Weather-adaptive highway event recognition method, system, device and medium

Abstract

The present invention relates to the field of intelligent traffic monitoring technology, and specifically to a weather-adaptive highway event recognition method, system, device and medium. The method comprises: obtaining historical video data and meteorological data of various abnormal events on highways and dividing them into data sets of different meteorological types; constructing a highway event recognition model, which introduces an MLLA layer in a feature extraction network of a YOLOv10 model; pre-training the highway event recognition model based on data sets of different meteorological types; obtaining the monitoring results of meteorological data of each section in real time and determining the meteorological type of each section; switching to a highway event recognition model of a corresponding meteorological type based on the meteorological type of each section to recognize abnormal events of each section, thereby improving the recognition accuracy and efficiency of abnormal events on highways under severe weather conditions, and timely responding to the monitoring needs of rapidly changing highway road conditions under severe weather conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic monitoring, and in particular to a weather-adaptive highway event recognition method, system, equipment and medium. Background Art

[0002] In recent years, the development of artificial intelligence technology, especially machine learning and deep learning algorithms, has made it possible to automatically identify and classify abnormal events on highways. Artificial intelligence technology can more accurately judge and warn of potential risks by training models to identify a series of features in abnormal events on highways, including vehicle behavior patterns and abnormal changes in the environment, significantly improving road safety and traffic efficiency. However, since the perception ability of artificial intelligence technology mainly relies on sensors and image recognition technology, it will inevitably lead to a decrease in the accuracy of artificial intelligence models in identifying abnormal events on highways under severe weather conditions such as foggy days and rainstorms. At the same time, since the conditions of roads and vehicles under severe weather conditions will be more complex and changeable, artificial intelligence models need to process and analyze more variable factors, resulting in low recognition efficiency and difficulty in meeting the monitoring needs for rapidly changing highway conditions under severe weather conditions.

[0003] To this end, Chinese patent CN117612115A proposes a vehicle recognition method based on highways, which obtains the meteorological information of the passage area for each day in the future within a preset period to extract multiple weather condition parameters of the passage area; determines the illumination compensation coefficient of the passage area according to each weather condition parameter; compensates the standard illumination intensity of the passage area according to the illumination compensation coefficient to obtain the target illumination intensity of the passage area in the future; updates the general neural network model according to the target illumination intensity to obtain the regional neural network model; obtains the vehicle image of the passing vehicle in the passage area, and recognizes the vehicle image according to the regional neural network model. This method updates the general neural network model based on the meteorological information of each day in the future, which helps to improve the visibility and quality of the image under different illumination conditions, thereby improving the accuracy of the vehicle image recognition results.

[0004] However, weather conditions are changeable, especially on highways at high altitudes. When bad weather occurs, there are local and sporadic characteristics. Therefore, if the future daily meteorological information obtained is inaccurate, it may lead to inaccurate illumination compensation coefficients, which in turn affects the calculation of target illumination intensity and the recognition accuracy of regional neural network models. Moreover, the calculation of illumination compensation coefficients, target illumination intensity, and the update of regional neural network models are complex and require a long analysis and processing time, which may cause the system to be unable to respond in time to the monitoring needs of rapidly changing highway conditions under bad weather conditions. In addition, neural network models still require a large amount of training data and computing resources to adapt to image recognition under bad weather conditions. Therefore, updating the general neural network model only by target illumination intensity has certain limitations in practical applications. Summary of the invention

[0005] In response to the above technical problems, the present invention proposes a weather-adaptive highway event identification method, system, equipment and medium, which aims to improve the accuracy and efficiency of identifying abnormal highway events under severe weather conditions, and promptly respond to the monitoring needs of rapidly changing highway road conditions under severe weather conditions.

[0006] In a first aspect, the present application provides a weather-adaptive highway event recognition method, comprising the following steps:

[0007] Step 102, obtaining historical video data and meteorological data of various abnormal events on the highway, and dividing the historical video data into data sets of different meteorological types based on the meteorological data;

[0008] Step 104, using the improved YOLOv10 model to build a highway event recognition model, wherein the highway event recognition model introduces an MLLA layer into the feature extraction network of the YOLOv10 model;

[0009] Step 106, pre-training the highway event recognition model based on the data sets of different weather types to obtain the highway event recognition model for different weather types;

[0010] Step 108, obtaining the monitoring results of the meteorological data of each section of the expressway in real time, and determining the meteorological type of each section based on the monitoring results;

[0011] Step 110 , based on the weather type of each road section, switching to a highway event recognition model of the corresponding weather type performs abnormal event recognition on each road section.

[0012] In some embodiments, in step 102, the historical video data is divided into data sets of different meteorological types based on meteorological data, including:

[0013] Match the historical video data of various abnormal events on the highway with the meteorological data in time to obtain the meteorological data corresponding to each abnormal event in the historical video data;

[0014] Based on the meteorological data corresponding to each abnormal event, determine the meteorological type when each abnormal event occurs;

[0015] Based on the meteorological type when each abnormal event occurs, the historical video data are divided into data sets of different meteorological types.

[0016] In some embodiments, the weather types include good weather, precipitation weather, snowy weather, windy weather, dusty weather, and haze weather.

[0017] In some embodiments, the meteorological data includes the total precipitation in 12 hours, the diameter of snowflakes, the snowfall per hour, the wind speed at 10 meters above the ground, the horizontal visibility, and the relative humidity of the air. Based on the meteorological data corresponding to each abnormal event, the meteorological type when each abnormal event occurs is determined, including:

[0018] Based on the 12-hour total precipitation corresponding to each abnormal event, the meteorological type when each abnormal event occurs is determined to be precipitation weather or good weather;

[0019] Based on the snowflake diameter and hourly snowfall corresponding to each abnormal event, determine the meteorological type when each abnormal event occurs as ice and snow weather or good weather;

[0020] Based on the comparison result of the wind speed at 10 meters above the ground on flat ground corresponding to each abnormal event and the first preset threshold value, determining whether the meteorological type when each abnormal event occurs is strong wind weather or good weather;

[0021] Based on the comparison result of the horizontal visibility corresponding to each abnormal event with the second preset threshold and the comparison result of the relative humidity of the air with the third preset threshold, it is determined that the meteorological type when each abnormal event occurs is sandstorm weather, haze weather or good weather.

[0022] In some embodiments, the highway event recognition model uses an iterative attention feature fusion method in the feature fusion part of the YOLOv10 model.

[0023] In some embodiments, in step 110, switching to a highway event recognition model of a corresponding meteorological type based on the meteorological type of each road section to recognize abnormal events in each road section includes:

[0024] Obtain camera monitoring data of the current section of the highway;

[0025] When the weather type determined for the current road section is a single weather type, the camera monitoring data of the current road section is input into the highway event recognition model of the corresponding weather type after switching, and the abnormal event recognition result of the current road section is output.

[0026] In some embodiments, in step 110, switching to a highway event recognition model of a corresponding meteorological type based on the meteorological type of each road section to recognize abnormal events in each road section further includes:

[0027] Obtain camera monitoring data of the current section of the highway;

[0028] When the weather type determined for the current road section is one of several weather types, the camera monitoring data of the current road section is input in parallel into the highway event recognition model of the corresponding weather type, and several abnormal event recognition results of the current road section are output;

[0029] The union of several abnormal event recognition results constitutes the final abnormal event recognition result of the current road section.

[0030] In a second aspect, the present application provides a weather-adaptive highway event recognition system, comprising:

[0031] The historical data acquisition module acquires historical video data and meteorological data of various abnormal events on the highway, and divides the historical video data into data sets of different meteorological types based on the meteorological data;

[0032] Model building module, building a highway event recognition model;

[0033] Model pre-training module, which pre-trains the highway event recognition model based on data sets of different meteorological types to obtain highway event recognition models for different meteorological types;

[0034] A real-time data acquisition module acquires the monitoring results of meteorological data of each section of the expressway in real time, and determines the meteorological type of each section based on the monitoring results;

[0035] The abnormal event recognition module switches to the highway event recognition model of the corresponding meteorological type based on the meteorological type of each road section to recognize abnormal events in each road section.

[0036] In a third aspect, the present application provides an electronic device, including a processor and a memory;

[0037] The processor is connected to the memory;

[0038] The memory is used to store executable program code;

[0039] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the aforementioned weather-adaptive highway event recognition method.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned weather-adaptive highway event recognition method.

[0041] The beneficial technical effects of the present invention include at least:

[0042] 1. Adopt weather-adaptive highway event recognition methods, systems, equipment and media, pre-train multiple highway event recognition models for different meteorological types based on historical video data, so that the model has stronger environmental adaptability. By introducing the MLLA layer in the feature extraction network of the YOLOv10 model, the improved YOLOv10 model is used as the highway event recognition model, taking into account the needs of non-autoregressive tasks, so that when dealing with the recognition task of highway abnormal events, it can better cope with multiple objects in the scene and the relationship between them, which helps the model to better learn and extract important information from the features, thereby improving the model's ability to understand highway abnormal events. By using position encoding (RoPE) instead of the "forget gate" in the MLLA layer, it can provide the necessary The location information of the vehicle and the accident in the image data can be used to accurately locate the vehicle and accident location in the highway event recognition. Combined with the real-time acquisition of meteorological data, it can dynamically switch to a suitable highway event recognition model for abnormal event recognition to ensure that it can respond to the monitoring needs of rapidly changing highway road conditions under severe weather conditions in a timely manner. It is especially suitable for areas with local and occasional severe weather such as high altitudes. Since the highway event recognition model corresponding to each meteorological type has a high target recognition rate under the corresponding weather conditions, the overall recognition accuracy and efficiency of highway abnormal events can be significantly improved by adaptively switching the model under different severe weather conditions. At the same time, it reduces the dependence of highway abnormal event recognition on computing resources, greatly reduces the complexity and processing time of recognition, and makes the model update and recognition process more efficient.

[0043] 2. By using specific meteorological data to accurately classify the meteorological types corresponding to each abnormal event, the subsequent pre-trained highway event recognition model can be better generalized to different meteorological conditions. Even under specific meteorological conditions that have not been encountered during the model pre-training, it can more accurately and quickly switch to the most matching highway event recognition model based on the acquired meteorological data and identify highway abnormal events under different meteorological conditions, thereby improving the model's adaptability and robustness to different meteorological conditions;

[0044] 3. By improving the feature fusion of the highway event recognition model, the iterative attention feature fusion method is used to adaptively adjust the weight of the feature map according to meteorological conditions, helping the model to focus on more stable features and reduce the adverse effects of noise or other visual interference on feature information in image data under bad weather conditions. This helps the model better understand and characterize abnormal events under different meteorological types, thereby improving the model's recognition accuracy for highway abnormal events and the model's robustness under bad weather conditions;

[0045] 4. Multiple models are used jointly to identify abnormal events on highways. When multiple weather types coexist, multiple models can be run in parallel to analyze camera monitoring data, thereby significantly improving the efficiency and accuracy of the model recognition process, ensuring that abnormal events can be accurately identified even under complex and changeable severe weather conditions, and effectively avoiding the technical problem that existing technologies are prone to false alarms or missed alarms when facing complex weather conditions.

[0046] Other features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below in conjunction with the accompanying drawings:

[0048] Figure 1 This is a flow chart of a weather-adaptive highway event recognition method according to an embodiment of the present invention.

[0049] Figure 2 It is a schematic diagram of the structure of the weather-adaptive highway event recognition system according to an embodiment of the present invention.

[0050] Figure 3 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The technical solutions of the embodiments of the present invention are explained and described below in conjunction with the drawings of the embodiments of the present invention, but the following embodiments are only preferred embodiments of the present invention, not all. Based on the embodiments in the implementation mode, other embodiments obtained by those skilled in the art without creative work are all within the protection scope of the present invention.

[0052] In the following description, terms such as "inside", "outside", "up", "down", "left", "right", etc. that indicate directions or positional relationships are only used to facilitate the description of the embodiments and simplify the description, and do not indicate or imply that the referred device or element must have a specific direction, be constructed and operate in a specific direction. Therefore, it should not be understood as a limitation of the present invention.

[0053] The present application embodiment provides a weather-adaptive highway event recognition method, see the attached Figure 1 , including the following steps:

[0054] Step 102, obtaining historical video data and meteorological data of various abnormal events on the highway, and dividing the historical video data into data sets of different meteorological types based on the meteorological data.

[0055] Among them, various abnormal events on the expressway include but are not limited to pedestrian crossing, spillage, vehicle breakdown, rear-end collision, illegal driving (such as illegal reversing, parking, lane changing, etc.), traffic congestion, road construction safety, and landslides and mud-rock flows on the expressway due to natural factors. In this embodiment, the historical video data is usually derived from surveillance cameras on various sections of the expressway, and the meteorological data is derived from traffic meteorological monitoring equipment near the location of the surveillance cameras.

[0056] Furthermore, in this embodiment, in step 102, the historical video data is divided into data sets of different meteorological types based on meteorological data, including:

[0057] Step 202 , time-matching the historical video data of various abnormal events on the highway with the meteorological data to obtain the meteorological data corresponding to each abnormal event in the historical video data.

[0058] Specifically, since the historical video data and meteorological data of various abnormal events on highways may have time differences, it is necessary to synchronize the time coordinates of the two, which can be achieved by recording the timestamps of the surveillance camera video and the timestamps of the meteorological data. First, the historical video data and meteorological data are associated with each other using timestamps to form a time-synchronized data set. Subsequently, the historical video data is analyzed to identify the time and location of the abnormal events. Next, the meteorological data corresponding to the time when each abnormal event occurred in the historical video data is found in the time-synchronized data set.

[0059] Step 204: Determine the meteorological type when each abnormal event occurs based on the meteorological data corresponding to each abnormal event.

[0060] Preferably, in this embodiment, the weather types include good weather, precipitation weather, snowy weather, windy weather, sandstorm weather and haze weather.

[0061] In this embodiment, by selecting meteorological types that have a significant impact on the occurrence and identification of abnormal events on highways, highway event recognition models of different meteorological types are pre-trained in a targeted manner so that the models can be adaptively switched under different weather conditions, thereby better capturing the characteristics under specific meteorological conditions and improving the accuracy of model recognition to a certain extent.

[0062] Further, in this embodiment, the meteorological data includes the total precipitation in 12 hours, the diameter of snowflakes, the snowfall per hour, the wind speed at 10 meters above the ground, the horizontal visibility, and the relative humidity of the air. Then, in step 204, based on the meteorological data corresponding to each abnormal event, the meteorological type when each abnormal event occurs is determined, including:

[0063] Based on the 12-hour total precipitation corresponding to each abnormal event, the meteorological type when each abnormal event occurs is determined to be precipitation weather or good weather;

[0064] Based on the snowflake diameter and hourly snowfall corresponding to each abnormal event, determine the meteorological type when each abnormal event occurs as ice and snow weather or good weather;

[0065] Based on the comparison result of the wind speed at 10 meters above the ground on flat ground corresponding to each abnormal event and the first preset threshold value, determining whether the meteorological type when each abnormal event occurs is strong wind weather or good weather;

[0066] Based on the comparison result of the horizontal visibility corresponding to each abnormal event with the second preset threshold and the comparison result of the relative humidity of the air with the third preset threshold, it is determined that the meteorological type when each abnormal event occurs is sandstorm weather, haze weather or good weather.

[0067] For example, based on the meteorological data corresponding to each abnormal event, the meteorological type when each abnormal event occurs may be determined by:

[0068] (1) If the total precipitation in 12 hours corresponding to the abnormal event is 0, the meteorological type when the abnormal event occurs is good weather. If the total precipitation in 12 hours corresponding to the abnormal event is greater than 0, the meteorological type when the abnormal event occurs is precipitation weather. Furthermore, the impact degree classification table of precipitation weather is shown in Table 1 below.

[0069] Table 1 Classification of the impact of precipitation weather

[0070] Precipitation Weather light rain moderate rain heavy rain rainstorm 12-hour total precipitation a (mm) 0＜a≤4.9 4.9＜a≤14.9 14.9＜a≤29.9 a＞29.9

[0071] (2) If the hourly snowfall corresponding to the abnormal event is 0 or no snowflakes appear, the meteorological type when the abnormal event occurs is good weather. If the hourly snowfall corresponding to the abnormal event is greater than 0 and the snowflake diameter is greater than 0, the meteorological type when the abnormal event occurs is ice and snow weather. Furthermore, the impact degree classification table of ice and snow weather is shown in Table 2 below.

[0072] Table 2 Classification of the impact of ice and snow weather

[0073] Snow and ice weather Light Snow Moderate Snow heavy snow Blizzard Snowflake diameter b1 (mm) 0＜b1≤10 10＜b1≤15 15＜b1≤20 b1＞20 Hourly snowfall b2 (mm) 0＜b2≤1 1＜b2≤3 3＜b2≤10 b2＞10

[0074] (3) If the wind speed at 10 meters above the ground on the flat ground corresponding to the abnormal event is less than or equal to the first preset threshold, the meteorological type when the abnormal event occurs is good weather; if the wind speed at 10 meters above the ground on the flat ground corresponding to the abnormal event is greater than the first preset threshold, the meteorological type when the abnormal event occurs is strong wind weather. Furthermore, the impact degree classification table of strong wind weather is shown in Table 3 below.

[0075] Table 3 Classification of the impact of strong wind weather

[0076] Windy weather Strong wind Strong wind Strong Wind hurricane Wind speed c (m / s) at 10 meters above the ground 10.8＜c≤17.2 17.2＜c≤24.5 24.5＜c≤32.7 c＞32.7

[0077] (4) If the horizontal visibility corresponding to the abnormal event is greater than the second preset threshold, the meteorological type when the abnormal event occurs is good weather; if the horizontal visibility corresponding to the abnormal event is less than or equal to the second preset threshold and the relative air humidity is less than or equal to the third preset threshold, the meteorological type when the abnormal event occurs is sandstorm weather; if the horizontal visibility corresponding to the abnormal event is less than or equal to the second preset threshold and the relative air humidity is greater than the third preset threshold, the meteorological type when the abnormal event occurs is haze weather. Further, the impact degree classification table of sandstorm weather is shown in Table 4 below, and the impact degree classification table of haze weather is shown in Table 5 below.

[0078] Table 4 Classification of the impact of sandstorms

[0079]

[0080] Table 5 Classification of the impact of haze weather

[0081]

[0082] It can be understood that this embodiment uses specific meteorological data (such as the total precipitation in 12 hours, the diameter of snowflakes, the hourly snowfall, the wind speed at 10 meters above the ground on flat ground, the horizontal visibility and the relative humidity of the air) to accurately divide the meteorological type corresponding to each abnormal event. The subsequent pre-trained highway event recognition model can be better generalized to different meteorological conditions. Even under specific meteorological conditions that have not been encountered during the model pre-training, it can more accurately and quickly switch to the most matching highway event recognition model according to the acquired meteorological data and identify highway abnormal events under different meteorological conditions, thereby improving the model's adaptability and robustness to different meteorological conditions, thereby improving the model's recognition accuracy and efficiency to a certain extent.

[0083] Step 206 , based on the meteorological type when each abnormal event occurs, the historical video data is divided into data sets of different meteorological types.

[0084] Specifically, in this embodiment, based on the meteorological type when each abnormal event occurs, the historical video data is divided into data sets of different meteorological types in the following manner:

[0085] First, the historical video data of a large number of abnormal events corresponding to each meteorological type can be converted into image data. Here, video processing tools (such as OpenCV) can be used to extract the video into image data at a certain time interval (such as one frame per second);

[0086] Secondly, the image data of abnormal events corresponding to each meteorological type are annotated using image annotation tools (such as LabelImg, etc.) to mark the location and category of the abnormal events in the image data;

[0087] Finally, the labeled image data are classified according to meteorological types and divided into data sets of different meteorological types.

[0088] Step 104: Use the improved YOLOv10 model to build a highway event recognition model, where the highway event recognition model introduces an MLLA layer into the feature extraction network of the YOLOv10 model.

[0089] Specifically, this embodiment uses the YOLOv10 model as the basis of the highway event recognition model. The input of the highway event recognition model is the image data of the abnormal event, and the output is the recognition result of the highway abnormal event. The recognition result includes the bounding box of each recognized target event, including its position (x, y coordinates) and size (width, height). These bounding boxes surround the abnormal event objects recognized in the image data, such as vehicles, pedestrians, spilled objects, road construction safety signs (such as construction vehicles, construction personnel, safety cones, crash buckets, construction signs, etc.) or other entities of interest.

[0090] Among them, the YOLOv10 model directly outputs the final prediction result after removing the non-maximum suppression (NMS) step, which greatly improves the processing speed. This step is removed because NMS needs to calculate a large number of overlapping boxes when processing the output, which is expensive for real-time applications. The structural diagram of yolov10 is basically the same as v8. The only difference is that yolov10 introduces a one-to-one head at the end. Both heads participate in the training during the training phase, so that the model can obtain rich supervision from one-to-many heads; then in the inference phase, only the one-to-one head is used, thus avoiding the post-processing of NMS. YOLOv10 also achieves an increase in recognition speed through improvements in lightweight classification heads and downsampling layers.

[0091] Although the YOLOv10 model improves the processing speed by removing the non-maximum suppression (NMS) step, which does help the real-time application of the highway event recognition model, it also inevitably leads to a decrease in the model's ability to process overlapping objects, especially when the object boundaries are unclear or multiple objects occlude each other, which has an adverse effect on the recognition of highway abnormal events. To this end, in this embodiment, the highway event recognition model introduces the MLLA layer in the feature extraction network of the YOLOv10 model.

[0092] Among them, the principle of MLLA (Mamba-Like Linear Attention) is to improve the performance of the model by integrating some core designs of the Mamba model into the linear attention mechanism. Specifically, MLLA mainly integrates the two key factors of "forget gate" and block design in Mamba. At the same time, MLLA replaces the forget gate by using position encoding (RoPE), thereby providing the necessary position information while maintaining parallel computing and fast inference speed, which makes MLLA more effective in processing non-autoregressive visual tasks.

[0093] For example, the highway event recognition model can be implemented by introducing the MLLA layer in the feature extraction network of the YOLOv10 model: define an MLLALayer class, which contains a convolution layer, a batch normalization layer, and a LeakyReLU activation function. Then in the DarkNet class, this MLLALayer is instantiated to build a feature extraction network. It can be understood that the feature extraction network constructed here is just a simplified example of the feature extraction part in the YOLOv10 model, which describes how to integrate the MLLA layer into the YOLOv10 model as a highway event recognition model.

[0094] This embodiment introduces the MLLA layer into the feature extraction network of the YOLOv10 model, and uses the improved YOLOv10 model as a highway event recognition model, taking into account the needs of non-autoregressive tasks, so that when processing the recognition task of highway abnormal events, it can better cope with multiple objects in the scene and the relationship between them. At the same time, through the special design of the MLLA layer, such as the "forget gate" and module design, it helps the model to better learn and extract important information in the features, thereby improving the model's ability to understand highway abnormal events, and by using position encoding (RoPE) instead of the "forget gate", it can provide necessary location information while maintaining parallel computing and fast inference speed, thereby accurately locating the vehicle and accident positions in the image data in highway event recognition.

[0095] Traditional feature fusion methods such as addition or concatenation are relatively simple and do not take into account the fusion applicability of specific objects. In different severe weather conditions, image data may be interfered with to varying degrees. Therefore, the traditional feature fusion method in the YOLOv10 model cannot effectively resist these interferences and is difficult to adapt to changes in image features in different severe weather conditions, thereby adversely affecting the accurate identification of abnormal events on highways under different meteorological types. For this reason, further, in this embodiment, the highway event recognition model adopts an iterative attention feature fusion method in the feature fusion part of the YOLOv10 model.

[0096] The main idea of ​​the iterative attention feature fusion (iAFF) method is to improve the detection accuracy by improving the feature fusion process. Specifically, the iterative attention feature fusion method better integrates features of different scales and semantic inconsistencies by introducing a multi-scale channel attention module. This method is a detail improvement and does not affect other modules in the YOLOv10 model.

[0097] Specifically, this embodiment introduces a multi-scale channel attention mechanism in the feature fusion part of YOLOv10, especially in the "neck" part of the model, to achieve effective fusion of multi-scale features. This multi-scale feature fusion method is more effective than traditional feature fusion methods and helps to improve the performance and accuracy of the model.

[0098] This embodiment improves the feature fusion of the highway event recognition model and uses an iterative attention feature fusion method to adaptively adjust the weight of the feature map according to meteorological conditions, helping the model focus on more stable features and reducing the adverse effects of noise or other visual interference on feature information in image data in bad weather, which helps the model better understand and characterize abnormal events under different meteorological types, thereby improving the model's recognition accuracy of highway abnormal events and the model's robustness in bad weather. This improvement not only improves the accuracy of the highway event recognition model, but also strikes a balance between the size and computational complexity of the model, making it possible to deploy the YOLOv10 model on low-end edge devices.

[0099] Step 106 , pre-training the highway event recognition model based on the data sets of different meteorological types, to obtain the highway event recognition model for different meteorological types.

[0100] It can be understood that in this embodiment, the highway event recognition model is pre-trained based on the data sets of different meteorological types to obtain highway event recognition models for different meteorological types, that is, the highway event recognition model is pre-trained based on the labeled abnormal event image data sets of different meteorological types to obtain several highway event recognition models for different meteorological types. The specific implementation method can be:

[0101] (1) Model configuration:

[0102] Adjust the hyperparameters of the highway event recognition model as needed, such as the learning rate, the number of images in each batch, the number of iterations, the size of the input image, and the patience value of the early stopping strategy (i.e., the number of consecutive rounds after which there is no performance improvement before stopping training).

[0103] Set the size and number of anchor boxes suitable for highway abnormal event recognition.

[0104] (2) Model pre-training:

[0105] The highway event recognition model was pre-trained using datasets of different meteorological types. For each meteorological type, the dataset was divided into a training set, a validation set, and a test set. Multiple rounds of training were performed, and the validation set was used to monitor the model performance and perform hyperparameter tuning.

[0106] (3) Model evaluation and adjustment:

[0107] Use the test set to evaluate the pre-trained highway event recognition model and calculate performance indicators such as accuracy, recall, mAP (mean Average Precision), etc. Fine-tune the structure or training parameters of the highway event recognition model based on the evaluation results to improve the performance of the highway event recognition model under specific meteorological conditions.

[0108] (4) Regular update of the model:

[0109] Monitor the performance of the highway event recognition model in real time, and regularly retrain the model with image data of new abnormal events to maintain its accuracy.

[0110] This embodiment pre-trains the highway event recognition model based on data sets of different meteorological types, and obtains multiple highway event recognition models for different meteorological types. This specific model pre-training can make each model more accurate and efficient in identifying abnormal highway events under corresponding severe weather conditions, avoiding the technical problem of low recognition efficiency of a single model under multiple severe weather conditions, and reducing false alarms and missed alarms caused by insufficient model generalization. At the same time, model pre-training can reduce the time consumption of real-time training, thereby speeding up the model's response speed to the monitoring needs of rapidly changing highway road conditions under severe weather conditions.

[0111] Step 108, obtaining the monitoring results of the meteorological data of each section of the expressway in real time, and determining the meteorological type of each section based on the monitoring results.

[0112] Optionally, in this embodiment, the monitoring results of the meteorological data of each section of the highway can be obtained by real-time requesting the traffic meteorological monitoring equipment near the location of the monitoring cameras of each section, or by periodically (for example, once every 2 minutes) requesting the traffic meteorological monitoring equipment near the location of the monitoring cameras of each section to update the current meteorological data.

[0113] Among them, the method of determining the meteorological type of each road section based on the monitoring results in step 108 of this embodiment is similar to the method of determining the meteorological type when each abnormal event occurs based on the meteorological data corresponding to each abnormal event in the aforementioned step 204, and this embodiment will not be repeated here.

[0114] Step 110 , based on the weather type of each road section, switching to a highway event recognition model of the corresponding weather type performs abnormal event recognition on each road section.

[0115] It can be understood that the abnormal event recognition results of each road section in step 110 of this embodiment are the recognition results of the abnormal event monitoring image of the highway, and the recognition results contain the bounding box of each recognized target event, including its position (x, y coordinates) and size (width, height). These bounding boxes surround the abnormal event objects recognized in the monitoring image, such as vehicles, pedestrians, spilled objects, road construction safety signs or other entities of interest, so as to provide an accurate basis for the subsequent detection of abnormal events on the highway (such as pedestrian crossing, spilling events, vehicle failures, rear-end collisions, illegal reversing, illegal parking, illegal lane changes, traffic congestion, road construction safety events, etc.). The subsequent detection method of abnormal events on the highway is such as Chinese patent CN111507237A, a method and device for detecting abnormal events based on video applied on the highway.

[0116] Furthermore, in this embodiment, the switching of the highway event recognition model of the corresponding meteorological type can select a single model or multiple models according to the actual meteorological type determined for each road section. In the case of a single model, a model switching process is involved, and in the case of multiple models, a model parallel recognition process is involved. For example, the meteorological type of a sunny day is relatively simple, and only a single highway event recognition model for good weather is needed to recognize abnormal events in each road section. However, sandstorms are often accompanied by strong winds, so the highway event recognition model for sandstorms and the highway event recognition model for strong winds can be used for parallel recognition. The general principle is to improve the accuracy of highway abnormal event recognition.

[0117] On the one hand, in this embodiment, in step 110, switching to a highway event recognition model of a corresponding meteorological type based on the meteorological type of each road section to recognize abnormal events of each road section includes:

[0118] Obtain camera monitoring data of the current section of the highway;

[0119] When the weather type determined for the current road section is a single weather type, the camera monitoring data of the current road section is input into the highway event recognition model of the corresponding weather type after switching, and the abnormal event recognition result of the current road section is output.

[0120] It can be understood that this embodiment only describes the method for identifying abnormal events in a certain section of a highway, and the method for identifying abnormal events in other sections can refer to this embodiment.

[0121] For example, a main road under a surveillance camera on a highway experienced multiple weather types within 1 hour. It was sunny at first, turned to strong winds after 10 minutes, and turned to heavy rain after half an hour. Vehicles slowed down on the main road in the heavy rain, and congestion gradually occurred. The specific implementation process of abnormal event recognition for each section based on the highway event recognition model switching to the corresponding weather type of each section is described as follows:

[0122] First, the surveillance camera requests the nearby traffic weather monitoring equipment to obtain the current weather data every 2 minutes. At the beginning, it determines that the weather type is sunny, so it switches to the highway event recognition model for good weather to identify the abnormal event objects on the main road under the current surveillance camera and detect whether any abnormal events have occurred. The detection result is that no abnormal events have occurred;

[0123] Then, 10 minutes later, the wind turns strong. At this time, the traffic weather monitoring equipment obtains the current weather data and determines that the weather type is strong wind weather. Then, the highway event recognition model for good weather is switched to the highway event recognition model for strong wind weather to identify the abnormal event objects of the main road under the current surveillance camera and detect whether any abnormal event occurs. The detection result is that no abnormal event occurs.

[0124] Then, half an hour later, it turned into heavy rain. At this time, the traffic meteorological monitoring equipment obtained the current meteorological data and determined that the meteorological type was precipitation weather and heavy rain. The highway event recognition model for strong winds was switched to the highway event recognition model for precipitation weather to identify abnormal event objects on the main road under the current surveillance camera. Since the model has been specially trained with a large number of samples under heavy rain weather in the early pre-training stage, when vehicles slow down and gradually become congested on the main road under the current surveillance camera, the model can quickly identify abnormal vehicles and detect abnormal events of traffic congestion.

[0125] On the other hand, in this embodiment, in step 110, switching to a highway event recognition model of a corresponding meteorological type based on the meteorological type of each road section to recognize abnormal events in each road section also includes:

[0126] Obtain camera monitoring data of the current section of the highway;

[0127] When the weather type determined for the current road section is one of several weather types, the camera monitoring data of the current road section is input in parallel into the highway event recognition model of the corresponding weather type, and several abnormal event recognition results of the current road section are output;

[0128] The union of several abnormal event recognition results constitutes the final abnormal event recognition result of the current road section.

[0129] It is understandable that “several” may include “two”, but its scope is not limited to two, and may be more than two.

[0130] For example, heavy rain is usually accompanied by strong winds, light snow is usually accompanied by mild haze, heavy snow and sandstorms are usually accompanied by strong winds, etc. The specific number of weather types is determined by actual meteorological data according to the aforementioned meteorological type judgment criteria, and this embodiment does not limit this.

[0131] For example, a main road under a surveillance camera on a highway experienced multiple weather types within one hour. At the beginning, it was light snow and light haze weather, and it turned to moderate rain half an hour later. In the light snow and light haze weather, a throwing event and an action event of stretching an arm out of the car window occurred on the main road. The specific implementation process of the highway event recognition model switching to the corresponding weather type based on the weather type of each section to identify abnormal events on each section is described:

[0132] First, the surveillance camera requests the nearby traffic meteorological monitoring equipment to obtain the current meteorological data every 2 minutes. At the beginning, it determines that the weather type is light snow weather and light haze weather, then switches to the highway event model of ice and snow weather and the highway event model of haze weather, and parallelly identifies abnormal event objects for the same monitoring image of the main road under the current surveillance camera. Since different models are specially trained with a large number of samples under the corresponding weather conditions in the early pre-training stage, it is possible that the highway event model of ice and snow weather only recognizes the occurrence of spilled objects, while the highway event model of haze weather not only recognizes the occurrence of spilled objects, but also recognizes the action of stretching arms out of the car window. Therefore, the union of the recognition results of the two models, that is, the recognition results of spilled objects and the action of stretching arms out of the car window, is used as the final abnormal event recognition result of the main road under the current surveillance camera, thereby detecting the abnormal events of spilling and the abnormal events of stretching arms out of the car window.

[0133] Then, half an hour later, it turned into moderate rain. At this time, the traffic meteorological monitoring equipment obtained the current meteorological data and determined that the meteorological type was precipitation weather with moderate rain. It then switched from the highway event recognition model for snowy weather and the highway event model for haze weather to a single highway event recognition model for precipitation weather to identify abnormal event objects on the main road under the current surveillance camera and detect whether any abnormal events occurred. The detection result was that no abnormal events occurred.

[0134] This embodiment adapts to different severe weather conditions by automatically switching to the pre-trained model of the corresponding meteorological type according to the determined meteorological type, and can respond to changes in severe weather conditions in real time, improve the adaptability and robustness of the algorithm to different environments, and when the meteorological type changes, it can quickly switch to the highway event recognition model of the corresponding meteorological type to ensure that abnormal events on the highway can be identified in a timely and accurate manner. At the same time, multiple models are jointly used to identify abnormal events on the highway. When multiple meteorological types coexist, multiple models can be run in parallel to analyze the camera monitoring data, thereby significantly improving the efficiency and accuracy of the model recognition process, ensuring that abnormal events can be accurately identified under complex and changeable severe weather conditions, and effectively avoiding the technical problem that the prior art is prone to false alarms or missed alarms when facing complex weather conditions.

[0135] In summary, this embodiment pre-trains multiple highway event recognition models for different meteorological types based on historical video data, so that the model has stronger environmental adaptability, obtains meteorological data in real time and dynamically switches to a suitable highway event recognition model, ensuring that it can respond in time to the monitoring needs of rapidly changing highway road conditions under severe weather conditions. It is especially suitable for areas with local and sporadic severe weather such as high altitudes. Since the highway event recognition model corresponding to each meteorological type has a high target recognition rate under the corresponding weather conditions, the overall recognition accuracy and efficiency of highway abnormal events can be significantly improved by adaptively switching the model under different severe weather conditions. At the same time, the dependence of highway abnormal event recognition on computing resources is reduced, the complexity and processing time of recognition are greatly reduced, and the model update and recognition process are more efficient.

[0136] Another embodiment of the present specification provides a weather-adaptive highway event recognition system, see the attached Figure 2 ,include:

[0137] The historical data acquisition module 1 acquires the historical video data and meteorological data of various abnormal events on the expressway, and divides the historical video data into data sets of different meteorological types based on the meteorological data;

[0138] Model building module 2, building a highway event recognition model;

[0139] Model pre-training module 3, pre-training the highway event recognition model based on data sets of different meteorological types, to obtain highway event recognition models for different meteorological types;

[0140] A real-time data acquisition module 4 acquires the monitoring results of the meteorological data of each section of the expressway in real time, and determines the meteorological type of each section based on the monitoring results;

[0141] The abnormal event recognition module 5 switches to a highway event recognition model of a corresponding meteorological type based on the meteorological type of each road section to recognize abnormal events in each road section.

[0142] It can be understood that the technical concept of the weather-adaptive highway event recognition system provided in this embodiment is similar to the technical concept of the aforementioned weather-adaptive highway event recognition method, and this embodiment will not be repeated here.

[0143] Another embodiment of the present specification provides an electronic device structure. Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 and at least one communication bus 302 .

[0144] The communication bus 302 may be used to realize the connection and communication among the above-mentioned components.

[0145] The user interface 303 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0146] The network interface 304 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0147] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts within the entire electronic device 300, and executes various functions and processes data of the electronic device 300 by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The processor 301 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to handle wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0148] Among them, the memory 305 may include RAM or ROM. Optionally, the memory 305 includes a non-transitory computer-readable medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also be at least one storage device located away from the aforementioned processor 301. The memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application. The processor 301 may be used to call the application stored in the memory 305 and execute the method in one or more of the above-mentioned embodiments.

[0149] Another embodiment of the present specification provides a computer-readable storage medium, which stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes one or more steps in the above-mentioned weather-adaptive highway event recognition method embodiment. If the components of the above-mentioned electronic device are implemented in the form of software functional units and used as independent downstream task predictions or uses, they can be stored in the computer-readable storage medium.

[0150] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of this specification is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state drive (SSD)).

[0151] The above description is only the preferred embodiment disclosed in this application and the description of the technical principle used. Those skilled in the art should understand that the scope of protection involved in this disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but also should cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this disclosure (but not limited to) to form a technical solution.

[0152] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

Claims

1. A weather-adaptive highway event recognition method, characterized in that: The following steps are involved: Step 102, obtaining historical video data and meteorological data of various abnormal events on the highway, and dividing the historical video data into data sets of different meteorological types based on the meteorological data; Step 104, using the improved YOLOv10 model to build a highway event recognition model, wherein the highway event recognition model introduces an MLLA layer into the feature extraction network of the YOLOv10 model; Step 106, pre-training the highway event recognition model based on the data sets of different weather types to obtain the highway event recognition model for different weather types; Step 108, obtaining the monitoring results of the meteorological data of each section of the expressway in real time, and determining the meteorological type of each section based on the monitoring results; Step 110, switching to a highway event recognition model of a corresponding meteorological type based on the meteorological type of each road section to perform abnormal event recognition on each road section; Wherein, step 110 includes: Obtain camera monitoring data of the current section of the highway; When the weather type determined for the current road section is one of several weather types, the camera monitoring data of the current road section is input in parallel into the highway event recognition model of the corresponding weather type, and several abnormal event recognition results of the current road section are output; The union of several abnormal event recognition results constitutes the final abnormal event recognition result of the current road section; Wherein, step 102 comprises: Match the historical video data of various abnormal events on the highway with the meteorological data in time to obtain the meteorological data corresponding to each abnormal event in the historical video data; Based on the meteorological data corresponding to each abnormal event, determine the meteorological type when each abnormal event occurs, the meteorological type including good weather, precipitation weather, ice and snow weather, strong wind weather, sandstorm weather and haze weather; Based on the meteorological type when each abnormal event occurred, the historical video data was divided into data sets of different meteorological types; Furthermore, the meteorological data includes the total precipitation in 12 hours, the diameter of snowflakes, the snowfall per hour, the wind speed at 10 meters above the ground, the horizontal visibility and the relative humidity of the air. Based on the meteorological data corresponding to each abnormal event, the meteorological type when each abnormal event occurs is determined, including: Based on the 12-hour total precipitation corresponding to each abnormal event, the meteorological type when each abnormal event occurs is determined to be precipitation weather or good weather; Based on the snowflake diameter and hourly snowfall corresponding to each abnormal event, determine the meteorological type when each abnormal event occurs as ice and snow weather or good weather; Based on the comparison result of the wind speed at 10 meters above the ground on flat ground corresponding to each abnormal event and the first preset threshold value, determining whether the meteorological type when each abnormal event occurs is strong wind weather or good weather; Based on the comparison result of the horizontal visibility corresponding to each abnormal event with the second preset threshold and the comparison result of the relative humidity of the air with the third preset threshold, it is determined that the meteorological type when each abnormal event occurs is sandstorm weather, haze weather or good weather.

2. The weather-adaptive highway event recognition method according to claim 1, characterized in that: The highway event recognition model adopts an iterative attention feature fusion method in the feature fusion part of the YOLOv10 model.

3. The weather-adaptive highway event recognition method according to claim 1, characterized in that: In step 110, based on the meteorological type of each road section, switching to a highway event recognition model of a corresponding meteorological type is performed to recognize abnormal events in each road section, including: Obtain camera monitoring data of the current section of the highway; When the weather type determined for the current road section is a single weather type, the camera monitoring data of the current road section is input into the highway event recognition model of the corresponding weather type after switching, and the abnormal event recognition result of the current road section is output.

4. Weather-adaptive highway event recognition system, characterized by: include: The historical data acquisition module acquires historical video data and meteorological data of various abnormal events on the highway, and divides the historical video data into data sets of different meteorological types based on the meteorological data; Model building module, building a highway event recognition model; Model pre-training module, which pre-trains the highway event recognition model based on data sets of different meteorological types to obtain highway event recognition models for different meteorological types; A real-time data acquisition module acquires the monitoring results of meteorological data of each section of the expressway in real time, and determines the meteorological type of each section based on the monitoring results; The abnormal event recognition module switches to the highway event recognition model of the corresponding meteorological type based on the meteorological type of each road section to recognize abnormal events in each road section; The abnormal event identification module is used to perform the following steps: Obtain camera monitoring data of the current section of the highway; When the weather type determined for the current road section is one of several weather types, the camera monitoring data of the current road section is input in parallel into the highway event recognition model of the corresponding weather type, and several abnormal event recognition results of the current road section are output; The union of several abnormal event recognition results constitutes the final abnormal event recognition result of the current road section; The historical data acquisition module is used to perform the following steps: Match the historical video data of various abnormal events on the highway with the meteorological data in time to obtain the meteorological data corresponding to each abnormal event in the historical video data; Based on the meteorological data corresponding to each abnormal event, determine the meteorological type when each abnormal event occurs, the meteorological type including good weather, precipitation weather, ice and snow weather, strong wind weather, sandstorm weather and haze weather; Based on the meteorological type when each abnormal event occurred, the historical video data was divided into data sets of different meteorological types; Furthermore, the meteorological data includes the total precipitation in 12 hours, the diameter of snowflakes, the snowfall per hour, the wind speed at 10 meters above the ground, the horizontal visibility and the relative humidity of the air. Based on the meteorological data corresponding to each abnormal event, the meteorological type when each abnormal event occurs is determined, including: Based on the 12-hour total precipitation corresponding to each abnormal event, the meteorological type when each abnormal event occurs is determined to be precipitation weather or good weather; Based on the snowflake diameter and hourly snowfall corresponding to each abnormal event, determine the meteorological type when each abnormal event occurs as ice and snow weather or good weather; Based on the comparison result of the wind speed at 10 meters above the ground on flat ground corresponding to each abnormal event and the first preset threshold value, determining whether the meteorological type when each abnormal event occurs is strong wind weather or good weather; Based on the comparison result of the horizontal visibility corresponding to each abnormal event with the second preset threshold and the comparison result of the relative humidity of the air with the third preset threshold, it is determined that the meteorological type when each abnormal event occurs is sandstorm weather, haze weather or good weather.

5. An electronic device, characterized in that: including a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the weather-adaptive highway event recognition method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the weather-adaptive highway event recognition method as described in any one of claims 1 to 3 is implemented.

Images