Method and System for Detecting and Warning Road Abnormal Events Based on Unsupervised Lifelong Learning

Abstract

The present invention discloses a method and system for detecting and warning road abnormal events based on unsupervised lifelong learning, including: collecting traffic scene data by using various sensors along the road and on the vehicle end and preprocessing it; using an autoencoder to perform unsupervised feature extraction on image data, and using the principal component analysis method to reduce the dimension of non-image data and extract main features; initializing an abnormal detection model, and using the K-Means clustering algorithm to divide the feature space into clusters to form an initial normal cluster set of normal traffic patterns; receiving new traffic data, and using the incremental learning method, the density-based clustering algorithm and the elastic weight integration method to continuously perform online learning and update on the model; inputting the newly collected data into the updated model for abnormal detection and road abnormal event alarm; the present invention realizes efficient, accurate detection and timely warning of road abnormal events through unsupervised feature extraction, model initialization, and lifelong learning and model update mechanisms.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic anomaly recognition, and more specifically, to a method and system for detecting and warning road anomaly events based on unsupervised lifelong learning. Background Art

[0002] With the continuous development of the traffic system, the traffic flow of vehicles and pedestrians on the road is increasing day by day. Timely detection and warning of various anomaly events (such as traffic accidents, road obstacles, traffic chaos under abnormal weather conditions, etc.) are crucial for ensuring traffic safety and improving traffic efficiency.

[0003] Currently, traditional road anomaly event detection methods mainly rely on supervised learning algorithms. These methods require a large amount of labeled data to train the model, and the labeling process is time-consuming and laborious and difficult to cover all possible anomaly situations.

[0004] Moreover, in practical applications, traffic scenarios are constantly changing, and new types of anomalies may appear at any time. Traditional models are difficult to adapt to this dynamic change and need to be retrained frequently, which is costly and inefficient.

[0005] Therefore, how to develop a road anomaly event detection and warning technology that can automatically adapt to new situations and does not require a large amount of labeled data to achieve efficient and accurate detection and timely warning of road anomaly events, so as to improve traffic safety and operation efficiency, is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for detecting and warning road anomaly events based on unsupervised lifelong learning to solve some of the technical problems mentioned in the background art.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for detecting and warning road anomaly events based on unsupervised lifelong learning, comprising the following steps:

[0009] S1. Collect traffic scene data using a variety of sensors distributed along the road and on the vehicle end, and perform preprocessing;

[0010] S2. Use an autoencoder to perform unsupervised feature extraction on the preprocessed image data, and use the principal component analysis method to reduce the dimension of non-image data to extract the main features;

[0011] S3. Initialize a clustering-based anomaly detection model using the extracted features, and use the K-Means clustering algorithm to divide the feature space into multiple clusters to form an initial set of normal clusters, and each cluster represents a normal traffic pattern;

[0012] S4. Receive new traffic data, and continuously perform online learning and update the normal traffic pattern and abnormal event pattern for the anomaly detection model by using the incremental learning method, density-based clustering algorithm, and elastic weight integration method;

[0013] S5. Input the newly collected data into the updated model for anomaly detection and road abnormal event alarm.

[0014] Preferably, the multiple sensors include but are not limited to cameras, millimeter-wave radars, lidars, ultrasonic sensors, GPS / Beidou positioning systems, vehicle speed sensors, acceleration sensors, and gyroscopes. The collected traffic scene data includes road images and videos, the speeds, distances, and angles of vehicles and objects, and the three-dimensional spatial information of the road and its surrounding environment.

[0015] Preferably, the autoencoder includes an encoder and a decoder. The encoder compresses the input image into a low-dimensional feature vector, and the decoder reconstructs the original image according to the feature vector;

[0016] Train the autoencoder with minimizing the reconstruction error as the objective of the autoencoder, so that the extracted features can effectively represent the key information of the image.

[0017] Preferably, the specific content of step S3 is:

[0018] S31. Assume a given historical traffic data points , and randomly initialize cluster centers ;

[0019] S32. For each data point , calculate its distance to each cluster center, and assign to the nearest cluster :

[0020] ;

[0021] S33. Update the center of each cluster:

[0022]

[0023] where, is the number of data points in cluster ;

[0024] S34. Repeat step S32 - step S33 until convergence, and form the initial set of normal clusters as clusters .

[0025] Preferably, the specific content of step S4 is as follows:

[0026] S41. Determine the similarity between the new traffic data and the existing clusters, and calculate the distances from the new data points to the centers of each cluster using distance metrics;

[0027] S42. If the distance between the new data point and a certain cluster is less than the set threshold, assign the new data point to that cluster, and update the cluster center and statistical information;

[0028] S43. For the data points whose distances from all cluster centers are greater than the threshold, they are judged as potential abnormal data, and the density-based clustering algorithm is further used to analyze the potential abnormal data, create new clusters and add them to the model to represent abnormal traffic patterns;

[0029] S44. During the model update process, the Elastic Weight Consolidation (EWC) method is adopted. When learning a new task, minimize the loss function of the new task while retaining the knowledge of the old task.

[0030] Preferably, the specific content of step S43 is as follows:

[0031] For each data point in the data set , calculate the number of neighbor points N within the given radius ϵ ϵ (x i );

[0032] If N ϵ (x i ) is greater than or equal to the minimum number of neighbor points , then mark as a core point, create a new cluster, add its neighbor points to the cluster, and recursively check the neighbor points of the neighbor points until no new points can be added to the cluster;

[0033] If N ϵ (x i ) is less than , but within the neighbor range of a certain core point, then mark as a boundary point and assign it to the corresponding cluster;

[0034] If the abnormal data forms a new dense area, it is added to the model as a new cluster to represent the abnormal traffic pattern.

[0035] Preferably, step S43 further includes that the data points not marked as core points or boundary points are regarded as noise points, and continuously monitor whether they form new patterns in the subsequent data accumulation.

[0036] Preferably, the specific content of step S44 is as follows:

[0037] S441. Train the model on the old task to obtain the parameters of the model after the training of the old task is completed;

[0038] S442. After the training of the old task is completed, calculate the importance weights of each model parameter for the old task:

[0039] S443. Conduct new task learning, and introduce a regularization term when updating the parameters to constrain the update of the parameters, punish the updates that deviate greatly from the important parameters of the old task, so that the important parameters related to the old task will not change too much, thereby avoiding forgetting old knowledge.

[0040] Preferably, in step S5, if the newly collected data point input to update the model is determined to belong to a known normal cluster, it is considered that the current traffic scene is normal; if the data point belongs to an abnormal cluster, a road anomaly event alarm is triggered;

[0041] The road anomaly events represented by the abnormal clusters include, but are not limited to, traffic accidents, road obstacles, traffic chaos under abnormal weather conditions, pedestrians breaking into the motor vehicle lane, abnormal aggravation of traffic congestion, vehicle out of control, road waterlogging or icing, traffic signal failures, animals breaking into the road, and illegal occupation of the emergency lane.

[0042] A road anomaly event detection and early warning system based on unsupervised lifelong learning, based on the described road anomaly event detection and early warning method based on unsupervised lifelong learning, includes: a data collection module, an unsupervised feature extraction and model initialization module, a lifelong learning and model update module, and an anomaly event detection and early warning module;

[0043] The data collection module is used to collect traffic scene data by using a variety of sensors distributed along the road and on the vehicle end, and perform preprocessing;

[0044] The unsupervised feature extraction and model initialization module is used to perform unsupervised feature extraction on the preprocessed image data by using an autoencoder, perform feature dimensionality reduction on non-image data by using the principal component analysis method, extract the main features, and initialize the anomaly detection model based on clustering by using the extracted features, and use the K-Means clustering algorithm to divide the feature space into multiple clusters to form an initial set of normal clusters, and each cluster represents a normal traffic pattern;

[0045] The lifelong learning and model update module is used to receive new traffic data, and continuously perform online learning and update the normal traffic pattern and anomaly event pattern on the anomaly detection model by using the incremental learning method, the density-based clustering algorithm, and the elastic weight integration method;

[0046] The anomaly event detection and early warning module is used to input the newly collected data into the updated model for anomaly detection and road anomaly event alarm.

[0047] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a method and system for detecting and warning road abnormal events based on unsupervised lifelong learning, which has the following beneficial effects:

[0048] High efficiency in data utilization: The present invention adopts an unsupervised learning method, combines the use of a unique autoencoder and PCA method, gets rid of the dependence on a large amount of labeled data, can directly extract key features from the original traffic data, greatly reduces the labor, material and time costs of data annotation, and at the same time improves the data utilization efficiency, enabling the system to be quickly deployed and adapted to different traffic scenarios. Even in the case of difficult data annotation or limited resources, it can effectively perform model training and abnormal event detection;

[0049] Strong adaptability of the model: Through the lifelong learning mechanism, the model can continuously learn new traffic data and abnormal event patterns online; Whether it is the seasonal change of traffic flow, the renovation of road facilities, or the emergence of new abnormal events, the model can dynamically update itself, continuously improve its adaptability to the complex and changeable road environment, always maintain a high detection accuracy, effectively avoid the problem of performance degradation of traditional models due to environmental changes, and truly achieve intelligent long-term stable operation;

[0050] High accuracy in abnormal detection: By combining various advanced clustering algorithms such as K-Means and DBSCAN and a carefully designed feature extraction and model update strategy, the present invention can more accurately identify the subtle differences between normal traffic patterns and abnormal events; The fusion of multi-source sensor data further enriches the information dimension, provides a comprehensive basis for accurately judging abnormal events, greatly reduces the situation of false alarms and missed detections, ensures that traffic management departments and road users can timely and accurately obtain road abnormal information, take corresponding measures, and improve road safety;

[0051] Strong anti-forgetting performance: The introduction of the Elastic Weight Consolidation (EWC) method solves the problem of catastrophic forgetting that may occur when the model learns new knowledge, enabling the model to firmly retain the key information about various traffic patterns and abnormal events learned previously during the process of continuously updating and expanding knowledge, ensuring the stability and reliability of the model during long-term operation, avoiding performance fluctuations caused by frequent retraining or forgetting old knowledge, and enabling the detection system to consistently provide high-quality abnormal event detection services. Brief Description of the Drawings

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0053] Figure 1 Schematic diagram of a road abnormal event detection and early warning method based on unsupervised lifelong learning provided by the present invention;

[0054] Figure 2 Schematic diagram of unsupervised feature extraction and model initialization provided by the present invention;

[0055] Figure 3 Schematic diagram of lifelong learning and model update provided by the present invention;

[0056] Figure 4 Schematic diagram of a road abnormal event detection and early warning system based on unsupervised lifelong learning provided by the present invention. Detailed implementation manners

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0058] An embodiment of the present invention discloses a road abnormal event detection and early warning method based on unsupervised lifelong learning, as Figure 1 shown, including the following steps:

[0059] S1. Collect traffic scene data using a variety of sensors distributed along the road and on the vehicle end, and perform preprocessing;

[0060] S2. Use an autoencoder to perform unsupervised feature extraction on the preprocessed image data, and use the principal component analysis method for non-image data to reduce the dimensionality of features and extract the main features;

[0061] S3. Use the extracted features to initialize a clustering-based anomaly detection model, and use the K-Means clustering algorithm to divide the feature space into multiple clusters to form an initial set of normal clusters, where each cluster represents a normal traffic pattern;

[0062] S4. Receive new traffic data, and use the incremental learning method, density-based clustering algorithm, and elastic weight integration method to continuously perform online learning and update the normal traffic pattern and abnormal event pattern of the anomaly detection model;

[0063] S5. Input the newly collected data into the updated model for anomaly detection and road anomaly event alarm.

[0064] To further implement the above technical solution, multiple sensors include but are not limited to cameras, millimeter-wave radars, lidars, ultrasonic sensors, GPS / Beidou positioning systems, vehicle speed sensors, acceleration sensors, and gyroscopes, and collect traffic scene data including road images and videos, speeds, distances, and angles of vehicles and objects, and three-dimensional spatial information of roads and their surrounding environments.

[0065] In this embodiment, the captured road images and videos are used to record visual information such as vehicles, pedestrians, and road conditions. For example, visual information in abnormal events such as vehicle collisions, pedestrians entering the motor vehicle lane, traffic signal failures, animals entering the road, and illegal occupation of the emergency lane is recorded, such as vehicle deformation, pedestrian positions, signal states, animal shapes, and vehicle driving positions; the speeds, distances, and angles of vehicles and objects can detect abnormal acceleration and angular velocity changes when the vehicle is out of control, sudden speed changes before vehicle collisions, and abnormal vehicle speeds and vehicle distances during traffic congestion; the three-dimensional spatial information of roads and their surrounding environments, including the positions and shapes of obstacles, etc., helps to identify the positions and shapes of road obstacles and detect abnormal changes in road surface reflection signals when the road is waterlogged or frozen; preprocess various types of collected data, including operations such as denoising, filtering, normalization, and image enhancement, to improve data quality, and store the preprocessed data in a database to provide a reliable data basis for subsequent analysis and processing.

[0066] To further implement the above technical solution, the autoencoder includes an encoder and a decoder. The encoder compresses the input image into a low-dimensional feature vector, and the decoder reconstructs the original image according to the feature vector;

[0067] Train the autoencoder with minimizing the reconstruction error as the goal of the autoencoder, so that the extracted features can effectively represent the key information of the image;

[0068] Specifically:

[0069] Let the input image be , the encoder function is , where is the low-dimensional feature vector, the decoder function is , the reconstructed image is , the goal of the autoencoder is to minimize the reconstruction error , and the reconstruction error function is the mean square error MSE:

[0070]

[0071] where is the number of image pixels, and are the -th pixel values of the input image and the reconstructed image, respectively;

[0072] The autoencoder is trained by minimizing the reconstruction error so that the extracted features can effectively represent the key information of the image; for non-image data, such as vehicle speed, position, etc., methods such as principal component analysis (PCA) are used for feature dimensionality reduction to extract the main features.

[0073] For example, in traffic accident detection, image features of events such as vehicle collisions, rear-end collisions, rollovers, etc. are extracted. Features such as vehicle deformation and scattered objects are reflected in the feature vector; when a pedestrian enters the motor vehicle lane, features such as the movement trajectory, speed of the pedestrian, and relative position with the vehicle are also encoded into the feature vector. For non-image data, methods such as principal component analysis (PCA) are used for feature dimensionality reduction to extract the main features. After data such as vehicle speed and position are processed by PCA, the main features related to abnormal events are highlighted, such as the key change features of traffic flow and vehicle speed when traffic congestion intensifies abnormally, and the abnormal speed and angle features when the vehicle is out of control.

[0074] To further implement the above technical solution, such as Figure 2 , the specific content of step S3 is:

[0075] S31. Given historical traffic data points , randomly initialize cluster centers ;

[0076] S32. For each data point , calculate its distance to each cluster center, and assign to the closest cluster :

[0077] ;

[0078] S33. Update the center of each cluster:

[0079]

[0080] where is the number of data points in cluster ;

[0081] S34. Repeat steps S32 - S33 until convergence, and form an initial set of normal clusters as clusters .

[0082] To further implement the above technical solution, as Figure 3 , the specific content of step S4 is as follows:

[0083] S41. Determine the similarity between the new traffic data and the existing clusters, and use a distance metric (such as the Euclidean distance) to calculate the distances from the new data points to the centers of each cluster;

[0084] S42. If the distance between the new data point and a certain cluster is less than the set threshold, assign the new data point to that cluster, and update the center and statistical information (such as mean, covariance, etc.) of the cluster;

[0085] S43. For data points whose distances from all cluster centers are greater than the threshold, they are judged as potential abnormal data, and the density-based clustering algorithm DBSCAN is further used to analyze the potential abnormal data, create new clusters and add them to the model to represent abnormal traffic patterns;

[0086] S44. During the model update process, the Elastic Weight Consolidation (EWC) method is adopted. When learning a new task, the loss function of the new task is minimized while retaining the knowledge of the old task.

[0087] Taking road obstacles as an example, when the lidar and millimeter-wave radar detect the position and contour data of the obstacles, and the image data identifies the type of the obstacles, if the distances between these data and the existing normal traffic pattern clusters are greater than the threshold, after analysis by the density-based clustering algorithm (such as DBSCAN), if a new dense area is formed, a new cluster is created to represent this abnormal event type of road obstacles; in the detection of traffic chaos under abnormal weather conditions, features such as the generally decreased vehicle speed, abnormal increase or decrease in the vehicle distance, and unstable vehicle driving direction in the new data, if they do not conform to the features of the existing clusters, may form new abnormal clusters or update the information of the existing abnormal clusters; for newly emerging abnormal event types, such as abnormal driving patterns caused by new types of vehicle failures, the model will, during the continuous learning process, gradually determine the data from potential abnormal data to a new abnormal event cluster, and incorporate it into the model by updating the center and statistical information of the cluster, etc., to achieve the recognition and learning of new abnormal event types.

[0088] To further implement the above technical solution, the specific content of step S43 is as follows:

[0089] For the data set in each data point , calculate the number of neighbor points N within the given radius ϵ ϵ (x i );

[0090] If N ϵ (x i ) is greater than or equal to the minimum number of neighbor points , then is marked as a core point, and a new cluster is created. Its neighboring points are added to the cluster, and the neighboring points of the neighboring points are recursively checked until no new points can be added to the cluster;

[0091] If N ϵ (x i ) is less than , but within the neighborhood of a core point, then is marked as a boundary point and assigned to the corresponding cluster;

[0092] If the abnormal data forms a new dense area, it is added to the model as a new cluster, representing an abnormal traffic pattern;

[0093] In this embodiment, the data points subsequently determined to belong to this new cluster will trigger an abnormal event alarm. For example, when there is a road obstacle, the vehicle speed and position data will deviate significantly from the normal cluster. Through the analysis of the DBSCAN algorithm, it is found that these data form a new dense area, a new cluster will be created and a warning will be issued when similar data is detected subsequently.

[0094] To further implement the above technical solution, step S43 further includes that the data points not marked as core points or boundary points are regarded as noise points, which do not trigger an alarm at the current stage, and it is continuously monitored whether they form a new pattern in the subsequent data accumulation.

[0095] To further implement the above technical solution, in the process of model update, in order to avoid the problem of catastrophic forgetting, the Elastic Weight Consolidation (EWC) method is adopted. When learning a new task, the loss function of the new task needs to be minimized while retaining the knowledge of the old task. The specific content of step S44 is:

[0096] S441. Train the model on the old task to obtain the parameters of the model after the old task training is completed , where represents the parameter value of the model after the old task training is completed, and n is the total number of model parameters;

[0097] S442. After the old task training is completed, calculate the importance weight of each model parameter for the old task;

[0098] The importance weight reflects the sensitivity of the parameter to the old task. In this embodiment, it is measured by the diagonal elements of the Fisher Information Matrix (FIM):

[0099] The definition of the Fisher Information Matrix F is:

[0100]

[0101] where is the input data, is the output data, is the output probability distribution of the model under and is the log-likelihood function, is the partial derivative of the log-likelihood function with respect to the i-th parameter, is the expectation of the input data x and the output data y;

[0102] The diagonal elements of the Fisher information matrix are:

[0103]

[0104] where, is the importance weight of the i-th parameter, is the i-th diagonal element of the Fisher information matrix;

[0105] The importance weight represents the sensitivity of the i-th parameter to the old task. The larger the value, the more important the parameter is to the old task;

[0106] S443. Perform new task learning, and introduce a regularization term when updating the parameters to constrain the update of the parameters, punish the updates that deviate greatly from the important parameters of the old task, so that the important parameters related to the old task will not change too much, thereby avoiding forgetting old knowledge;

[0107] The introduced regularization term is:

[0108]

[0109] where, is the positive regularization strength, which is used to balance the learning of the new task and the old task, is the value of the current parameter, is the value of the parameter after the old task training ends, is the importance weight of the parameter, which is calculated from the Fisher information matrix.

[0110] To further implement the above technical solution, in step S5, if the newly collected data point for updating the model is determined to belong to the known normal cluster, it is considered that the current traffic scene is normal; if the data point belongs to the abnormal cluster, a road anomaly event alarm is triggered;

[0111] In this embodiment, when a road anomaly event alarm is triggered, the early warning system sends out alarms to the traffic management center, road users, etc. in various ways. For example, it displays detailed information such as the location, type, and severity of the anomaly event on the monitoring screen of the traffic management center, and sends early warning messages to the in-vehicle terminals of nearby vehicles through wireless communication technology to inform the driver of possible anomalies ahead, reminding them to take corresponding safe driving measures such as decelerating, avoiding, and changing lanes. At the same time, the early warning message can also include some suggested information for dealing with the anomaly event, such as turning on the fog lights in foggy weather and the detour route near the accident site, etc., to improve the driver's ability and safety in dealing with anomaly events;

[0112] The road anomaly events represented by the anomaly clusters include but are not limited to traffic accidents, road obstacles, traffic chaos under abnormal weather conditions, pedestrians entering the motor vehicle lane, abnormal exacerbation of traffic congestion, vehicle out of control, road waterlogging or icing, traffic signal failures, animals entering the road, and illegal occupation of the emergency lane;

[0113] Specifically:

[0114] Traffic accidents: Such as vehicle collisions, rear-end collisions, rollovers, etc. are detected through sudden hard braking, abnormal speed changes, deviation of the driving trajectory in the sensor data, and features such as vehicle deformation and scattered objects in the images. During the learning process of the model, these feature data that are significantly different from the normal driving mode are clustered into anomaly event clusters. When new data matches this cluster, an early warning is triggered, sending information such as the accident location and severity to the traffic management center, and sending a warning to surrounding vehicles to remind the driver to pay attention to avoidance and slow down;

[0115] Road obstacles: Include stationary or slowly moving obstacles such as fallen goods, broken-down vehicles, and construction facilities. The lidar and millimeter-wave radar accurately detect the location and contour of the obstacles, and the image data helps to identify the type of the obstacles. When the model finds an object that does not conform to the normal traffic pattern on the vehicle's driving path, it is determined as a road obstacle anomaly event, and an early warning is sent in time to notify the driver to change lanes or brake and avoid in advance, and at the same time inform the traffic management department to arrange cleaning or traffic guidance work;

[0116] Traffic chaos under abnormal weather conditions: Such as situations where poor weather such as fog, heavy rain, heavy snow, and strong wind leads to reduced visibility, slippery road surfaces, slow and chaotic vehicle driving, etc. The sensor data reflects features such as a general decrease in vehicle speed, abnormal increase or decrease in vehicle distance, and unstable vehicle driving direction. The model can identify these abnormal patterns by learning traffic data under different weather conditions. Once such a situation is detected, an early warning is sent to the traffic management department and the driver, reminding the driver to turn on the fog lights and slow down, and prompting the traffic management department to take corresponding traffic control measures such as speed limits and road closures;

[0117] Pedestrians entering the motor vehicle lane: The camera image data can capture the abnormal behavior of pedestrians on the motor vehicle lane. The model learns and analyzes features such as the movement trajectory, speed, and relative position of pedestrians with vehicles. When it detects that pedestrians enter the motor vehicle lane, it quickly issues a warning to remind the driver to pay attention to avoiding pedestrians and prevent collision accidents;

[0118] Abnormal intensification of traffic congestion: This includes normal traffic peak congestion and abnormal intensification of local traffic congestion caused by emergencies (such as traffic accidents, road construction, etc.). By continuously monitoring and analyzing data such as traffic flow and vehicle speed on each section of the road, and comparing with historical data and normal congestion patterns, when it is found that the traffic flow on a certain section increases rapidly, the vehicle speed drops sharply and exceeds the normal congestion range, it is determined as an abnormal congestion event, and a warning is issued in a timely manner to help the traffic management department quickly locate the congestion source and take effective dredging measures to relieve traffic pressure;

[0119] Vehicle out of control: For example, the vehicle suddenly steers, accelerates, or decelerates out of control due to reasons such as a flat tire or mechanical failure. By monitoring the abnormal acceleration and angular velocity changes of the vehicle with a millimeter-wave radar, and combining visual features such as the distortion of the vehicle's driving posture and tire conditions in the camera image, the model can learn and identify this abnormal event pattern. Once it detects that the vehicle is out of control, it immediately sends a warning to surrounding vehicles, reminding them to keep a safe distance and avoid, and at the same time notifies the traffic management department to go to deal with it, reducing the risk of chain accidents;

[0120] Road waterlogging or icing: In rainy or low-temperature weather, road waterlogging or icing will seriously affect vehicle driving safety. The lidar detects abnormal changes in the road surface reflection signal, reflecting the existence of waterlogging or icing areas. At the same time, the slipping signs of the vehicle passing through these areas in the camera image (such as wheel spin, driving trajectory deviation, etc.) are also learned by the model as important features. When it detects road waterlogging or icing, the warning system issues a warning to passing vehicles, informing the driver to slow down and drive carefully, and avoid accidents due to the slippery road surface. At the same time, the traffic management department can take measures such as salting and drainage in a timely manner to improve the road conditions;

[0121] Traffic signal failure: The camera continuously monitors the status of traffic signals. When the model finds that the signal has faults such as the red light not switching for a long time, the green light still staying on when there is no vehicle passing, and abnormal signal flashing, it issues an alarm to the traffic management center in a timely manner, so that the management department can quickly arrange maintenance personnel to repair the signal, avoiding traffic chaos and accidents caused by signal failures. At the same time, it informs the driver of the signal failure information ahead through the in-vehicle terminal, reminding them to pass through the intersection carefully, pay attention to the dynamics of surrounding vehicles and pedestrians, and pass through the intersection in an orderly manner according to traffic rules;

[0122] Animal intrusion onto the road: Especially in some road sections near mountains, forests or farmlands, wild animals or livestock may intrude onto the road. The camera captures the moving images of animals on the road. The model identifies and judges based on information such as the external features, movement trajectories and speeds of the animals, and classifies them as abnormal events. Once an animal intrusion onto the road is detected, a warning is quickly sent to the driver to remind them to decelerate and avoid in advance, preventing collision accidents and ensuring the safety of people and animals on the road. At the same time, relevant animal management departments or surrounding residents can be notified to take measures to prevent animals from entering the road area again;

[0123] Illegal occupation of the emergency lane: Using the image recognition function of the camera, the driving positions of vehicles on the road are analyzed. When the model finds that a vehicle has stayed in the emergency lane for a long time without an emergency sign, it is determined as an illegal occupation of the emergency lane behavior. This not only affects the passage of emergency rescue vehicles but may also lead to increased traffic congestion. The system sends the location and license plate information of the illegal vehicle to the traffic management center for law enforcement officers to handle. At the same time, a warning is sent to surrounding vehicles to remind drivers to abide by traffic regulations and keep the emergency lane unobstructed, ensuring that rescue work can be carried out smoothly in case of an emergency and improving the road emergency response ability.

[0124] A road abnormal event detection and warning system based on unsupervised lifelong learning, such as Figure 4 ., based on a road abnormal event detection and warning method based on unsupervised lifelong learning, including: a data collection module, an unsupervised feature extraction and model initialization module, a lifelong learning and model update module, and an abnormal event detection and warning module;

[0125] The data collection module is used to collect traffic scene data using a variety of sensors distributed along the road and on the vehicle end, and perform preprocessing;

[0126] The unsupervised feature extraction and model initialization module is used to perform unsupervised feature extraction on the preprocessed image data using an autoencoder, use the principal component analysis method for non-image data for feature dimensionality reduction, extract the main features, and initialize a clustering-based anomaly detection model using the extracted features. The K-Means clustering algorithm is used to divide the feature space into multiple clusters to form an initial set of normal clusters, and each cluster represents a normal traffic pattern;

[0127] The lifelong learning and model update module is used to receive new traffic data, and use the incremental learning method, density-based clustering algorithm, and elastic weight integration method to continuously perform online learning and update the normal traffic pattern and abnormal event pattern for the anomaly detection model;

[0128] An abnormal event detection and early warning module, which is used to input newly collected data into the updated model for abnormal detection and road abnormal event alarm.

[0129] The various embodiments in this specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference can be made to the description of the method part for the relevant parts.

[0130] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting and warning road abnormal events based on unsupervised lifelong learning, characterized in that, Including the following steps: S1. Collect traffic scene data using a variety of sensors distributed along the road and at the vehicle end, and perform preprocessing; S2. Use an autoencoder to perform unsupervised feature extraction on the preprocessed image data, and use the principal component analysis method for non-image data to reduce the dimensionality of features and extract the main features; S3. Initialize a clustering-based anomaly detection model using the extracted features, and use the K-Means clustering algorithm to divide the feature space into multiple clusters to form an initial set of normal clusters, where each cluster represents a normal traffic pattern; S4. Receive new traffic data, and use the incremental learning method, density-based clustering algorithm, and elastic weight consolidation method to continuously perform online learning on the anomaly detection model and update the normal traffic pattern and anomaly event pattern; S5. Input the newly collected data into the updated model for anomaly detection and road anomaly event alarm; The specific content of step S4 is as follows: S41. Judge the similarity between the new traffic data and the existing clusters, and use the distance metric to calculate the distance from the new data point to the center of each cluster; S42. If the distance between the new data point and a certain cluster is less than the set threshold, assign the new data point to the cluster, and update the center and statistical information of the cluster; S43. For data points whose distances to the centers of all clusters are greater than the threshold, judge them as potential abnormal data, and further analyze the potential abnormal data using the density-based clustering algorithm, create a new cluster and add it to the model to represent the abnormal traffic pattern; S44. During the model update process, use the elastic weight consolidation EWC method. When learning a new task, minimize the loss function of the new task while retaining the knowledge of the old task; The specific content of step S43 is as follows: For each data point x in the dataset X i , calculate the number of neighbor points N within a given radius ∈ ∈ (x i ); If N ∈ (x i ) is greater than or equal to the minimum number of neighbor points MinPts, then mark x i as a core point, create a new cluster, add its neighbor points to the cluster, and recursively check the neighbor points of the neighbor points until no new points can be added to the cluster; If N ∈ (x i ) is less than MinPts but within the neighborhood of a core point, then mark x i as a border point and assign it to the corresponding cluster; If the abnormal data forms a new dense area, add it to the model as a new cluster to represent the abnormal traffic pattern; The specific content of step S44 is as follows: S441. Train the model on the old task to obtain the parameters of the model after the old task training ends; S442. After the old task training ends, calculate the importance weights of each model parameter for the old task: S443. Perform new task learning, and introduce a regularization term when updating the parameters to constrain the update of the parameters, punish the updates that deviate greatly from the important parameters of the old task, so that the important parameters related to the old task will not change too much, thereby avoiding forgetting the old knowledge.

2. The method for detecting and warning road abnormal events based on unsupervised lifelong learning according to claim 1, wherein The variety of sensors includes but is not limited to cameras, millimeter-wave radars, lidars, ultrasonic sensors, GPS / Beidou positioning systems, vehicle speed sensors, acceleration sensors, and gyroscopes. The traffic scene data collected includes road images and videos, the speeds, distances, and angles of vehicles and objects, and the three-dimensional spatial information of the road and its surrounding environment.

3. A method for detecting and warning road abnormal events based on unsupervised lifelong learning according to claim 1, characterized in that, The autoencoder includes an encoder and a decoder. The encoder compresses the input image into a low-dimensional feature vector, and the decoder reconstructs the original image according to the feature vector; Train the autoencoder with minimizing the reconstruction error as the goal of the autoencoder, so that the extracted features can effectively represent the key information of the image.

4. The method for detecting and warning road abnormal events based on unsupervised lifelong learning according to claim 1, wherein, The specific content of step S3 is as follows: S31. Let N given historical traffic data points be X = {x1, x2, …, x N}, and randomly initialize K cluster centers μ = {μ1, μ2, …, μ K}; S32. For each data point x i , calculate its distance d(x i , μ j ) to each cluster center, and assign x i to the nearest cluster C j : S33. Update the center of each cluster: where |C j | is the number of data points in cluster C j ; S34. Repeat steps S32 - S33 until convergence, and form an initial set of normal clusters into K clusters C = {C1, C2, …, C K}.

5. A method for detecting and warning road abnormal events based on unsupervised lifelong learning according to claim 1, characterized in that, Step S43 further includes that data points not marked as core points or boundary points are regarded as noise points, and it continuously monitors whether they form new patterns during subsequent data accumulation.

6. The method for detecting and warning road abnormal events based on unsupervised lifelong learning according to claim 1, wherein In step S5, if the newly collected data points input to the updated model are determined to belong to a known normal cluster, the current traffic scene is considered normal; if the data points belong to an abnormal cluster, a road anomaly event alarm is triggered. The road anomaly events represented by the abnormal clusters include but are not limited to traffic accidents, road obstacles, traffic chaos under abnormal weather conditions, pedestrians entering the motor vehicle lane, abnormal aggravation of traffic congestion, vehicle out of control, road waterlogging or icing, traffic signal failures, animals entering the road, and illegal occupation of the emergency lane.

7. A road abnormal event detection and warning system based on unsupervised lifelong learning, characterized in that, A road anomaly event detection and warning method based on unsupervised lifelong learning according to any one of claims 1-6, comprising: a data acquisition module, an unsupervised feature extraction and model initialization module, a lifelong learning and model update module, and an anomaly event detection and warning module. The data acquisition module is used to collect traffic scene data by using a variety of sensors distributed along the road and on the vehicle side, and perform preprocessing. The unsupervised feature extraction and model initialization module is used to perform unsupervised feature extraction on the preprocessed image data by using an autoencoder, perform feature dimensionality reduction on non-image data by using the principal component analysis method, extract the main features, and initialize the clustering-based anomaly detection model by using the extracted features, and divide the feature space into multiple clusters by using the K-Means clustering algorithm to form an initial set of normal clusters, and each cluster represents a normal traffic pattern. The lifelong learning and model update module is used to receive new traffic data, and continuously perform online learning and update the normal traffic pattern and anomaly event pattern of the anomaly detection model by using the incremental learning method, the density-based clustering algorithm, and the elastic weight integration method. The anomaly event detection and warning module is used to input the newly collected data into the updated model for anomaly detection and road anomaly event alarm.

Images