An abnormal event early warning method and system for multimodal perception of a traffic road network

Abstract

The present invention discloses an abnormal event early warning method and system for multimodal perception of a traffic road network, which relates to the technical field of data processing. The method includes: obtaining a set of frequently-occurring path accident data of a target traffic road network, constructing a set of abnormal event prototypes, and obtaining vehicle status information of a target vehicle based on V2X communication. Probability matching is performed in combination with multimodal perception data to determine a target abnormal event prototype; a set of historical abnormal event triggering features is called to perform feature enhancement on it, and the enhanced abnormal event triggering features are used as early warning conditions to perform real-time abnormal event early warning on the driving process of the target vehicle. The present invention solves the technical problems of the prior art that rely on a single data source and lack real-time dynamic response capabilities, resulting in incomplete detection of traffic abnormal events and early warning delays, and achieves the technical effect of realizing comprehensive and real-time detection and dynamic early warning of traffic abnormal events through multimodal perception and V2X communication technologies.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to an abnormal event warning method and system for multi-modal perception of a traffic road network. Background Art

[0002] In an intelligent transportation system, the detection and warning of abnormal events are key links to ensure road safety. However, existing technologies mainly rely on a single data source (such as a camera or a sensor), making it difficult to comprehensively capture complex traffic scenarios, and lacking real-time dynamic response capabilities, resulting in low detection accuracy and large warning delays. In addition, traditional methods do not make full use of historical data and cannot effectively identify potential risks. With the development of vehicle-to-everything (V2X) communication technology and multi-modal perception technology, how to integrate multi-source data and achieve real-time dynamic warning has become an urgent problem to be solved. Summary of the Invention

[0003] This application provides an abnormal event warning method and system for multi-modal perception of a traffic road network, which is used to solve the technical problems that existing technologies rely on a single data source and lack real-time dynamic response capabilities, resulting in incomplete detection of traffic abnormal events and warning delays.

[0004] In the first aspect of this application, an abnormal event warning method for multi-modal perception of a traffic road network is provided. The method includes: obtaining a set of frequently-occurring path accident data of a target traffic road network, traversing the set of frequently-occurring path accident data to construct an abnormal event prototype cluster for the target traffic road network, and obtaining a set of abnormal event prototypes; based on V2X communication, obtaining the vehicle status information of a target vehicle; performing linked multi-modal perception data collection to obtain a multi-modal perception data set; based on the vehicle status information and the multi-modal perception data set, performing probability matching on the set of abnormal event prototypes to obtain a target abnormal event prototype; invoking the historical abnormal event triggering feature set of the target abnormal event prototype, performing feature enhancement on the historical abnormal event triggering feature set to determine enhanced abnormal event triggering features; using the enhanced abnormal event triggering features as warning triggering conditions to perform abnormal event warning on the driving process of the target vehicle.

[0005] In the second aspect of the present application, an abnormal event early warning system for multi-modal perception of a traffic road network is provided. The system includes: an abnormal event prototype construction module, which is used to obtain a set of frequently-occurring path accident data of a target traffic road network, traverse the set of frequently-occurring path accident data to construct abnormal event prototype clusters for the target traffic road network, and obtain a set of abnormal event prototypes; a vehicle status information acquisition module, which is used to obtain the vehicle status information of a target vehicle based on V2X communication; a multi-modal perception data acquisition module, which is used to perform linked multi-modal perception data acquisition to obtain a multi-modal perception data set; a target abnormal event matching module, which is used to perform probability matching on the set of abnormal event prototypes based on the vehicle status information and the multi-modal perception data set to obtain a target abnormal event prototype; an event trigger feature enhancement module, which is used to call the historical abnormal event trigger feature set of the target abnormal event prototype, perform feature enhancement on the historical abnormal event trigger feature set, and determine enhanced abnormal event trigger features; an abnormal event early warning module, which is used to use the enhanced abnormal event trigger features as early warning trigger conditions to perform abnormal event early warning on the driving process of the target vehicle.

[0006] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0007] An abnormal event early warning method and system for multi-modal perception of a traffic road network provided in the present application relate to the technical field of data processing. By obtaining traffic road network accident data to construct a set of abnormal event prototypes, combining V2X communication and multi-modal perception data, matching a target abnormal event prototype, calling and enhancing a historical trigger feature set, and using the enhanced features as early warning conditions, the driving process of the target vehicle is monitored in real time to achieve accurate abnormal event early warning. This solves the technical problems in the prior art that rely on a single data source and lack real-time dynamic response capabilities, resulting in incomplete detection and early warning delay of traffic abnormal events, and realizes the technical effects of achieving comprehensive and real-time traffic abnormal event detection and dynamic early warning through multi-modal perception and V2X communication technologies, and improving traffic safety and management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0009] Figure 1Schematic diagram of the process of an abnormal event warning method for multimodal perception of a traffic road network provided by an embodiment of the present application;

[0010] Figure 2 Schematic diagram of the structure of an abnormal event warning system for multimodal perception of a traffic road network provided by an embodiment of the present application.

[0011] Explanation of reference numerals: Abnormal event prototype construction module 11, vehicle status information acquisition module 12, multimodal perception data acquisition module 13, target abnormal event matching module 14, event trigger feature enhancement module 15, abnormal event warning module 16. Specific implementation manners

[0012] The present application provides an abnormal event warning method and system for multimodal perception of a traffic road network, which are used to solve the technical problems that the existing technology relies on a single data source and lacks real-time dynamic response capabilities, resulting in incomplete detection of traffic abnormal events and delayed warnings.

[0013] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0014] It should be noted that the terms "first", "second", etc. in the specification and the above-mentioned accompanying drawings of the present application are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products or devices.

[0015] Embodiment 1, as Figure 1 shown, the present application provides an abnormal event warning method for multimodal perception of a traffic road network, and the method includes:

[0016] P10: Obtain the path accident frequent data set of the target traffic road network, traverse the path accident frequent data set to construct abnormal event prototype clusters for the target traffic road network, and obtain an abnormal event prototype set.

[0017] Further, step P10 of the embodiment of the present application further includes:

[0018] P11: Identify and authenticate the cluster centers multiple times from the set of frequently-occurring path accident data to determine the set of cluster centers; P12: Based on the set of cluster centers, perform cluster analysis on the set of frequently-occurring path accident data from two dimensions of time similarity and feature similarity to obtain clusters of frequently-occurring path accident data, where each cluster center corresponds to a set of frequently-occurring path accident data in a cluster of frequently-occurring path accident data; P13: Traverse the clusters of frequently-occurring path accident data for embedded feature averaging processing to construct a set of abnormal event prototypes.

[0019] It should be understood that obtaining the set of frequently-occurring path accident data of the target traffic road network, which contains historical accident data such as multi-dimensional information like the time, location, weather conditions, vehicle status, road conditions, etc. of the accident occurrence, is the basis for constructing the abnormal event prototypes.

[0020] First, to ensure the accuracy and representativeness of the clustering results, identify and authenticate the cluster centers multiple times from the set of frequently-occurring path accident data. Randomly select several cluster centers each time and calculate the similarity between any two cluster centers. If the similarity between all cluster centers is less than or equal to the preset similarity threshold, then these cluster centers are considered to pass the authentication and are uniformly divided into the set of cluster centers to ensure that the cluster centers are evenly distributed in the feature space and avoid the clustering results being too concentrated or overlapping.

[0021] Next, after determining the set of cluster centers, perform cluster analysis on the set of frequently-occurring path accident data from two dimensions of time similarity and feature similarity. Time similarity reflects the proximity of accidents in time, while feature similarity involves the similarity of features such as accident type, vehicle status, road environment, etc. Through the comprehensive analysis of these two dimensions, the system divides the accident data into multiple clusters of frequently-occurring path accident data, and each cluster center corresponds to a set of frequently-occurring path accident data. This process is similar to the clustering process of the K-Means algorithm, where each cluster corresponds to a cluster center, and the data within the cluster has high similarity in the time and feature dimensions. The purpose of this step is to group the historical accident data according to time and features to provide a basis for subsequent prototype construction.

[0022] Finally, traverse each cluster path accident-prone data cluster and perform embedded feature averaging on the data therein, that is, calculate the mean of the features of each data in the cluster, such as speed, acceleration, weather index, etc., to generate a feature vector. Calculate the mean of the features of each data in the cluster to generate a feature vector, which represents the typical features of the cluster. For example, if a cluster contains multiple accidents, with an average speed of 60 km / h and an average visibility of 500 meters, the generated feature vector is [60, 500]. Finally, the feature vectors of each cluster form an abnormal event prototype (i.e., a typical abnormal event pattern composed of feature vectors), and all prototypes form an abnormal event prototype set. The purpose of this step is to abstract the typical features of each cluster into prototypes, providing a basis for subsequent abnormal event detection and early warning. The embedded feature averaging process not only improves the representativeness of the features but also reduces the impact of noisy data on the clustering results.

[0023] Through the above steps, representative abnormal event prototypes are extracted from the complex accident-prone data, providing an accurate reference for subsequent real-time monitoring and early warning.

[0024] Furthermore, step P12 of the embodiment of the present application further includes:

[0025] P12-1: Obtain a cluster quality recognition function, where the cluster quality recognition function is: ; where is the cluster quality factor, is the total number of cluster centers, is a positive integer, is the th cluster center among the M cluster centers, corresponding to the number of path accident-prone data in the th cluster path accident-prone data set, is the path accident-prone data of the th cluster center, is the th path accident-prone data in the th cluster path accident-prone data set corresponding to the th cluster center, is the weight parameter, used to control the influence degree of time similarity in loss calculation, is the time point when the path accident-prone data of the th cluster center occurs, is the th time point when the th path accident-prone data in the th cluster path accident-prone data set corresponding to the is a scale parameter for presetting the adjustment time distance, used to determine the attenuation rate of time similarity; P12-2: Use the clustering quality identification function to identify the clustering quality of the clustering path accident-prone data cluster, and obtain the clustering quality factor; P12-3: Determine whether the clustering quality factor is greater than or equal to the preset clustering quality factor. If not, re-screen the clustering center.

[0026] Optionally, further evaluate the quality of the clustering path accident-prone data cluster through the clustering quality identification function to ensure the accuracy and reliability of the clustering analysis.

[0027] First, obtain the clustering quality identification function. This function can comprehensively quantify the clustering quality by considering the feature similarity and time similarity. Then, use this function to calculate the clustering quality factor of each cluster. By traversing each cluster center and its corresponding data set, calculate the feature similarity and time similarity between the data points and the cluster center, and perform weighted summation according to the formula to finally obtain the clustering quality factor. That is, calculate the LOSS value of each cluster. The smaller this value is, the more similar the data points within the cluster are in terms of features and time, and the higher the clustering quality. In this way, those tight and consistent clusters can be identified, and these clusters represent the typical accident-prone patterns in the traffic road network.

[0028] Finally, determine whether the calculated clustering quality factor (i.e., LOSS value) is greater than or equal to the preset clustering quality factor threshold. If the threshold is not reached, it means that the current clustering quality does not meet the requirements, and the system needs to re-screen the clustering center. For example, re-execute the clustering center identification and authentication steps, select a new clustering center, and re-perform the clustering analysis and clustering quality identification until the clustering quality factor reaches the preset threshold to improve the accuracy and reliability of the clustering analysis.

[0029] Through the above process, not only the selection of the clustering center is optimized, but also the accuracy and reliability of the clustering analysis are significantly improved through quantitative evaluation and dynamic adjustment, laying a solid foundation for subsequent abnormal event warning.

[0030] P20: Based on V2X communication, obtain the vehicle status information of the target vehicle.

[0031] Specifically, based on the V2X (Vehicle-to-Everything) communication technology, the vehicle status information of the target vehicle is obtained in real time, providing key data support for subsequent traffic monitoring, abnormal event detection and warning. The V2X communication technology is an advanced technology for vehicles to interact with the surrounding environment (including other vehicles, infrastructure, pedestrians, etc.), which can achieve low-latency and high-reliability data transmission, ensuring the real-time and accuracy of vehicle status information.

[0032] First, establish a connection with the target vehicle through V2X communication, and use various communication modes such as V2V (Vehicle-to-Vehicle), V2I (Vehicle-to-Infrastructure), and V2P (Vehicle-to-Pedestrian) to comprehensively cover the interaction scenarios between the vehicle and the surrounding environment. Through this technology, the system can obtain the dynamic data of the target vehicle in real time, including position information (such as latitude and longitude coordinates), speed information, acceleration information, direction information, vehicle status (such as engine status, brake status, light status), and environmental perception data (such as surrounding vehicles, pedestrians, traffic lights, etc.).

[0033] Next, the obtained vehicle status information is transmitted to the system background in real time through the V2X communication network. The system analyzes and preprocesses the received data to ensure the integrity and accuracy of the data. At the same time, the vehicle status information is associated and analyzed with historical data and environmental data to provide support for subsequent abnormal event detection and warning.

[0034] The vehicle status information obtained through V2X communication technology can be widely applied to various scenarios. For example, in real-time traffic monitoring, the system can monitor the driving status of the target vehicle and detect abnormal behaviors in a timely manner; in collision warning, the system predicts potential collision risks based on the vehicle status information and issues warnings to the driver; in path planning, combined with the vehicle status and traffic environment, it provides the driver with the optimal path suggestion; in accident analysis, it uses the vehicle status information to analyze the cause of the accident and provides a basis for accident handling.

[0035] Through the above steps, the vehicle status information of the target vehicle can be obtained efficiently and accurately, which not only improves the real-time performance and reliability of the system, but also provides important data support for traffic monitoring, abnormal event detection and warning.

[0036] P30: Perform associated multi-modal perception data acquisition to obtain a multi-modal perception data set.

[0037] Furthermore, step P30 of the embodiment of the present application further includes:

[0038] P31: Use traffic cameras to collect video streams and extract features from the target traffic road network to obtain a road network visual feature set; P32: Use geomagnetic sensors to extract lane occupancy features from the target traffic road network to obtain lane occupancy features; P33: Use an environmental perception sensor array to perform environmental perception on the target traffic road network to obtain an environmental perception feature set; P34: Summarize the road network visual feature set, lane occupancy features, and environmental perception feature set to obtain the multi-modal perception data set.

[0039] It should be understood that by integrating a variety of perception technologies, comprehensive and multi-angle data collection is carried out on the target traffic road network, and finally a multi-modal perception dataset is generated. This step can comprehensively reflect the real-time state of the traffic road network by integrating the data of various sensors, providing rich data support for subsequent traffic monitoring, abnormal event detection and decision-making analysis.

[0040] First, traffic cameras deployed at key locations are used to collect video streams of the target traffic road network. These cameras can capture the dynamic behaviors of vehicles, traffic flow, and possible abnormal events. Key visual features, such as vehicle types, colors, license plate numbers, and driving trajectories, can be extracted from the video streams through advanced image processing and computer vision technologies. This set of visual features provides intuitive and rich information for traffic monitoring.

[0041] Next, geomagnetic sensors are deployed under the lanes of the target traffic road network, and the geomagnetic sensors are used to extract lane occupancy features of the target traffic road network. Geomagnetic sensors can detect the magnetic field changes caused by the passing of vehicles, thereby accurately identifying the lane occupancy situation. These sensors can provide real-time lane occupancy data, including the number of vehicles, types, and residence times on each lane, providing important bases for traffic flow analysis and lane management, and helping the system to grasp the usage of lanes in real time.

[0042] In addition, an environmental perception sensor array is deployed in the target traffic road network, and the environmental perception sensor array is used to perform environmental perception on the target traffic road network. These sensors can monitor environmental factors such as weather conditions (such as rain, fog, snow), road surface conditions (such as wetness, icing), and visibility, generating a set of environmental perception features, providing important references for traffic management and safety warning, and helping the system to cope with complex environmental changes.

[0043] Finally, the set of road network visual features, lane occupancy features, and environmental perception features obtained from the video streams, geomagnetic sensors, and environmental perception sensor array are summarized to form a complete multi-modal perception dataset. Exemplarily, first, the data of different modalities are aligned according to timestamps and spatial positions to ensure the consistency and relevance of the data; then, through multi-modal data fusion, the visual features, lane occupancy features, and environmental features are organically combined to form a comprehensive description of the traffic road network state; finally, the fused multi-modal perception dataset is stored in the system database, providing a data basis for subsequent traffic analysis, abnormal event detection, and decision support. This dataset integrates multi-faceted information such as vision, physics, and environment, providing comprehensive data support for the detection and early warning of traffic abnormal events.

[0044] P40: Based on the vehicle state information and the multi-modal perception data set, perform probability matching on the abnormal event prototype set to obtain a target abnormal event prototype.

[0045] Further, step P40 of the embodiment of the present application further includes:

[0046] P41: Use the vehicle state information and the multi-modal perception data set as matching vectors; P42: Use a probability function to perform probability analysis on the matching vectors and the abnormal event prototype set respectively to obtain a matching probability set; P43: Use the abnormal event prototype corresponding to the maximum value in the matching probability set as the target abnormal event prototype.

[0047] Wherein, the probability function is: ; wherein, is the probability that the matching vector belongs to the abnormal event prototype, is the matching vector, is the event that the matching vector belongs to the abnormal event prototype, is the abnormal event prototype, is the distance metric function, and .

[0048] Optionally, by comprehensively analyzing the vehicle state information and the multi-modal perception data set, perform probability matching on the abnormal event prototype set to identify the most likely abnormal event to occur.

[0049] First, fuse the real-time acquired vehicle state information and the multi-modal perception data set into a comprehensive matching vector . This vector includes the current speed, acceleration, driving direction, position information of the vehicle, as well as visual features extracted from traffic cameras, lane occupancy features obtained from geomagnetic sensors, and environmental perception features collected from the environmental perception sensor array. These information together constitute a comprehensive description of the traffic situation, providing rich data support for the identification of abnormal events.

[0050] Next, use the probability function to calculate the matching probability between the matching vector and each abnormal event prototype. Here, is a distance metric function used to calculate the distance between the matching vector and the abnormal event prototype in the feature space. The distance metric function uses the Euclidean distance, which can intuitively reflect the similarity between the two in the multi-dimensional feature space. The probability function converts the distance into a probability through an exponential function. The smaller the distance, the higher the matching probability, indicating that the matching vector is more likely to belong to the abnormal event prototype.

[0051] During the calculation process, the system traverses each prototype in the abnormal event prototype set, calculates the distances and corresponding matching probabilities between them and the matching vector respectively. These probability values reflect the matching degrees of different abnormal event prototypes with the current traffic conditions, and can provide a basis for subsequent abnormal event recognition.

[0052] Finally, select the abnormal event prototype with the maximum value from the calculated matching probability set as the target abnormal event prototype. This prototype has the highest matching probability with the current traffic conditions, so it is most likely to represent the upcoming abnormal event. In this way, the most relevant one can be identified from many potential abnormal events, providing a clear direction for real-time early warning and taking preventive measures. By comprehensively analyzing vehicle states and multi-modal perception data, the accuracy and timeliness of abnormal event detection can be significantly improved.

[0053] P50: Invoke the historical abnormal event trigger feature set of the target abnormal event prototype, perform feature enhancement on the historical abnormal event trigger feature set, and determine the enhanced abnormal event trigger features.

[0054] Further, step P50 of the embodiment of the present application further includes:

[0055] P51: Randomly extract the first historical abnormal event trigger feature and the second historical abnormal event trigger feature from the historical abnormal event trigger feature set; P52: Perform inner product mapping on the first historical abnormal event trigger feature and the second historical abnormal event trigger feature to obtain the first feature similarity set; P53: Based on the first feature similarity set, perform feature enhancement on the second historical abnormal event trigger feature respectively to obtain the first enhanced historical abnormal event trigger feature; P54: Randomly extract the third historical abnormal event trigger feature from the historical abnormal event trigger feature set again, and perform enhancement on the third historical abnormal event trigger feature based on the first enhanced historical abnormal event trigger feature to obtain the second enhanced historical abnormal event trigger feature, and so on, to obtain the enhanced abnormal event trigger features.

[0056] It should be understood that through the method of feature enhancement, features are extracted and enhanced from the historical abnormal event trigger feature set of the target abnormal event prototype to determine a more robust and representative set of abnormal event trigger features, so as to improve the accuracy of abnormal event detection and the reliability of the early warning system.

[0057] First, randomly extract two different historical anomaly event trigger features from the set of historical anomaly event trigger features, which are respectively called the first historical anomaly event trigger feature and the second historical anomaly event trigger feature. These feature samples usually include various factors related to the anomaly event, such as environmental conditions (such as rain and snow weather), vehicle status (such as vehicle speed, acceleration), road conditions (such as congestion level), etc. By randomly extracting feature samples, the system can provide diverse input data for subsequent feature enhancement.

[0058] Next, perform an inner product mapping on these two extracted historical anomaly event trigger features. Calculate the similarity between them. Inner product mapping is a method to measure the correlation between features, which can help the system judge the similarity degree between different feature samples. For example, the system can analyze the correlation between environmental factors (such as rain and snow weather) and vehicle status (such as vehicle speed) through inner product mapping, so as to determine the trigger cause of the anomaly event. Finally, the system generates a first feature similarity set to record the similarity values between feature samples.

[0059] Then, based on the first feature similarity set, perform feature enhancement on the second historical anomaly event trigger feature. For example, by using the similarity value as a weight, perform weighted adjustment on the second historical anomaly event trigger feature, so as to generate the first enhanced historical anomaly event trigger feature. This process can highlight the important information in the feature sample, while suppressing the noise data, and improve the expression ability and discrimination of the feature.

[0060] Finally, randomly extract a third historical anomaly event trigger feature from the set of historical anomaly event trigger features again, and based on the previously obtained first enhanced historical anomaly event trigger feature, perform enhancement on this newly extracted feature, so as to obtain the second enhanced historical anomaly event trigger feature. This process can be iterated. Each iteration is based on the latest enhanced feature to enhance the new feature until a complete set of enhanced anomaly event trigger features is obtained. Through multiple iterations, the system can gradually optimize the feature samples, mine the potential laws in the historical data, and thus improve the accuracy and reliability of the anomaly event trigger features. This provides a more accurate and reliable feature basis for subsequent anomaly event detection and warning, thus significantly improving the safety and efficiency of the intelligent transportation system.

[0061] Furthermore, step P53 of the embodiment of the present application further includes:

[0062] P53-1: Perform normalization processing on the first feature similarity set to obtain a first feature similarity normalization value set; P53-2: Add the first feature similarity normalization value set into an initially empty matrix, and then use a graph convolutional network to perform convolution operation on it and the second historical anomaly event trigger feature to obtain the first enhanced historical anomaly event trigger feature.

[0063] In a possible embodiment of the present application, the feature enhancement process can be further refined to ensure the accuracy and effectiveness of feature enhancement.

[0064] First, the first feature similarity set is normalized. Normalization is a data preprocessing technique that scales feature values to a specific range (usually 0 to 1) to eliminate the influence of the dimension of different features and ensure that they have the same weight in subsequent processing. This step generates the first set of normalized feature similarity values, providing a standardized similarity measure for feature enhancement.

[0065] Next, the normalized feature similarity set is added to an initially empty matrix. This matrix will serve as the input to the graph convolutional network and perform convolutional operations together with the second historical anomaly event trigger feature. The Graph Convolutional Network (GCN) is a deep learning model that can effectively learn features on graph-structured data. By integrating feature similarity information into the graph convolutional network, complex relationships between features can be captured, and the expressive power of features can be enhanced.

[0066] During the convolutional operation process, the graph convolutional network uses the set of normalized feature similarity values to adjust the weights of the second historical anomaly event trigger feature, thereby obtaining the first enhanced historical anomaly event trigger feature. These enhanced features not only contain the information of the original features but also incorporate the normalized information of feature similarity, making the features more accurate and comprehensive in expressing anomaly events. This feature enhancement method not only improves the accuracy of features but also enhances the model's ability to identify anomaly events, providing a more reliable feature basis for subsequent anomaly event warnings.

[0067] P60: Using the enhanced anomaly event trigger feature as the warning trigger condition, perform anomaly event warning on the driving process of the target vehicle.

[0068] Specifically, using the enhanced anomaly event trigger feature as the warning trigger condition, perform anomaly event warning on the driving process of the target vehicle to ensure that the vehicle can immediately receive warning information once it approaches potential anomaly event conditions during the driving process.

[0069] Exemplarily, the system continuously tracks the driving status of the target vehicle and collects relevant multi-modal data in real time. This data is used to update the matching vector of the vehicle, reflecting the current traffic conditions. Then, this real-time updated matching vector is compared with the enhanced abnormal event trigger features. If there is a match, that is, the current situation meets the early warning trigger conditions defined by any of the enhanced features, the system will evaluate the degree of match. When the degree of match exceeds the preset threshold, the system determines that a warning needs to be issued. Immediately, a warning message is generated, including the type of abnormal event that may occur, recommended countermeasures, etc., and is quickly transmitted to the driver through channels such as the vehicle's central control screen, sound alarm, or mobile device.

[0070] The purpose of this warning mechanism is to give the driver or the autonomous driving system enough time and information to take risk avoidance measures to avoid or mitigate potential traffic accidents. Through this process, a closed-loop abnormal event detection and response system is realized, which can realize real-time monitoring and warning of the driving process of the target vehicle, and improve the intelligent level and safety of traffic management.

[0071] Furthermore, the embodiment of the present application further includes step P70: obtaining the warning trigger time and continuously monitoring the target vehicle in the warning feedback monitoring window.

[0072] Optionally, the specific time of warning trigger can be further obtained, and the target vehicle is continuously monitored in the set warning feedback monitoring window to ensure that the system can provide continuous monitoring and necessary follow-up guidance before and after the possible occurrence of abnormal events, thereby enhancing the reliability and effectiveness of the warning system.

[0073] Exemplarily, first record the specific time point of warning trigger as the starting time for subsequent monitoring. Then, within the preset warning feedback monitoring window, continuously collect the real-time driving data of the target vehicle, including information such as vehicle speed, acceleration, position, direction, etc., and dynamically compare it with the enhanced abnormal event trigger features to evaluate the vehicle state change after the warning. If it is detected that the abnormal event persists or deteriorates further, the system will upgrade the warning level and take more proactive intervention measures, such as sending an emergency reminder to the driver or directly linking with the traffic management center. At the same time, the system will also record all data during the monitoring period for subsequent analysis and optimization of the warning model. Through this step, the system can realize real-time evaluation and dynamic adjustment of the warning effect, ensure that abnormal events are handled in a timely and effective manner, and further improve the safety and reliability of vehicle driving.

[0074] In summary, the embodiment of the present application has at least the following technical effects:

[0075] This application obtains a set of frequently-occurring path accident data of a target traffic road network, constructs a set of abnormal event prototypes, and obtains the vehicle status information of the target vehicle based on V2X communication. By linking multi-modal perception data collection, combining the vehicle status information and the multi-modal perception data set, probability matching is performed on the set of abnormal event prototypes to determine the target abnormal event prototype. The historical abnormal event trigger feature set is called and feature enhancement is performed, and the enhanced abnormal event trigger feature is used as the early warning condition to perform real-time abnormal event early warning on the driving process of the target vehicle.

[0076] It achieves the technical effect of realizing comprehensive and real-time traffic abnormal event detection and dynamic early warning through multi-modal perception and V2X communication technologies, and improving traffic safety and management efficiency.

[0077] Embodiment 2, based on the same inventive concept as the abnormal event early warning method for multi-modal perception of a traffic road network in the foregoing embodiment, as Figure 2 shown, this application provides an abnormal event early warning system for multi-modal perception of a traffic road network. The system in the embodiment of this application and the method embodiment are based on the same inventive concept. Among them, the system includes:

[0078] An abnormal event prototype construction module 11, which is used to obtain a set of frequently-occurring path accident data of a target traffic road network, traverse the set of frequently-occurring path accident data, and perform abnormal event prototype clustering construction on the target traffic road network to obtain a set of abnormal event prototypes.

[0079] A vehicle status information acquisition module 12, which is used to obtain the vehicle status information of the target vehicle based on V2X communication.

[0080] A multi-modal perception data collection module 13, which is used to perform linked multi-modal perception data collection to obtain a multi-modal perception data set.

[0081] A target abnormal event matching module 14, which is used to perform probability matching on the set of abnormal event prototypes based on the vehicle status information and the multi-modal perception data set to obtain a target abnormal event prototype.

[0082] An event trigger feature enhancement module 15, which is used to call the historical abnormal event trigger feature set of the target abnormal event prototype, perform feature enhancement on the historical abnormal event trigger feature set, and determine the enhanced abnormal event trigger feature.

[0083] The abnormal event warning module 16 is used to take the enhanced abnormal event triggering feature as the warning triggering condition to perform abnormal event warning on the driving process of the target vehicle.

[0084] Furthermore, the abnormal event prototype construction module 11 is further used to execute the following steps:

[0085] Perform multiple clustering center identification and authentication from the set of frequently-occurring path accident data to determine the set of clustering centers; perform clustering analysis on the set of frequently-occurring path accident data based on the set of clustering centers from two dimensions of time similarity and feature similarity to obtain the clustering path accident frequently-occurring data clusters, where each clustering center corresponds to a set of clustering path accident frequently-occurring data in the clustering path accident frequently-occurring data cluster; traverse the clustering path accident frequently-occurring data clusters to perform embedded feature averaging processing to construct the abnormal event prototype set.

[0086] Furthermore, the abnormal event prototype construction module 11 is further used to execute the following steps:

[0087] Obtain the clustering quality recognition function, where the clustering quality recognition function is: ; where is the clustering quality factor, is the total number of clustering centers, is a positive integer, is the th clustering center among the M clustering centers corresponding to the th set of frequently-occurring path accident data in the clustering path accident frequently-occurring data cluster, is the th frequently-occurring path accident data of the clustering center, is the th clustering center corresponding to the th set of frequently-occurring path accident data in the clustering path accident frequently-occurring data cluster, the th frequently-occurring path accident data, is the weight parameter, used to control the influence degree of time similarity in loss calculation, is the th time point when the frequently-occurring path accident data of the clustering center occurs, is the th clustering center corresponding to the th set of frequently-occurring path accident data in the clustering path accident frequently-occurring data cluster, the th time point when the th frequently-occurring path accident data occurs, Used to determine the decay rate of time similarity; identify the clustering quality of the clustering path accident-prone data cluster using the clustering quality identification function to obtain the clustering quality factor; determine whether the clustering quality factor is greater than or equal to the preset clustering quality factor, and if not, re-screen the clustering center.

[0088] Further, the multimodal perception data acquisition module 13 is further configured to perform the following steps:

[0089] Collect video streams and extract features from the target traffic road network using traffic cameras to obtain a road network visual feature set; extract lane occupancy features from the target traffic road network using geomagnetic sensors to obtain lane occupancy features; perform environmental perception on the target traffic road network using an environmental perception sensor array to obtain an environmental perception feature set; summarize the road network visual feature set, lane occupancy features, and environmental perception feature set to obtain the multimodal perception data set.

[0090] Further, the target abnormal event matching module 14 is further configured to perform the following steps:

[0091] Use the vehicle status information and the multimodal perception data set as matching vectors; perform probability analysis on the matching vectors and the abnormal event prototype set respectively using a probability function to obtain a matching probability set; use the abnormal event prototype corresponding to the maximum value in the matching probability set as the target abnormal event prototype.

[0092] Further, the target abnormal event matching module 14 is further configured to perform the following steps:

[0093] The probability function is: ; where is the probability that the matching vector belongs to the abnormal event prototype, is the matching vector, is the event that the matching vector belongs to the abnormal event prototype, is the abnormal event prototype, is the distance metric function, .

[0094] Further, the event trigger feature enhancement module 15 is further configured to perform the following steps:

[0095] Randomly extract the first historical anomaly event triggering feature and the second historical anomaly event triggering feature from the set of historical anomaly event triggering features; perform an inner product mapping on the first historical anomaly event triggering feature and the second historical anomaly event triggering feature to obtain a first feature similarity set; perform feature enhancement on the second historical anomaly event triggering feature respectively based on the first feature similarity set to obtain a first enhanced historical anomaly event triggering feature; randomly extract a third historical anomaly event triggering feature from the set of historical anomaly event triggering features again, and perform enhancement on the third historical anomaly event triggering feature based on the first enhanced historical anomaly event triggering feature to obtain a second enhanced historical anomaly event triggering feature, and so on, to obtain the enhanced anomaly event triggering feature.

[0096] Further, the event triggering feature enhancement module 15 is further configured to perform the following steps:

[0097] Perform normalization processing on the first feature similarity set to obtain a first set of normalized feature similarity values; add the first set of normalized feature similarity values into an initially empty matrix, and then perform a convolution operation on it and the second historical anomaly event triggering feature using a graph convolutional network to obtain the first enhanced historical anomaly event triggering feature.

[0098] Further, the system further includes:

[0099] An early warning feedback monitoring module, which is configured to obtain the early warning trigger time and continuously perform feedback monitoring on the target vehicle in the early warning feedback monitoring window.

[0100] It should be noted that the above sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of this specification have been described. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

[0102] This specification and the drawings are only exemplary descriptions of the present application and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications.

Claims

1. An abnormal event warning method for multi-modal perception of a traffic road network, characterized in that, The method includes: Obtain a set of frequently-occurring path accident data for the target traffic road network, traverse the set of frequently-occurring path accident data to construct abnormal event prototype clusters for the target traffic road network, and obtain a set of abnormal event prototypes; Specifically: Perform multiple clustering center identification and authentication on the set of frequently-occurring path accident data to determine a set of clustering centers; Based on the set of clustering centers, perform clustering analysis on the set of frequently-occurring path accident data from two dimensions of time similarity and feature similarity to obtain clusters of frequently-occurring path accident data, where each clustering center corresponds to a set of frequently-occurring path accident data in a cluster of frequently-occurring path accident data; Traverse the clusters of frequently-occurring path accident data for embedded feature averaging processing to construct a set of abnormal event prototypes; Based on V2X communication, obtain the vehicle status information of the target vehicle; Execute associated multi-modal perception data collection to obtain a multi-modal perception data set; Based on the vehicle status information and the multi-modal perception data set, perform probability matching on the set of abnormal event prototypes to obtain a target abnormal event prototype; Call the set of historical abnormal event trigger features of the target abnormal event prototype, perform feature enhancement on the set of historical abnormal event trigger features, and determine enhanced abnormal event trigger features; Specifically: Randomly extract a first historical abnormal event trigger feature and a second historical abnormal event trigger feature from the set of historical abnormal event trigger features; Perform inner product mapping on the first historical abnormal event trigger feature and the second historical abnormal event trigger feature to obtain a first set of feature similarities; Based on the first set of feature similarities, perform feature enhancement on the second historical abnormal event trigger feature respectively to obtain a first enhanced historical abnormal event trigger feature; Randomly extract a third historical abnormal event trigger feature from the set of historical abnormal event trigger features again, and perform enhancement on the third historical abnormal event trigger feature based on the first enhanced historical abnormal event trigger feature to obtain a second enhanced historical abnormal event trigger feature, and so on, to obtain the enhanced abnormal event trigger features; Use the enhanced abnormal event trigger features as the warning trigger condition to perform abnormal event warning on the driving process of the target vehicle.

2. The abnormal event warning method for multimodal perception of a traffic road network according to claim 1, wherein, Based on the set of clustering centers, perform clustering analysis on the set of frequently-occurring path accident data from two dimensions of time similarity and feature similarity to obtain clusters of frequently-occurring path accident data, including: Obtain a clustering quality identification function, where the clustering quality identification function is: ； Among them, is the clustering quality factor, is the total number of cluster centers, is a positive integer, is the number of path accident frequent data in the th cluster path accident frequent data set corresponding to the th cluster center, is the path accident frequent data of the th cluster center, is the th path accident frequent data in the th cluster path accident frequent data set corresponding to the th cluster center, is the weight parameter, used to control the influence degree of time similarity in loss calculation, is the time point when the path accident frequent data of the th cluster center occurs, is the time point when the th path accident frequent data in the th cluster path accident frequent data set corresponding to the th cluster center occurs, is the scale parameter for presetting the adjusted time distance, used to determine the attenuation speed of time similarity; Use the clustering quality identification function to identify the clustering quality of the clusters of frequently-occurring path accident data to obtain a clustering quality factor; Judge whether the clustering quality factor is greater than or equal to a preset clustering quality factor, if not, re-screen the clustering centers.

3. The abnormal event warning method for multimodal perception of a traffic road network according to claim 1, wherein Based on the vehicle status information and the multi-modal perception data set, perform probability matching on the set of abnormal event prototypes to obtain a target abnormal event prototype, including: Use the vehicle status information and the multi-modal perception data set as matching vectors; Use a probability function to perform probability analysis on the matching vectors and the set of abnormal event prototypes respectively to obtain a set of matching probabilities; Take the abnormal event prototype corresponding to the maximum value in the set of matching probabilities as the target abnormal event prototype.

4. The abnormal event warning method for multimodal perception of a traffic road network according to claim 3, wherein, The probability function is: ； Among them, is the probability that the matching vector belongs to the abnormal event prototype, is the matching vector, is the event that the matching vector belongs to the abnormal event prototype, is the abnormal event prototype, is the distance metric function, .

5. The abnormal event warning method for multimodal perception of a traffic road network according to claim 1, characterized in that, Based on the first feature similarity set, perform feature enhancement on the second historical abnormal event triggering features respectively to obtain the first enhanced historical abnormal event triggering features, including: Perform normalization processing on the first feature similarity set to obtain the first feature similarity normalization value set; Add the first feature similarity normalization value set into an initially empty matrix, and then use a graph convolutional network to perform convolutional operations on it and the second historical abnormal event triggering features to obtain the first enhanced historical abnormal event triggering features.

6. The abnormal event warning method for multi-modal perception of a traffic road network according to claim 1, characterized in that Execute associated multi-modal perception data collection to obtain a multi-modal perception data set, including: Use traffic cameras to collect video streams and extract features from the target traffic road network to obtain a road network visual feature set; Use geomagnetic sensors to extract lane occupancy features from the target traffic road network to obtain lane occupancy features; Use an environmental perception sensor array to perform environmental perception on the target traffic road network to obtain an environmental perception feature set; Summarize the road network visual feature set, lane occupancy features, and environmental perception feature set to obtain the multi-modal perception data set.

7. The abnormal event warning method for multimodal perception of a traffic road network according to claim 1, characterized in that, Obtain the warning trigger time, and continuously monitor and feedback on the target vehicle in the warning feedback monitoring window.

8. An abnormal event warning system for multimodal perception of a traffic road network, characterized in that, Steps for implementing the abnormal event warning method for multi-modal perception of a traffic road network according to any one of claims 1 to 7, the system includes: An abnormal event prototype construction module, which is used to obtain a set of path accident frequent data of the target traffic road network, traverse the set of path accident frequent data, and perform clustering construction of abnormal event prototypes on the target traffic road network to obtain a set of abnormal event prototypes; A vehicle status information acquisition module, which is used to acquire the vehicle status information of the target vehicle based on V2X communication; A multi-modal perception data collection module, which is used to execute associated multi-modal perception data collection to obtain a multi-modal perception data set; A target abnormal event matching module, which is used to perform probability matching on the set of abnormal event prototypes based on the vehicle status information and the multi-modal perception data set to obtain a target abnormal event prototype; An event trigger feature enhancement module, which is used to call the set of historical abnormal event triggering features of the target abnormal event prototype, perform feature enhancement on the set of historical abnormal event triggering features, and determine the enhanced abnormal event triggering features; An abnormal event warning module, which is used to use the enhanced abnormal event triggering features as the warning trigger condition to perform abnormal event warning on the driving process of the target vehicle.

Images