Traffic network multi-mode perception abnormal event early warning method and system

Abstract

The invention discloses a traffic network multi-mode perception abnormal event early warning method and system, and relates to the technical field of data processing, and the method comprises the steps: obtaining a path accident frequent occurrence data set of a target traffic network, constructing an abnormal event prototype set, obtaining the vehicle state information of a target vehicle based on V2X communication, and obtaining the abnormal event prototype set; probability matching is carried out in combination with the multi-modal sensing data, and a target abnormal event prototype is determined; and calling a historical abnormal event trigger feature set to perform feature enhancement, and performing real-time abnormal event early warning on the driving process of the target vehicle by taking the enhanced abnormal event trigger feature as an early warning condition. According to the invention, the technical problems of incomplete traffic abnormal event detection and early warning delay caused by dependence on a single data source and lack of real-time dynamic response capability in the prior art are solved, and the technical effects of comprehensive and real-time traffic abnormal event detection and dynamic early warning are achieved through multi-modal sensing and V2X communication technologies.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to an abnormal event warning method and system for multimodal perception of a traffic road network. Background Art

[0002] In an intelligent transportation system, the detection and warning of abnormal events are key links to ensure road safety. However, existing technologies mainly rely on a single data source (such as a camera or a sensor), making it difficult to comprehensively capture complex traffic scenarios and lacking real-time dynamic response capabilities, resulting in low detection accuracy and large warning delays. In addition, traditional methods do not make full use of historical data and cannot effectively identify potential risks. With the development of vehicle-to-everything (V2X) communication technology and multimodal perception technology, how to integrate multi-source data and achieve real-time dynamic warning has become an urgent problem to be solved. Summary of the Invention

[0003] This application provides an abnormal event warning method and system for multimodal perception of a traffic road network, which is used to solve the technical problems that existing technologies rely on a single data source and lack real-time dynamic response capabilities, resulting in incomplete detection of traffic abnormal events and warning delays.

[0004] In the first aspect of this application, an abnormal event warning method for multimodal perception of a traffic road network is provided. The method includes: obtaining a set of frequently-occurring path accident data of a target traffic road network, traversing the set of frequently-occurring path accident data to construct an abnormal event prototype cluster for the target traffic road network, and obtaining a set of abnormal event prototypes; based on V2X communication, obtaining vehicle status information of a target vehicle; performing linked multimodal perception data collection to obtain a multimodal perception data set; based on the vehicle status information and the multimodal perception data set, performing probability matching on the set of abnormal event prototypes to obtain a target abnormal event prototype; invoking a set of historical abnormal event triggering feature sets of the target abnormal event prototype, performing feature enhancement on the set of historical abnormal event triggering feature sets to determine enhanced abnormal event triggering features; and using the enhanced abnormal event triggering features as warning triggering conditions to perform abnormal event warning on the driving process of the target vehicle.

[0005] In a second aspect of the present application, an abnormal event early warning system for multimodal perception of a traffic road network is provided. The system includes: an abnormal event prototype construction module, which is used to obtain a set of frequently-occurring path accident data of a target traffic road network, traverse the set of frequently-occurring path accident data, and construct an abnormal event prototype cluster for the target traffic road network to obtain a set of abnormal event prototypes; a vehicle status information acquisition module, which is used to acquire the vehicle status information of a target vehicle based on V2X communication; a multimodal perception data acquisition module, which is used to perform linked multimodal perception data acquisition to obtain a multimodal perception data set; a target abnormal event matching module, which is used to perform probability matching on the set of abnormal event prototypes based on the vehicle status information and the multimodal perception data set to obtain a target abnormal event prototype; an event trigger feature enhancement module, which is used to call the historical abnormal event trigger feature set of the target abnormal event prototype, enhance the features of the historical abnormal event trigger feature set, and determine enhanced abnormal event trigger features; and an abnormal event early warning module, which is used to use the enhanced abnormal event trigger features as early warning trigger conditions to perform abnormal event early warning on the driving process of the target vehicle.

[0006] One or more technical solutions provided in the present application have at least the following technical effects or advantages: An abnormal event early warning method and system for multimodal perception of a traffic road network provided in the present application relate to the technical field of data processing. By obtaining traffic road network accident data to construct a set of abnormal event prototypes, combining V2X communication and multimodal perception data, matching a target abnormal event prototype, calling and enhancing a historical trigger feature set, and using the enhanced features as early warning conditions, the driving process of a target vehicle is monitored in real time to achieve accurate abnormal event early warning. This solves the technical problems in the prior art that rely on a single data source and lack real-time dynamic response capabilities, resulting in incomplete detection and early warning delays of traffic abnormal events. It realizes comprehensive and real-time traffic abnormal event detection and dynamic early warning through multimodal perception and V2X communication technologies, and improves the technical effects of traffic safety and management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0008] Figure 1Schematic flowchart of an abnormal event warning method for multimodal perception of a traffic road network provided by an embodiment of the present application; Figure 2 Schematic structural diagram of an abnormal event warning system for multimodal perception of a traffic road network provided by an embodiment of the present application.

[0009] Explanation of reference numerals: Abnormal event prototype construction module 11, vehicle status information acquisition module 12, multimodal perception data acquisition module 13, target abnormal event matching module 14, event trigger feature enhancement module 15, abnormal event warning module 16. Detailed implementation manners

[0010] The present application provides an abnormal event warning method and system for multimodal perception of a traffic road network, which is used to solve the technical problems that the existing technology relies on a single data source and lacks real-time dynamic response capabilities, resulting in incomplete detection of traffic abnormal events and early warning delays.

[0011] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0012] It should be noted that the terms "first", "second", etc. in the specification of the present application and the above accompanying drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server including a series of steps or units does not necessarily need to be limited to those clearly listed steps or units, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0013] Embodiment 1, as Figure 1 shown, the present application provides an abnormal event warning method for multimodal perception of a traffic road network, and the method includes: P10: Obtain a set of frequently-occurring path accident data of the target traffic road network, traverse the set of frequently-occurring path accident data to construct abnormal event prototype clusters for the target traffic road network, and obtain a set of abnormal event prototypes.

[0014] Further, step P10 of the embodiment of the present application further includes: P11: Perform multiple clustering center identification and authentication on the set of frequently-occurring path accident data to determine the set of clustering centers; P12: Based on the set of clustering centers, perform clustering analysis on the set of frequently-occurring path accident data from two dimensions of time similarity and feature similarity to obtain clusters of frequently-occurring path accident data, where each clustering center corresponds to a set of frequently-occurring path accident data in a cluster of frequently-occurring path accident data; P13: Traverse the clusters of frequently-occurring path accident data for embedded feature averaging processing to construct a set of abnormal event prototypes.

[0015] It should be understood that obtaining the set of frequently-occurring path accident data of the target traffic road network, which contains historical accident data such as multi-dimensional information including the time, location, weather conditions, vehicle status, road conditions, etc. of the accident, is the basis for constructing abnormal event prototypes.

[0016] First, to ensure the accuracy and representativeness of the clustering results, perform multiple clustering center identification and authentication on the set of frequently-occurring path accident data. Randomly select several clustering centers each time, and calculate the similarity between any two clustering centers. If the similarity between all clustering centers is less than or equal to the preset similarity threshold, then these clustering centers are considered to pass the authentication and are uniformly classified into the set of clustering centers, ensuring that the clustering centers are evenly distributed in the feature space and avoiding the clustering results being too concentrated or overlapping.

[0017] Next, after determining the set of clustering centers, perform clustering analysis on the set of frequently-occurring path accident data from two dimensions of time similarity and feature similarity. Time similarity reflects the proximity of accidents in time, while feature similarity involves the similarity of features such as accident type, vehicle status, and road environment. Through the comprehensive analysis of these two dimensions, the system divides the accident data into multiple clusters of frequently-occurring path accident data, and each clustering center corresponds to a set of frequently-occurring path accident data. This process is similar to the clustering process of the K-Means algorithm, where each cluster corresponds to a clustering center, and the data within the cluster has high similarity in the time and feature dimensions. The purpose of this step is to group the historical accident data according to time and features, providing a basis for subsequent prototype construction.

[0018] Finally, traverse each cluster path accident - frequent data cluster, and perform embedded feature averaging on the data therein. That is, calculate the mean value of the features of each data within the cluster, such as speed, acceleration, weather index, etc., to generate a feature vector. Calculate the mean value of the features of each data within the cluster to generate a feature vector, which represents the typical features of the cluster. For example, if a cluster contains multiple accidents with an average speed of 60 km / h and an average visibility of 500 meters, the generated feature vector is [60, 500]. Finally, the feature vectors of each cluster form an abnormal event prototype (i.e., a typical abnormal event pattern composed of feature vectors), and all prototypes form an abnormal event prototype set. The purpose of this step is to abstract the typical features of each cluster into prototypes, providing a basis for subsequent abnormal event detection and early warning. The embedded feature averaging process not only improves the representativeness of the features but also reduces the influence of noise data on the clustering results.

[0019] Through the above steps, representative abnormal event prototypes are extracted from complex accident - frequent data, providing an accurate reference for subsequent real - time monitoring and early warning.

[0020] Furthermore, step P12 of the embodiment of the present application further includes: P12 - 1: Obtain a cluster quality recognition function, where the cluster quality recognition function is: ; where is the cluster quality factor, is the total number of cluster centers, is a positive integer, is the number of path accident - frequent data in the th cluster path accident - frequent data set corresponding to the th cluster center among M cluster centers, is the path accident - frequent data of the th cluster center, is the th path accident - frequent data in the th cluster path accident - frequent data set corresponding to the th cluster center, is the weight parameter, used to control the influence degree of time similarity in loss calculation, is the time point when the path accident - frequent data of the th cluster center occurs, is the th time point when the th path accident - frequent data in the th cluster path accident - frequent data set corresponding to the is the scale parameter for presetting the adjusted time distance, Used to determine the decay rate of time similarity; P12-2: Use the clustering quality recognition function to recognize the clustering quality of the frequently-occurring accident data clusters on the clustering path, and obtain the clustering quality factor; P12-3: Determine whether the clustering quality factor is greater than or equal to the preset clustering quality factor. If not, re-screen the clustering center.

[0021] Optionally, further perform quality assessment on the frequently-occurring accident data clusters on the clustering path through the clustering quality recognition function to ensure the accuracy and reliability of the clustering analysis.

[0022] First, obtain the clustering quality recognition function. By comprehensively considering the feature similarity and time similarity, this function can comprehensively quantify the quality of the clustering. Then, use this function to calculate the clustering quality factor of each cluster. By traversing each cluster center and its corresponding data set, calculate the feature similarity and time similarity between the data points and the cluster center, and perform weighted summation according to the formula to finally obtain the clustering quality factor. That is, calculate the LOSS value of each cluster. The smaller this value is, the more similar the data points within the cluster are in terms of features and time, and the higher the clustering quality. In this way, those tight and consistent clusters can be identified, and these clusters represent the typical patterns of frequent accidents in the traffic road network.

[0023] Finally, determine whether the calculated clustering quality factor (i.e., the LOSS value) is greater than or equal to the preset clustering quality factor threshold. If the threshold is not reached, it means that the current clustering quality does not meet the requirements, and the system needs to re-screen the clustering center. For example, re-execute the clustering center recognition and authentication steps, select a new clustering center, and re-perform the clustering analysis and clustering quality recognition until the clustering quality factor reaches the preset threshold to improve the accuracy and reliability of the clustering analysis.

[0024] Through the above process, not only the selection of the clustering center is optimized, but also the accuracy and reliability of the clustering analysis are significantly improved through quantitative evaluation and dynamic adjustment, laying a solid foundation for subsequent abnormal event warning.

[0025] P20: Based on V2X communication, obtain the vehicle status information of the target vehicle.

[0026] Specifically, based on the V2X (Vehicle-to-Everything) communication technology, the vehicle status information of the target vehicle is obtained in real time, providing key data support for subsequent traffic monitoring, abnormal event detection and warning. The V2X communication technology is an advanced technology for vehicles to interact with the surrounding environment (including other vehicles, infrastructure, pedestrians, etc.), which can achieve low-latency and high-reliability data transmission, ensuring the real-time and accuracy of vehicle status information.

[0027] First, establish a connection with the target vehicle through V2X communication, and use various communication modes such as V2V (Vehicle-to-Vehicle), V2I (Vehicle-to-Infrastructure), and V2P (Vehicle-to-Pedestrian) to comprehensively cover the interaction scenarios between the vehicle and the surrounding environment. Through this technology, the system can obtain the dynamic data of the target vehicle in real time, including position information (such as longitude and latitude coordinates), speed information, acceleration information, direction information, vehicle status (such as engine status, brake status, light status), and environmental perception data (such as surrounding vehicles, pedestrians, traffic lights, etc.).

[0028] Next, the obtained vehicle status information is transmitted to the system background in real time through the V2X communication network. The system analyzes and preprocesses the received data to ensure the integrity and accuracy of the data. At the same time, the vehicle status information is correlated and analyzed with historical data and environmental data to provide support for subsequent abnormal event detection and warning.

[0029] The vehicle status information obtained through V2X communication technology can be widely applied to various scenarios. For example, in real-time traffic monitoring, the system can monitor the driving status of the target vehicle and detect abnormal behaviors in a timely manner; in collision warning, the system predicts potential collision risks based on the vehicle status information and issues warnings to the driver; in path planning, combined with the vehicle status and traffic environment, it provides the driver with the optimal path suggestion; in accident analysis, the vehicle status information is used to analyze the cause of the accident and provide a basis for accident handling.

[0030] Through the above steps, the vehicle status information of the target vehicle can be obtained efficiently and accurately, which not only improves the real-time performance and reliability of the system, but also provides important data support for traffic monitoring, abnormal event detection and warning.

[0031] P30: Perform associated multi-modal perception data collection to obtain a multi-modal perception data set.

[0032] Furthermore, step P30 of the embodiment of the present application further includes: P31: Use a traffic camera to collect video streams and extract features from the target traffic road network to obtain a road network visual feature set; P32: Use a geomagnetic sensor to extract lane occupancy features from the target traffic road network to obtain lane occupancy features; P33: Use an environmental perception sensor array to perform environmental perception on the target traffic road network to obtain an environmental perception feature set; P34: Summarize the road network visual feature set, lane occupancy features, and environmental perception feature set to obtain the multi-modal perception data set.

[0033] It should be understood that by integrating a variety of sensing technologies, comprehensive and multi-angle data collection is carried out on the target traffic road network, and finally a multi-modal sensing data set is generated. This step can comprehensively reflect the real-time state of the traffic road network by integrating the data of a variety of sensors, providing rich data support for subsequent traffic monitoring, abnormal event detection and decision-making analysis.

[0034] First, traffic cameras deployed at key locations are used to collect video streams of the target traffic road network. These cameras can capture the dynamic behaviors of vehicles, traffic flow, and possible abnormal events. Key visual features such as vehicle type, color, license plate number, and driving trajectory can be extracted from the video stream through advanced image processing and computer vision technologies. This set of visual features provides intuitive and rich information for traffic monitoring.

[0035] Next, geomagnetic sensors are deployed under the lanes of the target traffic road network, and the geomagnetic sensors are used to extract lane occupancy features of the target traffic road network. Geomagnetic sensors can detect the magnetic field changes caused by the passing of vehicles, thereby accurately identifying the lane occupancy situation. These sensors can provide real-time lane occupancy data, including the number, type, and residence time of vehicles on each lane, providing an important basis for traffic flow analysis and lane management, and helping the system to grasp the usage of lanes in real time.

[0036] In addition, an environmental perception sensor array is deployed in the target traffic road network, and the environmental perception sensor array is used to perform environmental perception on the target traffic road network. These sensors can monitor environmental factors such as weather conditions (such as rain, fog, snow), road surface conditions (such as wet, icy), and visibility, generating a set of environmental perception features, providing an important reference for traffic management and safety warning, and helping the system to cope with complex environmental changes.

[0037] Finally, the set of road network visual features, lane occupancy features, and environmental perception features obtained from the video stream, geomagnetic sensors, and environmental perception sensor array are summarized to form a complete multi-modal sensing data set. Exemplarily, first, the data of different modalities are aligned according to timestamps and spatial positions to ensure the consistency and relevance of the data; then, through multi-modal data fusion, the visual features, lane occupancy features, and environmental features are organically combined to form a comprehensive description of the traffic road network state; finally, the fused multi-modal sensing data set is stored in the system database to provide a data basis for subsequent traffic analysis, abnormal event detection, and decision support. This data set synthesizes information from multiple aspects such as vision, physics, and environment, providing comprehensive data support for the detection and early warning of traffic abnormal events.

[0038] P40: Based on the vehicle state information and the multi-modal perception data set, perform probability matching on the abnormal event prototype set to obtain a target abnormal event prototype.

[0039] Further, step P40 of the embodiment of the present application further includes: P41: Use the vehicle state information and the multi-modal perception data set as matching vectors; P42: Use a probability function to perform probability analysis on the matching vectors and the abnormal event prototype set respectively to obtain a matching probability set; P43: Use the abnormal event prototype corresponding to the maximum value in the matching probability set as the target abnormal event prototype.

[0040] Wherein, the probability function is: ; wherein, is the probability that the matching vector belongs to the abnormal event prototype, is the matching vector, is the event that the matching vector belongs to the abnormal event prototype, is the abnormal event prototype, is the distance metric function, and .

[0041] Optionally, by comprehensively analyzing the vehicle state information and the multi-modal perception data set, perform probability matching on the abnormal event prototype set to identify the most likely abnormal event.

[0042] First, fuse the real-time acquired vehicle state information and the multi-modal perception data set into a comprehensive matching vector . This vector includes the current speed, acceleration, driving direction, position information of the vehicle, as well as visual features extracted from traffic cameras, lane occupancy features obtained from geomagnetic sensors, and environmental perception features collected from the environmental perception sensor array. These information together constitute a comprehensive description of the traffic condition, providing rich data support for the identification of abnormal events.

[0043] Next, use the probability function to calculate the matching probability between the matching vector and each abnormal event prototype. Here, is a distance metric function, which is used to calculate the distance between the matching vector and the abnormal event prototype in the feature space. The distance metric function adopts the Euclidean distance, which can intuitively reflect the similarity degree between the two in the multi-dimensional feature space. The probability function converts the distance into a probability through an exponential function. The smaller the distance, the higher the matching probability, indicating that the matching vector is more likely to belong to the abnormal event prototype.

[0044] During the calculation process, the system traverses each prototype in the abnormal event prototype set, calculates the distances between them and the matching vector respectively, and the corresponding matching probabilities. These probability values reflect the matching degrees of different abnormal event prototypes with the current traffic conditions, and can provide a basis for subsequent abnormal event recognition.

[0045] Finally, select the abnormal event prototype with the maximum value from the calculated matching probability set as the target abnormal event prototype. This prototype has the highest matching probability with the current traffic conditions, so it is most likely to represent the upcoming abnormal event. In this way, the most relevant one can be identified from numerous potential abnormal events, providing a clear direction for real-time warning and taking preventive measures. By comprehensively analyzing vehicle states and multi-modal perception data, the accuracy and timeliness of abnormal event detection can be significantly improved.

[0046] P50: Invoke the historical abnormal event trigger feature set of the target abnormal event prototype, perform feature enhancement on the historical abnormal event trigger feature set, and determine the enhanced abnormal event trigger features.

[0047] Furthermore, step P50 of the embodiment of the present application further includes: P51: Randomly extract the first historical abnormal event trigger feature and the second historical abnormal event trigger feature from the historical abnormal event trigger feature set; P52: Perform inner product mapping on the first historical abnormal event trigger feature and the second historical abnormal event trigger feature to obtain the first feature similarity set; P53: Based on the first feature similarity set, perform feature enhancement on the second historical abnormal event trigger feature respectively to obtain the first enhanced historical abnormal event trigger feature; P54: Randomly extract the third historical abnormal event trigger feature from the historical abnormal event trigger feature set again, and based on the first enhanced historical abnormal event trigger feature, perform enhancement on the third historical abnormal event trigger feature to obtain the second enhanced historical abnormal event trigger feature, and so on, to obtain the enhanced abnormal event trigger features.

[0048] It should be understood that by means of feature enhancement, features are extracted and enhanced from the historical abnormal event trigger feature set of the target abnormal event prototype to determine a more robust and representative set of abnormal event trigger features, so as to improve the accuracy of abnormal event detection and the reliability of the warning system.

[0049] First, randomly extract two different historical abnormal event trigger features from the set of historical abnormal event trigger features, which are respectively called the first historical abnormal event trigger feature and the second historical abnormal event trigger feature. These feature samples usually include various factors related to abnormal events, such as environmental conditions (such as rain and snow weather), vehicle states (such as vehicle speed, acceleration), road conditions (such as congestion level), etc. By randomly extracting feature samples, the system can provide diverse input data for subsequent feature enhancement.

[0050] Next, perform an inner product mapping on these two extracted historical abnormal event trigger features. Calculate the similarity between them. Inner product mapping is a method to measure the correlation between features, which can help the system judge the similarity degree between different feature samples. For example, the system can analyze the correlation between environmental factors (such as rain and snow weather) and vehicle states (such as vehicle speed) through inner product mapping, so as to determine the trigger cause of abnormal events. Finally, the system generates a first feature similarity set to record the similarity values between feature samples.

[0051] Then, based on the first feature similarity set, perform feature enhancement on the second historical abnormal event trigger feature. For example, by using the similarity value as a weight, perform weighted adjustment on the second historical abnormal event trigger feature to generate the first enhanced historical abnormal event trigger feature. This process can highlight the important information in feature samples, while suppressing noise data, and improve the expression ability and discrimination of features.

[0052] Finally, randomly extract a third historical abnormal event trigger feature from the set of historical abnormal event trigger features again, and based on the previously obtained first enhanced historical abnormal event trigger feature, perform enhancement on this newly extracted feature to obtain the second enhanced historical abnormal event trigger feature. This process can be iterated. Each iteration performs enhancement on the new feature based on the latest enhanced feature until a complete set of enhanced abnormal event trigger features is obtained. Through multiple iterations, the system can gradually optimize feature samples, mine potential laws in historical data, and thus improve the accuracy and reliability of abnormal event trigger features. This provides a more accurate and reliable feature basis for subsequent abnormal event detection and warning, thereby significantly improving the safety and efficiency of the intelligent transportation system.

[0053] Furthermore, step P53 of the embodiment of the present application further includes: P53-1: Perform normalization processing on the first feature similarity set to obtain a first feature similarity normalization value set; P53-2: Add the first feature similarity normalization value set into an initially empty matrix, and then perform convolution operation on it and the second historical abnormal event trigger feature by using a graph convolutional network to obtain the first enhanced historical abnormal event trigger feature.

[0054] In a possible embodiment of the present application, the feature enhancement process can be further refined to ensure the accuracy and effectiveness of feature enhancement.

[0055] First, normalize the first feature similarity set. Normalization is a data preprocessing technique that scales feature values to a specific range (usually 0 to 1) to eliminate the influence of the dimension of different features and ensure that they have the same weight in subsequent processing. This step generates the first set of normalized feature similarity values, providing a standardized similarity measure for feature enhancement.

[0056] Next, add the normalized feature similarity set to an initially empty matrix. This matrix will serve as the input to the graph convolutional network and perform convolutional operations together with the second historical anomaly event trigger feature. The Graph Convolutional Network (GCN) is a deep learning model that can effectively learn features on graph-structured data. By integrating the feature similarity information into the graph convolutional network, the complex relationships between features can be captured, and the expressive power of the features can be enhanced.

[0057] During the convolutional operation, the graph convolutional network uses the set of normalized feature similarity values to adjust the weights of the second historical anomaly event trigger feature, thereby obtaining the first enhanced historical anomaly event trigger feature. These enhanced features not only contain the information of the original features but also incorporate the normalized information of the feature similarity, making the features more accurate and comprehensive in expressing anomaly events. This feature enhancement method not only improves the accuracy of the features but also enhances the model's ability to identify anomaly events, providing a more reliable feature basis for subsequent anomaly event warnings.

[0058] P60: Use the enhanced anomaly event trigger feature as the warning trigger condition to perform anomaly event warnings on the driving process of the target vehicle.

[0059] Specifically, use the enhanced anomaly event trigger feature as the warning trigger condition to perform anomaly event warnings on the driving process of the target vehicle, ensuring that once the vehicle approaches potential anomaly event conditions during the driving process, it can immediately receive warning information.

[0060] Exemplarily, the system continuously tracks the driving status of the target vehicle and collects relevant multimodal data in real time. These data are used to update the vehicle's matching vector to reflect the current traffic conditions. Next, this real-time updated matching vector is compared with the enhanced abnormal event trigger feature. If there is a match, that is, the current situation meets the warning trigger condition defined by any enhanced feature, the system will evaluate the degree of match. When the degree of match exceeds the preset threshold, the system determines that an early warning is required. Immediately, early warning information is generated, including the type of abnormal event that may occur, recommended countermeasures, etc., and is quickly delivered to the driver through channels such as the vehicle's central control screen, sound alarms or mobile devices.

[0061] The purpose of this early warning mechanism is to give the driver or the autonomous driving system enough time and information to take risk avoidance measures to avoid or mitigate potential traffic accidents. Through this process, a closed-loop abnormal event detection and response system is realized, which can realize real-time monitoring and early warning of the target vehicle's driving process, and improve the intelligence level and safety of traffic management.

[0062] Furthermore, the embodiment of the present application also includes step P70: obtaining the warning trigger time, and performing continuous feedback monitoring on the target vehicle in the warning feedback monitoring window.

[0063] Optionally, the specific time when the warning is triggered can be further obtained, and continuous feedback monitoring of the target vehicle can be performed within the set warning feedback monitoring window to ensure that the system can provide continuous monitoring and necessary follow-up guidance before and after abnormal events may occur, thereby enhancing the reliability and effectiveness of the warning system.

[0064] Exemplarily, the specific time point when the warning is triggered is first recorded as the starting time for subsequent monitoring. Then, within the preset warning feedback monitoring window, the real-time driving data of the target vehicle is continuously collected, including information such as speed, acceleration, position, direction, etc., and dynamically compared with the enhanced abnormal event triggering characteristics to evaluate the change in vehicle status after the warning. If the abnormal event is detected to persist or further deteriorate, the system will upgrade the warning level and take more active intervention measures, such as sending an emergency reminder to the driver or directly linking with the traffic management center. At the same time, the system will also record all data during the monitoring period for subsequent analysis and optimization of the warning model. Through this step, the system can realize real-time evaluation and dynamic adjustment of the warning effect, ensure that abnormal events are handled in a timely and effective manner, and further improve the safety and reliability of vehicle driving.

[0065] In summary, the embodiments of the present application have at least the following technical effects: This application obtains a set of frequently-occurring path accident data of a target traffic road network, constructs a set of abnormal event prototypes, and obtains the vehicle status information of a target vehicle based on V2X communication. By linking multi-modal perception data collection, combining the vehicle status information and the multi-modal perception data set, probability matching is performed on the set of abnormal event prototypes to determine the target abnormal event prototype. The historical abnormal event trigger feature set is called and feature enhancement is performed to use the enhanced abnormal event trigger feature as a warning condition to perform real-time abnormal event warning on the driving process of the target vehicle.

[0066] It achieves the technical effect of realizing comprehensive and real-time traffic abnormal event detection and dynamic warning through multi-modal perception and V2X communication technologies, and improving traffic safety and management efficiency.

[0067] Embodiment 2, based on the same inventive concept as the abnormal event warning method for multi-modal perception of a traffic road network in the foregoing embodiment, as Figure 2 shown, this application provides an abnormal event warning system for multi-modal perception of a traffic road network. The system in the embodiment of this application and the method embodiment are based on the same inventive concept. Among them, the system includes: An abnormal event prototype construction module 11, which is used to obtain a set of frequently-occurring path accident data of a target traffic road network, traverse the set of frequently-occurring path accident data to construct abnormal event prototype clustering for the target traffic road network, and obtain a set of abnormal event prototypes.

[0068] A vehicle status information acquisition module 12, which is used to obtain the vehicle status information of a target vehicle based on V2X communication.

[0069] A multi-modal perception data collection module 13, which is used to perform linked multi-modal perception data collection to obtain a multi-modal perception data set.

[0070] A target abnormal event matching module 14, which is used to perform probability matching on the set of abnormal event prototypes based on the vehicle status information and the multi-modal perception data set to obtain a target abnormal event prototype.

[0071] An event trigger feature enhancement module 15, which is used to call the historical abnormal event trigger feature set of the target abnormal event prototype, perform feature enhancement on the historical abnormal event trigger feature set, and determine an enhanced abnormal event trigger feature.

[0072] An abnormal event warning module 16, which is used to use the enhanced abnormal event trigger feature as a warning trigger condition to perform abnormal event warning on the driving process of the target vehicle.

[0073] Further, the abnormal event prototype construction module 11 is further configured to perform the following steps: Perform multiple clustering center identification and authentication on the path accident frequent data set to determine a set of clustering centers; perform clustering analysis on the path accident frequent data set based on the set of clustering centers from two dimensions of time similarity and feature similarity to obtain a clustered path accident frequent data cluster, where each clustering center corresponds to a set of path accident frequent data in the clustered path accident frequent data cluster; traverse the clustered path accident frequent data cluster to perform embedded feature averaging processing to construct an abnormal event prototype set.

[0074] Further, the abnormal event prototype construction module 11 is further configured to perform the following steps: Obtain a clustering quality identification function, where the clustering quality identification function is: ; where is the clustering quality factor, is the total number of clustering centers, is a positive integer, is the th clustering center among M clustering centers corresponding to the number of path accident frequent data in the th set of path accident frequent data in the clustered path accident frequent data cluster, is the path accident frequent data of the th clustering center, is the th path accident frequent data in the th set of path accident frequent data corresponding to the th clustering center in the clustered path accident frequent data cluster, is the weight parameter, used to control the influence degree of time similarity in loss calculation, is the time point when the path accident frequent data of the th clustering center occurs, is the th time point when the th path accident frequent data in the th set of path accident frequent data corresponding to the th clustering center in the clustered path accident frequent data cluster occurs, is the scale parameter for presetting the adjusted time distance,

[0075] Further, the multi-modal perception data acquisition module 13 is further configured to perform the following steps: Collect video streams and extract features from a target traffic road network using traffic cameras to obtain a road network visual feature set; extract lane occupancy features from the target traffic road network using geomagnetic sensors to obtain lane occupancy features; perform environmental perception on the target traffic road network using an environmental perception sensor array to obtain an environmental perception feature set; summarize the road network visual feature set, lane occupancy features, and environmental perception feature set to obtain the multi-modal perception data set.

[0076] Further, the target abnormal event matching module 14 is further configured to perform the following steps: Use the vehicle state information and the multi-modal perception data set as matching vectors; perform probability analysis on the matching vectors and an abnormal event prototype set respectively using a probability function to obtain a matching probability set; use the abnormal event prototype corresponding to the maximum value in the matching probability set as the target abnormal event prototype.

[0077] Further, the target abnormal event matching module 14 is further configured to perform the following steps: The probability function is: ; where is the probability that the matching vector belongs to the abnormal event prototype, is the matching vector, is the event that the matching vector belongs to the abnormal event prototype, is the abnormal event prototype, is the distance metric function, .

[0078] Further, the event trigger feature enhancement module 15 is further configured to perform the following steps: Randomly extract a first historical abnormal event trigger feature and a second historical abnormal event trigger feature from the historical abnormal event trigger feature set; perform inner product mapping on the first historical abnormal event trigger feature and the second historical abnormal event trigger feature to obtain a first feature similarity set; perform feature enhancement on the second historical abnormal event trigger feature respectively based on the first feature similarity set to obtain a first enhanced historical abnormal event trigger feature; randomly extract a third historical abnormal event trigger feature from the historical abnormal event trigger feature set again, and perform enhancement on the third historical abnormal event trigger feature based on the first enhanced historical abnormal event trigger feature to obtain a second enhanced historical abnormal event trigger feature, and so on, to obtain the enhanced abnormal event trigger feature.

[0079] Further, the event trigger feature enhancement module 15 is further configured to perform the following steps: Normalize the first feature similarity set to obtain a first set of normalized feature similarity values; add the first set of normalized feature similarity values into an initially empty matrix, and then perform a convolution operation on it and the second historical abnormal event trigger feature using a graph convolutional network to obtain the first enhanced historical abnormal event trigger feature.

[0080] Further, the system further includes: An early warning feedback monitoring module, which is used to obtain the early warning trigger time and continuously monitor the target vehicle in the early warning feedback monitoring window.

[0081] It should be noted that the above sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of this specification have been described. Additionally, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0082] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

[0083] This specification and the drawings are only exemplary descriptions of the present application and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications.

Claims

1. A method for warning abnormal events based on multimodal perception of traffic network, characterized in that: The method comprises: Obtaining a path accident frequent occurrence data set of a target traffic network, traversing the path accident frequent occurrence data set to perform abnormal event prototype clustering construction on the target traffic network, and obtaining an abnormal event prototype set; Based on V2X communication, obtain the vehicle status information of the target vehicle; Perform linkage multimodal perception data collection to obtain a multimodal perception data set; Based on the vehicle state information and the multimodal perception data set, probability matching is performed on the abnormal event prototype set to obtain a target abnormal event prototype; Calling the historical abnormal event trigger feature set of the target abnormal event prototype, performing feature enhancement on the historical abnormal event trigger feature set, and determining an enhanced abnormal event trigger feature; Taking the enhanced abnormal event triggering feature as the warning triggering condition, an abnormal event warning is issued for the driving process of the target vehicle.

2. The abnormal event warning method of multimodal perception of traffic network as claimed in claim 1 is characterized in that: Obtaining a path accident frequent data set of a target traffic network, traversing the path accident frequent data set to perform abnormal event prototype clustering construction on the target traffic network, and obtaining an abnormal event prototype set, including: Perform multiple cluster center identification and authentication from the path accident frequent data set to determine a cluster center set; From the two dimensions of time similarity and feature similarity, cluster analysis is performed on the path accident frequent data set based on the cluster center set to obtain clustered path accident frequent data clusters, wherein each cluster center corresponds to a clustered path accident frequent data set in the clustered path accident frequent data cluster; The accident-frequent data clusters of the clustering path are traversed to perform embedding feature averaging processing to construct a set of abnormal event prototypes.

3. The abnormal event warning method of multimodal perception of traffic network as claimed in claim 2 is characterized in that: From the two dimensions of time similarity and feature similarity, cluster analysis is performed on the path accident frequent data set based on the cluster center set to obtain a clustered path accident frequent data cluster, including: Obtain a cluster quality identification function, wherein the cluster quality identification function is: ； in, is the clustering quality factor, is the total number of cluster centers, is a positive integer, is the first of the M cluster centers The cluster center corresponds to The number of frequent path accident data in the clustered frequent path accident data set, For the The accident frequency data of the paths in the cluster centers, For the The cluster center corresponds to In the clustered path accident-frequent data set, The accident frequency data of each route, is the weight parameter, Used to control the influence of temporal similarity in loss calculation. For the The time point at which the frequent accident data of the paths in the cluster centers occurred. For the The cluster center corresponds to In the clustered path accident-frequent data set, The time point at which the frequent accident data of each path occurs, Adjust the scale parameters of the time distance for the preset, Used to determine the decay rate of temporal similarity; The cluster quality identification function is used to identify the cluster quality of the accident-prone clusters and obtain the cluster quality factor. It is determined whether the cluster quality factor is greater than or equal to a preset cluster quality factor. If not, cluster center screening is performed again.

4. The abnormal event warning method for multimodal perception of traffic network as claimed in claim 2 is characterized in that: Based on the vehicle state information and the multimodal perception data set, probability matching is performed on the abnormal event prototype set to obtain a target abnormal event prototype, including: Using the vehicle state information and the multimodal perception dataset as a matching vector; Use probability function to perform probability analysis on matching vectors and abnormal event prototype sets respectively to obtain matching probability sets; The abnormal event prototype corresponding to the maximum value in the matching probability set is used as the target abnormal event prototype.

5. The abnormal event warning method of multimodal perception of traffic network as claimed in claim 4 is characterized in that: The probability function is: ； in, is the probability that the matching vector belongs to the prototype of the abnormal event, is the matching vector, To match events whose vectors belong to the prototype of abnormal events, is the prototype of abnormal event, is the distance metric function, .

6. The abnormal event warning method of multimodal perception of traffic network as claimed in claim 1, characterized in that: Calling the historical abnormal event trigger feature set of the target abnormal event prototype, performing feature enhancement on the historical abnormal event trigger feature set, and determining the enhanced abnormal event trigger feature, including: Randomly extracting a first historical abnormal event triggering feature and a second historical abnormal event triggering feature from the historical abnormal event triggering feature set; Perform inner product mapping on the first historical abnormal event triggering feature and the second historical abnormal event triggering feature to obtain a first feature similarity set; Based on the first feature similarity set, feature enhancement is performed on the second historical abnormal event trigger feature to obtain a first enhanced historical abnormal event trigger feature; A third historical abnormal event trigger feature is randomly extracted again from the historical abnormal event trigger feature set, and the third historical abnormal event trigger feature is enhanced based on the first enhanced historical abnormal event trigger feature to obtain a second enhanced historical abnormal event trigger feature, and so on to obtain the enhanced abnormal event trigger feature.

7. The abnormal event warning method of multimodal perception of traffic network as claimed in claim 6 is characterized in that: Based on the first feature similarity set, feature enhancement is performed on the second historical abnormal event trigger feature to obtain the first enhanced historical abnormal event trigger feature, including: Normalizing the first feature similarity set to obtain a first feature similarity normalized value set; The first feature similarity normalized value set is added to an initially empty matrix, and then a graph convolution network is used to perform a convolution operation on it and the second historical abnormal event trigger feature to obtain the first enhanced historical abnormal event trigger feature.

8. The abnormal event warning method for multimodal perception of traffic network as claimed in claim 1, characterized in that: Perform linked multimodal perception data collection to obtain a multimodal perception data set, including: Use traffic cameras to collect video streams and extract features of the target traffic road network to obtain a set of visual features of the road network; Using a geomagnetic sensor to extract lane occupancy features of the target traffic network to obtain lane occupancy features; Using an environment perception sensor array to perform environment perception on the target traffic network to obtain an environment perception feature set; The road network visual feature set, lane occupancy feature set and environment perception feature set are aggregated to obtain the multimodal perception data set.

9. The abnormal event warning method of multimodal perception of traffic network as claimed in claim 1, characterized in that: Obtain the warning trigger time and perform continuous feedback monitoring on the target vehicle in the warning feedback monitoring window.

10. A multi-modal perception abnormal event warning system for traffic network, characterized in that: The system comprises: An abnormal event prototype construction module, the abnormal event prototype construction module is used to obtain a path accident frequent data set of a target traffic network, traverse the path accident frequent data set to perform abnormal event prototype clustering construction on the target traffic network, and obtain an abnormal event prototype set; A vehicle status information acquisition module, wherein the vehicle status information acquisition module is used to acquire vehicle status information of a target vehicle based on V2X communication; A multimodal sensing data acquisition module, wherein the multimodal sensing data acquisition module is used to perform linked multimodal sensing data acquisition to obtain a multimodal sensing data set; A target abnormal event matching module, wherein the target abnormal event matching module is used to perform probability matching on the abnormal event prototype set based on the vehicle state information and the multimodal perception data set to obtain a target abnormal event prototype; An event trigger feature enhancement module, the event trigger feature enhancement module is used to call the historical abnormal event trigger feature set of the target abnormal event prototype, perform feature enhancement on the historical abnormal event trigger feature set, and determine an enhanced abnormal event trigger feature; An abnormal event warning module is used to use the enhanced abnormal event triggering feature as a warning triggering condition to issue an abnormal event warning for the driving process of the target vehicle.

Images