Highway traffic anomaly detection method and system based on multi-modal data fusion

Abstract

The invention discloses a road traffic anomaly detection method and system based on multi-modal data fusion, and relates to the field of traffic data processing, and the method comprises the steps: monitoring and obtaining multi-modal road traffic data, carrying out the traffic anomaly recognition based on the road traffic data of each modal, and obtaining a candidate abnormal event; obtaining candidate causes based on the space-time range of the candidate abnormal events, the similar historical abnormal events and the multi-modal road traffic data; based on the candidate abnormal events and the candidate causes, contribution degree analysis is carried out, multiple analysis results are obtained in combination with the multi-modal road traffic data, the analysis result with the highest credibility in the analysis results is extracted to serve as a road traffic anomaly detection result, and the analysis results comprise the abnormal events and the main causes. According to the invention, the technical problems of inaccurate road traffic abnormity detection result and insufficient reliability of cause analysis result in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of traffic data processing, and more specifically to a method and system for detecting highway traffic anomalies using multimodal data fusion. Background Technology

[0002] With the deepening of intelligent transportation construction, in order to reduce highway traffic accidents and ensure highway traffic conditions, the importance attached to condition monitoring and abnormal event detection of closed road sections such as tunnels is gradually increasing.

[0003] However, existing methods for detecting traffic anomalies cannot quantify the impact of different causes and lack analysis of the connections between anomaly causes. For diverse anomaly causes, it is impossible to quantify and assess the degree of influence of each candidate cause on the current anomaly, resulting in chaotic attribution of highway traffic anomaly detection results and hindering accurate emergency response. Summary of the Invention

[0004] This application provides a method and system for detecting highway traffic anomalies using multimodal data fusion, which addresses the problems of inaccurate detection results and insufficient reliability of causal analysis results in existing technologies.

[0005] In view of the above problems, this application provides a method and system for detecting highway traffic anomalies by multimodal data fusion.

[0006] In a first aspect, this application provides a method for detecting highway traffic anomalies based on multimodal data fusion, the method comprising: The system monitors and acquires multimodal highway traffic data, and identifies traffic anomalies based on the traffic data of each mode to obtain candidate anomaly events. Based on the spatiotemporal range of the candidate abnormal events, similar historical abnormal events, and multimodal highway traffic data, candidate causes are obtained; Based on the candidate abnormal events and candidate causes, a contribution analysis is performed. Combined with the multimodal highway traffic data, multiple analysis results are obtained. The analysis result with the highest confidence among the analysis results is extracted as the highway traffic anomaly detection result. The analysis result includes the abnormal event and its main causes.

[0007] Secondly, this application provides a highway traffic anomaly detection system based on multimodal data fusion, the system comprising: The abnormal event acquisition module is used to monitor and acquire multimodal highway traffic data, and to identify traffic anomalies based on the traffic data of each mode to acquire candidate abnormal events; The candidate cause acquisition module is used to acquire candidate causes based on the spatiotemporal range of the candidate abnormal events, similar historical abnormal events, and multimodal highway traffic data. The anomaly detection and analysis module is used to perform contribution analysis based on the candidate anomaly events and candidate causes, combine the multimodal highway traffic data to obtain multiple analysis results, and extract the analysis result with the highest confidence among the analysis results as the highway traffic anomaly detection result, wherein the analysis result includes the anomaly event and its main cause.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application first acquires multimodal highway traffic data through monitoring. Leveraging the complementarity of image, speed, and road data in terms of spatial, dynamic, and management information, it expands the detection coverage of abnormal events, improves the accuracy of initial screening, and lays a reliable data foundation for subsequent precise attribution. Second, by considering the spatiotemporal range of candidate abnormal events, similar historical abnormal events, and multimodal highway traffic data, candidate causes are identified. Cause deduction is performed based on spatial attributes, historical patterns, and multimodal data to achieve preliminary correlation of causes. Candidate abnormal events are proactively identified by fusing real-time road data. Next, contribution analysis is performed on candidate abnormal events. Combining multimodal highway traffic data, multiple analysis results are obtained. The evidence support, spatiotemporal correlation, and logical rationality of each candidate cause are calculated to obtain the causal contribution, generating multiple analysis results. Finally, by calculating the credibility of the analysis results, the analysis result with the highest credibility is selected as the highway traffic anomaly detection result, providing a reliable decision-making basis for highway traffic anomaly detection. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the highway traffic anomaly detection method based on multimodal data fusion proposed in this application. Figure 2 This is a schematic diagram of the structure of the highway traffic anomaly detection system based on multimodal data fusion proposed in this application.

[0010] In the attached diagram, the components represented by each number are as follows: Abnormal event acquisition module 11, candidate cause acquisition module 12, and abnormal detection and analysis module 13. Detailed Implementation

[0011] This application provides a method and system for detecting highway traffic anomalies using multimodal data fusion, which addresses the problems of inaccurate detection results and insufficient reliability of causal analysis results in existing technologies.

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] It should be noted that the terms "comprising" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0014] The present invention will now be described in detail with reference to the accompanying drawings.

[0015] Example 1, as Figure 1 As shown, this application provides a method for detecting highway traffic anomalies based on multimodal data fusion, the method comprising: S10: Monitor and acquire multimodal highway traffic data, and identify traffic anomalies based on each mode of highway traffic data to obtain candidate anomaly events; In this embodiment of the application, monitoring and acquisition are carried out in real time and continuously by a sensor network installed along the highway; multimodal highway traffic data are multi-source heterogeneous data that can reflect the highway traffic status and are obtained from different physical sensors or information systems; traffic anomaly identification is to use algorithms or rules to analyze data of a single mode and determine whether it deviates from the normal traffic pattern; candidate anomaly events are a set of possible anomalies that are preliminarily determined after analyzing different data sources individually.

[0016] Specifically, raw data is first collected in parallel from multiple data sources. Then, for each type of data, preliminary anomaly detection is performed, and all anomaly identifications are aggregated to obtain a set of candidate anomaly events.

[0017] Step S10 in the method provided in this application embodiment includes: Monitoring and acquiring multimodal highway traffic data, wherein the multimodal highway traffic data includes image data, speed data, and road data; Traffic anomalies are identified based on traffic data from each mode of highway, and candidate anomaly events are obtained.

[0018] In this embodiment of the application, multimodal highway traffic data is first monitored and acquired. The multimodal highway traffic data includes image data, speed data, and road data. The image data is a video stream captured by a visual sensor such as a camera; the speed data is a parameter that reflects the speed characteristics of traffic flow and is measured by a traffic flow detection device; and the road data is structured information related to the physical state, management state, and ancillary events of the road.

[0019] Specifically, image data is acquired from network cameras deployed at key locations such as tunnel entrances and exits; speed data is obtained by collecting vehicle speed data at a fixed frequency from traffic flow detection equipment deployed along the route; and road data is obtained from the road maintenance operation management system, traffic monitoring system, and tunnel equipment monitoring system. The timestamps of these three types of multimodal highway traffic data—image data, speed data, and road data—are then unified and sent to subsequent processing steps.

[0020] For example, real-time video streams from network cameras at two locations near the tunnel entrance and exit, with a resolution of 1920×1080, are used to obtain image data. Speed ​​measuring instruments installed at the entrance and exit sections obtain speed data for multiple passing vehicles; road data is obtained based on the road maintenance operation management system, such as the maintenance management system showing no planned maintenance or construction reports for the tunnel today; the traffic monitoring platform showing no speed limits, closures, or detours issued for the tunnel and connected road sections; and the tunnel equipment monitoring system providing status telemetry data showing normal tunnel lighting, normal fan operation, and no fire alarm signals.

[0021] Secondly, traffic anomaly identification is performed based on the traffic data of each modality to obtain candidate anomaly events. Specifically, traffic anomaly identification is performed in parallel on the three types of traffic data acquired: image data, speed data, and road data. The results of the traffic anomaly identification are then summarized to obtain candidate anomaly events that can reflect multi-dimensional anomalies.

[0022] In step S10 of the method provided in this application embodiment, the step of identifying traffic anomalies based on traffic data from each modal highway and obtaining candidate anomaly events includes: Based on the image data, vehicle information is analyzed and matched to obtain vehicle traffic data. Based on the speed data, vehicle speed statistical characteristics are analyzed and obtained, wherein the vehicle speed statistical characteristics include at least maximum speed, minimum speed, average speed and speed variation parameters; Based on the road data, road segment information is obtained through analysis, and combined with the image data, the road segment travel time is obtained. By combining the vehicle traffic data, the statistical characteristics of vehicle speed, and the travel time of the road segment, anomaly identification is performed to obtain candidate abnormal events.

[0023] In this embodiment, vehicle information is first obtained by analyzing image data and performing information pairing to obtain vehicle traffic data. Vehicle information is descriptive attributes about the vehicle parsed from a single frame or consecutive image frames. Information pairing is the process of associating vehicle information belonging to the same physical vehicle detected at different times and under different camera views in multiple consecutive image frames. Vehicle traffic data is process data reflecting traffic flow formed by continuously tracking and pairing individual vehicles.

[0024] Specifically, the process begins by identifying vehicles in each frame of the video stream using an object detection model and obtaining their bounding boxes and categories. Then, a multi-object tracking algorithm is used to compare the appearance and motion features of vehicles between adjacent frames, performing cross-frame tracking and information matching for each detected vehicle. Finally, for each vehicle, records are generated including the time and location coordinates before and after entering and exiting the tunnel, as well as the time spent in the tunnel, serving as vehicle passage data.

[0025] Secondly, based on the speed data, the statistical characteristics of vehicle speed are analyzed and obtained. These statistical characteristics include at least the maximum speed, minimum speed, average speed, and speed variation parameters. The speed data is the raw sampling data obtained from the cross-section detector. The statistical characteristics of vehicle speed are indicators obtained by mathematically statistically analyzing the speed values ​​of all passing vehicles within a specific time period and a specific cross-section. The speed variation parameters are indicators that measure the degree of dispersion of the speed distribution.

[0026] Specifically, statistical analysis is performed on the speed data of all passing vehicles within a fixed time window and spatial window. The maximum, minimum, and average speeds of all passing vehicles within the time period are calculated, along with a speed variation parameter describing the magnitude of speed fluctuations. The speed variation parameter is the ratio of the standard deviation to the average speed data of all passing vehicles within the fixed time window. When traffic is smooth in the tunnel, the average speed is high and the speed variation parameter is small; when traffic is congested in the tunnel, the average speed is low, and the speed variation parameter may initially increase and then decrease.

[0027] For example, based on the cross-section detectors at the tunnel entrance and exit, speed data for the tunnel entrance and exit are acquired within a time window from 2:00 PM to 2:02 PM: [78, 65, 75, 75, 70] km / h. The maximum speed is 78 km / h, the minimum speed is 65 km / h, the average speed is 72.6 km / h, the speed standard deviation is 4.59 km / h, and the speed variation parameter is 0.63.

[0028] Secondly, based on road data, road segment information is obtained through analysis and combined with image data to obtain road segment travel time. The road segment information consists of attributes related to travel time extracted from the road data, including the maintenance status and natural conditions of the road segment. The maintenance status may include the length of the road segment and the speed limit, while the natural conditions may include specific weather conditions.

[0029] Specifically, the process begins by analyzing road data to obtain information about the current road segment, including the tunnel's length and speed limit. Then, image data is analyzed in conjunction with this information: vehicle image data is used to determine the tunnel's entrance and exit. Combining this with the distance information from the road data and relevant management rules, the current travel time is calculated. The current travel time is calculated as the tunnel's length divided by the current speed limit. If the weather is sunny or cloudy, the travel time may not be significantly affected; however, if the weather is rainy or snowy, the travel time may be one or more times the calculated result.

[0030] For example, based on road data, the length of a tunnel is 5 kilometers, the speed limit is 80 km / h, and the weather is sunny. The current travel time is calculated as 5 / 80 × 3600 = 225 seconds.

[0031] Finally, by combining vehicle traffic data, vehicle speed statistical characteristics, and road segment travel time, anomaly identification is performed to obtain candidate abnormal events. Specifically, anomaly identification is performed by combining vehicle traffic data, vehicle speed statistical characteristics, and road segment travel time, and according to anomaly identification rules.

[0032] In step S10 of the method provided in this application embodiment, the step of combining the vehicle traffic data, the vehicle speed statistical characteristics, and the road segment travel time to perform anomaly identification and obtain candidate abnormal events includes: Acquire historical anomalies and integrate them into a sample output set; Obtain historical vehicle traffic data, historical vehicle speed statistical characteristics, and historical road segment travel time corresponding to the historical abnormal events, and integrate them into a sample input set; An anomaly detection plugin is constructed, and the anomaly detection plugin is trained using the sample input set and the sample output set until convergence; The vehicle traffic data, vehicle speed statistical characteristics, and road segment travel time are input into the anomaly identification plugin to obtain candidate anomaly events.

[0033] In this embodiment of the application, historical abnormal events are first obtained and integrated into a sample output set. The historical abnormal events are real instances of abnormal road traffic events that occurred in the past time period and were recorded by manual confirmation or by a verified automated system. The sample output set is a set of target variables or labels used for training supervised learning models in the context of machine learning.

[0034] Specifically, it involves acquiring historical anomalous events from a past period, integrating them, and using them as a sample output set.

[0035] Secondly, historical vehicle traffic data, historical vehicle speed statistical features, and historical road segment passage times corresponding to historical abnormal events are obtained and integrated into a sample input set. The sample input set is a set of feature vectors composed of multimodal features corresponding to each historical abnormal event in the sample output set in machine learning.

[0036] Specifically, based on the occurrence time and location of each historical anomaly in the sample output set, the original multimodal data of the corresponding road segment is obtained from the historical database. Then, using the same processing method, historical vehicle traffic data, historical vehicle speed statistical characteristics, and historical road segment travel time calculated based on the road conditions at that time are analyzed from the historical raw data. Finally, the three types of features are vectorized and concatenated to form the feature vector corresponding to each anomaly in the sample output set, i.e., the sample input set.

[0037] Next, an anomaly detection plugin is constructed and trained using the sample input set and the sample output set until convergence. Specifically, firstly, a suitable machine learning model is selected to construct the anomaly detection plugin based on the task characteristics. Then, the anomaly detection plugin is iteratively trained using the sample input set and the sample output set. In each training round, the model predicts the sample input set based on the current parameters, compares it with the true labels to calculate the error, and then automatically adjusts the parameters using the backpropagation algorithm to reduce the error. This process is repeated until the model's prediction accuracy on the training set stabilizes at a high level, achieving convergence, and finally obtaining the anomaly detection plugin.

[0038] For example, a fully connected neural network is first used to construct an anomaly detection plugin. The main structure of the anomaly detection plugin includes an input layer, a hidden layer, and an output layer. The input layer consists of multiple neurons and is used to directly receive the processed real-time feature vector. For example, the input feature vector is composed of vehicle traffic data hypothetically encoded as 10-dimensional features, vehicle speed statistical features hypothetically encoded as 5-dimensional features, and road segment travel time hypothetically encoded as 1-dimensional features, concatenated together. Therefore, the dimension of the input layer is 16, that is, the input layer contains 16 neurons, corresponding to 16 input features. The hidden layer is used to perform complex nonlinear transformations on the data to extract deep features of the data. For example, 16 neurons are used. Each neuron in the hidden layer obtains input from the neurons in the previous layer, and performs nonlinear transformations through weighted sum operations, bias term correction, and the ReLU activation function as the input to the next layer. The output layer is used to generate the final prediction result. Since this task is to identify whether an anomaly has occurred, it is a binary classification problem. For example, the output layer contains 1 neuron, which is fully connected to the neurons in the hidden layer, and the output value is compressed to the (0,1) interval through the Sigmoid activation function. The output value is the probability that the current traffic state is an abnormal event. The closer the value is to 1, the higher the confidence level of the model in judging it as a candidate abnormal event.

[0039] Secondly, the sample input set is used as the model input, and the sample output set is used as supervision. The anomaly detection plugin is trained using a fully connected neural network. The sample output set is used as the model input and is divided into training, validation and test sets in a 7:2:1 ratio.

[0040] Finally, an initial learning rate of 0.0001 was set, and the Adam optimizer was used for training. Mean squared error (MSE) was used to measure the difference between predicted and true values. Through forward propagation, the input data was weighted and summed using weights and biases, and a nonlinear transformation was performed using an activation function. Through multiple nonlinear transformations, the complex relationship between the input data and the target output was learned. Subsequently, backpropagation was used to calculate the gradient information of the output error, and gradient descent was used to update the weights and biases in the network. Through multiple iterations of optimization, the model's prediction error was continuously reduced. After training, the model's performance on a reserved validation set was evaluated. If the model's recognition accuracy reached above 95% and the loss function value no longer decreased, the training was considered converged, and the anomaly detection plugin was obtained.

[0041] Finally, vehicle traffic data, vehicle speed statistics, and road segment travel time are input into the anomaly detection plugin to obtain candidate anomaly events. Specifically, real-time features are integrated into a feature vector using the same format as the training samples. This real-time feature vector is then input into the already trained and converged anomaly detection plugin. Based on patterns learned from historical data, the anomaly detection plugin predicts the current traffic state and outputs candidate anomaly events.

[0042] In this embodiment, a complete data foundation is constructed using multimodal data, including image, speed, and road data, ensuring the comprehensiveness of the anomaly detection information sources. Subsequently, anomaly identification is performed by extracting vehicle traffic data, speed statistical features, and road segment travel time from each modal data. Through preliminary fusion of multimodal features, the accuracy of anomaly identification is improved. Furthermore, the anomaly identification plugin is trained based on historical data to enhance the generalization ability of candidate anomaly event identification, providing high-quality input for subsequent accurate attribution.

[0043] S20: Based on the spatiotemporal range of the candidate abnormal events, similar historical abnormal events, and multimodal highway traffic data, obtain candidate causes; In this embodiment of the application, the spatiotemporal range refers to the specific geographical location and time period in which the candidate abnormal event occurred; similar historical abnormal events are historical records in the historical database that are similar in characteristics to the current candidate abnormal event; and candidate causes are various potential cause hypotheses that may lead to the occurrence of the current abnormal event.

[0044] Specifically, firstly, based on the spatiotemporal scope of the event, common risk sources in the location of the event are associated with it. Secondly, similar historical anomalies are retrieved, and historical causes are considered as possible candidates. Subsequently, details in multimodal highway traffic data are examined to look for clues that may indicate specific causes. Finally, all possible causes discovered through various means are aggregated to form candidate causes.

[0045] Step S20 in the method provided in this application embodiment includes: Based on the spatiotemporal range of the candidate abnormal events, the causes of the abnormalities are obtained and added to the candidate causes; Based on the similar historical anomalies of the candidate anomalies, retrieve the corresponding historical main causes and add them to the candidate causes; Based on the multimodal highway traffic data, abnormal road sections are identified and added to the candidate causes.

[0046] In this embodiment of the application, the cause of the anomaly is first obtained based on the spatiotemporal range of the candidate abnormal event and added to the candidate cause. For example, if a traffic accident occurs within a 10-minute range on a road section within 5km of the candidate abnormal event, then this traffic accident is added to the candidate cause.

[0047] Secondly, based on similar historical anomalies of the candidate anomaly, the corresponding main historical causes are retrieved and added to the candidate causes. Similar historical anomalies are event records in the historical event database that are similar to the current candidate anomaly in terms of key features. The retrieval is based on the features of the current event as the query conditions, and similarity matching and searching are performed in the historical database to return the historical event records that are successfully matched.

[0048] Specifically, firstly, similar historical anomalies to the candidate anomaly are retrieved from the historical event database. Then, using these similar historical anomalies as search criteria, the primary historical causes of the anomalies are identified and added to the candidate causes. For example, if a candidate anomaly is slow vehicle traffic, and the primary causes of abnormal vehicle speeds in the historical event database include traffic accidents and road equipment malfunctions, then traffic accidents and road equipment malfunctions are added to the candidate causes.

[0049] Finally, based on multimodal highway traffic data, abnormal road sections are identified and added to the candidate causes. Abnormal road sections are tunnel sections where the road surface itself exhibits abnormalities. Specifically, based on the multimodal highway traffic data, the image data is analyzed for abnormal visibility, road surface debris, etc.; the speed data is analyzed to determine if abnormal patterns match the characteristics of traffic flow disturbances caused by equipment malfunctions; and the road data is checked for real-time alarms from the equipment monitoring system, such as tunnel ventilation fan malfunctions. After identifying abnormal road sections from the multimodal highway traffic data, they are added to the candidate causes.

[0050] For example, suppose we obtain abnormal road segments: vehicle malfunction, road equipment malfunction, traffic accident, etc., and add them to the candidate causes to facilitate subsequent anomaly attribution.

[0051] In this embodiment, by acquiring the spatiotemporal range and combining it with the inherent spatial attributes of roads to associate typical risks, prior knowledge reasoning about location is achieved. Through the retrieval of similar historical anomalies, statistical patterns in historical data are utilized to use historical high-incidence causes as important empirical references, enabling inductive reasoning based on historical patterns. Subsequently, by acquiring multimodal highway traffic data, the road segment's own condition is diagnosed in real time, and clues from images, speed, and road data are directly used to infer causes, achieving consistency analysis based on real-time evidence and ensuring the rationality of the candidate cause set.

[0052] S30: Based on the candidate abnormal events and candidate causes, perform contribution analysis, combine the multimodal highway traffic data, obtain multiple analysis results, and extract the analysis result with the highest confidence among the analysis results as the highway traffic anomaly detection result, wherein the analysis result includes abnormal events and main causes.

[0053] In this embodiment of the application, contribution analysis is a process of quantitatively evaluating the extent to which each candidate cause plays a role in the occurrence of the abnormal event; multiple analysis results are calculated by the system through different inference paths or weights, outputting multiple possible events and main cause hypotheses; credibility is a comprehensive evaluation index of the overall reliability of the analysis results, which is usually calculated by integrating multiple evaluation dimensions.

[0054] Specifically, when faced with anomalous events and multiple candidate causes, the system constructs an analytical model to perform deep correlation calculations between the candidate causes and multimodal highway traffic data, assessing the contribution of each cause. Based on different combinations of primary causes, several most probable hypotheses for analytical results are generated. Each result includes a description of the anomalous event and a quantification or ranking of its contribution to the cause. Subsequently, a final judgment is made on the multiple potentially correct analytical results generated. A credibility assessment model is used to analyze each result, and the analytical result with the highest credibility is taken as the highway traffic anomaly detection result.

[0055] Step S30 in the method provided in this application embodiment includes: For each of the candidate abnormal events, perform candidate cause matching to obtain multiple candidate cause sets; Contribution analysis is performed on multiple candidate causes in the candidate cause set to obtain the cause contribution. Based on the contribution of the causes, candidate main causes are obtained, and the candidate abnormal events and candidate main causes are integrated into candidate analysis results. The candidate analysis results are evaluated for credibility to obtain the credibility of the candidate analysis results, and the candidate analysis results with the highest credibility are selected as the highway traffic anomaly detection results.

[0056] In this embodiment of the application, candidate cause matching is first performed on each candidate abnormal event to obtain multiple candidate cause sets. The candidate cause matching set is a dataset consisting of a candidate abnormal event and one or more candidate causes, obtained by matching the causes of each independent candidate abnormal event from the candidate causes.

[0057] Furthermore, a contribution analysis is performed on multiple candidate causes in the candidate cause set to obtain the cause contribution degree. The contribution analysis is a process of quantitatively evaluating the impact of each cause in a candidate cause set on the candidate abnormal event in order to determine the extent to which each cause plays a role in the occurrence of the final abnormal event. The cause contribution degree is a quantitative value of the degree of influence of a specific cause on the abnormal event, calculated through the contribution analysis.

[0058] Specifically, the multiple candidate cause sets are analyzed separately, and the contribution of multiple candidate causes in each candidate cause set is analyzed to obtain the causal contribution of each candidate cause set. Then, based on the causal contribution, the specific cause of the abnormal event is obtained.

[0059] Next, based on the contribution of each cause, candidate principal causes are obtained, and the candidate anomalous events and candidate principal causes are integrated into candidate analysis results. Specifically, based on the calculated contribution of each cause, the cause with the largest contribution is selected as the candidate principal cause; then, the candidate principal cause is integrated with the original candidate anomalous events to form candidate analysis results, where one candidate principal cause may generate multiple candidate analysis results.

[0060] Finally, the credibility of the candidate analysis results is assessed to obtain their credibility level, and the candidate analysis result with the highest credibility level is selected as the highway traffic anomaly detection result. Specifically, multiple candidate analysis results are comprehensively evaluated, and each candidate analysis result is fused and calculated to obtain a corresponding final credibility score. Based on the credibility score, the candidate analysis results are selected, and the candidate analysis result with the highest credibility level is selected as the final highway traffic anomaly detection result.

[0061] In step S30 of the method provided in this application embodiment, contribution analysis is performed on multiple candidate causes in the candidate cause set to obtain the cause contribution, including: Spatiotemporal co-occurrence analysis was performed on multiple candidate causes to obtain spatiotemporal co-occurrence parameters; Modal data support analysis is performed on multiple candidate causes in conjunction with the multimodal highway traffic data to obtain modal data support parameters; Logical rationality analysis is performed on multiple candidate causes to obtain logical rationality parameters; By combining the spatiotemporal co-occurrence parameter, the modal data support parameter, and the logical rationality parameter, a contribution analysis is performed to obtain the causal contribution of multiple candidate causes.

[0062] In this embodiment, a spatiotemporal co-occurrence analysis is first performed on multiple candidate causes to obtain spatiotemporal co-occurrence parameters. Specifically, based on multiple candidate causes of candidate events obtained from historical data, the measured spatiotemporal characteristics of the current candidate abnormal event are matched with the spatiotemporal co-occurrence standard value of each cause, considering both time and spatial ranges. For example, the temporal and spatial similarities between multiple candidate causes and candidate abnormal events are calculated. If the time deviation is within 15 minutes, the temporal similarity is set to 1; if the time deviation is between 15 and 120 minutes, the temporal similarity decreases linearly to 0 as the time deviation increases. Similarly, spatial similarity is calculated based on the distance between the occurrence points of multiple candidate causes and candidate abnormal events. If the distance between occurrence points is within 3 km, the spatial similarity is set to 1; if the distance between occurrence points is between 3 km and 20 km, the spatial similarity decreases linearly to 0 as the distance between occurrence points increases. The temporal similarity and spatial similarity are then weighted and fused to obtain the spatiotemporal co-occurrence parameter. For each cause, the spatiotemporal co-occurrence parameter is output with a value between 0 and 1. The higher the value, the greater the possibility of coexistence with the current abnormal cause in the spatiotemporal dimension. The weight of the weighted fusion is determined according to the importance that the candidate cause attaches to time and space. For example, the spatial weight can be set to 0.6 and the temporal weight to 0.4.

[0063] Furthermore, modal data support analysis is performed on multiple candidate causes using multimodal highway traffic data to obtain modal data support parameters. Modal data support analysis involves mining raw multimodal data such as images, speeds, and roads, or their intermediate features, to find specific evidence that can directly or indirectly confirm or refute the existence of a candidate cause. The modal data support parameter represents the degree of support the current multimodal data provides for the hypothesis of that cause. Specifically, for each candidate cause, specific evidence that can directly or indirectly confirm or refute the existence of that candidate cause is searched in the multimodal highway traffic data, and the strength and reliability of this evidence are evaluated to obtain the modal data support parameter.

[0064] For example, in image data, if the cause is vehicle malfunction, the image data is checked for visual evidence such as parking or hazard lights; if the cause is slippery road surface, the road data is checked for records of water spraying or rainfall. In speed data, the waveform of sudden speed drops and the rate of congestion formation and dissipation are analyzed to determine whether they conform to typical traffic flow disturbance patterns of a certain cause. In road data, it is determined whether there are relevant reports or equipment status alarms for the time period and location of the cause. The more supporting evidence and the more reliable it is, the higher the parameter value. Assuming modal data support analysis is performed on candidate cause A, after support analysis of image data, speed data, and road data, the modal data support parameter is finally obtained as 0.7.

[0065] Specifically, the reliability of each piece of evidence is assessed, and evidence from different modalities is fused. For example, if an image shows suspected collision marks while the velocity data matches the shockwave of an accident, a strong chain of evidence is formed. Finally, the corresponding modal data support parameters are obtained for each cause.

[0066] Next, a logical rationality analysis is performed on multiple candidate causes to obtain logical rationality parameters. This analysis, based on common sense, traffic engineering principles, and causal relationships, assesses the logical rationality of a candidate cause. The logical rationality parameters are quantitative scores obtained after evaluating the logical chain. For example, traffic rules and common sense are used to determine the strength of the logical connection between each cause and the abnormal event. For instance, if slippery road surfaces easily lead to slow traffic, then the logical rationality parameter for the candidate abnormal event of slow traffic is 0.9; the connection between road equipment failure and slow traffic is relatively weak, and the rationality parameter can be set to 0.6.

[0067] Finally, by combining spatiotemporal co-occurrence parameters, modal data support parameters, and logical rationality parameters, a contribution analysis is performed to obtain the causal contribution of multiple candidate causes. The causal contribution is a comprehensive quantitative score calculated through fusion, representing the overall contribution of the cause to the anomalous event. Specifically, for each candidate cause, spatiotemporal co-occurrence parameters, modal data support parameters, and logical rationality parameters are obtained from different perspectives. Then, a weighted fusion method is used to calculate the final causal contribution. Finally, the causal contribution of each cause is obtained. The weights of the spatiotemporal co-occurrence parameter, modal data support parameter, and logical rationality parameter can be learned from historical data or set by expert experience based on the degree of influence of each of the three on the causal contribution. The sum of the weights of the three is 1. If the weights of the spatiotemporal co-occurrence parameter, modal data support parameter, and logical rationality parameter are represented as w1, w2, and w3, respectively, then the causal contribution = w1 × spatiotemporal co-occurrence parameter + w2 × modal data support parameter + w3 × logical rationality parameter.

[0068] For example, assuming the weights w1, w2, and w3 are 0.3, 0.4, and 0.3 respectively, then the causal contribution of candidate cause A is 0.52 × 0.3 + 0.7 × 0.4 + 0.55 × 0.3 = 0.601.

[0069] Step S30 in the method provided in this application embodiment further includes: Multimodal data consistency analysis is performed on the candidate analysis results to obtain modal credibility; The candidate analysis results are combined with similar historical anomalies to perform historical pattern matching and obtain the reliability of the patterns; The reasonableness of the contribution distribution of the candidate cause set corresponding to the candidate abnormal events in the candidate analysis results is evaluated to obtain the distribution credibility; The confidence level of the candidate analysis results is obtained by combining the modality confidence level, the regularity confidence level, and the distribution confidence level. The candidate analysis results with the highest credibility are selected and used as the results for detecting highway traffic anomalies.

[0070] In this embodiment of the application, a multimodal data consistency analysis is first performed on the candidate analysis results to obtain modal credibility. The multimodal data consistency analysis is to evaluate whether there are contradictions or conflicts between the candidate analysis results and all currently available, unprocessed multimodal raw data and intermediate features, as well as the degree of mutual corroboration as a whole. Modal credibility is the extent to which the analysis results are consistent with all observation data.

[0071] Specifically, the consistency analysis of event causes and modal data is performed using multimodal consistency analysis, and consistency with image data is also analyzed: if the results of image data do not contradict the accident conclusion, it indicates consistency with the analyzed event causes; otherwise, it indicates inconsistency. For speed data, if the measured speed is highly consistent with the accident conclusion, it indicates consistency between the event causes and speed data. For road data, if the road data does not contradict the accident conclusion, it indicates consistency between the event causes and road data. Finally, the modal credibility is obtained by integrating the results of the three consistency analyses.

[0072] Secondly, the candidate analysis results are combined with similar historical anomalies to perform historical pattern matching to obtain the reliability of the patterns. Similar historical anomalies are historical cases in the historical database that are similar to the anomalies described by the current candidate analysis results in terms of key characteristics such as type, location, and time period. Historical pattern matching compares the current candidate analysis results with statistical patterns extracted from similar historical events to assess whether they conform to historical experience. The reliability of the patterns is a quantitative score obtained through matching analysis, which indicates the extent to which the analysis results conform to historical norms or known patterns.

[0073] Specifically, based on the event characteristics described in the current candidate analysis results, similar historical anomalous events are retrieved from the historical database. Then, the distribution of the main causes of these historical events is analyzed, and subsequently, the main causes are matched with the historical distribution. If the main cause is the most frequent cause in the same historical context, the match is extremely high; if the most frequent cause in the same historical context does not match the main cause, and the cause frequency is low, the match is extremely low. Based on the match degree, the reliability of the pattern in the result is determined.

[0074] Next, the reasonableness assessment of the contribution distribution of the candidate cause set corresponding to the candidate abnormal events in the candidate analysis results is carried out to obtain the distribution credibility. The contribution distribution is the set of contribution values ​​of all candidate causes calculated for the current candidate abnormal event, reflecting the ranking and gap of the influence of each cause on the event as believed by the system. The reasonableness assessment is a logical and statistical review of the form of the contribution distribution to determine whether it is reasonable. The distribution credibility is a score obtained through the assessment, which reflects the reliability and quality of the contribution distribution itself.

[0075] For example, in assessing the reasonableness of contribution distribution, the distribution of contribution values ​​for each candidate cause in the set of causes corresponding to the candidate abnormal event is examined to see if it is clear and if the main cause is prominent. For instance, in the candidate event of slow-moving traffic, if a candidate cause is a traffic accident and its contribution of 0.75 is significantly higher than other causes, such as temporary construction (0.1) and slippery road surface (0.05), it indicates a concentrated distribution and high confidence in the judgment. Therefore, the distribution confidence obtained from the assessment is high, and the distribution confidence can be assigned based on the difference between the contribution of the largest candidate cause and the second largest cause, i.e., distribution confidence = 0.75 - 0.1 = 0.65. Conversely, if the contribution of the main cause "traffic accident" (0.45) is very close to the contribution of the competing cause "slippery road surface" (0.4), it indicates insufficient confidence in the judgment and a scattered distribution. Therefore, the distribution confidence obtained from the assessment is low, at 0.45 - 0.4 = 0.05. The evaluation process is achieved by calculating the difference between the contribution of the main cause and the contribution of the other causes. The greater the difference, the higher the rationality of the distribution and the higher the credibility of the corresponding distribution.

[0076] Simultaneously, the credibility of candidate analysis results is obtained by integrating modal credibility, pattern credibility, and distribution credibility. The credibility of candidate analysis results is calculated through fusion to obtain a final comprehensive score representing the overall credibility level of the candidate analysis result. Specifically, a weighted fusion of the three credibility factors can be used to calculate the final credibility.

[0077] Finally, the candidate analysis results with the highest credibility are selected as the highway traffic anomaly detection results. The selection process involves comparing the final comprehensive credibility of all candidate analysis results and selecting the candidate analysis result with the highest credibility as the highway traffic anomaly detection result.

[0078] Specifically, the above calculation steps are used to calculate all candidate causes, and the candidate analysis results with the highest credibility are selected and officially obtained as highway traffic anomaly detection results, which are then output to the monitoring center's large screen and emergency dispatch system.

[0079] For example, if after the final confidence comparison, cause A is found to have the highest confidence, then the candidate analysis result of cause A is used as the result of highway traffic anomaly detection.

[0080] In step S30 of the method provided in this application embodiment, the step of obtaining the candidate analysis result confidence level by combining the modal confidence level, the regularity confidence level, and the distribution confidence level includes: Based on similar analysis results from historical analysis, a result weighting reorganization is obtained, and the modality credibility, the pattern credibility, and the distribution credibility are weighted and calculated to obtain the credibility of the candidate analysis results.

[0081] In this embodiment, based on similar analysis results from historical analysis results, a result weighting reorganization is obtained. The modal credibility, pattern credibility, and distribution credibility are weighted and calculated to obtain the credibility of the candidate analysis result. Historical analysis results are archived records of final analysis results generated during past execution processes and verified as correct through manual review or practice. Each historical analysis result includes three sub-scores: modal credibility, pattern credibility, and distribution credibility, calculated at that time. Similar analysis results are historical records retrieved from the historical analysis result database within the feature space of the current candidate analysis result, which are similar to the current candidate analysis result in key attributes. The result weighting reorganization is a combination of weight coefficients corresponding to modal credibility, pattern credibility, and distribution credibility, respectively. The weights determine the relative importance of the three sub-scores when comprehensively calculating the final credibility.

[0082] Specifically, the result weighting for the current scenario is obtained, and three sub-scores of the current candidate analysis result are obtained: modal confidence, regularity confidence, and distribution confidence. These are then weighted and summed to obtain the final result. The final confidence = (modal confidence × modal confidence weight) + (regularity confidence × regularity confidence weight) + (distribution confidence × distribution confidence weight).

[0083] For example, if the weighting coefficients of modal confidence, regularity confidence, and distribution confidence are 0.60, 0.25, and 0.15 respectively, and assuming modal confidence = 0.95, regularity confidence = 0.90, and distribution confidence = 0.93, the weighted calculation is: candidate analysis result confidence = (0.95 × 0.60) + (0.9 × 0.25) + (0.93 × 0.15) ≈ 0.93.

[0084] In this embodiment, an analysis process of cause matching, contribution analysis, result integration, and credibility assessment is first established to structure the cause analysis process. Subsequently, contribution analysis is performed on three quantitative dimensions: spatiotemporal co-occurrence, modal data support, and logical rationality, and the results are fused to obtain the cause contribution, thereby assessing the degree of influence of candidate causes. A credibility assessment mechanism is adopted for the final analysis results, and credibility is assessed from three perspectives: consistency of multimodal data, matching degree of historical patterns, and rationality of internal contribution distribution, to ensure the reliability of the analysis results and provide a reliable analytical basis for highway anomaly detection.

[0085] The embodiments of this application, through the above specific implementation methods, achieve the following technical effects: In this embodiment, a complete data foundation is first constructed using multimodal data, including images, speed, and road data, ensuring the comprehensiveness and complementarity of anomaly detection information sources. Subsequently, anomaly identification is performed using vehicle traffic data, speed statistical features, and road segment travel time from each modal data. Through preliminary fusion of multimodal features, the accuracy of anomaly identification is improved. Then, an anomaly identification plugin is trained based on historical data to improve the generalization ability of candidate anomaly event identification, providing high-quality input for subsequent accurate attribution.

[0086] Secondly, by acquiring information across spatiotemporal ranges and combining it with the inherent spatial attributes of roads to associate typical risks, prior knowledge inference about location is achieved. Through the retrieval of similar historical anomalies, statistical patterns in historical data are utilized to identify historically frequent causes as important empirical references, enabling inductive reasoning based on historical patterns. Subsequently, by acquiring multimodal highway traffic data, the condition of road segments is diagnosed in real time, and clues from images, speed, and road data are directly used to infer causes, achieving consistent analysis based on real-time evidence and ensuring the rationality of the candidate cause set.

[0087] Finally, an analytical process was established, including causal matching, contribution analysis, result integration, and credibility assessment, thus structuring the causal analysis process. Subsequently, contribution analysis was performed on three quantitative dimensions: spatiotemporal co-occurrence, modal data support, and logical rationality, and the results were integrated to obtain the causal contribution, thereby assessing the degree of influence of candidate causes. A credibility assessment mechanism was adopted for the final analysis results, evaluating credibility from three perspectives: consistency of multimodal data, matching degree of historical patterns, and rationality of internal contribution distribution, to ensure the reliability of the analysis results and provide a reliable analytical basis for highway anomaly detection.

[0088] Example 2, as Figure 2 As shown, based on the same inventive concept as the highway traffic anomaly detection method using multimodal data fusion provided in Embodiment 1, this embodiment of the invention also provides a highway traffic anomaly detection system using multimodal data fusion, comprising: The abnormal event acquisition module 11 is used to monitor and acquire multimodal highway traffic data, and to identify traffic anomalies based on each mode of highway traffic data to acquire candidate abnormal events. The candidate cause acquisition module 12 is used to acquire candidate causes based on the spatiotemporal range of the candidate abnormal events, similar historical abnormal events, and multimodal highway traffic data. The anomaly detection and analysis module 13 is used to perform contribution analysis based on the candidate anomaly events and candidate causes, combine the multimodal highway traffic data to obtain multiple analysis results, and extract the analysis result with the highest confidence among the analysis results as the highway traffic anomaly detection result, wherein the analysis result includes the anomaly event and its main cause.

[0089] In one embodiment, the exception event acquisition module 11 includes: Monitoring and acquiring multimodal highway traffic data, wherein the multimodal highway traffic data includes image data, speed data, and road data; Traffic anomalies are identified based on traffic data from each mode of highway, and candidate anomaly events are obtained.

[0090] The step of identifying traffic anomalies based on traffic data from each mode of highway and obtaining candidate anomaly events includes: Based on the image data, vehicle information is analyzed and matched to obtain vehicle traffic data. Based on the speed data, vehicle speed statistical characteristics are analyzed and obtained, wherein the vehicle speed statistical characteristics include at least maximum speed, minimum speed, average speed and speed variation parameters; Based on the road data, road segment information is obtained through analysis, and combined with the image data, the road segment travel time is obtained. By combining the vehicle traffic data, the statistical characteristics of vehicle speed, and the travel time of the road segment, anomaly identification is performed to obtain candidate abnormal events.

[0091] In one embodiment, the exception event acquisition module 11 further includes: Acquire historical anomalies and integrate them into a sample output set; Obtain historical vehicle traffic data, historical vehicle speed statistical characteristics, and historical road segment travel time corresponding to the historical abnormal events, and integrate them into a sample input set; An anomaly detection plugin is constructed, and the anomaly detection plugin is trained using the sample input set until convergence. The vehicle traffic data, vehicle speed statistical characteristics, and road segment travel time are input into the anomaly identification plugin to obtain candidate anomaly events.

[0092] In one embodiment, the candidate cause acquisition module 12 includes: Based on the spatiotemporal range of the candidate abnormal events, the causes of the abnormalities are obtained and added to the candidate causes; Based on the similar historical anomalies of the candidate anomalies, retrieve the corresponding historical main causes and add them to the candidate causes; Based on the multimodal highway traffic data, abnormal road sections are identified and added to the candidate causes.

[0093] In one embodiment, the anomaly detection and analysis module 13 further includes: For each of the candidate abnormal events, perform candidate cause matching to obtain multiple candidate cause sets; Contribution analysis is performed on multiple candidate causes in the candidate cause set to obtain the cause contribution. Based on the contribution of the causes, candidate main causes are obtained, and the candidate abnormal events and candidate main causes are integrated into candidate analysis results. The candidate analysis results are evaluated for credibility to obtain the credibility of the candidate analysis results, and the candidate analysis results with the highest credibility are selected as the highway traffic anomaly detection results.

[0094] The contribution analysis of multiple candidate causes in the candidate cause set is performed to obtain the cause contribution, including: Spatiotemporal co-occurrence analysis was performed on multiple candidate causes to obtain spatiotemporal co-occurrence parameters; Modal data support analysis is performed on multiple candidate causes in conjunction with the multimodal highway traffic data to obtain modal data support parameters; Logical rationality analysis is performed on multiple candidate causes to obtain logical rationality parameters; By combining the spatiotemporal co-occurrence parameter, the modal data support parameter, and the logical rationality parameter, a contribution analysis is performed to obtain the causal contribution of multiple candidate causes.

[0095] In one embodiment, the anomaly detection and analysis module 13 further includes: Multimodal data consistency analysis is performed on the candidate analysis results to obtain modal credibility; The candidate analysis results are combined with similar historical anomalies to perform historical pattern matching and obtain the reliability of the patterns; The reasonableness of the contribution distribution of the candidate cause set corresponding to the candidate abnormal events in the candidate analysis results is evaluated to obtain the distribution credibility; The confidence level of the candidate analysis results is obtained by combining the modality confidence level, the regularity confidence level, and the distribution confidence level. The candidate analysis results with the highest credibility are selected and used as the results for detecting highway traffic anomalies.

[0096] The step of obtaining the credibility of candidate analysis results by combining the modality credibility, the regularity credibility, and the distribution credibility includes: Based on similar analysis results from historical analysis, a result weighting reorganization is obtained, and the modality credibility, the pattern credibility, and the distribution credibility are weighted and calculated to obtain the credibility of the candidate analysis results.

[0097] The embodiments of this application, through the above specific implementation methods, achieve the following technical effects: In this embodiment, the abnormal event acquisition module 11 first constructs a complete data foundation by using multimodal data, including images, speed, and road data, to ensure the comprehensiveness and complementarity of the sources of abnormal detection information. Subsequently, vehicle traffic data, speed statistical features, and road segment travel time from each modal data are used for abnormal identification. Through the preliminary fusion of multimodal features, the accuracy and robustness of abnormal identification are improved. Then, the abnormal identification plugin is trained based on historical data to improve the intelligence level and generalization ability of candidate abnormal event identification, providing high-quality input for subsequent accurate attribution.

[0098] Secondly, through the candidate cause acquisition module 12, based on the spatiotemporal range and combined with the inherent spatial attributes of the road to associate typical risks, prior knowledge reasoning about location is realized. By retrieving similar historical anomalies, statistical patterns in historical data are used to take historical high-incidence causes as important empirical references, realizing inductive reasoning based on historical patterns. Subsequently, by acquiring multimodal highway traffic data, the condition of the road segment itself is diagnosed in real time, and the causes are directly inferred using clues in image, speed, and road data, realizing consistency analysis based on real-time evidence and ensuring the rationality of the candidate cause set.

[0099] Finally, through the anomaly detection and analysis module 13, an analysis process of cause matching, contribution analysis, result integration, and credibility assessment is established, making the cause analysis process structured. Subsequently, contribution analysis is performed on three quantitative dimensions: spatiotemporal co-occurrence, modal data support, and logical rationality, and the results are integrated to obtain the cause contribution, thereby assessing the degree of influence of candidate causes. A credibility assessment mechanism is adopted for the final analysis results, and credibility is assessed from three perspectives: consistency of multimodal data, matching degree of historical patterns, and rationality of internal contribution distribution, to ensure the reliability of the analysis results and provide a reliable analytical basis for highway anomaly detection.

Claims

1. A method for detecting highway traffic anomalies based on multimodal data fusion, characterized in that, include: The system monitors and acquires multimodal highway traffic data, and identifies traffic anomalies based on the traffic data of each mode to obtain candidate anomaly events. Based on the spatiotemporal range of the candidate abnormal events, similar historical abnormal events, and multimodal highway traffic data, candidate causes are obtained; Based on the candidate abnormal events and candidate causes, a contribution analysis is performed. Combined with the multimodal highway traffic data, multiple analysis results are obtained. The analysis result with the highest confidence among the analysis results is extracted as the highway traffic anomaly detection result. The analysis result includes the abnormal event and its main causes.

2. The method for detecting highway traffic anomalies based on multimodal data fusion according to claim 1, characterized in that, Multimodal highway traffic data is monitored and acquired. Based on the traffic data of each modality, traffic anomalies are identified to obtain candidate anomaly events, including: Monitoring and acquiring multimodal highway traffic data, wherein the multimodal highway traffic data includes image data, speed data, and road data; Traffic anomalies are identified based on traffic data from each mode of highway, and candidate anomaly events are obtained.

3. The method for detecting highway traffic anomalies based on multimodal data fusion according to claim 2, characterized in that, Traffic anomaly identification is performed based on traffic data from each mode of highway, and candidate anomaly events are obtained, including: Based on the image data, vehicle information is analyzed and matched to obtain vehicle traffic data; Based on the speed data, vehicle speed statistical characteristics are analyzed and obtained, wherein the vehicle speed statistical characteristics include at least the maximum speed, minimum speed, average speed, and speed variation parameters; Based on the road data, road segment information is obtained through analysis, and combined with the image data, the road segment travel time is obtained. By combining the vehicle traffic data, the statistical characteristics of vehicle speed, and the travel time of the road segment, anomaly identification is performed to obtain candidate abnormal events.

4. The method for detecting highway traffic anomalies based on multimodal data fusion according to claim 3, characterized in that, Combining the vehicle traffic data, the statistical characteristics of vehicle speed, and the travel time of the road segment, anomaly identification is performed to obtain candidate abnormal events, including: Acquire historical anomalies and integrate them into a sample output set; Obtain historical vehicle traffic data, historical vehicle speed statistical characteristics, and historical road segment travel time corresponding to the historical abnormal events, and integrate them into a sample input set; An anomaly detection plugin is constructed, and the anomaly detection plugin is trained using the sample input set until convergence. The vehicle traffic data, vehicle speed statistical characteristics, and road segment travel time are input into the anomaly identification plugin to obtain candidate anomaly events.

5. The method for detecting highway traffic anomalies based on multimodal data fusion according to claim 1, characterized in that, Based on the spatiotemporal range of the candidate abnormal events, similar historical abnormal events, and multimodal highway traffic data, candidate causes are obtained, including: Based on the spatiotemporal range of the candidate abnormal events, the causes of the abnormalities are obtained and added to the candidate causes; Based on the similar historical anomalies of the candidate anomalies, retrieve the corresponding historical main causes and add them to the candidate causes; Based on the multimodal highway traffic data, abnormal road sections are identified and added to the candidate causes.

6. The method for detecting highway traffic anomalies based on multimodal data fusion according to claim 1, characterized in that, Based on the candidate anomaly events and their potential causes, a contribution analysis is performed. Combined with the multimodal highway traffic data, multiple analysis results are obtained. The analysis result with the highest confidence among these results is extracted as the highway traffic anomaly detection result, including: For each of the candidate abnormal events, perform candidate cause matching to obtain multiple candidate cause sets; Contribution analysis is performed on multiple candidate causes in the candidate cause set to obtain the cause contribution. Based on the contribution of the causes, candidate main causes are obtained, and the candidate abnormal events and candidate main causes are integrated into candidate analysis results. The candidate analysis results are evaluated for credibility to obtain the credibility of the candidate analysis results, and the candidate analysis results with the highest credibility are selected as the highway traffic anomaly detection results.

7. The method for detecting highway traffic anomalies based on multimodal data fusion according to claim 6, characterized in that, Contribution analysis is performed on multiple candidate causes in the candidate cause set to obtain the causal contribution, including: Spatiotemporal co-occurrence analysis was performed on multiple candidate causes to obtain spatiotemporal co-occurrence parameters; Modal data support analysis is performed on multiple candidate causes in conjunction with the multimodal highway traffic data to obtain modal data support parameters; Logical rationality analysis is performed on multiple candidate causes to obtain logical rationality parameters; By combining the spatiotemporal co-occurrence parameter, the modal data support parameter, and the logical rationality parameter, a contribution analysis is performed to obtain the causal contribution of multiple candidate causes.

8. The method for detecting highway traffic anomalies based on multimodal data fusion according to claim 6, characterized in that, The candidate analysis results are evaluated for credibility to obtain their credibility level. The candidate analysis results with the highest credibility are then selected as the highway traffic anomaly detection results, including: Multimodal data consistency analysis is performed on the candidate analysis results to obtain modal credibility; The candidate analysis results are combined with similar historical anomalies to perform historical pattern matching and obtain the reliability of the patterns; The reasonableness of the contribution distribution of the candidate cause set corresponding to the candidate abnormal events in the candidate analysis results is evaluated to obtain the distribution credibility; The confidence level of the candidate analysis results is obtained by combining the modality confidence level, the regularity confidence level, and the distribution confidence level. The candidate analysis results with the highest credibility are selected and used as the results for detecting highway traffic anomalies.

9. The method for detecting highway traffic anomalies based on multimodal data fusion according to claim 8, characterized in that, The confidence level of candidate analysis results is obtained by combining the modal confidence level, the regularity confidence level, and the distribution confidence level, including: Based on similar analysis results from historical analysis, a result weighting reorganization is obtained, and the modality credibility, the pattern credibility, and the distribution credibility are weighted and calculated to obtain the credibility of the candidate analysis results.

10. A highway traffic anomaly detection system based on multimodal data fusion, characterized in that, The system is used to implement the multimodal data fusion method for detecting highway traffic anomalies according to any one of claims 1-9, the system comprising: The abnormal event acquisition module is used to monitor and acquire multimodal highway traffic data, and to identify traffic anomalies based on the traffic data of each mode to acquire candidate abnormal events; The candidate cause acquisition module is used to acquire candidate causes based on the spatiotemporal range of the candidate abnormal events, similar historical abnormal events, and multimodal highway traffic data. The anomaly detection and analysis module is used to perform contribution analysis based on the candidate anomaly events and candidate causes, combine the multimodal highway traffic data to obtain multiple analysis results, and extract the analysis result with the highest confidence among the analysis results as the highway traffic anomaly detection result, wherein the analysis result includes the anomaly event and its main cause.

Images