Traffic monitoring method, device and medium based on multimodal and hypergraph structure

Abstract

The present application discloses a traffic monitoring method, device and medium based on multimodality and hypergraph structure, which relates to the field of smart transportation technology. The method includes: obtaining multimodal spatiotemporal sequence data through roadside monitoring points, collecting real-time visibility and precipitation intensity data through meteorological sensors; generating hypergraph nodes according to the spatial topological relationship of the monitoring points, and generating hyperedges according to the similarity of vehicle trajectories and the spatiotemporal coupling of V2X communication data to obtain a hypergraph structure; performing multi-scale spectral decomposition on the hypergraph structure, using a learnable filter to suppress high-frequency noise, and reconstructing spatial dependency features; generating anomaly thresholds based on historical data, comparing spatial dependency features with anomaly thresholds, and predicting anomaly diffusion paths through a hypergraph structure. The present application solves the problem of multi-scale spectral decomposition and learnable filtering to solve the sparsity and noise coupling problems of abnormal samples through the above method, uses a hypergraph structure to characterize complex associations, and combines multimodal verification to improve prediction accuracy.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a traffic monitoring method, device, and medium based on multimodality and hypergraph structure. Background Art

[0002] Current urban traffic anomaly detection relies primarily on fixed sensor networks and static graph models, making it difficult to capture the dynamic, high-order correlations between roads. While traditional spatiotemporal graph convolution methods can model binary connections between nodes, they are unable to characterize the complex patterns of coordinated changes across multiple road segments. This is especially true in scenarios where traffic flow changes are sudden due to abnormal events, significantly increasing the prediction error of existing technologies for congestion propagation paths. Furthermore, existing data augmentation strategies are often limited to single-dimensional perturbations in time or space, failing to effectively mitigate the coupled interference between the sparsity of abnormal samples in traffic data and environmental noise, leading to a sharp decline in the model's detection performance under complex meteorological conditions such as rain and fog.

[0003] In recent years, hypergraph-based traffic modeling technologies have partially addressed the problem of representing high-order relationships. However, their hyperedge weights often rely on manual settings or static road topology, lacking the ability to adapt to real-time traffic patterns. While the dynamic hypergraph method proposed in patent CN117648652A incorporates a time series similarity metric, it still suffers from three limitations: first, it fails to consider multimodal data; second, it fails to account for the sharp decline in detection performance under complex meteorological conditions such as rain and fog; and third, it fails to incorporate multiscale spectral decomposition to extract low-frequency basis functions that reflect steady-state traffic flow and high-frequency components that are sensitive to anomalies.

[0004] Through the above analysis, the problems and defects of the existing technology are as follows:

[0005] The traffic monitoring based on hypergraph structure in the existing technology does not consider multimodal data, meteorological data, multi-scale spectral decomposition, the coupling interference of abnormal sample sparsity and environmental noise. Summary of the Invention

[0006] The embodiments of the present application provide a traffic monitoring method, device and medium based on multimodality and hypergraph structure, which can solve the problems in the existing technology that traffic monitoring with hypergraph structure does not take into account multimodal data and meteorological data, as well as multi-scale spectral decomposition, sparsity of abnormal samples and coupling interference with environmental noise.

[0007] In a first aspect, an embodiment of the present application provides a traffic monitoring method based on multimodality and hypergraph structure, the method comprising: obtaining multimodal spatiotemporal sequence data through roadside monitoring points, collecting real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points including camera arrays and microwave radars, and the multimodal data including vehicle trajectories and V2X communication data; generating hypergraph nodes according to the spatial topological relationship of the monitoring points, and generating hyperedges according to the similarity of vehicle trajectories and the spatiotemporal coupling of V2X communication data to obtain a hypergraph structure; performing multi-scale spectral decomposition on the hypergraph structure, using a learnable filter to suppress high-frequency noise, and reconstructing spatial dependency features; generating anomaly thresholds based on historical data, comparing the spatial dependency features with the anomaly thresholds, and predicting the anomaly diffusion path through the hypergraph structure.

[0008] In one implementation of the present application, hypergraph nodes are generated based on the spatial topological relationship of the monitoring points, and hyperedges are generated based on the similarity of vehicle trajectories and the spatiotemporal coupling of V2X communication data to obtain a hypergraph structure, which specifically includes: calculating the shortest path distance between monitoring points and constructing an adjacency matrix; based on the adjacency matrix, a dynamic time warping algorithm is used to calculate the similarity of vehicle trajectories, and trajectories with a similarity higher than a preset threshold are constructed as hyperedges.

[0009] In one implementation of the present application, the method also includes: when a vehicle sudden braking event is detected in the V2X communication data, the hyperedge weight is enhanced to the associated road section according to the adjacency matrix with the location of the event as the center; based on the visibility data collected by the meteorological sensor, the hyperedge of the corresponding camera array and microwave radar is attenuated and compensated; a generative adversarial network is constructed, the noise distribution of the meteorological sensor under extreme weather is simulated by the generator, and the robust representation of the hyperedge weight is learned by the discriminator.

[0010] In one implementation of the present application, a multi-scale spectral decomposition is performed on the hypergraph structure, a learnable filter is used to suppress high-frequency noise, and spatial dependency features are reconstructed, specifically including: normalizing and multi-scale decomposing the hypergraph structure to obtain eigenvectors and corresponding eigenvalues, and extracting high-frequency components and low-frequency components based on the eigenvalues; suppressing high-frequency components through a learnable filter, where the high-frequency components include congestion below a preset time; performing correlation matching on the eigenvectors with historical data, and sorting the eigenvectors for phase correction; fusing the phase-corrected eigenvectors through an attention mechanism, and reconstructing the spatial dependency features using an inverse spectral domain transform.

[0011] In one implementation of the present application, the phase-corrected feature vectors are fused through the attention mechanism, and the spatial dependency features are reconstructed using the inverse spectral domain transform, specifically including: using a multi-head graph attention mechanism to calculate the association weights between nodes respectively, and capturing the spatial dependency relationships of local congestion propagation, road section flow and speed respectively; constructing a stacked dilated convolution layer, and the dilation factor of the dilated convolution layer increases exponentially to simultaneously capture congestion shorter than a preset time and congestion longer than a preset time; projecting the association weights and the feature vectors processed by the dilated convolution layer into the spectral domain, and obtaining the spatial dependency features through the inverse spectral domain transform.

[0012] In one implementation of the present application, an anomaly threshold is generated based on historical data, the spatial dependency characteristics are compared with the anomaly threshold, and the anomaly diffusion path is predicted through a hypergraph structure, specifically including: grouping historical data according to time granularity, extracting the flow and speed of each group of historical data, and using kernel density estimation to generate a baseline probability distribution space; projecting the current phase-corrected feature vector to the baseline probability distribution space, and calculating the Mahalanobis distance as the anomaly threshold; when it is detected that the anomaly threshold for multiple consecutive cycles exceeds the warning value, reducing the subsequent detection sensitivity threshold.

[0013] In one implementation of the present application, the anomaly diffusion path is predicted through a topological propagation model of a hypergraph structure, specifically including: based on a real-time updated adjacency matrix, simulating the propagation of anomalies from an initial node to adjacent nodes according to spatial dependency characteristics to obtain a predicted diffusion path; performing multimodal cross-validation on the diffusion path in combination with the geographic location of V2X communication data; when a vehicle sudden braking event in the V2X communication data overlaps with the predicted diffusion path, improving the warning confidence level of the diffusion path; and updating the detour diffusion path to the navigation platform in real time.

[0014] In one implementation of the present application, the method also includes: building a two-way data channel between the digital twin and the real road network in the cloud, injecting historical abnormal patterns for stress testing; performing activity analysis on the hypergraph nodes, and shutting down meteorological sensors whose activity is below the threshold.

[0015] In a second aspect, an embodiment of the present application further provides a traffic monitoring device based on multimodality and hypergraph structure, the device comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so as to enable the at least one processor to: obtain multimodal spatiotemporal series data through roadside monitoring points, and collect real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points comprising camera arrays and microwave radars, and the multimodal data comprising vehicle trajectories and V2X communication data; generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of the vehicle trajectories and the spatiotemporal coupling of the V2X communication data to obtain a hypergraph structure; perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependency features; generate anomaly thresholds based on historical data, compare the spatial dependency features with the anomaly thresholds, and predict the anomaly diffusion path through the hypergraph structure.

[0016] In a third aspect, an embodiment of the present application further provides a non-volatile computer storage medium for traffic monitoring based on multimodality and hypergraph structure, which stores computer executable instructions, and the computer executable instructions are set to: obtain multimodal spatiotemporal series data through roadside monitoring points, and collect real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points include camera arrays and microwave radars, and the multimodal data include vehicle trajectories and V2X communication data; generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of vehicle trajectories and the spatiotemporal coupling of V2X communication data to obtain a hypergraph structure; perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependency features; generate anomaly thresholds based on historical data, compare the spatial dependency features with the anomaly thresholds, and predict the anomaly diffusion path through the hypergraph structure.

[0017] The embodiments of the present application provide a traffic monitoring method, device, and medium based on multimodality and hypergraph structure. The method obtains multimodal spatiotemporal sequences and meteorological data including vehicle trajectories and V2X communication data through roadside monitoring points, generates a hypergraph structure based on the spatial topology and data association of the monitoring points, reconstructs spatial dependency features after noise suppression through multi-scale spectral decomposition and learnable filtering, generates dynamic anomaly thresholds based on historical data, and predicts diffusion paths through the hypergraph. The method integrates multimodality and meteorological data to improve environmental adaptability, solves the problem of sparse anomaly samples and noise coupling through multi-scale spectral decomposition and learnable filtering, uses a hypergraph structure to characterize complex associations and combines them with multimodal verification to improve prediction accuracy, and enhances system robustness through dynamic optimization of thresholds based on historical data and digital twin stress testing. This method systematically solves the defects of the prior art that do not consider multi-source data, multi-scale analysis, and complex interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A flow chart of a traffic monitoring method based on multimodality and hypergraph structure provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the internal structure of a traffic monitoring device based on multimodality and hypergraph structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] The embodiments of the present application provide a traffic monitoring method, device and medium based on multimodality and hypergraph structure, which solves the problem that traffic monitoring with hypergraph structure in the prior art does not take into account multimodal data and meteorological data, as well as multi-scale spectral decomposition, the sparsity of abnormal samples and the coupling interference of environmental noise.

[0023] The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 The flow chart of a traffic monitoring method based on multimodality and hypergraph structure provided in the embodiment of the present application is as follows. Figure 1 As shown, the embodiment of the present application provides a traffic monitoring method based on multimodality and hypergraph structure, which specifically includes the following steps:

[0025] Step 10: Obtain multimodal spatiotemporal sequence data through roadside monitoring points. Real-time visibility and precipitation intensity data are collected through meteorological sensors. Monitoring points include camera arrays and microwave radars. Multimodal data includes vehicle trajectories and V2X communication data.

[0026] In this step, radar and cameras use a time synchronization protocol to ensure data alignment. The spatiotemporal coordinate sequence of each vehicle reflects both individual movement patterns and group interactions. V2X communication data includes vehicle-to-infrastructure and vehicle-to-vehicle communication information, such as emergency braking signals and path planning intentions.

[0027] Step 20: Generate hypergraph nodes based on the spatial topological relationship of the monitoring points, and generate hyperedges based on the similarity of vehicle trajectories and the spatiotemporal coupling of V2X communication data to obtain a hypergraph structure.

[0028] As an optional embodiment, hypergraph nodes are generated according to the spatial topological relationship of the monitoring points, and hyperedges are generated according to the similarity of the vehicle trajectories and the spatiotemporal coupling of the V2X communication data to obtain a hypergraph structure. Specifically, the hypergraph structure may include: step 201: calculating the shortest path distance between the monitoring points and constructing an adjacency matrix; step 202: based on the adjacency matrix, using a dynamic time warping algorithm to calculate the similarity of the vehicle trajectories, and constructing trajectories with a similarity higher than a preset threshold as hyperedges.

[0029] In this step, camera and radar locations are treated as hypergraph nodes. Spatial topology is calculated from geographic locations. Physical accessibility between nodes is characterized by shortest path distance. The adjacency matrix A∈Rn×n, with elements Aij representing the connection weights between nodes vi and vj, reflects the physical structure of the road network and serves as the basis for subsequent hyperedge generation. A similarity threshold is set. If it exceeds the threshold, the monitoring points {vi, vj, …} that the trajectory passes through are connected into hyperedges in the adjacency matrix, indicating a strong correlation between these nodes at the trajectory level.

[0030] A communication event indicates that nodes vi and vj communicate at a certain moment. The coupling degree is defined as the normalized value of the number of communications per unit time. If the normalized value exceeds a threshold, the nodes involved in the communication, {vi, vj, …}, are connected into a hyperedge, reflecting the real-time communication relationship between the nodes. Each hyperedge is assigned a weight, such as a trajectory similarity value or a communication coupling value, to reflect the strength of the relationship.

[0031] As an optional embodiment, the method may also include: step 203: when a vehicle sudden braking event is detected in the V2X communication data, the hyperedge weight is enhanced to the associated road section according to the adjacency matrix with the location of the event as the center; step 204: based on the visibility data collected by the meteorological sensor, the hyperedge of the corresponding camera array and microwave radar is attenuated and compensated; step 205: constructing a generative adversarial network, simulating the noise distribution of meteorological sensors under extreme weather conditions through the generator, and learning the robust representation of the hyperedge weight through the discriminator.

[0032] Step 30: Perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependency features;

[0033] As an optional embodiment, multi-scale spectral decomposition is performed on the hypergraph structure, a learnable filter is used to suppress high-frequency noise, and spatial dependency features are reconstructed. Specifically, the process may include: step 301: normalizing and multi-scale decomposing the hypergraph structure to obtain eigenvectors and corresponding eigenvalues, and extracting high-frequency components and low-frequency components based on the eigenvalues;

[0034] The characteristic decomposition is L=UΛU T , where Λ=diag(λ1,λ2,…,λn) is the eigenvalue matrix and U is the eigenvector matrix. Arranging the eigenvalues in ascending order λ1≤λ2≤…≤λn corresponds to:

[0035] Low-frequency components: small eigenvalues: represent the global structure and long-term dependencies of the hypergraph, such as the propagation of congestion on main roads.

[0036] High-frequency components: Large eigenvalues: Indicates local fluctuations and short-term noise, such as occasional sudden braking and sensor errors.

[0037] Step 302: Suppressing high-frequency components using a learnable filter, where the high-frequency components include congestion below a preset time;

[0038] In this step, the filter can be learned to adaptively suppress high-frequency noise.

[0039] Step 303: performing correlation matching on the feature vectors and historical data, and sorting the feature vectors to perform phase correction;

[0040] In this step, the cosine similarity between the current feature vector and the history is calculated, sorted by similarity, and the most similar preset number of historical features are selected for phase correction. The feature vector is sorted and rotated to align the phase with the historical pattern.

[0041] Step 304: The phase-corrected feature vectors are fused through the attention mechanism, and the spatial dependency features are reconstructed using the inverse spectral domain transform.

[0042] In this step, the filtered frequency domain features are converted back to the spatial domain. This process restores the spatial dependencies between nodes. The reconstructed features retain and enhance low-frequency components, suppress high-frequency noise and random sudden braking, and improve the signal-to-noise ratio. This filter filters out short-term fluctuations such as occasional sudden braking, focusing on the propagation paths of persistent congestion on main roads and distinguishing between periodic morning rush hour traffic and sudden abnormal accidents.

[0043] As an optional embodiment, the phase-corrected feature vectors are fused through the attention mechanism, and the spatial dependency features are reconstructed using the inverse spectral domain transform. Specifically, the following may be included: Step 3041: The association weights between nodes are calculated respectively using the multi-head graph attention mechanism to capture the spatial dependency relationships among local congestion propagation, road traffic flow, and speed; Step 3042: A stacked dilated convolution layer is constructed, and the dilation factor of the dilated convolution layer increases exponentially to simultaneously capture congestion shorter than a preset time and congestion longer than a preset time; Step 3043: The association weights and the feature vectors processed by the dilated convolution layer are projected into the spectral domain, and the spatial dependency features are obtained through the inverse spectral domain transform.

[0044] In this step, for the node vi and vj , calculate the attention score, and use the attention weights of neighboring nodes to obtain how congestion spreads from one monitoring point to adjacent points. Different heads can focus on traffic changes, such as the spatial dependence of traffic density and speed fluctuations.

[0045] The receptive field of standard convolution is k. The dilated convolution expands the receptive field to k+(k-1)(d-1) by introducing the dilation factor d. For the input feature map x, the dilated convolution output is:

[0046]

[0047] Where r is the dilation factor, K is the convolution kernel size, and w[k] is the fixed filter coefficient.

[0048] Design stacked convolutional layers with an expansion factor of r=2n (n is the number of layers), with r=1 in the first layer: capturing short time scales, such as congestion changes within 30 seconds;

[0049] The second layer r=2: captures medium time scales, such as congestion propagation within 2 minutes;

[0050] The third layer, with r=4, captures long timescales, such as congestion evolution over 10 minutes or longer. By combining different dilation factors, the model can handle both short-term and long-term congestion. Multi-head attention, through different head parameterizations, captures spatial dependencies in different dimensions, such as flow rate and speed. Dilated convolution, through exponentially growing receptive fields, adaptively covers timescales from seconds to hours.

[0051] Step 40: Generate anomaly thresholds based on historical data, compare the spatial dependency features with the anomaly thresholds, and predict anomaly diffusion paths through the hypergraph structure.

[0052] As an optional embodiment, an anomaly threshold is generated based on historical data, the spatial dependency characteristics are compared with the anomaly threshold, and the anomaly diffusion path is predicted through a hypergraph structure. Specifically, the following may be included: Step 401: grouping historical data according to time granularity, extracting the flow and speed of each group of historical data, and using kernel density estimation to generate a reference probability distribution space; Step 402: projecting the current phase-corrected feature vector to the reference probability distribution space, and calculating the Mahalanobis distance as the anomaly threshold; Step 403: when it is detected that the anomaly threshold for multiple consecutive cycles exceeds the warning value, reducing the subsequent detection sensitivity threshold.

[0053] In this step, the historical data is sliced by time: hour, weekday / weekend, seasonal grouping, and including abnormal data during historical rainstorms. A two-dimensional probability distribution is constructed for the flow rate f and speed v of each set of data:

[0054]

[0055] Here, K(⋅) is the kernel function, x and y represent two-dimensional feature variables in the historical data, specifically corresponding to flow rate f and velocity v. h is the bandwidth parameter, which controls the smoothness of the distribution. This distribution reflects the joint probability density of flow rate and velocity under normal operating conditions. The current feature vector is projected onto the reference distribution space, and the Mahalanobis distance between it and the distribution center is calculated. The Mahalanobis distance accounts for inter-feature correlation, and since flow rate and velocity are negatively correlated, it is more reliable than the Euclidean distance. When continuous anomalies occur, the detection threshold is lowered to increase sensitivity and accelerate the detection of diffusion paths. When there are no anomalies, the default threshold is gradually restored to reduce false positives.

[0056] As an optional embodiment, predicting the anomaly diffusion path through a topological propagation model of a hypergraph structure may specifically include: step 404: based on a real-time updated adjacency matrix, simulating the propagation of anomalies from an initial node to adjacent nodes according to spatial dependency characteristics to obtain a predicted diffusion path; step 405: performing multimodal cross-validation on the diffusion path in combination with the geographic location of the V2X communication data; step 406: when a vehicle sudden braking event in the V2X communication data overlaps with the predicted diffusion path, improving the warning confidence level of the diffusion path; step 407: updating the detour diffusion path to the navigation platform in real time.

[0057] In this step, based on the hypergraph adjacency matrix A and hyperedge weight w e , a graph diffusion model is used to simulate anomaly propagation: s(t+1)=A⋅s(t), where s(t) is the node anomaly intensity vector. The initial value is si=1 for the node where the anomaly is detected, and 0 for all other nodes. If the predicted diffusion path contains the node sequence [v1, v2, v3], check whether any sudden braking events occur continuously on the spatial path v1→v2→v3 in the V2X data during the corresponding time period.

[0058] As an optional embodiment, the method may also include: building a two-way data channel between the digital twin and the real road network in the cloud, injecting historical abnormal patterns for stress testing; performing activity analysis on hypergraph nodes, and shutting down meteorological sensors whose activity is below a threshold.

[0059] In this step, real-time monitoring data, including camera video streams and radar point clouds, is transmitted via 5G and the Internet of Things to update the digital twin's status. Optimization strategies generated by stress testing, such as signal timing plans and emergency lane activation, are fed back to roadside equipment for execution. Furthermore, historical abnormal events, such as congestion on arterial roads caused by heavy rain, are replayed in the digital twin to verify the effectiveness of the current monitoring algorithm. Extreme scenarios are artificially created, such as not delivering updated detour routes, to assess the system's crash threshold.

[0060] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, this application embodiment also provides a traffic monitoring device based on multimodality and hypergraph structure, the structure of which is as follows: Figure 2 shown.

[0061] Figure 2 The internal structure diagram of a traffic monitoring device based on multimodality and hypergraph structure provided in the embodiment of the present application is as follows. Figure 2 As shown, the equipment includes:

[0062] at least one processor 201;

[0063] and, a memory 202 communicatively coupled to the at least one processor;

[0064] Among them, the memory 202 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 201 to enable the at least one processor 201 to: obtain multimodal spatiotemporal series data through roadside monitoring points, and collect real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points include camera arrays and microwave radars, and the multimodal data include vehicle trajectories and V2X communication data; generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of vehicle trajectories and the spatiotemporal coupling of V2X communication data to obtain a hypergraph structure; perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependency features; generate anomaly thresholds based on historical data, compare the spatial dependency features with the anomaly thresholds, and predict the anomaly diffusion path through the hypergraph structure.

[0065] Some embodiments of the present application provide corresponding Figure 1 A non-volatile computer storage medium for traffic monitoring based on multimodal and hypergraph structures stores computer-executable instructions, wherein the computer-executable instructions are configured to: obtain multimodal spatiotemporal sequence data through roadside monitoring points, and collect real-time visibility and precipitation intensity data through meteorological sensors. The monitoring points include camera arrays and microwave radars, and the multimodal data include vehicle trajectories and vehicle-to-everything (V2X) communication data; generate hypergraph nodes based on the spatial topological relationships of the monitoring points, and generate hyperedges based on the similarity of vehicle trajectories and the spatiotemporal coupling of V2X communication data to obtain a hypergraph structure; perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependency features; generate anomaly thresholds based on historical data, compare the spatial dependency features with the anomaly thresholds, and predict anomaly diffusion paths through the hypergraph structure.

[0066] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from the other embodiments. In particular, the IoT device and media embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.

[0067] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0068] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0070] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0072] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0073] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0074] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0075] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0076] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A traffic monitoring method based on multimodality and hypergraph structure, characterized in that: The method comprises: Acquire multimodal spatiotemporal sequence data through roadside monitoring points, including camera arrays and microwave radars, and collect real-time visibility and precipitation intensity data through meteorological sensors. The multimodal data includes vehicle trajectories and V2X communication data. Generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of the vehicle trajectories and the spatiotemporal coupling of the V2X communication data to obtain a hypergraph structure; Performing multi-scale spectral decomposition on the hypergraph structure, using a learnable filter to suppress high-frequency noise, and performing phase correction on the hypergraph structure to reconstruct spatial dependency features; Generate an anomaly threshold based on historical data, compare the spatial dependency feature with the anomaly threshold, and predict the anomaly diffusion path through the hypergraph structure, specifically including: Grouping the historical data by time granularity, extracting the flow rate and speed of each group of historical data, and generating a benchmark probability distribution space using kernel density estimation; Projecting the current phase-corrected hypergraph structure into the reference probability distribution space, and calculating the Mahalanobis distance as an anomaly threshold; When it is detected that the abnormal threshold value exceeds the warning value for multiple consecutive cycles, the subsequent detection sensitivity threshold is lowered; Predicting the abnormal diffusion path through the hypergraph structure specifically includes: Based on the real-time updated adjacency matrix, the propagation of anomalies from the initial node to the adjacent nodes is simulated according to the spatial dependency characteristics to obtain a predicted diffusion path; performing multimodal cross-validation on the diffusion path in combination with the geographic location of the V2X communication data; When the vehicle sudden braking event in the V2X communication data overlaps with the predicted diffusion path, increasing the warning confidence level of the diffusion path; The diffusion path is updated to the navigation platform in real time.

2. The traffic monitoring method based on multimodality and hypergraph structure according to claim 1, characterized in that: Generate hypergraph nodes based on the spatial topological relationship of the monitoring points, and generate hyperedges based on the similarity of the vehicle trajectories and the spatiotemporal coupling of the V2X communication data to obtain a hypergraph structure, specifically including: Calculating the shortest path distance between the monitoring points and constructing an adjacency matrix; Based on the adjacency matrix, a dynamic time warping algorithm is used to calculate the similarity of the vehicle trajectories, and the trajectories with similarity higher than a preset threshold are constructed as hyperedges.

3. The traffic monitoring method based on multimodality and hypergraph structure according to claim 2, characterized in that: The method further comprises: When a sudden braking event of a vehicle is detected in the V2X communication data, the hyperedge weight is enhanced toward the associated road segment based on the adjacency matrix with the location of the event as the center; Performing attenuation compensation on the corresponding hyperedges of the camera array and microwave radar according to the visibility data collected by the meteorological sensor; A generative adversarial network is constructed, in which the generator simulates the noise distribution of the meteorological sensor under extreme weather conditions, and the discriminator learns the robust representation of the hyperedge weights.

4. The traffic monitoring method based on multimodality and hypergraph structure according to claim 1, characterized in that: Perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependency features, specifically including: Normalizing and multi-scale decomposing the hypergraph structure to obtain eigenvectors and corresponding eigenvalues, and extracting high-frequency components and low-frequency components according to the eigenvalues; suppressing the high frequency components by a learnable filter, the high frequency components including congestion below a preset time; performing correlation matching on the feature vector and historical data, and sorting the feature vector to perform phase correction; The phase-corrected feature vectors are fused through an attention mechanism, and the spatial dependency features are reconstructed using an inverse spectral domain transform.

5. The traffic monitoring method based on multimodality and hypergraph structure according to claim 4 is characterized in that: The phase-corrected feature vectors are fused through the attention mechanism, and the spatial dependency features are reconstructed using the inverse spectral domain transform, specifically including: A multi-head graph attention mechanism is used to calculate the association weights between nodes, capturing the spatial dependencies of local congestion propagation, road flow, and speed. Constructing a stacked dilated convolutional layer, wherein the dilation factor of the dilated convolutional layer increases exponentially to simultaneously capture congestion shorter than a preset time and congestion longer than a preset time; The associated weights and the feature vector processed by the dilated convolution layer are projected into the spectral domain, and the spatial dependency features are obtained by inverse spectral domain transformation.

6. The traffic monitoring method based on multimodality and hypergraph structure according to claim 1, characterized in that: The method further comprises: Build a two-way data channel between the digital twin and the real road network in the cloud, inject historical abnormal patterns for stress testing; An activity analysis is performed on the hypergraph nodes, and meteorological sensors whose activity is lower than a threshold are turned off.

7. A traffic monitoring device based on multimodality and hypergraph structure, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least a processor to execute so that the at least one processor is capable of: Execute the steps of a traffic monitoring method based on multimodality and hypergraph structure as described in any one of claims 1-6.

8. A non-volatile computer storage medium for traffic monitoring based on multimodality and hypergraph structure, storing computer executable instructions, characterized in that: The computer executable instructions are configured to: Execute the steps of a traffic monitoring method based on multimodality and hypergraph structure as described in any one of claims 1-6.

Images