Traffic monitoring method and device based on multi-mode and hypergraph structure, and medium

Abstract

The invention discloses a traffic monitoring method and device based on multiple modes and a hypergraph structure, and a medium, and relates to the technical field of intelligent traffic. The method comprises the following steps: acquiring multi-modal time-space sequence data through a monitoring point on a roadside, and acquiring real-time visibility and rainfall intensity data through a meteorological sensor; hypergraph nodes are generated according to the spatial topological relation of the monitoring points, hyperedges are generated according to the similarity of vehicle tracks and the space-time coupling degree of V2X communication data, and a hypergraph structure is obtained; performing multi-scale spectral decomposition on the hypergraph structure, suppressing high-frequency noise by adopting a learnable filter, and reconstructing spatial dependency features; and generating an abnormal threshold based on historical data, comparing the spatial dependency feature with the abnormal threshold, and predicting an abnormal diffusion path through a hypergraph structure. According to the method, the problems of sparse abnormal sample cracking and noise coupling through multi-scale spectral decomposition and learnable filtering are solved, a hypergraph structure is utilized to describe complex association, and the prediction precision is improved in combination with multi-modal verification.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technologies, and particularly to a traffic monitoring method, device, and medium based on multimodality and hypergraph structures. Background Art

[0002] Current urban traffic anomaly detection mainly relies on fixed sensor networks and static graph models, making it difficult to capture the dynamically evolving high-order correlation relationships between roads. Although traditional spatio-temporal graph convolution methods can model binary connections between nodes, they cannot represent the complex patterns of co-variation among multiple road segments. Especially in scenarios where traffic flow suddenly changes caused by abnormal events, the prediction error of the existing technology for congestion propagation paths increases significantly. In addition, existing data augmentation strategies are mostly limited to single-dimensional perturbations in time or space, and fail to effectively alleviate the coupled interference of abnormal sample sparsity and environmental noise in traffic data, resulting in a sharp decline in the detection performance of the model under complex meteorological conditions such as rain and fog.

[0003] In recent years, although hypergraph-based traffic modeling technologies can partially solve the problem of high-order relationship representation, their hyperedge weights mostly rely on manual setting or static road topologies, lacking the ability to adaptively adjust to real-time traffic patterns. The dynamic hypergraph method proposed in Patent CN117648652A introduces time series similarity metrics, but still has three limitations: First, it does not consider multimodal data; second, it does not consider the sharp decline in the detection performance of the model under complex meteorological conditions such as rain and fog; third, it does not consider multi-scale spectral decomposition to extract low-frequency basis functions reflecting traffic flow steady states and high-frequency components sensitive to anomalies.

[0004] Through the above analysis, the problems and defects existing in the prior art are as follows: The traffic monitoring of the hypergraph structure in the prior art does not consider multimodal data, meteorological data, and does not consider multi-scale spectral decomposition, the coupled interference of abnormal sample sparsity and environmental noise. Summary of the Invention

[0005] The embodiments of this application provide a traffic monitoring method, device, and medium based on multimodality and hypergraph structures, which can solve the problem that the traffic monitoring of the hypergraph structure in the prior art does not consider multimodal data, meteorological data, and does not consider multi-scale spectral decomposition, the coupled interference of abnormal sample sparsity and environmental noise.

[0006] In a first aspect, an embodiment of the present application provides a traffic monitoring method based on multi-modal and hypergraph structures. The method includes: obtaining multi-modal spatio-temporal sequence data through roadside monitoring points, collecting real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points including a camera array and a microwave radar, and the multi-modal data including vehicle trajectories and V2X communication data; generating hypergraph nodes according to the spatial topological relationship of the monitoring points, and generating hyperedges according to the similarity of vehicle trajectories and the spatio-temporal coupling degree of V2X communication data to obtain a hypergraph structure; performing multi-scale spectral decomposition on the hypergraph structure, using a learnable filter to suppress high-frequency noise, and reconstructing spatially dependent features; generating an anomaly threshold based on historical data, comparing the spatially dependent features with the anomaly threshold, and predicting the anomaly diffusion path through the hypergraph structure.

[0007] In an implementation manner of the present application, generating hypergraph nodes according to the spatial topological relationship of the monitoring points, and generating hyperedges according to the similarity of vehicle trajectories and the spatio-temporal coupling degree of V2X communication data to obtain a hypergraph structure specifically includes: calculating the shortest path distance between the monitoring points, constructing an adjacency matrix; based on the adjacency matrix, using the dynamic time warping algorithm to calculate the similarity of vehicle trajectories, and constructing hyperedges for the trajectories with similarity higher than a preset threshold.

[0008] In an implementation manner of the present application, the method further includes: when a vehicle emergency braking event is detected in the V2X communication data, taking the event occurrence location as the center, and enhancing the hyperedge weight to the associated road section according to the adjacency matrix; performing attenuation compensation on the hyperedges of the corresponding camera array and microwave radar according to the visibility data collected by the meteorological sensors; constructing a generative adversarial network, simulating the noise distribution of the meteorological sensors under extreme weather through the generator, and learning the robust representation of the hyperedge weight through the discriminator.

[0009] In an implementation manner of the present application, performing multi-scale spectral decomposition on the hypergraph structure, using a learnable filter to suppress high-frequency noise, and reconstructing spatially dependent features specifically includes: normalizing and multi-scale decomposing the hypergraph structure to obtain eigenvectors and corresponding eigenvalues, and extracting high-frequency components and low-frequency components according to the eigenvalues; suppressing the high-frequency components through a learnable filter, and the high-frequency components include congestion below a preset time; performing correlation matching between the eigenvectors and historical data, sorting the eigenvectors to perform phase correction; fusing the phase-corrected eigenvectors through an attention mechanism, and reconstructing the spatially dependent features using spectral domain inverse transformation.

[0010] In an implementation of the present application, the feature vectors after phase correction are fused through an attention mechanism, and the spatial dependence features are reconstructed by inverse spectral domain transformation, which specifically includes: calculating the correlation weights between nodes respectively by using a multi-head graph attention mechanism to capture the spatial dependence relationships of local congestion propagation, road section traffic flow, and speed respectively; constructing a stacked dilated convolutional layer, where the dilation factor of the dilated convolutional layer increases exponentially to capture congestions shorter than a preset time and congestions longer than the preset time simultaneously; projecting the correlation weights and the feature vectors processed by the dilated convolutional layer into the spectral domain, and obtaining the spatial dependence features through inverse spectral domain transformation.

[0011] In an implementation of the present application, an anomaly threshold is generated based on historical data, the spatial dependence features are compared with the anomaly threshold, and the anomaly diffusion path is predicted through a hypergraph structure, which specifically includes: grouping the historical data according to time granularity, extracting the traffic flow and speed of each group of historical data, and generating a reference probability distribution space by using kernel density estimation; projecting the currently phase-corrected feature vector into the reference probability distribution space, and calculating the Mahalanobis distance as the anomaly threshold; when it is detected that the anomaly thresholds of multiple consecutive cycles exceed the warning value, reducing the subsequent detection sensitivity threshold.

[0012] In an implementation of the present application, the anomaly diffusion path is predicted through a topological propagation model of a hypergraph structure, which specifically includes: based on the real-time updated adjacency matrix, simulating the propagation of anomalies from the initial node to adjacent nodes according to the spatial dependence features to obtain the predicted diffusion path; performing multi-modal cross-verification on the diffusion path in combination with the geographical locations of V2X communication data; when the vehicle emergency braking event in the V2X communication data overlaps with the predicted diffusion path, increasing the warning confidence level of the diffusion path; and updating the detour diffusion path to the navigation platform in real time.

[0013] In an implementation of the present application, the method further includes: building a two-way data channel between the digital twin and the real road network in the cloud, injecting historical anomaly patterns for stress testing; performing activity analysis on the hypergraph nodes, and turning off the meteorological sensors with activity lower than the threshold.

[0014] Second aspect, an embodiment of the present application further provides a traffic monitoring device based on multimodality and hypergraph structure. The device includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: obtain multimodal spatio-temporal sequence data through monitoring points on the roadside, collect real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points include a camera array and a microwave radar, and the multimodal data includes vehicle trajectories and V2X communication data; generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of vehicle trajectories and the spatio-temporal coupling degree of V2X communication data to obtain a hypergraph structure; perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependence features; generate an anomaly threshold based on historical data, compare the spatial dependence features with the anomaly threshold, and predict the anomaly diffusion path through the hypergraph structure.

[0015] Third aspect, an embodiment of the present application further provides a non-volatile computer storage medium for traffic monitoring based on multimodality and hypergraph structure, storing computer-executable instructions, and the computer-executable instructions are set to: obtain multimodal spatio-temporal sequence data through monitoring points on the roadside, collect real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points include a camera array and a microwave radar, and the multimodal data includes vehicle trajectories and V2X communication data; generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of vehicle trajectories and the spatio-temporal coupling degree of V2X communication data to obtain a hypergraph structure; perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependence features; generate an anomaly threshold based on historical data, compare the spatial dependence features with the anomaly threshold, and predict the anomaly diffusion path through the hypergraph structure.

[0016] A traffic monitoring method, device and medium based on multimodality and hypergraph structure provided by an embodiment of the present application obtain multimodal spatio-temporal sequences and meteorological data including vehicle trajectories and V2X communication data through roadside monitoring points, generate a hypergraph structure based on the spatial topology and data association of the monitoring points, reconstruct spatial dependence features after suppressing noise through multi-scale spectral decomposition and learnable filtering, then generate a dynamic anomaly threshold in combination with historical data and predict the diffusion path through the hypergraph, fuse multimodal and meteorological data to improve environmental adaptability, solve the problems of sparse anomaly samples and noise coupling through multi-scale spectral decomposition and learnable filtering, use the hypergraph structure to describe complex associations and combine multimodal verification to improve prediction accuracy, dynamically optimize the threshold based on historical data and perform digital twin stress testing to enhance system robustness, and systematically solve the defects of the prior art that do not consider multi-source data, multi-scale analysis and complex interference. Description of the Drawings

[0017] The accompanying drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings: Figure 1 is a flowchart of a traffic monitoring method based on multimodal and hypergraph structures provided by an embodiment of the present application; Figure 2 is a schematic internal structure diagram of a traffic monitoring device based on multimodal and hypergraph structures provided by an embodiment of the present application. Detailed implementation manners

[0018] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0019] The embodiments of the present application provide a traffic monitoring method, device, and medium based on multimodal and hypergraph structures, which solve the problems in the prior art that traffic monitoring using hypergraph structures does not consider multimodal data, meteorological data, and does not consider the coupling interference of multi-scale spectral decomposition, anomaly sample sparsity, and environmental noise.

[0020] The technical solutions proposed in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0021] Figure 1 is a flowchart of a traffic monitoring method based on multimodal and hypergraph structures provided by an embodiment of the present application. As Figure 1 shown, a traffic monitoring method based on multimodal and hypergraph structures provided by an embodiment of the present application specifically includes the following steps: Step 10: Obtain multimodal spatio-temporal sequence data through roadside monitoring points, and collect real-time visibility and precipitation intensity data through meteorological sensors. The monitoring points include a camera array and a microwave radar, and the multimodal data includes vehicle trajectories and V2X communication data.

[0022] In this step, the radar and the camera adopt a time synchronization protocol to ensure data alignment; the spatio-temporal coordinate sequence of each vehicle reflects the individual movement pattern and the group interaction relationship. The V2X communication data includes communication information between vehicles and infrastructure, and between vehicles, such as emergency braking signals and path planning intentions.

[0023] Step 20: Generate hypergraph nodes based on the spatial topological relationship of monitoring points, and generate hyperedges based on the similarity of vehicle trajectories and the spatio-temporal coupling degree of V2X communication data to obtain a hypergraph structure; As an alternative embodiment, generating hypergraph nodes based on the spatial topological relationship of monitoring points, and generating hyperedges based on the similarity of vehicle trajectories and the spatio-temporal coupling degree of V2X communication data to obtain a hypergraph structure may specifically include: Step 201: Calculate the shortest path distance between monitoring points and construct an adjacency matrix; Step 202: Based on the adjacency matrix, use the dynamic time warping algorithm to calculate the similarity of vehicle trajectories, and construct hyperedges for trajectories with similarity higher than a preset threshold.

[0024] In this step, the positions of cameras and radars serve as hypergraph nodes, and the spatial topological relationship is calculated from geographical locations. The physical reachability between nodes is characterized by the shortest path distance. The adjacency matrix A ∈ Rn×n, and the element Aij represents the connection weight between nodes vi and vj, reflecting the physical structure of the road network, which is the basis for subsequent hyperedge generation. Set a similarity threshold. If it exceeds the threshold, in the adjacency matrix, connect the monitoring points {vi, vj,...} passed by the trajectory into a hyperedge, indicating that these nodes have a strong association at the trajectory level.

[0025] A communication event indicates that communication occurs between nodes vi and vj at a certain moment. The coupling degree is defined as the normalized value of the number of communications per unit time. If the normalized value exceeds the threshold, connect the nodes {vi, vj,...} involved in the communication into a hyperedge to reflect the real-time communication association between nodes. Each hyperedge is attached with a weight, such as the trajectory similarity value or the communication coupling degree value, reflecting the association strength.

[0026] As an alternative embodiment, the method may further include: Step 203: When a vehicle emergency braking event is detected in the V2X communication data, with the event occurrence location as the center, enhance the hyperedge weight to the associated road section according to the adjacency matrix; Step 204: According to the visibility data collected by the meteorological sensor, perform attenuation compensation on the hyperedges of the corresponding camera array and microwave radar; Step 205: Construct a generative adversarial network, and use the generator to simulate the noise distribution of the meteorological sensor under extreme weather, and use the discriminator to learn the robust representation of the hyperedge weight.

[0027] Step 30: Perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct the spatial dependence features; As an alternative embodiment, performing multi-scale spectral decomposition on the hypergraph structure, using a learnable filter to suppress high-frequency noise, and reconstructing the spatial dependence features may specifically include: Step 301: Normalize and perform multi-scale decomposition on the hypergraph structure to obtain eigenvectors and corresponding eigenvalues, and extract high-frequency components and low-frequency components according to the eigenvalues; The eigen decomposition is L = UΛUT , where $\Lambda = \text{diag}(\lambda_1, \lambda_2, \ldots, \lambda_n)$ is the eigenvalue matrix and $U$ is the eigenvector matrix. The eigenvalues are arranged in ascending order $\lambda_1 \leq \lambda_2 \leq \ldots \leq \lambda_n$, corresponding to: Low-frequency components: Small eigenvalues: Represent the global structure and long-term dependencies of the hypergraph, such as the congestion propagation on the main roads.

[0028] High-frequency components: Large eigenvalues: Represent local fluctuations and short-term noise, such as occasional sudden brakes and sensor errors.

[0029] Step 302: Suppress high-frequency components through a learnable filter, where the high-frequency components include congestion below a preset time; In this step, the learnable filter adaptively suppresses high-frequency noise.

[0030] Step 303: Perform correlation matching between the eigenvectors and historical data, and sort the eigenvectors for phase correction; In this step, calculate the cosine similarity between the current eigenvector and the history, sort by similarity, select the preset number of the most similar historical features for phase correction, and sort and rotate the eigenvectors to align the phase with the historical pattern.

[0031] Step 304: Fuse the phase-corrected eigenvectors through an attention mechanism and use the inverse spectral transform to reconstruct the spatial dependence features.

[0032] In this step, convert the filtered frequency-domain features back to the spatial domain. This process restores the spatial dependence relationship between nodes. For the reconstructed features, the low-frequency components are retained and strengthened, while high-frequency noise and random sudden brakes are suppressed, improving the signal-to-noise ratio. Filter short-term fluctuations such as occasional sudden brakes, focus on the propagation path of continuous congestion on the main roads, and distinguish between the periodic morning rush hour traffic flow and sudden abnormal accidents.

[0033] As an optional embodiment, fusing the phase-corrected eigenvectors through an attention mechanism and using the inverse spectral transform to reconstruct the spatial dependence features may specifically include: Step 3041: Use the multi-head graph attention mechanism to calculate the association weights between nodes respectively, and capture the spatial dependence relationships of local congestion propagation, section flow, and speed respectively; Step 3042: Construct a stacked dilated convolutional layer, where the dilation factor of the dilated convolutional layer increases exponentially to capture congestion shorter than the preset time and congestion longer than the preset time simultaneously; Step 3043: Project the association weights and the eigenvectors processed by the dilated convolutional layer into the spectral domain, and obtain the spatial dependence features through the inverse spectral transform.

[0034] In this step, for nodes vi and vj, calculate the attention scores. Through the attention weights of neighboring nodes, obtain how congestion spreads from one monitoring point to adjacent points. Different heads can respectively focus on traffic changes, such as the spatial dependencies of traffic flow density and speed fluctuations.

[0035] The receptive field of standard convolution is k. Dilated convolution expands the receptive field to k+(k - 1)(d - 1) by introducing a dilation factor d. For the input feature map x, the output of dilated convolution is:

[0036] where r is the dilation factor, K is the convolution kernel size, and w[k] are fixed filter coefficients.

[0037] Design a stacked convolutional layer where the dilation factor grows as r = 2^n (n is the layer number). For the first layer, r = 1: capture congestion changes on a short time scale, such as within 30 seconds; For the second layer, r = 2: capture congestion propagation on a medium time scale, such as within 2 minutes; For the third layer, r = 4: capture congestion evolution on a long time scale, such as over 10 minutes. In this way, through the combination of different dilation factors, the model can handle both short-term and long-term congestion simultaneously. Multi-head attention is parameterized by different heads to respectively capture the spatial dependencies of different dimensions such as traffic flow and speed. Dilated convolution adaptively covers time scales from seconds to hours through exponentially growing receptive fields.

[0038] Step 40: Generate an anomaly threshold based on historical data, compare the spatial dependence features with the anomaly threshold, and predict the anomaly diffusion path through a hypergraph structure.

[0039] As an optional embodiment, generating an anomaly threshold based on historical data, comparing the spatial dependence features with the anomaly threshold, and predicting the anomaly diffusion path through a hypergraph structure may specifically include: Step 401: Group the historical data by time granularity, extract the traffic flow and speed of each group of historical data, and use kernel density estimation to generate a reference probability distribution space; Step 402: Project the feature vector after current phase correction into the reference probability distribution space and calculate the Mahalanobis distance as the anomaly threshold; Step 403: When it is detected that the anomaly threshold exceeds the warning value for multiple consecutive cycles, reduce the subsequent detection sensitivity threshold.

[0040] In this step, slice the historical data by time: hours, weekdays / weekends, seasons, and include anomaly data during historical heavy rains. Construct a two-dimensional probability distribution for the traffic flow f and speed v of each group of data:

[0041] Among them, K(⋅) is the kernel function, x and y represent two-dimensional feature variables in historical data, specifically corresponding to the flow rate f and velocity v. h is the bandwidth parameter that controls the distribution smoothness, and this distribution reflects the joint probability density of the flow rate and velocity under normal operating conditions. Project the current feature vector into the reference distribution space and calculate its Mahalanobis distance from the distribution center. The Mahalanobis distance takes into account the correlation between features. Since the flow rate and velocity are negatively correlated, it is more reliable than the Euclidean distance. When there are consecutive anomalies, lower the detection threshold to increase the sensitivity and accelerate the capture of the diffusion path. When there are no anomalies, gradually restore the default threshold to reduce false alarms.

[0042] As an alternative embodiment, the abnormal diffusion path is predicted through the topological propagation model of the hypergraph structure, which may specifically include: Step 404: Based on the real-time updated adjacency matrix, simulate the propagation of the anomaly from the initial node to the adjacent nodes according to the spatial dependence features to obtain the predicted diffusion path; Step 405: Perform multimodal cross-verification on the diffusion path in combination with the geographical location of the V2X communication data; Step 406: When the vehicle emergency braking event in the V2X communication data overlaps with the predicted diffusion path, raise the warning confidence level of this diffusion path; Step 407: Real-time update the detour diffusion path to the navigation platform.

[0043] In this step, based on the adjacency matrix A and hyperedge weights w of the hypergraph e , the graph diffusion model is used to simulate the anomaly propagation s(t + 1)=A⋅s(t), where s(t) is the node anomaly intensity vector, and the initial value is the node si = 1 where the anomaly is detected, and the rest are 0. If the predicted diffusion path contains the node sequence [v1, v2, v3], check whether there is an emergency braking event continuously appearing on the spatial path of v1→v2→v3 in the V2X data during the corresponding time period.

[0044] As an alternative embodiment, the method may further include: constructing a two-way data channel between the digital twin and the real-world road network in the cloud and injecting historical anomaly patterns for stress testing; performing activity analysis on the hypergraph nodes and turning off the meteorological sensors with activity lower than the threshold.

[0045] In this step, real-time monitoring data such as camera video streams and radar point clouds are transmitted through 5G and the Internet of Things to update the digital twin state; the optimization strategies generated by the stress testing, such as signal timing schemes and emergency lane activation, are fed back to the roadside devices for execution. Further, historical anomaly events, such as the congestion of the main road caused by heavy rain, are replayed in the digital twin to verify the effectiveness of the current monitoring algorithm; extreme scenarios are artificially created, such as not pushing the currently updated detour roads, to evaluate the system crash threshold.

[0046] The above is the method embodiment proposed in this application. Based on the same inventive concept, the embodiments of this application also provide a traffic monitoring device based on multimodality and hypergraph structure, and its structure is asFigure 2 as shown

[0047] Figure 2 The figure is a schematic internal structure diagram of a traffic monitoring device based on multimodality and hypergraph structure provided by an embodiment of the present application. As Figure 2 shown, the device includes: at least one processor 201; and a memory 202 communicatively connected to the at least one processor; wherein, the memory 202 stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor 201 so that the at least one processor 201 can: obtain multimodal spatio-temporal sequence data through monitoring points on the roadside, collect real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points include a camera array and a microwave radar, and the multimodal data includes vehicle trajectories and V2X communication data; generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of vehicle trajectories and the spatio-temporal coupling degree of V2X communication data to obtain a hypergraph structure; perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependence features; generate an anomaly threshold based on historical data, compare the spatial dependence features with the anomaly threshold, and predict the anomaly diffusion path through the hypergraph structure.

[0048] Some embodiments of the present application provide a non-volatile computer storage medium corresponding to Figure 1 for traffic monitoring based on multimodality and hypergraph structure, storing computer-executable instructions, and the computer-executable instructions are set to: obtain multimodal spatio-temporal sequence data through monitoring points on the roadside, collect real-time visibility and precipitation intensity data through meteorological sensors, the monitoring points include a camera array and a microwave radar, and the multimodal data includes vehicle trajectories and V2X communication data; generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of vehicle trajectories and the spatio-temporal coupling degree of V2X communication data to obtain a hypergraph structure; perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct spatial dependence features; generate an anomaly threshold based on historical data, compare the spatial dependence features with the anomaly threshold, and predict the anomaly diffusion path through the hypergraph structure.

[0049] Each embodiment in the present application is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments. In particular, for the embodiments of the Internet of Things devices and media, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0050] The systems, media, and methods provided by the embodiments of the present application correspond one by one. Therefore, the systems and media also have beneficial technical effects similar to those of their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be elaborated here.

[0051] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0052] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one or more of the flows Figure 1 or blocks.

[0053] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the specified functions in Figure 1 one or more of the flows Figure 1 or blocks.

[0054] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in Figure 1 one or more of the flows Figure 1 or blocks.

[0055] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0056] The memory may include non - permanent memory in the form of computer - readable media, random access memory (RAM) and / or non - volatile memory such as read - only memory (ROM) or flash RAM. Memory is an example of computer - readable media.

[0057] Computer - readable media includes permanent and non - permanent, removable and non - removable media that can store information by any method or technology. The information can be computer - readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase - change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read - only memory (ROM), electrically erasable programmable read - only memory (EEPROM), flash memory or other memory technologies, compact disc read - only memory (CD - ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non - transitory media that can be used to store information that can be accessed by a computing device. As defined herein, computer - readable media does not include transitory computer - readable media such as modulated data signals and carrier waves.

[0058] It should also be noted that the term "comprises", "comprising" or any other variation thereof is intended to cover a non - exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements but also other elements not expressly listed or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus that comprises the element.

[0059] The above - described are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A traffic monitoring method based on multi-modal and hypergraph structure, characterized in that The method includes: Obtaining multi-modal spatio-temporal sequence data through roadside monitoring points, collecting real-time visibility and precipitation intensity data through meteorological sensors, where the monitoring points include a camera array and a microwave radar, and the multi-modal data includes vehicle trajectories and V2X communication data; Generating hypergraph nodes according to the spatial topological relationship of the monitoring points, and generating hyperedges according to the similarity of the vehicle trajectories and the spatio-temporal coupling degree of the V2X communication data to obtain a hypergraph structure; Performing multi-scale spectral decomposition on the hypergraph structure, using a learnable filter to suppress high-frequency noise, and reconstructing spatially dependent features; Generating an anomaly threshold based on historical data, comparing the spatially dependent features with the anomaly threshold, and predicting the anomaly diffusion path through the hypergraph structure.

2. The traffic monitoring method based on multi-modal and hypergraph structure according to claim 1, wherein Generating hypergraph nodes according to the spatial topological relationship of the monitoring points, and generating hyperedges according to the similarity of the vehicle trajectories and the spatio-temporal coupling degree of the V2X communication data to obtain a hypergraph structure, specifically including: Calculating the shortest path distance between the monitoring points and constructing an adjacency matrix; Based on the adjacency matrix, using the dynamic time warping algorithm to calculate the similarity of the vehicle trajectories, and constructing hyperedges for the trajectories with similarity higher than a preset threshold.

3. The traffic monitoring method based on multimodality and hypergraph structure according to claim 2, characterized in that, The method further includes: When a vehicle hard braking event is detected in the V2X communication data, taking the event occurrence location as the center, and enhancing the hyperedge weight to the associated road section according to the adjacency matrix; Performing attenuation compensation on the hyperedges of the corresponding camera array and microwave radar according to the visibility data collected by the meteorological sensor; Constructing a generative adversarial network, simulating the noise distribution of the meteorological sensor under extreme weather through a generator, and learning the robust representation of the hyperedge weight through a discriminator.

4. A traffic monitoring method based on multimodality and hypergraph structure according to claim 1, characterized in that, Performing multi-scale spectral decomposition on the hypergraph structure, using a learnable filter to suppress high-frequency noise, and reconstructing spatially dependent features, specifically including: Normalizing and multi-scale decomposing the hypergraph structure to obtain eigenvectors and corresponding eigenvalues, and extracting high-frequency components and low-frequency components according to the eigenvalues; Suppressing the high-frequency components through a learnable filter, where the high-frequency components include congestion within a preset time; Performing correlation matching between the eigenvectors and historical data, and sorting the eigenvectors for phase correction; Fusing the phase-corrected eigenvectors through an attention mechanism, and reconstructing spatially dependent features using spectral domain inverse transformation.

5. The traffic monitoring method based on multimodality and hypergraph structure according to claim 4, wherein, Fusing the phase-corrected eigenvectors through an attention mechanism, and reconstructing spatially dependent features using spectral domain inverse transformation, specifically including: Using a multi-head graph attention mechanism to calculate the association weights between nodes respectively, and capturing the spatial dependence relationships of local congestion propagation, road section traffic flow, and speed respectively; Constructing a stacked dilated convolutional layer, where the dilation factor of the dilated convolutional layer increases exponentially to capture congestion shorter and longer than a preset time simultaneously; Projecting the association weights and the eigenvectors processed by the dilated convolutional layer into the spectral domain, and obtaining spatially dependent features through spectral domain inverse transformation.

6. A traffic monitoring method based on multimodality and hypergraph structure according to claim 4, characterized in that Generating an anomaly threshold based on historical data, comparing the spatially dependent features with the anomaly threshold, and predicting the anomaly diffusion path through the hypergraph structure, specifically including: Group the historical data according to time granularity, extract the traffic flow and speed of each group of the historical data, and generate a benchmark probability distribution space using kernel density estimation; Project the feature vector after the current phase correction into the benchmark probability distribution space, and calculate the Mahalanobis distance as the anomaly threshold; When it is detected that the anomaly thresholds of multiple consecutive cycles exceed the warning value, reduce the subsequent detection sensitivity threshold.

7. A traffic monitoring method based on multimodality and hypergraph structure according to claim 6, characterized in that, Predict the anomaly diffusion path through the topological propagation model of the hypergraph structure, specifically including: Based on the adjacency matrix updated in real time, simulate the propagation of the anomaly from the initial node to the adjacent nodes according to the spatial dependence characteristics to obtain the predicted diffusion path; Perform multimodal cross-validation on the diffusion path in combination with the geographical location of the V2X communication data; When the vehicle hard braking event in the V2X communication data overlaps with the predicted diffusion path, increase the warning confidence level of the diffusion path; Update the navigation platform in real time to bypass the diffusion path.

8. A traffic monitoring method based on multimodality and hypergraph structure according to claim 1, characterized in that, The method further includes: Build a two-way data channel between the digital twin and the real road network in the cloud, and inject historical anomaly patterns for stress testing; Perform activity analysis on the hypergraph nodes, and turn off the meteorological sensors with activity lower than the threshold.

9. A traffic monitoring device based on multimodality and hypergraph structure, characterized in that The device includes: At least one processor; And a memory communicatively connected to the at least one processor; Wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: Obtain multi-modal spatio-temporal sequence data through roadside monitoring points, and collect real-time visibility and precipitation intensity data through meteorological sensors. The monitoring points include camera arrays and microwave radars, and the multi-modal data includes vehicle trajectories and V2X communication data; Generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of the vehicle trajectories and the spatio-temporal coupling degree of the V2X communication data to obtain a hypergraph structure; Perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct the spatial dependence characteristics; Generate an anomaly threshold based on historical data, compare the spatial dependence characteristics with the anomaly threshold, and predict the anomaly diffusion path through the hypergraph structure.

10. A non-volatile computer storage medium for traffic monitoring based on multi-modal and hypergraph structures, storing computer-executable instructions, characterized in that, The computer-executable instructions are set to: Obtain multi-modal spatio-temporal sequence data through roadside monitoring points, and collect real-time visibility and precipitation intensity data through meteorological sensors. The monitoring points include camera arrays and microwave radars, and the multi-modal data includes vehicle trajectories and V2X communication data; Generate hypergraph nodes according to the spatial topological relationship of the monitoring points, and generate hyperedges according to the similarity of the vehicle trajectories and the spatio-temporal coupling degree of the V2X communication data to obtain a hypergraph structure; Perform multi-scale spectral decomposition on the hypergraph structure, use a learnable filter to suppress high-frequency noise, and reconstruct the spatial dependence characteristics; Generate an anomaly threshold based on historical data, compare the spatial dependence characteristics with the anomaly threshold, and predict the anomaly diffusion path through the hypergraph structure.

Images