Methods, devices and electronic equipment for urban traffic congestion identification

Abstract

This invention discloses a method, device, and electronic device for urban traffic congestion identification. Applied to the field of artificial intelligence technology, the specific solution is as follows: First, acquire multimodal data on traffic information within a target area; second, perform time-delay compensation processing on the multimodal data to obtain input data, and process the input data based on a lightweight encoder corresponding to each modality to determine the embedding vector corresponding to each modality; third, determine the model processing depth value corresponding to each modality based on the embedding vectors corresponding to all modalities; wherein, the model processing depth value is used to characterize the number of processing layers of the backbone network for the embedding vectors; fourth, process the embedding vectors using the target model and the model processing depth value to obtain a fusion vector, and process the fusion vector based on the decision layer of the target model to obtain the traffic congestion state. This invention achieves accurate identification of traffic congestion state in a target area, improving the efficiency of congestion state identification.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus and electronic device for identifying urban traffic congestion. Background Technology

[0002] In urban traffic management, in order to improve road traffic efficiency, reduce travel delays, and optimize the allocation of traffic resources, traffic conditions are usually identified to determine the state of traffic congestion.

[0003] Currently, existing methods for identifying traffic congestion mainly include manual monitoring and traffic congestion identification based on single-modal data. Manual monitoring suffers from high labor costs, significant subjective identification errors, and difficulty in real-time monitoring of traffic congestion. Single-modal data-based traffic congestion identification, due to its limited information dimensions, suffers from insufficient robustness and difficulty in identifying congestion in complex traffic events, such as multi-dimensional events like accidents, construction, and low-altitude traffic interference. Summary of the Invention

[0004] This invention provides a method, device, and electronic device for identifying urban traffic congestion, which enables accurate identification of traffic congestion status in a target area and improves the efficiency of congestion status identification.

[0005] According to one aspect of the present invention, a method for identifying urban traffic congestion is provided, the method comprising: Acquire multimodal data on traffic information within the target area; wherein, the multimodal data includes at least two of the following: road image information collected by roadside camera devices, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information; Multimodal data undergoes time delay compensation to obtain input data, and the input data is processed based on the lightweight encoder corresponding to each modality to determine the embedding vector corresponding to each modality. Based on the embedding vectors corresponding to all modalities, determine the model processing depth value corresponding to each modal data; where the model processing depth value is used to characterize the number of processing layers of the backbone network for the embedding vectors; By using the target model and model processing depth values, the embedded vector is processed to obtain the fused vector. The fused vector is then processed based on the decision layer of the target model to obtain the traffic congestion status.

[0006] According to another aspect of the present invention, a device for identifying urban traffic congestion is provided, the device comprising: The data acquisition module is used to acquire multimodal data about traffic information within the target area; the multimodal data includes at least two of the following: road image information collected by roadside camera devices, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information. The embedding vector determination module is used to perform time delay compensation processing on multimodal data to obtain input data, and to process the input data based on the lightweight encoder corresponding to each modality to determine the embedding vector corresponding to each modality. The depth value determination module is used to determine the model processing depth value corresponding to each modality data based on the embedding vectors corresponding to all modalities; wherein, the model processing depth value is used to characterize the number of processing layers of the backbone network for the embedding vectors; The congestion status determination module is used to process the embedded vector using the target model and model processing depth values ​​to obtain a fused vector, and then process the fused vector based on the decision layer of the target model to obtain the traffic congestion status.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the urban traffic congestion identification method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided storing computer instructions for causing a processor to execute and implement the urban traffic congestion identification method of any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, characterized in that the computer program, when executed by a processor, implements the urban traffic congestion identification method as described in any embodiment of the present invention.

[0010] The technical solution of this invention achieves automatic acquisition of traffic information in unattended situations by obtaining multimodal data on traffic information within a target area, providing data support for subsequent determination of traffic congestion status. The multimodal data undergoes time-series delay compensation processing to obtain the output data, achieving time-series alignment of the multimodal data and ensuring consistency across the temporal dimension. A lightweight encoder corresponding to each modality processes the input data to obtain an embedding vector corresponding to each modality. Based on the embedding vectors corresponding to all modalities, the model processing depth value corresponding to each modality is determined. This allows for optimal allocation of computational resources to the target model based on the model processing depth value, ensuring the overall performance of the target model while avoiding the high computational cost of full-depth processing of all modal data, effectively reducing the computational cost of the target model. This invention utilizes a target model and model processing depth values ​​to process embedded vectors, obtaining a fused vector. The fused vector is then further processed based on the decision layer of the target model to determine traffic congestion status. By using model processing depth values, computational power is allocated differently to different modalities, ensuring sufficient processing of high-information-density modal data while effectively reducing the risk of overfitting to low-information-density modal data, thus improving the model's robustness and generalization performance. This invention solves the delays and misjudgments caused by human identification in existing technologies, significantly reducing operating costs. Furthermore, by employing multimodal data processing, it fully utilizes the complementary characteristics of each modality, improving the robustness and reliability of congestion status identification, making it particularly suitable for traffic status perception under complex urban road conditions and extreme weather conditions. Moreover, by using the target model and model processing depth values ​​to process embedded vectors, multimodal deep fusion at the feature level is achieved, avoiding the potential risks of "text-centric bias" and "large model illusion," thus improving the credibility and accuracy of congestion status identification. In addition, by determining the model processing depth value, each modal feature can exit the calculation in advance after reaching the preset depth, reducing the computing power requirement, reducing the risk of overfitting, and improving the efficiency of congestion state recognition.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a method for identifying urban traffic congestion provided in an embodiment of the present invention; Figure 2 This is a flowchart of a method for identifying urban traffic congestion provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an urban traffic congestion identification device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the urban traffic congestion identification method according to an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1 This is a flowchart of a method for identifying urban traffic congestion according to Embodiment 1 of the present invention. This embodiment is applicable to situations where traffic congestion status of a target area is identified based on multimodal data. This method can be executed by an urban traffic congestion identification device, which can be implemented in hardware and / or software. This device can be configured in electronic devices such as mobile phones, computers, or servers. Figure 1 As shown, the method includes: S110. Obtain multimodal data on traffic information within the target area.

[0017] The target area can be the area where traffic congestion identification is currently required. The multimodal data includes at least two of the following: road image information collected by roadside cameras, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information.

[0018] The roadside camera device can be a camera device located on both sides of the road in the target area. The roadside camera device can collect road image information of the target area in real time. Optionally, the road image information can be in the form of color video frames (RGB video frames). The road image information includes at least one traffic participation element in the corresponding road. Traffic participation elements can be understood as components that constitute the traffic system and affect traffic operation and safety. Optionally, traffic participation elements can include: road users (e.g., pedestrians, drivers of motor vehicles or non-motorized vehicles), infrastructure (e.g., roads, traffic signals, traffic facilities), and vehicles (e.g., motor vehicles, non-motorized vehicles).

[0019] Radar point cloud data can be 3D point cloud data of traffic participants. This radar point cloud data can be generated through urban infrastructure, such as sensor fusion base stations. Information such as the location and speed of traffic participants can be determined using this radar point cloud data.

[0020] The GPS trajectory of a floating car can be defined as the trajectory information formed by the location, speed, time, and other data collected by a floating car equipped with a Global Positioning System (GPS) positioning device during its operation. The floating car can be a moving vehicle that collects and transmits its trajectory, speed, and location data in real time by installing a GPS positioning device. Weather information can include meteorological data such as temperature, humidity, and rainfall within the target area.

[0021] Specifically, at least two of the following are acquired: road image information collected by roadside cameras within the target area, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information, in order to comprehensively determine the road congestion status of the target area through the above multimodal data.

[0022] S120. Perform time delay compensation processing on the multimodal data to obtain the input data, and process the input data based on the lightweight encoder corresponding to each modal data to determine the embedding vector corresponding to each modality.

[0023] The timing delay compensation process can involve timing alignment of the multimodal data to ensure its timing consistency. The input data can be the timing-aligned multimodal data.

[0024] For each modality of data, the corresponding lightweight encoder is different. By using a lightweight encoder specific to a particular modality, targeted processing of the data within that modality can be achieved. The lightweight encoder, also known as a lightweight specific encoder, is used to map the data within the input data to the corresponding modality to determine the embedding vector. The embedding vector is the vector obtained after mapping the data to the corresponding modality. The number of embedding vectors corresponds to the number of modalities in the input data.

[0025] Specifically, based on the time information corresponding to each modality in the multimodal data, one modality is used as a benchmark to perform time alignment and time delay compensation on the other modal data to determine the temporally consistent multimodal data, which is the input data. The data of each modality in the input data is then input into the corresponding lightweight encoder to obtain the embedding vector corresponding to each modality.

[0026] In this embodiment of the invention, the method for performing time delay compensation processing on multimodal data can be as follows: obtain the hardware timestamps of each sensor and record the time delay values ​​corresponding to different sensors; use the acquisition time information corresponding to the target sensor that meets the preset conditions as the reference time, and perform time alignment processing on the multimodal data based on the reference time and the time delay values; for any modal data, if the time difference between any two adjacent time points in the modal data is greater than a preset difference threshold, then interpolation is performed based on the modal data of the two time points and the number of interpolations to obtain the input data; wherein, the number of interpolations is related to the time difference; wherein, the target sensor corresponds to the roadside camera device.

[0027] Multimodal data includes modal data across multiple modalities, each of which can be acquired by a corresponding sensor. A hardware timestamp is a time stamp directly embedded by the corresponding sensor during modal data acquisition, synchronized with the hardware clock; it records the precise moment the modal data was generated. Time delay values ​​can be used to characterize the time delay information corresponding to different modal data.

[0028] The target sensor meeting the preset conditions can be a target sensor whose hardware timestamp matches the corresponding preset timestamp. Optionally, the target sensor can correspond to a roadside camera device. The acquisition time information corresponding to the target sensor can be the acquisition time corresponding to when the roadside camera device acquires road image information, i.e., the reference time. The reference time can be used for time alignment processing of modal data of other modalities.

[0029] The preset difference threshold can be a pre-set standard value for the time difference. Interpolation refers to the new modal data that needs to be inserted between two time points. The time corresponding to the interpolation is located between the two time points. The number of interpolations can be the number of modal data that needs to be inserted between the modal data of the two time points.

[0030] Specifically, the hardware timestamps of the sensors used to collect data from each modality are obtained. Based on the hardware timestamps of each sensor, the time delay values ​​corresponding to different sensors are determined. If the hardware timestamp of a sensor matches a preset timestamp, the sensor is determined to meet a preset condition, and the sensor that meets the preset condition is designated as the target sensor. Optionally, the target sensor may correspond to a roadside camera device.

[0031] Using the acquisition time information of the target sensor for the corresponding modal data as the reference time, and based on the reference time and the time delay values ​​corresponding to different sensors, delay compensation processing is performed on the modal data corresponding to other sensors to ensure the temporal consistency of each modal data in the multimodal data.

[0032] Because different sensors have different sampling frequencies for corresponding modal data—for example, a roadside camera can collect 30 frames of road image information per second, while a sensor used to collect weather information can collect weather information every tens of minutes—modal data corresponding to low sampling frequencies can be replicated, extended, or fitted with differences to match modal data corresponding to high sampling frequencies. Furthermore, since sensors may experience intermittent failures or lack effective modal data, modal data can be interpolated or smoothed to ensure that the time points corresponding to each modal data are consistent and that effective modal data exists in all cases. Specifically, for any modal data, if the time difference between any two adjacent time points (the time points when the modal data was collected) is greater than a corresponding preset difference threshold, interpolation can be determined based on the modal data corresponding to the two time points, and the number of interpolations can be determined based on the time difference. Interpolation processing is then performed on the modal data corresponding to the two time points based on the interpolation number, resulting in time-adjusted modal data corresponding to each modality. All time-adjusted modal data corresponding to all modalities are then used as input data.

[0033] In this embodiment of the invention, the specific method for determining the embedding vector corresponding to each modality may be: retrieving a pre-set lightweight encoder corresponding to each modality data; processing the input data of the corresponding modality based on the optimizable parameters in the lightweight encoder to obtain the embedding vector of the modality.

[0034] In the lightweight encoder, the optimizable parameters can be learnable parameters, which can be jointly optimized with the subsequent target model to achieve complete gradient propagation from the original input (modal data) to the final decision (traffic congestion state).

[0035] Specifically, based on the modalities corresponding to the multimodal data, a lightweight encoder corresponding to each modality is retrieved. The input data for the corresponding modality is preprocessed using the lightweight encoder to obtain the embedding vector corresponding to that modality. Optionally, for the modal data... It can be achieved through a lightweight, specific encoder. Map it to an embedding vector of uniform dimension. The specific process can be illustrated by the following function: ; in, Represents an embedding vector. This indicates a lightweight encoder for the corresponding mode. Represents modal data, This indicates parameters that can be optimized.

[0036] It should be noted that, based on the features corresponding to different modal data (e.g., spatial features of road image information, time-series features of floating car GPS trajectories), different optimization strategies can be adopted for lightweight encoders under different modalities to preserve key information and obtain embedding vectors. For example, for road image information, a lightweight encoder corresponding to a convolutional neural network can be used to extract features from the road image information to obtain embedding vectors. For weather information, word embedding encoders, such as Word2Vec and BERT-mini, can be used to encode the weather information to obtain semantic vectors (embedding vectors). Therefore, the embodiments of this invention do not limit the specific lightweight encoder used.

[0037] S130. Based on the embedding vectors corresponding to all modalities, determine the model processing depth value corresponding to each modal data; wherein, the model processing depth value is used to characterize the number of processing layers of the backbone network for the embedding vectors.

[0038] Optionally, the process of processing the embedded vectors to determine the model processing depth value can be implemented through a depth labeling module. The depth labeling module, namely the Traffic-Aware Depth Evaluator (TADE) module, is used to process the embedded vectors for each modality and generate the model processing depth value corresponding to the modality data.

[0039] Specifically, the embedding vectors of the corresponding modalities are processed by the depth labeling module to determine the model processing depth value corresponding to each modal data.

[0040] Optionally, for each modality's embedding vector The depth is processed by the traffic scene perception depth assessment module (depth labeling module) to obtain the model processing depth value. To process depth values ​​based on a model This controls the propagation depth of the embedding vector in the backbone network of the subsequent target model. The model processes depth values. A positive integer, representing the embedding vector. The number of modules that should participate in the calculation. When the embedding vector... Spread to the After the initial layer, it exits early and no longer participates in deeper neural network propagation. The features routed to the backbone network and then stored in the modality feature fusion layer are fused together after all modal embedding vectors have been processed.

[0041] S140. Using the target model and model processing depth value, the embedded vector is processed to obtain the fused vector, and the fused vector is processed based on the decision layer of the target model to obtain the traffic congestion status.

[0042] The target model can be a model used to process the embedding vectors corresponding to all modalities to determine the traffic congestion state. The fusion vector can be a fusion vector corresponding to the multimodal data. The decision layer can be a network layer used to identify the fusion vector to determine the traffic congestion state. The traffic congestion state is used to characterize the traffic congestion situation in the target area. Optionally, the traffic congestion state can be divided into smooth traffic, light congestion, moderate congestion, heavy congestion, etc.

[0043] Specifically, the backbone network based on the target model processes the corresponding embedding vectors according to the model's processing depth value, obtaining a processing result for each embedding vector. Modal feature fusion processing is then performed based on the processing results for each embedding vector to obtain a fused vector. Finally, the decision layer of the target model processes the fused vector to determine the traffic congestion state.

[0044] Optionally, embodiments of the present invention further include: when the traffic congestion state meets preset congestion conditions, retrieving a target map based on traffic congestion information corresponding to the traffic congestion state to generate a target strategy corresponding to the traffic congestion information; wherein the target strategy includes at least one or more of the following: extending the green light duration, activating the emergency lane, and guiding detour routes.

[0045] Specifically, if the current traffic congestion state matches any preset traffic congestion state, it can be determined that the traffic congestion state meets the preset congestion condition. There can be one or more preset traffic congestion states; optionally, the preset traffic congestion state can be moderate congestion or severe congestion. For example, if the current traffic congestion state is moderate congestion, then the current traffic congestion state is determined to meet the preset congestion condition.

[0046] Traffic congestion information corresponding to a traffic congestion status can be traffic information about the traffic congestion event that caused the congestion. For example, specific traffic information related to morning rush hour congestion, accident congestion, construction congestion, etc.

[0047] The target map can be a map corresponding to the target area. The target strategy is an executable traffic management strategy for the target area. Optionally, the target strategy may include at least one or more of the following: extending green light duration, activating emergency lanes, and guiding detour routes.

[0048] Specifically, after obtaining the traffic congestion status, if the traffic congestion status matches any of at least one preset traffic congestion status, then the traffic congestion status is determined to meet the preset congestion conditions. When the traffic congestion status meets the preset congestion conditions, the traffic congestion event corresponding to the traffic congestion status and the traffic congestion information corresponding to the traffic congestion event are identified. Based on the traffic congestion information, the target map is retrieved, and the target map and traffic congestion information are processed using a rule engine or lightweight generation model to determine the target strategy. The target strategy is then pushed to the traffic control center or navigation system via an API interface to enable traffic scheduling in the target area.

[0049] The technical solution of this embodiment achieves automatic acquisition of traffic information in unattended situations by obtaining multimodal data on traffic information within the target area, providing data support for subsequent determination of traffic congestion status. Temporal delay compensation processing is performed on the multimodal data to obtain the output data, achieving temporal alignment of the multimodal data and ensuring consistency in the temporal dimension. A lightweight encoder corresponding to each modality processes the input data to obtain an embedding vector corresponding to each modality. Based on the embedding vectors corresponding to all modalities, the model processing depth value corresponding to each modality is determined. This allows for optimal allocation of computational resources to the target model based on the model processing depth value, ensuring the overall performance of the target model while avoiding the high computational cost of full-depth processing of all modal data, effectively reducing the computational cost of the target model. This invention utilizes a target model and model processing depth values ​​to process embedded vectors, obtaining a fused vector. The fused vector is then further processed based on the decision layer of the target model to determine traffic congestion status. By using model processing depth values, computational power is allocated differently to different modalities, ensuring sufficient processing of high-information-density modal data while effectively reducing the risk of overfitting to low-information-density modal data, thus improving the model's robustness and generalization performance. This invention solves the delays and misjudgments caused by human identification in existing technologies, significantly reducing operating costs. Furthermore, by employing multimodal data processing, it fully utilizes the complementary characteristics of each modality, improving the robustness and reliability of congestion status identification, making it particularly suitable for traffic status perception under complex urban road conditions and extreme weather conditions. Moreover, by using the target model and model processing depth values ​​to process embedded vectors, multimodal deep fusion at the feature level is achieved, avoiding the potential risks of "text-centric bias" and "large model illusion," thus improving the credibility and accuracy of congestion status identification. In addition, by determining the model processing depth value, each modal feature can exit the calculation in advance after reaching the preset depth, reducing the computing power requirement, reducing the risk of overfitting, and improving the efficiency of congestion state recognition.

[0050] Example 2 Figure 2 This is a flowchart of a method for identifying urban traffic congestion according to Embodiment 2 of the present invention. This embodiment is a preferred embodiment of the above embodiments. For specific implementation details, please refer to the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here. Figure 2 As shown, the method includes: S210. Obtain multimodal data on traffic information within the target area.

[0051] The multimodal data includes at least two of the following: road image information collected by roadside cameras, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information.

[0052] S220. Perform time delay compensation processing on the multimodal data to obtain the input data, and process the input data based on the lightweight encoder corresponding to each modal data to determine the embedding vector corresponding to each modality.

[0053] S230. Input the embedding vectors of all modalities into the depth labeling module so that the depth labeling module outputs the enhanced feature vector corresponding to each modality.

[0054] The depth labeling module, also known as the Traffic-Aware Depth Evaluator (TADE), processes the embedding vectors for each modality to generate a model processing depth value corresponding to the modality data. The enhanced feature vectors can be the feature vectors determined by the depth labeling module through the extraction and processing of the embedding vectors.

[0055] Specifically, the embedding vectors of all modalities are input into the deep labeling module, which processes the embedding vectors corresponding to all modalities to determine the enhanced feature vector corresponding to each modality.

[0056] In this embodiment of the invention, the method for determining the enhanced feature vector by processing the embedded vector based on the deep labeling module can be as follows: the embedded vector is processed by the first unit in the deep labeling module to output a structured traffic context vector; wherein, the structured traffic context vector includes at least the results under the spatial attribute dimension, the results under the temporal attribute dimension, the results under the environmental state dimension, and the results under the traffic flow state dimension; the embedded vector is processed by the second unit in the deep labeling module to obtain an event stage identifier; wherein, the event stage identifier includes at least any one of the latent period, development period, peak period, and dissipation period of the traffic state; the structured traffic context vector, the event stage identifier, and the embedded vector are processed by the third unit in the deep labeling module to obtain the enhanced feature vector.

[0057] The first unit of the depth labeling module is used to initially fuse the embedding vectors corresponding to all modules to infer the current macroscopic traffic environment state and determine the structured traffic context vector. Optionally, the first unit may include a Multi-Layer Perceptron (MLP) structure and / or a lightweight Transformer encoder. The structured traffic context vector is used to represent the macroscopic traffic environment at the current spatiotemporal location. This structured traffic context vector serves as an auxiliary input for subsequent depth evaluation and modal control, improving the understanding of scene semantics. Optionally, the structured traffic context vector includes at least the results under the spatial attribute dimension, the results under the temporal attribute dimension, the results under the environmental state dimension, and the results under the traffic flow state dimension. Among them, the results under the spatial attribute dimension can be determined by the embedding vector corresponding to the road image information and the map information corresponding to the target area. For example, the results under the spatial attribute dimension may include: the current road segment type (expressway, arterial road, ramp, intersection), road geometry information (curves, slope, number of lanes), and whether there are special areas such as construction zones, schools, and tunnels nearby. The results under the temporal attribute dimension can be determined by the temporal information corresponding to the embedding vectors of each modality. For example, results under the time attribute dimension can include: the current time period (morning peak 7:00–9:00, evening peak 17:00–19:00, nighttime 22:00–6:00, etc.), and whether it is a holiday or special event day (such as a major event or a severe weather warning day). Results under the environmental state dimension can be determined by the embedding vectors corresponding to weather information and road image information. For example, results under the environmental state dimension can include: real-time weather (sunny, rainy, foggy, snowy), visibility, road surface moisture, lighting conditions (daytime, dusk, nighttime), etc. Results under the traffic flow state dimension can be determined by the embedding vectors corresponding to road image information, radar point cloud data, and floating car GPS trajectories. For example, results under the traffic flow state dimension can include: average vehicle speed, traffic density level (low / medium / high), etc.

[0058] The second unit is used to determine the current congestion stage based on the embedding vectors corresponding to all modalities. The event stage identifier is used to characterize the current congestion stage and guides the dynamic adjustment of modality depth. The latency period of traffic conditions can be the stage where traffic flow begins to gradually increase but has not yet reached the level of causing obvious congestion; signs and factors of congestion are quietly accumulating. The development period can be the stage where, as traffic flow continues to increase, potential congestion factors accumulate and amplify, vehicle speeds on the road decrease significantly, and traffic congestion begins to gradually appear and intensify. The peak period can be the stage where congestion develops to its most severe stage, at which point traffic flow reaches or exceeds the road's capacity limit, vehicles on the road are almost unable to move normally, and the congestion reaches its peak. The dissipation period can be the stage where, as the factors causing congestion gradually disappear or ease, traffic flow begins to decrease, vehicle speeds on the road gradually recover, and the congestion gradually lessens until it disappears completely.

[0059] The third unit is used to conditionally fuse structured traffic context vectors, event stage identifiers, and embedding vectors to generate context-aware enhanced feature vectors.

[0060] Specifically, the embedding vectors corresponding to all modalities are processed using the multilayer perceptron structure and / or lightweight Transformer encoder in the first unit of the deep labeling module to obtain a structured traffic context vector. The second unit of the deep labeling module processes the embedding vectors corresponding to all modalities to determine the event stage identifier. If the embedding vector is related to temporal information, trend features can be extracted using a lightweight Long Short-Term Memory (LSTM) network or a Temporal Multi-Layer Perceptron (Temporal MLP) in the second unit. If the embedding vector is the embedding vector corresponding to a single frame of the road image information, a Markov state transition judgment is performed by combining the output from the previous time step. The classifier structure of the second unit can be a multilayer perceptron (MLP), a convolutional neural network (CNN), or a small Transformer to output a four-class probability distribution, and the current event stage identifier is determined based on this four-class probability distribution.

[0061] The third unit of the deep labeling module performs conditional fusion of structured traffic context vectors, event stage identifiers, and embedding vectors to generate context-aware enhanced feature vectors.

[0062] Optionally, in conjunction with the above, the following description is provided, using the depth labeling module as the Traffic-Aware Depth Evaluator (TADE) module, the first unit as the traffic context vector extraction unit, the second unit as the event evolution stage identification unit, and the third unit as the modality-specific feature enhancement unit.

[0063] In real-world traffic environments, not all high-dimensional modal data always requires deep processing by the target model. For example, the semantic value of a road image showing "slow-moving traffic" is lower than that of a "sudden braking" event; the text "weather information: light rain" has little impact during the day, but can significantly increase risk at night and on curved roads; floating car GPS trajectories have warning value during the "latency period," but tend to stabilize during "peak hours," resulting in decreased information density. Therefore, the TADE module can be used to determine the model processing depth value corresponding to the embedding vector. In this process, the enhanced feature vector can be determined first using the TADE module, and then the model processing depth value can be determined based on the enhanced feature vector. Specifically, The embedding vectors corresponding to all modes are extracted using the traffic context vector extraction unit. Processing is performed to initially extract shallow features from different modalities and determine the structured traffic context vector. It should be noted that since this unit is only used to initially extract the traffic context vectors corresponding to the embedding vectors of the modalities, it can use a pre-trained model and is not required to perform co-optimization with the backbone network.

[0064] The event evolution stage identification unit identifies the embedding vectors corresponding to all modalities. The system processes and integrates shallow features from different modal temporal dynamic patterns to identify the current event stage, determine whether it is in the "latent period," "development period," "peak period," or "dissipation period" of congestion, and outputs an event stage identifier. ,in, .

[0065] It should be noted that since this unit is only used to fuse shallow features of different modal temporal dynamic modes, it can use a pre-trained model and is not required to perform co-optimization with the backbone network.

[0066] Based on the modality-specific feature enhancement unit, the embedding vectors corresponding to all modalities are... Structured traffic context vectors and event phase markers Perform conditional fusion processing to generate context-aware enhanced feature vectors. .

[0067] Optionally, for the embedding vector Structured traffic context vectors and event phase markers One way to perform conditional fusion processing is to apply a linear transformation matrix to the embedded vector. Structured traffic context vectors and event phase markers The feature vectors are then concatenated and fused to obtain the enhanced feature vectors, as shown in the function below: ; in, It is a linear transformation matrix (i.e., a fully connected layer or weight matrix), which can map the concatenated high-dimensional vector back to the appropriate feature dimension and fuse the semantics of different modalities to obtain an enhanced feature vector.

[0068] Optionally, for the embedding vector Structured traffic context vectors and event phase markers One way to perform conditional fusion is to use a sigmoid gate on the embedded vector. Structured traffic context vectors and event phase markers Gated fusion is performed, and the intensity of context information injection is controlled to obtain enhanced feature vectors.

[0069] Optionally, for the embedding vector Structured traffic context vectors and event phase markers Another way to perform conditional fusion processing is to use a small network, such as an MLP, to process the structured traffic context vector. Event phase identifiers Process and determine the scaling parameters. and offset parameters The embedded vector is normalized according to the scaling and offset parameters to obtain the enhanced feature vector. The scaling parameter... and offset parameters It can be determined using the following function: ; ; Using scaling parameters and offset parameters For embedded vectors Standard normalization can be achieved using the following function: ; in, This represents element-wise multiplication. Let be the mean of the embedding vectors corresponding to all modalities. This represents the standard deviation of the embedding vectors corresponding to all modes. This indicates a preset constant.

[0070] S240. The dynamic depth evaluation unit in the depth labeling module processes the enhanced feature vectors of all modes to obtain the propagation depth value on the backbone network where each mode is located.

[0071] The dynamic depth evaluation unit evaluates the augmented feature vectors corresponding to all modalities to determine the propagation depth value of the corresponding modality. The propagation depth value can be understood as the number of layers in the backbone network of the target model where the embedded vector propagates or is processed. Optionally, the dynamic depth evaluation unit may include a lightweight neural network or a small Transformer structure.

[0072] Specifically, the enhanced feature vectors of all modalities are nonlinearly transformed by the dynamic depth evaluation unit in the depth labeling module to extract intermediate vectors for determining information density. Based on the intermediate vectors corresponding to all modalities, a continuous information density evaluation attribute is generated. The information density evaluation attribute is then mapped to obtain the propagation depth value.

[0073] Optionally, in conjunction with the above, a learnable dynamic depth evaluation mechanism based on dynamic depth evaluation units can be developed. It is a learnable nonlinear mapping function that automatically optimizes the propagation depth value through end-to-end training. This reduces reliance on manual priors. It uses enhanced feature vectors corresponding to the modality. As input, output the integer propagation depth value corresponding to this mode. , It is a positive integer, and its value range is 1. ,in, This indicates the total number of modules in the backbone network.

[0074] The above process can be represented as: ; in, It represents the set of optimizable parameters for a dynamic deep evaluation unit, which shares the optimization objective with the subsequent target model and supports end-to-end joint training.

[0075] Optionally, based on dynamic depth evaluation units The specific method for processing the enhanced feature vectors of all modalities can be: The enhanced feature vector is processed by the feature transformation subunit in the dynamic depth evaluation unit. A nonlinear transformation is performed to extract an intermediate vector used to determine information density. The intermediate vector is evaluated through the scoring sub-unit in the dynamic depth evaluation unit. Processing is performed to generate a continuous information density score. The information density score is used to characterize the semantic complexity of the corresponding modality. It should be noted that the information density score corresponds to the information density evaluation attribute mentioned above. The information density score is evaluated through the depth mapping subunit in the dynamic depth evaluation unit. Perform mapping processing to determine integer training depth labels. This refers to the propagation depth value mentioned above.

[0076] It should be noted that the parameters of the aforementioned dynamic depth evaluation unit and the subsequent target model parameters can be optimized based on the overall loss function, supporting end-to-end training. The dynamic depth evaluation unit can be implemented using any lightweight neural network architecture. Preferably, the dynamic depth evaluation unit can be a multilayer perceptron (MLP) or a small Transformer structure.

[0077] Optionally, the input enhanced feature vector can be processed by the feature transformation subunit in the dynamic depth evaluation unit. A nonlinear transformation is performed to extract an intermediate vector used to determine information density. The specific method is as follows: enhance the feature vector based on one or more fully connected layers in the feature transformation subunit. A linear transformation is performed, and a nonlinear activation function is appended after each fully connected layer. This can be one or more of ReLU, GELU, Swish, Sigmoid, Tanh, or their variants. The linear transformation result is then processed based on at least one of these nonlinear activation functions to obtain the nonlinearly transformed result. The nonlinearly transformed result is then normalized to obtain an intermediate vector. The normalization process can be either layer normalization or batch normalization.

[0078] Optionally, the intermediate vector can be evaluated using the scoring sub-units within the dynamic depth evaluation unit. Processing is performed to generate a continuous information density score. The specific approach could be: based on the intermediate vector of the linear layer pair in the scoring subunit. The raw score is then processed to output the final score. Based on a monotonic nonlinear function (e.g., Sigmoid, Softmax), the raw score is normalized to a preset data interval to obtain the normalized result. The preset data interval can be [0,1]. The normalized result is then processed using an optimizable threshold function to generate a continuous representation before discretization, yielding the information density score. .

[0079] It should be noted that the feature transformation subunit and scoring subunit in the dynamic depth evaluation unit can be integrated into a single network or implemented step by step.

[0080] Optionally, the information density score is obtained by using the depth mapping subunit in the dynamic depth evaluation unit. Perform mapping processing to determine the propagation depth value. The specific method could be: scoring the information density through linear or nonlinear mapping and quantization functions. Processing is performed to generate propagation depth values. ,Right now: ; in, For optimizable parameters in linear or nonlinear mappings, This is the quantization function, including rounding down, rounding up, or rounding to the nearest integer. Or, Information density score using a piecewise linear function Interval partitioning is performed to obtain the propagation depth value corresponding to the information density score. For example, the [0,1] interval is divided into... Each sub-interval corresponds to a depth value. Or, Information density score using a learnable lookup table A nonlinear mapping is performed to obtain the propagation depth value corresponding to the information density score. Or, Information density score is obtained through a classification network. The process involves determining the probability distribution information related to the propagation depth value, and then processing this probability distribution information based on a preset function (argmax) to obtain the propagation depth value. .

[0081] It should be noted that the mapping function in the aforementioned deep mapping subunit can achieve gradient approximation backpropagation through a pass-through estimator during the training phase, ensuring end-to-end trainability.

[0082] The above describes the dynamic depth evaluation unit. All optimizable parameters (including but not limited to transformation weights, biases, and mapping coefficients) It shares the loss function with the subsequent target model and performs joint optimization through backpropagation.

[0083] S250, Use the propagation depth value as the model processing depth value.

[0084] Specifically, the propagation depth value determined by the dynamic depth evaluation unit of the depth labeling module is used as the model processing depth value, and the number of layers for propagation of the embedding vector corresponding to each modality in the target model is controlled based on the model processing depth value.

[0085] S260. The embedding vector is processed based on the backbone network in the target model and the model processing depth value corresponding to each modality. When the processing depth value is detected to have reached the model processing depth value, the feature processing of the modality is exited.

[0086] The backbone network can be understood as a multi-layered network with a modality training exit mechanism. The backbone network can consist of... A sequentially connected processing layer Composed of. Each processing layer It can implement the same nonlinear transformation function, and its structure is adaptable to various mainstream neural network paradigms. Optional processing layers... It can include at least: a Transformer block, a convolutional block, and a recurrent layer. The Transformer block contains multi-head self-attention (MSA), a feedforward network (FFN), layer normalization (LN), and residual connections. The convolutional block can be a stacked structure consisting of convolutional layers, normalization layers, and activation functions. The recurrent layer can be a single-layer structure of a Long Short-Term Memory (LSTM) network or a gated recurrent unit (GRU).

[0087] Specifically, based on the backbone network consisting of N processing layers in the target model, the embedding vector corresponding to each mode is processed according to the model processing depth value corresponding to each mode. When the processing depth value of the embedding vector is detected to reach the model processing depth value, the feature processing of the embedding vector of that mode is automatically terminated, and the feature processing result corresponding to that mode is obtained.

[0088] Optionally, all modalities share the backbone network parameters to achieve high efficiency and structural uniformity in network parameters. For the embedding vector corresponding to each modality... The embedding vectors corresponding to each mode As the first processing layer of the independent input parallel input backbone network, subsequent processing layers only perform transformation operations on the current "active mode".

[0089] Because the backbone network uses a sequential execution architecture, the current layer number is... It is a deterministic program state variable, when the embedding vector is input to the first... The processing depth value can be naturally obtained when processing a single layer, without the need to introduce additional layer index embeddings or learnable position encodings.

[0090] In the Each processing layer processes the embedded vector. After performing the forward computation, the following modal exit judgment can be executed: For each currently active mode ,like If the processing depth value is less than the model processing depth value, it means that the modality has not yet completed the predetermined feature extraction. In this case, the output corresponding to the modality can be... As the first The input to each processing layer.

[0091] like If the processing depth value reaches the model processing depth value, it means that the modality has completed the predetermined feature extraction process, triggering the exit mechanism and displaying its final output feature processing result. As input to the connection layer or modal feature fusion layer, it is used for fusion processing in the connection layer or modal feature fusion layer.

[0092] Since information from different modalities exists in the form of feature vectors, the separation of different modal information can be achieved simply through matrix segmentation. The processed modal information (feature processing results) is sent to the connection layer or modal feature fusion layer for temporary storage. The fusion of multimodal information is only performed in the connection layer or modal feature fusion layer at the end of the backbone network after feature extraction of all modalities has been completed.

[0093] S270. Based on the feature processing results of all modalities, determine the fusion vector.

[0094] The fusion vector can be the feature vector obtained by fusing all feature processing results. The number of feature processing results is consistent with the number of modes.

[0095] Specifically, the feature processing results corresponding to each modality are fused to obtain a fusion vector. Optionally, the fusion processing method can be one or more of the following: concatenation, attention weighting, lightweight neural networks, etc.

[0096] Optionally, in conjunction with the above, the feature processing results corresponding to all early-exiting modalities and all feature processing results not early-exiting are fused in the connection layer or modality feature fusion layer at the end of the backbone network to obtain a fused vector. .

[0097] S280. Collaborative modeling of fusion vectors is performed using a pre-determined fusion strategy to determine the correlation information between any two modalities.

[0098] The pre-determined fusion strategy can be used to collaboratively model multimodal information in the fusion vector. The association information can be the association feature information between two modalities determined based on the feature processing results corresponding to the two modalities.

[0099] Specifically, the fusion vector is collaboratively modeled based on a pre-determined fusion strategy to determine the correlation information between any two modalities. Based on the correlation information between any two modalities, the fusion vector after mining the correlation information can be obtained.

[0100] S290. Based on the decision layer, the fusion vector after mining the associated information is processed to obtain the traffic congestion status.

[0101] Specifically, the decision layer of the target model processes the fused vector after mining the associated information to determine the traffic congestion status of the target area.

[0102] The technical solution of this embodiment achieves automatic acquisition of traffic information in unattended situations by obtaining multimodal data on traffic information within the target area, providing data support for subsequent determination of traffic congestion status. The multimodal data undergoes time-series delay compensation processing to obtain the output data, achieving time-series alignment of the multimodal data and ensuring consistency across the temporal dimension. A lightweight encoder corresponding to each modality processes the input data to obtain an embedding vector corresponding to each modality. The embedding vectors of all modalities are input into a depth labeling module, enabling the depth labeling module to output an enhanced feature vector corresponding to each modality. The dynamic depth evaluation unit in the depth labeling module processes the enhanced feature vectors of all modalities to obtain the propagation depth value on the backbone network of each modality. This propagation depth value is used as the model processing depth value, achieving optimal allocation of computational resources for the target model based on the model processing depth value. This ensures the overall performance of the target model while avoiding the high computational cost of full-depth processing of all modal data, effectively reducing the computational cost of the target model. The embedded vectors are processed based on the backbone network of the target model and the model processing depth value corresponding to each modality. Feature processing for a modality is terminated when the processing depth value reaches the model processing depth value. A fusion vector is determined based on the feature processing results of all modalities. The fusion vectors are collaboratively modeled using a pre-determined fusion strategy to determine the correlation information between any two modalities. The fusion vector, after mining the correlation information, is processed based on the decision layer to obtain the traffic congestion status. Differentiated allocation of computing power to different modal data is achieved through the model processing depth value. This ensures sufficient processing of high-information-density modal data while effectively reducing the risk of overfitting to low-information-density modal data, thus improving the robustness and generalization performance of the model. This invention solves the delays and misjudgments caused by human identification in existing technologies, significantly reducing operating costs. Furthermore, by using multimodal data processing, it fully utilizes the complementary characteristics of each modality, improving the robustness and reliability of congestion status identification, and is particularly suitable for traffic status perception under complex urban road conditions and extreme weather conditions. Furthermore, by processing the embedding vectors using the target model and model processing depth values, multimodal deep fusion at the feature level is achieved, avoiding the potential risks of "text-centered bias" and "large model illusion," thus improving the reliability and accuracy of congestion state recognition. In addition, by determining the model processing depth value, each modal feature exits the computation early after reaching a preset depth, reducing computational requirements and the risk of overfitting, thereby improving the efficiency of congestion state recognition.

[0103] Example 3 Figure 3 This is a schematic diagram of a traffic congestion identification device for urban areas provided in Embodiment 3 of the present invention. Figure 3As shown, the device includes: a data acquisition module 310, an embedding vector determination module 320, a depth value determination module 330, and a congestion status determination module 340.

[0104] The data acquisition module 310 is used to acquire multimodal data about traffic information within a target area; wherein the multimodal data includes at least two of the following: road image information collected by a roadside camera device, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information; the embedding vector determination module 320 is used to perform time delay compensation processing on the multimodal data to obtain input data, and to process the input data based on the lightweight encoder corresponding to each modality to determine the embedding vector corresponding to each modality; the depth value determination module 330 is used to determine the model processing depth value corresponding to each modality based on the embedding vectors corresponding to all modalities; wherein the model processing depth value is used to characterize the number of processing modules of the backbone network for the embedding vector; the congestion state determination module 340 is used to process the embedding vector using the target model and the model processing depth value to obtain a fusion vector, and to process the fusion vector based on the decision layer of the target model to obtain the traffic congestion state.

[0105] The technical solution of this embodiment achieves automatic acquisition of traffic information in unattended situations by obtaining multimodal data on traffic information within the target area, providing data support for subsequent determination of traffic congestion status. Temporal delay compensation processing is performed on the multimodal data to obtain the output data, achieving temporal alignment of the multimodal data and ensuring consistency in the temporal dimension. A lightweight encoder corresponding to each modality processes the input data to obtain an embedding vector corresponding to each modality. Based on the embedding vectors corresponding to all modalities, the model processing depth value corresponding to each modality is determined. This allows for optimal allocation of computational resources to the target model based on the model processing depth value, ensuring the overall performance of the target model while avoiding the high computational cost of full-depth processing of all modal data, effectively reducing the computational cost of the target model. This invention utilizes a target model and model processing depth values ​​to process embedded vectors, obtaining a fusion vector. The fusion vector is then processed based on the decision layer of the target model to determine traffic congestion status. By using model processing depth values, computational power is allocated differently to different modalities, ensuring sufficient processing of high-information-density modal data while effectively reducing the risk of overfitting to low-information-density modal data, thus improving the model's robustness and generalization performance. This invention solves the delays and misjudgments caused by human identification in existing technologies, significantly reducing operating costs. Furthermore, by using multimodal data processing, it fully utilizes the complementary characteristics of each modality, improving the robustness and reliability of congestion status identification, especially suitable for traffic status perception under complex urban road conditions and extreme weather conditions. Moreover, by using the target model and model processing depth values ​​to process embedded vectors, multimodal deep fusion at the feature level is achieved, avoiding the potential risks of "text-centric bias" and "large model illusion," thus improving the credibility and accuracy of congestion status identification. In addition, by determining the model processing depth value, each modal feature can exit the calculation in advance after reaching the preset depth, reducing the computing power requirement, reducing the risk of overfitting, and improving the efficiency of congestion state recognition.

[0106] Based on the above embodiments, optionally, the embedded vector determination module includes: a data to be input determination unit, used to acquire the hardware timestamps of each sensor and record the time delay values ​​corresponding to different sensors; using the acquisition time information corresponding to the target sensor that meets preset conditions as a reference time, and performing time alignment processing on the multimodal data according to the reference reference time and the time delay values; for any modal data, if the time difference between any two adjacent time points in the modal data is greater than a preset difference threshold, then interpolation is performed based on the modal data of the two time points and the interpolation quantity to obtain the data to be input; wherein, the interpolation quantity is related to the time difference; wherein, the target sensor corresponds to a roadside camera device.

[0107] Optionally, the embedding vector determination module includes: a data encoding unit, used to retrieve a pre-set lightweight encoder corresponding to each modality data; and to process the input data of the corresponding modality based on the optimizable parameters in the lightweight encoder to obtain the embedding vector of the modality.

[0108] Optionally, the depth value determination module includes: an enhanced feature vector determination unit, used to input the embedding vectors of all modalities into the depth labeling module, so that the depth labeling module outputs an enhanced feature vector corresponding to each modality; a propagation depth value determination unit, used to process the enhanced feature vectors of all modalities based on the dynamic depth evaluation unit in the depth labeling module, to obtain the propagation depth value on the backbone network where each modality is located; and a model processing depth value determination unit, used to use the propagation depth value as the model processing depth value.

[0109] Optionally, an enhanced feature vector determination unit is used to process the embedded vector based on the first unit in the deep labeling module to output a structured traffic context vector; wherein the structured traffic context vector includes at least the results under the spatial attribute dimension, the results under the temporal attribute dimension, the results under the environmental state dimension, and the results under the traffic flow state dimension; the second unit in the deep labeling module processes the embedded vector to obtain an event stage identifier; wherein the event stage identifier includes at least any one of the traffic state's latency period, development period, peak period, and dissipation period; the third unit in the deep labeling module processes the structured traffic context vector, the event stage identifier, and the embedded vector to obtain the enhanced feature vector.

[0110] Optionally, the congestion state determination module includes: a fusion vector determination unit, used to process the embedding vector based on the backbone network in the target model and the model processing depth value corresponding to each mode, and to exit the feature processing of the mode when the processing depth value is detected to reach the model processing depth value; and to determine the fusion vector based on the feature processing results of all modes.

[0111] Optionally, the congestion state determination module includes: a fusion vector processing unit, used to collaboratively model the fusion vector through a pre-determined fusion strategy to determine the correlation information between any two modalities; and to process the fusion vector after mining the correlation information based on the decision layer to obtain the traffic congestion state.

[0112] Optionally, the device further includes: a scheduling strategy determination module, used to retrieve a target map based on traffic congestion information corresponding to the traffic congestion state when the traffic congestion state meets preset congestion conditions, so as to generate a target strategy corresponding to the traffic congestion information; wherein the target strategy includes at least one or more of the following: extending the green light duration, activating the emergency lane, and guiding detour routes.

[0113] The urban traffic congestion identification device provided in the embodiments of the present invention can execute the urban traffic congestion identification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0114] Example 4 Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0115] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0116] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0117] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for urban traffic congestion identification.

[0118] In some embodiments, the urban traffic congestion identification method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the urban traffic congestion identification method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the urban traffic congestion identification method by any other suitable means (e.g., by means of firmware).

[0119] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs for implementing the urban traffic congestion identification method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0121] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0122] Example 5 Embodiment 5 of the present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a method for identifying urban traffic congestion, the method comprising: Acquire multimodal data on traffic information within a target area; wherein the multimodal data includes at least two of the following: road image information collected by roadside cameras, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information; perform time delay compensation processing on the multimodal data to obtain input data, and process the input data based on the lightweight encoder corresponding to each modality to determine the embedding vector corresponding to each modality; determine the model processing depth value corresponding to each modality based on the embedding vectors corresponding to all modalities; wherein the model processing depth value is used to characterize the number of processing layers of the backbone network for the embedding vector; process the embedding vector using the target model and the model processing depth value to obtain a fusion vector, and process the fusion vector based on the decision layer of the target model to obtain the traffic congestion status.

[0123] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0124] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0125] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0126] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0127] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0128] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for identifying urban traffic congestion, characterized in that, include: Acquire multimodal data on traffic information within the target area; wherein, the multimodal data includes at least two of the following: road image information collected by roadside camera devices, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information; The multimodal data is subjected to time delay compensation to obtain the input data, and the input data is processed based on the lightweight encoder corresponding to each modality to determine the embedding vector corresponding to each modality. Based on the embedding vectors corresponding to all modalities, determine the model processing depth value corresponding to each modal data; wherein, the model processing depth value is used to characterize the number of processing layers of the backbone network for the embedding vector; The embedded vector is processed using the target model and the model processing depth value to obtain a fused vector. The fused vector is then processed based on the decision layer of the target model to obtain the traffic congestion status.

2. The method according to claim 1, characterized in that, The step of performing time delay compensation processing on the multimodal data to obtain the input data includes: Obtain the hardware timestamps of each sensor and record the time delay values ​​corresponding to different sensors; The acquisition time information corresponding to the target sensor that meets the preset conditions is used as the reference time, and the multimodal data is time-aligned according to the reference time and the time delay value. For any modal data, if the time difference between any two adjacent time points in the modal data is greater than a preset difference threshold, then interpolation is performed based on the modal data at the two time points and the number of interpolations to obtain the input data; wherein, the number of interpolations is related to the time difference. Among them, the target sensor corresponds to the roadside camera device.

3. The method according to claim 1, characterized in that, The process of processing the input data based on the lightweight encoder corresponding to each modality data to determine the embedding vector corresponding to each modality includes: Retrieve the pre-set lightweight encoder corresponding to each modal data; The input data for the corresponding mode is processed based on the optimizable parameters in the lightweight encoder to obtain the embedding vector of the mode.

4. The method according to claim 1, characterized in that, The step of determining the model processing depth value corresponding to each modality data based on the embedding vectors corresponding to all modalities includes: The embedding vectors of all modalities are input into the depth labeling module so that the depth labeling module outputs an enhanced feature vector corresponding to each modality; The enhanced feature vectors of all modes are processed by the dynamic depth evaluation unit in the depth labeling module to obtain the propagation depth value on the backbone network where each mode is located. The propagation depth value is used as the model processing depth value.

5. The method according to claim 4, characterized in that, The step of inputting the embedding vectors of all modalities into the depth labeling module, so that the depth labeling module outputs an enhanced feature vector corresponding to each modality, includes: The embedded vector is processed based on the first unit in the deep labeling module to output a structured traffic context vector; wherein the structured traffic context vector includes at least the results under the spatial attribute dimension, the results under the temporal attribute dimension, the results under the environmental state dimension, and the results under the traffic flow state dimension. The embedded vector is processed by the second unit in the depth labeling module to obtain an event stage identifier; wherein the event stage identifier includes at least one of the following: the incubation period, the development period, the peak period, and the dissipation period of the traffic state; The enhanced feature vector is obtained by processing the structured traffic context vector, the event stage identifier, and the embedding vector based on the third unit in the deep labeling module.

6. The method according to claim 1, characterized in that, The process of using the target model and the model's depth values ​​to process the embedding vector to obtain the fused vector includes: The embedding vector is processed based on the backbone network in the target model and the model processing depth value corresponding to each modality, and the feature processing of the modality is exited when the processing depth value is detected to reach the model processing depth value. The fusion vector is determined based on the feature processing results of all modalities.

7. The method according to claim 1, characterized in that, The decision layer based on the target model processes the fused vector to obtain the traffic congestion status, including: The fusion vector is collaboratively modeled using a pre-determined fusion strategy to determine the association information between any two modalities; The traffic congestion status is obtained by processing the fusion vector after mining the associated information based on the decision layer.

8. The method according to claim 1, characterized in that, When the traffic congestion condition meets preset congestion conditions, the method further includes: The target map is retrieved based on the traffic congestion information corresponding to the traffic congestion status, so as to generate a target strategy corresponding to the traffic congestion information; The target strategy includes at least one or more of the following: extending the green light duration, activating the emergency lane, and guiding detour routes.

9. A device for identifying urban traffic congestion, characterized in that, include: The data acquisition module is used to acquire multimodal data about traffic information within the target area; wherein, the multimodal data includes at least two of the following: road image information collected by roadside camera devices, radar point cloud data corresponding to at least one traffic participation element, floating car GPS trajectory, and weather information; The embedding vector determination module is used to perform time delay compensation processing on the multimodal data to obtain the input data, and to process the input data based on the lightweight encoder corresponding to each modality to determine the embedding vector corresponding to each modality. The depth value determination module is used to determine the model processing depth value corresponding to each modality data based on the embedding vectors corresponding to all modalities; wherein, the model processing depth value is used to characterize the number of processing layers of the backbone network for the embedding vectors; The congestion state determination module is used to process the embedded vector using the target model and the model processing depth value to obtain a fused vector, and then process the fused vector based on the decision layer of the target model to obtain the traffic congestion state.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the urban traffic congestion identification method according to any one of claims 1-8.

Images