Traffic interaction method, device and storage medium

Abstract

The present application discloses a traffic interaction method, device and storage medium for use in the field of intelligent traffic technology. The present application method includes: constructing an initial traffic interaction model and defining initial urban road condition data; processing the initial urban road condition data through preprocessing, multimodal Transformer model and knowledge graph embedding vector, and calculating the traffic flow prediction value based on the processing result; constructing a traffic environment graph structure based on the traffic flow prediction value and the signal light position data; initializing the signal light control time and constructing an evaluation module; obtaining the signal light control instruction based on the traffic flow prediction value and the initialization signal light control time; calculating the loss value after the signal light control instruction interacts with the traffic environment graph structure through the evaluation module; optimizing the initialization traffic interaction model through the loss value to generate a target traffic interaction model; inputting the acquired urban road condition data into the target traffic interaction model to obtain the target signal light control instruction.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a traffic interaction method, device and storage medium. Background Art

[0002] With the acceleration of urbanization, urban traffic problems have become increasingly prominent. Problems such as traffic congestion and frequent traffic accidents have become important factors restricting urban development.

[0003] To meet these challenges, existing technologies have adopted a traffic information collection technology, which solves the problems of urban traffic congestion and frequent traffic accidents by using induction coils to detect vehicle passing conditions, video surveillance to identify road congestion and vehicle behavior, GPS floating vehicles to collect vehicle speed and location information, and mobile Internet crowdsourcing to obtain user travel data.

[0004] Although traffic information collection technology has improved the management level of urban traffic to a certain extent, it often relies on manual settings and fixed rules, lacks the ability to automatically adjust and optimize traffic light control strategies according to real-time road conditions, and is difficult to adapt to the dynamic changes of urban traffic. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a traffic interaction method, device and storage medium.

[0006] The technical solution provided in this application is described below:

[0007] The first aspect of the present application provides a traffic interaction method, the method comprising:

[0008] Construct an initial traffic interaction model;

[0009] Define initial city traffic data;

[0010] Processing the initial urban traffic condition data to obtain multimodal traffic condition data;

[0011] Processing the multimodal road condition data using a preconfigured multimodal Transformer model to obtain a target multimodal feature representation;

[0012] Obtaining a multimodal and knowledge graph feature vector based on the target multimodal feature representation and a preconfigured knowledge graph embedding vector;

[0013] Processing the multimodal and knowledge graph feature vectors to obtain traffic flow prediction values;

[0014] Constructing a traffic environment graph structure according to the traffic flow prediction value and the signal light location data;

[0015] Initialize the control time of the signal light and build the evaluation module;

[0016] generating a signal light control instruction according to the traffic flow prediction value and the control time;

[0017] Interacting with the traffic environment graph structure according to the signal light control instruction to obtain an interaction result;

[0018] Calculating the interaction result according to the evaluation module to obtain a loss value;

[0019] Optimizing the control time according to the loss value to obtain an optimization result;

[0020] Training the traffic interaction model according to the optimization result to generate a target traffic interaction model;

[0021] The acquired urban road condition data is input into the target traffic interaction model for processing to obtain the target traffic light control instruction.

[0022] Optionally, the using a preconfigured multimodal Transformer model to process the multimodal road condition data to obtain a target multimodal feature representation includes:

[0023] Build and initialize a multimodal Transformer model;

[0024] Processing the multimodal road condition data using the initialized multimodal Transformer model to obtain a multimodal feature representation;

[0025] Calculating the difference between the multimodal feature representation and a preset multimodal feature representation using a preconfigured cross entropy loss function to obtain difference data;

[0026] Optimizing the initialized multimodal Transformer model based on the difference data to obtain a target multimodal Transformer model;

[0027] The multimodal road condition data is input into the target multimodal Transformer model for processing to obtain a target multimodal feature representation.

[0028] Optionally, the using the initialized multimodal Transformer model to process the multimodal road condition data to obtain a multimodal feature representation includes:

[0029] Processing the multimodal road condition data to obtain multimodal data feature codes;

[0030] The multimodal data feature encoding is input into the initialized multimodal Transformer model, so that the initialized multimodal Transformer model calculates the multimodal road condition data feature encoding according to a preconfigured attention mechanism to obtain a multimodal feature representation.

[0031] Optionally, obtaining a multimodal and knowledge graph feature vector based on the target multimodal feature representation and a preconfigured knowledge graph embedding vector includes:

[0032] Configure the traffic knowledge graph;

[0033] Processing the traffic knowledge graph to obtain a knowledge graph embedding vector;

[0034] The target multimodal feature representation and the knowledge graph embedding vector are processed to obtain multimodal and knowledge graph feature vectors.

[0035] Optionally, the initialized multimodal Transformer model calculates the feature encoding of the multimodal road condition data according to a preconfigured attention mechanism to obtain a multimodal feature representation, including:

[0036] Initialize the multimodal Transformer model to calculate the feature encoding of the multimodal road condition data according to a preconfigured attention mechanism to obtain a multimodal feature representation;

[0037] The preconfigured attention mechanism is calculated by the first formula:

[0038] ;

[0039] in, represents the calculated multimodal feature representation; Q represents the query vector in the multimodal road condition data feature encoding; K is the key vector in the multimodal road condition data feature encoding; V is the value vector in the multimodal road condition data feature encoding; Represents the similarity between each query vector and each key vector; Represents the scaling factor, used to maintain stability between values; represents the probability distribution matrix; Represents the weighted multimodal feature representation.

[0040] Optionally, the using a preconfigured cross entropy loss function to calculate the difference between the multimodal feature representation and a preset multimodal feature representation to obtain difference data includes:

[0041] Calculating the difference between the multimodal feature representation and a preset multimodal feature representation using a preconfigured cross entropy loss function to obtain difference data;

[0042] The preconfigured cross entropy loss function is calculated by the second formula:

[0043] ;

[0044] Wherein, Loss represents the difference data between the multimodal feature representation and the preset multimodal feature representation; N represents the total number of samples; Indicates the accumulation of difference data from the 1st sample to the Nth sample; Represents the preset multimodal feature representation of the i-th sample; represents the multimodal feature representation of the i-th sample calculated by the attention mechanism; and Represents the difference data of the current sample in the positive and negative categories.

[0045] Optionally, the optimizing the initialized multimodal Transformer model based on the difference data to obtain a target multimodal Transformer model includes:

[0046] Obtaining parameter values ​​of an optimized multimodal Transformer model based on the difference data;

[0047] The parameter values ​​of the optimized multimodal Transformer model are calculated by the third formula:

[0048] ;

[0049] in, Represents the parameter value of the optimized multimodal Transformer model; Represents the parameter value of the initialized multimodal Transformer model before optimization; Representation parameters Update the scaling factor to control the parameters of the multimodal Transformer model Update step size; Represents the parameters of the difference data loss relative to the multimodal Transformer model The gradient of To minimize discrepancy data;

[0050] The initialized multimodal Transformer model is optimized according to the parameter values ​​of the optimized multimodal Transformer model to obtain a target multimodal Transformer model.

[0051] A second aspect of the present application provides a traffic interaction device, the device comprising:

[0052] The first construction unit is used to construct an initial traffic interaction model;

[0053] A first setting unit, used to define initial city traffic data;

[0054] A first processing unit, configured to process the initial urban traffic condition data to obtain multimodal traffic condition data;

[0055] A second processing unit, configured to process the multimodal road condition data using a preconfigured multimodal Transformer model to obtain a target multimodal feature representation;

[0056] A third processing unit, configured to obtain a multimodal and knowledge graph feature vector based on the target multimodal feature representation and a preconfigured knowledge graph embedding vector;

[0057] A fourth processing unit, used for processing the multi-modal and knowledge graph feature vectors to obtain a traffic flow prediction value;

[0058] A second construction unit, configured to construct a traffic environment graph structure according to the traffic flow prediction value and the signal light position data;

[0059] A second setting unit is used to initialize the control time of the signal light and build an evaluation module;

[0060] a fifth processing unit, configured to generate a signal light control instruction according to the traffic flow prediction value and the control time;

[0061] a sixth processing unit, configured to interact with the traffic environment graph structure according to the signal light control instruction to obtain an interaction result;

[0062] a seventh processing unit, configured to calculate the interaction result according to the evaluation module to obtain a loss value;

[0063] an eighth processing unit, configured to optimize the control time according to the loss value to obtain an optimization result;

[0064] A training unit, used for training the traffic interaction model according to the optimization result to generate a target traffic interaction model;

[0065] The ninth processing unit is used to input the acquired urban road condition data into the target traffic interaction model for processing to obtain the target traffic light control instruction.

[0066] A third aspect of the present application provides a traffic interaction device, the device comprising:

[0067] Processor, memory, input-output unit, and bus;

[0068] The processor is connected to the memory, the input and output unit, and the bus;

[0069] The memory stores a program, and the processor calls the program to execute the first aspect and any optional method in the first aspect.

[0070] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional method in the first aspect.

[0071] It can be seen from the above technical solutions that this application has the following advantages:

[0072] 1. This application lays the foundation for subsequent model training by constructing an initial traffic interaction model and defining initial urban road condition data. Then, by using a multimodal Transformer model to process the initial urban road condition data, the city's road condition characteristics are effectively extracted, laying the foundation for subsequent accurate traffic flow prediction values, and enhancing the traffic interaction model's ability to understand complex traffic scenarios. Then, through the pre-configured knowledge graph embedding vector, the traffic field knowledge is effectively combined with the actual road condition data. Furthermore, the acquired multimodal and knowledge graph feature vectors are processed to obtain traffic flow prediction values, so that the traffic interaction model can better adapt to different traffic scenarios and complex road conditions.

[0073] 2. By constructing a traffic environment graph structure, the traffic flow prediction value is closely combined with the signal light position, achieving an accurate description of the traffic environment. Then, on this basis, the signal light control time is initialized, and an evaluation module is constructed to provide a scientific basis for the generation and optimization of signal light control instructions. Furthermore, through the real-time interaction between the signal light control instructions and the traffic environment graph structure, and the precise calculation of the interaction results by the evaluation module, the traffic interaction model is continuously optimized and improved to obtain a target traffic interaction model that can intelligently generate target signal light control instructions based on real-time urban road condition data, thereby improving the automation level of urban traffic management, providing strong data support for urban traffic planning and management, and promoting the development of smart transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0075] Figure 1A flow chart of an embodiment of the traffic interaction method provided by the present application;

[0076] Figure 2 A flow chart of an embodiment of obtaining a target multimodal feature representation in the traffic interaction method provided by the present application;

[0077] Figure 3 A flow chart of an embodiment of obtaining multimodal and knowledge graph feature vectors in the traffic interaction method provided in this application;

[0078] Figure 4 A flow chart of an embodiment of the traffic interaction device provided by the present application;

[0079] Figure 5 This is a flow chart of another embodiment of the traffic interaction device provided by the present application. DETAILED DESCRIPTION

[0080] It should be noted that the traffic interaction method provided by the present application can be applied to a terminal, a system, or a server. For example, the terminal can be a smart phone or a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, the present application uses the terminal as an example for explanation.

[0081] See also Figure 1 The present application first provides an embodiment of a traffic interaction method, which includes:

[0082] S101, constructing an initialization traffic interaction model;

[0083] In this embodiment, the traffic interaction model integrates multimodal data and knowledge graphs to achieve accurate perception of traffic conditions, accurate prediction of traffic flow, and optimized control of traffic lights. It can dynamically adjust traffic strategies based on real-time data to alleviate congestion, improve road traffic efficiency, reduce the risk of traffic accidents, and improve the operating efficiency and service quality of the entire transportation network.

[0084] When constructing the initialized traffic interaction model, we must first determine the model architecture based on the traffic management task, and then initialize the traffic interaction model parameters. We can assign initial values ​​to the traffic interaction model parameters according to a specific distribution to ensure that the initialized traffic interaction model can start training normally. Then, we set up a data processing module so that the initialized traffic interaction model can perform preprocessing operations such as cleaning and standardization on the collected data to ensure that the data quality and format meet the requirements of the initialized traffic interaction model. Furthermore, we clarify the input and output interfaces of the initialized traffic interaction model to prepare for the subsequent training and application of the initialized traffic interaction model.

[0085] S102, defining initial city traffic data;

[0086] In this embodiment, the urban road condition data mainly includes: real-time traffic flow, average speed and vehicle type distribution data on the road; visual related data such as the length of vehicle queues at intersections, the number of pedestrians and their movement trajectories; and geographic information data, such as the topological structure of the road network, the length, width and slope of each road section, etc.

[0087] When defining the initial urban road condition data, the initialization traffic interaction model integrates vehicle driving data such as real-time vehicle flow, average speed, vehicle type distribution, etc. collected from sensors such as geomagnetism and radar, visual data such as the length of vehicle queues at intersections, the number of pedestrians and movement trajectories captured by cameras, and geographic information data such as road network topology, section length, width and slope provided by the geographic information system. These multi-source data are then uniformly organized and associated according to specific data structure and format requirements, and the integrated data is further defined as the initial road condition data.

[0088] S103, processing the initial urban traffic condition data to obtain multimodal traffic condition data;

[0089] In this embodiment, the multimodal road condition data is a specific data form obtained after processing the urban road condition data, and is comprehensive data integrating visual modal data, sensor modal data, and geographic information modal data.

[0090] When processing the initial urban road condition data, the initial traffic interaction model will first perform image preprocessing such as cropping, scaling, and enhancement on the visual data in the urban road condition data, and then use the convolutional neural network to extract features to obtain visual modal data. At the same time, the acquired vehicle driving data will be cleaned of anomalies, filled with gaps, and normalized, and then the relevant features will be constructed through feature engineering to become sensor modal data. Subsequently, the geographic information data is converted into a graph structure and features are extracted with the help of a graph neural network to form geographic information modal data. Furthermore, the visual modal data, sensor modal data, and geographic information modal data are integrated to obtain multimodal road condition data.

[0091] S104, using a preconfigured multimodal Transformer model to process the multimodal road condition data to obtain a target multimodal feature representation;

[0092] In this embodiment, the multimodal Transformer model is mainly used as a model for processing multimodal traffic data. It is based on the Transformer architecture and uses the attention mechanism to process data of different modalities. By encoding and inputting modal data such as vision, sensor, and geographic information separately, the information is interactively fused internally, and the potential correlation between modalities is mined to provide comprehensive information for traffic analysis. The multimodal feature representation is a feature vector that integrates multimodal road condition data, including vehicle visual information, driving status, and geographic environment information, etc., reflecting the semantic correlation of each modality.

[0093] When processing multimodal road condition data, the multimodal road condition data is input into the trained multimodal Transformer model. For visual modal data, position encoding is performed based on the spatial distribution of its image features, so that the trained multimodal Transformer model can perceive the relative position relationship of traffic elements in the image, and then it is converted into feature encoding suitable for Transformer processing through the combination of convolutional layer and fully connected layer. For sensor modal data, time series encoding is performed according to its time series characteristics, highlighting the law of data change over time, and then mapped into feature encoding. For geographic information modal data, feature encoding of nodes and edges is performed based on the graph structure. The encoding process incorporates information such as the topological relationship of the road and converts it into feature encoding. Then, these feature encodings are input into the core encoder of the trained multimodal Transformer model. Furthermore, the self-attention mechanism in the encoder calculates the degree of correlation between different position elements from each part of the different feature encodings, such as the correlation between the vehicle position in the visual modality and the traffic flow data in the sensor modality. Finally, after being processed by the encoder, the data of each modality is continuously interactively fused, and the target multimodal feature representation covering vision, sensor, geographic information, etc. is finally output.

[0094] S105, obtaining a multimodal and knowledge graph feature vector based on the target multimodal feature representation and the preconfigured knowledge graph embedding vector;

[0095] In this embodiment, the knowledge graph embedding vector is the result of mapping the entities and relationships in the traffic knowledge graph to a low-dimensional vector space. For example, entities such as roads, intersections, vehicle types and the connection relationships between them are converted into vector form, so that in this vector space, entities and relationships represented by similar vectors have similar traffic semantics, which facilitates computer processing and understanding of traffic knowledge. Multimodal and knowledge graph feature vectors are vectors obtained by integrating multimodal feature representation and knowledge graph embedding vectors. It contains both real-time features of traffic scenes extracted from multimodal data such as vision, sensors, and geographic information, and integrates the semantic knowledge and logical relationships in the traffic field in the knowledge graph.

[0096] Before obtaining the multimodal and knowledge graph feature vectors, it is necessary to construct a knowledge graph in the traffic field, and then use knowledge extraction technology to obtain the entity and relationship information of the knowledge graph from multiple sources such as traffic regulations, historical traffic data, and screenshot field experience, and store it in the form of a graph database. Then, a graph embedding algorithm, such as the TransE algorithm, is used to map the entities and relationships in the knowledge graph into low-dimensional vector representations, so that the semantic information in the knowledge graph can be reflected in the vector space. For example, in the vector space, entities with similar traffic semantics, such as two parallel roads, have a close embedding vector distance. Furthermore, a fusion strategy, such as weighted summation or splicing operation, is used to fuse the multimodal feature vector with the corresponding knowledge graph entity embedding vector.

[0097] Among them, the ‌TransE algorithm‌ is an embedding representation algorithm for representing nodes and relationships in a graph structure, and is particularly suitable for representation learning of knowledge graphs. Its core idea is to transform the triples h, r, and t in the knowledge graph, where h is the head entity, r is the relationship, and t is the tail entity, into vector representations in the vector space, so that the sum of the head entity vector and the relationship vector is as close as possible to the tail entity vector.

[0098] S106, processing the multimodal and knowledge graph feature vectors to obtain a traffic flow prediction value;

[0099] In this embodiment, the multimodal and knowledge graph feature vectors are processed using a heterogeneous model collaborative processing algorithm, where heterogeneous models mainly refer to models with different structures and scales, such as complex large models: deep neural networks and relatively simple small models: small multi-layer perceptrons, etc. The collaborative processing algorithm utilizes the powerful learning and feature extraction capabilities of the large model, and outputs high-quality soft targets after training with a large amount of data. This soft target contains rich semantic information and a deep understanding of the data. Next, the small model defines the loss function based on the soft target and true label of the large model, and is trained through model distillation technology to learn the knowledge representation of the large model while maintaining its own high efficiency and lightweight characteristics.

[0100] Before obtaining the traffic flow prediction value, it is necessary to select a large-scale deep learning model as the teacher model, which has powerful feature learning and complex relationship modeling capabilities. At the same time, a lightweight neural network, such as a simple multi-layer perceptron or a small convolutional neural network, is selected as the student model. Next, a large number of historical traffic data samples with real labeled values ​​of traffic flow are used to train the teacher model. During the training process, the back propagation algorithm is used to adjust the parameters of the teacher model so that it can accurately predict traffic flow. Among them, the back propagation algorithm is a neural network training algorithm. The output and loss are calculated by forward propagation input data, and then the gradient is calculated and the network parameters are updated by back propagation to minimize the loss function. Further, after the teacher model training is completed, the student model is trained using the same historical traffic data. During the training process, the model distillation technology is used to minimize the loss function of the student model so that the student model learns the performance of the teacher model. Finally, after the training is completed, the student model can ensure that the reasoning efficiency and generalization ability are improved under the premise of a certain prediction accuracy, and finally output the traffic flow prediction value.

[0101] S107, constructing a traffic environment graph structure according to the traffic flow prediction value and the signal light location data;

[0102] In this embodiment, when constructing the traffic environment graph structure, it is necessary to use cameras and image recognition technology to capture images of traffic lights in the city, and use manual or automatic annotation tools to mark the location data of the traffic lights, and use the location data of the traffic lights as graph nodes in the graph structure. Next, the obtained traffic flow prediction value is associated with the corresponding traffic light node, and the traffic flow pressure index is calculated, and the predicted flow and the carrying capacity of the surrounding roads and the historical flow law are comprehensively considered to reflect the traffic congestion of the node. Further, according to the road network topology in the geographic information data, adjacent traffic light nodes are connected to form edges. For example, if two traffic lights are located on the same road and adjacent to each other, there is an edge connection between them. The attributes of the edge include information such as the length, width, number of lanes, road type, and road capacity limit of the connecting road. Finally, the data of the nodes and edges are integrated to construct a traffic environment graph structure that can fully reflect the association between the traffic flow situation and the traffic lights.

[0103] S108, initializing the control time of the signal light and constructing an evaluation module;

[0104] In this embodiment, the initial control time is set for each signal light according to the traffic rules. For example, for the signal lights at general intersections, the red light time is initially set to 30 to 60 seconds, the green light time is initially set to 30 to 90 seconds, and the yellow light time is fixed to 3 to 5 seconds. During the initialization process, different time periods, such as peak hours and off-peak hours, and traffic characteristics of different areas, such as commercial areas and residential areas, can also be considered to differentiate the initial settings of the signal light control time, which is not specifically limited here. Next, the evaluation module is constructed using a weight-based multi-index comprehensive evaluation technology. First, several key evaluation indicators are determined, such as the average vehicle waiting time, the road congestion index, and the traffic flow rate. For the average vehicle waiting time, an induction coil is set at the intersection or the camera image is used to analyze the vehicle stay time and calculate the average value. The road congestion index can be calculated based on the proportional relationship between the road section traffic volume and the road carrying capacity, and combined with the average vehicle speed. For example, when the traffic volume is close to saturation and the speed is slow, the congestion index increases. The traffic flow rate counts the number of vehicles passing through the intersection per unit time. Then assign weights to each indicator, for example, the average vehicle waiting time weight is set to 0.4, the road congestion index weight is 0.3, and the traffic flow rate weight is 0.3. The evaluation module collects the data of each indicator at a set time interval, such as after each signal cycle, and calculates the comprehensive evaluation score based on the weight.

[0105] S109, generating a traffic light control instruction according to the traffic flow prediction value and the control time;

[0106] In this embodiment, when generating signal light control instructions, it is necessary to judge the traffic pressure situation of each intersection based on the predicted traffic flow value. If a certain intersection predicts a large flow and is approaching congestion, and the current green light time is short, an instruction to extend the green light time is generated; conversely, when the flow is small and the red light is long, the red light time is shortened. Next, consider the synergy of adjacent intersections, and analyze the flow correlation and spacing of upstream and downstream intersections. For example, in order to avoid traffic congestion, when the green light at the upstream intersection is extended, the downstream intersection adjusts the green light start time according to the situation to ensure smooth connection of traffic and achieve a green wave band effect. After combining these factors, detailed control instructions such as the duration and switching time of the red, green, and yellow lights of each signal light are generated.

[0107] S110, interacting with the traffic environment graph structure according to the signal light control instruction to obtain an interaction result;

[0108] In this embodiment, after obtaining the signal light control instruction, the signal light will update its own state, such as switching the signal light color and the time for the signal light to switch colors, and feed back the current state change to the traffic environment graph structure to update the corresponding signal light node data. Then, based on the connection relationship and state data between the signal light nodes in the traffic environment graph structure, the downstream signal light responds according to its own control instruction and the state of the adjacent node. For example, when the green light at an intersection is on, its downstream node adjusts its own preparation time and green light duration based on the current road vehicle queue length, predicted traffic flow and preset coordination rules. Finally, the interactive results such as the average waiting time of vehicles at each intersection after the response, the change in road congestion level and the dynamic data of the overall traffic flow are obtained.

[0109] S111, calculating the interaction result according to the evaluation module to obtain a loss value;

[0110] In this embodiment, the loss value reflects the gap between the traffic operation status under the current signal light control instruction and the ideal status, which can be used for subsequent optimization and adjustment of the signal light control strategy. When calculating the interaction results, the evaluation module needs to calculate the average vehicle waiting time in the interaction results. , average vehicle waiting time The calculation is done by the following formula:

[0111] ;

[0112] The waiting time for each vehicle is The average vehicle waiting time is , the road congestion index is The traffic flow rate is , the total number of vehicles passing through the intersection during the statistical time period is .

[0113] Next, calculate the current road congestion index , road congestion index The calculation is done by the following formula:

[0114] ;

[0115] The current traffic volume is The road carrying capacity is , the current average speed of the vehicle is , is the weight coefficient, and the road congestion index is .

[0116] Then, calculate a signal period Traffic flow rate within , traffic flow rate The calculation is done by the following formula:

[0117] ;

[0118] in, For a signal cycle The number of vehicles passing through the intersection is designed to be The signal period The maximum number of vehicles that can pass through The signal period Traffic flow rate within the area.

[0119] Further, the comprehensive evaluation score is calculated , comprehensive evaluation score The calculation is done by the following formula:

[0120] ;

[0121] in, , as well as It is the time weight. You can set the time weight value according to the specific situation. There is no specific restriction here.

[0122] Finally, the evaluation module is scored based on the comprehensive evaluation Calculate the loss value , loss value The calculation is done by the following formula:

[0123] ;

[0124] Among them, the comprehensive evaluation score under ideal conditions is , which can be calculated by the indicator data of the period of good traffic operation in the historical data; the loss value is ; The current comprehensive traffic assessment score is .

[0125] S112, optimizing the control time according to the loss value to obtain an optimization result;

[0126] In this embodiment, after obtaining the loss value, the following optimization formula is first used to obtain the control time: Expected Return About Control Time Parameters The gradient is:

[0127] ;

[0128] in, Indicates control time Expected Return About Control Time Parameters The gradient of , which determines the update direction of the control time parameters; Indicates the control time based on The state obtained by sampling and actions Calculate the average of the gradients; Indicates control time About Control Time Parameters The logarithmic gradient of is used to measure the impact of the change of control time parameters on the probability of control time selection; It means that the optimization direction is to reduce the loss value.

[0129] Then, according to the obtained Update the control time parameters, which are calculated using the following formula:

[0130] ;

[0131] in, Indicates the updated control time parameters, Indicates the control time parameters before updating, Represents the learning rate, which is used to control the step size of each time control time parameter update.

[0132] Further, according to the updated control time parameters obtained, the control time is optimized to obtain the optimized control time, that is, the optimization result.

[0133] S113, training the traffic interaction model according to the optimization result to generate a target traffic interaction model;

[0134] In this embodiment, the obtained optimization results are used to train the initialized traffic interaction model. During the training process, the initialized traffic interaction model adjusts its internal parameters according to the input optimization results to minimize the error between the predicted results and the actual results. Through multiple iterations of training, the model gradually converges, that is, the loss value of the model no longer decreases significantly. At this time, the initialized traffic interaction model learns the optimal relationship between the signal light control time and the traffic environment graph structure, and can accurately generate optimized signal light control instructions based on the input traffic condition data, thereby generating a target traffic interaction model.

[0135] S114: input the acquired urban road condition data into the target traffic interaction model for processing to obtain a target traffic light control instruction.

[0136] In this embodiment, after obtaining the trained target traffic interaction model, the target traffic interaction model will obtain the urban road condition data for preprocessing, such as cropping, scaling and extracting visual features of image data, cleaning and normalizing sensor data, and converting geographic information data into a graph structure and extracting features, thereby forming multimodal road condition data. Next, the multimodal data feature fusion is performed on the multimodal road condition data using a preconfigured multimodal Transformer model, and the preconfigured knowledge graph embedding vector is combined with it to obtain a multimodal and knowledge graph feature vector. Subsequently, based on the multimodal and knowledge graph feature vectors, a heterogeneous model collaborative processing algorithm is used to predict traffic flow. Furthermore, a traffic environment graph structure is constructed based on the traffic flow prediction value and the signal light location data. Finally, combined with the traffic environment graph structure, an optimization algorithm is used to determine the duration and switching time of each light color of the signal light, thereby generating a target signal light control instruction.

[0137] This embodiment lays the foundation for subsequent model training by constructing an initial traffic interaction model and defining initial urban road condition data. Then, by using the multimodal Transformer model to process the initial urban road condition data, the road condition characteristics of the city are effectively extracted, which lays the foundation for the subsequent acquisition of accurate traffic flow prediction values ​​and enhances the traffic interaction model's ability to understand complex traffic scenarios. Then, through the pre-configured knowledge graph embedding vector, the traffic field knowledge is effectively combined with the actual road condition data. Further, the obtained multimodal and knowledge graph feature vectors are processed to obtain traffic flow prediction values, so that the traffic interaction model can better adapt to different traffic scenarios and complex road conditions; by constructing a traffic environment graph structure, the traffic flow prediction value is closely combined with the signal light position, realizing an accurate description of the traffic environment. Then, on this basis, the signal light control time is initialized, and an evaluation module is constructed, which provides a scientific basis for the generation and optimization of signal light control instructions. Furthermore, through the real-time interaction between the traffic light control instructions and the traffic environment graph structure, and the precise calculation of the interaction results by the evaluation module, the traffic interaction model is continuously optimized and improved to obtain a target traffic interaction model that can intelligently generate target traffic light control instructions based on real-time urban road condition data. This improves the level of automation in urban traffic management, provides strong data support for urban traffic planning and management, and promotes the development of smart transportation.

[0138] See also Figure 2 , Figure 2 Another embodiment of a traffic interaction method provided by the present application includes:

[0139] S201, construct and initialize a multimodal Transformer model;

[0140] In this embodiment, to construct an initialized multimodal Transformer model, it is first necessary to determine the overall framework of the multimodal Transformer model, then design independent input embedding layers for modal data such as vision, sensors, and geographic information, and further construct a multi-layer encoding structure, the core of which is a multi-head self-attention layer, where multiple heads focus on the relationship between the various parts of the multimodal data from different angles and perform weighted fusion. Finally, a feedforward neural network is used for nonlinear transformation, and the parameters of each layer are randomly initialized to give the initialized multimodal Transformer model initial learning ability, so that it can adjust parameters according to data features in subsequent training, and gradually improve the processing and feature extraction capabilities of multimodal road condition data.

[0141] S202, processing the multimodal road condition data to obtain multimodal data feature codes;

[0142] In this embodiment, multimodal data feature encoding refers to the process of converting data from different modalities, such as text, images, audio, etc., into numerical form so that the machine learning model can understand and process this data.

[0143] When processing multimodal road condition data, the visual modal data in the multimodal road condition data is firstly subjected to the convolutional neural network to capture local features, and the pooling layer is used to reduce the dimension to generate visual feature coding. Then, the sensor modal data in the multimodal road condition data is used to construct feature combinations based on traffic characteristics, such as the dynamic change feature sequence of traffic flow, to form sensor feature coding. Furthermore, the geographic information modality in the multimodal road condition data is converted into a graph structure and through the information transmission and aggregation of the graph neural network, feature coding is generated for the nodes and edges in the graph. Finally, the codes of each modality are integrated to obtain the multimodal feature coding.

[0144] S203, inputting the multimodal data feature code into the initialized multimodal Transformer model, so that the initialized multimodal Transformer model calculates the multimodal road condition data feature code according to a preconfigured attention mechanism to obtain a multimodal feature representation;

[0145] In this embodiment, the attention mechanism is a mechanism that simulates human attention in a deep learning model, which allows the model to focus on the most relevant parts of the current task when processing information. The core idea of ​​the attention mechanism is to enable the model to selectively pay attention to different parts of the input sequence and assign different weights to each part of the input sequence, so as to highlight the information that is more critical to the task.

[0146] When the multimodal data feature encoding is input into the initialized multimodal Transformer model, the multimodal Transformer model calculates the degree of correlation between each vector in the multimodal road condition data feature encoding through the internal pre-configured attention mechanism.

[0147] Among them, the preconfigured attention mechanism is calculated by the following first formula:

[0148] ;

[0149] in, represents the calculated multimodal feature representation; Q represents the query vector in the multimodal road condition data feature encoding; K is the key vector in the multimodal road condition data feature encoding; V is the value vector in the multimodal road condition data feature encoding; Represents the similarity between each query vector and each key vector; Represents the scaling factor, used to maintain stability between values; represents the probability distribution matrix; Represents the weighted multimodal feature representation.

[0150] S204, using a preconfigured cross entropy loss function to calculate the difference between the multimodal feature representation and a preset multimodal feature representation to obtain difference data;

[0151] In this embodiment, the cross entropy loss function mainly includes the cross entropy loss function of the single-label classification task and the cross entropy loss function of the multi-label classification task. The preset multimodal feature representation is an ideal feature representation pre-labeled in the training data set or determined by other prior knowledge, which represents the target feature expected to be output by the multimodal Transformer model.

[0152] Among them, the preconfigured cross entropy loss function is calculated by the following second formula:

[0153] ;

[0154] Wherein, Loss represents the difference data between the multimodal feature representation and the preset multimodal feature representation; N represents the total number of samples; Indicates the accumulation of difference data from the 1st sample to the Nth sample; Represents the preset multimodal feature representation of the i-th sample; represents the multimodal feature representation of the i-th sample calculated by the attention mechanism; and Represents the difference data of the current sample in the positive and negative categories.

[0155] S205, optimizing the initialized multimodal Transformer model based on the difference data to obtain a target multimodal Transformer model;

[0156] In this embodiment, when optimizing the initialized multimodal Transformer model based on the difference data, it is necessary to optimize the parameter values ​​of the initialized multimodal Transformer model to obtain the parameter values ​​of the optimized multimodal Transformer model.

[0157] The third formula below is used to calculate and obtain the parameter values ​​of the optimized multimodal Transformer model:

[0158] ;

[0159] in, Represents the parameter value of the optimized multimodal Transformer model; Represents the parameter value of the initialized multimodal Transformer model before optimization; Representation parameters Update the scaling factor to control the parameters of the multimodal Transformer model Update step size; Represents the parameters of the difference data loss relative to the multimodal Transformer model The gradient of To minimize the difference in data.

[0160] Next, the initialized multimodal Transformer model is optimized according to the parameter values ​​of the optimized multimodal Transformer model to obtain a target multimodal Transformer model.

[0161] S206: Input the multimodal road condition data into the target multimodal Transformer model for processing to obtain a target multimodal feature representation.

[0162] In this embodiment, the target multimodal Transformer model after training and optimization has learned the feature relationships and patterns in the multimodal road condition data. Next, the new multimodal road condition data is converted into feature encoding according to the processing flow described above and input into the target multimodal Transformer model. Furthermore, the target multimodal Transformer model accurately calculates the target multimodal feature representation based on its internal optimized parameters and structure through attention mechanism calculation.

[0163] In this embodiment, by constructing an initialized multimodal Transformer model, a special architecture is provided for multimodal road condition data processing, which can effectively integrate multiple modal data. Then, by encoding the features of the multimodal data, it is helpful to extract the key information in the data and remove redundancy. Then, the attention mechanism is used to calculate the multimodal feature representation, which can enable the multimodal Transformer model to focus on important data features and enhance the ability to capture multimodal data relationships. Further, the cross entropy loss function is used to calculate the difference data, which can accurately quantify the deviation between the model output and the preset value, and provide a basis for the subsequent optimization of the initialized multimodal Transformer model. Finally, the initialized multimodal Transformer model is optimized based on the difference data to obtain the target multimodal Transformer model, which provides a guarantee for the subsequent input of the multimodal road condition data into the target multimodal Transformer model to obtain the target multimodal feature representation, and the accurate target multimodal feature representation provides a basis for later operations such as path planning or traffic signal control.

[0164] See also Figure 3 , Figure 3 Another embodiment of a traffic interaction method provided by the present application includes:

[0165] S301, configuring a traffic knowledge graph;

[0166] In this embodiment, the traffic knowledge graph is a graphical tool for representing, organizing and sharing knowledge in the traffic field. It presents the associations between various key concepts, entities, attributes and relationships in the traffic field by establishing relationships between nodes and edges.

[0167] Before configuring the traffic knowledge graph, traffic-related data is collected from multiple sources, including but not limited to traffic rule manuals, historical traffic event records, road network topology, traffic flow statistics, speed limit information for different sections of roads, traffic signs, and the meaning of markings. Next, entity recognition and relationship extraction are performed on the collected data, and roads, intersections, and traffic signs are set as entities, and relationships such as roads connecting intersections are determined. Furthermore, a graph database is used to store the knowledge graph, where nodes represent entities and edges represent relationships. Finally, a classification standard for entities and relationships in the traffic knowledge graph is established.

[0168] S302, processing the traffic knowledge graph to obtain a knowledge graph embedding vector;

[0169] In this embodiment, the knowledge graph embedding vector refers to a representation method that maps entities and relationships in the knowledge graph into a continuous vector space. This representation method can learn vector representations of entities and relationships through supervised learning, making them semantically meaningful in a low-dimensional space, similar to word embedding. The knowledge graph embedding vector retains the structural information of the knowledge graph and can be used for various downstream tasks, such as question-answering systems or relationship extraction.

[0170] When processing the traffic knowledge graph, it is necessary to organize the entity and relationship data in the traffic knowledge graph as input, and then set the relationship between the head entity vector, relationship vector and tail entity vector based on the TransE algorithm hypothesis. Furthermore, prepare positive and negative samples, train them through the stochastic gradient descent optimization algorithm, minimize the marginal-based loss function, and continuously iterate and adjust the vector representation. After the training is completed, the knowledge graph embedding vector can be output.

[0171] S303. Process the multimodal feature representation and the knowledge graph embedding vector to obtain multimodal and knowledge graph feature vectors.

[0172] In this embodiment, when processing the multimodal feature representation and the knowledge graph embedding vector, a fusion feature method needs to be adopted. Taking the splicing method as an example, the dimension d1 of the multimodal feature representation vector and the dimension d2 of the knowledge graph embedding vector are connected end to end in sequence, and directly combined into a new vector with a dimension of d1+d2, that is, a fusion vector. Then the L2-norm is used to normalize the fusion vector to ensure that its numerical value is stable and comparable. Furthermore, depending on the specific application, if there is a need for dimensionality reduction, principal component analysis PCA and other technologies can be used to remove secondary information, reduce the dimension while retaining key features, and finally output the processed multimodal and knowledge graph feature vectors.

[0173] In this embodiment, multi-source traffic data is integrated to construct a knowledge graph, so that scattered traffic knowledge can be organized in a structured manner, providing a comprehensive and accurate knowledge reserve for the traffic interaction model. Next, the TransE algorithm is used to process the traffic knowledge graph to obtain the knowledge graph embedding vector, and the complex semantics and structural information in the graph is converted into a low-dimensional vector representation, which significantly reduces the complexity of data processing, while retaining key knowledge features, improving data processing efficiency and facilitating integration with other data types. Furthermore, the multimodal and knowledge graph feature vectors obtained by integrating the multimodal feature representation and the knowledge graph embedding vector realize the effective combination of multimodal perception data and prior traffic knowledge, enhancing the operational stability and fluency of the entire traffic interaction model.

[0174] See also Figure 4 The present application also provides a traffic interaction device, the system comprising:

[0175] The first construction unit 401 is used to construct an initialization traffic interaction model;

[0176] A first setting unit 402 is used to define initial city traffic data;

[0177] A first processing unit 403 is used to process the initial urban traffic condition data to obtain multi-modal traffic condition data;

[0178] A second processing unit 404 is used to process the multimodal road condition data using a preconfigured multimodal Transformer model to obtain a target multimodal feature representation;

[0179] A third processing unit 405 is used to obtain a multimodal and knowledge graph feature vector based on the target multimodal feature representation and the preconfigured knowledge graph embedding vector;

[0180] The fourth processing unit 406 is used to process the multi-modal and knowledge graph feature vectors to obtain a traffic flow prediction value;

[0181] A second construction unit 407, configured to construct a traffic environment graph structure according to the traffic flow prediction value and the signal light position data;

[0182] A second setting unit 408, used to initialize the control time of the signal light and construct an evaluation module;

[0183] A fifth processing unit 409, configured to generate a signal light control instruction according to the traffic flow prediction value and the control time;

[0184] A sixth processing unit 410, configured to interact with the traffic environment graph structure according to the signal light control instruction to obtain an interaction result;

[0185] A seventh processing unit 411 is used to calculate the interaction result according to the evaluation module to obtain a loss value;

[0186] An eighth processing unit 412, configured to optimize the control time according to the loss value to obtain an optimization result;

[0187] A training unit 413, configured to train the traffic interaction model according to the optimization result to generate a target traffic interaction model;

[0188] The ninth processing unit 414 is used to input the acquired urban road condition data into the target traffic interaction model for processing to obtain a target traffic light control instruction.

[0189] Optionally, the second processing unit 404 is further configured to:

[0190] Build and initialize a multimodal Transformer model;

[0191] Processing the multimodal road condition data using the initialized multimodal Transformer model to obtain a multimodal feature representation;

[0192] Calculating the difference between the multimodal feature representation and a preset multimodal feature representation using a preconfigured cross entropy loss function to obtain difference data;

[0193] Optimizing the initialized multimodal Transformer model based on the difference data to obtain a target multimodal Transformer model;

[0194] The multimodal road condition data is input into the target multimodal Transformer model for processing to obtain a target multimodal feature representation.

[0195] Optionally, the second processing unit 404 is further configured to:

[0196] Processing the multimodal road condition data to obtain multimodal data feature codes;

[0197] The multimodal data feature encoding is input into the initialized multimodal Transformer model, so that the initialized multimodal Transformer model calculates the multimodal road condition data feature encoding according to a preconfigured attention mechanism to obtain a multimodal feature representation.

[0198] Optionally, the third processing unit 405 is further configured to:

[0199] Configure the traffic knowledge graph;

[0200] Processing the traffic knowledge graph to obtain a knowledge graph embedding vector;

[0201] The target multimodal feature representation and the knowledge graph embedding vector are processed to obtain multimodal and knowledge graph feature vectors.

[0202] Optionally, the second processing unit 404 is further configured to:

[0203] Initialize the multimodal Transformer model to calculate the feature encoding of the multimodal road condition data according to a preconfigured attention mechanism to obtain a multimodal feature representation;

[0204] The preconfigured attention mechanism is calculated by the first formula:

[0205] ;

[0206] in, represents the calculated multimodal feature representation; Q represents the query vector in the multimodal road condition data feature encoding; K is the key vector in the multimodal road condition data feature encoding; V is the value vector in the multimodal road condition data feature encoding; Represents the similarity between each query vector and each key vector; Represents the scaling factor, used to maintain stability between values; represents the probability distribution matrix; Represents the weighted multimodal feature representation.

[0207] Optionally, the second processing unit 404 is further configured to:

[0208] Calculating the difference between the multimodal feature representation and a preset multimodal feature representation using a preconfigured cross entropy loss function to obtain difference data;

[0209] The preconfigured cross entropy loss function is calculated by the second formula:

[0210] ;

[0211] Wherein, Loss represents the difference data between the multimodal feature representation and the preset multimodal feature representation; N represents the total number of samples; Indicates the accumulation of difference data from the 1st sample to the Nth sample; Represents the preset multimodal feature representation of the i-th sample; represents the multimodal feature representation of the i-th sample calculated by the attention mechanism; and Represents the difference data of the current sample in the positive and negative categories.

[0212] Optionally, the second processing unit 404 is further configured to:

[0213] Obtaining parameter values ​​of an optimized multimodal Transformer model based on the difference data;

[0214] The parameter values ​​of the optimized multimodal Transformer model are calculated by the third formula:

[0215] ;

[0216] in, Represents the parameter value of the optimized multimodal Transformer model; Represents the parameter value of the initialized multimodal Transformer model before optimization; Representation parameters Update the scaling factor to control the parameters of the multimodal Transformer model Update step size; Represents the parameters of the difference data loss relative to the multimodal Transformer model The gradient of To minimize discrepancy data;

[0217] The initialized multimodal Transformer model is optimized according to the parameter values ​​of the optimized multimodal Transformer model to obtain a target multimodal Transformer model.

[0218] See also Figure 5 , the present application also provides a traffic interaction device, including:

[0219] Processor 501, memory 502, input and output unit 503, bus 504;

[0220] The processor 501 is connected to the memory 502, the input and output unit 503 and the bus 504;

[0221] The memory 502 stores a program, and the processor 501 calls the program to execute any of the above methods.

[0222] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program is run on a computer, the computer executes any of the above methods.

[0223] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0224] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0225] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0226] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0227] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A traffic interaction method, characterized in that: The method comprises: Construct an initial traffic interaction model; Define initial city traffic data; Processing the initial urban traffic condition data to obtain multimodal traffic condition data; Processing the multimodal road condition data using a preconfigured multimodal Transformer model to obtain a target multimodal feature representation; Obtaining a multimodal and knowledge graph feature vector based on the target multimodal feature representation and a preconfigured knowledge graph embedding vector; Processing the multimodal and knowledge graph feature vectors to obtain traffic flow prediction values; Constructing a traffic environment graph structure according to the traffic flow prediction value and the signal light location data; Initialize the control time of the signal light and build the evaluation module; generating a signal light control instruction according to the traffic flow prediction value and the control time; Interacting with the traffic environment graph structure according to the signal light control instruction to obtain an interaction result; Calculating the interaction result according to the evaluation module to obtain a loss value; Optimizing the control time according to the loss value to obtain an optimization result; Training the traffic interaction model according to the optimization result to generate a target traffic interaction model; The acquired urban road condition data is input into the target traffic interaction model for processing to obtain the target traffic light control instruction.

2. The traffic interaction method according to claim 1, characterized in that: The using a preconfigured multimodal Transformer model to process the multimodal road condition data to obtain a target multimodal feature representation includes: Build and initialize a multimodal Transformer model; Processing the multimodal road condition data using the initialized multimodal Transformer model to obtain a multimodal feature representation; Calculating the difference between the multimodal feature representation and a preset multimodal feature representation using a preconfigured cross entropy loss function to obtain difference data; Optimizing the initialized multimodal Transformer model based on the difference data to obtain a target multimodal Transformer model; The multimodal road condition data is input into the target multimodal Transformer model for processing to obtain a target multimodal feature representation.

3. The traffic interaction method according to claim 2, characterized in that: The using the initialized multimodal Transformer model to process the multimodal road condition data to obtain a multimodal feature representation includes: Processing the multimodal road condition data to obtain multimodal data feature codes; The multimodal data feature encoding is input into the initialized multimodal Transformer model, so that the initialized multimodal Transformer model calculates the multimodal road condition data feature encoding according to a preconfigured attention mechanism to obtain a multimodal feature representation.

4. The traffic interaction method according to claim 1, characterized in that: Obtaining a multimodal and knowledge graph feature vector based on the target multimodal feature representation and the preconfigured knowledge graph embedding vector, including: Configure the traffic knowledge graph; Processing the traffic knowledge graph to obtain a knowledge graph embedding vector; The target multimodal feature representation and the knowledge graph embedding vector are processed to obtain multimodal and knowledge graph feature vectors.

5. The traffic interaction method according to claim 3, characterized in that: The initialized multimodal Transformer model calculates the feature encoding of the multimodal road condition data according to a preconfigured attention mechanism to obtain a multimodal feature representation, including: Initialize the multimodal Transformer model to calculate the feature encoding of the multimodal road condition data according to a preconfigured attention mechanism to obtain a multimodal feature representation; The preconfigured attention mechanism is calculated by the first formula: ； in, represents the calculated multimodal feature representation; Q represents the query vector in the multimodal road condition data feature encoding; K is the key vector in the multimodal road condition data feature encoding; V is the value vector in the multimodal road condition data feature encoding; Represents the similarity between each query vector and each key vector; Represents the scaling factor, used to maintain stability between values; represents the probability distribution matrix; Represents the weighted multimodal feature representation.

6. The traffic interaction method according to claim 5, characterized in that: The using a preconfigured cross entropy loss function to calculate the difference between the multimodal feature representation and a preset multimodal feature representation to obtain difference data includes: Calculating the difference between the multimodal feature representation and a preset multimodal feature representation using a preconfigured cross entropy loss function to obtain difference data; The preconfigured cross entropy loss function is calculated by the second formula: ； Wherein, Loss represents the difference data between the multimodal feature representation and the preset multimodal feature representation; N represents the total number of samples; Indicates the accumulation of difference data from the 1st sample to the Nth sample; Represents the preset multimodal feature representation of the i-th sample; represents the multimodal feature representation of the i-th sample calculated by the attention mechanism; and Represents the difference data of the current sample in the positive and negative categories.

7. The traffic interaction method according to claim 6, characterized in that: The optimizing the initialized multimodal Transformer model based on the difference data to obtain a target multimodal Transformer model includes: Obtaining parameter values ​​of an optimized multimodal Transformer model based on the difference data; The parameter values ​​of the optimized multimodal Transformer model are calculated by the third formula: ； in, Represents the parameter value of the optimized multimodal Transformer model; Represents the parameter value of the initialized multimodal Transformer model before optimization; Representation parameters Update the scaling factor to control the parameters of the multimodal Transformer model Update step size; Represents the parameters of the difference data loss relative to the multimodal Transformer model The gradient of To minimize discrepancy data; The initialized multimodal Transformer model is optimized according to the parameter values ​​of the optimized multimodal Transformer model to obtain a target multimodal Transformer model.

8. A traffic interaction device, characterized in that: The device comprises: The first construction unit is used to construct an initial traffic interaction model; A first setting unit, used to define initial city traffic data; A first processing unit, configured to process the initial urban traffic condition data to obtain multimodal traffic condition data; A second processing unit, configured to process the multimodal road condition data using a preconfigured multimodal Transformer model to obtain a target multimodal feature representation; A third processing unit, configured to obtain a multimodal and knowledge graph feature vector based on the target multimodal feature representation and a preconfigured knowledge graph embedding vector; A fourth processing unit, used for processing the multi-modal and knowledge graph feature vectors to obtain a traffic flow prediction value; A second construction unit, configured to construct a traffic environment graph structure according to the traffic flow prediction value and the signal light position data; A second setting unit is used to initialize the control time of the signal light and build an evaluation module; a fifth processing unit, configured to generate a signal light control instruction according to the traffic flow prediction value and the control time; a sixth processing unit, configured to interact with the traffic environment graph structure according to the signal light control instruction to obtain an interaction result; a seventh processing unit, configured to calculate the interaction result according to the evaluation module to obtain a loss value; an eighth processing unit, configured to optimize the control time according to the loss value to obtain an optimization result; A training unit, used for training the traffic interaction model according to the optimization result to generate a target traffic interaction model; The ninth processing unit is used to input the acquired urban road condition data into the target traffic interaction model for processing to obtain the target traffic light control instruction.

9. A traffic interaction device, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the method according to any one of claims 1 to 7 is performed.

Images