Traffic scene training data generation method and device, electronic equipment and medium

Abstract

The invention discloses a traffic scene training data generation method and device, electronic equipment and a medium, and relates to the technical field of intelligent traffic, and the method comprises the steps: obtaining multi-modal traffic data, building a dynamic semantic interaction network according to the multi-modal traffic data, and enabling the dynamic semantic interaction network to comprise a plurality of traffic entities, the dynamic attribute of each traffic entity and the space-time relationship between the traffic entities are determined; in the dynamic semantic interaction network, labeling the target event and a core node of the target event according to a preset event specific sub-graph; performing causal chain backtracking on the labeled target event and the core node of the target event to obtain structured causal chain data corresponding to the target event; and performing question and answer pair generation processing on the structured causal chain data corresponding to the target event to obtain an instruction fine tuning data set for training the traffic large model. Therefore, automatic and high-quality generation of the traffic scene training data is realized, and the logical reasoning ability and interpretability of the model are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method, apparatus, electronic device, and medium for generating training data for traffic scenarios. Background Technology

[0002] With the rapid development of Large Language Models (LLMs) in natural language understanding and generation tasks, instruction fine-tuning has become an important means to improve the model's ability to understand complex semantics and follow human instructions. By fine-tuning the model, it can better transition from general language knowledge to domain-specific tasks. However, the effectiveness of instruction fine-tuning largely depends on the quality and construction of the instruction fine-tuning dataset. High-quality instruction fine-tuning datasets can effectively bridge the gap between the model's pre-training objectives (such as next-word prediction) and human task requirements (such as reasoning, judgment, and decision-making), thereby significantly improving the model's controllability and reasoning ability.

[0003] In related technologies, the construction of instruction fine-tuning datasets mainly includes three methods: manual generation, model generation, and collection and modification based on existing datasets. Manual generation methods typically rely on domain experts or human annotators to write questions and answers according to pre-set tasks. While the data quality is high and the language is natural, the construction cost is high and the efficiency is low, making it difficult to support large-scale instruction fine-tuning needs. Model generation methods automatically generate question-and-answer samples using existing large-scale language models, enabling rapid acquisition of large amounts of data. However, the factual accuracy and causal logic consistency of the generated results are often difficult to guarantee, requiring manual screening and revision. Modification based on existing datasets, while quickly integrating text resources from different sources and improving data diversity, suffers from significant differences in semantic annotation, format standards, and compliance among different sources, making it difficult to directly apply to instruction fine-tuning tasks in specific domains.

[0004] In the traffic scenario domain, traffic data typically comes from diverse sources and suffers from semantic complexity and inconsistent standards. Manually generated instruction samples cannot effectively cover the massive and dynamically changing traffic scenarios, while model-generated data often lacks accurate representation of causal logic, regulatory constraints, and other related knowledge. Furthermore, the lack of unified standards and high-quality traffic instruction fine-tuning datasets makes it difficult for existing methods to provide structured, semantically consistent, and interpretable instruction fine-tuning data for large traffic models, thus limiting the performance improvement of models in traffic scenario understanding, causal reasoning, and compliance decision-making tasks. Summary of the Invention

[0005] This invention aims to at least partially address one of the technical problems in related technologies. To this end, the first objective of this invention is to propose a method for generating traffic scene training data, enabling the efficient construction of a high-quality instruction fine-tuning dataset from multimodal traffic data. This achieves automated and high-quality generation of traffic scene training data, improving the model's logical reasoning ability and interpretability.

[0006] The second objective of this invention is to provide a traffic scene training data generation device.

[0007] The third objective of this invention is to provide an electronic device.

[0008] The fourth objective of this invention is to provide a computer-readable storage medium.

[0009] To achieve the above objectives, a first aspect of the present invention proposes a method for generating training data for traffic scenarios, comprising: acquiring multimodal traffic data and constructing a dynamic semantic interaction network based on the multimodal traffic data, wherein the dynamic semantic interaction network includes multiple traffic entities, dynamic attributes of each traffic entity, and spatiotemporal relationships between each traffic entity; in the dynamic semantic interaction network, labeling a target event and its core nodes according to a preset event-specific subgraph; performing causal chain backtracking on the labeled target event and its core nodes to obtain structured causal chain data corresponding to the target event; and performing question-and-answer pair generation processing on the structured causal chain data corresponding to the target event to obtain an instruction fine-tuning dataset for training a large traffic model.

[0010] According to the traffic scene training data generation method of the present invention, multimodal traffic data is acquired, and a dynamic semantic interaction network is constructed based on the multimodal traffic data. Then, in the dynamic semantic interaction network, target events and their core nodes are labeled according to a preset event-specific subgraph. Next, causal chain backtracking is performed on the labeled target events and their core nodes to obtain structured causal chain data for the corresponding target events. Finally, question-answer pair generation processing is performed on the structured causal chain data for the corresponding target events to obtain a command fine-tuning dataset for training a large traffic model. Thus, a high-quality command fine-tuning dataset is efficiently constructed from multimodal traffic data, thereby achieving automated and high-quality generation of traffic scene training data and improving the model's logical reasoning ability and interpretability.

[0011] In addition, the traffic scene training data generation method of the above embodiments of the present invention may also have the following additional technical features: According to an embodiment of the present invention, the step of constructing a dynamic semantic interaction network based on the multimodal traffic data includes: obtaining semantic tags of the multimodal traffic data; creating multiple nodes and multiple edges in the dynamic semantic interaction network based on the semantic tags, wherein each node is used to characterize the multiple traffic entities and the dynamic attributes of each traffic entity, and each edge is used to characterize the spatiotemporal relationship between each traffic entity.

[0012] According to an embodiment of the present invention, obtaining the structured causal chain data corresponding to the target event includes: obtaining the incoming edge of the core node of the target event and obtaining all parent nodes connected to the incoming edge; sequentially performing causal relationship filtering on all parent nodes connected to the incoming edge; constructing a causal fragment corresponding to each parent node based on each parent node that passes the causal relationship filtering, the core node of the target event, and the incoming edge, and storing the causal fragment of each parent node in a causal path list; after the causal chain backtracking is completed, obtaining the structured causal chain data corresponding to the target event based on all causal fragments stored in the causal path list.

[0013] According to an embodiment of the present invention, the causal backtracking ends when any of the following conditions are met: the parent node of the backtracking matches a preset root event pattern; the backtracking depth exceeds a preset maximum depth threshold, or the backtracking time exceeds a preset maximum time window; or there are no nodes to be backtracked.

[0014] According to one embodiment of the present invention, obtaining the structured causal chain data of the corresponding target event based on all causal fragments stored in the causal path list includes: reorganizing all causal fragments stored in the causal path list in chronological and logical order to output the structured causal chain data of the corresponding target event.

[0015] According to an embodiment of the present invention, obtaining the instruction fine-tuning dataset for training a large traffic model includes: forming causal inference question-and-answer pairs and compliance decision question-and-answer pairs for the corresponding target event based on the structured causal chain data of the corresponding target event, respectively through causal inference question-and-answer pair generation rules and compliance decision question-and-answer pair generation rules; and obtaining the instruction fine-tuning dataset for training the large traffic model based on the causal inference question-and-answer pairs and compliance decision question-and-answer pairs for the corresponding target event.

[0016] According to one embodiment of the present invention, the method further includes: converting the structured causal chain data of the corresponding target event into text data containing natural speech description by means of template filling or template generation.

[0017] To achieve the above objectives, a second aspect of the present invention proposes a traffic scene training data generation device, comprising: a network construction module for acquiring multimodal traffic data and constructing a dynamic semantic interaction network based on the multimodal traffic data, wherein the dynamic semantic interaction network includes multiple traffic entities, dynamic attributes of each traffic entity, and spatiotemporal relationships between each traffic entity; an event annotation module for annotating target events and core nodes of the target events in the dynamic semantic interaction network according to a preset event-specific subgraph; a causal backtracking module for performing causal chain backtracking on the annotated target events and core nodes of the target events to obtain structured causal chain data corresponding to the target events; and a data generation module for performing question-and-answer pair generation processing on the structured causal chain data of the corresponding target events to obtain an instruction fine-tuning dataset for training a large traffic model.

[0018] According to an embodiment of the present invention, a traffic scene training data generation device acquires multimodal traffic data through a network construction module and constructs a dynamic semantic interaction network based on the multimodal traffic data. The dynamic semantic interaction network includes multiple traffic entities, dynamic attributes of each traffic entity, and spatiotemporal relationships between each traffic entity. Then, an event annotation module annotates target events and their core nodes in the dynamic semantic interaction network based on a preset event-specific subgraph. Next, a causal backtracking module performs causal chain backtracking on the annotated target events and their core nodes to obtain structured causal chain data for the corresponding target events. Finally, a data generation module performs question-and-answer pair generation processing on the structured causal chain data for the corresponding target events to obtain a fine-tuning instruction dataset for training a large traffic model. This achieves efficient construction of a high-quality fine-tuning instruction dataset from multimodal traffic data, thereby realizing automated and high-quality generation of traffic scene training data and improving the model's logical reasoning ability and interpretability.

[0019] To achieve the above objectives, a third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described traffic scene training data generation method.

[0020] According to the electronic device of the present invention, by executing a computer program stored in the memory through a processor, a high-quality instruction fine-tuning dataset can be efficiently constructed from multimodal traffic data, thereby realizing the automated and high-quality generation of traffic scenario training data and improving the model's logical reasoning ability and interpretability.

[0021] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the steps of the traffic scene training data generation method described above.

[0022] According to embodiments of the present invention, a computer-readable storage medium can efficiently construct a high-quality instruction fine-tuning dataset from multimodal traffic data by executing a computer program stored thereon, thereby achieving automated and high-quality generation of traffic scenario training data and improving the model's logical reasoning ability and interpretability.

[0023] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a traffic scene training data generation method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a traffic scene training data generation method according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating a traffic scene training data generation method according to another embodiment of the present invention; Figure 4 This is a flowchart illustrating a traffic scene training data generation method according to yet another embodiment of the present invention; Figure 5 This is a flowchart illustrating a causal backtracking algorithm according to an embodiment of the present invention; Figure 6 This is a block diagram of a traffic scene training data generation device according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] The implementation details of the technical solutions in the embodiments of this application are described in detail below.

[0027] Figure 1 This is a flowchart illustrating a traffic scene training data generation method according to an embodiment of the present invention; Specifically, in some embodiments of the present invention, such as Figure 1 As shown, the method for generating training data for traffic scenarios may include the following steps: S101, acquire multimodal traffic data, and construct a dynamic semantic interaction network based on the multimodal traffic data. The dynamic semantic interaction network includes multiple traffic entities, the dynamic attributes of each traffic entity, and the spatiotemporal relationships between each traffic entity.

[0028] It is understood that, in this embodiment of the present invention, a dynamic semantic interaction network for describing traffic scenarios is constructed based on multimodal traffic data. The dynamic semantic interaction network includes multiple traffic entities (such as vehicles, pedestrians, traffic lights, etc.), dynamic attributes of each traffic entity (such as speed, acceleration, position, etc.), and spatiotemporal relationships between each traffic entity (such as following, parallelism, conflict, etc.).

[0029] Optionally, in the above embodiments of the present invention, the multimodal traffic data includes multi-source information from multimodal sensing sources such as cameras and radar. Since the data collected by different sensors usually differ in timestamps and coordinate systems, it is necessary to perform spatiotemporal alignment and intelligent completion on these data to achieve consistency and comparability of multimodal traffic data in time and space dimensions.

[0030] Furthermore, in some embodiments of the present invention, such as Figure 2 As shown, a dynamic semantic interaction network is constructed based on multimodal traffic data, including: S201, Obtain semantic labels for multimodal traffic data.

[0031] It is understood that, in this embodiment of the present invention, multimodal traffic data can be converted into structured data through high-fidelity semantic annotation. The semantic annotation process integrates algorithms such as computer vision recognition, object detection, and semantic segmentation to identify and classify various entities appearing in the traffic scene and extract their dynamic attribute information. Semantic tags are used to describe each entity (such as vehicles, pedestrians, and traffic lights) in the traffic scene and its dynamic attributes (such as speed, acceleration, and position) that change over time in a structured form. Specifically, a unique entity identifier is assigned to each identified entity, and a set of dynamic attributes including timestamps and perceived classification labels are extracted to obtain a set of semantic tags for the multimodal traffic data, such as {Entity ID: "Vehicle_01", Type: "Automobile", Position: [x,y,z], Speed: 50km / h}. This set of semantic tags can reflect the static composition and dynamic evolution characteristics of the traffic scene in a structured manner.

[0032] S202, create multiple nodes and multiple edges in a dynamic semantic interaction network based on semantic tags, where each node is used to represent multiple traffic entities and the dynamic attributes of each traffic entity, and each edge is used to represent the spatiotemporal relationship between each traffic entity.

[0033] It is understood that in this embodiment of the present invention, after obtaining semantic tags, a dynamic semantic interaction network for a traffic scenario is established based on the semantic tags. By automatically parsing the semantic tags, each traffic entity and its dynamic attributes are mapped to nodes in the network, wherein the node structure may include entity identifier, timestamp, and attribute fields; the spatiotemporal relationships between traffic entities are mapped to edges in the network used to connect nodes. In practical applications, the construction of the dynamic semantic interaction network is a dynamic graph structure in the time dimension, which can update nodes and edges according to the time series, thereby dynamically reflecting the interaction changes and event evolution processes between entities in the traffic scenario.

[0034] S102, In the dynamic semantic interaction network, the target event and the core node of the target event are labeled according to the preset event-specific subgraph.

[0035] It is understood that in this embodiment of the present invention, a target event matching a preset event-specific subgraph is searched within the constructed dynamic semantic interaction network. Specifically, a preset event-specific subgraph is first established for typical traffic events (such as accidents and congestion). This preset event-specific subgraph describes the entities, interaction relationships, and temporal attributes involved in the traffic event. Furthermore, by defining corresponding preset event-specific subgraphs for different types of traffic events, a traffic event template library can be formed to identify various target events in the semantic interaction network and to annotate the target events and their core nodes.

[0036] For example, a preset event-specific sub-event used to define a rear-end collision. Figure 1 This includes nodes representing car A, nodes representing car B, and edges representing the spatiotemporal relationship between car A and car B. A specific sub-event is defined for this preset event. Figure 1 Set the following matching criteria: (1) At t event At that moment, car A was directly behind car B; (2) The type of the edge between car A and car B changes from following to spatiotemporal conflict; (3) The state of car A changes from moving to its speed dropping to zero; (4) The state of car B changes to a sudden drop in speed to zero or close to zero.

[0037] For example, preset event-specific sub-presets used to define regional congestion. Figure 2 This includes multiple vehicle nodes (the node of vehicle A1, the node of vehicle A2, ..., the node of vehicle A) within a certain area (e.g., a certain road segment). n The nodes), and the edges representing the spatiotemporal relationships between these nodes. A specific sub-event is defined for this preset event. Figure 2 Set the following matching criteria: (1) At t eventAt any given time, the number of vehicle nodes n is greater than a preset threshold (e.g., n > 15). (2) At t event At any given moment, the average speed of these n vehicles is less than a preset threshold (e.g., average speed < 15 km / h).

[0038] It is understood that in the above embodiments of the present invention, during the event recognition process, the dynamic semantic interaction network is traversed and matched, and the nodes and their spatiotemporal edge relationships in the network are compared with the preset event-specific subgraph to determine their consistency in node type, relationship type and time evolution order. Then, when a certain local structure in the dynamic semantic interaction network satisfies the matching conditions of the preset event-specific subgraph, the traffic event corresponding to the structure is identified as the target event.

[0039] For identified target events, the core nodes of the event are further extracted. Core nodes represent key traffic entities directly involved in the event; that is, the set of main nodes constituting the event relationship chain in the predefined event-specific subgraph of the event definition. For example, if the target event is a rear-end collision, the corresponding core nodes are the nodes of vehicle A and vehicle B; if the target event is regional congestion, the corresponding core nodes are vehicle A1, vehicle A2, ..., vehicle A... n The node.

[0040] It should be noted that in practical applications, multiple target events can be identified and labeled separately for multiple events that may occur at the same time or in the same area.

[0041] S103, perform causal chain backtracking on the labeled target events and the core nodes of the target events to obtain the structured causal chain data of the corresponding target events.

[0042] It should be understood that the backtracking process of causal chain tracing starts from the core node of the target event and proceeds in reverse along the edges connected to the core node in the semantic interaction network to obtain the structured causal chain data of the corresponding target event.

[0043] In practical applications, structured causal chain data can describe the causal logical relationship of an event from its initial cause to its final result in the form of nodes and edges. Nodes are used to represent entities participating in the causal process, and edges are used to represent the type of causal relationship and time dependency.

[0044] S104, perform question-and-answer pair generation processing on the structured causal chain data corresponding to the target event to obtain the instruction fine-tuning dataset for training the traffic large model.

[0045] Specifically, in some embodiments of the present invention, such as Figure 3 As shown, the instruction fine-tuning dataset used to train the large traffic model is obtained, including: S301, based on the structured causal chain data of the corresponding target event, causal reasoning question-and-answer pairs and compliance decision-making question-and-answer pairs are formed for the corresponding target event through causal reasoning question-and-answer pair generation rules and compliance decision-making question-and-answer pair generation rules, respectively.

[0046] It is understood that, in this embodiment of the present invention, after obtaining the structured causal chain data corresponding to the target event, causal reasoning question-and-answer pair generation rules and compliance decision question-and-answer pair generation rules are respectively used to form causal reasoning question-and-answer pairs and compliance decision question-and-answer pairs corresponding to the target event.

[0047] The causal reasoning question-and-answer pair generation rule is as follows: Based on the final event in the structured causal chain data, the first question instruction is generated using a preset question template. By traversing the structured causal chain data in sequence, the structured events are transformed and combined into natural language statements to generate the first answer corresponding to the first question instruction.

[0048] The final event (final_event) portion of the structured causal chain data is analyzed. The final event refers to a traffic event that occurs at the very end of the time series and can be considered a result or direct manifestation of the event, such as a rear-end collision. Based on the final event, a first question instruction is generated using a pre-defined question template. This first question instruction guides the model in analyzing the causes of the final event. For example, the template could be "Please analyze in detail the causes of {event type} that occurred at {location}", where placeholders are filled with the event location and type of the final event, thus generating a specific question for a particular traffic scenario.

[0049] Subsequently, starting from the root event of the causal chain, each event node and state is traversed sequentially along the causal chain. The root event refers to the initial factor or direct precondition that leads to the final event, such as the sudden braking of the vehicle in front. During the traversal, each structured event or state is converted into a natural language description, and logical connectors (such as "because," "leading to," and "ultimately causing") are used to string the event statements into logically coherent paragraphs, forming the first answer corresponding to the first question instruction, i.e., the causal reasoning question-and-answer pair.

[0050] Compliance decision-making Q&A generates rules: Transform traffic scenario data into an understandable scenario description, which includes key elements and events in the scenario, such as vehicle location, traffic signals, road conditions, and dynamic behavior, to ensure that it accurately reflects the actual traffic environment and the regulatory constraints that may be involved.

[0051] Based on the scenario description, a second question instruction is generated using a regulatory knowledge base. This second question instruction guides the model to determine compliant operations or decisions in this scenario. For example, for the scenario of "a school bus stopping ahead with a stop sign extended," the question could be: "Faced with the scenario ahead, how should an autonomous vehicle make decisions to ensure compliance?"

[0052] Subsequently, the relevant regulatory entries that best match the scenario are retrieved from the regulatory knowledge base. Combined with the specific entities and behaviors in the scenario, the corresponding compliance requirements are extracted and transformed into standardized natural language answers, such as "must stop and wait, must not overtake", forming a second answer corresponding to the second question instruction, i.e., a compliance decision question-and-answer pair.

[0053] S302, Based on the causal reasoning question-and-answer pairs and compliance decision-making question-and-answer pairs corresponding to the target events, obtain the instruction fine-tuning dataset for training the traffic large model.

[0054] It is understood that, in this embodiment of the present invention, the aforementioned first question instruction and first answer (i.e., causal reasoning question-and-answer pair), and the aforementioned second question instruction and second answer (i.e., compliance decision question-and-answer pair) are integrated to construct a complete instruction fine-tuning dataset. Specifically, the first question instruction and first answer form a causal reasoning question-and-answer pair, which reflects the causal logic and evolution process of traffic events, providing training information for the model to understand the causal relationships in traffic scenarios; the second question instruction and second answer form a compliance decision question-and-answer pair, which reflects reasonable operating strategies under traffic regulations, providing training basis for the model to make decisions that comply with regulations in actual traffic scenarios.

[0055] It's important to note that the core goal of instruction fine-tuning is to enable the model to better understand and execute human-given instructions. Question-answer pairs provide training examples of these instructions and responses. In a question-answer pair, the question represents the instruction or task the model needs to execute, while the answer demonstrates the expected output or behavior. Based on this, the instruction fine-tuning dataset can improve the model's understanding, reasoning, and compliance decision-making abilities in complex traffic environments.

[0056] It should be understood that, in the above embodiments of the present invention, the structured causal chain data is subjected to textualization processing to generate an instruction fine-tuning dataset for training traffic scenario models. The textualization processing is used to reorganize the event elements, triggering conditions, and result relationships contained in the causal chain in the form of natural language, thereby forming text data with semantic coherence and logical integrity.

[0057] In this process, the causal chain of time can be transformed into a question-and-answer format that conforms to human language comprehension patterns, based on the temporal order and logical dependencies of events in the structured causal data chain. Through this transformation, the generated data can maintain the logical consistency of the causal chain while possessing linguistic features that the model can directly understand.

[0058] The result of this text-to-text conversion process is an instruction fine-tuning dataset containing a large amount of question-and-answer pairing data with causal logic. This embeds deep causal logic between traffic events at the linguistic level, enabling the model to not only learn "what happened" during training but also understand "why it happened." This data feature significantly enhances the model's causal reasoning and logical explanation capabilities for traffic scenarios, thus enabling more reasonable and interpretable outputs in subsequent applications.

[0059] In practical applications, the generated instruction fine-tuning dataset can be uniformly stored as a structured text file or database entry and used as data input for training large-scale traffic scenario models. This step realizes the transformation from causal structural information to semantic and task-oriented language data, making the model training data standardized in form and logically clear in semantics, laying the foundation for building intelligent traffic scenario models with deep causal understanding capabilities.

[0060] Furthermore, in some embodiments of the present invention, such as Figure 4 As shown, the structured causal chain data corresponding to the target event is obtained, including: S401: Obtain the incoming edges of the core node of the target event, and obtain all parent nodes connected to the incoming edges.

[0061] It is understood that, in this embodiment of the present invention, the parent node that has a direct spatiotemporal association with the current core node is determined based on the incoming edges pointing to the current core node in the dynamic semantic network.

[0062] Specifically, a dynamic semantic network containing spatiotemporal information is denoted as a spatiotemporal graph G = (V, E, T), where V is the set of nodes, including various traffic entities and attributes in the traffic scenario; E is the set of edges, used to represent the spatiotemporal relationships between nodes; and T is timestamp information, reflecting the temporal attributes of nodes and events. The core node set V of the target event... event It contains one or more core nodes related to the target event, and records the timestamp of the target event as t. event .

[0063] From the core node set V event Extract a node V current This refers to the current core node. Regarding the current core node V... currentSearch the spacetime graph G for all points pointing to the current core node V. current The inbound edge e in And determine each incoming edge e in The source node is the current core node V. current Parent node V parent Parent node V parent It is a potential cause of the target event.

[0064] S402, sequentially filter the causal relationships of all parent nodes connected to the incoming edge.

[0065] It is understood that in this embodiment of the present invention, the parent node is filtered for causal correlation based on timestamps and incoming edges.

[0066] Specifically, timestamps are used to reflect the chronological order of events, and incoming edges are used to represent the logical dependencies between nodes. Through incoming edge e... in and the parent node's timestamp t parent For parent node V parent Nodes that do not conform to causal logic in terms of time and relationship are filtered out. This filtering process yields a set of parent nodes with strong causal relationships to the core nodes, thus ensuring the logical accuracy of the constructed causal chain and the rationality of event evolution.

[0067] In one embodiment, timestamps t are filtered based on time constraints. parent Earlier than or equal to the current core node V current Corresponding timestamp t current The parent node is used to ensure the correctness of the causal direction, that is, the potential causal event must occur before or at the same time as the target event in time, thereby excluding nodes that do not have a logical sequential relationship in time.

[0068] In addition, according to the incoming edge e in This filters the causal relationship types of the parent nodes. In practical applications, a predefined set of causal relationship types is used, which includes relationship types that reflect the causal connections between events (such as "cause," "initiate," "precursor to conflict," etc.). Only when the incoming edge e... in Only when the relationship type belongs to this set is the parent node considered to have causal correlation and retained, thereby excluding unrelated parallel relationships.

[0069] S403: Construct a causal fragment for each parent node based on the parent node selected through causal relationship, the core node of the target event, and the incoming edges, and store the causal fragment of each parent node in the causal path list.

[0070] It is understood that in this embodiment of the present invention, the filtered parent node, the current core node and the corresponding edge are constructed as a causal segment.

[0071] Specifically, after obtaining the parent nodes filtered by causal correlation, for each pair of node relationships (V) that passed the filtering... parent V current Extract its corresponding edge e in The three are then encapsulated into causal segments according to a causal chain structure. Each causal segment is used to describe a local event evolution relationship in a traffic scenario, that is, a local causal structure from several upstream events (parent nodes) to the core event (core node).

[0072] In one embodiment, for the parent node V that passes the filtering... parent Match it with the corresponding timestamp t parent Stored together, to be used later for the parent node V that has passed the screening. parent Perform causal backtracking.

[0073] After completing the causal backtracking of all core nodes, based on the previously saved filtered parent node V... parent Match it with the corresponding timestamp t parent Then, perform causal backtracking on these parent nodes in sequence. That is, for the stored parent node V... parent Then, find the incoming edges pointing to the node again, identify its upstream parent node, perform causal correlation filtering, and store the filtering results.

[0074] Through this recursive backtracking process, the depth of the causal chain can be expanded layer by layer, enabling hierarchical identification from direct causes to indirect causes, thereby obtaining structured causal information covering the entire causal path of the target event.

[0075] In the actual execution of causal backtracking, to manage the backtracking path and node order, a causal path list (CausalPath) for storing causal fragments and a first-in-first-out backtracking queue (Queue) can be initialized. During the initial causal backtracking phase, the core node set V of the target event is... event and its corresponding event timestamp t event It is stored as the initial element in the backtracking queue (Queue) to initiate the causal backtracking process.

[0076] Take a node V from the backtracking queue Queue. current and its timestamp t current Following the aforementioned causal backtracking operation, the final selected parent node V is obtained. parent .

[0077] The parent node V that passes the filter parentCurrent core node V current and the relationship between them. in The causal fragments formed by combination (V) parent e in V current Store the causal path list CausalPath, and at the same time, select the parent node V that has been filtered. parent and timestamp t parent Record them together and push them into the tail of the backtracking queue, so that backtracking operations can be performed later.

[0078] After the parent node selection and enqueueing operations of the current node are completed, new node elements are sequentially retrieved from the backtracking queue, and the above backtracking process is repeated. It can be understood that in this first-in-first-out backtracking queue, the causal backtracking of the core nodes is completed first, followed by the causal backtracking of the selected parent nodes, thus ensuring the sequential backtracking of the causal chain in time, while maintaining the hierarchical integrity of the causal dependency structure in space.

[0079] S404 After the causal chain backtracking is completed, obtain the structured causal chain data of the corresponding target event based on all causal fragments stored in the causal path list.

[0080] Furthermore, in some embodiments of the present invention, obtaining structured causal chain data corresponding to the target event based on all causal fragments stored in the causal path list includes: reorganizing all causal fragments stored in the causal path list in chronological and logical order to output structured causal chain data corresponding to the target event.

[0081] It is understood that, in this embodiment of the present invention, each edge of each core node in the target event carries the spatiotemporal relationship type and time attribute between traffic entities. By analyzing these edge relationships and time series, the upstream behavior sequence that caused the target event and its influence path can be determined, thereby extracting the causal fragments related to the core node.

[0082] Causal fragments are used to represent the direct or indirect causes of a target event. They typically include upstream entity nodes that interacted with the core node in the period preceding the event, their behavioral state changes, and corresponding temporal information. For example, when the identified target event is a rear-end collision, the core nodes are the nodes corresponding to car A and car B. In causal backtracking, starting from the nodes corresponding to car A and car B, the associated nodes and temporal relationships are traced back in the dynamic semantic interaction network to identify the upstream causal fragments that may have caused the rear-end collision.

[0083] Based on the extracted causal fragments, they are combined in an orderly manner according to time sequence and causal direction to generate structured causal chain data corresponding to the target event.

[0084] Below is a data example of structured causal chain data in JSON format.

[0085] { "final_event": {"type": "congestion", "timestamp": "t1", ...}, "causal_chain": [ {"event": "lane occupancy", "timestamp": "t0", "details": "..."}, {"event": "rear-end collision", "timestamp": "t_root", "details": "..."} ] } In the data example above, `final_event` represents the target event, i.e., the endpoint of the causal backtracking, recording the event type, timestamp, and related details. `causal_chain` is an array of events arranged chronologically or logically, where each element represents an upstream event node, containing the event name, timestamp, and a detailed description. This chain reflects the causal relationship between events, with the root event influencing the final event layer by layer, forming a complete causal logical sequence.

[0086] Furthermore, in some embodiments of the present invention, causal backtracking terminates when any of the following conditions are met: The parent node being traced back matches the preset root event pattern; The backtracking depth exceeds the preset maximum depth threshold, or the backtracking time exceeds the preset maximum time window; There are no nodes to be backtracked.

[0087] It is understood that, in this embodiment of the present invention, to avoid infinite backtracking and ensure the rationality and controllability of the backtracking results, a termination condition can be set. The backtracking process stops when any of the following conditions are met: (1) If the parent node traced back matches a predefined root event pattern, the causal chain is considered to have been traced back to the most upstream key event, and the backtracking is terminated. For example, root event patterns may include traffic accidents, traffic light malfunctions, etc., and backtracking will not continue when these patterns are matched.

[0088] (2) When the backtracking depth exceeds the preset maximum depth threshold D max Or, the difference between the timestamp of the current node and the timestamp of the target event exceeds the preset maximum time window T. max (i.e., the timestamp t of the target event)event -t parent >T max ).

[0089] (3) If there are no more backtracking nodes, that is, the backtracking queue is empty, backtracking will also be terminated.

[0090] It should be noted that, in the process of causal backtracking, through multiple rounds of causal backtracking, all local causal fragments about the target event can be gradually extracted from the global semantic network, and thus used to generate a complete causal chain.

[0091] Figure 5 The flowchart of the causal backtracking algorithm is shown below. Figure 5 This further explains the entire process of causal retrospection.

[0092] At the beginning of the algorithm, a backtracking queue (Queue) is used to receive input. This backtracking queue contains the core nodes of the target event and their corresponding timestamps.

[0093] At the start of each backtracking round, the queue status is checked first. If the queue is empty, it means there are no nodes to backtrack to, and the algorithm ends; if the queue is not empty, the next step is performed.

[0094] Retrieve the head node and its corresponding timestamp from the queue, and denote it as the current core node V. current and timestamp t current .

[0095] Retrieving the current core node V in the semantic network current The causal parent node. Specifically, it points to the current core node V. current The inbound edge e in , Entering the edge e in The source node that is connected is the parent node V of the current node. parent This represents possible antecedent factors. For these parent nodes V... parent Perform causal correlation screening, and select the parent node V through the screening. parent Current core node V current With the corresponding edge e in Together they constitute a causal segment.

[0096] If based on the current core node V current The parent node V that is backtracked to parent If the root cause event is met, it is considered that the root cause of the target event has been located. At this point, the causal fragment result is output and the causal backtracking ends.

[0097] If no root event has been matched, continue to check if the current backtracking depth exceeds the maximum threshold D.max If the limit is exceeded, the causal backtracking process ends.

[0098] If the backtracking depth does not exceed the limit, then determine whether the backtracking time span exceeds the preset time window T. max If the time frame is exceeded, the causal backtracking will also end.

[0099] If the causal backtracking termination condition is not met, the selected parent node V will be used. parent With timestamp t parent Push the node back onto the tail of the backtracking queue so that it can continue tracing in the next loop. Then return to the queue empty check step and repeat the process until all nodes in the backtracking queue have been processed.

[0100] Furthermore, in some embodiments of the present invention, the method further includes: converting the structured causal chain data corresponding to the target event into text data containing natural speech descriptions by using template filling or template generation methods.

[0101] It is understood that, in this embodiment of the present invention, the structured causal chain data corresponding to the target event can also be converted into text data containing natural speech descriptions by template filling or template generation. For example, based on the node information and causal relationship in the structured causal chain data, the traffic entities, event types, spatiotemporal attributes, etc. involved in the causal chain can be mapped to text description elements.

[0102] Specifically, in the template-filling method, multiple natural language description templates are pre-defined, containing placeholders to correspond to key fields in structured causal chain data. By matching and filling the placeholders with key-value pairs in the structured data, the corresponding corpus can be obtained. For example, for the template "{event type} occurs at {location}, involving {number of vehicles}", the location information, event type, and number of vehicles involved in the structured causal chain can be filled into the placeholders to generate a specific natural language description. This method is simple to operate, generates text with a clear structure and complete information, and is suitable for quickly generating large-scale training corpora.

[0103] Furthermore, in the template generation method, the structured causal chain data is first serialized into a string with a specific format, such as "congestion | location: a certain road segment | number of vehicles: 2". The serialized string is then input into a finely tuned sequence-to-sequence (Seq2Seq) model, such as a text-to-text transfer transformer (T5) or a sequence-to-sequence generation model (BART, Bidirectional and Auto-Regressive Transformers). This model, after training, can convert structured information into fluent and natural text descriptions. This approach can generate traffic scene corpora with diverse language expressions and conforming to natural language logic, enriching the model's understanding and reasoning capabilities.

[0104] Through the above processing, the generated textual pre-training corpus not only retains the causal logic and key information of the events, but also reflects the causal progression between events at the language level, significantly improving the model's causal understanding and logical reasoning ability in traffic scenarios.

[0105] Specifically, the traffic scene training data generation method based on embodiments of the present invention acquires multimodal traffic data and constructs a dynamic semantic interaction network, which can accurately characterize each entity and its dynamic attributes in the traffic scene, as well as the spatiotemporal relationships between entities, thereby providing structured basic information for traffic event identification. On this basis, by matching preset subgraph patterns to identify target events and extracting core nodes, and performing causal backtracking based on the core nodes to generate structured causal chain data, it can not only reveal the upstream causes and impact paths of events, but also construct hierarchical causal logic chains, reflecting the causal relationships and evolutionary process of traffic events. Next, by performing text-based conversion processing on the structured causal chain data, it can generate an instruction fine-tuning dataset that can be directly used in large-scale traffic scene models, making the generated data consistent in form, coherent in semantics, and logically clear, thereby significantly improving the capabilities of large-scale traffic models in event understanding, logical reasoning, and instruction compliance, while simultaneously achieving efficient and automated generation of the instruction fine-tuning dataset.

[0106] In summary, the traffic scene training data generation method according to embodiments of the present invention acquires multimodal traffic data and constructs a dynamic semantic interaction network based on the multimodal traffic data. Then, within the dynamic semantic interaction network, target events and their core nodes are labeled according to a preset event-specific subgraph. Next, causal chain backtracking is performed on the labeled target events and their core nodes to obtain structured causal chain data corresponding to the target events. Finally, question-answer pair generation processing is performed on the structured causal chain data of the corresponding target events to obtain a command fine-tuning dataset for training a large traffic model. Thus, a high-quality command fine-tuning dataset is efficiently constructed from multimodal traffic data, thereby achieving automated and high-quality generation of traffic scene training data and improving the model's logical reasoning ability and interpretability.

[0107] Figure 6 This is a block diagram of a traffic scene training data generation device according to an embodiment of the present invention.

[0108] Specifically, in some embodiments of the present invention, such as Figure 6 As shown, the traffic scene training data generation device 100 includes: a network construction module 10, an event labeling module 20, a causal backtracking module 30, and a data generation module 40.

[0109] The network construction module 10 is used to acquire multimodal traffic data and construct a dynamic semantic interaction network based on the multimodal traffic data. The dynamic semantic interaction network includes multiple traffic entities, the dynamic attributes of each traffic entity, and the spatiotemporal relationships between each traffic entity. The event labeling module 20 is used to label target events and core nodes of target events in the dynamic semantic interaction network according to a preset event-specific subgraph. The causal backtracking module 30 is used to perform causal chain backtracking on the labeled target events and core nodes of target events to obtain the structured causal chain data of the corresponding target events. The data generation module 40 is used to perform question-and-answer pair generation processing on the structured causal chain data of the corresponding target events to obtain the instruction fine-tuning dataset for training the large traffic model.

[0110] It should be understood that the specific limitations of the traffic scene training data generation device 100 can be found in the limitations of the traffic scene training data generation method described above, and will not be repeated here. Each module in the aforementioned traffic scene training data generation device 100 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module. To reduce redundancy, further details will not be provided here.

[0111] In summary, the traffic scene training data generation apparatus according to embodiments of the present invention acquires multimodal traffic data through a network construction module and constructs a dynamic semantic interaction network based on the multimodal traffic data. The dynamic semantic interaction network includes multiple traffic entities, dynamic attributes of each traffic entity, and spatiotemporal relationships between each traffic entity. Then, an event annotation module annotates target events and their core nodes within the dynamic semantic interaction network according to a preset event-specific subgraph. Next, a causal backtracking module performs causal chain backtracking on the annotated target events and their core nodes to obtain structured causal chain data for the corresponding target events. Finally, a data generation module performs question-and-answer pair generation processing on the structured causal chain data for the corresponding target events to obtain a fine-tuning instruction dataset for training a large-scale traffic model. Thus, it efficiently constructs a high-quality fine-tuning instruction dataset from multimodal traffic data, thereby achieving automated and high-quality generation of traffic scene training data and improving the model's logical reasoning ability and interpretability.

[0112] Based on the traffic scene training data generation method of the foregoing embodiments of the present invention, the present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a traffic scene training data generation method.

[0113] It should be understood that the specific implementation of the electronic device in the embodiments of the present invention can be found in the specific implementation of the traffic scene training data generation method in the foregoing embodiments of the present invention, and will not be repeated here to reduce redundancy.

[0114] In summary, the electronic device according to the embodiments of the present invention, by executing a computer program stored in the memory through a processor, can efficiently construct a high-quality instruction fine-tuning dataset from multimodal traffic data, thereby realizing the automated and high-quality generation of traffic scenario training data and improving the model's logical reasoning ability and interpretability.

[0115] Based on the traffic scene training data generation method of the foregoing embodiments of the present invention, the present invention also proposes a computer storage medium storing a computer program thereon, which implements a traffic scene training data generation method when executed by a processor.

[0116] It should be understood that the specific implementation of the computer storage medium in the embodiments of the present invention can be found in the specific implementation of the traffic scene training data generation method in the foregoing embodiments of the present invention, and will not be repeated here to reduce redundancy.

[0117] In summary, the computer-readable storage medium according to embodiments of the present invention, by executing the computer program stored thereon, enables the efficient construction of a high-quality instruction fine-tuning dataset from multimodal traffic data, thereby achieving automated and high-quality generation of traffic scenario training data and improving the model's logical reasoning ability and interpretability.

[0118] Specifically, based on the traffic scene training data generation method, apparatus, electronic device, and medium of the above embodiments of the present invention, a dynamic semantic interaction network is constructed based on multimodal traffic data, realizing a unified expression of multiple traffic entities, dynamic attributes, and their spatiotemporal relationships in traffic scenes. On this basis, it can accurately identify and extract target events and their core nodes that match preset event definitions. Furthermore, by performing causal backtracking based on edge relationships and temporal attributes between nodes, structured causal chain data is formed, ensuring that the generation process of traffic events not only includes the surface-level spatiotemporal features of the events but also contains deeper causal logical connections. After textualization, the generated causal chain data can be directly used to construct a fine-tuning dataset for traffic scene models. This dataset possesses logical consistency and causal interpretability, significantly improving the model's logical reasoning ability and interpretability in traffic scenes. Simultaneously, by embedding expert knowledge such as event definitions into the data generation process, the present invention achieves fully automated generation from semantic structure to high-quality training text, significantly reducing the workload of manual annotation and data screening, and improving the efficiency and consistency of large-scale traffic model training data production.

[0119] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0120] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0121] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0122] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0123] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A traffic scene training data generation method, characterized in that, The method comprises: acquiring multi-modal traffic data, and constructing a dynamic semantic interaction network according to the multi-modal traffic data, wherein the dynamic semantic interaction network comprises a plurality of traffic entities, dynamic attributes of each traffic entity and spatio-temporal relationships between the traffic entities; in the dynamic semantic interaction network, labeling a target event and a core node of the target event according to a preset event-specific subgraph; performing causal chain backtracking on the labeled target event and the core node of the target event to obtain structured causal chain data corresponding to the target event; performing question and answer pair generation processing on the structured causal chain data corresponding to the target event to obtain an instruction fine-tuning data set for training a traffic large model.

2. The traffic scene training data generation method of claim 1, wherein, The construction of the dynamic semantic interaction network according to the multi-modal traffic data comprises: acquiring semantic labels of the multi-modal traffic data; creating a plurality of nodes and a plurality of edges in the dynamic semantic interaction network according to the semantic labels, wherein each node is used to represent the plurality of traffic entities and the dynamic attributes of each traffic entity, and each edge is used to represent the spatio-temporal relationships between the traffic entities.

3. The traffic scene training data generation method of claim 1, wherein, The acquisition of the structured causal chain data corresponding to the target event comprises: acquiring an incoming edge of the core node of the target event, and acquiring all parent nodes connected to the incoming edge; causal relationship screening is performed on all parent nodes connected to the incoming edge in sequence; a causal fragment corresponding to each parent node is constructed according to each parent node passing the causal relationship screening, the core node of the target event and the incoming edge, and the causal fragment of each parent node is stored in a causal path list; after the causal chain backtracking ends, the structured causal chain data corresponding to the target event is acquired according to all causal fragments stored in the causal path list.

4. The traffic scene training data generation method of claim 3, wherein, The causal backtracking ends when any of the following conditions is met: the backtracked parent node matches a preset root event mode; the depth of backtracking exceeds a preset maximum depth threshold, or the time of backtracking exceeds a preset maximum time window; there is no node to be backtracked.

5. The traffic scenario training data generation method of claim 4, wherein, The acquisition of the structured causal chain data corresponding to the target event according to all causal fragments stored in the causal path list comprises: all causal fragments stored in the causal path list are reorganized in chronological order and logical order to output the structured causal chain data corresponding to the target event.

6. The traffic scene training data generation method of claim 5, wherein, The acquisition of the instruction fine-tuning data set for training the traffic large model comprises: based on the structured causal chain data corresponding to the target event, a causal reasoning question and answer pair generation rule and a compliance decision question and answer pair generation rule are respectively formed to form a causal reasoning question and answer pair and a compliance decision question and answer pair corresponding to the target event; the instruction fine-tuning data set for training the traffic large model is acquired according to the causal reasoning question and answer pair and the compliance decision question and answer pair corresponding to the target event.

7. The traffic scene training data generation method of claim 1, wherein, The method further comprises: the structured causal chain data corresponding to the target event is converted into text data containing natural language descriptions by a template filling method or a template generation method.

8. A traffic scene training data generation apparatus characterized by comprising: The device comprises: The network construction module is configured to acquire multi-modal traffic data, and construct a dynamic semantic interaction network according to the multi-modal traffic data, wherein the dynamic semantic interaction network comprises a plurality of traffic entities, dynamic attributes of each traffic entity and spatio-temporal relationships between the traffic entities; The event labeling module is configured to label a target event and core nodes of the target event according to a preset event-specific subgraph in the dynamic semantic interaction network. The causal backtracking module is configured to perform causal chain backtracking on the labeled target event and the core nodes of the target event, and acquire structured causal chain data corresponding to the target event. The data generation module is configured to perform question-answer pair generation processing on the structured causal chain data corresponding to the target event, and acquire instruction fine-tuning data set for training a traffic large model. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to realize the steps of the traffic scene training data generation method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the traffic scene training data generation method in any one of claims 1 to 7.

Images