Tree search enhanced automated vehicle compliance behavior induction method

Abstract

The application provides a kind of dendritic search enhanced automatic driving car compliance behavior induction method, belongs to the technical field of automatic driving, including 1) reading traffic regulations text, it is blocked according to paragraph structure, further subdivides using large language model to block result, obtains several traffic regulations clauses 2) constructs road traffic scene tree structure and cross-regulation clause scene semantic analysis model, forms traffic regulations knowledge base of cross-regulation clause-element coding form 3) based on driving data extraction scene element, obtain the node distribution of each element in tree structure and build query representation, match the corresponding coding clause in cross-regulation knowledge base 4) constructs behavior constraint thinking chain, generates prohibited behavior list and mandatory behavior list, guides the compliance driving of automatic driving vehicle.The application constructs road traffic scene tree structure, proposes tree search algorithm, can simultaneously retrieve universal regulations and targeted regulations, improves the accuracy of traffic regulations retrieval and behavior constraint generation.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and in particular relates to a tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles. Background Technology

[0002] Compliant driving is a crucial prerequisite for the integration and large-scale application of autonomous vehicles into transportation systems. Traditional autonomous vehicles employ rule-based algorithms, summarizing human driving experience to design decision-making logic and supporting manually written rules to comply with traffic regulations. However, existing autonomous driving systems use data-driven algorithms, leveraging end-to-end models with strong fitting capabilities to directly learn the mapping relationship between sensor data and control signals. The internal mechanisms of these models are difficult to analyze, making compliance challenging. Therefore, there is an urgent need to accurately retrieve scene-related traffic regulations and summarize vehicle behavior constraints to support the training and optimization of data-driven algorithms.

[0003] Existing retrieval technologies are mainly based on the similarity relationship between query representations and knowledge documents. They use keywords such as "Li B, Wang Y, Mao J, et al. Driving everywhere with large language model policy adaptation[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition.2024:14948-14957" or natural language semantic features such as "Lewis P, Perez E, Piktus A, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks[J]. Advances in neural information processing systems,2020,33:9459-9474" to measure similarity and select knowledge documents with high similarity from the knowledge base as retrieval results.

[0004] However, driving scenarios are diverse and dynamic, and the hierarchical relationships between scenario elements are complex. Measuring only the similarity between the current scenario and the regulatory scenario can easily lead to omissions in universal regulations, resulting in incomplete summarization of vehicle behavior constraints. Summary of the Invention

[0005] The purpose of this invention is to solve the problems of low accuracy in traffic regulation retrieval and incomplete summarization of behavioral constraints by providing a tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles. The method comprises the following steps:

[0006] S1: Read the traffic regulations text, divide it into blocks according to paragraph structure, use a large language model to further subdivide the block results, obtain several traffic regulations clauses, and ensure that each clause corresponds to a single road traffic scenario.

[0007] S2: Referring to the six-layer model of scenario classification of roads-infrastructure-traffic management-traffic participants-environment-information, a tree structure of road traffic scenarios is constructed to represent the elements of each type of scenario and their hierarchical relationships. The full-path coding method is applied to generate the code corresponding to each element. The large language model prompt word fine-tuning technology is used to construct a semantic parsing model of traffic regulations clauses, automatically annotate the scenario elements associated with each clause, obtain the corresponding element codes, and form a traffic law knowledge base in the form of traffic regulations clauses-element codes.

[0008] S3: Combine vehicle location information with high-precision maps to extract static scene elements, and use detection algorithms to identify dynamic scene elements from vehicle-side perception data. Obtain the node distribution of each element in the tree structure, and use the element codes of the current node and its ancestor nodes to construct a query representation and match the corresponding coded clauses in the traffic regulations knowledge base.

[0009] S4: Construct a behavioral constraint mind chain, determine the type of behavioral constraint in each search clause, and extract the behavioral description text; summarize and generate a list of prohibited behaviors and a list of mandatory behaviors according to the type of behavioral constraint to guide autonomous vehicles to drive in compliance with regulations.

[0010] Furthermore, in S1, the paragraph structure includes line breaks and special identifiers for legal provisions (such as "Article 1").

[0011] Furthermore, in S1, the further subdivision is as follows: through prompt word engineering, the large language model determines whether the given text contains multiple road traffic scenarios. If it does, it is further subdivided; otherwise, no processing is done, ultimately ensuring that each clause corresponds to only a single traffic scenario.

[0012] Furthermore, the six-layer scene classification model includes: ISO 34504, ASAM OpenDRIVE, and ASAMOpenSCENARIO files.

[0013] Furthermore, S2 specifically includes the following steps:

[0014] S21: Use a multi-branch tree to represent the hierarchical relationship between different elements. The nodes of the multi-branch tree correspond one-to-one with the scene elements. The deeper the node in the multi-branch tree, the lower the level of the corresponding scene element and the finer the granularity of the scene information it contains.

[0015] S22: Use the full path encoding method to assign independent codes to each scene element in the S21 tree;

[0016] S23: Initialize trainable soft cue words for each scene element. Use the soft cue words, scene element name, and traffic regulation text as input signals to the large language model, and use whether the traffic regulation contains the element (i.e., "yes" or "no") as the output signal. Fine-tune the end-to-end learning of the soft cue words corresponding to each scene element using the cue words, and train the objective function, which is expressed as:

[0017]

[0018] p = f Θ (y|S i N i ,x j ))

[0019]

[0020] Where, x j Let N be the text of the j-th traffic regulation clause; i Let i be the name of the i-th scene element; This is the corresponding soft suggestion word, where n is the predefined soft suggestion word length, set to 20, and d is the embedding layer dimension of the large language model; f Θ (·) represents a pre-trained large language model with weights Θ; y ij Let j be the truth value of the j-th traffic regulation clause at the i-th scene element; To predict the probability distribution of the next token for a large language model, where, The size of the vocabulary is represented by CE(·), which is the cross-entropy loss function.

[0021] S24: After the objective function in S23 is trained, given a traffic regulation clause, perform model inference node by node to obtain all scene elements associated with the traffic regulation clause and read its corresponding code;

[0022] S25: Repeat S24 until all traffic regulations are marked, ultimately forming a structured traffic regulations knowledge base with traffic regulations clause-element codes.

[0023] Furthermore, in S2, the full path encoding method is specifically: a tree search path starting from the root node and ending at the current node.

[0024] Furthermore, S3 includes the following steps:

[0025] S31: Read road information of the vehicle's location from the high-precision map and extract static scene feature elements, including road type, facility type, lane type, lane right-of-way, lane width, horizontal alignment, longitudinal profile features, number of one-way lanes, changes in the number of lanes ahead, road traffic direction, surrounding built environment, traffic signs, traffic markings, and the setting of central road dividers.

[0026] S32: Based on the vehicle's forward video, the detection algorithm is used to identify weather conditions, visibility levels, and surrounding traffic participants in real time to obtain dynamic scene elements;

[0027] S33: Obtain the node distribution of each element in the tree structure, extract the element codes of each node and its ancestor nodes, and combine them to obtain the query representation;

[0028] S34: Traverse each clause in the traffic regulations knowledge base in S2. If the combination of scenario elements corresponding to the clause belongs to the query representation, it will be included in the search results, and finally a complete list of traffic regulations will be formed.

[0029] Furthermore, in S32, the detection algorithm includes: using the YOLO algorithm to identify surrounding traffic participants, and using a convolutional neural network to identify weather conditions and estimate visibility levels.

[0030] Furthermore, S4 specifically includes the following steps:

[0031] S41: Given the text of traffic regulations, construct a large language model thinking chain reasoning prompt, determine the type of behavioral constraints contained in the clause, and extract vehicle behavior descriptions based on this.

[0032] S42: Repeat S41 to process and retrieve traffic regulations one by one;

[0033] S43: According to the type of behavioral constraint, summarize the description text of prohibited behaviors and mandatory behaviors respectively, remove duplicate or synonymous behavior descriptions, and generate a list of prohibited behaviors and a list of mandatory behaviors.

[0034] Furthermore, in S41, the types of behavioral constraints include mandatory behaviors, prohibited behaviors, and authorized behaviors.

[0035] Compared with the prior art, the beneficial effects of the present invention are mainly reflected in:

[0036] 1. This invention fills the technical gap in accurately extracting vehicle behavior constraints in dynamic driving scenarios. It can be used to guide autonomous vehicles to drive in compliance with regulations, support automakers in optimizing autonomous driving algorithms, and is of great significance to promoting the development of autonomous driving technology.

[0037] 2. This invention utilizes large language model prompt word fine-tuning technology to automatically parse the scene semantics of traffic regulations and label their related elements, forming a traffic regulations knowledge base that can support the digital modeling and knowledge base construction of traffic regulations in different countries / regions.

[0038] 3. This invention constructs a tree structure for road traffic scenarios and proposes a tree-based retrieval algorithm that can simultaneously retrieve general and specific regulations, thereby improving the accuracy of traffic regulation retrieval.

[0039] 4. This invention applies the large language model thinking chain reasoning technology to extract the constraint types of traffic regulations and vehicle behavior, and generate a list of prohibited behaviors and a list of mandatory behaviors, providing refined compliant driving information for autonomous vehicles. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating a tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to the present invention.

[0041] Figure 2 This is the overall framework and implementation method of the present invention. Detailed Implementation

[0042] The following will describe in more detail a tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles, with reference to the schematic diagrams, which illustrate preferred embodiments of the invention. It should be understood that those skilled in the art can modify the invention described herein while still achieving its advantageous effects. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0043] Example

[0044] In the tree-structured search-enhanced method for summarizing compliant behaviors of autonomous vehicles, traffic regulations include: the Road Traffic Safety Law of the People's Republic of China, the Implementing Regulations of the Road Traffic Safety Law of the People's Republic of China, the Administrative Measures for Points Deduction of Road Traffic Safety Violations, Road Traffic Signs and Markings (GB5768.3), the Beijing Municipality Implementation Measures of the Road Traffic Safety Law of the People's Republic of China, and the Shanghai Municipality Road Traffic Management Regulations, as well as other national and local traffic regulations.

[0045] like Figure 1 As shown, the specific steps include:

[0046] Step 1:

[0047] 1.1) Divide the traffic regulations document text into blocks according to paragraph structure. The paragraph structure includes line breaks, special identifiers for the regulations (such as "Article 1"), etc.

[0048] 1.2) Utilize a large language model to subdivide the text into several clauses. Through cue word engineering, the large language model determines whether a given text contains multiple road traffic scenarios. If it does, it is further subdivided; otherwise, no processing is performed. Ultimately, this ensures that each clause corresponds to only a single traffic scenario.

[0049] Step 2:

[0050] 2.1) Use multi-branch trees to represent the hierarchical relationships between different elements. In this method, each node in the multi-branch tree corresponds one-to-one with a scene element. The deeper the node in the multi-branch tree, the lower the level of the corresponding scene element, and the finer the granularity of the scene information it contains.

[0051] 2.2) Use the full path encoding method to assign an independent code to each scene element in the tree in step 2.1.

[0052] 2.3) Initialize trainable soft cue words for each scene element. Use "soft cue word, scene element name, and traffic regulation text" as input signals to the large language model, and "whether the traffic regulation contains this element" (i.e., "yes" or "no") as the label (output signal). Fine-tune the end-to-end learning of the soft cue words corresponding to each scene element using these cue words. The training objective function is:

[0053]

[0054] p = f Θ (y|S i N i ,x j ))

[0055]

[0056] Where, x j Let N be the text of the j-th traffic regulation clause; i Let i be the name of the i-th scene element; This is the corresponding soft suggestion word, where n is the predefined soft suggestion word length, set to 20, and d is the embedding layer dimension of the large language model; f Θ (·) represents a pre-trained large language model with weights Θ; y ij Let j be the truth value of the j-th traffic regulation clause at the i-th scene element; To predict the probability distribution of the next token for a large language model, where, The size of the vocabulary is represented by CE(·), which is the cross-entropy loss function.

[0057] 2.4) After step 2.3 training is completed, given a traffic regulation clause, perform model inference node by node to obtain all scene elements associated with the traffic regulation clause and read its corresponding code.

[0058] 2.5) Repeat step 2.4 until all traffic regulations are marked, and finally form a structured traffic regulations knowledge base in the form of "traffic regulations clauses - element codes".

[0059] In this embodiment, a total of 310 traffic regulations were parsed and encoded, forming a traffic regulations knowledge base. Example data for the traffic regulations knowledge base is shown in Table 1.

[0060] Table 1. Examples of Traffic Regulations Knowledge Base

[0061]

[0062] Step 3:

[0063] 3.1) Read the road information of the vehicle's location from the high-precision map and extract the static scene feature elements, including road type, facility type, lane type, lane exclusive right, lane width, horizontal alignment, longitudinal profile features, number of one-way lanes, changes in the number of lanes ahead, road traffic direction, surrounding built environment, traffic signs, traffic markings, and the setting of road central isolation facilities.

[0064] 3.2) Based on the vehicle's forward video, a detection algorithm is used to identify weather conditions, visibility levels, and surrounding traffic participants in real time to obtain dynamic scene elements.

[0065] 3.3) Obtain the node distribution of each element in the tree structure, extract the element codes of each node and its ancestor nodes, and combine them to obtain the query representation.

[0066] 3.4) Traverse all clauses in the traffic regulations knowledge base. If the combination of scenario elements corresponding to a clause belongs to the query representation, it will be included in the search results, and finally a complete list of traffic regulations will be formed.

[0067] In this embodiment, by Figure 2 The driving scene shown has a total of 13 scene elements extracted (such as...).

[0068] As shown in Table 2), a total of 18 traffic regulations were extracted using a tree-structured retrieval algorithm (as shown in Table 3).

[0069]

[0070] Table 2. Examples of Scene Element Extraction

[0071]

[0072] Table 3. Examples of Traffic Regulations Search Results (Partial)

[0073] Step 4:

[0074] 4.1) Given the text of traffic regulations, construct a large language model thinking chain reasoning prompt words, determine the type of behavioral constraints contained in the clause, and extract vehicle behavior descriptions based on this.

[0075] 4.2) Repeat step 4.1 to process the retrieved traffic regulations one by one.

[0076] 4.3) According to the type of behavioral constraint, summarize the description text of prohibited behaviors and mandatory behaviors respectively, remove duplicate or synonymous behavior descriptions, and generate a list of prohibited behaviors and a list of mandatory behaviors.

[0077] In this embodiment, the example driving scenario includes a total of 7 prohibited behaviors and 11 mandatory behaviors (as shown in Table 4), which provides support for guiding compliance decisions of autonomous vehicles.

[0078]

[0079] Table 4. Examples of Behavioral Constraint Summarization

[0080] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles, characterized in that, Includes the following steps: S1: Read the traffic regulations text, divide it into blocks according to paragraph structure, use a large language model to further subdivide the block results, obtain several traffic regulations clauses, and ensure that each clause corresponds to a single road traffic scenario. S2: Referring to the six-layer model of scenario classification of roads-infrastructure-traffic management-traffic participants-environment-information, a tree structure of road traffic scenarios is constructed to represent the elements of each type of scenario and their hierarchical relationships. The full-path coding method is applied to generate the code corresponding to each element. The large language model prompt word fine-tuning technology is used to construct a semantic parsing model of traffic regulations clauses, automatically annotate the scenario elements associated with each clause, obtain the corresponding element codes, and form a traffic law knowledge base in the form of traffic regulations clauses-element codes. S3: Combine vehicle location information with high-precision maps to extract static scene elements, and use detection algorithms to identify dynamic scene elements from vehicle-side perception data. Obtain the node distribution of each element in the tree structure, and use the element codes of the current node and its ancestor nodes to construct a query representation and match the corresponding coded clauses in the traffic regulations knowledge base. S4: Construct a behavioral constraint mind chain, determine the type of behavioral constraint in each search clause, extract the behavioral description text, and summarize and generate a list of prohibited behaviors and a list of mandatory behaviors according to the type of behavioral constraint to guide autonomous vehicles to drive in compliance with regulations.

2. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 1, wherein in step S1, the paragraph structure includes line breaks and special identifiers for regulatory clauses.

3. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 1, characterized in that, In S1, the further subdivision is as follows: through prompt word engineering, the large language model determines whether the given text contains multiple road traffic scenarios. If it does, it is further subdivided; otherwise, no processing is performed, ultimately ensuring that each clause corresponds to only a single traffic scenario.

4. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 1, characterized in that, The six-layer model for scene classification includes: ISO 34504, ASAM OpenDRIVE, and ASAM OpenSCENARIO files.

5. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 1, characterized in that, S2 specifically includes the following steps: S21: Use a multi-branch tree to represent the hierarchical relationship between different elements. The nodes of the multi-branch tree correspond one-to-one with the scene elements. The deeper the node in the multi-branch tree, the lower the level of the corresponding scene element and the finer the granularity of the scene information it contains. S22: Use the full path encoding method to assign independent codes to each scene element in the S21 tree; S23: Initialize trainable soft cue words for each scene element. Use the soft cue words, scene element name, and traffic regulation text as input signals to the large language model, and use whether the traffic regulation contains the element as the output label. Fine-tune the end-to-end learning of the soft cue words corresponding to each scene element using the cue words, and train the objective function, which is expressed as: ； ； ； in, For the first The text of each traffic regulation clause; For the first The names of the scene elements; These are the corresponding soft suggestion words, among which, The predefined soft suggestion word length is set to 20. For large language model embedding layer dimension; For the pre-trained weights Large language models; For the first The traffic regulations clause is in the [number]th [section]. The truth value at each scene element; To predict the probability distribution of the next token for a large language model, where, The size of the vocabulary list; The cross-entropy loss function; S24: After the objective function in S23 is trained, given a traffic regulation clause, perform model inference node by node to obtain all scene elements associated with the traffic regulation clause and read its corresponding code; S25: Repeat S24 until all traffic regulations are marked, ultimately forming a structured traffic regulations knowledge base with traffic regulations clause-element codes.

6. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 1, characterized in that, In S2, the full path encoding method specifically refers to a tree search path that starts from the root node and ends at the current node.

7. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 1, characterized in that, S3 includes the following steps: S31: Read road information of the vehicle's location from the map and extract static scene feature elements, including road type, facility type, lane type, lane right-of-way, lane width, horizontal alignment, longitudinal profile features, number of one-way lanes, changes in the number of lanes ahead, road traffic direction, surrounding built environment, traffic signs, traffic markings, and the setting of central road dividers. S32: Based on the vehicle's forward video, the detection algorithm is used to identify weather conditions, visibility levels, and surrounding traffic participants in real time to obtain dynamic scene elements; S33: Obtain the node distribution of each element in the tree structure, extract the element codes of each node and its ancestor nodes, and combine them to obtain the query representation; S34: Traverse each clause in the traffic regulations knowledge base in S25. If the combination of scenario elements corresponding to the clause belongs to the query representation, it will be included in the search results, and finally a complete list of traffic regulations will be formed.

8. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 7, characterized in that, In step S32, the detection algorithm includes: using the YOLO algorithm to identify surrounding traffic participants, and using a convolutional neural network to identify weather conditions and estimate visibility levels.

9. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 1, characterized in that, S4 specifically includes the following steps: S41: Given the text of traffic regulations, construct a large language model thinking chain reasoning prompt, determine the type of behavioral constraints contained in the clause, and extract vehicle behavior descriptions based on this. S42: Repeat S41 to process and retrieve traffic regulations one by one; S43: According to the type of behavioral constraint, summarize the description text of prohibited behaviors and mandatory behaviors respectively, remove duplicate or synonymous behavior descriptions, and generate a list of prohibited behaviors and a list of mandatory behaviors.

10. The tree-structured retrieval-enhanced method for summarizing compliant behaviors of autonomous vehicles according to claim 9, characterized in that, In S41, the types of behavioral constraints include mandatory behaviors, prohibited behaviors, and authorized behaviors.

Images