Method and system for traffic scene understanding based on logical symbol tracing

Abstract

The application provides a traffic scene understanding method and system based on logical symbol traceability, which converts natural language texts related to traffic scenes and to-be-judged query propositions into first-order logical symbol representations, and generates corresponding logical premise sets and query propositions. The logical premise sets in underdetermined scenes are completed with implicit rules, and then the determinable logical premise sets are converted into initial posterior distributions of large language model agents. An active exploration is carried out by using a posterior sampling reinforcement learning framework, the posterior distribution is updated according to the environment feedback, and the ability of the agent to continuously refine the understanding of the traffic scene and generate vehicle control instructions in the interaction is driven. In the scheme, by designing a task-driven logical symbolization mechanism, an implicit rule completion strategy in an underdetermined scene and an active interaction framework based on the posterior distribution, the depth of the understanding of the natural language traffic scene by the automatic driving system, the reasoning rigor and the decision robustness are fundamentally improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically, to a traffic scene understanding method and system based on logical symbolic reasoning. Background Technology

[0002] As autonomous driving technology evolves from the perception layer to the decision-making layer, the vehicle's semantic understanding and safety decision-making capabilities in complex traffic scenarios have become core bottlenecks restricting the large-scale deployment of advanced autonomous driving. Traditional autonomous driving decision-making systems mainly rely on rule-based finite state machines or learning-based end-to-end models. While rule-based methods offer good interpretability and logical rigor, they depend on manually enumerating all possible traffic scenarios and interaction rules. When facing open, dynamic, and uncertain real-world traffic environments, the completeness and consistency of the rule base are difficult to guarantee, easily leading to decision failures due to missing or conflicting rules. Learning-based methods train neural network strategies using large-scale driving data, demonstrating strong generalization capabilities in conventional scenarios. However, their decision-making process lacks a transparent reasoning chain, and its unpredictability and lack of interpretability in safety-critical scenarios pose significant safety hazards, making it difficult to pass functional safety certification.

[0003] In recent years, the emergence of Large Language Models (LLMs) has opened up new paths for autonomous driving decision-making. LLMs learn rich world knowledge, common-sense reasoning abilities, and semantic generalization capabilities from massive text corpora, enabling them to understand complex traffic scenarios described in natural language and generate reasonable driving behavior suggestions. However, existing LLM-driven autonomous driving methods still suffer from three key limitations. First, insufficient coupling between symbolic reasoning and neural networks: existing methods often treat LLMs as direct action generators, failing to effectively integrate the semantic understanding capabilities of LLMs with the rigorous verifiability of symbolic logic. This leads to the possibility of common-sense logical fallacies when faced with precise geometric constraints or traffic regulations. Second, lack of knowledge underdeterminacy and abductive reasoning: Natural language descriptions of real-world traffic scenarios are often sparse and incomplete, with many implicit preconditions (such as unstated traffic conventions or implicit right-of-way rules) missing. Existing LLM methods typically reason directly based on given text, lacking an abductive reasoning mechanism to proactively fill in missing common-sense information, resulting in a significant decrease in accuracy in underdetermined problems. Third, there is a disconnect between passive perception and active exploration: current LLM autonomous driving agents are mostly based on a passive response architecture, meaning they receive sensory input and then output decisions, lacking the ability to quantitatively model environmental uncertainties and actively explore them. The agents cannot continuously update their belief distribution regarding the scene during interactions, nor can they eliminate uncertainties in critical states through proactive information gathering.

[0004] Meanwhile, in the field of neural symbolic fusion technology, existing work mostly focuses on injecting symbolic knowledge as constraints into neural network training or using neural networks to extract symbolic rules. However, few methods organically unify LLM's natural language understanding, symbolic logic satisfiability determination, and Bayesian posterior sampling into a closed-loop framework. Especially when processing natural language descriptions of traffic scenarios, systematic technical solutions are still lacking in how to accurately convert unstructured text into first-order logic formulas, how to use LLM to generate candidate common sense rules and rigorously screen them when logic is undecidable, and how to transform logical reasoning conclusions into the agent's prior beliefs to guide active exploration.

[0005] In summary, existing technologies face three core disconnects in the field of LLM-driven autonomous driving decision-making: first, the disconnect between LLM semantic understanding and symbolic logic verification results in decisions lacking both the flexibility of neural networks and the rigor of symbolic systems; second, the disconnect between given knowledge and implicit common sense prevents the system from autonomously completing the reasoning chain when information is incomplete; and third, the disconnect between static reasoning and dynamic exploration prevents the agent from eliminating uncertainty in scene cognition through interaction. Therefore, there is an urgent need for an integrated method that can tightly couple LLM semantic parsing, symbolic abductive reasoning, and Bayesian active exploration. Summary of the Invention

[0006] The purpose of this invention is to provide a traffic scene understanding method and system based on logical symbolic abduction, so as to improve the understanding depth, reasoning rigor and decision robustness of autonomous driving systems in natural language traffic scenes.

[0007] In a first aspect, the present invention provides a traffic scene understanding method based on logical symbolic reasoning, the method comprising: The natural language text related to traffic scenarios and the query propositions to be decided are converted into first-order logic symbolic representations, and the corresponding logical premise set and query propositions are generated. In each iteration, the logical skeleton in the logical premise set is extracted, and the logical text in the logical skeleton is used as the antecedent to generate candidate implicit rules through a large language model. The candidate implicit rules are then filtered and added to the logical premise set. The iteration continues until the obtained logical premise set and query proposition meet the judgment requirements. Based on the logical premise set and query propositions obtained through iteration, the posterior distribution of the big language model agent is initialized. In each round of sampling, the agent samples a hypothetical environment model from the current posterior distribution and generates a decision action for the current traffic state based on the hypothetical environment model through reasoning of the big language model. After the agent executes the decision action, it obtains the current round trajectory and updates the historical interaction trajectory and posterior distribution until the sampling stopping condition is met. Based on the obtained decision action, it generates a sequence of driving behavior planning instructions and outputs the accumulated round trajectory set of all rounds and the finally converged posterior distribution.

[0008] In an optional implementation, the logical premise set is a collection of multiple first-order logic formulas, each of which consists of entity constants, variables, predicates, quantifiers, and logical connectors, representing a fact or rule extracted from the natural language text. The query proposition is a single first-order logic formula, representing the target to be determined obtained by transforming the query proposition to be determined. The query proposition is one of attribute query, relation query, event prediction query, and behavior rationality query.

[0009] In an optional implementation, the step of generating candidate implicit rules using the logical text in the logical skeleton as a precondition through a large language model includes: For each pair of characters in the logical text of the logical skeleton, a priority score is obtained for each pair of characters based on the degree of relevance between each pair of characters and other characters in the logical text. Each pair of texts is traversed in descending order of priority score. Each pair of texts is used as the antecedent prompt in the large language model to generate the corresponding consequent text, thus obtaining the corresponding candidate implicit rules.

[0010] In an optional implementation, the step of filtering the candidate implicit rules and adding them to the logical premise set includes: The candidate implicit rules are evaluated based on the common sense compliance and scenario relevance of the settings; The candidate implicit rules are filtered based on the evaluation results and added to the logical premise set.

[0011] In an optional implementation, the step of evaluating the candidate implicit rules based on the set common sense compliance and scenario relevance includes: The logical values ​​of the candidate implicit rules in the common sense dimension are output using a large language model, and the corresponding common sense conformity score is obtained based on the logical values ​​in the common sense dimension. The logical values ​​of the candidate implicit rules and the current traffic scenario are evaluated based on the large language model, and the corresponding scenario relevance score is obtained based on the logical values ​​of the relevant dimensions. The evaluation result is obtained based on the common sense conformity score and the scenario relevance score.

[0012] In an optional implementation, the step of initializing the posterior distribution of the large language model agent based on the iteratively obtained logical premise set and query propositions includes: The logical premise set and query propositions obtained through iteration are encoded into natural language statements to form a structured text description; The structured text description is converted into an initial posterior distribution for a large language model agent, which is used to describe the agent's perception of the current state of the traffic scene.

[0013] In an optional implementation, the step of generating a decision action for the current traffic state based on the hypothetical environment model through large language model reasoning includes: The value of each candidate action is estimated using context learning based on the reward function and transition function in the hypothetical environment model through a large language model. Based on the value of each candidate action, a decision action is generated for the current traffic situation.

[0014] In an optional implementation, the step of the agent obtaining the current round trajectory and updating the historical interaction trajectory and posterior distribution after executing the decision action includes: After the agent executes the decision action, it observes the corresponding reward and the next state from the real environment to form the experience tuple for the current time step. After the current round ends, the complete round trajectory is obtained based on the experience tuple. The current round trajectory is then merged with the historical interaction trajectory to obtain the updated historical interaction trajectory. An updated posterior distribution is generated based on the current posterior distribution and the current round trajectory.

[0015] In an optional implementation, the posterior distribution includes states, actions, a reward function, and a transition function; The step of generating an updated posterior distribution based on the current posterior distribution and the current round trajectory includes: Extract the empirical tuple for each time step from the current round trajectory, and evaluate the consistency of the empirical tuple with the uncertainties in the current posterior distribution; The confidence levels of the reward function and the transition function are adjusted based on the consistency to obtain the updated posterior distribution.

[0016] Secondly, the present invention provides a traffic scene understanding system based on logical symbolic reasoning, the system comprising: The symbolization module is used to convert traffic-related natural language text and query propositions to be determined into first-order logical symbolic representations, and generate corresponding logical premise sets and query propositions; The completion module is used to extract the logical skeleton from the logical premise set in each iteration, use the logical text in the logical skeleton as antecedent to generate candidate implicit rules through a large language model, filter the candidate implicit rules and add them to the logical premise set, and iterate until the obtained logical premise set and query proposition meet the judgment requirements. The reasoning module is used to initialize the posterior distribution of the big language model agent based on the logical premise set and query propositions obtained through iteration. In each round of sampling, the agent samples a hypothetical environment model from the current posterior distribution and generates a decision action for the current traffic state based on the hypothetical environment model through reasoning of the big language model. The understanding and reasoning module is further used to obtain the current round trajectory after the agent executes the decision action, and update the historical interaction trajectory and posterior distribution until the sampling stopping condition is met. Based on the obtained decision action, it generates a sequence of driving behavior planning instructions and outputs the set of round trajectories accumulated over all rounds and the finally converged posterior distribution.

[0017] This invention provides a traffic scene understanding method and system based on logical symbolic abduction. It converts traffic scene-related natural language text and undecided query propositions into first-order logical symbolic representations, generating corresponding logical premise sets and query propositions. For underdetermined scenarios, it performs implicit rule completion on the logical premise set, then transforms the decidable logical premise set into the initial posterior distribution of a large language model agent. A posterior sampling reinforcement learning framework is used for active exploration, updating the posterior distribution based on environmental feedback, driving the agent to continuously refine its understanding of traffic scenes and its ability to generate vehicle control commands through interaction. This solution, by designing a task-driven logical symbolization mechanism, an implicit rule completion strategy for underdetermined scenarios, and an active interaction framework based on posterior distribution, fundamentally improves the depth of understanding, reasoning rigor, and decision robustness of autonomous driving systems regarding natural language traffic scenes. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating the traffic scene understanding method based on logical symbolic reasoning provided in this embodiment of the invention; Figure 2 for Figure 1 A flowchart of the sub-steps included in S12; Figure 3for Figure 1 Another flowchart of the sub-steps included in S12; Figure 4 for Figure 1 A flowchart of the sub-steps included in S13; Figure 5 for Figure 1 Another flowchart of the sub-steps included in S13; Figure 6 for Figure 1 A flowchart of the sub-steps included in S14; Figure 7 This is a functional block diagram of a traffic scene understanding system based on logical symbolic reasoning, provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0021] Please see Figure 1 The above is a flowchart of a traffic scene understanding method based on logical symbol abduction provided in an embodiment of the present invention. This traffic scene understanding method based on logical symbol abduction can be executed by a traffic scene understanding system based on logical symbol abduction. This traffic scene understanding system based on logical symbol abduction can be implemented by software and / or hardware and can be configured in an electronic device, such as a computer device or a server. The detailed steps of the traffic scene understanding method based on logical symbol abduction are described below.

[0022] S11, convert the natural language text related to the traffic scenario and the query proposition to be determined into a first-order logic symbol representation, and generate the corresponding logical premise set and query proposition; S12, In each iteration, extract the logical skeleton from the logical premise set, use the logical text in the logical skeleton as antecedent to generate candidate implicit rules through the large language model, filter the candidate implicit rules and add them to the logical premise set, iterate until the obtained logical premise set and query proposition meet the judgment requirements. S13, Based on the logical premise set and query proposition obtained through iteration, the posterior distribution of the big language model agent is initialized. In each round of sampling, the agent samples a hypothetical environment model from the current posterior distribution and generates a decision action for the current traffic state based on the hypothetical environment model through reasoning of the big language model. S14, after the agent executes the decision action, it obtains the current round trajectory and updates the historical interaction trajectory and posterior distribution until the sampling stopping condition is met. Based on the obtained decision action, it generates a driving behavior planning instruction sequence and outputs the accumulated round trajectory set of all rounds and the finally converged posterior distribution.

[0023] The traffic scene understanding method based on logical abduction provided in this embodiment achieves end-to-end processing from natural language scene description to data-driven efficient driving decision-making by designing a task-driven logical symbolization mechanism, a common sense completion strategy for underdetermined scenarios, and an active interaction framework based on posterior beliefs. This fundamentally improves the autonomous driving system's understanding depth, reasoning rigor, and decision robustness in natural language traffic scenes.

[0024] The specific implementation methods of each of the above steps will be explained in detail below.

[0025] First, we receive two types of input data: the first type is natural language text T representing the traffic scenario, and the second type is query propositions related to the traffic scenario that need to be decided. Natural language text T contains declarative descriptions of traffic participants, road structure, environmental conditions, and dynamic events, and queries on propositions. This is a target statement that needs to be determined for the current traffic scenario. Employing a semantic parser based on a large language model Natural language text T and Convert to first-order logic symbol representation. Parser Guided by a small number of sample prompts, the examples include natural language statements and their corresponding first-order logical forms, covering a variety of typical traffic scenarios, thus enhancing the parser's generalization ability. Define a finite set of entities ,in Represents dynamic traffic participant entities, including the ego of the vehicle itself and other vehicles. non-motorized vehicles ,pedestrian Each participant entity carries a unique identifier and category label; Represents static environment entities, including lanes. Intersection Traffic lights Stop line ,pedestrian crossing ; It represents abstract conceptual entities, including speed, acceleration, distance, direction, and time.

[0026] Define the set of predicate symbols Each predicate represents a relationship between entities or between entity attributes. Unary attribute predicates include: IsVehicle(x) indicates that entity x belongs to the vehicle category, IsPedestrian(x) indicates that entity x belongs to the pedestrian category, IsRed(y) indicates that traffic light y is currently red, IsGreen(y) indicates that traffic light y is currently green, and IsStopped(z) indicates that entity z is currently stationary. Binary relation predicates include: InFrontOf(x,y) indicates that entity x is in front of entity y, Behind(x,y) indicates that entity x is behind entity y, LeftOf(x,y) indicates that entity x is to the left of entity y, RightOf(x,y) indicates that entity x is to the right of entity y, and DistanceLess. This indicates that the distance between entity x and entity y is less than a threshold. VelocityGreaterThan(x,v) indicates that the speed of entity x is greater than the value v, Following(x,y) indicates that entity x is following entity y, and Occupies(x,l) indicates that entity x is currently occupying lane l. The ternary predicates include: Between(x,y,z) indicates that entity x is located between entity y and entity z, Approaching(x,y,t) indicates that entity x is approaching entity y with the remaining time t, and YieldRequired(x,y) indicates that entity x needs to perform a yield operation on entity y. First-order logic formulas allow the use of universal quantifiers. With existential quantifiers Universal quantifiers are used to express rules that apply to all members of a certain entity class, while existential quantifiers are used to express the existence of at least one entity that satisfies a specific attribute. Logical connectives include conjunction (logical AND), disjunction (logical OR), implication (if...then...), and negation (logical NOT), and all connectives follow the semantic definitions of standard first-order logic. First-order logic formulas can be transformed into equivalent propositional logic formulas by instantiating each predicate F(x) and entity A as propositional variables, and... Expand as ,Will Expand as . parser The output is defined as ,in, The logical premise set is a collection of multiple first-order logic formulas. Each first-order logic formula consists of entity constants, variables, predicates, quantifiers, and logical connectors, representing a fact or rule extracted from natural language text. Q is the query proposition, which is a single first-order logic formula representing the target to be decided obtained from the query proposition to be decided. The query proposition can be one of the following: attribute query, relation query, event prediction query, and behavior rationality query.

[0027] For each statement in the natural language text T The parser identifies the subject, predicate, object, and modifiers in a sentence, maps them to corresponding entities and predicates, and combines them into logical formulas based on the syntactic structure. For complex sentences containing conditional structures (such as "if...then..."), the parser identifies their logical structure and generates the corresponding implication; for complex sentences containing parallel structures, the parser breaks them down into multiple independent formulas and connects them by conjunction.

[0028] The generated logical premise set Based on information type, it is divided into three subsets: the fact subset. It contains declarative descriptions of directly observable facts in the current scene, each fact being an atomic formula without verbs or logical connectors. (Rule subset) Includes general traffic rules or common sense knowledge, typically expressed using implied quantifiers. (Constraint subset) Includes numerical constraint boundary conditions on scene parameters. The query proposition Q is a first-order logic formula, and its form depends on the requirements of the downstream task. In traffic scene understanding tasks, the query proposition Q typically takes one of the following forms: Attribute query, such as Determine whether the vehicle can safely turn at this time; Relationship lookup, such as Determine the right-of-way relationship between the vehicle and vehicle 1; Event prediction queries, such as Determine whether there is a collision risk within the time window T; Behavioral reasonableness query, such as Determine whether a vehicle should yield to a pedestrian. The final output is a structured triple. ,in For containing a subset of facts Rule subset and constraint subsets The set of logical premises, where Q is the query proposition. This is a lexical mapping table that records the correspondence between each entity constant, predicate symbol, and original natural language fragment. It is used to translate logical symbols back into natural language when necessary for human understanding. The output triples serve as input for subsequent steps, including SAT solving, logical skeleton extraction, and implicit rule generation.

[0029] The traffic scenario can also be defined as a first-order logical structure. Where E is a finite set of entities, P is a finite set of predicate symbols, containing unary, binary, and ternary predicates. I is an interpretation function that maps entities to constant symbols and predicates to relations. A scene parser can be defined as a mapping from the natural language space to a set of logical formulas: : ,in, For a finite set of premises, Here is the query formula, and V is the vocabulary mapping table.

[0030] Based on the above, regarding the obtained set of logical premises With the query proposition Q, where It may not be possible to contain Q or Q alone. This means the problem is in an undetermined state and requires the supplementation of missing common-sense premises.

[0031] Based on this, in this embodiment, multiple iterations are set up. In each iteration, the logical skeleton of the logical premise set is extracted, and the logical text in the logical skeleton is used as the antecedent to generate candidate implicit rules through a large language model. For details, please refer to [link to relevant documentation]. Figure 2 This step can be achieved in the following ways: S121, for each pair of characters in the logical text of the logical skeleton, a priority score is obtained for each pair of characters based on the degree of correlation between each pair of characters and other characters in the logical text. S122, traverse each pair of texts in descending order of priority score, use each pair of texts as the antecedent prompt in the large language model, generate the corresponding consequent text, and obtain the corresponding candidate implicit rules.

[0032] Then, the candidate implicit rules are filtered and added to the logical premise set. For details, please refer to [link to relevant documentation]. Figure 3 This can be achieved in the following ways: S123, Evaluate the candidate implicit rules based on the set common sense compliance and scenario relevance; Specifically, the logical values ​​of the candidate implicit rules in the common sense dimension are output using a large language model, and the corresponding common sense compliance score is obtained based on the logical values ​​in the common sense dimension; the logical values ​​of the candidate implicit rules and the current traffic scenario in the relevance dimension are evaluated based on the large language model, and the corresponding scenario relevance score is obtained based on the logical values ​​in the relevance dimension; the evaluation result is obtained based on the common sense compliance score and the scenario relevance score.

[0033] S124, Based on the evaluation results, the candidate implicit rules are filtered and added to the logical premise set.

[0034] In this embodiment, the common sense set is first initialized. It is used to store the implicit rules that are subsequently generated and filtered.

[0035] Call the SAT solver on the current logical premise set (This can be understood as the set of common sense being contained within the set of logical premises.) The satisfiability of the query proposition Q is then determined. The SAT solver is a complete propositional logic solver used to determine whether a given set of formulas is satisfiable.

[0036] Specifically, the SAT solver was tested separately. and Two decision questions. If the SAT solver returns... or If the problem is solved, terminate the process and output the current set of common sense. And the determination conclusion. If the SAT solver cannot reach any of the above conclusions, then the SAT solver is invoked to extract the current logical premise set. The logical framework. Defined as the set of explicit propositions of the current logical premise set and the textual units directly implied therein, as follows:

[0037] in, Let be the set of all possible literals, where a literal is a propositional variable or its negation. The logical skeleton reflects propositions whose truth values ​​have been determined under the current knowledge conditions; these literals serve as candidate antecedents for generating new rules.

[0038] Extracting the logical skeleton Next, the implicit rule generation and filtering stage begins. For the skeleton... Each pair of characters and The traversal is performed according to priority. Priority is determined by the relevance of the text to other texts in the skeleton. Specifically, the priority score for text L is defined as follows:

[0039] Prioritize traversing text pairs with higher priority scores. For each text pair... and The large language model is invoked to generate candidate implicit rules. Before generating new rules, the logical skeleton of the current perceptual proposition set is first calculated using the SAT solver. The antecedent of new rules generated by a large language model must include Literal quantity in Chinese.

[0040] by As a predicate prompting language model, it generates the corresponding consequent text. , forming a shape like The implicit rules. Large language models can incorporate the current set of logical premises during the generation process. (Includes common sense collection) By introducing new variables and predicates that do not appear in the original problem, we can break through the limitations of the entity set in the original problem and achieve true abductive reasoning.

[0041] For each generated candidate implicit rule A dual-scoring screening process is implemented.

[0042] The first level of scoring is the common sense compliance score, which uses a large language model to evaluate whether the rule conforms to common sense without any context.

[0043] Specifically, ask the large language model questions such as "Does the following rule violate common sense?" or "Is the following rule true?", and record the logical values ​​corresponding to "yes" and "no" in the model output. Calculate the common sense compliance score, as follows:

[0044] The second level of scoring is scenario relevance scoring, which utilizes a large language model within the current logical premise set. (Includes common sense collection) The rule is evaluated within the context of the current scene. Specifically, the known facts are provided to the large language model, and the question "Is the following new rule relevant to the current context?" is asked. The output logical value is recorded, and the scene relevance score is calculated, as shown below:

[0045] This section presets a dual scoring threshold. .like If the candidate implicit rule is accepted as a valid implicit rule, it will be added to the commonsense set. (That is, added to the logical premise set). If none of the candidate implicit rules generated by the current text pair pass the double-scoring screening, continue to traverse the next text pair. If multiple candidate implicit rules pass the screening, select the first candidate implicit rule that passes the screening and add it to the commonsense set, or add it according to the score product. Select the optimal rule in descending order.

[0046] Adding new implicit rules to the common sense set Then, the SAT solver is called again to solve the expanded logical premise set. Perform a satisfiability check and repeat the above process. If the SAT solver successfully determines satisfiability in a certain iteration... or If the judgment requirements are met, the iteration terminates and the judgment conclusion is output.

[0047] Furthermore, if no acceptable implicit rule is found after traversing all word pair combinations, or if no new implicit rule is added in multiple consecutive iterations, the iteration terminates. In this case, a self-consistent solution is called from the large language model as an alternative. The self-consistent solution uses multiple independent prompts to the large language model, takes the majority vote as the final conclusion, and calculates the vote confidence.

[0048] Here, a self-consistency threshold is also maintained. After each iteration, Subtract fixed step size When the confidence level of self-consistency voting exceeds the current threshold. When the iteration terminates, a self-consistent conclusion is output. This mechanism ensures that a reasonable decision result can still be output even when a deterministic conclusion cannot be obtained through symbolic solution.

[0049] After the above processing, the final output consists of two parts: The logical premise set (including the common sense set) after iterative completion This contains all the implicit rules that passed the filtering; The conclusion regarding the judgment of the proposition Q is either that Q is true or that Q is false.

[0050] This output serves as input for subsequent steps, initializing the text posterior distribution and driving the active exploration process. If the conclusion is obtained through the SAT solver, it is determined to be logically rigorous; if the conclusion is obtained through self-consistency, it is determined to be based on the statistical consistency of the large language model.

[0051] Based on the above, in this embodiment, the posterior distribution of the large language model agent is initialized based on the logical premise set obtained through iteration and the query proposition. For details, please refer to [link to relevant documentation]. Figure 4 This can be achieved through the following steps: S131, the logical premise set and query proposition obtained by iteration are encoded into natural language statements to form a structured text description; S132, the structured text description is converted into an initial posterior distribution of the large language model agent, the posterior distribution being used to describe the agent's perception of the current state of the traffic scene.

[0052] Building upon this, a multi-round sampling method is implemented. In each round, the agent samples a hypothetical environment model from the current posterior distribution. Based on this hypothetical environment model, a large language model is used to infer and generate a decision action for the current traffic state. For details, please refer to [link to relevant documentation]. Figure 5 This step can be achieved in the following ways: S133, using a large language model to estimate the value of each candidate action based on the reward function and transfer function in the hypothetical environment model, and employing context learning; S134, Generate a decision action for the current traffic state based on the value of each candidate action.

[0053] Then, after executing the decision action, the agent obtains the trajectory of the current round and updates the historical interaction trajectories and posterior distribution. For details, please refer to [link to relevant documentation]. Figure 6 This can be achieved in the following ways: S141, after the agent executes the decision action, it observes the corresponding reward and the next state from the real environment to form the experience tuple for the current time step; S142, after the current round ends, obtain the complete round trajectory based on the experience tuple, merge the current round trajectory with the historical interaction trajectory to obtain the updated historical interaction trajectory; S143, Generate an updated posterior distribution based on the current posterior distribution and the current round trajectory.

[0054] The posterior distribution includes states, actions, reward functions, and transition functions; the step of generating an updated posterior distribution based on the current posterior distribution and the current round trajectory is implemented in the following way: Extract the empirical tuple for each time step from the current round trajectory, evaluate the consistency between the empirical tuple and the uncertainties in the current posterior distribution, and adjust the confidence of the reward function and the transition function according to the consistency to obtain the updated posterior distribution.

[0055] Specifically, the logical premise set obtained by iterating through the above output. The judgment conclusion of the query proposition Q, as input, provides a structured understanding of the current traffic scenario. This structured understanding is then transformed into the initial beliefs of the large language model agent, i.e., the posterior distribution in text form.

[0056] The agent maintains a probabilistic posterior belief about the real environment. ,in, A collection of historical trajectories. Initial beliefs. Initialize with the structured knowledge output from the above steps. Posterior sampling: Every k rounds, sample a hypothesis model from the current posterior distribution: This sampling process is implemented by a large language model based on textual belief descriptions. Bayesian update: Obtaining the new round trajectory. Then, the posterior distribution is updated according to Bayes' rule: .in This represents the model likelihood. The large language model (posterior update module) approximates this update through textual reasoning, outputting a new belief description. .

[0057] Specifically, the posterior distribution of the large language model agent is first initialized. ,in, Represents a real Markov decision process in an environment, including a state space. Action space Reward function Transfer function Initial state distribution And the maximum step size H. The posterior distribution is represented in text form, describing the agent's perception of the current state of the traffic environment.

[0058] The initial posterior distribution is obtained by transforming the set of decidable logical premises output from the above steps, which encodes logical facts into natural language statements to form a structured text description.

[0059] The Markov decision process in a traffic scenario is defined as follows: state , represents the agent's observation of the current traffic scene at time step h, including the vehicle's position, speed, orientation, and the status information of surrounding vehicles, pedestrians, and traffic lights.

[0060] action This indicates the driving operations that the intelligent agent can perform, including but not limited to discrete or continuous control commands such as acceleration, deceleration, steering, lane changing, and stopping.

[0061] reward function Output a scalar reward value to evaluate the state. Next action The immediate effect of the reward design should reflect the goals of the driving task, such as safe driving, efficient passage, and comfort.

[0062] Specifically, the reward function can be defined as a weighted combination of multiple objectives: ,in, TTC stands for collision time. ; ; Penalties for rule violations; It is an adjustable hyperparameter.

[0063] Transfer function , describing in state Next action The environment then transitions to the next state. The probability distribution of the transition may be random, such as the unpredictability of the behavior of other vehicles.

[0064] The maximum step size H is the maximum number of time steps for each round, such as the maximum number of decision steps for a single driving mission.

[0065] In each sampling In this process, the agent first starts from the current posterior distribution. Sampling a hypothetical environment model ,in For all historical interaction trajectories up to round k, each trajectory It records the agent's complete experience in round i.

[0066] Taking the posterior distribution of the current text form as input, the large language model is required to generate a specific context hypothesis that includes a complete description of the transition function and the reward function.

[0067] In obtaining a hypothetical environment model Then, based on the hypothetical model, a decision action for the current traffic state is generated through large language model reasoning. Decision Action Based on the hypothesis model of the large language model Inference generation, based on The reward and transition functions defined in the model are used to estimate the value of each candidate action through context learning. The large language model receives the current state. and hypothetical models The text description is taken as input, and the output is the decision action considered reasonable under this assumption. Specifically, the large language model does not rely on explicitly calculating the analytical solution of the Bellman equation for the action value function, but instead outputs the action selection directly through reasoning based on the text description of state-action-reward.

[0068] Among them, decision-making actions refer to the driving actions of the vehicle, and driving actions are specifically the sequence of control commands for the vehicle.

[0069] The agent executes the decision actions output by the large language model. Subsequently, rewards were observed in a real-world environment. and the next state These observations constitute the empirical tuple for the current time step. After a round ends, i.e., after H time steps or when the termination condition is met, the agent obtains the complete round trajectory, as shown below:

[0070] The complete trajectory of the current round Interaction trajectory with history Merge to obtain the updated historical interaction trajectory Then, the posterior update of the large language model is invoked, based on the posterior distribution. And the newly observed round trajectory Generate the updated posterior distribution The posterior update of the large language model receives prior beliefs and trajectory descriptions in text form and outputs a belief state reflecting the integration of new knowledge. This update process follows the Bayesian inference principle, i.e., the posterior is proportional to the likelihood multiplied by the prior, but in the large language model implementation, it is approximated using text-based inference. When the inference latency exceeds a safety threshold, the system forcibly switches to a preset safe default strategy.

[0071] The specific operations for updating the posterior distribution include: First, from the round trajectory First, extract the state-action-reward-next state empirical tuples for each time step. Second, for each empirical tuple, evaluate its consistency with the uncertainties in the current posterior distribution. Then, adjust the confidence in the transition function and reward function based on this consistency; for example, if the same transition result is observed multiple times under a certain state-action pair, increase the certainty of the transition probability. Finally, generate an updated textual posterior distribution that retains the parts of the original belief that have not been overturned by the new observation, while integrating the information brought by the new observation.

[0072] In each round, the closed-loop process of sampling, decision-making, execution, and updating is repeated. As the number of rounds K increases, the posterior distribution gradually concentrates on the real-world model, driving the agent to continuously refine its understanding of the traffic scenario and its ability to generate vehicle control commands during interactions. This process realizes an information-oriented proactive exploration mechanism. The agent not only uses its current beliefs to select actions to obtain rewards but also explores the action space of states with high uncertainty by sampling hypothetical models, thereby accelerating the learning of effective strategies adapted to the real-world model.

[0073] The final output is a sequence of specific driving behavior planning instructions generated by the agent in the current traffic scenario. Simultaneously, the output includes the accumulated set of decision trajectories from all rounds. and the posterior distribution that eventually converges. The latter is used to characterize the logical interpretability and confidence assessment of decisions. These outputs can be used to evaluate the driving performance of an agent in traffic scenarios, or as input for subsequent tasks such as policy optimization and behavior planning.

[0074] In this embodiment, through the above-mentioned logical symbolization, abductive reasoning completion, and posterior sampling active exploration, an end-to-end processing flow is realized from natural language traffic scene description to a sequence of safe driving action instructions that can be directly executed by the vehicle control system.

[0075] Based on the same inventive concept, please refer to Figure 7 This invention also provides a traffic scene understanding system based on logical symbolic reasoning, the system comprising: The symbolization module is used to convert traffic-related natural language text and query propositions to be determined into first-order logical symbolic representations, and generate corresponding logical premise sets and query propositions; The completion module is used to extract the logical skeleton from the logical premise set in each iteration, use the logical text in the logical skeleton as antecedent to generate candidate implicit rules through a large language model, filter the candidate implicit rules and add them to the logical premise set, and iterate until the obtained logical premise set and query proposition meet the judgment requirements. The reasoning module is used to initialize the posterior distribution of the big language model agent based on the logical premise set and query propositions obtained through iteration. In each round of sampling, the agent samples a hypothetical environment model from the current posterior distribution and generates a decision action for the current traffic state based on the hypothetical environment model through reasoning of the big language model. The understanding and reasoning module is further used to obtain the current round trajectory after the agent executes the decision action, and update the historical interaction trajectory and posterior distribution until the sampling stopping condition is met. Based on the obtained decision action, it generates a sequence of driving behavior planning instructions and outputs the set of round trajectories accumulated over all rounds and the finally converged posterior distribution.

[0076] The traffic scene understanding system based on logical symbolic abduction provided in this embodiment can be used to execute the traffic scene understanding method based on logical symbolic abduction under any of the above embodiments. For details not covered in this embodiment, please refer to the corresponding descriptions in the above embodiments. This embodiment will not elaborate further here.

[0077] Furthermore, embodiments of the present invention also provide an electronic device, which may be a computer device, a server, or the like. The electronic device includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor. The processor executes the computer-executable instructions to implement the traffic scene understanding method based on logical symbolic reasoning in any of the above embodiments.

[0078] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0079] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] Furthermore, in the various embodiments of the present invention, the functional modules can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0081] It should be noted that if the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A traffic scene understanding method based on logical symbolic abduction, characterized in that, The method includes: The natural language text related to traffic scenarios and the query propositions to be decided are converted into first-order logic symbolic representations, and the corresponding logical premise set and query propositions are generated. In each iteration, the logical skeleton in the logical premise set is extracted, and the logical text in the logical skeleton is used as the antecedent to generate candidate implicit rules through a large language model. The candidate implicit rules are then filtered and added to the logical premise set. The iteration continues until the obtained logical premise set and query proposition meet the judgment requirements. Based on the logical premise set and query propositions obtained through iteration, the posterior distribution of the big language model agent is initialized. In each round of sampling, the agent samples a hypothetical environment model from the current posterior distribution and generates a decision action for the current traffic state based on the hypothetical environment model through reasoning of the big language model. After the agent executes the decision action, it obtains the current round trajectory and updates the historical interaction trajectory and posterior distribution until the sampling stopping condition is met. Based on the obtained decision action, it generates a sequence of driving behavior planning instructions and outputs the accumulated round trajectory set of all rounds and the finally converged posterior distribution.

2. The traffic scene understanding method based on logical symbols according to claim 1, characterized in that, The logical premise set is a collection of multiple first-order logic formulas. Each first-order logic formula consists of entity constants, variables, predicates, quantifiers, and logical connectors, representing a fact or rule extracted from the natural language text. The query proposition is a single first-order logic formula, representing the target to be determined obtained by transforming the query proposition to be determined. The query proposition is one of attribute query, relation query, event prediction query, and behavior rationality query.

3. The traffic scene understanding method based on logical symbolic abduction according to claim 1, characterized in that, The step of generating candidate implicit rules using the logical text in the logical skeleton as a precondition through a large language model includes: For each pair of characters in the logical text of the logical skeleton, a priority score is obtained for each pair of characters based on the degree of relevance between each pair of characters and other characters in the logical text. Each pair of texts is traversed in descending order of priority score. Each pair of texts is used as the antecedent prompt in the large language model to generate the corresponding consequent text, thus obtaining the corresponding candidate implicit rules.

4. The traffic scene understanding method based on logical symbolic abduction according to claim 1, characterized in that, The step of filtering the candidate implicit rules and adding them to the logical premise set includes: The candidate implicit rules are evaluated based on the common sense compliance and scenario relevance of the settings; The candidate implicit rules are filtered based on the evaluation results and added to the logical premise set.

5. The traffic scene understanding method based on logical symbolic abduction according to claim 4, characterized in that, The step of evaluating the candidate implicit rules based on the set common sense compliance and scenario relevance includes: The logical values ​​of the candidate implicit rules in the common sense dimension are output using a large language model, and the corresponding common sense conformity score is obtained based on the logical values ​​in the common sense dimension. The logical values ​​of the candidate implicit rules and the current traffic scenario are evaluated based on the large language model, and the corresponding scenario relevance score is obtained based on the logical values ​​of the relevant dimensions. The evaluation result is obtained based on the common sense conformity score and the scenario relevance score.

6. The traffic scene understanding method based on logical symbolic abduction according to claim 1, characterized in that, The steps for initializing the posterior distribution of the large language model agent based on the logical premise set and query propositions obtained through iteration include: The logical premise set and query propositions obtained through iteration are encoded into natural language statements to form a structured text description; The structured text description is converted into an initial posterior distribution for a large language model agent, which is used to describe the agent's perception of the current state of the traffic scene.

7. The traffic scene understanding method based on logical symbolic abduction according to claim 1, characterized in that, The step of generating decision actions for the current traffic state based on the hypothetical environment model through large language model reasoning includes: The value of each candidate action is estimated using context learning based on the reward function and transition function in the hypothetical environment model through a large language model. Based on the value of each candidate action, a decision action is generated for the current traffic situation.

8. The traffic scene understanding method based on logical symbolic abduction according to claim 1, characterized in that, The steps of obtaining the current round trajectory and updating the historical interaction trajectory and posterior distribution after the agent executes the decision action include: After the agent executes the decision action, it observes the corresponding reward and the next state from the real environment to form the experience tuple for the current time step. After the current round ends, the complete round trajectory is obtained based on the experience tuple. The current round trajectory is then merged with the historical interaction trajectory to obtain the updated historical interaction trajectory. An updated posterior distribution is generated based on the current posterior distribution and the current round trajectory.

9. The traffic scene understanding method based on logical symbolic abduction according to claim 8, characterized in that, The posterior distribution includes states, actions, reward functions, and transition functions; The step of generating an updated posterior distribution based on the current posterior distribution and the current round trajectory includes: Extract the empirical tuple for each time step from the current round trajectory, and evaluate the consistency of the empirical tuple with the uncertainties in the current posterior distribution; The confidence levels of the reward function and the transition function are adjusted based on the consistency to obtain the updated posterior distribution.

10. A traffic scene understanding system based on logical symbolic abduction, characterized in that, The system includes: The symbolization module is used to convert traffic-related natural language text and query propositions to be determined into first-order logical symbolic representations, and generate corresponding logical premise sets and query propositions; The completion module is used to extract the logical skeleton from the logical premise set in each iteration, use the logical text in the logical skeleton as antecedent to generate candidate implicit rules through a large language model, filter the candidate implicit rules and add them to the logical premise set, and iterate until the obtained logical premise set and query proposition meet the judgment requirements. The reasoning module is used to initialize the posterior distribution of the big language model agent based on the logical premise set and query propositions obtained through iteration. In each round of sampling, the agent samples a hypothetical environment model from the current posterior distribution and generates a decision action for the current traffic state based on the hypothetical environment model through reasoning of the big language model. The understanding and reasoning module is further used to obtain the current round trajectory after the agent executes the decision action, and update the historical interaction trajectory and posterior distribution until the sampling stopping condition is met. Based on the obtained decision action, it generates a sequence of driving behavior planning instructions and outputs the set of round trajectories accumulated over all rounds and the finally converged posterior distribution.

Images