A method and device for controlling reasoning state based on prior knowledge of large language model

Abstract

The present invention relates to the field of data processing technology, and in particular to a method and device for controlling the reasoning state based on prior knowledge of a large language model. The method is applied to the problem of autonomous driving discretized navigation, container stacking, cargo stacking, and text numerical calculation under natural language text. The method includes: obtaining a sample data set of decision reasoning of an unmanned system; using the sample data set and a loss function composed of an ATE causal effect estimation algorithm and cross entropy to train the initial large language model; obtaining a long-range action decision problem, constructing a causal probability tree, determining the optimization problem of pruning and expansion directions, and then dividing the long-range action decision problem into multiple short-range action decision problems; using the trained large language decision reasoning model to solve multiple short-range action decision problems, and then completing the action. The present invention can solve the problem of causal hallucination of the large language model in long-range reasoning tasks and improve the accuracy of reasoning.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for controlling an inference state based on prior knowledge of a large language model. Background Art

[0002] The core part of intelligent decision-making in large language models (LLMs) lies in planning, which involves generating a series of actions aimed at achieving a predetermined goal. Classical planning methods have been widely used in robotics and embodied environments, and are often used to guide external decision-making processes. Recent advances such as the Chain-of-Thought Model have greatly enhanced the ability of LLMs to perform detailed reasoning. This model breaks down complex queries into a series of manageable steps, thereby improving the decision-making ability of large language models. Subsequent initiatives such as ReACT have modified this approach to improve reasoning capabilities in decision-making environments using a Chain-of-Thought-based framework. In addition, "reflection" also provides a correction mechanism that enables local language learners to recognize their mistakes in the decision-making process, reflect on these mistakes, and make accurate decisions in subsequent attempts. Further developments have led to the birth of decision-making frameworks based on tree structures, which can adjust the capabilities of LLMs according to specific scenarios. Thought trees facilitate decision-making using breadth-first search (BFS) and depth-first search (DFS) algorithms. Meanwhile, planning reasoning (RAP) uses Monte Carlo tree search techniques to optimize the solution of the task. DFSDT proposes an efficient version of DFS for LLMs to make decisions, but it lacks the judgment ability to evaluate different decisions. JUDEC uses the Elo rating system to let LLMs develop self-assessment capabilities, allowing them to generate optimal solutions for a wide range of real-world tasks without being influenced by any specific task expertise. Finally, Graph-of-Thought represents thoughts as nodes in a graph, combining thoughts non-sequentially.

[0003] Integrating causal analysis in LLM to facilitate multi-step decision making: Causal analysis aims to discern and elucidate the causal relationships between actions, situations, or decisions. This approach requires investigating the origins or causes that led to an event, as well as the potential consequences that followed. Although various causal models may produce the same distribution of observations, they may produce different distributions when interventions are applied. Therefore, the use of interventions can distinguish possible causal frameworks that are consistent with the observed data. Previous studies have shown that while Chain-of-Thought has been hailed for its potential to improve task performance, its application does not always lead to better results. In addition, studies have shown that statistical pre-training of LLMs leads to high empirical performance of the model, but not necessarily for inference.

[0004] Solving Multi-Step Problems with LLMs: Recent studies have shown that, with careful design, LLMs are capable of performing not only basic arithmetic tasks, but also complex multi-step reasoning. For example, increasing computational resources can significantly improve accuracy on datasets such as GSM8K. At the same time, studies have shown that a 2B-parameter LLM can achieve 89.9% accuracy in a 5x5 multiplication task through curriculum learning with 50 million training instances. This evidence suggests that appropriately sized LLMs can effectively handle multiple reasoning steps internally. While trees are often used to represent games (especially Cantonese games) and sequential reasoning problems, it was Schaeffer's pioneering work that initially established a framework for understanding causal relationships by using probability trees. LLMs tend to struggle with long-range reasoning problems involving multiple steps.

[0005] However, their performance and efficiency are limited when dealing with complex navigation reasoning problems that require multi-step long-range reasoning.

[0006] The first challenge is causal hallucination, that is, the causal relationship between one-step reasoning (OSR) and state transitions in LLM is not always guaranteed. Similar to the fact that pre-trained LLMs are prone to hallucinations when processing certain factual information, causal hallucination reflects the fact that LLMs lack rigor when completing complex mathematical logic or common sense reasoning due to inherent randomness, which is to some extent deeply rooted in statistical necessity and has nothing to do with the Transformer architecture or data quality. For example, the finite-step reasoning method based on Chain-of-Thought suffers from causal hallucination and cannot effectively ensure the causal relationship between OSR and state transitions in LLM, resulting in unreliable reasoning and relatively low success rate (especially for long-range reasoning problems with significant error accumulation effects). The rationality between OSR and state transition can be summarized as follows: there is a causal relationship between a reasonable OSR and state transition. At the same time, there is only a correlation or no relationship between an unreasonable OSR and state transition, which shows that training with cross-entropy loss alone does not make the model have sufficient causal rigor.

[0007] The second challenge is that long-range reasoning problems have a huge search space. Although complex architectures such as Chain-of-Thought, Tree of Thought (ToT), and Program of Thought (PoT) can effectively improve the reasoning accuracy of LLM through external guidance, they are limited in dealing with long-range reasoning processes and task decomposition. One of the important reasons is that long-range reasoning has a huge state space, that is, each branch in the state transition process will expand the search space approximately exponentially. Most of the existing LLM-based methods (such as Monte Carlo search) are based on one-way reasoning, which makes them both time-consuming and prone to falling into local optimal states when dealing with reasoning problems with huge search spaces. Summary of the invention

[0008] The present invention provides a method and device for controlling the inference state based on a priori knowledge of a large language model. The technical solution is as follows:

[0009] On the one hand, a method for controlling an inference state based on a priori knowledge of a large language model is provided, the method being implemented by an inference state control device based on a priori knowledge of a large language model, the method comprising:

[0010] S1. Obtain a sample data set of unmanned system decision reasoning for model training;

[0011] S2, using the sample data set and the loss function composed of the ATE causal effect estimation algorithm and the cross entropy, the initial large language model is trained to obtain a trained large language decision reasoning model;

[0012] S3. Obtain a long-range action decision problem, construct a causal probability tree based on the long-range action decision problem, determine the optimization problem of pruning and expanding directions based on the causal probability tree, prune and expand the causal probability tree by solving the optimization problem, and divide the long-range action decision problem into multiple short-range action decision problems;

[0013] S4. Use the trained large language decision reasoning model to solve multiple short-term action decision problems, obtain the action plan corresponding to each short-term action decision problem, and complete the action according to the action plans corresponding to the multiple short-term action decision problems.

[0014] On the other hand, a device for controlling a reasoning state based on a priori knowledge of a large language model is provided, and the device is applied to a method for controlling a reasoning state based on a priori knowledge of a large language model, and the device comprises:

[0015] An acquisition unit, used for acquiring a sample data set of unmanned system decision reasoning for model training;

[0016] A training unit is used to train the initial large language model using a sample data set and a loss function composed of an ATE causal effect estimation algorithm and a cross entropy to obtain a trained large language decision reasoning model;

[0017] A segmentation unit is used to obtain a long-range action decision problem, construct a causal probability tree based on the long-range action decision problem, determine the optimization problem of pruning and expanding directions based on the causal probability tree, prune and expand the causal probability tree by solving the optimization problem, and segment the long-range action decision problem into multiple short-range action decision problems;

[0018] The decision-making unit is used to solve multiple short-term action decision problems using the trained large language decision reasoning model, obtain the action plan corresponding to each short-term action decision problem, and complete the action according to the action plans corresponding to the multiple short-term action decision problems.

[0019] On the other hand, an inference state control device based on large language model prior knowledge is provided, and the inference state control device based on large language model prior knowledge includes: a processor; a memory, on which computer readable instructions are stored, and when the computer readable instructions are executed by the processor, any one of the above-mentioned inference state control methods based on large language model prior knowledge is implemented.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned inference state control methods based on large language model prior knowledge.

[0021] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0022] By introducing the causal relationship enhancement (CRE) mechanism to improve the rigor of the LLM-based long-range reasoning method, using structural causal modeling (SCM) to enhance the causal relationship between OSR and state transitions, causal intervention is performed during the training process and the absolute value of ATE is optimized, which effectively alleviates the problem of causal hallucination in large language models in long-range reasoning tasks and improves the accuracy and reliability of reasoning. At the same time, a bidirectional double-end search (DES) method is proposed. After constructing a causal probability tree with the initial state as the starting point and the target state as the end point, long-range reasoning (such as 12 steps) is converted into a more manageable combination of small segments (such as 2 or 4 steps) by minimizing the distance between leaves and using end matching technology. By avoiding long-range sequential search from scratch, the DES method greatly reduces the complexity of solving long-range reasoning problems and significantly improves the decision-making efficiency and accuracy of large language models in complex reasoning tasks. Especially in the fields of autonomous driving and logistics transportation, accurate decision-making is crucial, which will greatly improve the safety and reliability of related systems and reduce the occurrence of misjudgments and accidents. The technical solution of the present invention is not only applicable to multiple fields such as autonomous driving, logistics transportation, container stacking and cargo stacking, but also has broad application prospects and important commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a flow chart of a method for controlling an inference state based on a priori knowledge of a large language model provided by an embodiment of the present invention;

[0025] Figure 2 is a schematic diagram of state transition provided by an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of a causal relationship enhancement structure provided by an embodiment of the present invention;

[0027] Figure 4 This is a schematic case diagram of a processing method of the CRE causal relationship enhancement model training method provided by an embodiment of the present invention in three application fields: container stacking, natural language text data processing, and cargo stacking;

[0028] Figure 5 It is a block diagram of a reasoning state control device based on a priori knowledge of a large language model provided by an embodiment of the present invention;

[0029] Figure 6 It is a structural diagram of an inference state control device based on large language model prior knowledge provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0031] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0032] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0033] In the embodiments of the present invention, sometimes the subscripts such as W 1 It may be written in non-subscript form such as W1. When the difference is not emphasized, the meaning is the same.

[0034] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0035] The embodiment of the present invention provides a method for reasoning state control based on large language model prior knowledge, which is applied to autonomous driving discrete navigation problems, container stacking problems, cargo stacking problems, and text numerical calculation problems under natural language texts. The method can be implemented by a reasoning state control device based on large language model prior knowledge, which can be a terminal or a server. Figure 1 The flowchart of the inference state control method based on the prior knowledge of the large language model is shown in FIG. 1 . The processing flow of the method may include the following steps:

[0036] S1. Obtain a sample dataset of unmanned system decision reasoning for model training.

[0037] In a feasible implementation, raw data in actual business scenarios, including autonomous driving navigation data, container stacking data, natural language text data, and cargo stacking data, are converted into training data for training a large language model. Optionally, S1 obtains a sample data set of unmanned system decision reasoning for model training, including:

[0038] S11. Convert the multi-source heterogeneous raw action decision data in various formats into a string sequence, convert the string sequence into a vector, and store it in a vector database.

[0039] In a feasible implementation, data in multiple formats of multi-source heterogeneous original business data (including but not limited to PDF format, HTML format, NEO4J format, etc.) are converted into string sequences using technologies such as OCR, read in JSON format with the help of API, converted into vectors, and stored in vector databases, instead of storing the original files in the system in isolation as attachments in the conventional way, specifically including the following S111-S114:

[0040] S111: Autonomous driving navigation data: including vehicle location information (GPS coordinates), navigation start and end location information (GPS coordinates), speed, direction, and road environment data (such as intersection information, etc.);

[0041] S112: container stacking data: including the initial state stacking level, position coordinates and stacking order of the container, and the target state stacking level, position coordinates and stacking order;

[0042] S113: Natural language text data: including text content obtained from various documents, reports, conversation records and other sources, which is unstructured and divides the problem, intermediate process and result according to the initial state and target state;

[0043] S114: Goods stacking data: including the size of the goods, the stacking order in the initial state, the stacking order in the target state, and whether the stacking rules are met.

[0044] S12, performing noise filtering, outlier detection, standardization processing and structure processing on the vector data in the vector database to obtain preprocessed sample data.

[0045] In a feasible implementation, in the data preprocessing stage, enhanced Kalman filtering, DBSCAN algorithm, time series convolutional neural network, coordinate transformation and normalization, advanced NLP technology, deep learning image processing, and three-dimensional space reconstruction are used to filter noise, detect outliers, standardize and structure the raw data. Specifically, the following S121-S124 may be included:

[0046] S121: Autonomous driving navigation data: Use Apache NiFi to process real-time data streams and integrate the processed navigation data streams into a unified time series database. Use automated labeling tools (such as Labelbox) to label navigation data based on the start and end point GPS coordinates and intersection information.

[0047] S122: Container stacking data: Talend is used to aggregate and merge container data from different sources to form a comprehensive database. Weakly supervised learning tools such as Snorkel are used to automatically annotate the stacking data, including the initial stacking level, location coordinates, and stacking order of the container, and the target stacking level, location coordinates, and stacking order.

[0048] S123: Natural language text data: Use ETL tools to integrate pre-processed text data from multiple sources into a unified data warehouse. Use a machine learning-driven annotation platform to automatically label and annotate text data, combined with semantic analysis technology to enhance data readability and context.

[0049] S124: Cargo stacking data: Apache NiFi is used to integrate cargo stacking data from different image sources to build a multidimensional dataset. Labelme is used to annotate images and combine the 3D point cloud data generated by SLAM to perform detailed labeling of each cargo location.

[0050] S13: label the preprocessed sample data to obtain labeled sample data.

[0051] In a feasible implementation, efficient ETL tools and automated annotation platforms are used, combined with weakly supervised learning and active learning methods, to integrate and label the pre-processed data. Specifically, S131-S134:

[0052] S131, Autonomous Driving Navigation Data: Use Apache Avro to convert the integrated time series data into JSON format to facilitate time series analysis of large language models. Use Protocol Buffers for data compression to optimize storage and transmission efficiency.

[0053] S132: Container stacking data: Convert the integrated container data into CSV format to meet the structured data input requirements of the large language model. Use Apache Parquet for data serialization to improve big data processing capabilities.

[0054] S133: Natural language text data: Use Thrift to convert the annotated text data into a dedicated binary format to ensure data compatibility and efficient transmission. Generate metadata files in JSON format to provide semantic and structural information of the data.

[0055] S134: Cargo stacking data: Use Apache Parquet to convert 3D point cloud data and image data into binary format to reduce storage space. Convert text label data into CSV format to facilitate text data processing for large language models.

[0056] S14. Perform data format consistency processing on the labeled sample data to obtain a sample data set for unmanned system decision reasoning used for model training.

[0057] In a feasible implementation, advanced data format conversion tools and data compression serialization technology are used to ensure the consistency and compatibility of data formats, thereby generating a high-quality data set suitable for large language model training. Specifically, the following S141-S144 may be included:

[0058] S141: Autonomous driving navigation data: format: starting point coordinates||destination coordinates####<intersection coordinates 1, intersection action 1>···<intersection coordinates N, intersection action N>.

[0059] S142: Container stacking data: format: starting state stacking state||destination state stacking state####<immediate stacking state 1, container moving action 1>···<immediate stacking state N, container moving action N>.

[0060] S143: Natural language text data: in the form of: original text question||final text answer####<intermediate solution process 1, intermediate answer 1>···<intermediate solution process 1, intermediate answer N>.

[0061] S144: The goods stacking data is in the form of: starting state stacking state||destination state stacking state####<immediate stacking state 1, goods moving action 1>···<immediate stacking state N, goods moving action N>.

[0062] It should be noted that the data in S14 format is divided into training set: test set: validation set = 7:2:1 ratio. The training set and test set are used in the training phase, and the validation set is only used in the testing phase.

[0063] The reasoning state control method based on the prior knowledge of the large language model refers to the use of the prior knowledge internalized in the pre-training process of the large language model, while establishing a spatiotemporal simulation system based on the combination of virtual and real to represent the key numerical information in the reasoning process, and finally integrating the model internalized knowledge and spatiotemporal simulation information to construct the state transition event of the finite state machine, so as to establish a comprehensive spatiotemporal causal representation of the automation of the long-process multi-state unmanned system, and unfold it on the time axis. Based on the advantage of the finite state machine in limiting the action search space and realizing decision search in the spatiotemporal continuous reasoning process of the automation of the long-process multi-state unmanned system, the method proposed in the present invention solves the problems of discreteness, high non-convexity, and huge action space existing in the traditional causal representation method.

[0064] A finite state machine is a computational model used to describe the behavior of a system or program. It consists of a set of states, transition conditions, and state transitions. The states not only represent different operating conditions of the system, but also store information about the past, that is, they reflect the input changes from the beginning of the system to the present moment. Figure 2 The state transition diagram shown in the figure, the transition condition defines the condition from one state to another. The state transition indicates the change of state and describes the transition of the system between different states. The next state of the finite state machine is determined by the input and the current state.

[0065] According to the definition of finite state machine, its mathematical model can be described as a seven-tuple ,here: is the input alphabet (a non-empty finite set of input events); is a non-empty finite set of states; is the initial state or initial state, it belongs to Elements in is the set of final states, A is a subset of (possibly empty); A is a possible action to take in each state. is the state transition function: , which can be further expressed as ,in , , , This formula includes two main parts: 1) Event-driven state transition: This transition is usually the most basic state transition, such as from state Transfer to state . This transition is usually triggered by a specific event e. This transition is one of the basic characteristics of a finite state machine, which triggers state transfer based on a specific event received. This transition can be seen as a "trigger", and when a specific event occurs, the state changes. 2) Conditional state transition: This transition is performed after the event-driven state transition. It determines whether to transfer the state based on whether a specific condition c is met. If condition c is met, the state is transferred to , otherwise no state transfer will be made. This transition is performed after the event-driven state transition, and it determines whether to perform state transfer based on specific conditions. The condition can be regarded as a kind of "filter" or "gate", and the state will only transfer when the condition is met. If the condition is not met, the state transfer will not occur. In addition, the execution of action a does not necessarily lead to state transfer, it may just be to perform certain tasks (such as printing logs, updating internal variables, etc.). The condition c can be regarded as a certain judgment standard, and state transfer is possible only when this standard is met.

[0066] In the present invention, the finite state machine specifically refers to a specific type of finite state machine used in long-process multi-state unmanned systems. In the scenario of automated spatiotemporal continuous reasoning of long-process multi-state unmanned systems, the large language model needs to perform different operations according to different goals, environments, and tasks, which requires modeling and control of its behavior. The finite state machine defines a set of states, each of which represents a different behavior or operation mode of the large language model in the air, and then specifies the transition conditions and state transitions between states to achieve control and coordination of the behavior of the large language model.

[0067] S2. Use the sample data set and the loss function composed of the ATE causal effect estimation algorithm and the cross entropy to train the initial large language model to obtain a trained large language decision reasoning model.

[0068] Optionally, S2 uses a sample data set and a loss function composed of an ATE causal effect estimation algorithm and a cross entropy to train the initial large language model, including the following steps S21-S23:

[0069] S21. The form of the samples in the sample data set is recorded as: IT||GT####<OSR,SR> Among them, IT represents the initial state, GT represents the target state, OSR represents the transfer action, and SR represents the state transition.

[0070] Optionally, the sample data set is autonomous driving navigation sample data, container stacking sample data, or natural language text sample data:

[0071] (1) When the sample data set is autonomous driving navigation sample data, S21 records the samples in the training sample data set in the form of IT||GT####<OSR,SR> ,include:

[0072] The format of the autonomous driving navigation sample data is recorded as: starting point coordinates||destination coordinates####<intersection coordinates 1, intersection action 1>…<intersection coordinates N, intersection action N>;

[0073] (2) When the sample data set is container stacking sample data, S21 records the samples in the training sample data set in the form of IT||GT####<OSR,SR> ,include:

[0074] The format of container stacking sample data is recorded as: starting state stacking state||destination state stacking state####<immediate stacking state 1, container moving action 1>…<immediate stacking state N, container moving action N>.

[0075] (3) When the sample data set is natural language text sample data, S21 records the sample in the training sample data set in the form of IT||GT####<OSR,SR> ,include:

[0076] The format of natural language text sample data is recorded as: original text question||final text answer####<intermediate solution process 1, intermediate answer 1>…<intermediate solution process 1, intermediate answer N>.

[0077] S22. Given binary variables X and Y, X is used to represent the correctness of OSR and Y is used to represent the correctness of SR.

[0078] ATE is defined as follows:

[0079] (1)

[0080] in, , X=1 means OSR is correct, Y=1 means SR is correct, represents the expected calculation formula, Refers to Do-calculus, which means external intervention in the value of Y without affecting the actual state of X.

[0081] The cross entropy loss function is defined as follows (2):

[0082] (2)

[0083] in, Indicates a given Status The conditional probability of being 1, Indicates a given Status The conditional probability of being 0.

[0084] The loss function composed of the ATE causal effect estimation algorithm and the cross entropy is defined as follows (3):

[0085] (3).

[0086] In one feasible implementation, all samples are divided into two categories: correct (T) and incorrect (F). In the incorrect category, there are three cases, namely, correct OSR leads to incorrect SR, incorrect OSR leads to incorrect SR, and incorrect OSR leads to correct SR.

[0087] In CRE, ATE is first used to quantitatively estimate the causal relationship between OSR and state transitions, and then ATE is embedded in the loss function during training to enhance the causal relationship of state transitions. The OSR selection is left to the cross-entropy loss control, while the suppression of hallucinations (reducing the occurrence of false SRs, i.e., SRs that do not match OSR, including SRs caused by false OSRs) is handled by ATE loss. Perplexity (PPL) is an indicator used to evaluate the performance of LLM, which indicates the ability of the model to predict the next word in the sequence. The lower the value, the higher the prediction accuracy. The estimation method of ATE is detailed as follows:

[0088] Given binary variables X and Y, representing the correctness of OSR and SR respectively, that is (or ) is correct. First, the causal intervention between X and Y is calculated, and then the distribution of Y is modified by intervening in X. From a statistical correlation perspective, if X and Y are correlated, then X can predict Y. However, if there is no causal relationship between X and Y, intervening in X will not change the distribution of Y. Therefore, if X and Y are correlated but not causally related, then manipulating or intervening in X will not cause any change in the distribution of Y. This distinction is critical in statistical analysis and experimental design because it addresses the potential fallacy that correlation inherently implies causation.

[0089]

[0090] The above formula, Refers to Do-calculus, which means external intervention on the value of Y without affecting the actual state of X. Using intervention independent of other variables, it is possible to find out whether the treated variable X will lead to the target variable Y.

[0091] Do-Calculus in causal inference is a set of rules proposed by Judea Pearl for calculating causal effects. Do-Calculus provides a mathematical tool for dealing with causal relationships in intervention situations. Here are the three basic rules of Do-Calculus:

[0092] Rule 1:

[0093] if and There is no path through 𝑍 between them:

[0094]

[0095] Rule 2:

[0096] If 𝑍 is descendants of and There is no parent node of 𝑍 on the path between them, then:

[0097]

[0098] Rule 3:

[0099] If 𝑍 blocks and All backdoor paths between , then:

[0100]

[0101] Among the above three rules, , and 𝑍 represent three different nodes respectively.

[0102] Use ATE to estimate the effect of an intervention by comparing the distribution of the target variable Y with and without the treatment. It is an intervention Down The potential result of ATE is as follows:

[0103]

[0104] According to the above formula, under intervention, the ratio of TT (true correct) samples and TF (false error) (hallucination) samples in the model output samples remains basically unchanged; the stronger the causal relationship between different OSRs and SRs, the lower the ATE. The reason is cross entropy Basically, it can ensure the majority of positive examples T. At the same time, ATE can reduce the appearance of negative examples F, making the distribution of positive and negative cases more stable.

[0105] Therefore, incorporating ATE into the loss function, and They represent the conditional probabilities that Y is 1 and 0 given the state X.

[0106]

[0107]

[0108] The probabilities of correct and incorrect (hallucination) samples in each OSR were estimated separately, and according to the SR between different OSRs, As part of the loss function under OSR in each case, the classes with strong causal effects are This process allows the model to internalize the logical judgment between OSR and SR during training, that is, the causal effect of the correct answer is strong. Low; the causal effect of incorrect answers is weak, Therefore, using and The loss function composed of these two can simultaneously achieve the coordinated optimization of OSR selection and SR hallucination elimination.

[0109] The cross entropy loss controls the accuracy of the state transition path, while ATE reduces the probability of causal illusion during OSR and SR reasoning under the state transition path. It reduces the causal illusion of current commonly used reasoning methods such as CoT (Chain of Thought). Causal illusion refers to the situation where OSR is correct and SR is wrong, or OSR is wrong and SR is correct. The coordinated optimization of the two jointly achieves the guarantee of the correctness of state transition path selection and the reduction of causal illusion.

[0110] S23. Input the input data in the sample data set into the initial large language model, compare the output prediction value of the initial large language model with the sample true value in the sample data set, and adjust the parameters in the initial large language model according to the loss function composed of the ATE causal effect estimation algorithm and the cross entropy until the loss function converges, stop training, and determine the obtained large language model as the trained large language decision inference model.

[0111] S3. Obtain a long-range action decision problem, construct a causal probability tree based on the long-range action decision problem, determine the optimization problem of pruning and expansion directions based on the causal probability tree, prune and expand the causal probability tree by solving the optimization problem, and divide the long-range action decision problem into multiple short-range action decision problems.

[0112] Optionally, the specific operations of S3 include S31-S36:

[0113] S31. Obtain a long-range action decision problem, which includes an initial state and a target state.

[0114] S32. Construct two causal probability trees based on the initial state and the target state, wherein a node in each causal probability tree represents a state in the reasoning process, and an arrow represents a causal relationship.

[0115] In one possible implementation, two causal probability trees are constructed based on the initial state (IT) and the target state (GT). Each node represents a state (SR) in the reasoning process (OSR), and the arrows represent causal relationships. These trees outline possible reasoning results within a limited number of intermediate steps. By matching the leaves of these two trees, several end-to-end permutations are determined to form a continuous feasible path.

[0116] The finite state machine of the present invention divides the basic state of the large language model into four basic states: current state (IT), target state (GT), transfer action reasoning (OSR), and state transition reasoning (SR). Among them, the IT state refers to the current position or state of the large language model; the GT state refers to the target position or state that the large language model needs to reach; the OSR state refers to the large language model performing process reasoning and calculating the action sequence required to achieve GT; the SR state refers to the state transition that occurs when the large language model is reasoned based on the aforementioned action sequence.

[0117] IT includes: for the discrete navigation problem of autonomous driving, the GPS coordinates of the starting point; for the container stacking problem, the spatial position of each box in the initial state of the container; for the cargo stacking problem, the spatial position of each cargo box in the initial state and the size (or weight) of each cargo box; for the text numerical calculation problem under natural language text, the natural language text of the input model.

[0118] GT includes: for the discrete navigation problem of autonomous driving, the GPS coordinates of the destination; for the container stacking problem, the spatial position of each box in the container target state; for the cargo stacking problem, the spatial position of each cargo box in the cargo target state and the size (or weight) of each cargo box; for the text numerical calculation problem under natural language text, the calculation result output text corresponding to the natural language text input to the model.

[0119] OSR includes: for the discrete navigation problem of autonomous driving, the car operation action group, namely, going straight, turning left, turning right, and U-turn; for the container stacking problem, the container movement action group, namely, grabbing and putting down; for the cargo stacking problem, the cargo box movement action group, namely, grabbing and putting down; for the text numerical calculation problem under natural language text, the calculation method action group, namely, addition, subtraction, multiplication, division, square, square root, exponent, trigonometric function, and logarithm.

[0120] SR includes: for the discrete navigation problem of autonomous driving, the GPS coordinates of each intersection; for the container stacking problem, the real-time spatial position of each container; for the cargo stacking problem, the real-time spatial position of each cargo box and the size (or weight) of each cargo box; for the text numerical calculation problem under natural language text, the intermediate process output text obtained after the input text is processed by the calculation method action group.

[0121] S33. Calculate the ATE between the expansion and distance reduction between two causal probability trees according to the following formula (4):

[0122] (4)

[0123] in, yes Relative to The reduced distance , Is the current leaf located at The distance D is calculated based on the spatial coordinates by calculating the Euclidean distance between the current position of the block and the target position coordinates.

[0124] In a feasible implementation, DES first calculates the ATE between the expansion of the tree and the reduction of the distance, and then clarifies the causal relationship between the expansion of the tree and the reduction of the distance, and then infers a better expansion direction and pruning process. At this time, the ATE calculation formula is as shown in the above formula (4). Among them, the distance D is calculated using spatial coordinates (GPS coordinates for autonomous driving navigation data, and coordinates in the spatial rectangular coordinate system modeled in the stacking state for container stacking data and cargo stacking data) and the number of layers relative to the tree expansion is estimated. The distance reduction is expressed as The distance is obtained by calculating the Euclidean distance between the block's current position and the target position coordinates.

[0125] S34. Based on the causal probability tree and the above formula (4), the optimization problem of the following formula (5) is constructed:

[0126] (5)

[0127] in, Indicates the number of levels relative to the tree expansion The distance reduction, represents the head tree of the causal probability tree, Represents the tail tree of the causal probability tree.

[0128] In one possible implementation, at each level of the tree expansion, the distance is calculated by comparing the current position with the target position, ensuring that the reduction in distance and the direction of the tree expansion have a strong causal effect. To avoid the expansion direction falling into a local optimum, a counterfactual evaluation is performed, assuming other expansion routes that may be taken during the random expansion process, and taking into account the causal effects of these assumed routes. These two values ​​are added to form the loss function, while taking into account the head and tail Tree.

[0129] S35, by minimizing the optimization problem , the optimization problem is solved and the schemes for pruning and expanding the causal probability tree are obtained.

[0130] In one feasible implementation, during the expansion process at both ends of the probability tree, by minimizing To intervene and guide the development towards the desired outcome. Minimize The judgment of pruning and expansion direction is realized, and priority is given to The smallest direction is used as the expansion direction.

[0131] Counterfactual evaluation is a method in the field of causal inference to analyze "what if" scenarios, that is, to evaluate the results under different conditions. It involves making assumptions about events that did not actually happen and predicting the results of these hypothetical events through models. Counterfactual evaluation is often used to understand causal relationships, by comparing actual results with hypothetical results, and evaluating causal effects. For example, evaluating how a student's grades would change if he did not attend a training course. This method is very important in fields such as policy evaluation, medical research and social sciences because it helps researchers understand potential causal relationships, not just observed associations.

[0132] S36. Prune and expand the causal probability tree according to the pruning and expansion scheme of the causal probability tree, so as to obtain multiple short-term action decision problems separated from the long-term action decision problem.

[0133] like Figure 3 The schematic diagram of the causal relationship enhancement structure shown in the figure first infers the short-range action from the beginning to the end of the long-range action decision problem. The expansion process of the causal probability tree is constrained by ATE, and pruning and optimization are achieved. For example, three sequential states are inferred from state 0 to the back and from state 6 to the front. Then the short-range problem is handed over to the CRE model for one-time output, and the multiple outputs are connected to each other end to end but not in a loop, forming the reasoning process required for the original long-range problem. The reason is that in the CoT method or ToT method, the model requires single-step reasoning for each state transition, and the chain expansion or tree expansion (the expansion space based on MCTS is large), which causes the model to perform many reasonings and the length of the single-step reasoning is too short. The 3-step or 4-step reasoning is regarded as a state transition process and handed over to the model for reasoning, and the constraint of the tree expansion direction is realized by the two-end approximation and ATE enhancement. Under the same computer hardware conditions, the amount of calculation is reduced and the calculation speed is accelerated.

[0134] S4. Use the trained large language decision reasoning model to solve multiple short-term action decision problems, obtain the action plan corresponding to each short-term action decision problem, and complete the action according to the action plans corresponding to the multiple short-term action decision problems.

[0135] In a feasible implementation, the following descriptions are respectively given for the problem of autonomous driving discretization navigation, the problem of container stacking, the problem of cargo stacking, and the problem of text numerical calculation under natural language text:

[0136] (1) Discrete navigation for autonomous driving: Use the DES-enhanced long-range reasoning method based on the CRE short-range reasoning module for decision-making and path planning, such as determining the current intersection and the action to be taken at the current intersection (left turn, right turn, U-turn, straight ahead), controlling the acceleration, braking and steering of the vehicle, and converting the instructions generated by the decision-making and planning system into control signals to achieve precise conversion from micro instructions to macro actions, including acceleration, braking and steering, to minimize control delays and execution deviations. Realize real-time transmission of control instructions (delay less than 5ms), and adjust the control strategy according to vehicle dynamics and road conditions through adaptive control algorithms, dynamically adjust path planning, and select the optimal driving path.

[0137] (2) Container stacking: Use the DES-enhanced long-range reasoning method based on the CRE short-range reasoning module to make decisions and plan actions, such as judging the current container stacking status and the actions to be taken (grasping, lifting, lowering, moving, and releasing), controlling the tightening, releasing, lifting, lowering, and moving actions of the mechanical crane, and converting the instructions generated by the decision-making and planning system into control signals to achieve precise conversion from micro-instructions to macro-actions, including tightening, releasing, lifting, lowering, and moving, to minimize control delays and execution deviations. Realize real-time transmission of control instructions, and adjust the control strategy according to the dynamics and blocking conditions of the mechanical crane through an adaptive control algorithm.

[0138] In a feasible implementation, Figure 4 This is a case diagram showing the processing method of the CRE causal relationship enhanced model training method in three application areas: container stacking, natural language text data processing, and cargo stacking. For easy distinction, different colors are used to mark containers and cargo. In the container stacking scenario, the input content of the training phase includes: initial state, target state, and path, and the output content includes: reasoning path; the input content of the testing phase includes: initial state, target state, and output content includes: reasoning path. Examples are: initial state: orange tiles are on the table, blue tiles are on the table, red tiles are on the table, and yellow tiles are on top of red tiles; target state: orange tiles are on blue tiles, yellow tiles are on orange tiles, blue tiles are on the table, and red tiles are on the table. The first step of the reasoning process, the OSR options are picking up orange and picking up blue. Picking up blue violates the cross entropy restriction. By Figure 4 As shown in Figure 2, the two situations will lead to an increase in |ATE| (i.e., OSR and SR lose their causal relationship). Therefore, |ATE| achieves the correct SR selection under the premise of correct OSR. The second step of the reasoning process simplifies the description of the process.

[0139] (3) Cargo stacking: Use the DES-enhanced long-range reasoning method based on the CRE short-range reasoning module to make decisions and plan actions, such as judging the current cargo stacking status and the actions to be taken (grasping, lifting, lowering, moving, and releasing), controlling the tightening, releasing, lifting, lowering, and moving actions of the mechanical crane, and converting the instructions generated by the decision-making and planning system into control signals to achieve accurate conversion from micro-instructions to macro-actions, including tightening, releasing, lifting, lowering, and moving, so as to minimize control delays and execution deviations. Realize real-time transmission of control instructions (delay less than 5ms), and adjust the control strategy according to the dynamics and blocking conditions of the mechanical crane through an adaptive control algorithm.

[0140] like Figure 4As shown in the figure, in the cargo stacking scenario, the input content of the training phase includes: initial state, target state and path, and the output content includes: reasoning path; the input content of the test phase includes: initial state, target state, and output content includes: reasoning path. For example: initial state: pile A: 1 pile B: 2 pile C: 3 target state pile A: {null} pile B: {null} pile C: 123. The first step of the reasoning process OSR options are to move 2 from B to C and move 1 from B to C. Moving 1 from B to C violates the cross entropy limit. By Figure 4 As shown in the figure, the two situations will lead to an increase in |ATE| (i.e., OSR and SR lose causal connection). Therefore, |ATE| achieves the correct SR selection under the premise of correct OSR. The second step of the reasoning process simplifies the description of the process. Compared with the container stacking scenario, the cargo stacking scenario further strengthens the stacking order restrictions of the weight size of the object blocks.

[0141] (4) Numerical calculation of text in natural language text:

[0142] like Figure 4 As shown in the figure, in the natural language text data scenario, we focus on processing English text data. The input content of the training phase includes: initial state, target state and path, and the output content includes: reasoning path. An example is: Andy receives a monthly salary of $800 but he has to pay a tax of 7%.

[0143] How much is his net salary?

[0144] Example Step 1: Similar to the container stacking scenario, the left and right selections of the table are controlled by cross entropy, and the selection between A1 and A2 is controlled by |ATE|. The combination of the two realizes the causal control between OSR and SR.

[0145]

[0146] Example step 2:

[0147]

[0148] Final answer: 744.

[0149] It should be noted that in order to ensure the continuous updating and iteration of the causal enhancement module, the causal domain knowledge can be extracted through multi-machine interaction and expert feedback during the process of reasoning state control according to the above steps, which can specifically include the following steps Z1-Z3:

[0150] Z1 targets the following business scenarios: autonomous driving navigation, container stacking, natural language text processing, and cargo stacking.

[0151] Z2, multi-person computer interaction mode: It can construct demonstration data for imitation learning through domain experts, and can also enhance business data through the data generated during the actual operation of the large language decision model. Types of human-computer interaction modes include: manually setting positive and negative samples, and manually evaluating the model output and correcting the positive and negative feedback of sample classification. Specifically include:

[0152] Z21, Demonstration Data Construction: Domain experts can provide high-quality training samples for the model to imitate and learn by constructing demonstration data. These demonstration data can significantly improve the initial learning ability of the model.

[0153] Z22, Data Enhancement: The data generated during the actual operation of the large language decision model can be used to enhance business data. By combining the actual operation data with the original training data, the model can better adapt to the actual application scenario.

[0154] Z3, multi-expert feedback type: including expert voting, confidence analysis and uncertainty analysis methods, to monitor and evaluate the entire process of model training, testing and application. Specifically include:

[0155] Z31, Expert Voting: Determine the accuracy and reliability of the model output through voting decisions by multiple field experts.

[0156] Z32, Confidence Analysis: Analyze the confidence of the model, identify the uncertainty areas of the model during the prediction process, and make corresponding adjustments and optimizations.

[0157] Z33, Uncertainty Analysis: By analyzing the uncertainty of the model output, evaluate its adaptability and reliability in different business scenarios, and ensure that the model can maintain a high level of performance in various application environments.

[0158] The expert feedback system for human-computer interaction is implemented based on the Unreal Engine (UE) and JSBSim platforms. Unreal Engine is an advanced game engine and development framework for creating highly realistic 3D games and applications based on graphics technology and hardware acceleration. Users are able to customize specific tasks, as well as build intelligent decision-making (control) systems through intelligent algorithms. The platform also allows the development of deployment of intelligent agents and multi-agent tasks.

[0159] The UE+JSBSim platform framework can run on Windows. Unreal Engine and JSBSim can be used for C++ and blueprint plug-ins. UE+JSBSim further integrates it into integrated components such as physical modeling, simulation components, environment customization, and rendering pipelines. UE+JSBSim has defined scene APIs, rendering pipeline APIs, VR APIs, physics APIs, audio APIs, and communication APIs to support the development of application platforms. Virtual reasoning simulation environment construction:

[0160] Based on the original communication, motion and physical models of the platform, this project has developed some additional interfaces: 1) API for the reasoning state control method based on the prior knowledge of the large language model. 2) API for the causal domain knowledge extraction method of multi-machine interaction and expert feedback type. In order to provide expert feedback data support for the reasoning state control method based on the prior knowledge of the large language model, this project implements the API for the causal domain knowledge extraction method of multi-machine interaction and expert feedback type to collect external evaluation feedback information of humans on the large model reasoning process. 3) API for human-machine interaction module support.

[0161] The platform implements control and first-person perception based on the A10C Warthog flight stick and Meta Quset Pro, respectively. First, the control mapping and input perception of the flight stick are exposed based on the UE plug-in and joystick package, thereby realizing it. Stream to Meta Quest Pro through SteamVR and Meta Quest Link. Next, configure the UE project to support OpenXR, connect Quest Pro to the PC via USB or wirelessly, and turn on Quest Link mode. After starting SteamVR on the PC, Meta Quest Link will automatically connect to Quest Pro. In the UE editor, set the project parameters, select the "VRPreview" mode to point to OpenXR, and start the application. At this time, the UE program will be displayed directly in the Quest Pro headset.

[0162] The experimental process and results are introduced below:

[0163] 1. The actual business scenario experimental test data is as follows:

[0164] (1) Comparison benchmark method used:

[0165] RAP: Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992, 2023.

[0166] CoT: Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022.

[0167] RoT: Wenyang Hui, Yan Wang, Kewei Tu, and Chengyue Jiang. Rot: Enhancing large language models with reflection on search trees. arXiv preprint arXiv:2404.05449, 2024.

[0168] (2) The comparison benchmark models adopted:

[0169] Llama-2-7B: Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.

[0170] Phi-2-7B: Yuanzhi Li, Sebastian Bubeck, Ronen Eldan, Allie DelGiorno, Suriya Gunasekar, and Yin Tat Lee. ii:phi-1.5 technical report ii:phi-1.5 technical report. arXiv preprint arXiv:2309.05463,

[0171] Mistral-7B: Albert Q Jiang, Alexander Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego De Las Casas, Florian Bressand, Gianna Lengyel, William Lample, Lucile Saulnier, et al. Mistral 7b. arXivpreprint arXiv:2310.06825,

[0172] Mistral-8x7B: Albert Q Jiang, Alexander Sablayrolles, Anthony Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego De Las Casas, Emma Bou Hanna, Florian Bressand, et al. ArXiv Preprint arXiv:2401.04088,

[0173] Qwen1.5-7B: Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609,

[0174] Mpt-7B: MN Team et al. Introducing mpt-7b: A new standard for open-source, commercially usable llms, 2023.

[0175] Baichuan-7B: Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, CeBian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. Baichuan 2:Open large-scale language models. arXiv preprint arXiv:2309.10305, 2023.

[0176] Table 1 below shows the success rate of solving the container stacking problem using different pre-trained models and comparison methods.

[0177] Table 1

[0178]

[0179] Table 2 below shows the success rates of different pre-training models and comparison methods for natural language processing problems.

[0180] Table 2

[0181]

[0182] Table 3 below shows the success rate of solving the problem of cargo stacking using different pre-trained models and comparison methods.

[0183] Table 3

[0184]

[0185] Table 4 below is a performance comparison table of adjusting the pre-trained model by the CRE method in an embodiment of the present invention. AQUA and QASC are both commonly used benchmark performance evaluation data sets in this field.

[0186] Table 4

[0187]

[0188] After testing, a control experiment was designed to ensure that the method does not impair the inherent problem-solving ability and reasoning ability of the model. Since DES is designed specifically for Blocksworld, it is a task step with a long reasoning time, and the listed control experiment does not involve such a lengthy reasoning step; therefore, DES performance was not tested. The experimental results show that the CRE method enhances the model's problem-solving ability on other baseline tasks to a certain extent and does not cause performance degradation.

[0189] In the embodiment of the present invention, a causal relationship enhancement (CRE) mechanism is introduced to improve the rigor of the long-range reasoning method based on LLM, and structural causal modeling (SCM) is used to enhance the causal relationship between OSR and state transition. Causal intervention is performed during the training process and the absolute value of ATE is optimized, which effectively alleviates the problem of causal hallucination of large language models in long-range reasoning tasks and improves the accuracy and reliability of reasoning. At the same time, a bidirectional double-end search (DES) method is proposed. After constructing a causal probability tree with the initial state as the starting point and the target state as the end point, long-range reasoning (such as 12 steps) is converted into a more manageable small segment combination (such as 2 or 4 steps) by minimizing the distance between leaves and using end matching technology. By avoiding long-range sequential search from the beginning, the DES method greatly reduces the complexity of solving long-range reasoning problems and significantly improves the decision-making efficiency and accuracy of large language models in complex reasoning tasks. Especially in the fields of autonomous driving and logistics transportation, accurate decision-making is crucial, which will greatly improve the safety and reliability of related systems and reduce the occurrence of misjudgments and accidents. The technical solution of the present invention is not only applicable to multiple fields such as autonomous driving, logistics transportation, container stacking and cargo stacking, but also has broad application prospects and important commercial value.

[0190] Figure 5 1 is a block diagram of a device for controlling a reasoning state based on a priori knowledge of a large language model according to an exemplary embodiment. The device is used in a method for controlling a reasoning state based on a priori knowledge of a large language model. Figure 5 The device includes an acquisition unit 510, a training unit 520, a segmentation unit 530 and a decision unit 540. Among them:

[0191] An acquisition unit 510 is used to acquire a sample data set of unmanned system decision reasoning for model training;

[0192] A training unit 520 is used to train the initial large language model using a sample data set and a loss function composed of an ATE causal effect estimation algorithm and a cross entropy to obtain a trained large language decision reasoning model;

[0193] A segmentation unit 530 is used to obtain a long-term action decision problem, construct a causal probability tree according to the long-term action decision problem, determine an optimization problem of pruning and expanding directions according to the causal probability tree, prune and expand the causal probability tree by solving the optimization problem, and segment the long-term action decision problem into multiple short-term action decision problems;

[0194] The decision unit 540 is used to solve multiple short-term action decision problems using the trained large language decision reasoning model, obtain an action plan corresponding to each short-term action decision problem, and complete the action according to the action plans corresponding to the multiple short-term action decision problems.

[0195] In the embodiment of the present invention, by introducing the causal relationship enhancement (CRE) mechanism, the problem of causal hallucination of large language models in long-range reasoning tasks is solved, and the accuracy and reliability of reasoning are improved. At the same time, a bidirectional double-end search (DES) method is proposed, which significantly improves the decision-making efficiency and accuracy of large language models in complex reasoning tasks by decomposing the long-range reasoning problem into multiple short-range reasoning problems and searching from the initial state and the target state at the same time. Especially in the fields of autonomous driving and logistics transportation, accurate decision-making is crucial, which will greatly improve the safety and reliability of related systems and reduce the occurrence of misjudgments and accidents. The technical solution of the present invention is not only applicable to multiple fields such as autonomous driving, logistics transportation, container stacking and cargo stacking, but also has broad application prospects and important commercial value.

[0196] Figure 6 It is a structural diagram of an electronic device 600 provided in an embodiment of the present invention. The electronic device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 601 and one or more memories 602, wherein the memory 602 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 601 to implement the steps of the above-mentioned inference state control method based on large language model prior knowledge.

[0197] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, and the instructions can be executed by a processor in a terminal to complete the above-mentioned inference state control method based on a priori knowledge of a large language model. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0198] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

Claims

1. A method for inference state control based on prior knowledge of a large language model, which is applied to autonomous driving discrete navigation problems, container stacking problems, cargo stacking problems, and text numerical calculation problems under natural language texts, characterized in that: The method comprises: S1. Obtain a sample data set of unmanned system decision reasoning for model training; S2, using the sample data set and the loss function composed of the ATE causal effect estimation algorithm and the cross entropy, the initial large language model is trained to obtain a trained large language decision reasoning model; S3. Obtain a long-range action decision problem, construct a causal probability tree based on the long-range action decision problem, determine the optimization problem of pruning and expanding directions based on the causal probability tree, prune and expand the causal probability tree by solving the optimization problem, and divide the long-range action decision problem into multiple short-range action decision problems; S4. Use the trained large language decision-making reasoning model to solve multiple short-term action decision problems, obtain the action plan corresponding to each short-term action decision problem, and complete the action according to the action plans corresponding to the multiple short-term action decision problems; The S2 uses a sample data set and a loss function composed of an ATE causal effect estimation algorithm and a cross entropy to train the initial large language model to obtain a trained large language decision reasoning model, including: S21. The form of the samples in the sample data set is recorded as: IT||GT####<OSR,SR> ; Among them, IT represents the initial state, GT represents the target state, OSR represents the transfer action, and SR represents the state transition; S22. Given binary variables X and Y, X is used to represent the correctness of OSR and Y is used to represent the correctness of SR; ATE is defined as follows: ATE=E(Y|do(X))-E(Y)=E[Y1-Y0] (1) Among them, X, Y ~ B (0, 1), X = 1 means that OSR is correct, Y = 1 means that SR is correct, E () represents the expected calculation formula, do (·) refers to Do-calculus, which means external intervention on the value of Y without affecting the actual state of X; The cross entropy loss function is defined as follows (2): Among them, p 1|X represents the conditional probability that Y is 1 given the X state, p 0|X It represents the conditional probability that Y is 0 given the X state; The loss function composed of the ATE causal effect estimation algorithm and the cross entropy is defined as follows (3): S23, inputting the input data in the sample data set into the initial large language model, comparing the output prediction value of the initial large language model with the sample true value in the sample data set, adjusting the parameters in the initial large language model according to the loss function composed of the ATE causal effect estimation algorithm and the cross entropy until the loss function converges, stopping the training, and determining the obtained large language model as the trained large language decision reasoning model; The sample data set is autonomous driving navigation sample data, container stacking sample data or natural language text sample data; When the sample data set is autonomous driving navigation sample data, the S21 records the format of the samples in the sample data set as IT||GT####<OSR,SR> ,include: The format of the autonomous driving navigation sample data is recorded as: starting point coordinates||destination coordinates####<intersection coordinates 1, intersection action 1>…<intersection coordinates N, intersection action N>; When the sample data set is container stacking sample data, the S21 records the form of the samples in the sample data set as IT||GT####<OSR,SR> ,include: The format of container stacking sample data is recorded as: starting state stacking state||destination state stacking state####<immediate stacking state 1, container moving action 1>...<immediate stacking state N, container moving action N>; When the sample data set is natural language text sample data, the S21 records the form of the samples in the sample data set as IT||GT####<OSR,SR> ,include: The form of natural language text sample data is recorded as: original text question||final text answer####<intermediate solution process 1, intermediate answer 1>…<intermediate solution process 1, intermediate answer N>; Among them, the S3 obtains the long-term action decision problem, constructs a causal probability tree according to the long-term action decision problem, determines the optimization problem of pruning and expanding directions according to the causal probability tree, and prunes and expands the causal probability tree by solving the optimization problem, and divides the long-term action decision problem into multiple short-term action decision problems, including: S31, obtaining a long-range action decision problem, wherein the long-range action decision problem includes an initial state and a target state; S32. construct two causal probability trees according to the initial state and the target state, wherein a node in each causal probability tree represents a state in the reasoning process, and an arrow represents a causal relationship; S33. Calculate the ATE between the expansion and distance reduction between two causal probability trees according to the following formula (4): ATE(A)=E(A|do(B))-E(A) (4) Where A is N i Relative to N i-1 The reduced distance D, B is the distance between the current leaf and N i The number of expansion layers, the distance D is calculated according to the spatial coordinates, by calculating the Euclidean distance between the current position of the block and the target position coordinates; S34. Based on the causal probability tree and the above formula (4), an optimization problem of the following formula (5) is constructed: Among them, δD represents the number of tree expansion layers N i The distance reduction, T head Represents the head tree of the causal probability tree, T tail Represents the tail tree of the causal probability tree; S35, by minimizing the optimization problem Solving the optimization problem to obtain a scheme for pruning and expanding the causal probability tree; S36. Prune and expand the causal probability tree according to the pruning and expansion scheme of the causal probability tree, so as to obtain multiple short-term action decision problems separated from the long-term action decision problem.

2. The inference state control method based on large language model prior knowledge according to claim 1 is characterized in that: The S1 acquires a sample data set of unmanned system decision reasoning for model training, including: S11, converting the multi-source heterogeneous original action decision data in various formats into a string sequence, converting the string sequence into a vector, and storing it in a vector database; S12, performing noise filtering, outlier detection, standardization processing, and structure processing on the vector data in the vector database to obtain preprocessed sample data; S13, labeling the preprocessed sample data to obtain labeled sample data; S14. Perform data format consistency processing on the labeled sample data to obtain a sample data set for unmanned system decision reasoning used for model training.

3. A reasoning state control device based on a priori knowledge of a large language model, the device is applied to autonomous driving discretized navigation problems, container stacking problems, cargo stacking problems and text numerical calculation problems under natural language texts, the reasoning state control device based on a priori knowledge of a large language model is used to implement the reasoning state control method based on a priori knowledge of a large language model as described in any one of claims 1-2, characterized in that, The device comprises: An acquisition unit, used for acquiring a sample data set of unmanned system decision reasoning for model training; A training unit is used to train the initial large language model using a sample data set and a loss function composed of an ATE causal effect estimation algorithm and a cross entropy to obtain a trained large language decision reasoning model; A segmentation unit is used to obtain a long-range action decision problem, construct a causal probability tree based on the long-range action decision problem, determine the optimization problem of pruning and expanding directions based on the causal probability tree, prune and expand the causal probability tree by solving the optimization problem, and segment the long-range action decision problem into multiple short-range action decision problems; A decision-making unit is used to solve multiple short-term action decision problems using a trained large language decision reasoning model, obtain an action plan corresponding to each short-term action decision problem, and complete the action according to the action plans corresponding to the multiple short-term action decision problems; Wherein, the training unit is used for: S21. The form of the samples in the sample data set is recorded as: IT||GT####<OSR,SR> ; Among them, IT represents the initial state, GT represents the target state, OSR represents the transfer action, and SR represents the state transition; S22. Given binary variables X and Y, X is used to represent the correctness of OSR and Y is used to represent the correctness of SR; ATE is defined as follows: ATE=E(Y|do(X))-E(Y)=E[Y1-Y0] (1) Among them, X, Y ~ B (0, 1), X = 1 means that OSR is correct, Y = 1 means that SR is correct, E () represents the expected calculation formula, do (·) refers to Do-calculus, which means external intervention on the value of Y without affecting the actual state of X; The cross entropy loss function is defined as follows (2): Among them, p 1|X represents the conditional probability that Y is 1 given the X state, p 0|X It represents the conditional probability that Y is 0 given the X state; The loss function composed of the ATE causal effect estimation algorithm and the cross entropy is defined as follows (3): S23, inputting the input data in the sample data set into the initial large language model, comparing the output prediction value of the initial large language model with the sample true value in the sample data set, adjusting the parameters in the initial large language model according to the loss function composed of the ATE causal effect estimation algorithm and the cross entropy until the loss function converges, stopping the training, and determining the obtained large language model as the trained large language decision reasoning model; The sample data set is autonomous driving navigation sample data, container stacking sample data or natural language text sample data; When the sample data set is autonomous driving navigation sample data, the training unit is used to: The format of the autonomous driving navigation sample data is recorded as: starting point coordinates||destination coordinates####<intersection coordinates 1, intersection action 1>…<intersection coordinates N, intersection action N>; When the sample data set is container stacking sample data, the S21 records the form of the samples in the sample data set as IT||GT####<OSR,SR> ,include: The format of container stacking sample data is recorded as: starting state stacking state||destination state stacking state####<immediate stacking state 1, container moving action 1>...<immediate stacking state N, container moving action N>; When the sample data set is natural language text sample data, the S21 records the form of the samples in the sample data set as IT||GT####<OSR,SR> ,include: The form of natural language text sample data is recorded as: original text question||final text answer####<intermediate solution process 1, intermediate answer 1>…<intermediate solution process 1, intermediate answer N>; Wherein, the segmentation unit is used for: S31, obtaining a long-range action decision problem, wherein the long-range action decision problem includes an initial state and a target state; S32. construct two causal probability trees according to the initial state and the target state, wherein a node in each causal probability tree represents a state in the reasoning process, and an arrow represents a causal relationship; S33. Calculate the ATE between the expansion and distance reduction between two causal probability trees according to the following formula (4): ATE(A)=E(A|do(B))-E(A) (4) Where A is N i Relative to N i-1 The reduced distance D, B is the distance between the current leaf and N i The number of expansion layers, the distance D is calculated according to the spatial coordinates, by calculating the Euclidean distance between the current position of the block and the target position coordinates; S34. Based on the causal probability tree and the above formula (4), an optimization problem of the following formula (5) is constructed: Among them, δD represents the number of tree expansion layers N i The distance reduction, T head Represents the head tree of the causal probability tree, T tail Represents the tail tree of the causal probability tree; S35, by minimizing the optimization problem Solving the optimization problem to obtain a scheme for pruning and expanding the causal probability tree; S36. Prune and expand the causal probability tree according to the pruning and expansion scheme of the causal probability tree, so as to obtain multiple short-term action decision problems separated from the long-term action decision problem.

4. The inference state control device based on large language model prior knowledge according to claim 3 is characterized in that: The acquisition unit is used to: S11, converting the multi-source heterogeneous original action decision data in various formats into a string sequence, converting the string sequence into a vector, and storing it in a vector database; S12, performing noise filtering, outlier detection, standardization processing, and structure processing on the vector data in the vector database to obtain preprocessed sample data; S13, labeling the preprocessed sample data to obtain labeled sample data; S14. Perform data format consistency processing on the labeled sample data to obtain a sample data set for unmanned system decision reasoning used for model training.

5. A reasoning state control device based on a priori knowledge of a large language model, characterized in that: The reasoning state control device based on large language model prior knowledge includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 2 is implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 2.

Images