An intelligent reasoning method and system based on global search combined with external information

Abstract

The present invention relates to the field of artificial intelligence technology, and specifically to an intelligent reasoning method and system based on global search combined with external information, wherein an intelligent reasoning method based on global search combined with external information includes the following steps: S1: using a pre-trained large language model to generate candidate actions in the current state, synchronously outputting local generation probabilities, retrieving external support data based on the current state, and calculating the external support compensation weight of the candidate action; S2: calculating the immediate benefits of the candidate action through the local generation probability and external support compensation weight of the candidate action; S3: calculating the comprehensive selection value of the candidate action through an upper confidence interval algorithm; S4: outputting the global optimal reasoning chain. The present invention can achieve a deep fusion of local generation and global search, has significant improvements in candidate action evaluation and dynamic integration of external information, and greatly improves the accuracy, coherence and system robustness of multi-step reasoning tasks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent reasoning method and system based on global search combined with external information. Background Art

[0002] In recent years, large language models have achieved breakthroughs in natural language understanding and generation. For example, the excellent performance of the GPT series and BERT models in tasks such as question answering, text generation, and translation has been widely verified. However, when dealing with multi-step complex reasoning, building problem chains, or invoking cross-domain knowledge, relying solely on the local generation capabilities of large language models can easily lead to local optimality. Long-range dependencies often lead to incoherent reasoning chains and accumulated errors. Furthermore, large language models have limitations in knowledge representation and external information invocation. When internal knowledge is insufficient or when facing cross-domain problems, relying solely on model generation often fails to cover all necessary information.

[0003] On the other hand, Monte Carlo Tree Search, a global search algorithm based on random simulation and statistical feedback, has demonstrated high effectiveness in decision-making problems such as Go and chess, exemplified by the successful implementation of AlphaGo. However, traditional Monte Carlo Tree Search methods rely primarily on fixed reward feedback and static exploration parameters, making them inadequate for addressing the uncertainty and ambiguity inherent in natural language generation. Furthermore, when handling complex multi-step logical reasoning, a single search mechanism often overlooks the rich semantics and contextual relationships inherent in the information generated by large language models.

[0004] Currently, some existing technologies attempt to combine candidate steps generated by large language models with traditional search algorithms, such as thought chaining and self-consistency strategies, to improve inference accuracy. However, these methods typically use fixed thresholds or static weights to determine external knowledge retrieval, failing to dynamically adjust based on the uncertainty expressed by candidate actions. Consequently, they lack flexible adaptive mechanisms during knowledge replenishment and global optimization, limiting the deep integration of local generation and global search. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: the existing intelligent reasoning method mainly relies on the confidence generated locally by the model itself, and is prone to falling into local optimality during multi-step reasoning. At the same time, external information is insufficiently called, and the reasoning effect is poor. In addition, since the existing global optimization mechanism is difficult to adapt to the ambiguity and uncertainty in natural language generation, it is difficult to achieve the optimal balance between efficiency and quality in the search process.

[0006] To solve the above technical problems, the first aspect of the present invention adopts the following technical solution: an intelligent reasoning method based on global search combined with external information, comprising the following steps:

[0007] S1: Use the pre-trained large language model to generate candidate actions in the current state, synchronously output the local generation probability, retrieve external support data based on the current state, and calculate the external support compensation weight of the candidate action;

[0008] S2: Calculate the immediate benefit of the candidate action through the local generation probability of the candidate action and the external support compensation weight;

[0009] S3: Calculate the comprehensive selection value of the candidate action through the upper confidence interval algorithm, where a dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight to modify the exploration term in real time;

[0010] S4: Select the candidate action with the highest comprehensive selection value, update the current state, and go to step S1 until the global optimal reasoning chain is output.

[0011] When the present invention works, it can achieve a deep fusion of local generation and global search, with significant improvements in candidate action evaluation and dynamic integration of external information, greatly improving the accuracy, coherence and system robustness of multi-step reasoning tasks. By combining a fusion mechanism with external support compensation weights, it overcomes the problems of insufficient information and error accumulation caused by a single reliance on the local confidence of a large language model, and can automatically adjust the exploration intensity according to the generation quality of the current candidate step and the support of external information, ensuring the efficiency and quality of the search process.

[0012] Preferably, step S1 further includes the following steps: dynamically calculating the uncertainty index of the candidate action; when the uncertainty index of the candidate action is greater than a preset threshold, determining that insufficient information is generated internally in the large language model, and actively calling external support data to supplement the semantic information of the current state.

[0013] When the present invention is working, by dynamically calculating the uncertainty index of candidate actions, it can provide real-time feedback and adjustment on the call of external information, ensure the supplementation of external information in the generation process, and can timely and accurately supplement the necessary external knowledge and correct the generation confidence when the confidence level of the large language model is low, the semantics are vague, or even the cross-domain knowledge is insufficient, thereby improving the global consistency and stability of the entire reasoning chain.

[0014] Preferably, in step S1, when retrieving external support data according to the current state and calculating the external support compensation weight of the candidate action, the following steps are specifically adopted:

[0015] A1: Use the encoder to map the current state and candidate actions into embedding vectors, retrieve external support data based on the current state, and obtain the support vector;

[0016] A2: Calculate the similarity between the embedding vector and the support vector and normalize the similarity;

[0017] A3: Obtain the statistical characteristics of the normalized similarity in the candidate set, combine its mean and standard deviation, perform nonlinear scaling on the normalized similarity, calculate the external support compensation weight, and output the external support compensation weight used to adjust the local generation probability output by the large language model.

[0018] When the present invention works, it calculates the external support compensation weight of the external information according to the statistical distribution of the candidate set through a nonlinear scaling model, thereby dynamically adjusting the locally generated information. Through a multi-dimensional evaluation method, it can effectively overcome the error accumulation and lack of robustness that may occur when relying solely on local information.

[0019] Preferably, in step A1, when external support data is retrieved according to the current state and a support vector is obtained, the following steps are adopted to measure the frequency of occurrence of words and phrases in the current state, and to make corrections by measuring the rarity of words and phrases in the external support data, to extract keywords in the current state, and at the same time, by constructing a dependency tree of the current state, to supplement keywords that have not been extracted before, to integrate and output a keyword set, to query external support data through parallel processing, and to merge the query results to output the final support vector.

[0020] When the present invention is working, by constructing a dependency tree, it can supplement the keywords not extracted by frequency statistics, thereby improving the comprehensiveness and accuracy of keyword extraction. At the same time, by querying external support data through parallel processing, the query process of external support data is accelerated, which can further improve the generation efficiency and generation quality of large language models.

[0021] Preferably, the step A1 further includes the following steps: performing fine-grained analysis on the candidate actions in conjunction with the multi-head attention mechanism to improve the matching degree between the output candidate actions and the actual semantic requirements.

[0022] Preferably, in step A2, when calculating the similarity between the embedding vector and the support vector, the following steps are adopted to calculate the cosine similarity between the embedding vector and the support vector on several attention heads, calculate the multi-head attention weights by weighted summation, and output the similarity between the embedding vector and the support vector.

[0023] When working, the present invention adopts a multi-head attention mechanism to perform fine-grained analysis of candidate actions, combining global feature matching with the importance allocation of local subspaces, so that the large language model can focus on the similarities of different semantic spaces, and enhance the robustness of similarity calculation through context-aware similarity enhancement.

[0024] Preferably, in step A3, the statistical characteristics of the normalized similarity in the candidate set are obtained, and the normalized similarity is nonlinearly scaled in combination with its mean and standard deviation, and the external support compensation weight is calculated. When the external support compensation weight for adjusting the local generation probability output by the large language model is output, the following steps are adopted:

[0025] B1: Obtain the normalized similarity of candidate actions in the candidate set and calculate the mean and standard deviation of the normalized similarity;

[0026] B2: Convert the normalized similarity of each candidate action into a standardized score;

[0027] B3: Combine the local generation probability of the candidate action and the preset hyperparameters to set a dynamic adjustment coefficient, and use the dynamic adjustment coefficient to correct the standardized score of the candidate action;

[0028] B4: Map the normalized scores of the candidate actions to a smooth nonlinear interval, and after making corrections based on the local generation probability of the candidate actions, output an external support compensation weight for adjusting the local generation probability output by the large language model.

[0029] Preferably, in step S3, the comprehensive selection value of the candidate action is calculated by the upper limit confidence interval algorithm, wherein the dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight to correct the exploration item in real time, the following steps are adopted: the comprehensive selection value of the candidate action is calculated by the upper limit confidence interval algorithm, the dynamic exploration constant is set, the dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight, and when the local generation probability of the candidate action is lower than a preset threshold or the external support compensation weight is lower than a preset threshold, the dynamic exploration constant is increased.

[0030] When the present invention is working, by setting a dynamically changing dynamic exploration constant when calculating the comprehensive selection value of candidate actions in the upper confidence interval algorithm, further automatic tuning of the reinforcement learning method is achieved. When faced with candidate actions with insufficient local confidence or insufficient external support, the exploration probability can be automatically increased, thereby avoiding falling into the local optimum and being unable to search for the best reasoning chain globally.

[0031] Preferably, in step S4, the candidate action with the highest comprehensive selection value is selected, the current state is updated, and the process goes to step S1 until the global optimal reasoning chain is output. Then, the following steps are adopted to select the candidate action with the highest comprehensive selection value, update the current state, and go to step S1. The Monte Carlo tree search is repeatedly called to obtain the cumulative reward of each iteration and update the statistical data of each path under the root node. After the iteration is completed, the global optimal reasoning chain is extracted and output based on the statistical results of the search tree.

[0032] To solve the above technical problems, the second aspect of the present invention adopts the following technical solution: an intelligent reasoning system based on global search combined with external information, applying the above-mentioned intelligent reasoning method based on global search combined with external information, including:

[0033] Large language model module, used to deploy a large language model to generate candidate actions based on the current state;

[0034] The Monte Carlo tree search module is used to construct an inference state tree and output the global optimal inference chain by balancing local generation and global exploration;

[0035] External information module, used to provide external support data;

[0036] The large language model module, the Monte Carlo tree search module, and the external information module are data-connected. The large language model generates a plurality of candidate actions to the Monte Carlo tree search module. The Monte Carlo tree search module outputs a global optimal reasoning chain by balancing local generation and global exploration. When the external information module generates insufficient information within the large language model module, it outputs external support data to the large language module. When the Monte Carlo tree search module balances local generation and global search, the external information module outputs external support data to the Monte Carlo tree search module.

[0037] When the present invention works, a closed-loop feedback system is constructed through the mutual cooperation of the large language model module, the Monte Carlo tree search module and the external information module. The large language model generates candidate actions, the Monte Carlo tree global search, and the automatic call of external support data are efficiently linked. In each step of the generation and search process, resource calls and parameter settings are automatically and dynamically adjusted, achieving real-time information transmission and optimization of the collaborative work of each module, greatly improving the overall reasoning accuracy, coherence and robustness in multi-task scenarios.

[0038] The beneficial technical effects of the present invention include:

[0039] 1. The present invention can achieve a deep fusion of local generation and global search, with significant improvements in candidate action evaluation and dynamic integration of external information, greatly improving the accuracy, coherence and system robustness of multi-step reasoning tasks. By combining a fusion mechanism with external support compensation weights, it overcomes the problems of insufficient information and error accumulation caused by a single reliance on the local confidence of a large language model, and can automatically adjust the exploration intensity based on the generation quality of the current candidate step and the support of external information, ensuring the efficiency and quality of the search process.

[0040] 2. By dynamically calculating the uncertainty index of candidate actions, the present invention can provide real-time feedback and adjustments to the call of external information, ensuring the supplementation of external information during the generation process. When the confidence level of the large language model is low, the semantics are ambiguous, or there is a lack of cross-domain knowledge, the necessary external knowledge can be supplemented and the generation confidence can be corrected in a timely and accurate manner, thereby improving the global consistency and stability of the entire reasoning chain.

[0041] 3. The present invention uses a nonlinear scaling model to calculate the external support compensation weight of external information based on the statistical distribution of the candidate set, thereby dynamically adjusting the locally generated information. Through a multi-dimensional evaluation method, it can effectively overcome the error accumulation and lack of robustness that may occur when relying solely on local information.

[0042] 4. By constructing a dependency tree, the present invention can supplement the keywords not extracted by frequency statistics, thereby improving the comprehensiveness and accuracy of keyword extraction. At the same time, by querying external support data through parallel processing, the query process of external support data is accelerated, which can further improve the generation efficiency and quality of large language models.

[0043] 5. The present invention adopts a multi-head attention mechanism to perform fine-grained analysis of candidate actions, combining global feature matching with the importance allocation of local subspaces, so that the large language model can focus on the similarities of different semantic spaces. Through context-aware similarity enhancement, the robustness of similarity calculation is improved.

[0044] 6. The present invention achieves further automatic tuning of the reinforcement learning method by setting a dynamically changing dynamic exploration constant when calculating the comprehensive selection value of candidate actions using the upper confidence interval algorithm. When faced with candidate actions with insufficient local confidence or insufficient external support, it can automatically increase their exploration probability, thereby avoiding falling into the local optimum and being unable to search for the best reasoning chain globally.

[0045] 7. The present invention constructs a closed-loop feedback system through the mutual cooperation of the large language model module, the Monte Carlo tree search module and the external information module, and efficiently links the large language model model to generate candidate actions, the Monte Carlo tree global search, and the automatic call of external support data. In each step of the generation and search process, the resource call and parameter setting are automatically and dynamically adjusted, realizing the real-time information transmission and the optimization of the collaborative work of each module, greatly improving the overall reasoning accuracy, coherence and robustness in multi-task scenarios.

[0046] Other features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below with reference to the accompanying drawings:

[0048] Figure 1 This is a workflow diagram of an intelligent reasoning method based on global search combined with external information;

[0049] Figure 2 This is a workflow diagram of step S1 in an intelligent reasoning method based on global search combined with external information;

[0050] Figure 3 This is a workflow diagram for step A3 in an intelligent reasoning method based on global search combined with external information;

[0051] Figure 4 This is a structural diagram of an intelligent reasoning system based on global search combined with external information. DETAILED DESCRIPTION

[0052] The following is an explanation and description of the technical solutions of the embodiments of the present invention in conjunction with the drawings of the embodiments of the present invention. However, the following embodiments are only preferred embodiments of the present invention and are not exhaustive. Based on the embodiments in the implementation manner, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0053] In the following description, terms such as "inside", "outside", "up", "down", "left", "right", etc. that indicate directions or positional relationships are only used to facilitate the description of the embodiments and simplify the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, they should not be understood as limiting the present invention. Example 1:

[0054] See also Figure 1 This embodiment discloses an intelligent reasoning method based on global search combined with external information, including the following steps:

[0055] S1: Use the pre-trained large language model to generate candidate actions in the current state, synchronously output the local generation probability, retrieve external support data based on the current state, and calculate the external support compensation weight of the candidate action;

[0056] S2: Calculate the immediate benefit of the candidate action through the local generation probability of the candidate action and the external support compensation weight;

[0057] S3: Calculate the comprehensive selection value of the candidate action through the upper confidence interval algorithm, where a dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight to modify the exploration term in real time;

[0058] S4: Select the candidate action with the highest comprehensive selection value, update the current state, and go to step S1 until the global optimal reasoning chain is output.

[0059] When this embodiment is working, it can achieve a deep fusion of local generation and global search, with significant improvements in candidate action evaluation and dynamic integration of external information, greatly improving the accuracy, coherence and system robustness of multi-step reasoning tasks. By combining the fusion mechanism of external support compensation weights, it overcomes the problems of insufficient information and error accumulation caused by the single reliance on the local confidence of a large language model, and can automatically adjust the exploration intensity according to the generation quality of the current candidate step and the support of external information, ensuring the efficiency and quality of the search process.

[0060] See also Figure 2 Preferably, in step S1, when retrieving external support data according to the current state and calculating the external support compensation weight of the candidate action, the following steps are specifically adopted:

[0061] A1: Use the encoder to map the current state and candidate actions into embedding vectors, retrieve external support data based on the current state, and obtain the support vector;

[0062] A2: Calculate the similarity between the embedding vector and the support vector and normalize the similarity;

[0063] A3: Obtain the statistical characteristics of the normalized similarity in the candidate set, combine its mean and standard deviation, perform nonlinear scaling on the normalized similarity, calculate the external support compensation weight, and output the external support compensation weight used to adjust the local generation probability output by the large language model.

[0064] When this embodiment is working, the external support compensation weight of the external information is calculated according to the statistical distribution of the candidate set through a nonlinear scaling model, thereby dynamically adjusting the locally generated information. Through a multi-dimensional evaluation method, it can effectively overcome the error accumulation and lack of robustness that may occur when relying solely on local information.

[0065] In this embodiment, in step A1, external support data is retrieved according to the current state. When the support vector is obtained, the following steps are adopted to measure the frequency of occurrence of words and phrases in the current state, and to make corrections by measuring the rarity of words and phrases in the external support data, so as to extract the keywords in the current state. In specific implementation, important keywords in the current state can be extracted by a hierarchical keyword extraction method. At the same time, by constructing a dependency tree of the current state, previously unextracted keywords are supplemented, and the keyword set is integrated and output. By querying the external support data through parallel processing, the query results are merged to output the final support vector.

[0066] When this embodiment is working, by constructing a dependency tree, it can supplement the keywords that are not extracted by frequency statistics, thereby improving the comprehensiveness and accuracy of keyword extraction. At the same time, by querying external support data through parallel processing, the query process of external support data is accelerated, which can further improve the generation efficiency and generation quality of large language models.

[0067] As a further improvement of this embodiment, in step A1, the following steps are also included, which cooperate with the multi-head attention mechanism to perform fine-grained analysis on the candidate actions to improve the matching degree between the output candidate actions and the actual semantic requirements.

[0068] Preferably, in step A2, when calculating the similarity between the embedding vector and the support vector, the following steps are adopted to calculate the cosine similarity of the embedding vector and the support vector on several attention heads, calculate the multi-head attention weight by weighted summation, and output the similarity between the embedding vector and the support vector.

[0069] When this embodiment works, a multi-head attention mechanism is used to perform fine-grained analysis of candidate actions, combining global feature matching with the importance allocation of local subspaces, so that the large language model can focus on the similarities of different semantic spaces. Through context-aware similarity enhancement, the robustness of similarity calculation is improved.

[0070] In a specific implementation, in step A2, the similarity between the embedding vector and the support vector is calculated and normalized using the following calculation formula:

[0071] ;

[0072] in, is the cosine similarity between the embedding vector and the support vector, is the embedding vector mapping the state s to the candidate action a through the encoder, Support vectors for external support data retrieval, is the normalized similarity between the embedding vector and the support vector.

[0073] In the specific implementation, in the step S1, the following steps are also included: dynamically calculating the uncertainty index of the candidate action; when the uncertainty index of the candidate action is greater than a preset threshold, it is determined that the internal generated information of the large language model is insufficient, and external support data is actively called to supplement the semantic information of the current state.

[0074] When this embodiment is working, by dynamically calculating the uncertainty index of the candidate action, it can provide real-time feedback and adjustment to the call of external information, ensure the supplementation of external information in the generation process, and can timely and accurately supplement the necessary external knowledge and correct the generation confidence when the confidence level of the large language model is low, the semantics are vague, or even the cross-domain knowledge is insufficient, thereby improving the global consistency and stability of the entire reasoning chain.

[0075] As a further improvement to this example, the uncertainty index of the candidate action is calculated using the following formula:

[0076] ;

[0077] in: is the uncertainty index of the candidate action. When the uncertainty index of the candidate action exceeds the preset threshold, it triggers the re-call and supplement of external information to ensure the integrity of information in the reasoning process. Preferably, the threshold can be further dynamically adjusted according to the generation effect to enhance or suppress the frequency of calling external support data, thereby maintaining a balance between generation quality and generation efficiency. is the local generation probability of the candidate action, Generate logits for action a in state s for the large language model. Example 2:

[0078] See also Figure 3 This embodiment provides an intelligent reasoning method based on global search combined with external information. The similarities with other embodiments will not be repeated here, and the differences will be described in detail below.

[0079] In this embodiment, in step A3, the statistical characteristics of the normalized similarity in the candidate set are obtained, and the normalized similarity is nonlinearly scaled in combination with its mean and standard deviation. The external support compensation weight is calculated, and the external support compensation weight for adjusting the local generation probability output by the large language model is output. The following steps are adopted:

[0080] B1: Obtain the normalized similarity of candidate actions in the candidate set and calculate the mean and standard deviation of the normalized similarity;

[0081] B2: Convert the normalized similarity of each candidate action into a standardized score;

[0082] B3: Combine the local generation probability of the candidate action and the preset hyperparameters to set a dynamic adjustment coefficient, and use the dynamic adjustment coefficient to correct the standardized score of the candidate action;

[0083] B4: Map the normalized scores of the candidate actions to a smooth nonlinear interval, and after making corrections based on the local generation probability of the candidate actions, output an external support compensation weight for adjusting the local generation probability output by the large language model.

[0084] In step B1, the normalized similarity of the candidate actions in the candidate set is obtained, and the mean and standard deviation of the normalized similarity are calculated using the following formula:

[0085] ;

[0086] in: is the mean of the candidate set, which is used to measure the central tendency of the similarity of all candidate actions. N is the number of candidate actions in the candidate set. is the standard deviation of the candidate set, which is used to quantify the degree of dispersion of the similarity distribution.

[0087] In step B2, the normalized similarity of each candidate action is converted into a standardized score using the following calculation formula:

[0088] ;

[0089] in: is the standardized score, is a minimum value used to prevent the denominator from being zero.

[0090] In step B3, the following calculation formula is used to set the dynamic adjustment coefficient in combination with the local generation probability of the candidate action and the preset hyperparameters:

[0091] ;

[0092] in: is a dynamic adjustment coefficient, and base_gamma is a preset hyperparameter used to control the initial strength. By setting the dynamic adjustment coefficient to adjust the local generation probability, the influence of normalized similarity on the external support compensation weight is amplified or suppressed. When the large language model is more confident in the candidate action and the local generation probability of the candidate action is higher, the reward for high similarity is weakened to avoid over-reliance on a single indicator. Conversely, when the local generation probability of the candidate action becomes smaller, it is necessary to rely on similarity judgment to increase the frequency of calling external support data, thereby dynamically balancing the confidence of the large language model and the matching degree of external support data in complex scenarios.

[0093] In step B4, the normalized scores of the candidate actions are mapped to a smooth nonlinear interval and corrected according to the local generation probability of the candidate actions. When outputting the external support compensation weight for adjusting the local generation probability output by the large language model, the following formula is used:

[0094] ;

[0095] in: Compensation weights for external support, is a nonlinear scaling model, It is an attenuation factor. When the local generation probability increases, the amplitude of weight adjustment is reduced. Through conservative adjustment, the call of external support data is reduced, and the generation efficiency is improved.

[0096] Preferably, in step S2, when calculating the immediate benefit of the candidate action by using the local generation probability of the candidate action and the external support compensation weight, the following calculation formula is used:

[0097] ;

[0098] in: is the immediate benefit of the candidate action. Example 3:

[0099] This embodiment provides an intelligent reasoning method based on global search combined with external information. The similarities with other embodiments are not repeated here, and the differences are described in detail below.

[0100] In this embodiment, in step S3, the comprehensive selection value of the candidate action is calculated by the upper limit confidence interval algorithm, wherein the dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight to correct the exploration item in real time. The following steps are adopted: the comprehensive selection value of the candidate action is calculated by the upper limit confidence interval algorithm, the dynamic exploration constant is set, and the dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight. When the local generation probability of the candidate action is lower than a preset threshold or the external support compensation weight is lower than a preset threshold, the dynamic exploration constant is increased.

[0101] When this embodiment is working, by setting a dynamically changing dynamic exploration constant when calculating the comprehensive selection value of candidate actions in the upper confidence interval algorithm, further automatic tuning of the reinforcement learning method is achieved. When faced with candidate actions with insufficient local confidence or insufficient external support, the exploration probability can be automatically increased, thereby avoiding falling into the local optimum and being unable to search for the best reasoning chain globally.

[0102] In specific implementation, the comprehensive selection value of the candidate action is calculated by the upper confidence interval algorithm. When the dynamic exploration constant is generated based on the local generation probability of the candidate action and / or the external support compensation weight to modify the exploration item in real time, the following formula is used:

[0103] ;

[0104] in: is the comprehensive selection value, is the average income, is the number of visits, is the number of visits under candidate action a, It is the dynamic exploration constant. When the local generation probability of a candidate action is low or the external information compensation is insufficient, the dynamic exploration constant automatically increases to promote more exploration of the candidate action, thereby avoiding falling into the local optimum. Example 4:

[0105] This embodiment provides an intelligent reasoning method based on global search combined with external information. The similarities with other embodiments are not repeated here, and the differences are described in detail below.

[0106] In this embodiment, in step S4, the candidate action with the highest comprehensive selection value is selected, the current state is updated, and the process proceeds to step S1. Until the global optimal reasoning chain is output, the following steps are adopted: the candidate action with the highest comprehensive selection value is selected, the current state is updated, and the process proceeds to step S1. The Monte Carlo tree search is repeatedly called to obtain the cumulative reward of each iteration and update the statistical data of each path under the root node. After the iteration is completed, the global optimal reasoning chain is extracted and output based on the statistical results of the search tree.

[0107] In the specific implementation, the following pseudo code shows the main process of an intelligent reasoning method based on global search combined with external information:

[0108] Input: Problem P, pre-trained large model LLM, external information module ExternalModule;

[0109] Output: optimal reasoning chain S*;

[0110] / / Initialize the root state s0 (i.e. problem P);

[0111] s0←P;

[0112] N(s0)←1;

[0113] Function MCTS(s):

[0114] if Terminal(s) or Depth(s) ≥ T_max:

[0115] return Evaluate(s);

[0116] for each candidate action a∈GenerateCandidates(LLM,s):

[0117] / / 1. Use the LLM module to generate candidate actions and calculate the local generation probability pLLM;

[0118] pLLM←Softmax(logits(s,a) output by LLM);

[0119] / / 2. Use the encoder to generate state-action embedding E_sa:

[0120] E_sa←Encoder(s,a);

[0121] / / 3. Obtain support vector r_ext from the external information module, and optimize through hierarchical keyword extraction and cache:

[0122] keywords←TF-IDF weighted extraction(s)+dependency parsing extraction(s);

[0123] r_ext←ExternalModule(keywords,useparallelprocessing);

[0124] / / 4. Calculate the cosine similarity of multi-head attention weights:

[0125] attn_weights←MultiHeadAttention(E_sa,r_ext);

[0126] cos_sim←dot(E_sa,r_ext) / (||E_sa||*||r_ext||);

[0127] weighted_cos_sim←Σ(attn_weights_i*partial_cos_sim_i);

[0128] cos_sim_norm←clip((weighted_cos_sim+1) / 2,0,1);

[0129] / / 5. Adaptive nonlinear scaling calculation compensation weight:

[0130] mean_sim←calculate the mean of all candidate cos_sim_norm;

[0131] std_sim←calculate the standard deviation of all candidate cos_sim_norm;

[0132] z_score←(cos_sim_norm-mean_sim) / (std_sim+ε);

[0133] gamma←base_gamma*(1-(1-pLLM));

[0134] beta←sigmoid(gamma*z_score)*(1-log(1+pLLM) / log(2))+baseline;

[0135] / / 6. Calculate the instant benefit R of the candidate action (composed of LLM confidence and external compensation weight):

[0136] R←pLLM+beta;

[0137] / / 7. Dynamic exploration constant alpha calculation: Combine the square root of (1-pLLM) and (1-cos_sim_norm) and tune through reinforcement learning:

[0138] alpha←sqrt((1-pLLM)+(1-cos_sim_norm));

[0139] / / 8. According to the improved UCT strategy, calculate the exploration item U and the comprehensive selection value F:

[0140] U←alpha*sqrt(ln(N(s)) / (N(s,a)+1));

[0141] F←Q(s,a)+U;

[0142] Select a*=argmax_aF(s,a);

[0143] / / Extend to generate the next state s'=s∪{a*}:

[0144] ifs' is not in the tree:

[0145] Initialize s', N(s')←1, Q(s')←R(s,a*);

[0146] else:

[0147] s'←existing node;

[0148] / / Simulation phase (Rollout): Perform a complete reasoning from state s', and the accumulated reward is recorded as R_total:

[0149] R_total←Rollout(s');

[0150] / / Backtrack update: Update the statistics and Q value of each node and action along the path from s0 to s';

[0151] For each state-action pair (s,a) on the path s0 to s':

[0152] Q(s,a)←(Q(s,a)*N(s,a)+R_total) / (N(s,a)+1);

[0153] N(s,a)←N(s,a)+1,N(s)←N(s)+1;

[0154] returnR_total;

[0155] functionRollout(s):

[0156] R_sum←0,t←0;

[0157] while not Terminal(s) and t <T_max:

[0158] a←GenerateCandidate(LLM,s);

[0159] pLLM←Softmax(logits(s,a) output by LLM);

[0160] E_sa←Encoder(s,a);

[0161] r_ext←ExternalModule(RetrieveKeywords(s));

[0162] cos_sim←cosine similarity(E_sa,r_ext);

[0163] cos_sim_norm←(cos_sim+1) / 2;

[0164] / / Calculate the uncertainty indicator delta (combining pLLM and cos_sim_norm):

[0165] delta←1-(pLLM+cos_sim_norm) / 2;

[0166] if delta>τ:

[0167] r_ext←ExternalModule(RetrieveKeywords(s));

[0168] cos_sim←cosinesimilarity(Encoder(s,a),r_ext);

[0169] cos_sim_norm←(cos_sim+1) / 2;

[0170] beta←NonLinearScaling(cos_sim_norm,current candidate average similarity,standard deviation,ε);

[0171] R←pLLM+beta;

[0172] R_sum←R_sum+R;

[0173] / / Update status: add currently selected action a:

[0174] s←s∪{a};

[0175] t←t+1;

[0176] return R_sum;

[0177] / / Main loop: repeatedly call MCTS(s0) to continuously update the search tree;

[0178] foriteration=1 to N_iter:;

[0179] R_iter←MCTS(s0);

[0180] / / The cumulative reward R_iter of each iteration updates the statistics of each path under the root node;

[0181] / / After the iteration, extract the optimal reasoning chain based on the search tree statistics;

[0182] S*←ExtractBestPath(s0);

[0183] return S*. Embodiment 5:

[0184] See also Figure 4 This embodiment provides an intelligent reasoning system based on global search combined with external information, which applies the intelligent reasoning method based on global search combined with external information described in the above embodiment, including:

[0185] Large language model module, used to deploy a large language model to generate candidate actions based on the current state;

[0186] The Monte Carlo tree search module is used to construct an inference state tree and output the global optimal inference chain by balancing local generation and global exploration;

[0187] External information module, used to provide external support data;

[0188] The large language model module, the Monte Carlo tree search module, and the external information module are data-connected. The large language model generates a plurality of candidate actions to the Monte Carlo tree search module. The Monte Carlo tree search module outputs a global optimal reasoning chain by balancing local generation and global exploration. When the external information module generates insufficient information within the large language model module, it outputs external support data to the large language module. When the Monte Carlo tree search module balances local generation and global search, the external information module outputs external support data to the Monte Carlo tree search module.

[0189] When this embodiment is working, a closed-loop feedback system is constructed through the mutual cooperation of the large language model module, the Monte Carlo tree search module and the external information module. The large language model model generates candidate actions, the Monte Carlo tree global search, and the automatic call of external support data are efficiently linked. Resource calls and parameter settings are automatically and dynamically adjusted in each generation and search process, achieving real-time information transmission and optimization of the collaborative work of each module, greatly improving the overall reasoning accuracy, coherence and robustness in multi-task scenarios.

[0190] The beneficial technical effects of this embodiment include: the present invention can achieve a deep fusion of local generation and global search, has significant improvements in candidate action evaluation and dynamic integration of external information, greatly improves the accuracy, consistency and system robustness of multi-step reasoning tasks, and overcomes the problems of insufficient information and error accumulation caused by a single reliance on the local confidence of a large language model by combining a fusion mechanism of external support compensation weights. It can also automatically adjust the exploration intensity based on the generation quality of the current candidate step and the support of external information, thereby ensuring the efficiency and quality of the search process.

[0191] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art will understand that the present invention includes, but is not limited to, the contents described in the drawings and the above specific embodiments. Any modifications that do not deviate from the functional and structural principles of the present invention are intended to be included within the scope of the claims.

Claims

1. An intelligent reasoning method based on global search combined with external information, characterized in that: The following steps are involved: S1: Generate candidate actions in the current state using the pre-trained large language model, output local generation probabilities simultaneously, retrieve external support data based on the current state, and calculate the external support compensation weight of the candidate actions. Specifically, the following steps are used: The encoder is used to map the current state and candidate actions into embedding vectors. External support data is retrieved based on the current state. The frequency of words and phrases in the current state is measured and corrected by measuring the rarity of words and phrases in the external support data. Keywords in the current state are extracted. At the same time, a dependency tree of the current state is constructed to supplement previously unextracted keywords, and a keyword set is integrated and output. External support data is queried through parallel processing, and the query results are merged to output the final support vector. The similarity between the embedding vector and the support vector is calculated and normalized to obtain the normalized similarity of the candidate actions in the candidate set. The mean and standard deviation of the normalized similarity are calculated, and the normalized similarity of each candidate action is converted into a standardized score. A dynamic adjustment coefficient is set based on the local generation probability of the candidate action and preset hyperparameters. The standardized score of the candidate action is corrected by the dynamic adjustment coefficient. The standardized score of the candidate action is mapped to a smooth nonlinear interval and corrected according to the local generation probability of the candidate action. The external support compensation weight used to adjust the local generation probability output by the large language model is output. S2: Calculate the immediate benefit of the candidate action through the local generation probability of the candidate action and the external support compensation weight; S3: Calculate the comprehensive selection value of the candidate action through the upper confidence interval algorithm, where a dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight to modify the exploration term in real time; S4: Select the candidate action with the highest comprehensive selection value, update the current state, and go to step S1 until the global optimal reasoning chain is output.

2. The intelligent reasoning method based on global search combined with external information according to claim 1, characterized in that: In the step S1, the following steps are also included: dynamically calculating the uncertainty index of the candidate action; when the uncertainty index of the candidate action is greater than a preset threshold, it is determined that the internal generated information of the large language model is insufficient, and external support data is actively called to supplement the semantic information of the current state.

3. The intelligent reasoning method based on global search and external information according to claim 1, characterized in that: The encoder is used to map the current state and candidate actions into embedding vectors. External support data is retrieved based on the current state. When the support vector is obtained, the following steps are also included to cooperate with the multi-head attention mechanism to perform fine-grained analysis on the candidate actions to improve the matching degree between the output candidate actions and the actual semantic requirements.

4. The intelligent reasoning method based on global search and external information according to claim 3, characterized in that: When calculating the similarity between the embedding vector and the support vector, the following steps are used to calculate the cosine similarity of the embedding vector and the support vector on several attention heads, calculate the multi-head attention weights by weighted summation, and output the similarity between the embedding vector and the support vector.

5. The intelligent reasoning method based on global search and external information according to claim 1, characterized in that: In step S3, the comprehensive selection value of the candidate action is calculated by the upper limit confidence interval algorithm, wherein the dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight to correct the exploration item in real time. The following steps are adopted: the comprehensive selection value of the candidate action is calculated by the upper limit confidence interval algorithm, the dynamic exploration constant is set, and the dynamic exploration constant is generated according to the local generation probability of the candidate action and / or the external support compensation weight. When the local generation probability of the candidate action is lower than a preset threshold or the external support compensation weight is lower than a preset threshold, the dynamic exploration constant is increased.

6. The intelligent reasoning method based on global search and external information according to claim 1, characterized in that: In step S4, the candidate action with the highest comprehensive selection value is selected, the current state is updated, and the process goes to step S1. When the global optimal reasoning chain is output, the following steps are adopted: the candidate action with the highest comprehensive selection value is selected, the current state is updated, and the process goes to step S1. The Monte Carlo tree search is repeatedly called to obtain the cumulative reward of each iteration and update the statistical data of each path under the root node. After the iteration is completed, the global optimal reasoning chain is extracted and output according to the statistical results of the search tree.

7. An intelligent reasoning system based on global search combined with external information, applying the intelligent reasoning method based on global search combined with external information as described in any one of claims 1 to 6, characterized in that: include: Large language model module, used to deploy a large language model to generate candidate actions based on the current state; The Monte Carlo tree search module is used to construct an inference state tree and output the global optimal inference chain by balancing local generation and global exploration; External information module, used to provide external support data; The large language model module, the Monte Carlo tree search module, and the external information module are data-connected. The large language model generates a plurality of candidate actions to the Monte Carlo tree search module. The Monte Carlo tree search module outputs a global optimal reasoning chain by balancing local generation and global exploration. When the external information module generates insufficient information within the large language model module, it outputs external support data to the large language module. When the Monte Carlo tree search module balances local generation and global search, the external information module outputs external support data to the Monte Carlo tree search module.

Images