Method, system, device and medium for processing exception of large model inference track

Abstract

The purpose of this invention is to provide a method, system, device, and medium for handling anomalies in large model inference trajectories. The method includes: acquiring the inference trajectory generated when the large model processes a target sample; sequentially dividing the inference trajectory into multiple inference steps; the target sample being a sample from which the large model outputs erroneous inference results; calculating the semantic entropy of each inference step; traversing the semantic entropy of each inference step in pairs to determine the maximum entropy change value; locating a breakpoint in the inference trajectory based on the maximum entropy change value; truncating the inference trajectory based on the position of the breakpoint; and retaining the inference trajectory before the breakpoint as a valid inference trajectory. This solution addresses the problems of wasted valid inference information, high cost, and low efficiency in handling anomalies in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, device, and medium for handling anomalies in large model inference trajectories. Background Technology

[0002] In the field of artificial intelligence, large models (such as generative language models based on the ransformer architecture) have been widely used in many tasks such as text generation, knowledge question answering, and logical reasoning due to their powerful semantic understanding and logical reasoning capabilities.

[0003] However, the reasoning process of large models is inherently a complex black box mechanism. When erroneous reasoning results are output, existing technologies often struggle to pinpoint the specific point where the error occurred. Current mainstream anomaly handling methods mostly focus on correcting the final output or regenerating the entire result, but this approach often discards the entire reasoning process, resulting in the waste of a significant amount of valid reasoning information. Furthermore, current anomaly handling methods cannot trace logical breakpoints in the reasoning process, leading to high costs and low efficiency in handling abnormal reasoning trajectories. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, device and medium for handling anomalies in large model inference trajectories, which solves the problems of a large amount of effective inference information being wasted and the high cost and low efficiency of handling anomaly inference trajectories in the prior art.

[0005] To address the above problems, embodiments of the present invention provide an anomaly handling method for large model inference trajectories, including: The inference trajectory generated when the large model processes the target sample is obtained, and the inference trajectory is divided into multiple inference steps in an orderly manner; the target sample is the sample in which the large model outputs erroneous inference results; the semantic entropy of each inference step is calculated, and the semantic entropy of each inference step is traversed in pairs to determine the maximum entropy change value. Based on the maximum entropy change value, the breakpoint is located in the inference trajectory; the inference trajectory is truncated based on the position of the breakpoint, and the inference trajectory before the breakpoint is retained as the valid inference trajectory.

[0006] In addition, the steps for determining the target sample as described above include: inputting each sample in the preset sample set into the large model for multiple query tests to obtain the inference results output by the large model; calculating the inference accuracy of the large model for each sample based on the inference results, and taking the samples with an accuracy lower than a preset threshold as the target samples.

[0007] In addition, as described above, dividing the reasoning trajectory into multiple reasoning steps in an orderly manner includes: dividing the reasoning trajectory into a sequence of reasoning steps according to preset logical nodes; the preset logical nodes include at least one of: newline character, splitting punctuation, and logical connector; numbering the reasoning steps one by one according to the sequence of reasoning steps to obtain multiple independent reasoning steps consistent with the reasoning order.

[0008] In addition, calculating the semantic entropy of each of the reasoning steps as described above includes: calculating the semantic entropy based on the following formula: ; in, For the current reasoning step s t The semantic entropy, |s t | indicates s t The number of tokens contained in s <t This indicates that the reasoning trajectory is in the current reasoning step s. t All previous historical reasoning steps; w <i In the current reasoning step s t The sequence of all tokens preceding the i-th token, where v is a token in the vocabulary V, representing the candidate output at the current position. The large model is based on s <t and w <i Given the context, predict the conditional probability that the next token is v.

[0009] In addition, as described above, locating breakpoints in the inference trajectory based on the maximum entropy change value includes: taking adjacent inference steps for which the maximum entropy change value is calculated as target inference step pairs; locating the target inference step pairs in the inference trajectory; and setting breakpoints between the target inference step pairs.

[0010] In addition, the method described above also includes: combining the effective reasoning trajectory, the target sample, the standard answer corresponding to the target sample, and the preset reflection guidance into a correction prompt word according to a preset template; inputting the correction prompt word into the large model to obtain the corrected reasoning trajectory.

[0011] This invention also proposes an anomaly handling system for large model inference trajectories, comprising: a data preprocessing module for acquiring the inference trajectory generated when the large model processes a target sample, and orderly dividing the inference trajectory into multiple inference steps; the target sample being a sample in which the large model outputs erroneous inference results; a semantic entropy monitoring module for calculating the semantic entropy of each inference step, traversing the semantic entropy of each inference step in pairs to determine the maximum entropy change value, and locating a breakpoint in the inference trajectory based on the maximum entropy change value; and an anomaly trajectory processing module for truncating the inference trajectory based on the position of the breakpoint, and retaining the inference trajectory before the breakpoint as a valid inference trajectory.

[0012] This invention also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executed by the at least one processor to enable the at least one processor to perform the anomaly handling method for large model inference trajectories as described above.

[0013] This invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for handling anomalies in large model inference trajectories.

[0014] This invention also proposes a computer program product, including computer instructions, which, when executed by a processor, implement the exception handling method for large model inference trajectories as described above.

[0015] This embodiment achieves the location of near-collapse points in the erroneous reasoning process of a large model by orderly segmenting the reasoning trajectory, quantifying semantic entropy, and locating breakpoints. By truncating the reasoning trajectory, a logically coherent and effective reasoning trajectory can be extracted, solving the problems of existing technologies being unable to locate erroneous reasoning nodes and wasting effective information, and reducing the correction cost of anomaly handling. Attached Figure Description

[0016] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0017] Figure 1 This is a flowchart illustrating the anomaly handling method for large model inference trajectories according to the first embodiment of the present invention. Figure 2 This is a flowchart illustrating the anomaly handling method for large model inference trajectories according to the second embodiment of the present invention. Figure 3This is a flowchart illustrating step 101 in the third embodiment of the present invention. Figure 4 This is a flowchart illustrating the anomaly handling method for large model inference trajectories according to the fourth embodiment of the present invention. Figure 5 This is a schematic diagram of the structure of the anomaly handling system for large model inference trajectories according to the fifth embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of the electronic device according to the sixth embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] The first embodiment of this invention relates to an anomaly handling method for large model inference trajectories, such as... Figure 1 As shown, the method in this embodiment specifically includes the following steps 101-103: Step 101: Obtain the inference trajectory generated when the large model processes the target sample, and divide the inference trajectory into multiple inference steps in an orderly manner; The target sample is the sample of incorrect inference results output by the large model.

[0021] Specifically, the inference trajectory refers to the series of intermediate, step-by-step natural language steps generated by the large model in response to the input question before providing the final answer. It represents the complete search path of the large model during the inference process. Large models with different architectures can extract this inference trajectory through their reserved intermediate layer output interfaces. This embodiment does not limit the specific large model or the method of extracting the inference trajectory. The target sample refers to the sample that outputs an incorrect inference result. The determination method can be completed by semantic similarity matching with the preset standard answer or by manual annotation. This embodiment does not impose specific limitations on this.

[0022] After obtaining the inference trajectory, it is divided into inference steps in an orderly manner, following the temporal continuity of the large model's inference process. Specifically, segmentation rules can be pre-defined based on the token generation granularity, such as generating 3-5 consecutive tokens as one step in a text generation task; alternatively, it can be achieved through semantic boundary detection, by calculating the semantic similarity of adjacent tokens, and segmenting when the similarity abruptly exceeds a threshold. This embodiment does not impose specific restrictions on this, as long as each inference step after segmentation has complete logical semantics.

[0023] Step 102: Calculate the semantic entropy of each reasoning step, traverse the semantic entropy of each reasoning step in pairs to determine the maximum entropy change value, and locate the breakpoint in the reasoning trajectory based on the maximum entropy change value.

[0024] Semantic entropy is a core indicator that quantifies the uncertainty and confusion of semantic information in a reasoning step. It is constructed based on the principles of information theory. The higher the semantic entropy, the more ambiguous the reasoning logic of the step, the more scattered the information, and the greater the probability of reasoning bias.

[0025] Specifically, semantic entropy calculation can be implemented by mapping the intermediate semantic representations of each inference step to a probability distribution. For example, the hidden state vector can be converted into the generation probability distribution of each token using a softmax function, and then substituted into the entropy calculation formula in information theory. Furthermore, for specific professional fields, weighted semantic entropy can be introduced, assigning higher weights to key semantic units such as core entities and logical connectors, thereby improving the targeting of entropy calculation. This embodiment does not limit the specific semantic entropy calculation method.

[0026] After calculating the semantic entropy of each reasoning step, for every two adjacent reasoning steps, the entropy change value is calculated sequentially, which is the difference between the semantic entropy of the subsequent step and the semantic entropy of the preceding step. By traversing the calculation, the maximum entropy change value is found, and then the logical breakpoint in the reasoning trajectory is located. Since the semantic entropy remains relatively stable during normal reasoning, but when a logical break or erroneous derivation occurs, the semantic entropy between adjacent steps will undergo a significant abrupt change. The connection point between adjacent steps corresponding to the maximum entropy change value is the node where the logical transition in reasoning is most drastic, which is also the key breakpoint where the error occurred.

[0027] Step 103: Cut off the inference trajectory based on the position of the breakpoint, and retain the inference trajectory before the breakpoint as a valid inference trajectory.

[0028] Specifically, the semantic entropy fluctuations in the reasoning steps before the breakpoint are gradual, the reasoning logic is coherent, and the information is valid. However, the reasoning after the breakpoint is semantically chaotic due to logical breaks, and is considered an invalid reasoning step. In practical applications, the breakpoint can be located by indexing the reasoning trajectory, cutting off and discarding all reasoning data after the breakpoint, while retaining complete data such as intermediate semantic representations, attention weights, and generated token sequences before the breakpoint, forming a valid reasoning trajectory. Furthermore, the current steps in this embodiment can be extended according to the task type: for knowledge-based question-answering tasks, if the valid reasoning trajectory already contains core knowledge points, subsequent reasoning can be directly supplemented based on this trajectory to generate the correct result; for text creation tasks, semantic coherence can be optimized based on the valid reasoning trajectory to avoid repeated generation; at the same time, the valid reasoning trajectory can be stored in a dedicated database as a reference for subsequent reasoning of similar samples, thereby improving the processing efficiency of similar tasks.

[0029] Compared with related technologies, this embodiment achieves the location of near-collapse points in the erroneous reasoning process of large models by orderly segmenting the reasoning trajectory, quantifying the semantic entropy, and locating breakpoints. By truncating the reasoning trajectory, a logically coherent and effective reasoning trajectory can be extracted, solving the problems of existing technologies being unable to locate reasoning error nodes and wasting effective information, and reducing the correction cost of anomaly handling.

[0030] The second embodiment of this invention relates to an anomaly handling method for large model inference trajectories. This embodiment supplements the foregoing embodiments, specifically by providing supplementary explanations of the steps for determining the target sample, such as... Figure 2 As shown, the specific steps include 201-202: Step 201: Input the samples from the preset sample set into the large model one by one for multiple query tests, and obtain the inference results output by the large model.

[0031] Specifically, the preset sample set in this embodiment refers to a standardized sample set constructed based on the application scenario and task type of the large model. Appropriate data can be selected to construct the sample set according to the actual training needs of the student model and the application domain. The preset sample set needs to contain a large number of question samples with standard answers (such as math problems, logic reasoning problems, etc.), and the standard answers can serve as the basis for judging the accuracy of subsequent reasoning results. Since the generation process of the large model has a certain degree of randomness, a single reasoning result is difficult to reflect the model's stable processing ability for that sample. Multiple tests can effectively offset the interference of accidental factors and highlight the model's inherent reasoning defects on that sample. Therefore, for each sample (question) in the sample set, multiple inputs are made into the large model for sampling, and the corresponding reasoning results of the large model are obtained. The reasoning results can be obtained through the API call interface of the large model, recording the complete output text, generation time, and other auxiliary information for each reasoning.

[0032] Step 202: Calculate the reasoning accuracy of the large model for each sample based on the reasoning results, and use the samples with an accuracy lower than the preset threshold as target samples.

[0033] Specifically, the correctness of the reasoning result is determined by comparing it with the standard answer of the sample. The reasoning accuracy of each sample is determined by statistically analyzing the reasoning accuracy across multiple reasoning results. The specific calculation method for the reasoning accuracy is as follows: First, each obtained reasoning result is judged as "correct or incorrect." The judgment criteria can be flexibly adjusted according to the task type. For example, for objective questions, a field matching method can be used; if the reasoning result is completely consistent with the standard answer, it is considered correct. For subjective questions, the semantic similarity between the reasoning result and the standard answer can be calculated; if the similarity is higher than a set threshold, it is considered correct. This embodiment does not limit the specific judgment method. If the reasoning accuracy of a certain question q in the N reasoning results output by the large model is less than the preset threshold, it indicates that the sample has inherent characteristics that cause the model's reasoning to fail, and it belongs to the sample that needs to be analyzed in detail, thus being marked as a target sample. This target sample can be used for subsequent anomaly identification of reasoning trajectory.

[0034] Compared with related technologies, this embodiment determines the target sample by repeatedly asking the sample questions in the preset sample set, avoiding the randomness error of a single inference, ensuring that the selected target sample can truly reflect the inference defects of the model, and providing a reliable data foundation for the subsequent anomaly analysis of the inference trajectory.

[0035] The third embodiment of the present invention relates to an anomaly handling method for large model inference trajectories. This embodiment is a supplement to the foregoing embodiments, and the supplement lies in refining the process of orderly dividing the inference trajectory into multiple inference steps in step 101, such as... Figure 3 As shown, the specific steps include the following.

[0036] Step 301: Divide the reasoning trajectory into a sequence of reasoning steps according to preset logical nodes; the preset logical nodes include at least one of the following: line break, dividing punctuation, and logical connector.

[0037] Step 302: Number each reasoning step according to the reasoning step sequence to obtain multiple independent reasoning steps that are consistent with the reasoning order.

[0038] Specifically, in this embodiment, the selection of preset logical nodes is based on the expression rules of natural language reasoning text, such as line breaks, punctuation marks, and logical connectors, to ensure that each reasoning step has complete semantic connotation and independent logical function. This embodiment does not impose specific restrictions on the selection of preset logical nodes. The specific segmentation method can be: traversing each character of the thought chain text, and when a preset logical node is detected, splitting the text with that node as the boundary to form a preliminary sequence of reasoning steps. To avoid invalid segmentation, semantic integrity and logical coherence can be checked for each segmented reasoning step. Semantic integrity can be ensured by subject-verb-object structure detection and semantic vector variance calculation, ensuring that the steps express a single and clear reasoning logic; logical coherence can be determined by calculating the semantic similarity between adjacent steps, ensuring that there is a coherent reasoning connection between steps and avoiding logical breaks caused by segmentation. If a step does not meet the semantic integrity requirement, or the semantic similarity between adjacent steps is lower than a preset threshold, the segmentation boundary of the current reasoning step is readjusted until all steps meet the definition of a valid reasoning unit.

[0039] After the node segmentation of the inference trajectory is completed, a unique number (such as S1, S2, ..., Sn) is assigned from front to back according to the step sequence generated by the segmentation. The number is completely consistent with the inference time sequence and semantic derivation order when the large model processes the target sample. Each number uniquely corresponds to an independent inference step. The number information can be associated with and stored with data such as intermediate semantic representations, generation timestamps, and attention weights of the inference steps for easy subsequent analysis.

[0040] In one example, the semantic entropy of each reasoning step is calculated, including: Semantic entropy is calculated based on the following formula: ; in, For the current reasoning step s t The semantic entropy of the current inference step s t It consists of a sequence of tokens, such as {w1, w2, ..., w}. N},|s t | indicates s t The number of tokens contained in s <t This indicates that in the inference trajectory at the current inference step st All previous historical reasoning steps; w <i Indicates the current reasoning step s t The sequence of all tokens preceding the i-th token, where v is a token in the vocabulary V, representing the candidate output at the current position. The large model is based on s <t and w <i Given the context, predict the conditional probability that the next token is v.

[0041] Based on the above formula, the semantic entropy corresponding to each inference step can be calculated. Since the semantic entropy of each inference step reflects the uncertainty and confusion of the semantic information at that step, during normal inference, the semantic entropy change between inference steps should fluctuate slightly within an ideal range. However, once the model faces inference collapse, its inference logic will abruptly change from coherent and clear to chaotic and ambiguous. At this point, the entropy change between the inference steps on both sides of the critical point of this logical abrupt change will reach an extreme value, which is the maximum entropy change value between the inference steps of the inference trajectory.

[0042] In another example, locating breakpoints in the inference trajectory based on the maximum entropy change value includes: The adjacent reasoning steps that calculate the maximum entropy change value are taken as the target reasoning step pairs; Locate the target reasoning step pair in the reasoning trajectory and set breakpoints between the target reasoning step pairs.

[0043] Specifically, the ordered numbers assigned to the inference steps can be directly invoked. By traversing the entropy change value data corresponding to each pair of numbers, extreme value comparisons can be completed to determine the target inference step pair (the two adjacent inference steps with the maximum entropy change value). For special cases where multiple sets of equal maximum entropy change values ​​appear in complex inference scenarios, auxiliary filtering rules can also be set. For example, priority can be given to selecting step pairs with earlier inference sequences, or the optimal target inference step pair can be locked in combination with step semantic integrity verification to ensure the uniqueness of breakpoint location.

[0044] Subsequently, the two target inference steps of the target inference step pair are accurately located in the inference trajectory, and breakpoints are set between them. Since the two target inference steps correspond to the last step of valid inference and the starting step of invalid inference, respectively, the boundary between the steps is the node where inference errors occur and model inference collapses. Setting breakpoints at this point can maximize the preservation of valid inference logic. In specific implementation, the physical location of the target inference step in the original inference trajectory can be matched based on the inference step number, text character position, or model intermediate state timestamp. Breakpoints are marked at the preset logical node boundaries between steps. This marking method is adaptable to both structured and unstructured inference trajectory storage formats, ensuring that subsequent truncation operations can be directly identified and executed.

[0045] Compared with related technologies, the method in this embodiment transforms the abstract entropy change value into a concrete reasoning near-collapse point in the reasoning trajectory, thereby achieving precise location of the breakpoint in the reasoning trajectory. This effectively avoids the truncation error caused by traditional fuzzy location, ensuring the integrity of the effective reasoning trajectory while completely eliminating invalid reasoning content. This significantly improves the execution efficiency of anomaly handling and comprehensively enhances the reliability and applicability of anomaly handling for large model reasoning trajectories.

[0046] The fourth embodiment of this invention relates to an anomaly handling method for large model inference trajectories. This embodiment is a supplement to the foregoing embodiments, such as... Figure 4 As shown, the method in this embodiment specifically includes the following steps 401-402: Step 401: Combine the valid reasoning trajectory, target sample, standard answer corresponding to the target sample, and preset reflection prompts into correction prompts according to the preset template.

[0047] Step 402: Input the corrected prompt words into the large model to obtain the corrected inference trajectory.

[0048] Specifically, to achieve low-cost correction of the reasoning trajectory, this embodiment combines the effective reasoning trajectory, the target sample, the standard answer corresponding to the target sample, and the preset reflection prompts into correction prompts according to a preset template. The correction prompts must provide the large model with complete information on the clear task (target sample), correct basis (effective reasoning trajectory), standard basis (standard answer), and direction of thinking (reflection prompts), ensuring that the large model can correct erroneous reasoning after the breakpoint based on existing correct logic, rather than regenerating the reasoning process, thus improving correction efficiency and ensuring logical coherence. Among them, the effective reasoning trajectory needs to be presented in a format that the large model can understand, which can be transformed into a logical deduction process described in natural language; the target sample directly adopts the original input format, and the standard answer needs to be optimized in presentation according to the task type. Objective questions can be given a clear answer directly, while subjective questions or logical reasoning questions need to be broken down into core points to avoid the large model misunderstanding the direction of correction; the reflection guidance is the core of guiding the large model to review errors and needs to be aligned with the breakpoint location results. For example, the reflection guidance can be: "Combining the previous effective reasoning and the standard answer, analyze the reasons for the logical break at the breakpoint, and based on the core logic of the effective reasoning, supplement a complete and correct reasoning process." or "Please try to answer the above question. Please note that it is strictly forbidden to mention the reference answer during the thinking process, and ensure that you do not reveal that you already know the reference answer in the thinking process and the answer. That is to say, it is strictly forbidden to use expressions such as 'because the reference answer is A' or 'this does not match the reference answer.'" The target sample question q, effective reasoning trajectory t, standard answer a, and reflection guidance p are combined to form a complete teacher model prompt. The template for the combination is as follows: “target sample {q} Standard answer: {a} Valid inference trajectory: {t} Reflection prompts {p}.

[0049] It should be noted that this embodiment does not impose specific restrictions on the format of the splicing template or the order of the prompts in each part, as long as the prompts contain the target sample, the valid reasoning trajectory, the standard answer, and the reflection guidance.

[0050] The correction prompts are input into the large model to obtain the corrected reasoning trajectory. Since the large model possesses the ability to perform semantic understanding and logical completion based on structured prompts, the correct logical starting point is anchored through the effective reasoning trajectory provided by the correction prompts. Combined with the standard answer, the correction direction is clarified. Under the guidance of the reflection prompts, reasoning errors at breakpoints are avoided, generating a coherent and correct complete reasoning trajectory. Furthermore, for long reasoning trajectory correction scenarios, a segmented input strategy can be adopted. First, the effective reasoning trajectory and reflection prompts are input to allow the large model to complete error review. Then, the target sample and standard answer are input to guide the generation of the correction trajectory, improving the accuracy of the correction. In addition, this embodiment also sets up an additional verification mechanism for the correction results. The semantic entropy and entropy change value of the obtained corrected reasoning trajectory are recalculated. If there is no sudden change in entropy exceeding a specified threshold, and the consistency between the reasoning result and the standard answer meets the preset requirements, the correction is deemed effective. If the correction effect is unsatisfactory, the expression of the reflection prompts or the preset template structure can be adjusted, and correction prompts can be regenerated for secondary correction, forming an iterative optimization mechanism until the correction is completed.

[0051] Compared with related technologies, this embodiment integrates effective reasoning trajectories, standard answers, and reflection prompts to construct structured and targeted correction prompts. This makes the correction process of large models no longer blind, but based on existing correct logic for correction and completion. It not only ensures the logical coherence and correctness of the corrected reasoning trajectory, but also avoids the waste of resources caused by repeated reasoning, thus greatly improving the correction efficiency. It solves the problems of lack of basis and logical disconnect in existing correction methods. It can also associate and store the corrected reasoning trajectory with the original effective reasoning trajectory and target samples, providing rich and high-quality data for subsequent model fine-tuning and similar sample processing, and continuously improving the reasoning reliability and robustness of large models.

[0052] The fifth embodiment of the present invention relates to an anomaly handling system for large model inference trajectories, such as... Figure 5 As shown, it includes: The data preprocessing module 501 is used to acquire the inference trajectory generated when the large model processes the target sample, and to divide the inference trajectory into multiple inference steps in an orderly manner; the target sample is the sample of the erroneous inference result output by the large model. The semantic entropy monitoring module 502 is used to calculate the semantic entropy of each reasoning step, traverse the semantic entropy of each reasoning step in pairs to determine the maximum entropy change value, and locate the breakpoint in the reasoning trajectory based on the maximum entropy change value. The abnormal trajectory processing module 503 is used to truncate the inference trajectory based on the position of the breakpoint and retain the inference trajectory before the breakpoint as a valid inference trajectory.

[0053] The system in this embodiment also includes a trajectory correction module, which is specifically used to: combine the effective reasoning trajectory, the target sample, the standard answer corresponding to the target sample, and the preset reflection guidance into correction prompts according to a preset template; input the correction prompts into the large model to obtain the corrected reasoning trajectory.

[0054] To better understand this solution, a specific application example will be used for illustration.

[0055] A highly challenging math competition problem, q, was selected as the target sample: "Suppose the sum of a set of numbers is the sum of its elements. Let S be a set of positive integers, where each number is no greater than 15. Assume that no two disjoint subsets of S have the same sum. What is the maximum possible sum of the set S with these properties?" The standard answer is 61.

[0056] The processing flow for the inference trajectory is as follows: The original error trajectory generated by the teacher model is as follows (the delimiter is "\n\n"): s1: To solve this problem, we have a set S consisting of positive integers, where each number is no more than 15, and any two disjoint subsets of S have different sums. The goal is to find the maximum possible sum of such a set.

[0057] s2: Key observation – Set S must be a set of distinct positive integers (because mathematically it is a set).

[0058] s3: Since any two subsets with the same sum imply the existence of two disjoint subsets with the same sum (by removing their intersection), the problem actually requires all 2^n subsets to have the same sum. |S| Each subset must have a different sum.

[0059] s4: In this case, to maximize the sum, we can choose the set of the four largest consecutive integers {12, 13, 14, 15}. This guarantees the uniqueness of the sum because the differences between the elements are too small to cause conflicts in the sums between subsets of different sizes. (Incorrect step: The selected set {12, 13, 14, 15} does not satisfy the condition "no two disjoint subsets have the same sum" because 12 + 15 = 27 and 13 + 14 = 27, making the set invalid, and its sum of 54 is not the maximum value.) s5: ... (The following is all based on incorrect derivations of s4)... The system calculates the entropy value for each step: H(s1) ≈ 0.2 (very low, indicating that the teacher model is very confident at this point) H(s2) ≈ 0.3 (very low) H(s3) ≈ 0.4 (lower) H(s4) ≈ 2.8 (At this point, the value surges from 0.4 to 2.8, indicating that the model doesn't know how to proceed and starts to make random assumptions, resulting in extremely high uncertainty.) ... Calculate the gradient: Δ = H(s4) - H(s3). This is the global maximum point, so the breakpoint t* is determined to be located between s3 and s4. That is, the system decides to keep s1, s2, and s3, and cut off s4 and the subsequent inference trajectory.

[0060] After local hint injection, the overall hint given to the teacher model is: "Suppose the sum of a set of numbers is the sum of its elements. Let S be a set of positive integers, where each number is no greater than 15. Assume that no two disjoint subsets of S have the same sum. What is the largest possible sum of the set S with these properties?" Suggested answer: 61 Partial thought process: To solve this problem, we know that a set S consists of positive integers, each number being at most 15, and that the sums of any two disjoint subsets are different. The goal is to find the maximum possible sum of such a set S.

[0061] Key observation – Set S must consist of distinct positive integers (from a mathematical perspective).

[0062] Since any two subsets with the same sum imply the existence of two disjoint subsets with the same sum (which can be solved by removing their intersection), the problem actually requires all 2^2 subsets to have the same sum. |S| Each subset must have a distinct sum.

[0063] To maximize the sum under this condition, we can choose four largest consecutive integers {12, 13, 14, 15}. This guarantees the uniqueness of the sum because the differences between the elements are too small to cause conflicts in the sum between subsets of different sizes.

[0064] Please attempt to answer the above questions. Please note that you are strictly prohibited from mentioning the reference answer during your thinking process, and you must ensure that you do not reveal that you already know the reference answer in your thought process and solutions. In other words, you are strictly prohibited from using statements such as "because the reference answer is A" or "this does not match the reference answer." After re-entering the large model, the model is guided by the answer and trajectory, and no longer generates the incorrect s4. Instead, it begins to try to apply the cosine theorem, and finally samples and generates multiple correct trajectories that derive the answer 61.

[0065] In summary, it is not difficult to see that this embodiment is a system embodiment corresponding to the aforementioned method embodiments. The implementation details described in the above method embodiments are still applicable and effective in this embodiment, and will not be repeated here.

[0066] The sixth embodiment of the present invention relates to an electronic device, such as... Figure 6 As shown, it includes at least one processor 602; and a memory 601 communicatively connected to at least one processor 602; wherein the memory 601 stores instructions executable by at least one processor 602, the instructions being executed by at least one processor 602 to enable at least one processor 602 to perform any of the above method embodiments.

[0067] The memory 601 and processor 602 are connected via a bus, which may include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 602 and memory 601 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 602 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 602.

[0068] Processor 602 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 601 can be used to store data used by processor 602 during operation.

[0069] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements any of the above-described method embodiments.

[0070] Another embodiment of the present invention relates to a computer program product, including computer instructions that, when executed by a processor, implement any of the above-described method embodiments.

[0071] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0072] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0073] While the embodiments disclosed herein are as described above, the foregoing content is merely for the purpose of facilitating understanding of this disclosure and is not intended to limit this disclosure. Any person skilled in the art to which this disclosure pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope of this disclosure; however, the scope of patent protection of this disclosure shall still be determined by the scope defined in the appended claims.

Claims

1. A method for handling anomalies in the inference trajectory of a large model, characterized in that, include: Obtain the inference trajectory generated when a large model processes target samples, and divide the inference trajectory into multiple inference steps in an orderly manner; The target sample is a sample of the incorrect inference results output by the large model; Calculate the semantic entropy of each reasoning step, traverse the semantic entropy of each reasoning step in pairs to determine the maximum entropy change value, and locate the breakpoint in the reasoning trajectory based on the maximum entropy change value; The inference trajectory is truncated based on the location of the breakpoint, and the inference trajectory before the breakpoint is retained as a valid inference trajectory.

2. The method according to claim 1, characterized in that, The steps for determining the target sample include: The samples in the preset sample set are input one by one into the large model for multiple query tests to obtain the inference results output by the large model. The inference accuracy of the large model for each sample is calculated based on the inference results, and samples with an accuracy lower than a preset threshold are selected as target samples.

3. The method according to claim 1, characterized in that, The reasoning trajectory is divided into multiple reasoning steps in an orderly manner, including: The reasoning trajectory is divided into a sequence of reasoning steps according to preset logical nodes; the preset logical nodes include at least one of the following: line break, dividing punctuation, and logical connector. The reasoning steps are numbered one by one according to the sequence of reasoning steps, resulting in multiple independent reasoning steps that are consistent with the reasoning order.

4. The method according to claim 1, characterized in that, Calculating the semantic entropy of each of the aforementioned reasoning steps includes: The semantic entropy is calculated based on the following formula: ； in, For the current reasoning step s t The semantic entropy, |s t | indicates s t The number of tokens contained in s <t This indicates that the reasoning trajectory is in the current reasoning step s. t All previous historical reasoning steps; w <i In the current reasoning step s t The sequence of all tokens preceding the i-th token, where v is a token in the vocabulary V, representing the candidate output at the current position. The large model is based on s <t and w <i Given the context, predict the conditional probability that the next token is v.

5. The method according to claim 1, characterized in that, Locating breakpoints in the inference trajectory based on the maximum entropy change value includes: The adjacent inference steps that calculate the maximum entropy change value are taken as the target inference step pair; Locate the target reasoning step pair in the reasoning trajectory and set the breakpoints between the target reasoning step pairs.

6. The method according to claim 5, characterized in that, The method further includes: The effective reasoning trajectory, the target sample, the standard answer corresponding to the target sample, and the preset reflection guidance are combined into correction prompts according to the preset template. Input the corrected prompts into the large model to obtain the corrected inference trajectory.

7. An anomaly handling system for large model inference trajectories, characterized in that, include: The data preprocessing module is used to obtain the inference trajectory generated when the large model processes the target sample, and to divide the inference trajectory into multiple inference steps in an orderly manner. The target sample is a sample of the incorrect inference results output by the large model; The semantic entropy monitoring module is used to calculate the semantic entropy of each reasoning step, traverse the semantic entropy of each reasoning step in pairs to determine the maximum entropy change value, and locate the breakpoint in the reasoning trajectory based on the maximum entropy change value. An abnormal trajectory processing module is used to truncate the inference trajectory based on the position of the breakpoint, and retain the inference trajectory before the breakpoint as a valid inference trajectory.

8. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the exception handling method for large model inference trajectories as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the exception handling method for the large model inference trajectory as described in any one of claims 1-6.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the exception handling method for the large model inference trajectory as described in any one of claims 1-6.

Images