Inference method and device of AI model

Abstract

The application provides an inference method and device of an AI model, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a sequence generated by an AI model, the AI model having a thought chain inference capability, and the sequence being generated by the AI model based on an input query request; then, based on the sequence, identifying whether a redundant reflection exists in a thought chain generated by the AI model; when it is identified that the AI model has a redundant reflection, performing a termination operation on the thought chain, so that the AI model terminates thinking and outputs a response content corresponding to the query request. The inference method of the AI model provided by the application effectively alleviates the problem of redundant reflection of the AI model on the basis of ensuring the accuracy of the AI model, avoids the AI model from falling into long thinking, and effectively reduces the computing power consumption of the AI model inference.

Description

[0001] This application claims priority to Chinese patent application No. 202510537928.1, filed on April 27, 2025, entitled "A Reasoning Method and Apparatus for an AI Model", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence (AI) technology, and in particular to a reasoning method and apparatus for an AI model. Background Technology

[0003] Since OpenAI released its O1 series models, test-time compute (TCT) and large-scale models combined with search trees have become hot research topics in academia and industry. By combining reinforcement learning and supervised learning during training, large models can learn multi-step reasoning capabilities—that is, performing long thought chains of reasoning before drawing conclusions—significantly improving the model's reasoning ability by providing more computation. However, this "long thought chain / slow thinking" reasoning greatly increases the computational cost required for reasoning. Moreover, the model is prone to getting stuck in long thought processes, unable to actively complete the thinking and draw conclusions within the constraints of the model's context. Summary of the Invention

[0004] The embodiments of this application provide an AI model reasoning method and apparatus, which can effectively alleviate the problem of redundant reflection in AI models, prevent AI models from getting stuck in long thinking, and effectively reduce the computing power consumption of AI model reasoning.

[0005] In a first aspect, this application provides a reasoning method for an AI model. The method includes obtaining a sequence generated by the AI ​​model, which has the ability to reason through thought chains. The sequence is generated by the AI ​​model based on an input query request. Then, based on the sequence, it is identified whether there is redundant reflection in the thought chain generated by the AI ​​model. When redundant reflection is identified in the AI ​​model, a termination operation is performed on the thought chain to stop the AI ​​model from thinking and output the response content corresponding to the query request.

[0006] This application embodiment can actively identify whether there is redundant reflection in the thought chain generated by the AI ​​model. When redundant reflection is detected in the AI ​​model, a termination operation is performed on the thought chain, which effectively compresses the length of the thought chain, effectively alleviates the problem of redundant reflection in the AI ​​model, avoids the AI ​​model from getting stuck in long thinking, and effectively reduces the computing power consumption of AI model inference.

[0007] In one possible implementation, a specific approach to identifying redundant reflections in the thought chain generated by AI model inference based on a sequence is to divide the sequence into multiple segments; and then identify redundant reflections in the thought chain generated by AI model inference based on these multiple segments.

[0008] In one possible implementation, multiple segments include multiple reflection segments; a specific implementation of dividing the sequence into multiple segments is: dividing the sequence based on a first reflection flag to obtain multiple reflection segments.

[0009] It's important to clarify that the meaning of "reflection marker" is that once the AI ​​model encounters this type of marker during inference (e.g., certain marker words in a large language model), it indicates that the AI ​​model is about to enter the next round of reflection. Therefore, reflection markers can be used to segment the sequence generated by the AI ​​model, obtaining multiple reflection fragments to facilitate redundant reflection identification in subsequent steps. It can be understood that reflection markers are different modalities in different AI models. For example, when the AI ​​model is a large language model, the reflection markers are reflection marker words; when the AI ​​model is a multimodal large model, the reflection markers are in the form of multimodal tokens.

[0010] In another possible implementation, a specific implementation of segmenting the sequence based on the reflection flag to obtain multiple reflection segments is as follows: the sequence is segmented based on the first reflection flag to obtain multiple thinking segments; the thinking segments containing conclusive content among the multiple thinking segments are identified as reflection segments.

[0011] In this possible implementation, after dividing the sequence into multiple thought segments using reflection markers, it is then checked whether each thought segment contains conclusive content. Thought segments with conclusive content are identified as reflection segments, ensuring that the division of reflection segments is more accurate.

[0012] In another possible implementation, a specific approach to identifying redundant reflections in the thought chain generated by AI model reasoning based on multiple fragments is as follows: perform a repetition calculation on multiple reflection fragments to obtain the repetition calculation result; if the repetition calculation result meets a preset condition, it is determined that the thought chain generated by AI model reasoning contains redundant reflections.

[0013] By using repetition calculation, we can better distinguish whether the reflection results of the preceding and following reflection segments are consistent, accurately identify whether there is redundant thinking in the thought chain, and ensure the inference accuracy of the AI ​​model.

[0014] In another possible implementation, the repetition calculation result includes the repetition score of adjacent reflection segments, or the repetition score of N consecutive reflection segments, where N is an integer greater than 2; the preset conditions include one or more of the following: the repetition score is greater than a first threshold, the repetition score is greater than a second threshold for M consecutive rounds, and the repetition score increases for H consecutive rounds, where M and H are both positive integers.

[0015] In another possible implementation, the termination operation includes a soft termination operation; a specific implementation of performing the termination operation on the thought chain is to reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag.

[0016] By adjusting the sampling probabilities of the reflection flag and the reflection end flag, the AI ​​model can be guided to end its thinking process more naturally, thus ensuring the quality of the response output by the AI ​​model.

[0017] In another possible implementation, the decrease in the sampling probability of the second reflection flag is positively correlated with the sequence length; the increase in the sampling probability of the reflection end flag is also positively correlated with the sequence length. That is, as the length of the sequence generated by the AI ​​model's inference increases, the AI ​​model is more likely to get stuck in prolonged thinking. Therefore, in the soft termination operation, the greater the decrease in the sampling probability of the second reflection flag and the greater the increase in the sampling probability of the reflection end flag, the more the model is guided to end its thinking as quickly as possible.

[0018] In another possible implementation, the termination operation includes a hard termination operation; a specific implementation of performing a termination operation on the thought chain is to generate pre-set reflection termination content.

[0019] In other words, a hard termination operation can forcibly stop the model's thinking and generate a pre-set reflection termination content. For example, the sampling probability of the token in the reflection termination content can be set to 100%, so that the model generates the pre-set reflection termination content 100% of the time, forcing the AI ​​model to stop thinking, freeing the AI ​​model from long thinking, and outputting the response content.

[0020] In another possible implementation, the termination operation includes a soft termination operation and a hard termination operation; a specific implementation of performing the termination operation on the thought chain is as follows: a soft termination operation is performed on the thought chain to reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag; if it is determined that the conditions for performing the hard termination operation are met, then the hard termination operation is performed on the thought chain to make the AI ​​model generate the preset reflection termination content.

[0021] When redundant reflection is detected in the thought chain of the AI ​​model, this application prioritizes a soft termination operation to guide the AI ​​model to end its thinking naturally. If the soft termination operation cannot effectively terminate the AI ​​model's thinking, a hard termination operation is initiated to force the AI ​​model to exit thinking, ensuring that the AI ​​model can end its thinking normally during redundant reflection and return the response content, thereby reducing user waiting time and improving user experience.

[0022] In another possible implementation, the conditions for performing a hard termination operation include one or more of the following: the AI ​​model does not terminate thinking within a preset inference step, or the AI ​​model generates a reflection flag.

[0023] By setting reasonable judgment conditions, the failure of soft termination operation can be accurately determined, and hard termination operation can be initiated in a timely manner to forcibly stop the AI ​​model's thinking, force the AI ​​model to output response content, and reduce response latency.

[0024] Optionally, the AI ​​model mentioned above can be a large language model (LLM) or a multimodal large model (MLLM), or other neural network models that may produce multi-round reasoning results.

[0025] Secondly, this application also provides an AI model reasoning device, which includes an acquisition module, a redundancy reflection identification module, and a termination module. The acquisition module is used to acquire the sequence generated by the AI ​​model, which has the ability to reason through thought chains. The sequence is generated by the AI ​​model based on the input query request. The redundancy reflection identification module is used to identify redundancy reflection in the thought chain generated by the AI ​​model based on the sequence. The termination module is used to perform a termination operation on the thought chain so that the AI ​​model stops thinking and outputs the response content corresponding to the query request.

[0026] In one possible implementation, the redundancy reflection identification module is specifically used to: divide the sequence into multiple segments; and based on the multiple segments, identify redundant reflections in the thought chain generated by the AI ​​model's reasoning.

[0027] In one possible implementation, multiple segments include multiple reflection segments; a specific implementation of dividing the sequence into multiple segments is: dividing the sequence based on a first reflection flag to obtain multiple reflection segments.

[0028] In another possible implementation, a specific implementation of segmenting the sequence based on the reflection flag to obtain multiple reflection segments is as follows: the sequence is segmented based on the first reflection flag to obtain multiple thinking segments; the thinking segments containing conclusive content among the multiple thinking segments are identified as reflection segments.

[0029] In another possible implementation, a specific approach to identifying redundant reflections in the thought chain generated by AI model reasoning based on multiple fragments is as follows: perform a repetition calculation on multiple reflection fragments to obtain the repetition calculation result; if the repetition calculation result meets a preset condition, it is determined that the thought chain generated by AI model reasoning contains redundant reflections.

[0030] In another possible implementation, the repetition calculation result includes the repetition score of adjacent reflection segments, or the repetition score of N consecutive reflection segments, where N is an integer greater than 2; the preset conditions include one or more of the following: the repetition score is greater than a first threshold, the repetition score is greater than a second threshold for M consecutive rounds, and the repetition score increases for H consecutive rounds, where M and H are both positive integers.

[0031] In another possible implementation, the termination operation includes a soft termination operation; the termination module is specifically used to: reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag.

[0032] In another possible implementation, the decrease in the sampling probability of the second reflection flag is positively correlated with the length of the sequence; the increase in the sampling probability of the end-of-reflection flag is positively correlated with the length of the sequence.

[0033] In another possible implementation, the termination operation includes a hard termination operation; the termination module is specifically used to generate pre-set reflection termination content.

[0034] In another possible implementation, the termination operation includes a soft termination operation and a hard termination operation; the termination module is specifically used to: perform a soft termination operation on the thought chain to reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag; if the conditions for performing a hard termination operation are met, then perform a hard termination operation on the thought chain to make the AI ​​model generate the preset reflection end content.

[0035] In another possible implementation, the conditions for performing a hard termination operation include one or more of the following: the AI ​​model does not terminate thinking within a preset inference step, or the AI ​​model generates a reflection flag.

[0036] Optionally, the above AI model can be a large language model or a multimodal large model.

[0037] Thirdly, embodiments of this application provide a computing device, including a memory and a processor, wherein the memory stores instructions that, when executed by the processor, cause the method described in the first aspect or any possible implementation of the first aspect to be implemented.

[0038] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the method described in the first aspect or any possible implementation thereof to be implemented.

[0039] Fifthly, embodiments of this application also provide a computer program or computer program product, which includes instructions that, when executed, cause a computer to perform the method described in the first aspect or any possible implementation thereof.

[0040] Sixthly, embodiments of this application also provide a chip including at least one processor and a communication interface, the processor being used to perform the method described in the first aspect or any possible implementation of the first aspect.

[0041] It is understandable that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here for the sake of brevity. Attached Figure Description

[0042] Figure 1 A schematic diagram of a system architecture is shown;

[0043] Figure 2 This diagram illustrates the main implementation steps of the AI ​​model inference method provided in this application when applied to a large language model;

[0044] Figure 3 This illustration shows a schematic diagram of the implementation architecture when the inference method of the AI ​​model provided in this application is applied to a large language model;

[0045] Figure 4 A flowchart illustrating an AI model inference method provided in an embodiment of this application;

[0046] Figure 5 This is a schematic diagram of the structure of an AI model inference device provided in an embodiment of this application;

[0047] Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0048] The term "and / or" used in this article describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0049] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same properties in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such processes, methods, systems, products, or apparatus.

[0050] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0051] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0052] The industry is actively exploring training methods and deep applications of large O1 / R1-like models. In China, Alibaba has launched the QwQ-32B model, and DeepSeek has launched the DeepSeek R1 model. R1-like models have become the standard for large-scale model deployment, serving as a benchmark for "deep thinking." However, as mentioned in the background, although these models perform excellently in inference, they suffer from problems such as repetitive thinking and redundant reflection, leading to high computational overhead and long response latency. Therefore, how to reasonably compress the length of the thought chain while ensuring the accuracy of large models remains a challenge.

[0053] There are several technical solutions to address the problem of redundancy in large models. For example, one solution involves fusing the weights of slow-thinking and fast-thinking models to balance inference performance and inference length; another solution uses reinforcement learning (RL) to incorporate length penalties into the training loss, thereby reducing the model's inference length.

[0054] However, this approach has drawbacks such as the need for retraining, significant accuracy loss, and a complex model weight fusion scheme that makes it difficult to balance accuracy and length.

[0055] The second related technical solution is to add a character token count requirement or other requirements to reduce the thought length in the prompt, thereby limiting the output length of the model.

[0056] This scheme suffers from several drawbacks, including impacting the thought process, significantly affecting accuracy, having low generalization ability, and difficulty in accurately specifying the required number of tokens, especially for difficult problems that require extensive thinking to solve.

[0057] The third related technical solution is to compress the thought chain into a minimum number of special characters for the reasoning of the next thought segment, thereby reducing the sequence length of the reasoning.

[0058] While this approach can reduce the computational cost of model inference, it cannot solve the problem of redundant thinking.

[0059] The fourth related technical solution is to summarize the thinking process periodically, use the summarized content for subsequent reasoning, and remove the existing thinking content.

[0060] However, this approach suffers from the following drawbacks: it requires retraining, suffers significant accuracy loss, introduces new latency for model inference, and results in low end-to-end performance gains.

[0061] The fifth related technical solution is to first run BoN (Best of N), and then remove N-1 generated items based on the confidence of the large model.

[0062] This approach cannot solve the problem of redundant model thinking and can only optimize inference performance in BoN scenarios.

[0063] The sixth related technical solution involves directly deploying the thought chain into the implicit embedding, without reflecting the explicit thought process, thus reducing token usage.

[0064] This solution cannot solve the problem of frequent reflection, the lack of a thought process results in a poor user experience, the absence of a visible token significantly impacts accuracy, requires retraining the model, is costly, and cannot meet the acceleration requirements of hundreds of models and thousands of states.

[0065] In view of this, this application proposes an AI model inference method that actively identifies whether there is redundant reflection in the thought chain generated by the AI ​​model. When redundant reflection is identified in the AI ​​model, the length of the thought chain is compressed. Under the condition that the reduction in accuracy is controllable, the amount of inference tokens generated is effectively reduced, the computing power consumption of the computing inference device is reduced, and there is no need to retrain the AI ​​model, thus saving training costs.

[0066] To facilitate understanding of the solutions in the embodiments of this application, the technical terms involved in the solutions provided in this application will be briefly introduced below.

[0067] Large language models are deep learning models trained on massive amounts of text data. By learning the structure, rules, and semantics of language through training, they can generate text with a natural language style or answer questions in natural language.

[0068] A character token is the smallest unit in a sequence generated by a large language model.

[0069] Instruction-type models are large models that have the ability to follow instructions.

[0070] Sampling is a process in which a large language model selects a final output token from all possible tokens based on the model's confidence / probability (i.e., the probability distribution output by the model, which indicates the probability of each token in the vocabulary being sampled).

[0071] The following describes in detail, with reference to the accompanying drawings and embodiments, the specific implementation of the AI ​​model inference method and apparatus provided in the embodiments of this application.

[0072] Figure 1 A schematic diagram of a system architecture is shown. The system includes a terminal 110, a server 130, and a network 120 that communicatively connects the terminal 110 and the server 130. The server 130 may include one or more servers (…). Figure 1 (This example uses only one server.) Server 130 is equipped with an AI model that has the ability to form thought chains. Server 130 can provide AI services to one or more terminals 110, such as more complex AI reasoning tasks including intelligent question answering and long text generation.

[0073] In some embodiments, server 130 may also provide other services or software applications, including both non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to end users under a Software as a Service (SaaS) model.

[0074] exist Figure 1 In the system shown, server 130 may include one or more components that implement the functions performed by server 130. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. A user of operating terminal 110 may sequentially interact with server 130 and utilize the services provided by these components using one or more client applications. It should be understood that... Figure 1 This is merely one example of a system for implementing the various methods described in the embodiments of this application; other different configurations are possible, and the embodiments of this application do not limit this.

[0075] The terminal 110 can have AI-related applications installed (such as intelligent question-and-answer software, smartphone assistants, such as Huawei's Xiaoyi Smart Assistant, which can provide Xiaoyi knowledge question-and-answer services), or open AI-related web pages. The aforementioned applications and web pages can provide an interactive interface. Optionally, users can input query text through this interactive interface, which can also be called user questions. For example, the interactive interface provides a text input box, through which users can input the question they want to query, such as "Please compare the Wenjie M7, Li Auto L7 and NIO ES7 and provide purchase advice." Users can also input the question they want to query through voice input. For example, the interactive interface provides a voice input button, and users can press and hold the voice input button and say the question they want to query, such as "Please compare the Wenjie M7, Li Auto L7 and NIO ES7 and provide purchase advice." The voice-to-text conversion module then converts the input voice into text. Terminal 110 can receive query text input by the user and send the query text to server 130. Server 130 can perform reasoning based on the received query text using the reasoning method of the AI ​​model provided in this application embodiment, avoiding redundant reflection of the AI ​​model and obtaining response content that takes into account both latency and quality requirements.

[0076] In another example, users can also input their queries via voice wake-up. For example, a user can call out "Xiaoyi, Xiaoyi" to wake up the Xiaoyi AI assistant and then input their query via voice, such as "Please compare the Wenjie M7, Li Auto L7, and NIO ES7 and provide purchase advice." The terminal 110 sends the received voice input to the server 130, which converts the voice into text and then uses the reasoning method of the AI ​​model provided in this application embodiment to perform reasoning, avoiding redundant reflection of the AI ​​model and obtaining a response that balances latency and quality requirements.

[0077] Terminal 110 may include various types of computer devices, such as portable handheld devices (e.g., smartphones), general-purpose computers (e.g., personal computers or laptops), workstation computers, wearable devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (e.g., Google Chrome OS), Harmony OS; or various mobile operating systems, such as Microsoft Windows Mobile OS, Windowsphone, Android, iPhone OS, HarmonyOS NEXT, etc. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays and smartwatches, etc.

[0078] Network 120 can be any type of network well known to those skilled in the art, and can use any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, 3G, 4G, 5G, etc., to support data communication. For example, one or more networks 120 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, the public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, Wi-Fi), and / or any combination of these and / or other networks.

[0079] Server 130 may include one or more general-purpose computers, special-purpose server computers (e.g., personal computer servers, UNIX servers, terminal servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement or combination. Server 330 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization, such as one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers. In various embodiments, server 130 may run one or more services or software applications that provide the functionality described below.

[0080] In some implementations, server 130 can be a server for a distributed system or a server integrated with blockchain. Server 130 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and virtual private servers (VPS) services, such as high management difficulty and weak business scalability.

[0081] It should be noted that in some other examples, the terminal 110 can also execute the inference method of the AI ​​model provided in the embodiments of this application on its own to obtain response content that takes into account both latency and quality requirements, without the cooperation of the server 130. Alternatively, the server 130 can also execute the inference method of the AI ​​model provided in the embodiments of this application on its own to optimize and obtain response content that takes into account both latency and quality requirements, without the cooperation of the terminal 110. This application does not make specific limitations on this.

[0082] It should be noted that the AI ​​model reasoning method and apparatus provided in this application embodiment can be applied to any AI model with thought chain capabilities, or any AI model capable of generating multi-round reasoning and thinking results. For example, large language models, multimodal large models, and other large models mostly possess multi-step reasoning capabilities, performing long thought chain reasoning before giving a conclusion. This can be called a "slow thinking" model. Although these models perform well in reasoning and can solve various complex problems, they are prone to getting stuck in long thinking and cannot actively complete thinking and give a conclusion within the constraints of the model context, resulting in prolonged response time and a poor user experience. Using the AI ​​model reasoning method provided in this application embodiment embodiment, it is possible to actively identify whether there is redundant reflection in the thought chain generated by the AI ​​model. When redundant reflection is identified in the AI ​​model, the thought chain is compressed, and the length of the compressed thought chain is controlled, effectively alleviating the problem of redundant reflection in the AI ​​model, preventing the AI ​​model from getting stuck in long thinking, and effectively reducing the computational cost of AI model reasoning.

[0083] The following section uses a large language model with slow thinking capabilities as an example to detail the specific implementation of the reasoning method and apparatus of the AI ​​model provided in this application.

[0084] Figure 2 This diagram illustrates the main implementation steps of the AI ​​model inference method provided in this application when applied to a large language model. For example... Figure 2 As shown, the reasoning method of the AI ​​model provided in this application mainly includes a reflection recognition step, a redundant reflection recognition step, a soft termination step of the thought chain, and a hard termination step of the thought chain.

[0085] The main function of the reflection identification step is to divide the text sequence generated by the large language model into multiple reflection segments, so that the subsequent redundant reflection identification step can analyze and identify whether there are redundant reflections in the large language model.

[0086] For example, in a large language model inference task similar to O1 / R1, the user inputs a question into the large language model, which then performs inference based on the input question to generate a text sequence. The reflection recognition step in this embodiment captures the reflective thought chain generated during inference in real time and identifies whether an answer has been found. If the reflective thought chain has been solved, it is called an independent reflective thought chain fragment (which can be simply referred to as a reflective fragment). For example... Figure 2 Conclusion 1, Reflection Fragment 1, Reflection Fragment 2, Reflection Fragment 3, and Reflection Fragment 4, that is, the text sequence generated by the large language model is divided into multiple reflection fragments through the reflection recognition step.

[0087] In this embodiment of the application, the reflection recognition step can use various methods to segment the text sequence generated by the large language model inference to obtain multiple reflection segments.

[0088] For example, the generated text sequence can be segmented into multiple reflection segments using rule matching. For instance, the generated text sequence can be segmented based on reflection markers to obtain multiple reflection segments. For example, a reflection lexicon can be pre-configured, which includes multiple reflection markers (these multiple reflection markers can be called the first reflection markers). Regular expression matching is used to match the generated text sequence by the large language model. When a reflection marker is matched for the first time in the generated text sequence, it indicates that the large language model has begun reflection. At this point, the text sequence can be segmented from the position before the reflection marker appears, resulting in conclusion 1 (also called reflection segment 0). Matching continues. When a reflection marker is matched again in the generated text sequence, it indicates that the model's current round of reflection has ended and a new round of reflection has begun. At this point, the text sequence can be segmented from the position before the newly matched reflection marker appears, resulting in reflection segment 1. Matching continues to obtain subsequent reflection segments 2, 3, ..., N. In other words, each time a reflection marker is matched, a reflection segment can be segmented. For example, if the first reflection marker is matched from the generated text sequence, the text segment before the first reflection marker (excluding the first reflection marker) forms reflection segment 0. If the second reflection marker is matched, the text segment between the second reflection marker (excluding the second reflection marker) and the first reflection marker (including the first reflection marker) forms reflection segment 1. If the third reflection marker is matched, the text segment between the third reflection marker and the second reflection marker forms reflection segment 2, and so on. The text segment between the Nth reflection marker and the (N-1)th reflection marker forms reflection segment N-1, and the text segment after the Nth reflection marker forms reflection segment N.

[0089] In the above reflection identification scheme, it is not necessary to verify whether the segmented fragments contain conclusive statements; any fragment can be defined as a reflection fragment. For example, some fragments may simply make some inferences without reaching a conclusion, or they may merely describe some irrelevant information; such fragments can also be identified as reflection fragments.

[0090] It should be noted that the first reflection marker, the second reflection marker, the third reflection marker, ..., the (N-1)th reflection marker and the Nth reflection marker mentioned above are only to distinguish reflection markers matched in different instances. These reflection markers can be the same or different from each other. For example, if the first and second reflection markers matched in the generated text sequence are both "Wait," then the first and second reflection markers are the same. However, if the third reflection marker is "But" and the fourth reflection marker is "However," then the third and fourth reflection markers are different.

[0091] For example, a neural network model can also be used to segment the generated text sequence into multiple reflection segments. For instance, a neural network model specifically designed for reflection segmentation (which can be called a reflection segmentation model, establishing a mapping between the text sequence and reflection segments, mapping the input text sequence to several reflection segments) can be trained. The generated text sequence can be used as input to the reflection segmentation model, and the output can be the segmented reflection segments. The output of the reflection segmentation model might be, for example,... Figure 2 As shown, there are Conclusion 1, Reflection Fragment 1, Reflection Fragment 2, Reflection Fragment 3, and Reflection Fragment 4.

[0092] In another example, to ensure the accuracy of the reflection segmentation, the segmented segments are further verified after segmentation. Only those segments that pass the verification are confirmed as reflection segments. For example, the segmented segments are checked for concluding statements; if they do, the segment is considered a complete reflection segment.

[0093] In other words, in this reflection identification scheme, only reflection fragments containing concluding statements will be identified as reflection fragments, while fragments without concluding statements will be excluded and will not participate in subsequent redundant reflection identification.

[0094] There are several methods to verify reflective passages. For example, they can be verified by detecting keywords. For instance, by detecting specific keywords such as "the answer is," if the segmented passage contains "the answer is," then the passage is considered to contain a conclusive statement and is thus identified as a reflective passage.

[0095] For example, reflection segments can also be verified by detecting their length. For instance, the length of a segment can be checked to see if it exceeds a preset threshold. If it does, the segment is considered long enough and likely contains a concluding statement, thus identifying it as a reflection segment.

[0096] In this embodiment, the generated text sequence can be subjected to reflective recognition in real time. That is, each time the large language model generates a token, it identifies whether the token is a reflective marker. Alternatively, a reflective recognition can be performed on the newly generated tokens after a preset number of tokens are generated. For example, if the number of tokens triggering reflective recognition is set to 100, the large language model performs reflective recognition every 100 tokens generated to identify whether the newly generated token sequence contains reflective segments.

[0097] The redundant reflection identification step is used to identify whether redundant reflection exists in the thought chain generated by the reasoning of the large language model based on the reflection fragments obtained from the segmentation.

[0098] For example, in the redundant reflection identification step, a scorer can be used to score the repetition of adjacent reflection segments. The higher the score, the higher the repetition of adjacent reflection segments. Based on the scoring results, it can be determined whether there is redundant reflection in the thought chain generated by the current reasoning of the large language model.

[0099] Scoring mechanisms can be implemented in several ways. For example, they can be rule-based, such as using word frequency detection to score adjacent reflection segments. This involves detecting whether adjacent reflection segments share the same token and the number of identical tokens to measure the repetition degree and use this as the basis for scoring. Alternatively, they can be scored by calculating similarity, such as by calculating the cosine similarity of the embeddings of adjacent reflection segments to determine their repetition.

[0100] For example, a repetition scoring model can be used to score adjacent reflective segments, such as an outcome reward model (ORM) or a process reward model (PRM). Adjacent reflective segments are input into the scoring model, and the scoring model outputs a repetition score.

[0101] This application does not limit the specific implementation of the scoring device; an appropriate method can be selected based on the actual situation.

[0102] In some other examples, a scorer can also be used to score several consecutive reflection segments, such as scoring three consecutive reflection segments, to obtain a repetition score.

[0103] There are several possible logics for determining whether redundant reflection exists in the thought chain generated by the current inference of the large language model based on the scoring results. For example, if the repetition score is greater than the first threshold T1, then the thought chain generated by the current inference of the large language model is considered to have redundant reflection; otherwise, it is not. Another example is if the repetition score exceeds the second threshold T2 for M consecutive rounds, then the thought chain generated by the current inference of the large language model is considered to have redundant reflection; otherwise, it is not. Yet another example is if the repetition score increases for H consecutive rounds, then the thought chain generated by the current inference of the large language model is considered to have redundant reflection; otherwise, it is not. Yet another example is if the repetition score is greater than the first threshold T1 and increases for H consecutive rounds, then the thought chain generated by the current inference of the large language model is considered to have redundant reflection; otherwise, it is not. Finally, another example is if the repetition score is greater than the first threshold T1 and exceeds the second threshold T2 for M consecutive rounds, then the thought chain generated by the current inference of the large language model is considered to have redundant reflection; otherwise, it is not. For example, if the repetition score exceeds the first threshold T1, and exceeds the second threshold T2 for M consecutive rounds, and the repetition score increases for H consecutive rounds, then it is determined that the thought chain generated by the current reasoning of the large language model has redundant reflection; otherwise, there is no redundant reflection. This ensures the accuracy of redundant reflection identification.

[0104] The soft termination step for thought chains is used to softly terminate thought chains when redundant reflections are detected in the current thought chain generated by the large language model. For example, the sampling probability of reflection marker words is reduced, and the sampling probability of reflection end marker words is increased to guide the large language model to naturally end its thinking and output the response text for the query request based on the content of the current thinking.

[0105] It is easy to understand that the thought chain generated by the model's current inference is, for example, Figure 2 The text consists of Conclusion 1, Reflection Fragment 1, Reflection Fragment 2, Reflection Fragment 3, and Reflection Fragment 4.

[0106] It should be explained that the redundant reflection mentioned in the embodiments of this application includes overthinking and underthinking. Overthinking means that the model has thought of the correct answer, but is still reflecting. For example, reflection segment 1 includes the correct answer, but the model still reasoning and generating reflection segment 2, reflection segment 3, etc. Subsequent thinking segments are all overthinking. Underthinking means that the model has not thought of the correct answer, but is still reflecting. That is to say, the model generates multiple reflection segments, each of which does not include the correct answer (but may include the wrong answer). Multiple rounds of reflection are all ineffective.

[0107] It should be noted that during soft termination, the sampling probability of all reflection markers in the reflection vocabulary can be reduced, or only the sampling probability of some reflection markers can be reduced (to distinguish them from the reflection markers mentioned above, these some reflection markers can be called second reflection markers). In the soft termination operation, the reflection markers whose sampling probability needs to be reduced can be determined experimentally. The reflection markers with better quality of the final output response text and better termination effect should be selected for sampling probability reduction.

[0108] The decrease in the sampling probability of reflection markers is positively correlated with the length of the currently generated text sequence; the increase in the sampling probability of reflection end markers is also positively correlated with the length of the currently generated text sequence. For example, during soft termination, the sampling probabilities of reflection markers such as Wait, But, and However are adjusted to their original values ​​f(len), where len is the length of the currently generated token sequence in the large language model, and f(len) is a function that decreases with the length of the generated token sequence. The sampling probabilities of reflection end markers such as So and Therefore are adjusted to their original values ​​g(len), where len is the length of the currently generated token sequence in the large language model, and g(len) is a function that increases with the length of the generated token sequence.

[0109] A hard termination step is initiated when soft termination fails, forcing the large language model to output a pre-defined reflective termination sentence, such as "I think I have thought for a long time. I should be able to answer the question now." The conditions for determining whether soft termination has failed, or the conditions for initiating hard termination, can be that the large language model has not terminated thinking within a pre-defined inference step (e.g., G-step) or that the large language model has generated one or more of the reflective marker words.

[0110] For example, if after a soft termination step, the large language model continues reasoning for G more steps, generating G tokens, but still hasn't generated a thought-ending word, then a hard termination step is executed. Alternatively, if after a soft termination step, a reflection marker is detected in the token sequence generated by the large language model, indicating that the large language model has started reflecting again, and the soft termination failed to effectively stop the reflection, then a hard termination step is executed. Or, if after a soft termination step, the large language model continues reasoning for G more steps, generating G tokens, and a reflection marker is detected among these G tokens, then a hard termination step is executed.

[0111] In this embodiment of the application, when redundant reflection is detected in the thought chain of the large language model, a soft termination step is first executed to guide the large language model to end its thinking naturally. If the soft termination step cannot effectively terminate the thinking of the large language model, a hard termination step is initiated to force the large language model to exit thinking, ensuring that the large language model can end its thinking normally during redundant reflection and return the response text, thereby reducing user waiting time and improving user experience.

[0112] In some other examples, the reasoning method of the AI ​​model provided in this application embodiment may also include only a soft termination step of the thought chain, that is, when it is recognized that there is redundant reflection in the thought chain generated by the current reasoning of the large language model, a soft termination operation is directly performed on the thought chain to guide the large language model to stop thinking naturally; the reasoning method of the AI ​​model provided in this application embodiment may also include only a hard termination step of the thought chain, that is, when it is recognized that there is redundant reflection in the thought chain generated by the current reasoning of the large language model, a hard termination operation is directly performed on the thought chain to forcibly stop the thinking of the large language model and output a pre-set reflection termination sentence.

[0113] As described above, the AI ​​model reasoning method provided in this application embodiment can actively identify whether there is redundant reflection in the thought chain generated by the AI ​​model. When redundant reflection is detected, the thought chain is terminated, effectively compressing the length of the thought chain, alleviating the problem of redundant reflection in the AI ​​model, preventing the AI ​​model from getting stuck in long thinking, effectively reducing the computing power overhead of AI model reasoning, and eliminating the need to retrain the AI ​​model, thus saving training costs. Furthermore, this application embodiment terminates the AI ​​model's thinking only when redundant thinking is detected, so it does not affect the normal thinking of the AI ​​model, and the accuracy of the AI ​​model's output will not deteriorate or will not deteriorate significantly.

[0114] In another example, the reasoning method of the AI ​​model mentioned in this application embodiment may also exclude the reflection identification step. The redundancy reflection identification step directly identifies whether there is redundant reflection in the thought chain generated by the large language model's reasoning based on the text sequence generated by the large language model. For example, a neural network classification model with redundancy reflection identification capability is trained, the text sequence generated by the large language model is input into the neural network classification model, and the output is a classification result indicating whether the thought chain generated by the large language model's reasoning has redundant reflection or not.

[0115] Figure 3 This diagram illustrates the implementation architecture of the AI ​​model inference method provided in this application when applied to a large language model. For example... Figure 3As shown, the architecture includes a reflection identification module, a redundancy reflection identification module, a thought chain soft termination module, and a thought chain hard termination module. The reflection identification module performs the reflection identification steps described above, segmenting the text sequence generated by the large language model's inference to obtain multiple reflection fragments. The redundancy reflection identification module performs the redundancy reflection identification steps described above, identifying whether redundant reflections exist in the thought chain generated by the large language model's inference based on the segmented reflection fragments. The thought chain soft termination module performs the thought chain soft termination steps described above, performing soft termination on the thought chain when redundant reflections are detected in the currently generated thought chain by the large language model. The thought chain hard termination module performs the thought chain hard termination steps described above, activating when soft termination fails, forcing the large language model to output a pre-defined reflection termination sentence.

[0116] like Figure 3 As shown, the model with the ability to reason using thought chains outputs the following text sequence based on the user's input:

[0117] <think>Okay, so the problem is:...;

[0118] Wait,...Factors of 42:1,2,3,6,7,14,21,42.\n\nComparing these twolists, the common factors are 1,2,7,and 14.

[0119] Wait,...the common factors are 1,2,7,14.That`s four numbers.

[0120] But,...So, all four numbers are indeed common factors.

[0121] Therefore, there are four possible numbers they could be thinking of.

[0122] The reflection recognition module segments the text sequence generated by the model's current inference into reflection segments, resulting in the following multiple reflection segments:

[0123] Reflection Fragment N-2: Wait,...Factors of 42:1,2,3,6,7,14,21,42.\n\nComparingthese two lists, the common factors are 1,2,7,and 14.

[0124] Reflection passage N-2: Wait,...the common factors are 1, 2, 7, 14. That's four numbers.

[0125] Reflection fragment N: But,...So, all four numbers are indeed common factors.Therefore, there are four possible numbers they could be thinking of.

[0126] At this point, N equals 3, meaning that the reflection recognition module divides the text sequence generated by the current inference of the model into three reflection segments: reflection segment 1, reflection segment 2, and reflection segment 3.

[0127] The redundant reflection identification module identifies whether redundant reflections exist in the thought chains generated by the large language model's reasoning, based on the segmented reflection fragments. For example... Figure 3 The redundant reflection identification module uses a repetition score scorer to score adjacent reflection segments, obtaining a repetition score for each adjacent reflection segment. For example, inputting reflection segment 1 and reflection segment 2 into the scorer outputs a score of 1; inputting reflection segment 2 and reflection segment 3 into the scorer outputs a score of 2; and so on, inputting reflection segment N-1 and reflection segment N into the scorer outputs a score of N-1, meaning that N reflection segments will generate N-1 repetition scores. Then, it is determined whether the scores of all M rounds among the N-1 repetition scores are greater than a threshold. If so, a soft termination operation is performed on the current thought chain through the thought chain soft termination module.

[0128] When the thought chain soft termination module detects redundant reflections in the thought chain currently generated by the large language model, it performs a soft termination operation on the thought chain. For example, it modifies the sampling probability of the model's inference output, that is, it lowers the sampling probability of reflection marker words and increases the sampling probability of reflection end marker words. After performing the soft termination operation, it determines whether the model outputs a reflection end word within G steps.< / think> If yes, the model ends its thinking process; otherwise, the hard termination module of the thought chain is activated.

[0129] The hard termination module of the thought chain is activated when soft termination fails, forcing the large language model to output a pre-set reflection termination sentence. For example, it modifies the sampling probability distribution of the model's inference output, sets the sampling probability of each token in the pre-set reflection termination sentence to 100%, and forces the model to output the pre-set reflection termination sentence.

[0130] It should be noted that, Figure 3 The implementation architecture shown is merely one possible implementation architecture for the AI ​​model inference method provided in this application embodiment, and does not constitute a limitation on the embodiments of this application. For example, this implementation architecture may not include a reflection recognition module, and the generated sequence may be processed directly through a redundant reflection recognition module to identify whether there is redundant reflection in the thought chain generated by the model inference. Another example is that this implementation architecture may only include a thought chain soft termination module or only include a thought chain hard termination module.

[0131] Figure 4This is a flowchart illustrating an AI model inference method provided in an embodiment of this application. This method can be executed by any computing-capable device, equipment, platform, or cluster of devices. This application does not specifically limit the specific computing device executing this method; a suitable computing device can be selected as needed. For example, it can be implemented by a server, meaning the AI ​​model inference method can be implemented by a server, providing AI model inference acceleration services. Alternatively, it can be implemented on a cloud device, providing AI model inference acceleration services to cloud users in the form of cloud services. For ease of description, the form of the executing entity will not be distinguished in the following text; all will be described as executing devices. Figure 4 As shown, the inference method of the AI ​​model provided in this application embodiment includes at least steps S401 to S404.

[0132] In step S401, the sequence generated by the AI ​​model is obtained.

[0133] AI models possess reasoning capabilities and the ability to solve complex problems, but they may get stuck in prolonged thought processes, unable to proactively complete their reasoning and arrive at a conclusion within the constraints of the model's context. For example, a certain slow-thinking large language model, when faced with a user-input question, got stuck in prolonged thought, generating 300,000 tokens before still not finishing its thought process.

[0134] Optionally, the AI ​​model can be a multi-modal AI model. The generated sequences are of different modalities for different modal AI models. For example, for a large language model, the generated sequences are text sequences; for a multi-modal large model, the generated sequences can be text sequences, speech frame sequences, video frame sequences, etc.

[0135] In one example, the sequence generated by the AI ​​model inference can be obtained in real time, or the sequence generated by the AI ​​model inference can be obtained periodically, for example, once every 100 tokens are generated.

[0136] In step S402, based on the sequence, it is identified whether there is redundant reflection in the thought chain generated by the AI ​​model reasoning.

[0137] In one example, the acquired sequence can be directly processed to identify whether there is redundant reflection in the thought chain generated by the AI ​​model's inference. For instance, rule matching can be used to identify whether there is redundant reflection in the thought chain generated by the AI ​​model's inference. For example, regular expression matching can be used to match how many times the summary content (e.g., "the answer is...") appears in the text sequence generated by the AI ​​model. If the number of times the summary content appears exceeds a set threshold, it is determined that there is redundant reflection in the thought chain generated by the AI ​​model's inference; otherwise, there is no redundant reflection.

[0138] Alternatively, a neural network classification model with the ability to recognize redundant reflection can be trained. The text sequence generated by the large language model is input into the neural network classification model, and the output is the classification result of whether the thought chain generated by the large language model has redundant reflection or not.

[0139] Optionally, the classification model can be an MLP, a classification model based on a transformer architecture, etc.

[0140] In another example, the sequence generated by the AI ​​model can be segmented, and then based on multiple segments, redundant reflections can be identified in the thought chain generated by the AI ​​model's reasoning.

[0141] For example, after obtaining the sequence generated by the AI ​​model, the sequence is segmented into multiple segments. This can be done according to specific rules: for text sequences, each complete sentence is segmented into a text segment; for video frames, a fixed number of frames are segmented into a video frame segment, such as every 50 frames; and for audio frame sequences, a fixed number of frames are segmented into an audio frame segment, such as every 30 audio frames. Subsequent redundancy reflection identification is also based on these segmented segments. For example, adjacent segments are scored for repetition, and the scoring results (e.g., whether the repetition score is greater than a threshold T for M consecutive rounds) determine whether the thought chain generated by the AI ​​model's current inference contains redundant reflections.

[0142] In another example, the sequence generated by the model's inference could be segmented into multiple reflection segments based on the reflection recognition module mentioned above. For instance, the sequence generated by the model could be segmented into multiple reflection segments based on reflection markers. It's easy to understand that the reflection markers differ depending on the modality of the sequence generated by the AI ​​model. For example, if the sequence is a text sequence, the reflection markers are reflection marker words. If the sequence is a sequence of audio frames, the reflection markers are reflection marker audio frames, such as audio frames containing the semantics of "Wait," "But," and "However." The appearance of such an audio frame indicates that the model has begun to reflect. Similarly, if the sequence is a video frame sequence, the reflection markers are reflection marker video frames. The appearance of such a video frame indicates that the model has begun to reflect.

[0143] In another example, to ensure the accuracy of the reflection fragment segmentation, the segmented fragments are further verified after segmentation. Only those that pass verification are identified as reflection fragments. For example, checking whether the segmented fragment contains conclusive content; if so, the fragment is considered a complete reflection fragment. Similarly, the method for identifying conclusive content differs for different modalities of the sequence. For example, for text modal sequences, the conclusive content is a sentence containing specific keywords, such as "the answer is." When the segmented fragment contains "the answer is," it is determined to contain a conclusive statement and is identified as a reflection fragment. For image modal sequences, the conclusive content is the bounding box. For example, in a face recognition scenario, the bounding box is used to mark the location of faces. The model continuously adjusts the position of the bounding box during the thinking process. The position of the bounding box changes and adjusts in each round of reflection; therefore, the bounding box can be used as the basis for identifying conclusive content.

[0144] For a detailed description of the reflection segmentation implementation, please refer to the detailed description of the reflection recognition module above. For the sake of brevity, it will not be repeated here.

[0145] Redundant reflections in the thought chain generated by the AI ​​model can be identified by evaluating the repetition of reflection segments. For example, a repetition scorer can be used to score the repetition of adjacent reflection segments, obtaining repetition scores s_[X, X+1], X=1, …, N-1 for reflection segment x and reflection segment x+1. If the reflection repetition score exceeds a pre-set threshold T (s_[NM, N-M+1],…, s_[N-1, N]) for M consecutive rounds, the thought chain is marked as a thought chain that needs to be terminated, and a termination operation is used to terminate the thought chain generated by the current inference of the AI ​​model.

[0146] For a detailed description of the scoring system in the redundant reflection and identification module above, please refer to the detailed description of the scoring system above. For the sake of brevity, it will not be repeated here.

[0147] In some other examples, a scorer can also be used to score several consecutive reflection segments, such as scoring three consecutive reflection segments, to obtain a repetition score.

[0148] The logic for judging redundant reflection can also be varied. For example, if the repetition score exceeds the first threshold T1, then the thought chain generated by the current reasoning of the large language model is judged to have redundant reflection; otherwise, there is no redundant reflection. Another example is if the repetition score exceeds the second threshold T2 for M consecutive rounds, then the thought chain generated by the current reasoning of the large language model is judged to have redundant reflection; otherwise, there is no redundant reflection. Yet another example is if the repetition score increases for H consecutive rounds, then the thought chain generated by the current reasoning of the large language model is judged to have redundant reflection; otherwise, there is no redundant reflection. Yet another example is if the repetition score exceeds the first threshold T1 and increases for H consecutive rounds, then the thought chain generated by the current reasoning of the large language model is judged to have redundant reflection; otherwise, there is no redundant reflection. Finally another example is if the repetition score exceeds the first threshold T1 and exceeds the second threshold T2 for M consecutive rounds, then the thought chain generated by the current reasoning of the large language model is judged to have redundant reflection; otherwise, there is no redundant reflection. For example, if the repetition score exceeds the first threshold T1, and exceeds the second threshold T2 for M consecutive rounds, and the repetition score increases for H consecutive rounds, then it is determined that the thought chain generated by the current reasoning of the large language model has redundant reflection; otherwise, there is no redundant reflection. This ensures the accuracy of redundant reflection identification.

[0149] In step S403, a termination operation is performed on the thought chain to stop the AI ​​model from thinking and output the response content corresponding to the query request.

[0150] To prevent the AI ​​model from getting bogged down in redundant reflection and generating a large number of redundant output sequences, a termination operation (also known as a soft termination operation) is performed on the thought chain when redundant reflection is detected in the current reasoning of the AI ​​model. This terminates the AI ​​model's thinking and outputs the response content corresponding to the query request. The termination operation can include only the soft termination operation of the thought chain; that is, when redundant reflection is detected in the current thought chain generated by the AI ​​model, a soft termination operation is performed on the thought chain. For example, the sampling probability of the reflection flag is reduced, and the sampling probability of the reflection end flag is increased to guide the AI ​​model to naturally end its thinking and output the response content for the query request based on the content of its current thinking.

[0151] Termination operations can also include only hard termination operations on thought chains (or simply hard termination operations). That is, when it is recognized that there is redundant reflection in the thought chain generated by the current reasoning of the AI ​​model, a hard termination operation is directly performed on the thought chain, forcibly stopping the AI ​​model's thinking and outputting the pre-set reflection termination content.

[0152] Understandably, the modality of the reflection termination content preset for the hard termination operation of the thought chain differs for different modal AI models. For example, for natural language processing AI models, the reflection termination content is the reflection termination sentence; for speech processing AI models, the reflection termination content is the reflection termination speech frame; and for visual AI models, the reflection termination content is the reflection termination video frame.

[0153] Termination operations can also include soft termination and hard termination of the thought chain. When redundant reflection is detected in the thought chain generated by the AI ​​model's reasoning, a soft termination operation is first executed to guide the AI ​​model to end its thinking naturally. If the soft termination operation cannot effectively terminate the AI ​​model's thinking, a hard termination operation is initiated to force the AI ​​model to exit thinking. This ensures that the AI ​​model can end its thinking normally during redundant reflection and return the response content, reducing user waiting time and improving user experience.

[0154] For a detailed explanation of the soft and hard termination operations of the MindChain, please refer to the detailed description of the MindChain soft termination module and the MindChain hard termination module above. For the sake of brevity, it will not be repeated here.

[0155] In another example, the AI ​​model inference method provided in this application embodiment can also support controlling whether to enable the AI ​​model inference method provided in this application embodiment by configuring an on / off control parameter. If configured to be enabled, the AI ​​model inference method provided in this application embodiment is executed, actively identifying whether there is redundant reflection in the thought chain generated by the AI ​​model inference. When redundant reflection is identified, a termination operation is performed on the thought chain to stop the redundant reflection of the AI ​​model. If configured to be disabled, the AI ​​model performs normal inference without interfering with the inference output of the AI ​​model.

[0156] As described above, the AI ​​model reasoning method provided in this application does not interfere with the normal reasoning of the AI ​​model when it is not caught in prolonged thinking. When redundant reflection is detected in the thought chain generated by the AI ​​model's reasoning, the probability distribution of the AI ​​model's output is adjusted to guide the AI ​​model to end its thinking, thus avoiding prolonged thinking and the generation of a large amount of redundant content. In other words, the AI ​​model reasoning method provided in this application does not adjust or update the parameters of the AI ​​model, and does not require retraining, saving training costs.

[0157] To verify the actual effect of the inference method of the AI ​​model provided in this application embodiment, the Qwq32B large model and the Qwen2.7Binstruct large model were tested on the MATH500 mathematical problem dataset. The test results showed that the total number of tokens output by the large model using the inference method of the AI ​​model provided in this application embodiment decreased by 16.54% compared with the large model not using the inference method of the AI ​​model provided in this application embodiment, and the model accuracy did not show significant degradation.

[0158] Practical verification shows that the inference method of the AI ​​model provided in this application embodiment can reduce the length of the token sequence output by the AI ​​model, avoid the token sequence output by the AI ​​model being too long, effectively alleviate the long-tail problem of the AI ​​model output, and ensure the accuracy of the model.

[0159] Based on the same concept as the aforementioned embodiment of an AI model inference method, this application also provides an AI model inference device 500. This AI model inference device 500 uses an AI model inference framework as a carrier and is used as an inference acceleration plugin. This plugin can accelerate the inference function of the AI ​​model during redundant reflection, effectively alleviating the problem of redundant reflection in the AI ​​model. The AI ​​model inference device 500 includes components for implementing... Figure 2-4 The units or modules of each step in the reasoning method of the AI ​​model shown.

[0160] Furthermore, the inference device of the AI ​​model provided in this application embodiment can be deployed in the cloud. In the cloud tenant scenario, the inference acceleration module provided to cloud users can accelerate the inference of the AI ​​model when it is caught in long thinking, effectively alleviating the problem of redundant reflection of the AI ​​model.

[0161] Figure 5 This is a schematic diagram of the structure of an AI model inference device provided in an embodiment of this application. Figure 5 As shown, the reasoning device 500 of the AI ​​model includes an acquisition module 501, a redundancy reflection identification module 503, and a termination module 504. The acquisition module 501 is used to acquire the sequence generated by the AI ​​model. The AI ​​model has the ability to reason through thought chains, and the sequence is generated by the AI ​​model based on the input query request. The redundancy reflection identification module 503 is used to identify redundancy reflection in the thought chain generated by the AI ​​model based on the sequence. The termination module 504 is used to perform a termination operation on the thought chain so that the AI ​​model stops thinking and outputs the response content corresponding to the query request.

[0162] In one possible implementation, the redundancy reflection identification module 502 is specifically used to: divide the sequence into multiple segments; and based on the multiple segments, identify the existence of redundant reflections in the thought chain generated by the AI ​​model inference.

[0163] In one possible implementation, multiple segments include multiple reflection segments; a specific implementation of dividing the sequence into multiple segments is: dividing the sequence based on a first reflection flag to obtain multiple reflection segments.

[0164] In another possible implementation, a specific implementation of segmenting the sequence based on the reflection flag to obtain multiple reflection segments is as follows: the sequence is segmented based on the first reflection flag to obtain multiple thinking segments; the thinking segments containing conclusive content among the multiple thinking segments are identified as reflection segments.

[0165] In another possible implementation, the redundant reflection identification module 503 is specifically used to: calculate the repetition of multiple reflection segments and obtain the repetition calculation result; if the repetition calculation result meets the preset conditions, it is determined that there is redundant reflection in the thought chain generated by the AI ​​model reasoning.

[0166] In another possible implementation, the repetition calculation result includes the repetition score of adjacent reflection segments, or the repetition score of N consecutive reflection segments, where N is an integer greater than 2; the preset conditions include one or more of the following: the repetition score is greater than a first threshold, the repetition score is greater than a second threshold for M consecutive rounds, and the repetition score increases for H consecutive rounds, where M and H are both positive integers.

[0167] In another possible implementation, the termination operation includes a soft termination operation; the termination module 504 is specifically used to: reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag.

[0168] In another possible implementation, the decrease in the sampling probability of the second reflection flag is positively correlated with the length of the sequence; the increase in the sampling probability of the end-of-reflection flag is positively correlated with the length of the sequence.

[0169] In another possible implementation, the termination operation includes a hard termination operation; the termination module 504 is specifically used to generate pre-set reflection termination content.

[0170] In another possible implementation, the termination operation includes a soft termination operation and a hard termination operation; the termination module 504 is specifically used to: perform a soft termination operation on the thought chain to reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag; if it is determined that the conditions for performing a hard termination operation are met, then perform a hard termination operation on the thought chain to make the AI ​​model generate the preset reflection end content.

[0171] In another possible implementation, the conditions for performing a hard termination operation include one or more of the following: the AI ​​model does not terminate thinking within a preset inference step, or the AI ​​model generates a reflection flag.

[0172] Optionally, the above AI model can be a large language model or a multimodal large model.

[0173] The AI ​​model inference device 500 according to the embodiments of this application can correspond to executing the methods described in the embodiments of this application, and the above and other operations and / or functions of each module in the AI ​​model inference device 500 are respectively for implementing Figure 2-4 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0174] This application embodiment also provides a computing device, including at least one processor, a memory, and a communication interface, wherein the processor is used to execute... Figure 2-4 The method described.

[0175] Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0176] like Figure 6 As shown, the computing device 600 includes at least one processor 601, a memory 602, and a communication interface 603. The processor 601, memory 602, and communication interface 603 are communicatively connected, which can be achieved via a wired (e.g., bus) or wireless connection. The communication interface 603 is used to send and / or receive data from other devices. The memory 602 stores computer instructions, which the processor 601 executes to perform the methods described in the foregoing method embodiments, thereby effectively mitigating the redundancy and reflexivity problem of the AI ​​model.

[0177] It should be understood that in the embodiments of this application, the processor 601 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0178] The memory 602 may include read-only memory and random access memory, and provides instructions and data to the processor 601. The memory 602 may also include non-volatile random access memory. Optionally, the random access memory may be, for example, high bandwidth memory (HBM).

[0179] The memory 602 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM).

[0180] It should be understood that the computing device 600 according to the embodiments of this application can perform the implementation of the embodiments of this application. Figure 2-4 The method shown is described in detail above, and will not be repeated here for the sake of brevity.

[0181] Embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer instructions are executed by a processor, the aforementioned method is implemented.

[0182] An embodiment of this application provides a chip including at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the method mentioned above.

[0183] Embodiments of this application provide a computer program or computer program product that includes instructions that, when executed, cause a computer to perform the methods mentioned above.

[0184] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0185] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0186] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A reasoning method for an AI model, characterized in that, include: Obtain the sequence generated by the AI ​​model, which has the ability to reason through thought chains, and the sequence is generated by the AI ​​model based on the input query request; Based on the sequence, redundant reflections are identified in the thought chain generated by the AI ​​model's reasoning. A termination operation is performed on the thought chain to stop the AI ​​model from thinking and output the response content corresponding to the query request.

2. The method according to claim 1, characterized in that, The step of identifying redundant reflections in the thought chain generated by the AI ​​model's reasoning based on the sequence includes: The sequence is divided into multiple segments; Based on the aforementioned multiple fragments, redundant reflections are identified in the thought chain generated by the AI ​​model's reasoning.

3. The method according to claim 2, characterized in that, The multiple segments include multiple reflection segments; The step of dividing the sequence into multiple segments includes: The sequence is segmented based on the first reflection marker to obtain multiple reflection segments.

4. The method according to claim 3, characterized in that, The sequence is segmented based on reflection markers to obtain multiple reflection segments, including: The sequence is segmented based on the first reflection marker to obtain multiple thought fragments; The thought segments containing conclusive content among the multiple thought segments are identified as reflection segments.

5. The method according to claim 3 or 4, characterized in that, The identification of redundant thinking in the thought chain generated by the AI ​​model's reasoning based on the multiple fragments includes: The repetition rate of the multiple reflection segments is calculated to obtain the repetition rate calculation results; If the repetition calculation result meets the preset conditions, it is determined that there is redundant reflection in the thought chain generated by the AI ​​model reasoning.

6. The method according to claim 5, characterized in that, The repetition calculation result includes the repetition score of adjacent reflection segments, or the repetition score of N consecutive reflection segments, where N is an integer greater than 2; The preset conditions include one or more of the following: the repetition score is greater than a first threshold, the repetition score is greater than a second threshold for M consecutive rounds, and the repetition score increases for H consecutive rounds, where M and H are both positive integers.

7. The method according to any one of claims 1-6, characterized in that, The termination operation includes a soft termination operation; The termination operation on the thought chain includes: Decrease the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag.

8. The method according to claim 7, characterized in that, The decrease in the sampling probability of the second reflection flag is positively correlated with the length of the sequence; The increase in the sampling probability of the reflection end marker is positively correlated with the length of the sequence.

9. The method according to any one of claims 1-6, characterized in that, The termination operation includes a hard termination operation; The termination operation on the thought chain includes: Generate pre-set reflection conclusion content.

10. The method according to any one of claims 1-6, characterized in that, The termination operation includes soft termination and hard termination. The termination operation on the thought chain includes: A soft termination operation is performed on the thought chain to reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag. If the conditions for performing a hard termination operation are met, then the hard termination operation is performed on the thought chain so that the AI ​​model generates preset reflection termination content.

11. The method according to claim 10, characterized in that, The conditions for performing the hard termination operation include the AI ​​model not terminating its thinking within a preset inference step, and the AI ​​model generating one or more of the reflection flags.

12. An inference device for an AI model, characterized in that, include: The acquisition module is used to acquire sequences generated by the AI ​​model, which has the ability to reason through thought chains, and the sequences are generated by the AI ​​model based on the input query request. A redundant reflection identification module is used to identify redundant reflections in the thought chain generated by the AI ​​model's reasoning, based on the sequence. The termination module is used to perform a termination operation on the thought chain, so that the AI ​​model stops thinking and outputs the response content corresponding to the query request.

13. The apparatus according to claim 12, characterized in that, The redundancy reflection identification module is specifically used for: The sequence is divided into multiple segments; Based on the aforementioned multiple fragments, redundant reflections are identified in the thought chain generated by the AI ​​model's reasoning.

14. The apparatus according to claim 13, characterized in that, The multiple segments include multiple reflection segments; The step of dividing the sequence into multiple segments includes: The sequence is segmented based on the first reflection marker to obtain multiple reflection segments.

15. The apparatus according to claim 14, characterized in that, The sequence is segmented based on reflection markers to obtain multiple reflection segments, including: The sequence is segmented based on the first reflection marker to obtain multiple thought fragments; The thought segments containing conclusive content among the multiple thought segments are identified as reflection segments.

16. The apparatus according to claim 14 or 15, characterized in that, Based on the multiple fragments, the identification of redundant reflections in the thought chain generated by the AI ​​model's reasoning includes redundant reflections: The repetition rate of the multiple reflection segments is calculated to obtain the repetition rate calculation results; If the repetition calculation result meets the preset conditions, it is determined that there is redundant reflection in the thought chain generated by the AI ​​model reasoning.

17. The apparatus according to claim 16, characterized in that, The repetition calculation result includes the repetition score of adjacent reflection segments, or the repetition score of N consecutive reflection segments, where N is an integer greater than 2; The preset conditions include one or more of the following: the repetition score is greater than a first threshold, the repetition score is greater than a second threshold for M consecutive rounds, and the repetition score increases for H consecutive rounds, where M and H are both positive integers.

18. The apparatus according to any one of claims 12-17, characterized in that, The termination operation includes a soft termination operation; The termination module is specifically used to: reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag.

19. The apparatus according to claim 18, characterized in that, The decrease in the sampling probability of the second reflection flag is positively correlated with the length of the sequence; The increase in the sampling probability of the reflection end marker is positively correlated with the length of the sequence.

20. The apparatus according to any one of claims 12-17, characterized in that, The termination operation includes a hard termination operation; The termination module is specifically used to generate pre-set reflection termination content.

21. The apparatus according to any one of claims 12-17, characterized in that, The termination operation includes soft termination and hard termination. The termination module is specifically used to: perform a soft termination operation on the thought chain, so as to reduce the sampling probability of the second reflection flag and increase the sampling probability of the reflection end flag; If the conditions for performing a hard termination operation are met, then the hard termination operation is performed on the thought chain so that the AI ​​model generates preset reflection termination content.

22. The apparatus according to claim 21, characterized in that, The conditions for performing the hard termination operation include the AI ​​model not terminating its thinking within a preset inference step, and the AI ​​model generating one or more of the reflection flags.

23. A chip comprising at least one processor and a communication interface, the processor being configured to perform the method as described in any one of claims 1-11.

24. A computer program or computer program product comprising instructions that, when executed, cause a computer to perform the method as described in any one of claims 1-11.

25. A computing device, comprising a memory and a processor, characterized in that, The memory stores instructions that, when executed by a processor, cause the method described in any one of claims 1-11 to be implemented.

26. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it causes the method as described in any one of claims 1-11 to be implemented.

Images