Inference question processing method, apparatus and system

Abstract

The present application provides an inference question processing method, apparatus and system. The method comprises: acquiring a question to be inferred; and then on the basis of some of intermediate results provided by an intermediate template and an inference model, determining a result of said question. The present application reduces the time for calculating the intermediate results in the process of inferring said question, and increases the speed of inferring said question.

Description

Methods, apparatus and systems for handling reasoning problems

[0001] This application claims priority to Chinese Patent Application No. 202411596089.2, filed on November 8, 2024, entitled "Method, Apparatus and System for Processing Reasoning Problems", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of cloud technology, and in particular to a method, apparatus and system for processing reasoning problems. Background Technology

[0003] Large Language Model (LLM) is a core component of intelligent question-answering systems and intelligent assistant systems. Users or system programs input questions into the LLM in the form of questions, and then the LLM generates and outputs the corresponding answers in text form.

[0004] Typically, LLMs are deployed in the cloud to enable rapid response to problems and more efficient problem handling. This deployment approach fully leverages the various advantages of cloud computing platforms, such as powerful computing capabilities and high availability.

[0005] However, this deployment method requires transmitting issues to the cloud for processing. During the transmission of issues and the processing of issues in the cloud, users' privacy data may be at risk of being leaked. Summary of the Invention

[0006] This application provides a method, apparatus, and system for processing reasoning problems. After obtaining the problem to be reasoned, the result of the problem to be reasoned is determined based on some intermediate results and reasoning models provided by intermediate templates. This reduces the time spent calculating intermediate results during the reasoning process and speeds up the reasoning of the problem to be reasoned.

[0007] Firstly, this application provides a method for processing reasoning problems, applied to a terminal. The method includes: the terminal acquiring a problem to be reasoned about; then, the terminal determining the result of the problem to be reasoned about based on an intermediate template and a reasoning model. The intermediate template provides partial intermediate results from the reasoning process of the reasoning model in reasoning about the problem to be reasoned about. These partial intermediate results are derived from a test question, and the test question has the same text template as the problem to be reasoned about. Thus, this application's implementation is executed by the terminal, avoiding the uploading of questions containing user privacy information to the cloud, thereby protecting user privacy and security. Furthermore, in this embodiment, the result of the problem to be reasoned about can be determined based on the intermediate template and the reasoning model. During the reasoning process, using partial intermediate results provided by the intermediate template can effectively reduce the time spent calculating intermediate results and accelerate the processing efficiency of reasoning problems.

[0008] In one possible implementation, the intermediate template is also used to provide location information of the intermediate result to be calculated, wherein the intermediate result to be calculated is an intermediate result that the intermediate template did not provide during the reasoning process of the inference model based on the intermediate template to reason about the problem to be reasoned.

[0009] In the implementation of this application, the intermediate template provides the location information of the result to be calculated, eliminating the need to determine which intermediate results need to be calculated based on the intermediate template. This further reduces the time required to determine the location of the result to be calculated and speeds up the processing of reasoning problems.

[0010] In one possible implementation, the intermediate template is further used to provide a category feature of at least one result of the text template, the category feature being used to indicate the correspondence between the intermediate result to be computed and at least one result of the text template.

[0011] In the implementation of this application, it can be determined whether the result of the reasoning problem can be directly obtained from the determined result to be calculated based on the correspondence between the intermediate result to be calculated and the above-mentioned relationship, thereby further accelerating the processing speed of the reasoning problem.

[0012] In one possible implementation, the inference model includes multiple processing layers. The processing layers are used to determine intermediate results that are not provided by the intermediate template during the inference process of the inference model inferring the problem to be inferred. The processing layers are also used to determine whether the result of the problem to be inferred can be determined based on the unprovided intermediate results.

[0013] In the implementation of this application, the result of the problem to be reasoned can be determined based on the unprovided intermediate results. If the result of the problem to be reasoned can be determined, the result of the problem to be reasoned can be obtained directly by the processing layer. This is different from the prior art, where only the last processing layer can obtain the result of the problem to be reasoned, thus speeding up the reasoning process.

[0014] In one possible implementation, before obtaining the problem to be reasoned, the method further includes: receiving multiple intermediate templates sent from the cloud; and before determining the result of the problem to be reasoned based on the intermediate templates and the reasoning model, the method further includes: determining whether there is an intermediate template among the multiple intermediate templates that corresponds to the problem to be reasoned.

[0015] In the implementation of this application, the intermediate template sent by the cloud is received, without the terminal needing to determine the intermediate template corresponding to different text templates, which speeds up the process of obtaining the intermediate template and further accelerates the reasoning process.

[0016] Furthermore, if an intermediate template corresponding to the problem to be reasoned is determined to exist, the result of the problem to be reasoned can be obtained based on the intermediate template and the model to be reasoned. If no intermediate template corresponding to the problem to be reasoned is determined to exist, the result of the problem to be reasoned can still be obtained through the problem to be reasoned and the reasoning model, thus ensuring the stability of obtaining the result of the problem to be reasoned.

[0017] In one possible implementation, determining whether there exists an intermediate template corresponding to the problem to be reasoned among multiple intermediate templates includes: determining a text template; and determining whether there exists an intermediate template matching the problem to be reasoned based on the text template and a first mapping relationship, wherein the first mapping relationship includes a mapping relationship between the text template and the intermediate template.

[0018] In the implementation of this application, the same text template can correspond to multiple different questions to be reasoned. By transforming the determination of whether there is an intermediate template corresponding to the question to be reasoned into first determining the text template of the question to be reasoned, and then determining whether there is an intermediate template corresponding to the text template, the size of the first mapping relationship is reduced, and the time to determine the intermediate template by searching the first mapping relationship is further reduced, thus speeding up the reasoning process.

[0019] In one possible implementation, the inference model is a large language model, and the processing layer is a self-attention conversion layer.

[0020] In the implementation of this application, the self-attention mechanism allows the model to consider the entire context, rather than just local information, when processing text sequences. Based on a large language model that includes a self-attention transformation layer, determining the outcome of the question to be reasoned enables the model to better understand the meaning of sentences or paragraphs, thereby improving the understanding and accuracy of answers to complex questions.

[0021] Secondly, this application provides a method for processing reasoning problems, which is applied in the cloud. The method includes: obtaining intermediate results for each of the multiple test problems, wherein the text templates corresponding to the multiple test problems are identical; determining the intermediate template corresponding to the text template based on the intermediate results of the multiple test problems; and sending the intermediate template to a terminal.

[0022] In the implementation method of this application, the cloud determines the intermediate template corresponding to the text template based on the intermediate results of multiple test questions, and templates the intermediate results of a type of question. When there is a corresponding intermediate template for the question to be reasoned, the intermediate results that need to be calculated can be reduced, and the speed of reasoning can be accelerated.

[0023] In addition, the cloud sends the intermediate template to the terminal, which can directly use the intermediate template determined by the cloud to reason about the problem to be reasoned, further accelerating the reasoning speed.

[0024] In one possible implementation, determining the intermediate template corresponding to the text template based on the intermediate results of multiple test questions includes: determining the position information of a first intermediate result and a second intermediate result of the intermediate template based on the intermediate results of multiple test questions, wherein the first intermediate result is an intermediate result that does not need to be calculated when the inference model infers the problem to be inferred based on the intermediate template, and the second intermediate result is an intermediate result that needs to be calculated when the inference model infers the problem to be inferred based on the intermediate template, and the text template corresponding to the problem to be inferred is the same as that of the test questions; and determining the correspondence between the second intermediate result and at least one result of the text template based on the second intermediate result.

[0025] In the implementation of this application, the inference model is determined to be based on an intermediate template. The intermediate results do not need to be calculated during the inference process, which can reduce the time spent calculating intermediate results and speed up the inference process.

[0026] In addition, determining the location information of the second intermediate result and the correspondence between the second intermediate result and at least one result of the text template helps to directly determine which intermediate results need to be calculated, and to judge whether the result of the problem to be reasoned can be obtained directly based on the calculation results and the correspondence, thereby further accelerating the speed of reasoning.

[0027] Thirdly, this application provides a reasoning problem processing apparatus, which includes: an acquisition module for acquiring a problem to be reasoned; and a first determination module for determining the result of the problem to be reasoned based on an intermediate template and a reasoning model, wherein the intermediate template is used to provide some intermediate results in the process of reasoning the problem to be reasoned by the reasoning model, and the some intermediate results are obtained based on a test question, and the test question is the same as the text template corresponding to the problem to be reasoned.

[0028] In one possible implementation, the intermediate template is also used to provide location information of the intermediate result to be calculated, wherein the intermediate result to be calculated is an intermediate result that the intermediate template did not provide during the reasoning process of the inference model based on the intermediate template to reason about the problem to be reasoned.

[0029] In one possible implementation, the intermediate template is further used to provide a category feature of at least one result of the text template, the category feature being used to indicate the correspondence between the intermediate result to be computed and at least one result of the text template.

[0030] In one possible implementation, the inference model includes multiple processing layers. The processing layers are used to determine intermediate results that are not provided by the intermediate template during the inference process of the inference model inferring the problem to be inferred. The processor layers are also used to determine whether the result of the problem to be inferred can be determined based on the unprovided intermediate results.

[0031] In one possible implementation, the device further includes: a receiving module for receiving multiple intermediate templates sent from the cloud; and a second determining module for determining whether there is an intermediate template among the multiple intermediate templates that corresponds to the problem to be reasoned.

[0032] In one possible implementation, the second determining module is specifically used to: determine a text template; and determine whether there exists an intermediate template that matches the problem to be reasoned, based on the text template and a first mapping relationship, wherein the first mapping relationship includes the mapping relationship between the text template and the intermediate template.

[0033] In one possible implementation, the inference model is a large language model, and the processing layer is a self-attention conversion layer.

[0034] The effects of the reasoning problem processing devices of the above-mentioned implementations are similar to the effects of the reasoning problem processing methods of the first aspect or any implementation of the first aspect, and will not be repeated here.

[0035] Fourthly, this application provides a processing apparatus for reasoning problems, comprising: an acquisition module for acquiring intermediate results of each of the multiple test problems, wherein the text templates corresponding to the multiple test problems are identical; a determination module for determining an intermediate template corresponding to the text template based on the intermediate results of the multiple test problems; and a sending module for sending the intermediate template to a terminal.

[0036] In one possible implementation, the determining module is specifically used to: determine the location information of a first intermediate result and a second intermediate result of an intermediate template based on the intermediate results of multiple test questions; wherein, the first intermediate result is an intermediate result that does not need to be calculated when the inference model infers the question to be inferred based on the intermediate template, and the second intermediate result is an intermediate result that needs to be calculated when the inference model infers the question to be inferred based on the intermediate template, and the text templates corresponding to the question to be inferred and the test questions are the same; and based on the second intermediate result, determine the correspondence between the second intermediate result and at least one result of the text template.

[0037] The effects of the reasoning problem processing devices in the above-described implementations are similar to those of the reasoning problem processing methods in the second aspect or any of the implementations of the second aspect, and will not be repeated here.

[0038] Fifthly, this application provides a reasoning problem processing system, which includes a terminal and a cloud. The terminal includes a reasoning problem processing apparatus as provided in the third aspect or any possible implementation thereof; the cloud includes a reasoning problem processing apparatus as provided in the fourth aspect or any possible implementation thereof.

[0039] The effect of the reasoning problem processing system of this embodiment is similar to that of the reasoning problem processing method of the first aspect or any possible implementation of the first aspect described above, and will not be repeated here.

[0040] Sixthly, embodiments of this application provide a device cluster including at least one device, each device including a processor and a memory. The processor of the at least one device is configured to execute instructions stored in the memory of the at least one device, causing the device cluster to perform a reasoning problem processing method in the first aspect or any possible implementation of the first aspect, or to perform a reasoning problem processing method in the second aspect or any possible implementation of the second aspect.

[0041] The effect of the device cluster in this embodiment is similar to the effect of the reasoning problem processing method of the first aspect or any possible implementation of the first aspect described above, or similar to the effect of the reasoning problem processing method of the second aspect or any possible implementation of the second aspect described above, and will not be repeated here.

[0042] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when executed by a device cluster, cause the device cluster to perform a reasoning problem processing method in the first aspect or any possible implementation of the first aspect, or to perform a reasoning problem processing method in the second aspect or any possible implementation of the second aspect.

[0043] The effect of the computer program product of this embodiment is similar to that of the reasoning problem processing method of the first aspect or any possible implementation of the first aspect, or similar to that of the reasoning problem processing method of the second aspect or any possible implementation of the second aspect, and will not be repeated here.

[0044] Eighthly, embodiments of this application provide a computer-readable storage medium including computer program instructions. When the computer program instructions are executed by a device cluster, the device cluster executes the reasoning problem processing method in the first aspect or any possible implementation of the first aspect, or executes the reasoning problem processing method in the second aspect or any possible implementation of the second aspect.

[0045] The effect of the computer-readable storage medium of this embodiment is similar to the effect of the reasoning problem processing method of the first aspect or any possible implementation of the first aspect described above, or similar to the effect of the reasoning problem processing method of the second aspect or any possible implementation of the second aspect described above, and will not be repeated here. Attached Figure Description

[0046] Figure 1 is a flowchart illustrating an exemplary method for processing reasoning problems;

[0047] Figure 2 is a flowchart illustrating another method for handling reasoning problems;

[0048] Figure 3 is a schematic diagram illustrating an intermediate result as an example;

[0049] Figure 4 is a schematic diagram illustrating another intermediate result as an example;

[0050] Figure 5 is a schematic diagram illustrating the distribution of intermediate results as an example;

[0051] Figure 6 is a schematic diagram of the structure of an exemplary device for processing reasoning problems;

[0052] Figure 7 is a schematic diagram of the structure of another example of a reasoning problem processing device;

[0053] Figure 8 is a schematic diagram of the structure of another reasoning problem processing system, which is an example of this.

[0054] Figure 9 is a schematic diagram of the structure of an exemplary computing device;

[0055] Figure 10 is a schematic diagram of the structure of an exemplary storage device;

[0056] Figure 11 is a schematic diagram of the structure of an exemplary computing device cluster;

[0057] Figure 12 is a schematic diagram of the structure of an exemplary storage device cluster;

[0058] Figure 13 is an exemplary schematic diagram of the interaction between a computing device and a storage device. Detailed Implementation

[0059] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0060] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0061] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0062] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0063] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0064] Before describing the technical solutions of the embodiments of this application, a brief introduction to the background technology and technical terms involved in the embodiments of this application will be given first:

[0065] Full reasoning: The process of inputting a problem to be reasoned about and outputting the first word of the result to be reasoned about.

[0066] Incremental reasoning: the process of deducing the next character based on the previous character, such as the process of deducing the second character of the result to be inferred based on the first character of the result to be inferred.

[0067] Large Language Model (LLM) is a core component of intelligent question-answering systems and intelligent assistant systems. Users or system programs input questions into the LLM in the form of questions, and then the LLM generates and outputs the corresponding answers in text form.

[0068] In the process of reasoning about a problem, LLM not only generates the answer to the problem but also produces intermediate results, such as query (Q) vectors, key (K) vectors, and value (V) vectors. LLM obtains the answer to the problem by generating it word by word. That is, LLM requires one full inference and multiple incremental inferences to reason about the answer to a problem. Both full inference and incremental inference can be understood as the process of deriving the next word based on the previous words.

[0069] Currently, in order to improve the speed of LLM reasoning problems, LLM caches the intermediate results of historical text and concatenates the intermediate results of the newly calculated input text with the intermediate results of the historical text. The answer to the problem is generated using the concatenated intermediate results, reducing the repeated calculation of historical text.

[0070] For example, if the historical text is A, the new input text is B, and the desired next character is c, the query vector for the historical text A is QA, the key vector is KA, and the value vector is VA. LLM transforms the new input text B into its corresponding query vector QB. In scenarios like A+B, where the historical text A serves as a prefix to the question, LLM calculates the key vector KB and value vector VB for the new input text B based on QB, KA, and VA. It then concatenates KA and KB to obtain the concatenated key vector K-AB, and concatenates VA and VB to obtain the concatenated value vector V-AB. LLM then infers c based on K-AB1 and V-AB1. This eliminates the need to infer the entire "A+B" structure to generate the overall key vector K-AB2 and value vector V-AB2, thus accelerating the reasoning process. In this context, uppercase letters represent one or more characters, and lowercase letters represent a single character.

[0071] However, for problem types where historical text and new input text are intertwined, such as A1+B1+A2+B2…, there is still a lack of efficient ways to accelerate reasoning.

[0072] Furthermore, LLMs are typically deployed in the cloud to enable rapid response to and more efficient problem handling. However, this deployment method requires transmitting issues to the cloud for processing, and during the transmission and processing of issues in the cloud, users' privacy data may be at risk of being leaked.

[0073] In addition, when a user initiates a request to obtain the result of a question and transmits the question to the cloud, the cloud infers the result of the question and returns it to the user. The end-to-end latency in transmitting the question and its result is relatively long, which increases the time it takes for the user to obtain the result.

[0074] To address the numerous problems existing in the aforementioned related technologies, this application provides a method, apparatus, and system for processing reasoning problems. This reasoning problem processing apparatus can be deployed in a terminal, avoiding the uploading of problems containing user privacy information to the cloud and protecting user privacy. Furthermore, in this embodiment, the result of the problem to be reasoned can be determined based on an intermediate template and a reasoning model. During the reasoning process, using some intermediate results provided by the intermediate template reduces the time spent calculating intermediate results, thus accelerating the reasoning process. Moreover, the terminal can obtain the intermediate template determined by the cloud before triggering the reasoning of the problem, avoiding data transmission between the terminal and the cloud during the reasoning process. This ensures that the time for the user to obtain the result does not include end-to-end latency, reducing the time it takes for the user to receive the result.

[0075] Furthermore, the inference model includes multiple processing layers. Unlike existing technologies, where each processing layer needs to calculate its output based on the output of the previous processing layer and the input of the current layer, and determine the result of the inference problem based on the output of the last processing layer, in this embodiment, each processing layer infers a partial intermediate result based on an intermediate template. This partial intermediate result is not provided in the intermediate template. Based on this partial intermediate result and the category features provided by the intermediate template, it is determined whether the result of the inference problem can be determined based on this partial intermediate result. If the result of the inference problem can be determined based on this partial intermediate result, the result of the inference problem can be obtained directly from this processing layer. Unlike existing technologies, where only the last processing layer can obtain the result of the inference problem, this allows for skipping layers to obtain the result of the inference problem, further accelerating the inference process.

[0076] Figure 1 is a flowchart illustrating an exemplary method for processing reasoning problems. This method can be applied to a terminal, specifically, to a service or application within the terminal. The terminal can be a smartphone, tablet, laptop, or desktop computer, etc., and the service or application can be a recommendation service or application, etc.

[0077] Please refer to Figure 1. The method flow may include, but is not limited to, the following steps:

[0078] S101. The terminal receives multiple intermediate templates sent from the cloud.

[0079] For example, the terminal can obtain multiple intermediate templates generated by the cloud from the cloud side. In some embodiments, the terminal sends a request to the cloud to obtain an intermediate template, and the cloud can respond to the request by sending the intermediate template generated by the cloud to the terminal, and the terminal receives the intermediate template sent by the cloud.

[0080] In a specific embodiment, the terminal sends a first message including the acquisition request to the cloud. After receiving the first message, the cloud sends a second message including an intermediate template to the terminal. The terminal receives the second message and obtains the intermediate template from the second message.

[0081] In other embodiments, the cloud sends an intermediate template generated by the cloud to the terminal at preset intervals, and the terminal receives the intermediate template sent by the cloud.

[0082] In a specific embodiment, the cloud sends a third message including an intermediate template to the terminal at preset intervals. The terminal receives the third message and obtains the intermediate template from the third message.

[0083] In an optional embodiment, after receiving multiple intermediate templates sent by the cloud, the terminal stores the multiple intermediate templates. Before the terminal obtains the question to be reasoned, the terminal obtains the multiple intermediate templates stored by the terminal.

[0084] The intermediate template can be used to provide some intermediate results in the process of reasoning about the problem to be reasoned. These intermediate results are derived from the reasoning of the test question, which is the same as the text template corresponding to the problem to be reasoned.

[0085] Intermediate templates can also be used to provide location information for intermediate results to be calculated. These intermediate results are those not provided by the intermediate template during the reasoning process of the problem to be reasoned.

[0086] The intermediate template can also be used to provide a category feature of at least one result of the text template, the category feature being used to indicate the correspondence between the intermediate result to be calculated and at least one result of the text template.

[0087] In this context, a text template refers to a type of question with a fixed format. By filling in the blanks in the text template, a reasoning question can be obtained. Different types of questions correspond to different text templates. If at least one blank is filled in differently in the same text template, different reasoning questions will be obtained.

[0088] In a specific embodiment, text template 1 can be as follows: "There is a user named xx who uses a xx mobile phone. He was active for xx days in the past month. The approximate click-through rate is xx%. Within a month, he clicked on xx out of the xx promoted apps that exposed him to him. Determine whether the user will click on this ad." Here, xx represents the parts to be filled in.

[0089] In one embodiment, by filling in the text template 1 with the part to be filled, the following reasoning question 1 can be obtained: "There is a user Y1 who uses a phone A1. In one month, he was active for 10 days and clicked on 4 out of 20 promotions recommended to him. The click-through rate of the promotions is 20%. Determine whether the user will click on this promotion."

[0090] In another embodiment, by filling in the text template 1 with the part to be filled, the following reasoning question 2 can be obtained: "There is a user Y1 who uses a phone A1. In one month, he was active for 15 days and clicked on 10 out of 20 recommended promotions. The click-through rate of the promotions is 50%. Determine whether the user will click on this promotion."

[0091] It should be noted that the text templates for Question 1 and Question 2 above are the same. However, the user's active days, click-through rate, and the number of promoted applications clicked are different in Question 1 and Question 2 above. Therefore, Question 1 and Question 2 are different questions to be inferred.

[0092] In an optional embodiment, the reasoning problem 1 can be the test problem described above, and the reasoning problem 2 can be the reasoning problem described above.

[0093] S102. Terminal obtains the problem to be inferred.

[0094] For example, the terminal may obtain the reasoning question in response to a received user operation, or the terminal may generate the reasoning question corresponding to a task in response to a task. Optionally, an application or service in the terminal may also automatically generate the reasoning question, and the terminal may obtain the reasoning question generated by the application or service.

[0095] Here, the question to be reasoned refers to the question that requires feedback from the terminal, and the type of question to be reasoned can be question-and-answer type, recommendation type, etc.

[0096] In one example, the question to be reasoned about could be "Is tomorrow's weather most likely to be cloudy or sunny?" In another example, the question to be reasoned about could be "Where is roujiamo (Chinese hamburger) a specialty?" In yet another example, the question to be reasoned about could be question 2 mentioned above.

[0097] In an optional embodiment, the terminal may provide a question-and-answer page for users to input questions. Users input questions to be reasoned about based on this page, and the terminal obtains the input questions. For example, the terminal includes a human-computer interaction component, through which users input questions to be reasoned about. The human-computer interaction component may be a voice input component, a touch display component, etc., and the touch display component may be a touchscreen.

[0098] In an optional embodiment, after acquiring a task, the terminal can determine whether a text template corresponding to the task exists among multiple text templates based on the task type and mapping relationship 1. If a text template corresponding to the task exists, the terminal can generate a reasoning question corresponding to the task based on the text template; if no text template corresponding to the task exists, the terminal can generate a reasoning question corresponding to the task based on the task. For example, the terminal parses the task to generate the reasoning question corresponding to the task.

[0099] In an optional embodiment, mapping relationship 1 includes multiple task types and text templates that correspond one-to-one with each of the multiple task types. The terminal can search mapping relationship 1 according to the task type of the task to determine whether there is a text template corresponding to the task among the multiple text templates.

[0100] In one example, mapping relationship 1 is shown in Table 1 below.

[0101] Table 1

[0102] For example, the task type of this task is task type 1. The terminal device looks up the mapping relationship 1 shown in Table 2 based on "task type 1" to determine the text template 1 corresponding to "task type 1". Therefore, the terminal determines that the text template corresponding to this task is text template 1.

[0103] The task type is task type x. The terminal device searches for mapping relationship 1 as shown in Table 2 based on "task type x" to determine that there is no text template corresponding to this task among the multiple text templates.

[0104] In an optional embodiment, the terminal stores the plurality of text templates. After obtaining the text template corresponding to the task, the terminal fills in the part to be filled in the text to obtain the reasoning question corresponding to the task.

[0105] In a specific embodiment, the terminal obtains historical data regarding user Y1's active days in the past month, the number of promotions recommended to user A1 by the terminal, and the number of promotions clicked by user A1. The terminal determines the click-through rate (CTR) of user A1's clicks based on the number of promotions recommended to user A1 and the number of promotions clicked by user A1. The terminal obtains the terminal type used by user A1 using a terminal type acquisition command. The terminal fills the obtained and calculated relevant information into text template 1 to obtain the reasoning question 1.

[0106] S103. The terminal determines whether there is an intermediate template among multiple intermediate templates that corresponds to the problem to be reasoned.

[0107] The terminal determines the text template corresponding to the question to be reasoned. Based on the text template and mapping relationship 2 (i.e. the first mapping relationship), the terminal determines whether there is an intermediate template that matches the question to be reasoned. Mapping relationship 2 includes the mapping relationship between the text template and the intermediate template.

[0108] In some embodiments, the problem to be reasoned is generated by the terminal based on a text template, and the terminal determines the text template as the text template corresponding to the problem to be reasoned.

[0109] In other embodiments, the problem to be reasoned is either a user input problem obtained by the terminal, or a problem generated by the terminal's parsing task. The terminal matches the problem to be reasoned with the plurality of text templates to determine the text template corresponding to the problem to be reasoned.

[0110] In some specific embodiments, the terminal determines one text template. Based on the text template and mapping relationship 2, the terminal determines the intermediate template corresponding to the text template and identifies the intermediate template as the text template corresponding to the question to be reasoned.

[0111] In some other specific embodiments, the terminal determines at least two text templates. After determining at least two intermediate templates corresponding to the at least two text templates and mapping relationship 2, the terminal concatenates the at least two intermediate templates in the order in which the corresponding text templates appear in the problem to be reasoned to obtain the text template corresponding to the problem to be reasoned.

[0112] For example, the at least two text templates are text template 2 and text template 6, where text template 2 corresponds to intermediate template 2, and text template 6 corresponds to intermediate template 6. If text template 2 appears before text template 6, the text template corresponding to the reasoning question determined by the terminal is "intermediate template 2 intermediate template 6"; if text template 2 appears after text template 6, the text template corresponding to the reasoning question determined by the terminal is "intermediate template 6 intermediate template 2". Here, the position of text template 2 refers to its location within the problem to be processed, and the position of text template 6 refers to its location within the problem to be processed.

[0113] In an optional embodiment, the mapping relationship 2 includes multiple text templates and intermediate templates that correspond one-to-one with the multiple text templates. The terminal searches the mapping relationship 2 based on the text templates to determine whether there is an intermediate template that matches the question to be reasoned.

[0114] In one example, mapping relationship 2 is shown in Table 2 below.

[0115] Table 2

[0116] For example, the text template is text template 2. The terminal device looks up the mapping relationship 2 shown in Table 2 based on "text template 2" and determines that "text template 2" corresponds to intermediate template 2. Therefore, the terminal determines that the intermediate template corresponding to this text template is intermediate template 2.

[0117] The text template is text template y. The terminal device searches for mapping relationship 2 as shown in Table 2 based on "text template y" and determines that there is no intermediate template corresponding to text template y.

[0118] S104. The terminal determines the result of the problem to be reasoned based on the intermediate template and the reasoning model.

[0119] The reasoning model includes multiple processing layers. The processing layers are used to determine intermediate results that are not provided by the intermediate template during the reasoning process of the reasoning model in reasoning the problem to be reasoned. The processing layers are also used to determine whether the result of the problem to be reasoned can be determined based on the unprovided intermediate results.

[0120] The inference model is a Large Language Model (LLM), and the processing layer is a self-attention Transformer layer. This LLM can be a megabyte-scale LLM or a byte-scale LLM.

[0121] In an optional embodiment, an intermediate template can be used to provide partial intermediate results during the reasoning process of the problem to be reasoned. The terminal inputs the problem to be reasoned and the partial intermediate results provided by the intermediate template into the reasoning model, the reasoning model outputs the result of the problem to be reasoned, and the terminal obtains the result of the problem to be reasoned output by the reasoning model.

[0122] In a specific embodiment, for each of the plurality of processing layers: the processing layer determines the location information of the intermediate result that it needs to calculate, and calculates the intermediate result. The inference model performs inference based on the intermediate result calculated by each processing layer, the intermediate result provided by the intermediate layer, and the question to be inferred, to obtain the result of the inference question.

[0123] It should be noted that the process of the processing layer calculating the intermediate results to be calculated in the processing layer is similar to the process of the Transformer layer calculating the K vector and V vector. The specific implementation process of the inference model to obtain the result of the inference problem based on the problem to be inferred and the intermediate results is similar to the specific implementation process of the LLM based on the problem to be inferred and the intermediate results. This application embodiment will not elaborate on these aspects.

[0124] In an optional embodiment, the intermediate template is further used to provide location information of the intermediate results to be calculated. The terminal inputs the problem to be reasoned, the partial intermediate results provided by the intermediate template, and the location information of the intermediate results to be calculated into the inference model. The inference model outputs the result of the problem to be reasoned, and the terminal obtains the result of the problem to be reasoned output by the inference model.

[0125] In a specific embodiment, the multiple processing layers calculate intermediate results indicating the location information. The inference model performs inference based on the intermediate results calculated by each processing layer, the intermediate results provided by the intermediate layers, and the question to be inferred, to obtain the result of the inference question.

[0126] In an optional embodiment, the intermediate template is further used to provide at least one category feature of the text template. The terminal inputs the question to be reasoned, some intermediate results provided by the intermediate template, the location information of the intermediate results to be calculated, and the category feature of at least one result of the text template into the reasoning model. The reasoning model provides the result of the question to be reasoned, and the terminal obtains the result of the question to be reasoned output by the reasoning model.

[0127] In a specific embodiment, the i-th processing layer calculates the intermediate result of the position to be calculated in the i-th processing layer. The i-th processing layer determines whether the result of the problem to be reasoned can be determined based on the intermediate result. If the result of the problem to be reasoned can be determined by the i-th processing layer, the i-th processing layer outputs the result of the problem to be reasoned. If the result of the problem to be reasoned cannot be determined by the i-th processing layer, the (i+1)-th processing layer calculates the intermediate result of the position to be calculated in the (i+1)-th processing layer, and determines whether the result of the problem to be reasoned can be determined based on the intermediate result calculated by the (i+1)-th processing layer.

[0128] For example, based on the category characteristics of the at least one result, the i-th processing layer determines whether there is an intermediate result in the intermediate results of the position to be calculated in the i-th processing layer that corresponds to at least one result of the text template. If there is an intermediate result that corresponds to the text template, the i-th processing layer can determine the result of the question to be reasoned; if there is no intermediate result that corresponds to the text template, the i-th processing layer cannot determine the result of the question to be reasoned.

[0129] In existing technologies, each processing layer needs to calculate all intermediate results of the current processing layer based on the output of the previous processing layer and the input of the current layer, and then determine the result of the reasoning problem based on the intermediate results of the last processing layer. In the embodiments of this application, each processing layer calculates the intermediate results of the position to be calculated in the processing layer based on the intermediate template, without calculating all intermediate results of the processing layer. Based on the intermediate results and the aforementioned category features, it is determined whether the result of the reasoning problem can be determined based on the intermediate results. If the result of the reasoning problem can be determined based on the intermediate results, the result of the reasoning problem can be obtained directly from that processing layer. This differs from existing technologies, where only the last processing layer can obtain the result of the reasoning problem, thus enabling skipping layers to obtain the result of the reasoning problem and accelerating the reasoning process.

[0130] The reasoning model needs to go through one full inference and multiple incremental inferences to reason about the result of a problem to be reasoned. For example, the time for one full inference is Q, the time for one incremental inference is S, the result of the problem to be reasoned is G words, and the time for the reasoning model to obtain the result is T = Q + S × G.

[0131] The time required for full inference depends on the length of the problem to be inferred. The longer the problem is, the longer it takes for the inference model to perform a full inference. The shorter the problem is, the shorter it takes for the inference model to perform a full inference.

[0132] In this embodiment, a problem to be inferred can be obtained by filling in a text template with the part to be filled in. If the text template contains predetermined intermediate results, the inference model only needs to calculate the intermediate results of the part to be filled in to perform a full inference. This differs from related technologies, where a full inference requires calculating all intermediate results, and the time for the inference model to perform a full inference is related to the length of the entire problem. The method in this embodiment makes the time for the inference model to perform a full inference positively correlated with the length of the part to be filled in, effectively reducing the time spent calculating intermediate results and accelerating the processing efficiency of the inference problem.

[0133] Furthermore, the time it takes for the inference model to obtain a result is related to the number of words in the result generated by the inference model. The more words in the result, the longer it takes for the inference model to obtain a result, and the fewer words in the result, the shorter it takes for the inference model to obtain a result.

[0134] In this embodiment, the reasoning problem can be designed as, for example, "Please answer yes / no", "Please output the most likely category A / B / C / D", or "Determine whether the user will click on this promotion". The terminal only needs to output a single word (e.g., yes, no, A, etc.) to achieve the desired answer effect, reducing the number of words in the result of the reasoning problem. This allows the terminal to perform a full reasoning based on the reasoning problem without needing to perform incremental reasoning to obtain the result of the reasoning problem, further reducing the time to obtain the result and speeding up the processing speed of the reasoning problem.

[0135] Figure 2 is a flowchart illustrating another exemplary method for processing reasoning problems, applied in the cloud. The cloud can be a server cluster, which may include one or more cloud servers. Referring to Figure 2, the method flow may include, but is not limited to, the following steps:

[0136] S201. The cloud retrieves the intermediate results of each of the multiple test questions, and the text templates corresponding to these multiple test questions are the same.

[0137] In an optional embodiment, an inference model is deployed in the cloud. For each of the multiple test questions, the cloud can input the test question into the inference model to obtain an intermediate result for that test question. The inference model can be an LLM (Liquid Dynamics Model), and the intermediate result can be the K-vector and V-vector generated during the LLM inference process of the test question's result.

[0138] In an optional embodiment, the intermediate results can be K vectors and V vectors. Given a fixed number of Transformer layers in the LLM, fixed LLM internal parameters, and fixed test problem length, the number of K vectors and V vectors are also fixed. As shown in Figure 3, the LLM includes M Transformer layers (the number can be set according to actual needs, and this application does not limit it), and each Transformer layer includes N K vectors and N V vectors. The specific value of N is directly proportional to the length of the test problem; that is, the longer the test problem, the larger the value of N; the shorter the test problem, the smaller the value of N. Each K or V vector can be a numerical value or a tensor, and this embodiment does not limit this.

[0139] The cloud inputs the test question into the LLM to obtain the contents of the N K vectors and N V vectors determined by each Transformer layer during the LLM's inference of the test question.

[0140] In specific embodiments, the cloud also includes a Neural Processing Unit (NPU) or a Graphics Processing Unit (GPU), which provides efficient parallel computing capabilities and rich interfaces. The cloud runs LLM on the NPU or GPU to obtain the generated K and V vectors during the process of obtaining the results of the LLM inference test problem through the interfaces provided by the NPU or GPU.

[0141] In an optional embodiment, the cloud fills in different parts to be filled in the same text template to obtain multiple test questions corresponding to the same text template.

[0142] In an optional embodiment, if the user agrees to upload terminal data to the cloud, the cloud obtains the test questions uploaded by the terminal. Furthermore, the cloud uses the intermediate results of the obtained test questions to determine the intermediate template of the corresponding text template, and then sends the obtained test question results to the terminal.

[0143] S202. Based on the intermediate results of the multiple test questions, the cloud determines the intermediate template corresponding to the text template.

[0144] Based on the intermediate results of multiple test questions, the cloud determines the location information of the first intermediate result and the second intermediate result of the intermediate template. Based on the second intermediate result, the cloud determines the correspondence between the second intermediate result and at least one result of the text template.

[0145] The first intermediate result is an intermediate result that does not need to be calculated when the inference model infers the problem to be inferred based on the intermediate template. The second intermediate result is an intermediate result that needs to be calculated when the inference model infers the problem to be inferred based on the intermediate template. The text templates corresponding to the problem to be inferred and the test problem are the same.

[0146] In the case where the intermediate results are K vectors and V vectors, the intermediate template can also be called a KV (Key-value) template or a KVCache template.

[0147] In an optional embodiment, each test question corresponds to an intermediate result, and each intermediate result includes multiple sub-results. For each sub-result at a given position among these multiple sub-results: the cloud compares the sub-results at that position for the multiple test questions to determine the position information of the first intermediate result and the second intermediate result.

[0148] In some embodiments, the cloud determines the maximum and minimum sub-results, and the difference between the maximum and minimum sub-results is determined as the difference at that position. If the difference at that position is less than a preset difference, the cloud determines a first intermediate result at that position based on multiple sub-results at that position. If the difference at that position is greater than or equal to the preset difference, the cloud determines multiple sub-results at that position as second intermediate results at that position, and determines the information at that position as the position information of a second intermediate result.

[0149] In other embodiments, after the cloud determines the differences between multiple sub-results at a location, the cloud determines the number of differences within a certain range. If the number of differences is less than or equal to a preset number, the cloud determines a first intermediate result for that location based on the multiple sub-results. If the number of differences is greater than the preset number, the cloud determines all the multiple sub-results at that location as second intermediate results for that location, and determines the location information of that location as location information of a second intermediate result. Here, "within a preset range" means greater than or equal to the minimum value of the preset range and less than or equal to the maximum value of the preset range. "Outside a preset range" means greater than the maximum value of the preset range or less than the minimum value of the preset range.

[0150] In one embodiment, the cloud determines that all intermediate results are second results, indicating that the intermediate template cannot represent the intermediate results of the text template. That is, when the inference model infers the reasoning question corresponding to the text template based on the intermediate template, it needs to calculate all intermediate results. In another embodiment, the cloud determines that all intermediate results are first results, indicating that the intermediate template can completely represent the intermediate results of the text template. That is, when the inference model infers the reasoning question corresponding to the text template based on the intermediate template, it does not need to calculate intermediate results.

[0151] In one embodiment, the cloud determines the average of multiple sub-results at that location as the first intermediate result for that location. In another embodiment, the cloud performs a weighted average of multiple sub-results at that location and determines the weighted average as the first intermediate result for that location.

[0152] For example, multiple test questions are test question 1, test question 2, ..., test question c. The multiple sub-results corresponding to test question 1 are sub-result 11, sub-result 12, ..., sub-result 1z. The multiple sub-results corresponding to test question 2 are sub-result 21, sub-result 22, ..., sub-result 2z, and so on. The multiple sub-results corresponding to test question c are sub-result c1, sub-result c2, ..., sub-result cz, where z is a positive integer.

[0153] For position z: The cloud compares sub-result 1z, sub-result 22, ..., sub-result cz to determine the position information of the first or second intermediate result of position z, as well as the second intermediate result.

[0154] In one example, the largest sub-result is sub-result 6z, and the smallest sub-result is sub-result 1z. The cloud determines the difference between sub-result 6z and sub-result 1z as the difference at position z. If the difference at position z is less than a preset difference, the cloud determines the first intermediate result at position z based on the multiple sub-results at position z. If the difference at position z is greater than or equal to the preset difference, the cloud determines sub-result 1z, sub-result 22, ..., sub-result cz as the second intermediate result at position z, and defines the information at position z as the position information of a second intermediate result.

[0155] In an optional embodiment, for each location information of a second intermediate result, the cloud determines whether there is a correspondence between the content of the location indicated by the location information and at least one result of the text template.

[0156] In a specific embodiment, the text template includes two results: a first result and a second result. If the cloud determines the result of the text template to be the first result, the range of multiple intermediate second results at position w belongs to the first range. If the cloud determines the result of the text template to be the second result, the range of multiple intermediate second results at position w belongs to the second range. If the first and second ranges do not overlap, the cloud determines that there is a correspondence between the content at position w and at least one result of the text template. The cloud determines that the first result corresponds to the second result at position w that belongs to the first range, and the second result corresponds to the second result at position w that belongs to the second range. If the first and second ranges overlap, the cloud determines that there is no correspondence between the content at position w and at least one result of the text template. Here, "the first and second ranges do not overlap" means that the minimum value of the first range is greater than the maximum value of the second range, or the minimum value of the second range is greater than the maximum value of the first range.

[0157] In an optional embodiment, as shown in Figure 4, the white square represents the first intermediate result, and the black square represents the second intermediate result; that is, the black square represents the position to be calculated. The numerical distribution of the K vector for the position to be calculated in the second Transformer layer is shown in Figure 5. When the text template result is the first result, the value of the K vector belongs to the range Y; when the text template result is the second result, the value of the K vector belongs to the range N. In other words, if the value of the K vector belongs to the range Y, the result of the test question corresponding to the text template can be determined to be the first result; if the value of the K vector belongs to the range N, the result of the test question corresponding to the text template can be determined to be the second result.

[0158] S203. The cloud sends the intermediate template to the terminal.

[0159] In some embodiments, the terminal sends a request to the cloud to obtain an intermediate template, and the cloud may respond to the request by sending the intermediate template to the terminal. After receiving the intermediate template, the terminal can execute the processes shown in Figure 1.

[0160] In a specific embodiment, the terminal sends a fourth message including the acquisition request to the cloud, and after receiving the fourth message, the cloud sends a fifth message including the intermediate template to the terminal.

[0161] In other embodiments, after the cloud determines the intermediate template corresponding to the text template, the cloud sends the cloud intermediate template to the terminal at preset intervals.

[0162] In a specific embodiment, the cloud sends a sixth message, including an intermediate template, to the terminal at preset intervals.

[0163] It should be noted that the fourth message and the first message mentioned above, the fifth message and the second message mentioned above, and the sixth message and the third message mentioned above may be the same message or may not be the same message. This application embodiment does not impose any restrictions on this.

[0164] In an optional embodiment, the cloud sends the intermediate template and the corresponding text template to the terminal together. The terminal stores the intermediate template and the corresponding text template in a one-to-one correspondence relationship, forming the above-mentioned mapping relationship 2.

[0165] In this embodiment, the cloud determines the intermediate template corresponding to the text template offline based on the intermediate results of multiple test questions corresponding to the same text template, and sends the intermediate template determined offline by the cloud to the terminal. The cloud assists the terminal in obtaining the intermediate template, avoiding the terminal's own conversion of the text template into an intermediate template, thus speeding up the acquisition of the intermediate template. After obtaining the intermediate template sent by the cloud, the terminal can store it so that it can subsequently use it to reason about the question to be reasoned, thereby accelerating the processing speed of the reasoning problem.

[0166] The cloud and the terminal collaborate with each other. The cloud determines the intermediate template offline, and the terminal uses the intermediate template determined by the cloud to reason about the problem to be reasoned. This avoids uploading problems containing user privacy information to the cloud, thus protecting user privacy and security.

[0167] In addition, the terminal can obtain the intermediate template determined offline by the cloud before triggering the reasoning problem, avoiding data transmission between the terminal and the cloud during the reasoning process of the problem, so that the time for the user to obtain the result does not include the end-to-end latency, thus reducing the time for the user to obtain the result.

[0168] Corresponding to the above-mentioned method for processing reasoning problems, this application also provides a processing device for reasoning problems. This processing device is deployed on a terminal. Figure 6 is a schematic diagram of the structure of an exemplary processing device 800 for reasoning problems. Referring to Figure 6, the processing device 800 includes:

[0169] The acquisition module 601 is used to acquire the question to be reasoned; the first determination module 602 is used to determine the result of the question to be reasoned based on the intermediate template and the reasoning model, wherein the intermediate template is used to provide some intermediate results in the process of the reasoning model reasoning the question to be reasoned, and the some intermediate results are obtained based on the reasoning of a test question, and the test question is the same as the text template corresponding to the question to be reasoned.

[0170] In one possible implementation, the intermediate template is further used to provide location information of the intermediate result to be calculated, wherein the intermediate result to be calculated is an intermediate result that the intermediate template did not provide during the process of the inference model reasoning the problem to be reasoned based on the intermediate template.

[0171] In one possible implementation, the intermediate template is further configured to provide a category feature of at least one result of the text template, the category feature being used to indicate the correspondence between the intermediate result to be calculated and at least one result of the text template.

[0172] In one possible implementation, the inference model includes multiple processing layers, which are used to determine intermediate results not provided by the intermediate template during the inference process of the inference model inferring the inference problem, and the processor layers are also used to determine whether the result of the problem to be inferred can be determined based on the unprovided intermediate results.

[0173] In one possible implementation, the device further includes: a receiving module 603, for receiving a plurality of intermediate templates sent from the cloud; and a second determining module 604, for determining whether there is an intermediate template among the plurality of intermediate templates that corresponds to the problem to be reasoned.

[0174] In one possible implementation, the second determining module 604 is specifically used to: determine the text template; and determine whether there exists an intermediate template that matches the problem to be reasoned, based on the text template and a first mapping relationship, wherein the first mapping relationship includes a mapping relationship between the text template and the intermediate template.

[0175] In one possible implementation, the inference model is a large language model, and the processing layer is a self-attention conversion layer.

[0176] The effect of the above-described reasoning problem processing device is similar to that of the reasoning problem processing method executed by the terminal in the above embodiments, and will not be described again here.

[0177] It should be noted that the product forms provided in this application embodiment include, but are not limited to: terminals that store the above-mentioned intermediate template and / or deploy the above-mentioned inference model.

[0178] Without obtaining the intermediate template sent from the cloud, the difference between the time S1 for a terminal that stores the intermediate template to perform a full inference and the time S2 for a terminal that does not store the intermediate template to perform a full inference is greater than a preset time difference. Furthermore, when obtaining the question to be inferred based on a text template, the time for a terminal that stores the intermediate template to perform a full inference is only related to the length of the part to be filled, while the time for a terminal that does not store the intermediate template to perform a full inference is related to the length of the question to be inferred.

[0179] Furthermore, the aforementioned products can be charged on a per-request basis. For example, application A sends a text template to the cloud, requesting the cloud to generate an intermediate template corresponding to the text template. The cloud stores the generated intermediate template in its network storage system and generates an access path. The cloud then distributes the access path of the intermediate template to all terminals, which download and store the intermediate template. The cloud can record the number of times a terminal requests to obtain the intermediate template and generate billing information based on this number.

[0180] It should be noted that multiple new test questions are provided to the cloud, causing the cloud to generate new intermediate templates. If there is content on the order of the intermediate template download on the terminal (e.g., if the inference model is a megabyte-scale LLM, this content is tens of megabytes in size), and the first character length of the result of the inference question decreases after downloading, it can be determined that this content is an intermediate template generated by the cloud, where the text template corresponding to the question to be processed is the same as that corresponding to the test question.

[0181] It should be noted that providing cache paths to the terminal through network storage means using network storage systems (such as cloud storage, distributed file systems, etc.) to store caches and providing the access paths (or addresses) of these caches to the terminal.

[0182] If the cache is used in every processing layer during the reasoning process of the inference model (for example, the number of accesses to the cache is the same as the number of processing layers), then the cache can be identified as the aforementioned intermediate template.

[0183] Corresponding to the above-mentioned method for processing reasoning problems, this application also provides another device for processing reasoning problems, which is deployed in the cloud. Figure 7 is a schematic diagram of the structure of an exemplary device 700 for processing reasoning problems. Referring to Figure 7, the processing device 700 includes:

[0184] The acquisition module 701 is used to acquire intermediate results for each of the multiple test questions, wherein the text templates corresponding to the multiple test questions are the same; the determination module 702 is used to determine the intermediate template corresponding to the text template based on the intermediate results of the multiple test questions; and the sending module is used to send the intermediate template to the terminal.

[0185] In one possible implementation, the determining module 702 is specifically configured to: determine the position information of a first intermediate result and a second intermediate result of the intermediate template based on the intermediate results of the plurality of test questions; wherein, the first intermediate result is an intermediate result that does not need to be calculated when the inference model infers the question to be inferred based on the intermediate template, and the second intermediate result is an intermediate result that needs to be calculated when the inference model infers the question to be inferred based on the intermediate template, and the text template corresponding to the question to be inferred is the same as that of the test questions; and determine the correspondence between the second intermediate result and at least one result of the text template based on the second intermediate result.

[0186] The effect of the above-mentioned reasoning problem processing device is similar to that of the reasoning problem processing method executed in the cloud in the above embodiments, and will not be described again here.

[0187] Corresponding to the above-mentioned method for processing reasoning problems, this application also provides a system for processing reasoning problems. Figure 8 is a schematic diagram of the structure of a system 800 for processing reasoning problems, as exemplarily shown. Referring to Figure 8, the processing device 800 includes a terminal 810 and a cloud 820. The terminal 810 includes the above-mentioned processing device 600 for processing reasoning problems; the cloud 820 includes the above-mentioned processing device 700 for processing reasoning problems.

[0188] The effect of the above-described reasoning problem processing system is similar to that of the reasoning problem processing methods in the above embodiments, and will not be repeated here.

[0189] All of the above modules can be implemented in software or hardware. As an example of a software functional unit, a module may include code running on a computing instance. A computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Furthermore, there may be one or more computing instances. For example, module 601 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0190] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0191] As an example of a hardware functional unit, the module mentioned above may include at least one computing device, such as a server. Alternatively, the module may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0192] The multiple computing devices included in the aforementioned terminal 810 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the aforementioned terminal 810 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the aforementioned terminal 810 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0193] It should be noted that, in other embodiments, the above modules can be used to execute the corresponding steps in the reasoning problem processing method to realize all the functions of the reasoning problem processing system 800.

[0194] As an example of a hardware functional unit, the module may also include at least one storage device, such as a server. Alternatively, the module may be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0195] The multiple storage devices included in the aforementioned cloud 820 can be distributed within the same region or in different regions. Similarly, the multiple storage devices included in the aforementioned cloud 820 can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the multiple storage devices included in the aforementioned cloud 820 can be distributed within the same VPC or in multiple VPCs. These multiple storage devices can be any combination of storage devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0196] It should be noted that, in other embodiments, the above modules can be used to execute the corresponding steps in the reasoning problem processing method to realize all the functions of the reasoning problem processing system 800.

[0197] This application also provides a computing device 900. As shown in FIG9, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 909. The processor 904, the memory 906, and the communication interface 909 communicate with each other via the bus 902. The computing device 900 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 900.

[0198] Bus 902 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 9, but this does not imply that there is only one bus or one type of bus. Bus 902 can include pathways for transmitting information between various components of computing device 900 (e.g., memory 906, processor 904, communication interface 909).

[0199] Processor 904 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0200] Memory 906 may include volatile memory, such as random access memory (RAM). Memory 906 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0201] The memory 906 stores executable program code, which the processor 904 executes to implement the functions of the aforementioned acquisition module 601, first determination module 602, receiving module 603, and second determination module 604, thereby realizing the reasoning problem processing method. In other words, the memory 906 stores instructions for executing the reasoning problem processing method.

[0202] The communication interface 909 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 900 and other devices or communication networks.

[0203] This application also provides a storage device 1000. As shown in FIG10, the storage device 1000 includes: a bus 1002, a processor 1004, a memory 1006, and a communication interface 1009. The processor 1004, the memory 1006, and the communication interface 1009 communicate with each other via the bus 1002. The storage device 1000 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the storage device 1000.

[0204] Bus 1002 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 10, but this does not imply that there is only one bus or one type of bus. Bus 1002 can include pathways for transmitting information between various components of storage device 1000 (e.g., memory 1006, processor 1004, communication interface 1009).

[0205] The processor 1004 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0206] The memory 1006 may include volatile memory, such as random access memory (RAM). The memory 1006 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0207] The memory 1006 stores executable program code, and the processor 1004 executes the executable program code to implement the functions of the aforementioned acquisition module 701 and determination module 702, thereby realizing the reasoning problem processing method. That is, the memory 1006 stores instructions for executing the reasoning problem processing method.

[0208] The communication interface 1009 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the storage device 1000 and other devices or communication networks.

[0209] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0210] As shown in Figure 11, the computing device cluster includes at least one computing device 1100. The memory 1106 of one or more computing devices 1100 in the computing device cluster may store the same instructions for performing processing methods for reasoning problems.

[0211] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the processing method for the reasoning problem. In other words, a combination of one or more computing devices 1100 can jointly execute the instructions for executing the processing method for the reasoning problem.

[0212] It should be noted that the memory 1106 in different computing devices 1100 within the computing device cluster can store different instructions, which are used to execute parts of the functions of the reasoning problem processing device 600. That is, the instructions stored in the memory 1106 of different computing devices 1100 can implement the functions of one or more modules among the acquisition module 601, the first determination module 602, the receiving module 603, and the second determination module 604.

[0213] This application also provides a storage device cluster. The storage device cluster includes at least one storage device. The storage device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the storage device can also be a terminal device such as a desktop computer, laptop computer, or smartphone.

[0214] As shown in Figure 12, the storage device cluster includes at least one storage device 1200. The memory 1206 in one or more storage devices 1200 within the storage device cluster may store the same instructions for performing processing methods for reasoning problems.

[0215] In some possible implementations, the memory 1206 of one or more storage devices 1200 in the storage device cluster may also store partial instructions for executing the processing method for the reasoning problem. In other words, a combination of one or more storage devices 1200 can jointly execute the instructions for executing the processing method for the reasoning problem.

[0216] It should be noted that the memory 1206 in different storage devices 1200 within the storage device cluster can store different instructions, which are used to execute parts of the functions of the reasoning problem processing device 700. That is, the instructions stored in the memory 1206 in different storage devices 1200 can implement the functions of one or more modules in the acquisition module 701 and the determination module 702.

[0217] In some possible implementations, the computing devices and storage devices in the reasoning problem processing system can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 13 illustrates one possible implementation. As shown in Figure 13, computing device 1300A and storage device 1300B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1306 in computing device 1300A stores instructions for executing the functions of the acquisition module 601, the first determination module 602, the receiving module 603, and the second determination module 604. Simultaneously, the memory 1306 in storage device 1300B stores instructions for executing the functions of the acquisition module 701 and the determination module 702.

[0218] It should be understood that the functions of computing device 1300A shown in Figure 13 can also be performed by multiple computing devices 1300. Similarly, the functions of storage device 1300B can also be performed by multiple storage devices 1300.

[0219] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the reasoning problem processing method described in the above embodiments.

[0220] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the reasoning problem processing method described in the above embodiments.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A method for processing reasoning problems, characterized in that, Applied to a terminal, the method includes: Obtain the problem to be reasoned; Based on the intermediate template and the inference model, the result of the problem to be inferred is determined. The intermediate template is used to provide some intermediate results in the process of the inference model inferring the problem to be inferred. The partial intermediate results are obtained based on the inference of a test question, which is the same as the text template corresponding to the problem to be inferred.

2. The method according to claim 1, characterized in that, The intermediate template is also used to provide location information of intermediate results to be calculated, wherein the intermediate results to be calculated are intermediate results that are not provided by the intermediate template during the reasoning process of the inference model based on the intermediate template to reason about the problem to be reasoned.

3. The method according to claim 2, characterized in that, The intermediate template is also used to provide a category feature of at least one result of the text template, the category feature being used to indicate the correspondence between the intermediate result to be calculated and at least one result of the text template.

4. The method according to claim 1, characterized in that, The reasoning model includes multiple processing layers. The processing layers are used to determine intermediate results that are not provided by the intermediate template during the reasoning process of the reasoning model for the problem to be reasoned. The processing layers are also used to determine whether the result of the problem to be reasoned can be determined based on the unprovided intermediate results.

5. The method according to claim 1, characterized in that, Before obtaining the problem to be reasoned, the method further includes: Receive multiple intermediate templates sent from the cloud; Before determining the result of the problem to be reasoned based on the intermediate template and the reasoning model, the method further includes: Determine whether there exists an intermediate template among the plurality of intermediate templates that corresponds to the problem to be reasoned.

6. The method according to claim 5, characterized in that, Determining whether there exists an intermediate template among the plurality of intermediate templates that corresponds to the problem to be reasoned includes: Determine the text template; Based on the text template and the first mapping relationship, it is determined whether there exists an intermediate template that matches the problem to be reasoned. The first mapping relationship includes the mapping relationship between the text template and the intermediate template.

7. The method according to any one of claims 1 to 6, characterized in that, The reasoning model is a large language model, and the processing layer is a self-attention conversion layer.

8. A method for processing reasoning problems, characterized in that, Applied to the cloud, the method includes: Based on multiple test questions, obtain the intermediate result of each of the multiple test questions, where the text templates corresponding to the multiple test questions are the same; Based on the intermediate results of the multiple test questions, the intermediate template corresponding to the text template is determined; The intermediate template is sent to the terminal.

9. The method according to claim 8, characterized in that, The process of determining the intermediate template corresponding to the text template based on the intermediate results of the multiple test questions includes: Based on the intermediate results of the multiple test questions, the position information of the first intermediate result and the second intermediate result of the intermediate template are determined. The first intermediate result is an intermediate result that does not need to be calculated when the inference model infers the question to be inferred based on the intermediate template. The second intermediate result is an intermediate result that needs to be calculated when the inference model infers the question to be inferred based on the intermediate template. The text template corresponding to the question to be inferred is the same as that of the test question. Based on the second intermediate result, a correspondence is determined between the second intermediate result and at least one result of the text template.

10. A device for processing reasoning problems, characterized in that, Applied to a terminal, the device includes: The acquisition module is used to acquire the problem to be reasoned. The first determining module is used to determine the result of the problem to be reasoned based on an intermediate template and a reasoning model. The intermediate template is used to provide some intermediate results in the process of the reasoning model reasoning the problem to be reasoned. The partial intermediate results are obtained based on a test question, and the test question is the same as the text template corresponding to the problem to be reasoned.

11. The apparatus according to claim 10, characterized in that, The intermediate template is also used to provide location information of intermediate results to be calculated, wherein the intermediate results to be calculated are intermediate results that are not provided by the intermediate template during the reasoning process of the inference model based on the intermediate template to reason about the problem to be reasoned.

12. The apparatus according to claim 11, characterized in that, The intermediate template is also used to provide a category feature of at least one result of the text template, the category feature being used to indicate the correspondence between the intermediate result to be calculated and at least one result of the text template.

13. The apparatus according to claim 10, characterized in that, The reasoning model includes multiple processing layers. The processing layers are used to determine intermediate results that are not provided by the intermediate template during the reasoning process of the reasoning model for the problem to be reasoned. The processing layers are also used to determine whether the result of the problem to be reasoned can be determined based on the unprovided intermediate results.

14. The apparatus according to claim 10, characterized in that, The device further includes: The receiving module is used to receive multiple intermediate templates sent from the cloud. The second determining module is used to determine whether there is an intermediate template among the plurality of intermediate templates that corresponds to the problem to be reasoned.

15. The apparatus according to claim 14, characterized in that, The second determining module is specifically used for: Determine the text template; Based on the text template and the first mapping relationship, it is determined whether there exists an intermediate template that matches the problem to be reasoned. The first mapping relationship includes the mapping relationship between the text template and the intermediate template.

16. The apparatus according to any one of claims 10 to 15, characterized in that, The reasoning model is a large language model, and the processing layer is a self-attention conversion layer.

17. A device for processing reasoning problems, characterized in that, The device, applied in the cloud, includes: The acquisition module is used to acquire intermediate results for each of the multiple test questions, where the text templates corresponding to the multiple test questions are the same. The determination module is used to determine the intermediate template corresponding to the text template based on the intermediate results of the multiple test questions; The sending module is used to send the intermediate template to the terminal.

18. The apparatus according to claim 17, characterized in that, The determining module is specifically used for: Based on the intermediate results of the multiple test questions, the position information of the first intermediate result and the second intermediate result of the intermediate template are determined. The first intermediate result is an intermediate result that does not need to be calculated when the inference model infers the question to be inferred based on the intermediate template. The second intermediate result is an intermediate result that needs to be calculated when the inference model infers the question to be inferred based on the intermediate template. The text template corresponding to the question to be inferred is the same as that of the test question. Based on the second intermediate result, a correspondence is determined between the second intermediate result and at least one result of the text template.

19. A system for processing reasoning problems, characterized in that, Including terminals and the cloud, The terminal includes a reasoning problem processing device as described in any one of claims 10 to 16; The cloud includes a processing apparatus for reasoning problems as described in claim 17 or 18.

20. A cluster of devices, characterized in that, It includes at least one device, each device including a processor and memory; The processor of the at least one device is configured to execute instructions stored in the memory of the at least one device to cause the cluster of devices to perform the method as described in any one of claims 1 to 9.

21. A computer program product containing instructions, characterized in that, When the instruction is executed by the device cluster, the device cluster causes the device cluster to perform the method as described in any one of claims 1 to 9.

22. A computer-readable storage medium, characterized in that, Includes computer program instructions, which, when executed by a cluster of devices, perform the method as described in any one of claims 1 to 9.

Images