A question and answer method, apparatus and computing device cluster

Abstract

A question and answer method, comprising: obtaining a first query input by a user; sending the first query and a predicted reasoning scale related to the first query to a slow thinking model, wherein the slow thinking model is used to reason the first query with the predicted reasoning scale as a constraint to obtain a reply, and the predicted reasoning scale related to the first query is obtained based on a budget evaluation model performing budget evaluation on the first query. By taking the predicted reasoning scale as an upper limit, the slow thinking model can prioritize the reasoning path with high information density within the limited budget, skip redundant steps, and reduce repeated reasoning.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a question-answering method, apparatus, and cluster of computing devices. Background Technology

[0002] With the continuous advancement of artificial intelligence technology, slow-thinking models, due to their multi-step reasoning mechanism, have significantly surpassed traditional neural network models in terms of reasoning accuracy, and have thus gradually attracted widespread attention.

[0003] However, current slow-thinking models may repeatedly perform the same reasoning steps or calculate information that has already been processed during the reasoning process. This repetitive thinking phenomenon leads to low reasoning efficiency and high computational resource consumption when processing a single query. How to effectively improve the reasoning efficiency of slow-thinking models and reduce their reasoning overhead has become an urgent technical challenge. Summary of the Invention

[0004] This application provides a question-answering method, apparatus, computing device cluster, computer storage medium, and computer product that can improve the reasoning efficiency of slow-thinking models and avoid repetitive thinking.

[0005] In a first aspect, this application provides a question-answering method, which includes: obtaining a first query input by a user; sending the first query and the predictive inference scale associated with the first query to a slow-thinking model, wherein the slow-thinking model is used to infer the first query with the predictive inference scale as a constraint to obtain a response, and the predictive inference scale associated with the first query is obtained by budgeting the first query based on a budget evaluation model.

[0006] For example, slow thinking involves multiple steps of reasoning when forming a recovery. The constraint of predicted inference scale means that the inference scale corresponding to the inference content obtained by the slow thinking model when reasoning about the first query does not exceed the predicted inference scale. Inference scale is used to characterize the quantitative features in the inference content generated by the slow thinking model. Inference scale can be characterized using various granularities. Typically, the predicted inference related to the first query will be smaller than the inference scale of the inference content generated by the slow thinking model under natural reasoning conditions.

[0007] Thus, by evaluating the first query using a budget assessment model, a relatively reasonable predictive inference scale for the slow-thinking model is obtained while ensuring accurate responses. This predictive inference scale can reasonably control the inference scale of the slow-thinking model when reasoning about the first query, thereby reducing resource overhead and computational consumption during the inference process. Furthermore, since the inference content is effectively compressed within the predictive inference scale, the occurrence of redundant thinking can be significantly reduced.

[0008] From another perspective, using the scale of predictive inference as an upper limit on inference allows slow-thinking models to complete inference within a limited budget. This is equivalent to setting boundaries on the depth and breadth of the model's thought processes. Under this constraint, slow-thinking models may tend to choose more information-dense inference paths, skipping or compressing repetitive steps to prioritize the output of high-value inference content. Therefore, using the scale of predictive inference as a constraint not only limits the amount of content generated by the slow-thinking model but also prompts the model to strategically avoid inefficient and redundant thinking, thereby significantly reducing the occurrence of repetitive inference.

[0009] In one possible implementation, the method further includes: based on the similarity between the first query and candidate queries in the database, filtering at least one first candidate query and a first candidate inference scale corresponding to the at least one first candidate query from the database, wherein the database contains: at least one candidate query and a candidate inference scale corresponding to the at least one candidate query, and the candidate inference scale refers to an inference scale smaller than the inference scale associated with the candidate query under natural inference; inputting the first query, at least one first candidate query, and the first candidate inference scale corresponding to the at least one first candidate query into a trained budget evaluation model, so that the budget evaluation model obtains the predicted inference scale of the first query with reference to the first candidate query and the first candidate inference scale.

[0010] For example, candidate queries are pre-defined queries, and the candidate inference size corresponding to a candidate query is an optimized inference size reference value. The optimization of the candidate inference size is relative to the inference size of the slow-thinking model during natural inference. In the database, one candidate query corresponds to one candidate inference size. In this embodiment, the candidate query selected from the data is called the first candidate query, and the selected candidate inference size is called the first candidate inference size. At least one first candidate query and the first candidate inference size corresponding to at least one first candidate query can be simply referred to as the first data.

[0011] In this way, on the one hand, the primary data can enhance the generalization ability of the budget assessment model. The examples most similar to the query allow the budget assessment model to refer to the primary data when performing budget assessments on the query, making the predicted inference scale more accurate. On the other hand, as the database is continuously updated, budget samples not covered by the budget assessment model during the training phase can be continuously accumulated in the database, thereby improving the adaptability of the budget assessment model in real-world scenarios.

[0012] In one possible implementation, the budget evaluation model is trained based on budget samples. The budget samples are generated through the following steps: obtaining candidate queries, a first inference scale, and a first response, wherein the first inference scale is used to characterize the inference scale of the slow-thinking model when performing natural inference on the candidate queries, and the first response is the inference result in the natural inference process; based on the slow-thinking model, the test inference scales that meet the first condition are filtered to obtain candidate inference scales, wherein at least one test inference scale is obtained by reducing the first inference scale, and the first condition includes: the test inference scale is smaller than the first inference scale, and the test response corresponding to the test inference scale is consistent with the first response, wherein the test response corresponding to the test inference scale refers to the inference result obtained by the slow-thinking model when inferring the candidate queries with the test inference scale as a constraint.

[0013] For example, being constrained by the test inference size means that when the slow-thinking model performs inference, the inference size corresponding to the generated inference content will not exceed the test inference size. For example, the test inference size can be a numerical value or a range. Consistency between the test response and the first response means that they are substantially the same. For example, when both the test response and the first response are numerical, consistency can be determined directly through numerical comparison. For non-numerical responses, neural network models or other text similarity calculation methods can be used to determine consistency.

[0014] In this way, candidate queries and their corresponding candidate inference sizes are used as the budget sample. In this sample, the candidate inference sizes are smaller than the inference sizes associated with candidate queries under natural inference. Therefore, training the budget evaluation model using the budget sample allows it to learn to predict more compact and resource-efficient inference sizes for different queries while ensuring response accuracy, thereby improving overall inference efficiency.

[0015] In one possible implementation, based on the slow-thinking model, the test inference scales that meet the first condition are filtered to obtain candidate inference scales. This includes: based on the slow-thinking model, multiple rounds of inference are performed on candidate queries with different test inference scales as constraints, so that a test response corresponding to the test inference scale of the current round is obtained in each round of inference, wherein the test inference scale of the current round is adjusted based on the first inference scale or the test inference scale of the previous round; and the test inference scales that meet the first condition are selected according to preset rules to obtain candidate inference scales.

[0016] For example, preset rules are rules set in advance. These preset rules can be implemented using one or more methods, including the number of successful inference rounds, the consistency between the test response corresponding to the test inference size and the first response, and whether the test inference content of the current round meets the constraints of the test inference size of the current round. For instance, it can be preset that after successfully running n rounds of inference, the test inference size of the current round is used as a candidate inference size; if the number of successful rounds is less than n, the next round of inference continues. Here, n is a positive integer. Successfully running n rounds of inference means that the test inference content of the current round meets the constraints of the test inference size of the current round, and the test response of the current round is consistent with the first response; otherwise, it is considered a failure.

[0017] In this way, by using multiple rounds of reasoning and dynamically adjusting the test inference size, we can explore the convergence process of the inference size more precisely while ensuring response consistency, thereby improving the accuracy and compactness of the candidate inference size.

[0018] In one possible implementation, the steps further include: in the process of adjusting the test inference scale of the previous round to obtain the test inference scale of the current round, if the test inference scale of the previous round satisfies the first condition, then the test inference scale of the previous round is reduced to obtain the test inference scale of the current round; in the process of adjusting the first inference scale to obtain the test inference scale of the current round, the first inference scale is reduced to obtain the test inference scale of the current round.

[0019] In this way, by further compressing the inference scale while satisfying the first condition, we can more effectively approach the inference boundary and improve the accuracy and tightness of inference resource control.

[0020] In one possible implementation, the test inference scale that satisfies the first condition is selected according to a preset rule to obtain a candidate inference scale, including: when it is determined that the inference has ended and the test inference scale of the current round does not satisfy the first condition, the test inference scale of the previous round is used as a candidate inference scale; when it is determined that the inference has ended and the test inference scale of the current round satisfies the first condition, the test inference scale of the current round is used as a candidate inference scale.

[0021] In this way, by flexibly selecting an appropriate test inference size at the end of the inference process, we can ensure that if the first condition is not met, we can revert to the previous inference size, thereby avoiding excessive reduction of the inference size and ensuring a balance between response accuracy and inference efficiency.

[0022] In one possible implementation, the first condition also includes: the test inference content of the current round satisfies the constraint of the test inference size of the current round.

[0023] In this way, if the test reasoning content of the current round does not meet the constraints of the test reasoning scale of the current round, it may be that the slow thinking model is experiencing an illusion. In this case, reasoning can be performed again until the constraints are met.

[0024] Secondly, this application provides a question-answering device, which includes: an acquisition module and a processing module; the acquisition module is used to acquire a first query input by a user; the processing module is used to send the first query and the predictive inference scale related to the first query to a slow thinking model, wherein the slow thinking model is used to infer the first query with the predictive inference scale as a constraint to obtain a response, and the predictive inference scale related to the first query is obtained by budgeting the first query based on a budget evaluation model.

[0025] In one possible implementation, the processing module is further configured to: filter at least one first candidate query and a first candidate inference scale corresponding to the at least one first candidate query from the database based on the similarity between the first query and candidate queries in the database, wherein the database contains: at least one candidate query and a candidate inference scale corresponding to the at least one candidate query, the candidate inference scale being a smaller inference scale than the inference scale associated with the candidate query under natural inference; input the first query, at least one first candidate query, and the first candidate inference scale corresponding to the at least one first candidate query into a trained budget evaluation model, so that the budget evaluation model obtains the predicted inference scale of the first query with reference to the first candidate query and the first candidate inference scale.

[0026] In one possible implementation, the processing module is further configured to: obtain a candidate query, a first inference scale, and a first response, wherein the first inference scale is used to characterize the inference scale of the slow-thinking model when performing natural inference on the candidate query, the inference scale is used to characterize the quantitative features in the inference content generated by the slow-thinking model, and the first response is the inference result in the natural inference process; based on the slow-thinking model, the test inference scales that meet the first condition are filtered to obtain candidate inference scales, wherein at least one test inference scale is obtained by reducing the first inference scale, the first condition including: the test inference scale is smaller than the first inference scale, and the test response corresponding to the test inference scale is consistent with the first response, and the test response corresponding to the test inference scale refers to the inference result obtained by the slow-thinking model when inferring the candidate query with the test inference scale as a constraint.

[0027] In one possible implementation, the processing module is further configured to: filter test inference scales that meet the first condition based on the slow thinking model to obtain candidate inference scales: based on the slow thinking model, perform multiple rounds of inference on candidate queries with different test inference scales as constraints, so that a test response corresponding to the test inference scale of the current round is obtained in each round of inference, wherein the test inference scale of the current round is adjusted based on the first inference scale or the test inference scale of the previous round; select test inference scales that meet the first condition according to preset rules to obtain candidate inference scales.

[0028] In one possible implementation, the processing module is further configured to: in the process of adjusting the test inference scale of the previous round to obtain the test inference scale of the current round, if the test inference scale of the previous round meets the first condition, then reduce the test inference scale of the previous round to obtain the test inference scale of the current round; in the process of adjusting the first inference scale to obtain the test inference scale of the current round, reduce the first inference scale to obtain the test inference scale of the current round.

[0029] In one possible implementation, the processing module is further configured to: in the process of selecting test inference scales that meet the first condition according to preset rules to obtain candidate inference scales: when it is determined that the inference is to end and the test inference scale of the current round does not meet the first condition, the test inference scale of the previous round is used as a candidate inference scale; when it is determined that the inference is to end and the test inference scale of the current round meets the first condition, the test inference scale of the current round is used as a candidate inference scale.

[0030] Thirdly, this application provides a method for compressing inference scale, which includes: obtaining a candidate query, a first inference scale, and a first response, wherein the first inference scale is used to characterize the inference scale of the slow-thinking model when performing natural inference on the candidate query, the inference scale is used to characterize the quantitative features in the inference content generated by the slow-thinking model, and the first response is the inference result in the natural inference process; based on the slow-thinking model, screening test inference scales that meet a first condition to obtain candidate inference scales, wherein at least one test inference scale is obtained by reducing the first inference scale, the first condition including: the test inference scale is smaller than the first inference scale, and the test response corresponding to the test inference scale is consistent with the first response, and the test response corresponding to the test inference scale refers to the inference result obtained by the slow-thinking model when inferring the candidate query with the test inference scale as a constraint.

[0031] In one possible implementation, based on the slow-thinking model, the test inference scales that meet the first condition are filtered to obtain candidate inference scales. This includes: based on the slow-thinking model, multiple rounds of inference are performed on candidate queries with different test inference scales as constraints, so that a test response corresponding to the test inference scale of the current round is obtained in each round of inference, wherein the test inference scale of the current round is adjusted based on the first inference scale or the test inference scale of the previous round; and the test inference scales that meet the first condition are selected according to preset rules to obtain candidate inference scales.

[0032] In one possible implementation, the steps further include: in the process of adjusting the test inference scale of the previous round to obtain the test inference scale of the current round, if the test inference scale of the previous round satisfies the first condition, then the test inference scale of the previous round is reduced to obtain the test inference scale of the current round; in the process of adjusting the first inference scale to obtain the test inference scale of the current round, the first inference scale is reduced to obtain the test inference scale of the current round.

[0033] In one possible implementation, the test inference scale that satisfies the first condition is selected according to a preset rule to obtain a candidate inference scale, including: when it is determined that the inference has ended and the test inference scale of the current round does not satisfy the first condition, the test inference scale of the previous round is used as a candidate inference scale; when it is determined that the inference has ended and the test inference scale of the current round satisfies the first condition, the test inference scale of the current round is used as a candidate inference scale.

[0034] In one possible implementation, the first condition also includes: the test inference content of the current round satisfies the constraint of the test inference scale of the current round.

[0035] Fourthly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0036] Fifthly, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the method described in the first aspect or any possible implementation thereof. Exemplarily, the computing device cluster may include one or more computing devices.

[0037] Sixthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in the first aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0038] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of multi-step reasoning using existing slow-thinking models;

[0040] Figure 2 This is a schematic diagram illustrating the technical concept of a question-and-answer system provided in an embodiment of this application;

[0041] Figure 3 This is a schematic diagram of a prompt word template provided in an embodiment of this application;

[0042] Figure 4 This is a schematic diagram illustrating the technical concept of another question-answering system provided in an embodiment of this application;

[0043] Figure 5 This is a schematic diagram illustrating the interaction between an example retrieval module and a database, provided in an embodiment of this application.

[0044] Figure 6 yes Figure 4 A schematic diagram of the predictive inference module is obtained;

[0045] Figure 7 yes Figure 6 A schematic diagram of prompt word A in the text;

[0046] Figure 8 This is a schematic diagram illustrating the training of a budget assessment model according to an embodiment of this application;

[0047] Figure 9 This is a schematic diagram illustrating the training of a budget evaluation model based on budget samples, provided in an embodiment of this application.

[0048] Figure 10 This is a schematic diagram illustrating how a budget sample is obtained from a training sample through multiple rounds of inference, as provided in an embodiment of this application.

[0049] Figure 11 This is a flowchart illustrating a process for determining whether to proceed to the next round of reasoning in any round, as provided in an embodiment of this application.

[0050] Figure 12 This is a schematic diagram illustrating a comparison between different rounds of test inference scales provided in an embodiment of this application;

[0051] Figure 13 This is a flowchart illustrating how to select a specified test inference scale according to preset rules, provided in an embodiment of this application.

[0052] Figure 14 This is a schematic diagram illustrating an embodiment of obtaining a budget sample provided in this application;

[0053] Figure 15 This is a schematic diagram illustrating another comparison between different rounds of test inference scales provided in an embodiment of this application;

[0054] Figure 16 This is a schematic diagram illustrating a user interacting with a cloud computing platform, provided in an embodiment of this application.

[0055] Figure 17 This is a flowchart illustrating a question-and-answer method provided in an embodiment of this application;

[0056] Figure 18 This is a schematic diagram of a question-and-answer method provided in an embodiment of this application;

[0057] Figure 19 This application provides a flowchart illustrating a method for generating budget samples.

[0058] Figure 20 This is a schematic diagram of the structure of a question-and-answer device provided in an embodiment of this application;

[0059] Figure 21 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0060] Figure 22 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0061] Figure 23This is a schematic diagram of another computing device cluster structure provided in an embodiment of this application. Detailed Implementation

[0062] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0063] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0064] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0065] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0066] First, the relevant technical terms involved in the technical solution provided in this application will be introduced.

[0067] (1) Large language model (LLM): refers to a very large deep learning model that can be pre-trained based on a large number of training samples. Typically, large language models can have a huge number of parameters and can understand and generate human language, so they usually belong to natural language processing models.

[0068] (2) Token: In natural language processing, a "token" generally refers to a text unit such as a word or character, which is the smallest unit when a neural network model processes text data. For example, the sentence "who are you" can be divided into three tokens: "who", "are", and "you", with each word corresponding to one token.

[0069] (3) Prompt: This refers to a piece of input content provided when using a neural network model to guide the model in generating the corresponding output. The prompt can contain information such as task objectives, process constraints, or examples. The content of the prompt directly affects the model's reasoning and output.

[0070] (4) Slow-thinking model: also known as a slow-thinking system, the slow-thinking model refers to generating a more comprehensive, accurate, and well-reasoned solution by thinking and reasoning for a longer period of time before answering the query. Generally speaking, ordinary large language models will directly output a response based on the query, but the slow-thinking model 203 can output a response after receiving the query through multiple reasoning steps.

[0071] Figure 1 This is a diagram illustrating the thought process of a query using a slow-thinking model. Figure 1 In this system, users can access the slow-thinking model through a client and send queries to it. Upon receiving the query, the slow-thinking model will perform reasoning based on the query and return the resulting response to the user. The response refers to the reasoning result of the slow-thinking model, typically the content generated in the last step of a multi-step reasoning process. Figure 1 In the slow-thinking model, reasoning about user queries uses a thought chain approach. Figure 1 It can be seen that the model's reasoning process for a user query consists of four steps. Assuming the user query is: "How many 'r's are there in the word 'strawberry'?", this slow-thinking model can break down the entire reasoning process into four steps, and perform thinking in each step. Figure 1 In step 1, the thought process is: "I want to find out how many times the letter 'r' appears in the word 'strawberry,' and I can try checking each character individually." Then, based on the thought process in step 1, steps 2, 3, and 4 are considered respectively, and the output of step 4 is returned to the user as the query response. It should be noted that although the response to the user can use only the output of step 4, the response in step 4 depends on the thought process in steps 1 to 3. In this embodiment, the content considered in all steps of a complete reasoning process in the slow-thinking model is referred to as the reasoning content. Figure 1 In this context, the reasoning content is the sum of the content generated in steps 1 to 4. Additionally, in... Figure 1 As can be seen, the slow-thinking model is prone to repetitive thinking when performing multi-step reasoning. Figure 1 Taking steps 3 and 4 as examples, the model provides a clear answer in step 3, but outputs the same or similar content again in step 4, without substantially supplementing the reasoning result. This repetitive behavior leads to a waste of resources in the reasoning process and reduces reasoning efficiency.

[0072] Therefore, despite Figure 1 The slow-thinking model in this context can perform multi-step reasoning on user queries through thought chains or thought trees, thus outperforming traditional neural network models in terms of accuracy. However, this reasoning approach also incurs higher computational overhead. On the one hand, adopting a multi-step reasoning strategy significantly increases the number of tokens consumed in the overall reasoning process, typically several times that of conventional large language models. On the other hand, as mentioned earlier, this model may also encounter the problem of repetitive thinking during the reasoning process, further reducing reasoning efficiency.

[0073] In view of the above problems, this application provides a question-answering method that evaluates the user-input query, predicts the reasonable budget required for reasoning in a slow-thinking model, and uses this budget as a constraint to guide the slow-thinking model to control the scale of reasoning during the reasoning process. This method effectively reduces the computational cost of the reasoning process compared to the resource overhead of a slow-thinking model performing natural reasoning under unconstrained conditions. In this embodiment, natural reasoning refers to the reasoning process in which the slow-thinking model autonomously analyzes and deduces based on input information without external constraints.

[0074] For example, Figure 2 A schematic diagram illustrating the technical concept of a question-and-answer system provided in an embodiment of this application is shown. Figure 2As shown, the question-answering system includes: a budget assessment model 201, a prompt word construction module 202, and a slow-thinking model 203. The budget assessment model 201 is used to evaluate the reasonable budget (or "inference scale") required for the slow-thinking model 203 to perform query-related inference based on the user-input query (or "first query"), thus obtaining a predicted inference scale (or "candidate inference scale"). Here, budget assessment is the assessment of the inference scale. The inference scale is used to characterize the quantitative features in the inference content. The inference scale can be characterized using multiple granularities. For example, when the inference scale uses "token" as the granularity, it can characterize the total number of tokens in the inference content. Additionally, when the inference scale uses "step" as the granularity, it can characterize the number of steps in the inference content. Of course, the inference scale can also use multiple granularities simultaneously to more comprehensively reflect the resource overhead required for inference. Those skilled in the art can also conceive of other quantitative features related to characterizing the inference content based on the above description, which will not be elaborated here. The predicted inference size is used to constrain the slow-thinking model 203, ensuring that the inference size related to the inference content obtained by the slow-thinking model 203 when inferring the query does not exceed the predicted inference size. For example, the budget assessment model 201 receives a query and predicts a reasonable inference size constraint required by the slow-thinking model 203 to ensure response quality. Taking the inference size at the step level as an example, if the budget assessment model 201 estimates the predicted inference size related to the query to be 5 steps, it indicates that the budget assessment model 201 determines that the slow-thinking model 203 can obtain an accurate response after 5 steps. In contrast, if the slow-thinking model 203 is allowed to perform natural inference on the same query, it may take 8 steps to obtain a response.

[0075] The suggestion word construction module 202 is used to construct suggestion words based on the predictive inference scale output by the budget assessment model 201 and the user-input query. As one possible implementation, the suggestion word construction module 202 can call a preset suggestion word template, filling the template with both the predictive inference scale and the query, thereby generating suggestion words. For example... Figure 3 As shown, the predicted inference scale and the user-input query can be input together into the prompt word template to obtain prompt words. The prompt words constrain the slow-thinking model 203 during its inference of the query, ensuring that the inference scale related to the inference content in this inference does not exceed the predicted inference scale. For example, when the predicted inference scale is [step = 3, token count = 500], the prompt words constructed from the prompt word template will instruct the slow-thinking model 203 to ensure that the inference content generated during the inference process meets the constraint of the predicted inference scale.

[0076] The slow-thinking model 203 receives prompts generated by the prompt word construction module 202 and infers the user query based on these prompts. During inference, the slow-thinking model 203 follows the constraints in the prompts to ensure that the generated inference content does not exceed the limit of the predicted inference scale. Subsequently, the slow-thinking model 203 completes the inference and generates a query-related response, which is returned to the user.

[0077] Therefore, by evaluating the query using the budget assessment model 201, a relatively reasonable predicted inference scale for the slow-thinking model 203 is obtained while ensuring the accuracy of the response. Subsequently, this predicted inference scale is embedded as constraint information into the prompt words, so that the slow-thinking model 203 generates a response with the predicted inference scale as a constraint during the actual inference process. This method can control the inference scale of the slow-thinking model 203 when reasoning about the query while ensuring the accuracy of the response, thereby reducing resource overhead and computational consumption during the inference process. In addition, since the inference content is effectively compressed within the predicted inference scale, the occurrence of redundant thinking can also be significantly reduced.

[0078] To improve the generalization ability of the budget assessment model 201 when assessing queries within the aforementioned question-and-answer system, an example retrieval module 204 can be added to the system framework. In this case, the aforementioned system framework can be transformed into... Figure 4 The system framework is shown below. Figure 4 The system mainly includes: an example retrieval module 204, a database 205, a budget assessment model 201, a prompt word construction module 202, and a slow thinking model 203. Figure 4 The prompt word construction module 202 and the slow thinking model 203 can be found in the aforementioned... Figure 2 The relevant descriptions in the text will not be repeated here.

[0079] exist Figure 4Database 205 contains multiple candidate queries and corresponding candidate inference scales. The candidate queries are pre-defined queries, and the corresponding candidate inference scales are optimized inference scale reference values. The optimization of the candidate inference scale is relative to the inference scale of the slow-thinking model during natural inference. For example, a candidate (hereinafter referred to as "candidate query-A") corresponds to a candidate inference scale (hereinafter referred to as "candidate inference scale A"). When the slow-thinking model performs natural inference on candidate query-A, it can obtain the inference scale A based on the inference content. The relationship between candidate inference scale A and inference scale A is that candidate inference scale A is less than or equal to inference scale A. Therefore, the candidate inference scale is an optimized inference scale, allowing the slow-thinking model 203 to consume less resources and be more efficient than natural inference. For ease of description, a candidate query and its corresponding candidate inference scale will be considered as a candidate evaluation pair. In other words, database 205 can include multiple candidate evaluation pairs (or "budget samples"), and the candidate queries and candidate inference scales in database 205 can be mapped.

[0080] exist Figure 4 The example retrieval module 204 is used to retrieve at least one candidate query matching the user-input query from the database 205, and the candidate inference scale associated with the at least one candidate query, based on the user-input query. For example, the example retrieval module 204 can calculate the similarity between the user-input query and the candidate queries in the database 205 based on similarity algorithms such as cosine similarity, and select the top k candidate queries as the desired candidate queries. Further, the candidate inference scales corresponding to the selected top K candidate queries can be used as the desired candidate inference scales. In this way, candidate queries matching the user-input query and candidate inference scales associated with the retrieved candidate queries are retrieved from the database 205. As one possible implementation, such as... Figure 5As shown, a candidate query search module and a candidate inference scale search module can be configured. The candidate query search module can be used to calculate the similarity between candidate queries and the user-input query, and based on the calculation results, select the top K candidate queries as candidate queries that match the user-input query. Before searching in the candidate query search module, the user-input query can be vectorized first, and then the similarity search can be performed based on the vectorized query vector. The candidate inference scale search module can be used to search for candidate inference scales related to the top K candidate queries from database 205. For example, when calculating similarity and / or filtering, methods such as cosine similarity, Jaccard similarity coefficient, and edit distance can be used, or similarity calculation models such as the Reranker model, Sentence-BERT model, and DUET model can be used for similarity calculation and / or filtering.

[0081] exist Figure 4 In this process, the budget assessment model 201 can receive a user-input query, at least one candidate query retrieved by the example retrieval module 204, and candidate inference scales associated with the at least one candidate query (hereinafter referred to as "first data"). The budget assessment model 201 then performs a budget assessment based on the query and the first data to obtain the predicted inference scale. The first data enables the budget assessment model 201 to evaluate the predicted inference scale associated with the query using the first data as a reference. In some embodiments, such as... Figure 6 As shown, the prompt words (hereinafter referred to as "prompt words A") used to construct the budget assessment model 201 can be based on the query and the first data. During the construction of prompt words A, a prompt word template (hereinafter referred to as "prompt word A template") pre-set for prompt words A can be used. The prompt word A template can guide the budget assessment model 201 to predict the inference scale of the user-input query based on the first data provided by prompt words A. As one possible implementation, such as... Figure 7 As shown, the constructed prompt A can include a system message prompt (system_message_prompt), an in-context example (in-context example), and a user message prompt (human_message_prompt). The system message prompt can be used to instruct the behavior or role of the budget assessment model 201, clarifying its task. The in-context example can consist of the first data retrieved by the example retrieval module 204, which can be used to provide examples and references for the budget assessment model 201 when making inference-scale predictions. The user message prompt can be used to instruct the user on the query input. For example, the prompt A template can be as follows: Figure 7 As shown in (A). Figure 7In (A), the first data is written to the reference example of prompt word A template and used as a context example. The user-input query is then written to the input portion of the user message prompt word. After writing is complete, the following can be obtained: Figure 7 The prompt word A is shown in (B). Subsequently, the budget evaluation model 201 evaluates based on prompt word A to obtain the predictive inference scale. For example, the budget evaluation model 201 can be a large language model with natural language processing capabilities or other neural network models with natural language processing capabilities.

[0082] Therefore, on the one hand, the first data can enhance the generalization ability of the budget assessment model 201. The example closest to the query allows the budget assessment model 201 to use the first data as a reference when performing budget assessment on the query, making the predicted inference scale more accurate. On the other hand, as the database 205 is continuously updated, budget samples not covered by the budget assessment model 201 during the training phase can be continuously accumulated in the database 205, thereby improving the adaptability of the budget assessment model 201 in real-world scenarios. Through the example retrieval module 204, even when facing queries outside the training distribution, the retrieved first data can still assist the prediction process, enhancing the generalization ability of the budget assessment model 201, thereby effectively controlling the consumption of inference resources while maintaining the correctness of inference.

[0083] like Figure 8 As shown, the budget assessment model 201 can be trained based on budget samples, thereby enhancing its budget assessment capabilities. In this embodiment, the budget samples can serve as both candidate examples and be used to train the budget assessment model 201. For example, Figure 9 This is a schematic diagram illustrating the training process of the budget sample on the budget evaluation model 201. Figure 9 In this process, candidate queries from the budget sample are used as input to the budget evaluation model 201, enabling the budget evaluation model 201 to predict the inference scale of the slow-thinking model 203 based on the candidate queries, thus obtaining the predicted inference scale. The budget sample also contains candidate inference scales related to the candidate queries. Subsequently, the candidate inference scales are treated as labels for the candidate queries and backpropagated in combination with the obtained inference scales to train the budget evaluation model 201. This results in the trained budget evaluation model 201.

[0084] In this embodiment, the budget sample can be obtained through manual design. However, manual design is time-consuming and labor-intensive, and the standards vary, which has limited effect on improving the capabilities of the budget evaluation model 201.

[0085] In view of this, this application also proposes a technical concept for generating budget samples. Budget samples can be generated from training samples. The training samples include candidate queries, inference content corresponding to the candidate queries, and inference scale related to the inference content (hereinafter referred to as "original inference scale"). Subsequently, for the candidate query, a suitable test inference scale can be determined by gradually adjusting the test inference scale during multiple rounds of constrained inference, and the determined test inference scale is used as the candidate inference scale. This candidate inference scale and the candidate query constitute the budget sample. The candidate inference scale satisfies the following conditions: on the one hand, it is less than or equal to the original inference scale; on the other hand, it enables the slow-thinking model 203 to still generate a correct response under the constraint of the candidate inference scale. For example, the training samples can be obtained through a publicly available dataset, or the candidate query can be input into the slow-thinking model 203 to obtain the inference content. The original inference scale is then obtained by analyzing the inference content. As a possible implementation, the original inference scale can refer to the inference scale of the slow-thinking model 203 when performing natural inference on the candidate query.

[0086] In multi-round reasoning, different test reasoning scales are used to constrain the reasoning in the current round. Figure 10 This is a schematic diagram illustrating how budget samples are obtained from training samples through multiple rounds of inference. Figure 10 In the first round of reasoning, the original reasoning scale is shortened to a test reasoning scale smaller than the original. Then, the test reasoning scale and candidate queries are input into the slow-thinking model 203, which infers the candidate queries using the test reasoning scale as a constraint, obtaining the current reasoning content (hereinafter referred to as "test reasoning content"). Subsequently, at least based on the responses obtained from the reasoning content and the test reasoning content, it is determined whether to proceed to the next round of reasoning. If it is determined to proceed to the next round of reasoning, the test reasoning scale of the current round can be adjusted to obtain the test reasoning scale for the next round. Then, the new test reasoning scale and candidate queries are input into the slow-thinking model 203, which infers the candidate queries using the updated test reasoning scale as a constraint, obtaining the test reasoning content for this round. When it is determined that the reasoning is to end, a candidate reasoning scale is specified from the test reasoning scale according to preset rules, and the candidate reasoning scale is obtained based on the specified test reasoning scale. Thus, a budget sample consisting of candidate queries and candidate reasoning scales is obtained. In summary, during multiple rounds of reasoning, the test reasoning scale is adjusted before the next round of reasoning, and the trend in adjusting the test reasoning scale is to optimize it towards a smaller scale.

[0087] Subsequently, the following two parts will be explained separately: 1) how to determine whether to enter the next round of reasoning, and how to adjust the test reasoning scale before entering the next round of reasoning; 2) the judgment conditions for the termination of reasoning, and how to select the specified test reasoning scale according to the preset rules when reasoning terminates.

[0088] 1) How to determine whether to proceed to the next round of reasoning, and how to adjust the scale of test reasoning before proceeding to the next round of reasoning.

[0089] Figure 11 This is a flowchart illustrating the process of determining whether to proceed to the next round of reasoning in any given round. Figure 11 In the process, at least based on the scale of the current round of test reasoning, the content of the test reasoning, and the first response, it is determined whether to proceed to the next round of reasoning.

[0090] As one possible implementation, such as Figure 11 As shown in (A), we can first determine whether the test inference content of the current round satisfies the constraints of the current round's test inference scale and test inference content. We can also determine whether the test response obtained in the current round is correct. Then, based on whether the constraints are satisfied and whether the test response is correct, we can determine whether to proceed to the next round of inference. For example, if the test inference content of the current round satisfies the current round's test inference scale, it means that the slow-thinking model's inference process in this round satisfies the constraints of the current round's test inference scale. Furthermore, we can compare the test response with the correct response in the training samples (hereinafter referred to as "the first response"). If the current round's test response is consistent with the first response, it means that the answer output under the current round's test inference scale is accurate; if the current round's test response is inconsistent with the first response, it means that the slow-thinking model can no longer successfully output the correct answer when using the current round's test inference scale as a constraint. Therefore, we can proceed to the next round of inference when the current round's test inference content satisfies the constraints of the current round's test inference scale and the current round's test response is consistent with the first response. In this embodiment, consistency between the current round's test response and the first response means that their content is substantially the same. For example, when both the test response and the first response are numerical, consistency can be determined directly through numerical comparison. For non-numerical responses, neural network models or other text similarity calculation methods can be used to determine consistency. Those skilled in the art can choose appropriate consistency comparison methods according to actual needs, and no limitations are imposed here.

[0091] As another possible implementation, such as Figure 11 As shown in (B), it is possible to Figure 11Based on scheme (A), an additional judgment regarding the number of rounds is introduced. For example, it can be preset that after successfully running n rounds of reasoning, the test reasoning size of the current round is used as a candidate reasoning size. If the number of successfully run rounds is less than n rounds, the next round of reasoning continues. Here, n is a positive integer. Successfully running n rounds of reasoning means that the test reasoning content of the current round satisfies the constraint of the test reasoning size of the current round, and the test response of the current round is consistent with the first response. Otherwise, it is considered a failure.

[0092] Alternatively, it can be like Figure 11 As shown in (C), Figure 11 Scheme (C) and Figure 11 The schemes in (A) are largely the same, with the main difference being: Figure 11 In scheme (C), the next round of reasoning will proceed regardless of whether the current round's test response matches the first response of the current round. For example... Figure 12 As shown in (A), if the test response in the current round is inconsistent with the first response, then in the next round of reasoning, the test inference size will be larger than the inference size in the current round. Conversely, as... Figure 12 As shown in (B), if the test inference content of the current round meets the test inference size constraint of the current round, and the test response of the current round is consistent with the first response, then the test inference size of the next round will be smaller than the inference size of the current round. This strategy aims to explore the compression limit of the candidate query inference size of the slow-thinking model during the inference process, and ultimately obtain a smaller candidate inference size. Those skilled in the art can conceive of other criteria for determining whether to proceed with the next round of inference based on the three schemes described above, which will not be elaborated here.

[0093] 2) The conditions for determining the termination of inference, and how to select the specified test inference scale according to preset rules when inference terminates.

[0094] The decision to end the reasoning process can be determined based on one or more of the following: whether the current round's test response is consistent with the first response; whether the current round's test reasoning content meets the current round's test reasoning scale; or whether the predetermined round has been reached.

[0095] As one possible implementation, such as Figure 13 As shown, inference can end when the current round's test response is inconsistent with the first response, or when the current round's test inference content does not meet the current round's test inference scale, and the test inference scale used in the previous round is used as the candidate inference scale. Alternatively, inference can end after successfully running n rounds of inference, and the test inference scale of the current round is used as the candidate inference scale. Those skilled in the art can set the criteria for determining the end of inference according to the actual situation, and no restrictions are imposed here.

[0096] As another possible implementation, when the test inference content does not meet the constraint of the test inference scale used in the current round, the current round may be re-inferred.

[0097] When it is determined that the inference is ended, the test inference scale of the current round or the test inference scale of the previous round is selectively taken as the candidate inference scale. For the criteria of the preset rule, reference may be made to the foregoing Figure 11 solution of (C), which will not be repeated here.

[0098] To better understand the technical concept of generating budget samples, the following description will be combined with Figure 14 the examples to describe the present technical concept in detail.

[0099] In Figure 14 , one training sample is processed through the technical concept of budget sample generation to obtain the candidate inference scale under the same candidate query. The manner of acquiring the training sample has been described in the foregoing, and will not be repeated here.

[0100] In Figure 14 , the inference of the first round is performed first. In the inference of the first round, a range M0 related to the original inference scale is obtained according to the original inference scale. Then, the range M0 related to the original inference scale is subjected to inference scale reduction to obtain a first test inference scale M1. In this embodiment, the test inference scale may be represented by a numerical value or a range to indicate the meaning of the test inference scale. For example Figure 15 as shown, the range (0, m1] may be used to represent the meaning of the first test inference scale M1. That is, the constraint of the first test inference scale M1 will constrain the inference scale of the inference content output by the slow thinking model 203 within the range of (0, m1]. In Figure 15 , the range M0 related to the original inference scale is determined according to the original inference scale, and the range is (0, m0]. Then, the range of M0 is reduced to obtain M1. M1 is (0, m1], where m1<m0. For example, m1 may be half of m0, that is, m1=1 / 2*m0. Certainly, m1 may also be 1 / 3 of m0 or other proportions, which is not limited herein. For example, the original inference scale and the test inference scale use the same standard quantitative feature, that is, when the original inference scale is the number of steps, the test inference scale is also the number of steps. Taking m1=1 / 2*m0 as an example, when the range M0 related to the original inference scale is (0, step=6], the first test inference scale M1 may be (0, step=3]. For another example, when M0 is: {(0, step=6], (0, token=1000]}, M1 may be: {(0, step=6], (0, token=1000]}. Certainly, when the inference scale is reduced, only one granularity feature may be reduced, and the other remains unchanged, which will not be repeated here.

[0101] Subsequently, the first test inference scale M1 and the candidate query are input into the slow-thinking model 203, ensuring that the inference scale of the inference content performed by the slow-thinking model 203 satisfies the constraint of the first test inference scale M1 when reasoning about the candidate query. After completing the inference, the slow-thinking model 203 obtains the first test inference content. The first test inference content is used to determine whether to continue reasoning. For example, the decision to continue reasoning can be made by comparing whether the first test inference content satisfies the constraint of the first test inference scale and whether the first test inference content is correct. Figure 14 In the process, when the first test inference content satisfies the constraint of the first test inference size, and the test response in the first test inference content is consistent with the response in the training sample, the next round of inference is performed. When the test response in the first test inference content is inconsistent with the response in the training sample, or when the first test inference content does not satisfy the constraint of the first test inference size, the inference ends, and the original inference size M0 is used as the candidate inference size. Thus, a budget sample consisting of candidate queries and candidate inference sizes of M0 is obtained.

[0102] In another possible implementation, considering that the slow-thinking model 203 may exhibit some illusions, if the first test inference content does not meet the constraint of the first test inference size, the inference in this round can be repeated until the first test inference content meets the constraint of the first test inference size. Under the premise that the first test inference content meets the constraint of the first test inference size, when the test response in the first test inference content is inconsistent with the response in the training samples, the original inference size M0 is used as the candidate inference size, and a budget sample is obtained. For example, when the first test inference size M1 and the candidate query are input into the slow-thinking model 203, a prompt word (hereinafter referred to as "prompt word A") can be constructed based on the first test inference size M1 and the candidate query. Prompt word A is used to instruct the slow-thinking model 203 that the inference content generated during the inference process of the candidate query must meet the constraint of the first test inference size M1.

[0103] If it is determined that a second round of reasoning is to be performed, then a second round of reasoning is conducted. During the second round of reasoning, the reasoning size of the first test case M1 is first reduced to obtain the reasoning size of the second test case M2. For example... Figure 15as shown, the range of M2 is (0, m2], where m2 < m1. The selection manner of the second test inference scale M2 may be the same as the selection manner of the first test inference scale M1, which will not be repeated here. Subsequently, the second test inference scale M2 and the candidate query are input into the slow thinking model 203 together, so that when the slow thinking model 203 performs inference on the candidate query, the inference content thereof is constrained by the second test inference scale M2. After the current inference is completed, the second test inference content is obtained. Next, it is determined whether the second test inference content satisfies a preset condition, and this determination process has been described in detail in the foregoing content, which will not be repeated here. Subsequently, when it is determined that the inference is ended, the first test inference scale M1 is taken as a candidate inference scale, and a budget sample is obtained.

[0104] When the determination result is that a third round of inference is to be performed, the second test inference scale M2 is reduced, and the third test inference scale M3 used for the inference of the current round is obtained. As Figure 15 shown, the range of M3 is (0, m3], where m3 < m2. In the inference of the current round, the third test inference scale M3 and the candidate query are input into the slow thinking model 203 together, so that when the slow thinking model 203 performs inference on the candidate query, the inference scale of the inference content does not exceed the third test inference scale M3. After the inference, the third test inference content is obtained. Subsequently, it is determined whether the inference is ended. When it is determined that the inference is ended, the second test inference scale M2 is taken as a candidate inference scale, and a budget sample is obtained. When it is determined that the inference is not ended, the third test inference scale M3 is reduced, and the inference of the next round is performed according to the reduced test inference scale. This will not be repeated here.

[0105] In this way, in multiple rounds of inference, by gradually reducing the value of the test inference scale round by round, a relatively small test inference scale of the slow thinking model 203 on the premise of satisfying inference correctness is determined, thereby obtaining the candidate inference scale and the budget sample. Compared with the existing chain-of-thought samples, the candidate inference scale of the budget sample is optimized and simultaneously satisfies the constraint condition of the accuracy requirement of the slow thinking model 203. Therefore, after the foregoing budget evaluation model 201 is trained with the budget sample, the budget evaluation model 201 can perform budget evaluation on different queries and output an appropriate predicted inference scale. This inference scale can not only limit the resource consumption of the slow thinking model 203 when responding to a query, but also ensure the response accuracy of the slow thinking model 203 when responding to the query.

[0106] In some embodiments, each component of the above-described solution framework can be configured on a cloud computing platform, for example, deployed on at least one instance such as a virtual machine or container, so that the cloud computing platform can provide a question-and-answer service. Of course, each component of the above-described solution framework can also be configured on nodes other than the cloud computing platform, for example, deployed in at least one data center or on at least one server, depending on the actual situation, and is not limited here. The cloud computing platform can provide pages related to public cloud services for users to remotely access public cloud services. In this embodiment, users can pre-purchase question-and-answer services on the cloud computing platform. For ease of understanding, the interaction pattern between the user and the cloud computing platform is described below. Figure 16 As shown, the interaction between the user and the cloud computing platform mainly includes: the user logs into the cloud computing platform 1600 through a client webpage, selects and purchases a question-and-answer service on the cloud computing platform 1600, and after purchase, the user can rewrite SQL queries on the cloud computing platform 1600 based on the functions provided by the question-and-answer service. The cloud computing platform 1600 is primarily used to manage the infrastructure running the question-and-answer service. For example, the infrastructure running the question-and-answer service may include multiple data centers located in different regions, each data center including multiple servers. The data centers can provide basic resources for the question-and-answer service, such as computing resources and storage resources. Therefore, when purchasing and using the question-and-answer service, the user is mainly paying for the resources used. When using the question-and-answer service, the user can input SQL statements through the configuration interface, application programming interface, or user interaction interface provided by the cloud computing platform 1600. The cloud computing platform 1600 can then rewrite the SQL queries according to the user's input. In some embodiments, the various parts of the above-described solution framework can also be configured on a local server, depending on the actual situation, and are not limited here.

[0107] The following describes the specific implementation process of the above technical concept.

[0108] For example, Figure 17 This diagram illustrates a flowchart of a question-answering method provided in an embodiment of this application. It is understood that this method can be executed by any device, apparatus, platform, or cluster of devices with computing and processing capabilities. For example, the method can be executed by a question-answering device, which can be implemented by software and / or hardware, and can be, but is not limited to, configured in an electronic device or server; typically, it can be configured on a server. Figure 17 As shown, the inference scale prediction method may include the following steps:

[0109] S2101: Obtain the first query input by the user.

[0110] In this embodiment, the first query can be understood as a question or request raised by the user. In this embodiment, the first query can be input by the user, or sent by a device, apparatus, or service, etc., without limitation. For example, when the user inputs the first query through the client, the client sends the first query to the server. The server then obtains the first query and performs subsequent processing. As mentioned above, this server can be a cloud server such as a cloud computing platform, or a local server, which will not be elaborated further here.

[0111] S2102: Send the first query and the predictive inference scale associated with the first query to the slow-thinking model.

[0112] In this embodiment, the slow thinking model is used to reason about the first query with the predictive inference scale as a constraint to obtain a response. The predictive inference scale associated with the first query is obtained by budgeting the first query based on the budget evaluation model.

[0113] In this embodiment, the budget evaluation module can perform a budget evaluation on the first query to obtain the predicted inference scale. Budget evaluation refers to the evaluation of the inference scale. The inference scale can be represented using various granularities. The predicted inference scale is used to allow the slow-thinking model to infer the first query with the predicted inference scale as a constraint. For example, the predicted inference scale can be represented numerically or as a range. When the predicted inference scale is numerical, using the predicted inference scale as a constraint means that the inference scale corresponding to the inference content obtained by the slow-thinking model when inferring the first query is less than the predicted inference scale. When the predicted inference scale is a range, using the predicted inference scale as a constraint means that the inference scale corresponding to the inference content obtained by the slow-thinking model when inferring the first query should be within the range of the predicted inference scale. For example, the detailed process of this step can be found in the foregoing. Figure 2 The relevant descriptions of the budget evaluation model are not repeated here.

[0114] Therefore, by evaluating the first query using a budget assessment model, a relatively reasonable predictive inference scale for the slow-thinking model is obtained while ensuring accurate responses. This predictive inference scale can reasonably control the inference scale of the slow-thinking model when reasoning about the first query, thereby reducing resource overhead and computational consumption during the inference process, while ensuring response accuracy.

[0115] Furthermore, as previously explained, slow-thinking models, under natural reasoning conditions, often expand their thought processes through multiple steps to enhance the comprehensiveness and rigor of their answers. However, without length constraints, the model may exhibit repetitive thinking (e.g., repeatedly judging or stating the same reasoning conclusion in multiple steps) or generate redundant content (e.g., repeatedly expanding upon intermediate conclusions), leading to lengthy reasoning processes, excessive token counts, and wasted computational resources. In this approach, the predicted reasoning scale is used as an upper limit for reasoning, allowing the slow-thinking model to complete reasoning within a limited budget. This is equivalent to setting boundaries on the depth and breadth of the model's thought process. Under this constraint, the slow-thinking model may tend to choose more information-dense reasoning paths, skipping or compressing repetitive steps to prioritize high-value reasoning content. Therefore, using the predicted reasoning scale as a constraint not only limits the amount of content generated by the slow-thinking model but also prompts the model to strategically avoid inefficient and redundant thinking, thereby significantly reducing the occurrence of repetitive reasoning.

[0116] However, when the budget evaluation model predicts the first query, some first queries may be outside the distribution of the training data (or "budget samples") during the training of the budget evaluation model. Therefore, in order to further improve the generalization ability of the budget evaluation model, S2102 may also include: S2102-1 and S2102-2.

[0117] S2102-1: Based on the similarity between the first query and candidate queries in the database, at least one first candidate query and the first candidate inference scale corresponding to the at least one first candidate query are selected from the database.

[0118] In this embodiment, the database contains at least one candidate query and a corresponding candidate inference size. The candidate query is a pre-defined query, and the corresponding candidate inference size is an optimized inference size reference value. The optimization of the candidate inference size is relative to the inference size of the slow-thinking model during natural inference. In the database, one candidate query corresponds to one candidate inference size. In this embodiment, the candidate query selected from the data is called the first candidate query, and the selected candidate inference size is called the first candidate inference size.

[0119] S2102-2: Input the first query, at least one first candidate query, and the first candidate inference size corresponding to at least one first candidate query into the trained budget evaluation model, so that the budget evaluation model obtains the inference size of the first query with reference to the first candidate query and the first candidate inference size.

[0120] In this embodiment, at least one first candidate query and the first candidate inference scale corresponding to at least one first candidate query can be simply referred to as first data. The first query and the filtered first data are input into a trained budget assessment model, allowing the budget assessment model to perform a budget assessment on the first query and obtain the predicted inference scale. For example, a preset prompt word A template can be used to construct prompt word A. During the construction of prompt word A, the first data is used as an example part, and the first query is used as the user's input part. Then, prompt word A is input into the budget assessment model so that the example is used as a reference when performing budget assessment. For example, the detailed process of this step can be found in the foregoing. Figure 4 The relevant descriptions of the budget evaluation model are not repeated here.

[0121] like Figure 18 As shown, the first query can be transformed into a query vector. The example retrieval module then retrieves constrained samples from the database that match the first query. These constrained samples include candidate queries and candidate inference sizes. The retrieved constrained samples serve as examples to instruct the budget evaluation model to perform a budget evaluation on the first query, obtaining a predicted inference size. Afterward, the predicted inference size and the first query are sent to the slow-thinking model, allowing it to infer the first query with the predicted inference size as a constraint, thus obtaining a response.

[0122] Therefore, on the one hand, the first set of data can enhance the generalization ability of the budget assessment model. Examples most similar to the query allow the model to use this first set of data as a reference when assessing the budget, resulting in more accurate predictions of the inference scale. On the other hand, as the database is continuously updated, it can accumulate budget samples not covered during the training phase, thereby improving the model's adaptability to real-world scenarios. Through the example retrieval module, even when faced with queries outside the training distribution, the retrieved first set of data can still assist the prediction process, enhancing the generalization ability of the budget assessment model and effectively controlling inference resource consumption while maintaining inference accuracy.

[0123] For example, Figure 19 A flowchart illustrating a budget sample generation method provided in an embodiment of this application is shown. The budget sample generation method may include the following steps:

[0124] S1801: Obtain candidate queries, first inference size, and first response related to the first inference size.

[0125] In this embodiment, the first inference scale is used to characterize the inference scale of the slow-thinking model when performing natural inference on candidate queries. The inference scale is used to characterize the quantitative features in the inference content generated by the slow-thinking model. The first response is the inference result in the natural inference process. Natural inference refers to the inference process in which the slow-thinking model autonomously analyzes and deduces based on input information without external constraints.

[0126] S1802: Based on the slow thinking model, the test inference scales that meet the first condition are filtered to obtain candidate inference scales. Among them, at least one test inference scale is obtained by shrinking the first inference scale. The first condition includes: the test inference scale is smaller than the first inference scale, and the test response corresponding to the test inference scale is consistent with the first response. The test response corresponding to the test inference scale refers to the inference result obtained by the slow thinking model in reasoning the candidate query when the test inference scale is a constraint.

[0127] In this embodiment, using the test inference scale as a constraint means that when the slow-thinking model performs inference, the inference scale corresponding to the generated inference content will not exceed the test inference scale. For example, the test inference scale can be a numerical value or a range. When the test inference scale is a range, reducing the test inference scale means reducing its right boundary. Furthermore, consistency between the test response and the first response means that they are substantially the same. For example, when both the test response and the first response are numerical values, consistency can be determined directly through numerical comparison. For non-numerical responses, neural network models or other text similarity calculation methods can be used to determine consistency.

[0128] Additionally, the first condition may also include: the test reasoning content of the current round satisfies the constraint of the test reasoning size of the current round. If the test reasoning content of the current round does not satisfy the constraint of the test reasoning size of the current round, it may be that the slow-thinking model is experiencing an illusion. In this case, reasoning can be performed again until the constraint is satisfied.

[0129] Therefore, the candidate queries and their corresponding candidate inference sizes are used as the budget sample. In this sample, the candidate inference size is smaller than the inference size associated with the candidate query under natural inference, which can reduce repetitive thinking during the slow-thinking model's response.

[0130] Furthermore, to improve the quality of the obtained budget samples, S1802, during the screening process, can use a slow-thinking model to perform multiple rounds of reasoning on candidate queries with different test inference scales as constraints. This ensures that in each round of reasoning, a test response corresponding to the current round's test inference scale is obtained. The current round's test inference scale is adjusted based on the first inference scale or the previous round's test inference scale. Candidate inference scales are then selected based on preset rules that meet the first condition.

[0131] In this embodiment, a test inference size is used as a constraint in each round of inference, allowing the slow-thinking model to infer candidate queries. The test inference size used in different rounds of inference differs. This multi-round inference and dynamic adjustment of the test inference size allows for a more refined exploration of the convergence process of the inference size while ensuring response consistency, thereby improving the accuracy and compactness of the candidate inference size. For example, the test inference size used in the first round of inference is adjusted based on a first inference size. The test inference size used in subsequent rounds of inference is adjusted based on the test inference size of the previous round. For example, in adjusting the first inference size to obtain the test inference size for the current round, the first inference size can be reduced to obtain the test inference size for the current round. In adjusting the test inference size of the previous round to obtain the test inference size for the current round, if the test inference size of the previous round satisfies a first condition, then the test inference size of the previous round is reduced to obtain the test inference size for the current round. If the test inference size of the previous round does not satisfy the first condition, the inference process can end. Alternatively, in the process of obtaining the test inference scale of the current round, the test inference scale of the current round may be made larger than the test inference scale of the previous round.

[0132] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, the various embodiments described above can be combined according to actual conditions, and the combined solutions are still within the protection scope of this application.

[0133] Next, based on the methods in the above embodiments, a question-and-answer device provided in this application will be introduced.

[0134] For example, Figure 20 A schematic diagram of the structure of a question-and-answer device provided in an embodiment of this application is shown. Figure 20 As shown, the question-answering device 2000 includes: an acquisition module 2001 and a processing module 2002; the acquisition module 2001 is used to acquire a first query input by the user; the processing module 2002 is used to send the first query and the predictive inference scale associated with the first query to a slow-thinking model, wherein the slow-thinking model is used to infer the first query with the predictive inference scale as a constraint to obtain a response, and the predictive inference scale associated with the first query is obtained by budgeting the first query based on a budget evaluation model.

[0135] In one possible implementation, the processing module 2002 is further configured to: based on the similarity between the first query and candidate queries in the database, filter at least one first candidate query and a first candidate inference scale corresponding to the at least one first candidate query from the database, wherein the database contains: at least one candidate query and a candidate inference scale corresponding to the at least one candidate query, the candidate inference scale being a smaller inference scale than the inference scale associated with the candidate query under natural inference; input the first query, at least one first candidate query, and the first candidate inference scale corresponding to the at least one first candidate query into a trained budget evaluation model, so that the budget evaluation model obtains the predicted inference scale of the first query with reference to the first candidate query and the first candidate inference scale.

[0136] In one possible implementation, the processing module 2002 is further configured to: obtain a candidate query, a first inference scale, and a first response, wherein the first inference scale is used to characterize the inference scale of the slow-thinking model when performing natural inference on the candidate query, the inference scale is used to characterize the quantitative features in the inference content generated by the slow-thinking model, and the first response is the inference result in the natural inference process; based on the slow-thinking model, the test inference scales that meet the first condition are filtered to obtain candidate inference scales, wherein at least one test inference scale is obtained by reducing the first inference scale, the first condition including: the test inference scale is smaller than the first inference scale, and the test response corresponding to the test inference scale is consistent with the first response, and the test response corresponding to the test inference scale refers to the inference result obtained by the slow-thinking model when inferring the candidate query with the test inference scale as a constraint.

[0137] In one possible implementation, the processing module 2002 is further configured to: filter test inference scales that meet the first condition based on the slow thinking model to obtain candidate inference scales: based on the slow thinking model, perform multiple rounds of inference on candidate queries with different test inference scales as constraints, so that a test response corresponding to the test inference scale of the current round is obtained in each round of inference, wherein the test inference scale of the current round is adjusted based on the first inference scale or the test inference scale of the previous round; select test inference scales that meet the first condition according to preset rules to obtain candidate inference scales.

[0138] In one possible implementation, the processing module 2002 is further configured to: in the process of adjusting the test inference scale of the previous round to obtain the test inference scale of the current round, if the test inference scale of the previous round meets the first condition, then reduce the test inference scale of the previous round to obtain the test inference scale of the current round; in the process of adjusting the first inference scale to obtain the test inference scale of the current round, reduce the first inference scale to obtain the test inference scale of the current round.

[0139] In one possible implementation, the processing module 2002 is further configured to: in the process of selecting test inference scales that meet the first condition according to preset rules to obtain candidate inference scales: when it is determined that the inference is to end and the test inference scale of the current round does not meet the first condition, the test inference scale of the previous round is used as a candidate inference scale; when it is determined that the inference is to end and the test inference scale of the current round meets the first condition, the test inference scale of the current round is used as a candidate inference scale.

[0140] In some embodiments, Figure 20 Both the acquisition module 2001 and the processing module 2002 shown can be implemented in software or in hardware. For example, the implementation of the acquisition module 2001 will be described below. Similarly, the implementation of the processing module 2002 can refer to the implementation of the acquisition module 2001.

[0141] As an example of a software functional unit, module 2001 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Furthermore, the aforementioned computing instance may be one or more. For example, module 2001 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0142] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0143] As an example of a hardware functional unit, the acquisition module 2001 may include at least one computing device, such as a server. Alternatively, the acquisition module 2001 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0144] The multiple computing devices included in the acquisition module 2001 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the acquisition module 2001 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the acquisition module 2001 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0145] It should be noted that, in other embodiments, the acquisition module 2001 can be used to execute any step in the question-and-answer method described in the above embodiments, and the processing module 2002 can also be used to execute any step in the question-and-answer method described in the above embodiments. Furthermore, the steps implemented by the acquisition module 2001 and the processing module 2002 can be specified as needed, and different steps in the question-and-answer method described in the above embodiments can be implemented by the acquisition module 2001 and the processing module 2002 respectively. Figure 20 The question-and-answer device 700 shown has all the functions.

[0146] This application also provides a computing device 2100. For example... Figure 21 As shown, the computing device 2100 includes a bus 2102, a processor 2104, a memory 2106, and a communication interface 2108. The processor 2104, the memory 2106, and the communication interface 2108 communicate with each other via the bus 2102. The computing device 2100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 2100.

[0147] Bus 2102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 21 The bus 2104 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 2104 may include a path for transmitting information between various components of the computing device 2100 (e.g., memory 2106, processor 2104, communication interface 2108).

[0148] Processor 2104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0149] The memory 2106 may include volatile memory, such as random access memory (RAM). The processor 2104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0150] The memory 2106 stores executable program code, and the processor 2104 executes the executable program code to implement the aforementioned functions respectively. Figure 20 The functions of the acquisition module 2001 and processing module 2002 shown are used to implement the question-and-answer method described in the above embodiments. That is, the memory 2106 stores instructions for executing the question-and-answer method described in the above embodiments.

[0151] Alternatively, the memory 2106 stores executable code, and the processor 2104 executes the executable code to implement the aforementioned functions respectively. Figure 20 The question-and-answer device 2000 shown in the figure functions to implement the question-and-answer method described in the above embodiments. That is, the memory 2106 stores instructions for executing the question-and-answer method described in the above embodiments.

[0152] The communication interface 2108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 2100 and other devices or communication networks.

[0153] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0154] like Figure 22 As shown, the computing device cluster includes at least one computing device 2100. The memory 2106 of one or more computing devices 2100 in the computing device cluster may store the same instructions for executing the question-and-answer method described in the above embodiments.

[0155] In some possible implementations, the memory 2106 of one or more computing devices 2100 in the computing device cluster may also store partial instructions for executing the question-and-answer method described in the above embodiments. In other words, a combination of one or more computing devices 2100 can jointly execute instructions for executing the question-and-answer method described in the above embodiments.

[0156] It should be noted that the memory 2106 in different computing devices 2100 within the computing device cluster can store different instructions, each used to execute the aforementioned instructions. Figure 20 The question-and-answer device 700 shown contains some of the functions. That is, the instructions stored in the memory 2106 of different computing devices 2100 can implement the functions of one or more modules in the acquisition module 2001 and the processing module 2002.

[0157] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN), a local area network (LAN), or similar. Figure 23 One possible implementation is shown. For example... Figure 23 As shown, the two computing devices 2100A and 2100B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device. In this type of possible implementation, the memory 2106 in computing device 2100A stores instructions for executing the functions of the acquisition module 2001. Simultaneously, the memory 2106 in computing device 2100B stores instructions for executing the functions of the processing module 2002.

[0158] It should be understood that Figure 23The functions of computing device 2100A shown can also be performed by multiple computing devices 2100. Similarly, the functions of computing device 2100B can also be performed by multiple computing devices 2100.

[0159] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 22 and Figure 23 The connection method of the computing device cluster. The difference is that the memory 2106 of one or more computing devices 2100 in the computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0160] In some possible implementations, the memory 2106 of one or more computing devices 2100 in the computing device cluster may also store partial instructions for executing the aforementioned question-and-answer method. In other words, a combination of one or more computing devices 2100 can jointly execute the instructions for executing the aforementioned question-and-answer method.

[0161] It should be understood that each step of the above method embodiments can be accomplished by hardware logic circuits or software instructions in a processor.

[0162] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When executed by a cluster of computing devices including at least one computing device, the computer program instructions cause the cluster of computing devices to perform the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium that the computing device can store, or a data storage device such as a data center containing one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0163] Based on the methods in the above embodiments, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices containing at least one computing device, cause the cluster of computing devices to perform the methods in the above embodiments.

[0164] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0165] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0166] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0167] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A question-and-answer method, characterized in that, include: Get the first query input by the user; The first query and the predictive inference scale associated with the first query are sent to the slow thinking model, wherein the slow thinking model is used to infer the first query with the predictive inference scale as a constraint to obtain a response, and the predictive inference scale associated with the first query is obtained by budgeting the first query based on the budget evaluation model.

2. The method according to claim 1, characterized in that, The method further includes: Based on the similarity between the first query and candidate queries in the database, at least one first candidate query and a first candidate inference scale corresponding to the at least one first candidate query are selected from the database. The database contains at least one candidate query and a candidate inference scale corresponding to the at least one candidate query. The candidate inference scale refers to an inference scale that is smaller than the inference scale associated with the candidate query under natural inference. The first query, at least one first candidate query, and the first candidate inference size corresponding to at least one first candidate query are input into the trained budget evaluation model so that the budget evaluation model obtains the predicted inference size of the first query with reference to the first candidate query and the first candidate inference size.

3. The method according to claim 1 or 2, characterized in that, The budget evaluation model is trained based on budget samples, which are generated through the following steps: Obtain candidate queries, a first inference scale, and a first response, wherein the first inference scale is used to characterize the inference scale of the slow-thinking model when performing natural inference on the candidate queries, the inference scale is used to characterize the quantitative features in the inference content generated by the slow-thinking model, and the first response is the inference result in the natural inference process; Based on the slow-thinking model, test inference scales that meet the first condition are filtered to obtain candidate inference scales. At least one of the test inference scales is obtained by shrinking the first inference scale. The first condition includes: the test inference scale is smaller than the first inference scale, and the test response corresponding to the test inference scale is consistent with the first response. The test response corresponding to the test inference scale refers to the inference result obtained by the slow-thinking model in reasoning about the candidate query when the test inference scale is a constraint.

4. The method according to claim 3, characterized in that, The slow-thinking model filters test inference sizes that meet the first condition to obtain candidate inference sizes, including: Based on the slow thinking model, the candidate query is subjected to multiple rounds of reasoning with different test reasoning scales as constraints, so that a test response corresponding to the test reasoning scale of the current round is obtained in each round of reasoning. The test reasoning scale of the current round is adjusted based on the first reasoning scale or the test reasoning scale of the previous round. Based on preset rules, test inference sizes that meet the first condition are selected to obtain candidate inference sizes.

5. The method according to claim 4, characterized in that, The steps also include: In the process of adjusting the test inference scale of the previous round to obtain the test inference scale of the current round, if the test inference scale of the previous round meets the first condition, then the test inference scale of the previous round is reduced to obtain the test inference scale of the current round. In the process of adjusting the first inference scale to obtain the test inference scale for the current round, the first inference scale is reduced to obtain the test inference scale for the current round.

6. The method according to claim 4 or 5, characterized in that, The step of selecting test inference sizes that meet the first condition according to preset rules to obtain candidate inference sizes includes: When it is determined that the reasoning has ended and the test reasoning size of the current round does not meet the first condition, the test reasoning size of the previous round is used as the candidate reasoning size. When it is determined that the reasoning has ended and the test reasoning size of the current round meets the first condition, the test reasoning size of the current round is taken as the candidate reasoning size.

7. The method according to any one of claims 3-6, characterized in that, The first condition also includes: the test reasoning content of the current round satisfies the constraint of the test reasoning scale of the current round.

8. A question-and-answer device, characterized in that, include: Acquisition module and processing module; The acquisition module is used to acquire the first query input by the user; The processing module is used to send the first query and the predictive inference scale associated with the first query to the slow thinking model, wherein the slow thinking model is used to infer the first query with the predictive inference scale as a constraint to obtain a response, and the predictive inference scale associated with the first query is obtained by budgeting the first query based on a budget evaluation model.

9. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The method includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-7, wherein the cluster of computing devices includes at least one computing device.

11. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-7, wherein the computing device cluster includes at least one computing device.

Images