A method and apparatus for collaborative reasoning of multiple heterogeneous models

Abstract

This invention discloses a method and apparatus for collaborative reasoning using multiple heterogeneous models, relating to the field of artificial intelligence reasoning and prediction technology. The method includes: receiving a reasoning request and a prompt word submitted by a client; selecting multiple matching heterogeneous models to simultaneously perform token prediction based on the context information and preset conditions of the reasoning request; selecting the best prediction result from the prediction results of all heterogeneous models; concatenating the best prediction result with the prompt word; if the concatenated result does not complete the reasoning, submitting the concatenated result as a new prompt word to the multiple matching heterogeneous models, and continuing iterative reasoning until the reasoning is completed. This invention improves overall reasoning performance and efficiency, and enhances the model's ability to perform tasks in specific domains.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence reasoning and prediction technology, and in particular to a method and apparatus for collaborative reasoning of multiple heterogeneous models. Background Technology

[0002] In existing technologies, AI models with different architectures (such as Large Language Models (LLMs) typically operate independently, with each model performing inference based on its weights and architectural characteristics. However, a single model may perform poorly on a specific task or be inefficient when handling certain types of queries. Furthermore, a single model may not fully leverage the domain or task-specific expertise of other models. Therefore, AI models with a single architecture generally exhibit low overall inference performance and efficiency when handling inference tasks, and their ability to perform specific tasks is weak. Summary of the Invention

[0003] In view of the aforementioned deficiencies or shortcomings in the prior art, the present invention provides a method and apparatus for collaborative reasoning among multiple heterogeneous models. By enabling collaborative work between models with different architectures, the overall reasoning performance and efficiency are improved, and the ability of the models to perform tasks in specific domains is enhanced.

[0004] One aspect of the present invention provides a method for collaborative reasoning of multiple heterogeneous models, comprising: receiving a reasoning request and a prompt word submitted by a client; selecting multiple matching heterogeneous models to simultaneously perform token prediction based on context information and preset conditions of the reasoning request; selecting the best prediction result from the prediction results of all heterogeneous models; concatenating the best prediction result with the prompt word; if the concatenation result does not complete the reasoning, submitting the concatenation result as a new prompt word to the multiple matching heterogeneous models, and continuing iterative reasoning until the reasoning is completed.

[0005] Furthermore, the preset conditions include at least one of the following: the model's expertise or area of ​​expertise, inference cost and resource consumption, response time and real-time requirements, probability distribution and confidence level of the predicted token, and the labeling of a specific token.

[0006] Furthermore, the step of selecting multiple matching heterogeneous models to perform token prediction simultaneously based on the context information and preset conditions of the inference request includes: if the label information in the inference request indicating the domain of expertise of the model matches the labels declared by multiple heterogeneous models, then the inference request is sent to the multiple heterogeneous models, and the multiple heterogeneous models that receive the inference request perform inference prediction simultaneously, with each model generating no more than a predetermined number of tokens.

[0007] Furthermore, the step of selecting multiple matching heterogeneous models to perform token prediction simultaneously based on the context information and preset conditions of the inference request includes: if the inference request does not contain label information indicating the domain that the model is good at, then the prompt word is predicted and classified by a language model, and the inference request is sent to multiple heterogeneous models corresponding to the declaration labels that match the classification results. Multiple heterogeneous models perform inference prediction simultaneously, and each model generates no more than a predetermined number of tokens.

[0008] Furthermore, the step of selecting the best prediction result from all heterogeneous models includes: recording the prediction results generated by each model. The probability value of the token with the highest probability during the process of selecting tokens. The probability value of the selected token Record the first to the last generation of each model. The average probability of the token with the highest probability during the process of generating tokens. and the average probability of the selected token If one of the following conditions is met, the model will stop generating tokens and end the current round of inference and prediction: Condition 1: Deviation The set threshold has been reached; Condition 2: Deviation The set threshold has been reached; Condition 3: Deviation The set threshold has been reached; Condition 4: A predetermined number of tokens have been generated; Condition 5: The model has marked this inference as ended; Calculate the average deviation of each heterogeneous model in conditions 1, 2 and 3 respectively, and take the prediction result of the model with the smallest average deviation as the best prediction result.

[0009] In another aspect, the present invention provides an apparatus for collaborative reasoning of multiple heterogeneous models, comprising: a receiving module for receiving a reasoning request and a prompt word submitted by a client; a scheduling module for selecting multiple matching heterogeneous models to perform token prediction simultaneously based on the context information and preset conditions of the reasoning request; and a reasoning module for selecting the best prediction result from the prediction results of all heterogeneous models, concatenating the best prediction result with the prompt word, and if the concatenation result does not complete the reasoning, submitting the concatenation result as a new prompt word to the multiple matching heterogeneous models, and continuing iterative reasoning until the reasoning is completed.

[0010] Furthermore, the preset conditions include at least one of the following: the model's expertise or area of ​​expertise, inference cost and resource consumption, response time and real-time requirements, probability distribution and confidence level of the predicted token, and the labeling of a specific token.

[0011] Furthermore, the scheduling module is also used to: if the label information indicating the domain of expertise of the model in the inference request matches the labels declared by multiple heterogeneous models, then the inference request is sent to the multiple heterogeneous models, and the multiple heterogeneous models that receive the inference request simultaneously perform inference prediction, with each model generating no more than the predetermined number of Tokens.

[0012] Furthermore, the scheduling module is also used to: if the inference request does not contain label information indicating the domain that the model is good at, then the prompt word is predicted and classified by the language model, and the inference request is sent to multiple heterogeneous models corresponding to multiple declaration labels that match the classification results. Multiple heterogeneous models perform inference prediction at the same time, and each model generates no more than a predetermined number of tokens.

[0013] Furthermore, the inference module is also used to: record the number of times each model generates the first... The probability value of the token with the highest probability during the process of selecting tokens. The probability value of the selected token Record the first to the last generation of each model. The average probability of the token with the highest probability during the process of generating tokens. and the average probability of the selected token If one of the following conditions is met, the model will stop generating tokens and end the current round of inference and prediction: Condition 1: Deviation The set threshold has been reached; Condition 2: Deviation The set threshold has been reached; Condition 3: Deviation The set threshold has been reached; Condition 4: A predetermined number of tokens have been generated; Condition 5: The model has marked this inference as ended; Calculate the average deviation of each heterogeneous model in conditions 1, 2 and 3 respectively, and take the prediction result of the model with the smallest average deviation as the best prediction result.

[0014] This invention provides a method and apparatus for collaborative reasoning using multiple heterogeneous models. It innovatively integrates multiple models with different architectures into a single inference system, supporting collaborative reasoning among heterogeneous models, fully leveraging the strengths of each model, and dynamically selecting the most suitable model for token generation based on preset conditions. This invention improves overall inference performance and efficiency, and enhances the model's ability to perform tasks in specific domains. Attached Figure Description

[0015] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0016] Figure 1 This is a flowchart illustrating a method for collaborative inference of multiple heterogeneous models provided in one embodiment of this application;

[0017] Figure 2 This is a schematic diagram of the structure of a device for collaborative inference of multiple heterogeneous models provided in one embodiment of this application;

[0018] Figure 3 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should be understood that although the terms first, second, third, etc., may be used to describe the acquisition modules in the embodiments of the present invention, these acquisition modules should not be limited to these terms. These terms are only used to distinguish the acquisition modules from each other.

[0022] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0023] It should be noted that the directional terms such as "upper," "lower," "left," and "right" used in the embodiments of the present invention are used to describe the angles shown in the accompanying drawings and should not be construed as limiting the embodiments of the present invention. Furthermore, in the context, it should be understood that when it is mentioned that an element is formed "upper" or "lower" of another element, it can not only be formed directly "upper" or "lower" of the other element, but also indirectly "upper" or "lower" of the other element through an intermediate element.

[0024] This application provides a method and apparatus for collaborative reasoning of multiple heterogeneous models, which can be implemented based on the following system:

[0025] Multiple language models with different architectures are available, each capable of independent inference. These model services can be large, open-source models such as Qwen, GPT-3, BERT, and XLNet, or smaller, self-trained models focused on specific domains or tasks.

[0026] The scheduler is responsible for determining which model to use to generate the next token based on preset conditions. The scheduler has real-time analysis capabilities, enabling it to dynamically select the most suitable model service based on model feedback, prediction results, and the current inference context. The method of this application can implement the functions of the scheduler through a computer program.

[0027] The communication interface allows for information exchange between models and between models and the scheduler. This interface supports efficient, low-latency data transmission, ensuring that models can quickly respond to scheduler requests.

[0028] The API gateway provides a unified service interface to the outside world, distributes inference requests to the scheduler internally, and returns the scheduler's decision results to the user.

[0029] See Figure 1 One embodiment of this application provides a method for collaborative inference of multiple heterogeneous models, including the following steps:

[0030] Step S101: Receive the reasoning request and prompt words submitted by the client;

[0031] Specifically, the client submits an inference request and a prompt word through the API gateway. The inference request includes the text to be inferred and possible contextual information. The API gateway forwards the inference request to the scheduler module. The scheduler module receives the inference request and prompt word submitted by the client and implements the corresponding function using the collaborative inference method of multiple heterogeneous models in this embodiment. Optionally, there can be one or more schedulers, such as multiple heterogeneous models with scheduling modules installed, or a separately configured scheduler module.

[0032] Step S102: Based on the context information of the inference request and preset conditions, select multiple matching heterogeneous models to perform token prediction simultaneously.

[0033] In natural speech processing, contextual information in an inference request refers to the textual information surrounding a specific word or phrase, including the words preceding (overture) and following (context) the target word. This information is crucial for understanding the meaning of the target word, disambiguating, and performing semantic reasoning. Preset conditions are conditions pre-defined in the scheduler algorithm, including but not limited to: the model's expertise or domain of strength, inference cost and resource consumption, response time and real-time requirements, the probability distribution and confidence level of predicted tokens, and the labeling of specific tokens. Based on the current inference request contextual information and preset conditions, the scheduler selects multiple heterogeneous models to simultaneously perform token prediction. Specifically, each inference node (i.e., model service) can declare one or more labels to indicate the types of reasoning the model excels at, such as: news facts, security knowledge, medical diagnosis, mathematical logic, code generation, etc. If the label information indicating the domain of expertise of the inference request matches the labels declared by multiple heterogeneous models, the inference request is sent to the multiple heterogeneous models. The multiple heterogeneous models receiving the inference request simultaneously perform inference prediction, with each model generating no more than a predetermined number of tokens. If the inference request does not contain label information indicating the domain of expertise of the model, the scheduler uses its own trained small language model to determine which capability labels are needed based on the prompt. For example, the scheduler uses a trained language model for classification. This language model predicts the classification based on the prompt. If the prompt is "Please provide a weekly meal plan for a 70-year-old diabetic patient," it should predict the classification as "medical health." The inference request is then sent to the heterogeneous models corresponding to the declared labels that match the classification results. The multiple heterogeneous models simultaneously perform inference prediction, with each model generating no more than a predetermined number of tokens. It should be noted that token generation can stop before reaching the predetermined number, meaning that generation can be completed ahead of schedule. In this case, the scheduler determines through its own algorithm that a complete sentence has been generated and no further token generation is needed.

[0034] Step S103: Select the best prediction result from all the prediction results of the heterogeneous models, and concatenate the best prediction result with the prompt word. If the concatenation result does not complete the reasoning, submit the concatenation result as a new prompt word to the multiple matching heterogeneous models and continue iterative reasoning until the reasoning is completed.

[0035] First, record the generation of each model. The probability value of the token with the highest probability during the process of selecting tokens. The probability value of the selected token For example, if the prompt is "What is the capital of France?", and the model predicts the generated text "The capital of France is Paris", and further assumes the first token is "France", the second token is "of", the third token is "capital", and the fourth token is "Paris", then after generating the third token, the fourth token could be "Paris", "Beijing", or "London", with varying probabilities. Assuming "London" has the highest probability, the probability value of this token, "London", would be denoted as... The model ultimately selected "Paris" as the fourth token, and the probability value of "Paris" is recorded as follows: .

[0036] Next, record the first to the last generation of each model. The average probability of the token with the highest probability during the process of generating tokens. and the average probability of the selected token For example: Two tokens have been generated, and the probability of the token selected by the scheduler algorithm is... =40% =60%, while the actual probability of the highest probability token is 60%. =90%, =90%, then =50% is the first two probabilities , The average value, =90% is the first two probabilities , The average value. Now, to generate a third token, if... =20%, =60%, which can be calculated as follows: =40%, which is the average of 40%, 60%, and 20%. It can also be calculated that... =80%, which is the average of 90%, 90%, and 60%, and so on.

[0037] Next, it is determined whether one of the following conditions is met. If any condition is met, the model stops generating tokens and ends the current round of inference and prediction:

[0038] Condition 1: Deviation Reaching the set threshold (experienced value is 10%).

[0039] Condition 2: Deviation Reaching the set threshold (experienced value is 10%).

[0040] Condition 3: Deviation Reaching the set threshold (experienced value is 15%)

[0041] Condition 4: A predetermined number of tokens have been generated;

[0042] Condition 5: The model has marked this inference as complete;

[0043] It should be noted that token generation will only stop for the model that meets the above conditions. If other model inference services have not reached the stopping condition, token generation will continue until the preset number is reached, or if a model inference service determines through its own algorithm that a complete sentence has been generated and token generation is no longer needed, token generation will also stop.

[0044] Next, the average deviation of each heterogeneous model under conditions 1, 2 and 3 is calculated, and the prediction result of the model with the smallest average deviation is taken as the best prediction result.

[0045] For example, consider two heterogeneous models, Model 1 and Model 2. Model 1 has a deviation of 10% in condition 1, 5% in condition 2, and 15% in condition 3. Therefore, the average deviation of Model 1 is (10% + 5% + 15%) / 3 = 15%. Model 2 has a deviation of 5% in condition 1, 5% in condition 2, and 10% in condition 3. Therefore, the average deviation of Model 2 is (5% + 5% + 10%) / 3 = 6.67%. Thus, Model 2 is the best prediction result.

[0046] Finally, the best prediction result is concatenated with the prompt word. If the concatenation result is marked as completed, the prediction result is returned to the client. If the concatenation result has not completed inference, the concatenation result is submitted as a new prompt word to the multiple matching heterogeneous models, and the iterative inference continues until the inference is completed.

[0047] For example, if the prompt is "What is the capital of France?", assuming it will generate "The capital of France is Paris", and the best prediction result in the first step generates "The capital of France", then the prompt and the best prediction result in the first step are concatenated to get "What is the capital of France? The capital of France". At this point, the generation is not complete, so this concatenated result is submitted to the scheduler again as a new prompt, and the reasoning continues from step S101. Assuming the second step generates "It is Paris", and the generation is complete, then "It is Paris" and the prompt "What is the capital of France? The capital of France" are directly concatenated to get the final result "What is the capital of France? The capital of France is Paris". The model reasoning service uses its built-in algorithm to identify whether a complete sentence has been generated to determine whether generation is complete.

[0048] The above embodiments of this application innovatively integrate multiple models with different architectures into one inference system, supporting collaborative inference of heterogeneous models, giving full play to the advantages of each model, dynamically selecting the most suitable model for token generation according to preset conditions, improving overall inference performance and efficiency, and enhancing the model's ability to perform tasks in specific domains.

[0049] See Figure 2 Another embodiment of the present invention provides an apparatus 200 for collaborative reasoning of multiple heterogeneous models, including a receiving module 201, a scheduling module 202 and a reasoning module 203. The apparatus 200 is capable of executing the method for collaborative reasoning of multiple heterogeneous models in the method embodiment.

[0050] Specifically, device 200 includes:

[0051] The receiving module 201 is used to receive the reasoning request and prompt words submitted by the client;

[0052] The scheduling module 202 is used to select multiple matching heterogeneous models to perform token prediction simultaneously based on the context information of the inference request and preset conditions.

[0053] The inference module 203 is used to select the best prediction result from the prediction results of all heterogeneous models, and concatenate the best prediction result with the prompt word. If the concatenation result does not complete the inference, the concatenation result is submitted as a new prompt word to the multiple matching heterogeneous models, and the inference continues iteratively until the inference is completed.

[0054] Furthermore, the preset conditions include at least one of the following: the model's expertise or area of ​​expertise, inference cost and resource consumption, response time and real-time requirements, probability distribution and confidence level of the predicted token, and the labeling of a specific token.

[0055] Furthermore, the scheduling module 202 is also used to: if the label information indicating the domain of expertise of the model in the inference request matches the labels declared by multiple heterogeneous models, then the inference request is sent to the multiple heterogeneous models, and the multiple heterogeneous models that receive the inference request simultaneously perform inference prediction, with each model generating no more than a predetermined number of Tokens.

[0056] Furthermore, the scheduling module 202 is also used to: if the inference request does not contain label information indicating the domain that the model is good at, then the prompt word is predicted and classified by the language model, and the inference request is sent to multiple heterogeneous models corresponding to multiple declaration labels that match the classification results. Multiple heterogeneous models perform inference prediction at the same time, and each model generates no more than a predetermined number of tokens.

[0057] Furthermore, the inference module 203 is also used to: record the number of times each model generates the first... The probability value of the token with the highest probability during the process of selecting tokens. The probability value of the selected token Record the first to the last generation of each model. The average probability of the token with the highest probability during the process of generating tokens. and the average probability of the selected token If one of the following conditions is met, the model will stop generating tokens and end the current round of inference and prediction: Condition 1: Deviation The set threshold has been reached; Condition 2: Deviation The set threshold has been reached; Condition 3: Deviation The set threshold has been reached; Condition 4: A predetermined number of tokens have been generated; Condition 5: The model has marked this inference as ended; Calculate the average deviation of each heterogeneous model in conditions 1, 2 and 3 respectively, and take the prediction result of the model with the smallest average deviation as the best prediction result.

[0058] It should be noted that the technical solutions corresponding to the device 200 for collaborative reasoning of multiple heterogeneous models provided in this embodiment, which can be used to execute various method embodiments, are similar in implementation principle and technical effect to the methods, and will not be repeated here.

[0059] See Figure 3 Another embodiment of the present invention provides a schematic diagram of the structure of an electronic device 300, which is used to implement the method of collaborative inference of multiple heterogeneous models in the method embodiment. The electronic device 300 in the embodiments of the present invention may include, but is not limited to, a PC and a server. Figure 3The electronic device 300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0060] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing device 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0061] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0062] The above description is merely a preferred embodiment of the present invention. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to the specific combination of the above-described technical features, but should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A method for collaborative reasoning using multiple heterogeneous language models, characterized in that, include: Receive reasoning requests and prompts submitted by the client; Based on the context information of the inference request and preset conditions, multiple matching heterogeneous language models are selected to simultaneously perform token prediction, including: if the label information in the inference request indicating the model's area of ​​expertise matches the labels declared by multiple heterogeneous language models, then the inference request is sent to the multiple heterogeneous language models, and the multiple heterogeneous language models receiving the inference request simultaneously perform inference prediction, with each model generating no more than a predetermined number of tokens; if the inference request does not contain label information indicating the model's area of ​​expertise, then the prompt word is predicted and classified by the language model, and the inference request is sent to the heterogeneous language models corresponding to the multiple declared labels that match the classification results, and the multiple heterogeneous language models simultaneously perform inference prediction, with each heterogeneous language model generating no more than a predetermined number of tokens; the preset conditions include at least one of the following: the model's professional knowledge or area of ​​expertise, inference cost and resource consumption, response time and real-time requirements, probability distribution and confidence level of predicted tokens, and the label of a specific token; Record the generation of each heterogeneous language model The probability value of the token with the highest probability during the process of selecting tokens. The probability value of the selected token Record the first to the last generated by each heterogeneous language model. The average probability of the token with the highest probability during the process of generating tokens. and the average probability of the selected token If any of the following conditions are met, the heterogeneous language model will stop generating tokens and end the current round of inference and prediction: Condition 1: Deviation The set threshold has been reached; Condition 2: Deviation The set threshold has been reached; Condition 3: Deviation The set threshold has been reached; Condition 4: A predetermined number of tokens have been generated; Condition 5: The model has marked this inference as complete; Calculate the average deviation of each heterogeneous language model under conditions 1, 2, and 3 respectively, and take the prediction result of the model with the smallest average deviation as the best prediction result. Concatenate the best prediction result with the prompt word. If the concatenation result does not complete the reasoning, submit the concatenation result as a new prompt word to the multiple matching heterogeneous language models and continue iterative reasoning until the reasoning is completed.

2. A device for collaborative reasoning using multiple heterogeneous language models, characterized in that... include: The receiving module is used to receive reasoning requests and prompts submitted by the client; The scheduling module is used to select multiple matching heterogeneous language models to simultaneously perform token prediction based on the context information of the inference request and preset conditions. This includes: if the label information in the inference request indicating the model's area of ​​expertise matches the labels declared by multiple heterogeneous language models, then the inference request is sent to the multiple heterogeneous language models, and the multiple heterogeneous language models receiving the inference request simultaneously perform inference prediction, with each heterogeneous language model generating no more than a predetermined number of tokens; if the inference request does not contain label information indicating the model's area of ​​expertise, then the prompt word is predicted and classified using a language model, and the inference request is sent to the heterogeneous language models corresponding to the multiple declared labels matching the classification results, with the multiple heterogeneous language models simultaneously performing inference prediction, and each heterogeneous language model generating no more than a predetermined number of tokens; the preset conditions include at least one of the following: the model's professional knowledge or area of ​​expertise, inference cost and resource consumption, response time and real-time requirements, the probability distribution and confidence level of the predicted tokens, and the labeling of a specific token. The inference module is used to record the generation of the first iteration by each heterogeneous language model. The probability value of the token with the highest probability during the process of selecting tokens. The probability value of the selected token Record the first to the last generated by each heterogeneous language model. The average probability of the token with the highest probability during the process of generating tokens. and the average probability of the selected token If any of the following conditions are met, the heterogeneous language model will stop generating tokens and end the current round of inference and prediction: Condition 1: Deviation The set threshold has been reached; Condition 2: Deviation The set threshold has been reached; Condition 3: Deviation The set threshold has been reached; Condition 4: A predetermined number of tokens have been generated; Condition 5: The model has marked this inference as complete; Calculate the average deviation of each heterogeneous language model under conditions 1, 2, and 3 respectively, and take the prediction result of the model with the smallest average deviation as the best prediction result. Concatenate the best prediction result with the prompt word. If the concatenation result does not complete the reasoning, submit the concatenation result as a new prompt word to the multiple matching heterogeneous language models and continue iterative reasoning until the reasoning is completed.

Images