Large language model reasoning capability method, device, equipment and computer program product

Abstract

The invention provides a big language model reasoning capability method, device and equipment and a computer program product, and relates to the technical field of computers and internet. The method comprises the steps of obtaining a single-language original question; determining a plurality of translation languages; translating the single-language original question through a plurality of translation languages to obtain a plurality of language version questions, the plurality of language version questions including the single-language original question; reasoning is conducted on the multiple language version questions through a large language model, multiple candidate answers are obtained, and one language version question corresponds to one candidate answer; and integrating the plurality of candidate answers to obtain a target answer of the single-language original question. According to the embodiment of the invention, the big language model reasoning capability can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer and Internet technologies, and in particular to a method, device, electronic device, computer-readable storage medium, and computer program product for large language model reasoning capability. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present disclosure that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] Large language model reasoning, a popular technology in the current field of artificial intelligence, demonstrates strong potential in scenarios such as natural language processing, intelligent question answering, and content generation. However, the inference results of existing large language models suffer from significant instability and inaccuracy, severely limiting their reliability in practical applications.

[0004] Therefore, how to improve the accuracy of the reasoning process has become a key technical challenge that needs to be solved urgently. Summary of the Invention

[0005] The present disclosure aims to provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for large language model reasoning, which can improve the accuracy of large language model reasoning.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0007] An embodiment of the present disclosure provides a method for reasoning with a large language model, comprising: obtaining an original question in a single language; determining multiple translated languages; translating the original question in the single language into the multiple translated languages ​​to obtain multiple language version questions, wherein the multiple language version questions include the original question in the single language; reasoning on the multiple language version questions separately using a large language model to obtain multiple candidate answers, wherein one candidate answer corresponds to one language version question; and integrating the multiple candidate answers to obtain a target answer to the original question in the single language.

[0008] In some embodiments, integrating the multiple candidate answers to obtain a target answer to the original question in a single language includes: selecting the candidate answer with the highest frequency of occurrence from the multiple candidate answers as the target answer to the original question in a single language.

[0009] In some embodiments, the multiple candidate answers are integrated to obtain the target answer of the single-language original question, including: setting a weight for each language according to corpus richness of each language in the large language model; performing weighted processing on the occurrence frequency of the candidate answers corresponding to each language to obtain a weighted frequency; and selecting the candidate answer with the highest weighted frequency as the target answer.

[0010] In some embodiments, the multiple candidate answers are integrated to obtain the target answer of the single-language original question, including: performing vectorization processing on the multiple candidate answers to obtain an answer semantic vector corresponding to each candidate answer; performing clustering processing on the multiple answer semantic vectors to obtain multiple answer clusters; selecting a cluster with the most answer semantic vectors as a target cluster from the multiple answer clusters; and selecting a centroid of the target cluster as the target answer of the single-language original question or selecting an answer closest to an average semantic vector of the target cluster as the target answer of the single-language original question.

[0011] In some embodiments, the multiple candidate answers are integrated to obtain the target answer of the single-language original question, including: performing judgment sorting on the multiple candidate answers by the large language model; and determining the target answer of the single-language original question from the multiple candidate answers according to the judgment sorting result.

[0012] In some embodiments, the multiple translation languages are determined, including: determining a domain and a category of the single-language original question; determining a candidate language set according to the domain and the category of the single-language original question; and determining a language with a corpus richness of N from the candidate language set as the translation language according to the corpus richness of different languages in the large language model, where N is an integer greater than or equal to 1.

[0013] In some embodiments, the method further includes: calculating a marginal benefit of each translation language on inference accuracy of the large language model; and periodically removing a language with a marginal benefit lower than a target threshold from the candidate language set according to the marginal benefit of each translation language on inference accuracy of the large language model.

[0014] In some embodiments, obtaining the candidate answer corresponding to the single-language original question through reasoning of the large language model on the single-language original question comprises: performing complexity evaluation on the single-language original question to obtain a complexity evaluation value of the single-language original question; adjusting a sampling parameter of the large language model and performing load scheduling on the large language model according to the complexity evaluation value of the single-language original question; and reasoning the single-language original question through the large language model after the sampling parameter is adjusted and the load is scheduled to obtain the candidate answer corresponding to the single-language original question.

[0015] The embodiment of the present disclosure provides a large language model reasoning capability device, comprising: a question obtaining module, a language determining module, a translation module, a reasoning module and an answer integrating module.

[0016] The question obtaining module is configured to obtain a single-language original question; the language determining module is configured to determine a plurality of translation languages; the translation module is configured to translate the single-language original question into a plurality of language version questions through the plurality of translation languages, wherein the plurality of language version questions comprise the single-language original question; the reasoning module is configured to reason the plurality of language version questions through a large language model respectively to obtain a plurality of candidate answers, wherein one language version question corresponds to one candidate answer; and the answer integrating module is configured to integrate the plurality of candidate answers to obtain a target answer of the single-language original question.

[0017] The embodiment of the present disclosure provides an electronic device, comprising: a memory and a processor; the memory is configured to store computer program instructions; and the processor is configured to invoke the computer program instructions stored in the memory to implement the large language model reasoning capability method.

[0018] The embodiment of the present disclosure provides a computer readable storage medium having computer program instructions stored thereon, and the computer program instructions implement the large language model reasoning capability method.

[0019] The embodiment of the present disclosure provides a computer program product or a computer program, which comprises computer program instructions stored in a computer readable storage medium. The computer program instructions are read from the computer readable storage medium, and a processor executes the computer program instructions to implement the large language model reasoning capability method.

[0020] The method, device, electronic device, computer readable storage medium and computer program product for large language model inference capability provided by the embodiments of the present disclosure can obtain candidate answers corresponding to multiple reasoning paths by inputting multiple language version questions, and then aggregate the candidate answers under the multiple reasoning paths to obtain a final answer. The method can improve the accuracy and robustness of large language model inference, improve the credibility of large model inference results, and enhance the practicality of large language model in high reliability scenarios.

[0021] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0023] Figure 1 A scene schematic diagram of a large language model inference capability method or a large language model inference capability device that can be applied to the embodiments of the present disclosure is shown.

[0024] Figure 2 is a flowchart of a large language model inference capability method according to an exemplary embodiment.

[0025] Figure 3 is a flowchart of an answer integration method according to an exemplary embodiment.

[0026] Figure 4 is a flowchart of an answer integration method according to an exemplary embodiment.

[0027] Figure 5 is a flowchart of an answer integration method according to an exemplary embodiment.

[0028] Figure 6 is a flowchart of a translation language determination method according to an exemplary embodiment.

[0029] Figure 7 is a flowchart of a translation language adjustment method according to an exemplary embodiment.

[0030] Figure 8 is a flowchart of a resource configuration method for a large model inference process according to an exemplary embodiment.

[0031] Figure 9is a schematic diagram of a large model inference framework according to an example embodiment.

[0032] Figure 10 is a block diagram of a large language model inference capability apparatus according to an example embodiment.

[0033] Figure 11 A structural schematic diagram of an electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0034] Example embodiments now will be described more fully hereinafter with reference to the accompanying drawings. Example embodiments, however, can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example embodiments to those skilled in the art. Like reference numerals refer to like elements throughout the figures, and descriptions of the same elements will be omitted from the descriptions of the figures.

[0035] Those skilled in the art know that the embodiments of the present disclosure can be a system, apparatus, device, method or computer program product. Therefore, the present disclosure can be embodied in the form of a complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0036] The features, structures or characteristics described in the present disclosure can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present disclosure. One skilled in the relevant art will recognize, however, that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0037] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0038] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain principles of the present disclosure. In the drawings:

[0039] The flowcharts shown in the drawings are only illustrative and do not necessarily include all contents and steps, nor are they necessarily executed in the order described. For example, some steps can be further divided, and some steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0040] In the description of the present disclosure, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this document only describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean that A exists alone, A and B exist together, and B exists alone. In addition, "one or more" means one or more, and "multiple" means two or more. "First", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. do not necessarily mean different; the terms "include", "contain" and "have" mean open inclusion and mean that in addition to the listed elements / components / etc. there can be other elements / components / etc.

[0041] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in conjunction with the drawings and specific embodiments. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0042] It should be noted that in the technical solutions of the present application, the collection, use, storage, sharing and transfer of user personal information and other processing comply with relevant laws and regulations, and the user needs to be informed and the user's consent or authorization needs to be obtained. When applicable, the user's personal information has been de-identified and / or anonymized and / or encrypted technical processing.

[0043] First, some of the terms involved in the embodiments of the present disclosure will be explained to facilitate understanding by those skilled in the art.

[0044] Large Language Model (LLM): refers to a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks such as text classification, question answering, dialogue, etc., and are an important way to artificial intelligence.

[0045] The foregoing introduces some concepts related to the embodiments of the present disclosure, and the technical features related to the embodiments of the present disclosure are introduced below.

[0046] The example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0047] Figure 1 A scene schematic diagram of a large language model inference capability method or a large language model inference capability device that can be applied to the embodiments of the present disclosure is shown.

[0048] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment provided by an example embodiment of the present disclosure.

[0049] As Figure 1 shown, the system architecture 100 can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a communication link medium between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0050] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Among them, the terminal devices 101, 102, 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, wearable devices, virtual reality devices, smart homes, etc.

[0051] The server 105 can be a server that provides various services, such as a background management server that provides support for the devices operated by the user using the terminal devices 101, 102, 103. The background management server can analyze and process received request data, etc., and feed back the processing results to the terminal device.

[0052] The server can be a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, and the like, and the present disclosure does not limit the same.

[0053] The server 105 can obtain, for example, a single-language original question; the server 105 can determine, for example, a plurality of translated languages; the server 105 can translate, for example, the single-language original question into a plurality of language version questions through the plurality of translated languages, wherein the plurality of language version questions include the single-language original question; the server 105 can infer, for example, the plurality of language version questions through a large language model respectively to obtain a plurality of candidate answers, wherein one language version question corresponds to one candidate answer; and the server 105 can integrate, for example, the plurality of candidate answers to obtain a target answer of the single-language original question.

[0054] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above-mentioned system is only illustrative, and the server 105 can be a server of one entity, and can also be composed of multiple servers. According to actual needs, the server 105 can have any number of terminal devices, networks, and servers.

[0055] Figure 2 FIG. 1 is a flowchart of a large language model inference capability method according to an example embodiment. The method provided by the embodiment of the present disclosure can be executed by any electronic device with computing processing capability, for example, the method can be executed by the server or the terminal device in the above-mentioned system, and can also be executed by the server and the terminal device together. In the following embodiments, the server is taken as an example for illustration, but the present disclosure is not limited thereto. Figure 1

[0056] Referring to Figure 2 , the large language model inference capability method provided by the embodiment of the present disclosure can include the following steps.

[0057] Step S202: obtaining a single-language original question.

[0058] The single-language original question mentioned above can refer to an initial question expressed in only one language, without translation or language conversion processing.

[0059] The above-mentioned one language can be Chinese, English, German, French, Russian, Italian, or any other language, and the present disclosure does not limit the same.

[0060] ​The original question can refer to an initial question that is initially proposed, unmodified or unprocessed. For example, a question text directly input by a user without any preprocessing (such as translation). For example, the original question can be "How to fix the computer blue screen?"

[0061] Step S204, determining a plurality of translation languages.

[0062] In some embodiments, the plurality of translation languages can be extended for the above-mentioned original question, so as to translate the original question through the translation languages.

[0063] In some embodiments, the plurality of translation languages can be randomly determined for the original question, or can be determined for the original question according to a field (for example: natural field, scientific field, literary field, historical field, etc.) or category (for example: computing category, reasoning category, medical category, etc.) of the original question, or can be determined for the original question according to corpus information richness in a large language model, and the present application does not limit this.

[0064] In some embodiments, the candidate language set can be automatically generated according to a multi-language support range of a target LLM, and then a language with rich resources (such as English, Chinese) is selected as a translation language, and a low-resource language (such as Vietnamese, Swahili) is supplemented as a translation language, so as to ensure the breadth and quality of language coverage. Among them, the rich resource refers to a language (for example, a language with corpus information greater than a threshold one) with relatively rich corpus information in the current large language model; the low-resource language refers to a language (for example, a language with corpus information less than a threshold two) with less corpus information in the large language model.

[0065] In some embodiments, a language diversity degree index (such as Jaccard distance) can also be set to enhance the difference between languages and avoid redundant language selection. Among them, the above-mentioned language diversity degree index is used to measure the difference between two languages. One of the two languages with smaller language difference can be excluded.

[0066] Step S206, translating the single-language original question through the plurality of translation languages to obtain a plurality of language version questions, wherein the plurality of language version questions comprises the single-language original question.

[0067] In some embodiments, the original question can be translated according to the above-mentioned translation languages by using an API (Application Programming Interface, application programming interface) service or an open-source translation model.

[0068] In some embodiments, a semantic consistency checking mechanism can be introduced to verify the semantic fidelity before and after translation. For example, BLEURT (BLEU-based evaluation metric for pre-trained models) or BERTScore (BERT-based scoring metric) can be used to detect the semantic fidelity before and after translation, and automatically filter or retranslate distorted versions.

[0069] In some embodiments, each multilingual version of the question can be independently called to a large language model for inference to generate diversified problem-solving paths and candidate answers. The core goal is to use language expression differences to drive different thinking paths to improve result diversity and accuracy.

[0070] In some embodiments, different language version questions can call different large language models, or the same large language model, which is not limited by the present application.

[0071] The above process is responsible for automatically converting the input single-language original question into multiple language versions, aiming to introduce language diversity to enrich the subsequent large language model (LLM) inference path. Language differences can guide the large language model to activate different inference knowledge, thereby improving the robustness and accuracy of the overall inference.

[0072] Step S208, the multiple language version questions are respectively inferred by a large language model to obtain multiple candidate answers, wherein one language version question corresponds to one candidate answer.

[0073] Step S210, the multiple candidate answers are integrated to obtain the target answer of the single-language original question.

[0074] The above method can reduce the ambiguity or data bias that may exist in a single language through the complementarity of multilingual input.

[0075] In some embodiments, the multiple candidate answers output by the multilingual inference engine module can be integrated, and a consistency analysis result can be provided as a decision confidence index.

[0076] In some embodiments, the candidate answer with the highest frequency of occurrence can be selected from the multiple candidate answers as the target answer of the single-language original question. For example, for deterministic tasks, the candidate answer with the highest frequency of occurrence can be selected from the multiple candidate answers as the target answer of the single-language original question. Deterministic tasks can refer to questions whose correct answers are explicit, unique, and objective, usually based on facts, data, or fixed rules, rather than subjective or open-ended answers. For example, mathematical problems, questions following natural laws, etc.

[0077] In some embodiments, a plurality voter can be used for deterministic tasks. Specifically, the frequency of the final answer generated by each language version can be counted; the answer with the highest frequency is selected as the final output, and if there is a tie, a language weight mechanism can be introduced (for example, English and Chinese have a high proportion of weight) to resolve ambiguity.

[0078] In some embodiments, a fault tolerance threshold (such as the difference in votes needs to exceed 2) can be configured to enhance robustness. When voting or counting high-frequency answers, the fault tolerance threshold is a minimum difference value set by a person to determine whether the current screening result is reliable. Only when the difference between the highest number of votes and other answers exceeds the threshold, the system adopts the answer, otherwise it is considered that the result may be ambiguous or noisy.

[0079] The above scheme can solve the problems of single reasoning path, unstable results, and insufficient reliability of large language models in complex reasoning tasks. Existing models are easily affected by prompt methods or input bias, leading to inconsistent answers or reasoning failure. The method improves the accuracy and robustness of reasoning through multi-path reasoning and answer aggregation mechanism, and enhances the practicality of the model in high reliability scenarios.

[0080] The above method obtains candidate answers corresponding to multiple reasoning paths by inputting multiple language version questions, and then aggregates the candidate answers under the multiple reasoning paths to obtain the final answer. This method can improve the accuracy and robustness of large language model reasoning, improve the reliability of large model reasoning results, and enhance the practicality of large language model in high reliability scenarios.

[0081] Figure 3 is a flowchart of an answer integration method according to an example embodiment.

[0082] Reference Figure 3 The above answer integration method can include the following steps.

[0083] Step S302, set weights for each language according to the corpus richness of each language in the large language model.

[0084] The above "corpus richness" refers to the sufficiency of resources of a certain language in the training data of the large language model (LLM).

[0085] In some embodiments, the corpus richness of each language in each large language model can be known in advance.

[0086] In some embodiments, the weight of a language can be proportional to the corpus richness of the language in the large language model. The richer the corpus of a language, the higher the corresponding weight, and the poorer the corpus of a language, the lower the corresponding weight. It should be noted that the weight of each language can be equal to 1.

[0087] Step S304, the frequency of occurrence of each language corresponding candidate answer is weighted to obtain the weighted frequency.

[0088] In some embodiments, the same candidate answer can be weighted and summed by the weight corresponding to each language to obtain the weighted frequency of each candidate answer. For example, the weight of language A is a, the weight of language B is b, and the candidate answer of language A and language B is C, then the weighted frequency corresponding to C can be a+b.

[0089] Step S306, the candidate answer with the highest weighted frequency is selected as the target answer.

[0090] The above method can integrate the candidate answers under the plurality of reasoning paths to obtain the final answer. This method can improve the accuracy and robustness of large language model reasoning, improve the credibility of large model reasoning results, and enhance the practicality of large language model in high reliability scenarios.

[0091] Figure 4 is a flowchart of an answer integration method according to an example embodiment.

[0092] Reference Figure 4 The above answer integration method can include the following steps.

[0093] Step S402, vectorizing the plurality of candidate answers to obtain the answer semantic vector corresponding to each candidate answer.

[0094] In some embodiments, the pre-trained model can be used to vectorize the plurality of candidate answers respectively to obtain the answer semantic vector corresponding to each candidate answer.

[0095] For example, the model can be used to encode each candidate answer into an answer semantic vector.

[0096] Step S404, clustering the plurality of answer semantic vectors to obtain a plurality of answer clusters.

[0097] In some embodiments, a clustering method (such as a k-means clustering method) can be used to cluster the plurality of answer semantic vectors to obtain a plurality of (such as two, three, four, etc.) answer clusters.

[0098] Step S406, selecting the cluster with the most answer semantic vectors from the plurality of answer clusters as the target cluster.

[0099] Step S408, selecting the centroid of the target cluster as the target answer of the single language original question or selecting the answer closest to the average semantic vector of the target cluster as the target answer of the single language original question.

[0100] In some embodiments, the answer corresponding to the answer semantic vector closest to the centroid in the target cluster can be selected as the target answer.

[0101] In some embodiments, the average vector of all answer semantic vectors in the target cluster can be calculated, and then the answer corresponding to the answer semantic vector closest to the average vector is selected as the target answer.

[0102] The above method can integrate the candidate answers under the plurality of reasoning paths to obtain the final answer. This method can improve the accuracy and robustness of large language model reasoning, improve the credibility of large model reasoning results, and enhance the practicality of large language model in high reliability scenarios.

[0103] Figure 5 is a flowchart of an answer integration method according to an example embodiment.

[0104] Referring to Figure 5 The above answer integration method can include the following steps.

[0105] Step S502, judging and sorting a plurality of candidate answers by a large language model.

[0106] In some embodiments, a large language model prompt can be generated according to the above plurality of candidate answers, so that the large language model makes a second judgment on the above plurality of candidate answers.

[0107] For example, a prompt can be designed: please judge which one of answers A, B, and C is the answer to the XXX question.

[0108] The design of the above prompt is not limited by the present application.

[0109] Step S504, determining the target answer of the single language original question from the plurality of candidate answers according to the judgment and sorting result.

[0110] The above method can integrate the candidate answers under the plurality of reasoning paths to obtain the final answer. This method can improve the accuracy and robustness of large language model reasoning, improve the credibility of large model reasoning results, and enhance the practicality of large language model in high reliability scenarios.

[0111] Figure 6 is a flowchart of a translation language determination method according to an example embodiment.

[0112] Referring to Figure 6 The above translation language determination method can include the following steps.

[0113] Step S602, determining the domain and category of the single language original question.

[0114] In some embodiments, the inference performance of different languages in different fields or categories can be different. For example, for a mathematical field problem, the answer accuracy obtained by asking in Chinese can be higher.

[0115] Therefore, different language sets can be set for different fields and categories according to the performance of different languages in different fields or categories in advance. The language set can be dynamically adjusted according to the inference performance of each language later.

[0116] Step S604 determines a candidate language set according to the field and category of the single language original question.

[0117] In some embodiments, the language set corresponding to the single language original question can be determined as the candidate language set according to the field and category of the single language original question.

[0118] Step S606 determines the top N languages in the candidate language set as the translation languages according to the corpus richness of different languages in the large language model, where N is an integer greater than or equal to 1.

[0119] In some embodiments, the top N languages in the candidate language set can also be determined as the translation languages of the single language original question in combination with the corpus richness in the large language model.

[0120] Through the above method, some translation languages that perform well in the inference process of the original question can be screened out to improve the inference ability of the large language model.

[0121] It can be understood that the above Figures 3-6 The answer integration method provided can be used in combination. If Figure 3 the answer obtained by the scheme shown in is not ideal or cannot give a unique answer, the scheme shown in Figure 4 can be used to integrate the answer, and so on.

[0122] Figure 7 is a flowchart of a translation language adjustment method according to an example embodiment.

[0123] In some embodiments, therefore, different language sets can be set for different fields and categories according to the performance of different languages in different fields or categories in advance. The language set can be dynamically adjusted according to the inference performance of each language later.

[0124] In the following, the present embodiment will illustrate how to adjust the translation languages in a field or category.

[0125] ReferenceFigure 7 The translation language adjustment method can include the following steps.

[0126] Step S702, calculate the marginal benefit of each translation language on the inference accuracy of the large language model.

[0127] In some embodiments, the increase in inference accuracy of the large language model when each additional translation language is added as input can be calculated to determine the marginal benefit of each translation language on the inference accuracy of the large language model.

[0128] Step S704, according to the marginal benefit of each translation language on the inference accuracy of the large language model, periodically remove languages in the candidate language set whose marginal benefit is lower than the target threshold.

[0129] In some embodiments, a lightweight recommendation model can be trained based on historical experimental data to predict the optimal language subset combination suitable for the current problem in the field or category; then automatically remove poorly performing languages (such as a language with a high long-term error rate) to narrow the inference range and improve efficiency.

[0130] The above methods can quantify the effectiveness of each language on the inference accuracy of the large language model, and preferentially retain the translation language that has the most significant improvement in accuracy.

[0131] Through the above method, the language that shares a large inference accuracy for the original problem in a certain field or category can be accurately and specifically determined.

[0132] In summary, the above method can quantify the contribution of each language and preferentially retain the language translation that has the most significant improvement in accuracy.

[0133] Figure 8 is a flowchart of a resource configuration method of a large model inference process according to an example embodiment.

[0134] Reference Figure 8 The resource configuration method in the large model inference process can include the following steps.

[0135] Step S802, complexity evaluation of the single language original problem is performed to obtain a complexity evaluation value of the single language original problem.

[0136] The complexity evaluation can include at least one of quantitative evaluation of syntactic complexity (such as sentence length, nesting structure, syntactic complexity), lexical complexity (rare word frequency, professional term density, lexical diversity), semantic complexity (depth of implied logic, polysemy, context dependence), symbol density, and syntactic ambiguity, thereby obtaining a complexity evaluation value.

[0137] In some embodiments, the quantitative evaluation values of each aspect of the single-language original question can be weighted and averaged as the complexity evaluation value of the single-language original question.

[0138] Step S804, adjusting the sampling parameters of the large language model according to the complexity evaluation value of the single-language original question and performing load scheduling on the large language model.

[0139] In some embodiments, the sampling parameters of the large language model can include at least one of Temperature, Top-p, Max length, and Beam search width.

[0140] In some embodiments, the large language model can automatically adjust the decoding parameters and computational resource allocation when generating answers according to the complexity of the processed question.

[0141] In some questions, complex questions can reduce the temperature (such as 0.3) to obtain more certain answers; creative questions can increase the temperature (such as 0.7) to increase diversity.

[0142] In some embodiments, technical questions can expand the candidate range (such as top_k=50); simple factual queries can narrow the range (such as top_k=10).

[0143] In some embodiments, complex questions increase the reasoning steps (such as chain-of-thought); simple questions directly affect a single step.

[0144] Step S806, reasoning the single-language original question through the large language model after adjusting the sampling parameters and performing load scheduling to obtain the candidate answers corresponding to the single-language original question.

[0145] The above scheme can automatically adjust the reasoning sampling parameters (such as temperature, top-k, and reasoning rounds) according to the question complexity (such as text length, symbol density, and syntax ambiguity), on the one hand; on the other hand, it can realize load-aware scheduling: complex questions allow more computational resources and parallelism, while simple questions respond quickly.

[0146] In some embodiments, a large language model reasoning function architecture can also be provided, which can be as shown in Figure 9 It can include four core modules: a multi-language question expansion module 901, a multi-language reasoning engine module 902, a decision aggregation module 903, and a performance optimization module 904. Below, the functions of each module will be introduced one by one.

[0147] 1. Multi-language question expansion module 901

[0148] This module is responsible for automatically converting input single-language questions into multiple language versions to generate a multilingual question set, aiming to introduce language diversity to enrich the reasoning paths of subsequent large language models (LLMs). Language differences can guide the model to activate different reasoning knowledge, thereby improving the robustness and accuracy of overall reasoning. Input of this module: single-language original question; output: multilingual version question set, structured as a mapping of language tags and text content.

[0149] As shown in Figure 9 The multilingual question expansion module 901 can include a language selector and a translation engine.

[0150] 1.1. Language selector

[0151] Function description: automatically generate a candidate language set according to the multilingual support range of the target LLM;

[0152] Provide a configurable language priority strategy: prefer languages with abundant resources (such as English and Chinese), and then supplement low-resource languages (such as Vietnamese and Swahili) to ensure the breadth and quality of language coverage;

[0153] Set a language diversity degree index to remove one of the languages with higher relevance to enhance the difference between languages and avoid redundant language selection.

[0154] 1.2. Translation engine

[0155] Integrate high-quality translation services, including API services or open-source translation models.

[0156] Introduce a semantic consistency verification mechanism: detect the semantic fidelity before and after translation, and automatically filter or retranslate distorted versions.

[0157] 2. Multilingual reasoning engine module 902

[0158] Function description: This module independently calls large language models for each multilingual version of the question to generate diversified problem-solving paths and candidate answers. Its core goal is to use language expression differences to drive different thinking paths to improve result diversity and accuracy. Input: multilingual question set. Output: structured candidate answer set, each record contains: language tag, complete reasoning path, final answer, confidence score.

[0159] The above-mentioned multilingual reasoning engine module 902 can include a prompt word generator and a reasoning scheduler.

[0160] 2.1 Prompt word generator

[0161] Prepare specially designed prompt templates for each language to guide LLMs to perform step-by-step reasoning.

[0162] A configurable template management mechanism is provided to automatically adapt the prompt style and format according to different model characteristics.

[0163] Language-level parallel reasoning is implemented using the inference acceleration engine.

[0164] Language-level parallel reasoning is implemented using the inference acceleration engine.

[0165] Task distribution in a multi-GPU (Graphics Processing Unit) environment is supported, significantly improving inference throughput.

[0166] A model quantization mechanism is introduced to reduce inference delay and resource occupation while maintaining accuracy.

[0167] 3. Decision aggregation module 903

[0168] Function description: The multiple candidate answers output by the multi-language inference engine module are integrated, and different aggregation strategies are used to obtain a unified final answer according to deterministic or non-deterministic task types, while providing consistency analysis results as decision confidence indicators. Input: Multi-language candidate answer set. Output: Final unified answer, with consistency score (such as highest vote percentage, average semantic similarity, etc.). The above decision aggregation module 903 can include majority voting rights and a semantic clusterer.

[0169] 3.1. Majority voter (for deterministic tasks)

[0170] The frequency of the final answer generated by each language version is counted.

[0171] The answer with the highest frequency is selected as the final output, and if there is a tie, a language weight mechanism (such as giving higher weight to English and Chinese when the proportion is high) can be introduced to resolve ambiguity.

[0172] A configurable fault tolerance threshold (such as a vote difference of more than 2) can be used to enhance robustness.

[0173] 3.2. Semantic clusterer (for generation class, subjective class tasks)

[0174] Each answer is encoded into a semantic vector.

[0175] K-Means clustering is used to extract semantic consistent answer clusters, and the answer represented by the largest cluster center or average vector is selected as the final result.

[0176] Confidence weighted clustering is combined to improve aggregation effect.

[0177] 4. Performance optimization module 904

[0178] Function Description: This module realizes the resource utilization and performance optimization of the overall system. By analyzing historical running data and real-time feedback, it dynamically adjusts the language combination and computing resource configuration, thereby reducing computing consumption and response time while ensuring accuracy. Input: Current problem features and historical reasoning statistics. Output: Optimized language combination strategy and computing resource allocation parameters. The above performance optimization module 904 can include a language combination optimizer and a resource allocator.

[0179] 4.1 Language Combination Optimizer

[0180] Train a lightweight recommendation model based on historical experimental data to predict the optimal language subset combination for the current problem.

[0181] Automatically exclude poorly performing languages (e.g., a language with a high long-term error rate) to narrow the reasoning range and improve efficiency.

[0182] Support online fine-tuning: dynamically update through user feedback / verification results to improve the adaptability of the recommendation strategy.

[0183] 4.2 Resource Allocator

[0184] Automatically adjust reasoning sampling parameters (e.g., temperature, top-k, reasoning rounds) based on problem complexity (e.g., text length, symbol density, syntax ambiguity).

[0185] Implement load-aware scheduling: allow more computing resources and parallelism for complex problems, and fast response for simple problems.

[0186] In combination with the above large language model reasoning function architecture, the present application also provides a large language model reasoning method. The method can include the following steps.

[0187] Step 1: Multi-language problem expansion

[0188] Input: Single-language original problem. Output: Multi-language problem set (e.g., N language versions such as Chinese, English, French, Japanese, etc., with language labels).

[0189] Step 1.1. Language Selection

[0190] Retrieve the list of languages supported by the target large language model (LLM). According to the configuration strategy, prefer high-resource languages (e.g., English, Chinese) to ensure semantic clarity and reasoning stability. Supplement some low-resource languages based on high-resource languages to enhance language diversity.

[0191] Support dynamic configuration of language pool: can customize language combination according to task type, runtime limit, prior experience, etc. Generally, 6 language combinations achieve a good balance between accuracy and computing resources.

[0192] Step 1.2. Question Translation

[0193] For each target language, call the corresponding translation engine: Commercial translation API for high-resource languages. Open-source models for low-resource languages or private deployment needs.

[0194] Enable semantic consistency verification mechanism, which can use the following mechanisms:

[0195] Back-translation comparison: Perform forward and reverse translation, and compare the semantic consistency with the original question;

[0196] Confidence assessment: Set a minimum threshold to filter distorted samples. If the translation quality does not meet the standard, trigger automatic retranslation or switch to the backup translation engine.

[0197] Step 1.3 Multi-language question set generation

[0198] Encapsulate each translated question as a structured object, add language identification and coding information, and support subsequent modules for language filtering, weight allocation, etc.

[0199] Step 2. Multi-language reasoning path generation

[0200] Input: Multi-language question set (including language label). Output: Structured reasoning result set (including reasoning path, final answer, language, confidence, etc.

[0201] Step 2.1 Prompt construction

[0202] Configure a dedicated prompt template for each language. At the same time, adapt to different LLM format preferences, such as: DeepSeek-Math bias structured Chinese prompt. ChatGLM prefers natural language style prompts. Template parameterization: allows embedding example formats, custom thinking path length, etc.

[0203] Step 2.2 Parallel inference generation

[0204] Use the inference framework to deploy large language model instances, and generate reasoning paths in parallel by language dimension.

[0205] Inference acceleration optimization: Enable model weight quantization to reduce memory overhead. Support mixed precision and batch inference to improve throughput.

[0206] Step 2.3 Answer extraction and storage

[0207] Parse the output text and use regular expressions to extract the answer. Structured storage results field, which can be estimated by LLM output probability or external scorer.

[0208] Step 3. Cross-language answer aggregation decision

[0209] Input: Multilingual candidate answer set (including language tags, reasoning paths, confidence). Output: Final decision answer and consistency score indicators.

[0210] Step 3.1 for deterministic tasks

[0211] Majority voting strategy: Count the number of occurrences of each candidate answer, and select the one with the highest frequency as the final answer. Support weighted voting: English weight = 1.2, Telugu weight = 0.8.

[0212] Flat vote processing mechanism: If the number of votes is the same, prefer to select the answer in high resource language. Or call LLM for secondary reasoning verification or sorting voting.

[0213] Step 3.2 Non-deterministic tasks (such as text summary, opinion integration)

[0214] Semantic clustering: Generate semantic vector of answers. Apply K-Means clustering to obtain main answer clusters. Extract the "centroid" answer or average answer of the largest cluster as the result.

[0215] LLM arbitration mechanism: If the clustering is not clear, construct a multi-candidate prompt and call LLM for judgment and sorting.

[0216] Step 4. Dynamic optimization

[0217] Input: Historical reasoning records (language combinations, answer correctness, time consumption, confidence, etc.). Output: Optimized language selection strategy and resource allocation parameters.

[0218] Step 4.1 Language combination optimization

[0219] Based on historical data, train a lightweight prediction model (such as random forest) to estimate the marginal benefit of language combination on reasoning accuracy.

[0220] Update the combination score table after each round of reasoning, and regularly eliminate underperforming languages.

[0221] You can set a cold start strategy: first use the recommended 6 language combinations, and then optimize according to the result feedback.

[0222] Step 4.2 Resource allocation adjustment

[0223] Complexity assessment (such as length, structure nesting degree, symbol density) is performed for each question, and reasoning resource parameters (such as the number of generated paths K, the number of reasoning times T) are dynamically allocated. Support parallel limit and resource upper limit setting to avoid waste of supercomputing resources.

[0224] The multi-path reasoning mechanism of the method is different from the existing method which relies on fixed prompts or single language input. Through multi-language extended problem input, the potential reasoning space of the model is stimulated, and diversified reasoning paths are systematically constructed, thereby improving the cognitive depth and thinking coverage of the model from the source.

[0225] In addition, the method also introduces majority voting and semantic clustering decision mechanism to aggregate and analyze the candidate answers generated by multiple reasoning paths, effectively improving the accuracy and stability of the reasoning results, and significantly reducing the interference of false judgment rate and random fluctuations on the output results.

[0226] The above embodiments can be applied to various application scenarios for improving the reasoning ability of large language models (LLM), especially when facing complex, multi-step, and high-uncertainty problems. The application scenarios include but are not limited to mathematical problem solving, logical reasoning, question and answer systems, automated decision support, education evaluation, intelligent customer service, etc. The method generates diversified reasoning paths, structures candidate answers, aggregates different reasoning results, and performs consistency judgment, effectively improving the comprehensive performance of LLM in reasoning accuracy, stability, and robustness, especially suitable for AI reasoning tasks with high reliability requirements such as financial risk control, medical diagnosis, judicial assistance, and other key fields.

[0227] It should be particularly pointed out that each step in each embodiment of the above-mentioned large language model reasoning ability method can be crossed, replaced, added, deleted. Therefore, these reasonable permutations and combinations of the large language model reasoning ability method should also belong to the protection scope of the present disclosure, and the protection scope of the present disclosure should not be limited to the embodiments.

[0228] Based on the same inventive concept, the present disclosure also provides a large language model reasoning ability device in the embodiments, as follows. Since the principle of solving problems of the device embodiment is similar to that of the above-mentioned method embodiment, the implementation of the device embodiment can be referred to the implementation of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0229] Figure 10 is a block diagram of a large language model reasoning ability device according to an exemplary embodiment. Referring to Figure 10 The large language model reasoning ability device 1000 provided by the embodiments of the present disclosure can include a question acquisition module 1001, a language determination module 1002, a translation module 1003, a reasoning module 1004, and an answer integration module 1005.

[0230] The problem obtaining module 1001 can be configured to obtain a single-language original question; the language determining module 1002 can be configured to determine a plurality of translation languages; the translation module 1003 can be configured to translate the single-language original question into a plurality of language version questions through the plurality of translation languages, where the plurality of language version questions include the single-language original question; the reasoning module 1004 can be configured to respectively reason the plurality of language version questions through a large language model to obtain a plurality of candidate answers, where one language version question corresponds to one candidate answer; and the answer integrating module 1005 can be configured to integrate the plurality of candidate answers to obtain a target answer of the single-language original question.

[0231] It should be noted that the above problem obtaining module 1001, language determining module 1002, translation module 1003, reasoning module 1004 and answer integrating module 1005 correspond to S202-S210 in the method embodiment, and the above modules and the examples and application scenarios realized by the corresponding steps are the same, but are not limited to the disclosed content of the above method embodiment. It should be noted that the above modules as part of the device can be executed in a computer system such as a group of computer executable instructions.

[0232] In some embodiments, the answer integrating module 1005 can include a frequency screening unit.

[0233] The frequency screening unit can be configured to screen the candidate answer with the highest frequency from the plurality of candidate answers as the target answer of the single-language original question.

[0234] In some embodiments, the answer integrating module 1005 can include a weight setting unit, a weighted processing unit and an answer screening unit.

[0235] The weight setting unit can be configured to set a weight for each language according to the corpus richness of the language in the large language model; the weighted processing unit can be configured to perform weighted processing on the frequency of occurrence of the candidate answer corresponding to each language to obtain a weighted frequency; and the answer screening unit can be configured to select the candidate answer with the highest weighted frequency as the target answer.

[0236] In some embodiments, the answer integrating module 1005 can include a vectorization processing unit, a clustering unit, a target cluster determining unit and a target answer determining unit.

[0237] The vectorization processing unit can be configured to perform vectorization processing on the multiple candidate answers to obtain answer semantic vectors corresponding to the candidate answers; the clustering unit can be configured to perform clustering processing on the answer semantic vectors to obtain multiple answer clusters; the target cluster determination unit can be configured to select, from the multiple answer clusters, a cluster with the largest number of answer semantic vectors as a target cluster; and the target answer determination unit can be configured to select a centroid of the target cluster as the target answer of the single-language original question or select an answer closest to an average semantic vector of the target cluster as the target answer of the single-language original question.

[0238] In some embodiments, the answer integration module 1005 can include an ordering unit and an ordering result screening unit.

[0239] The ordering unit can be configured to perform judgment ordering on the multiple candidate answers by using the large language model, and the ordering result screening unit can be configured to determine the target answer of the single-language original question from the multiple candidate answers according to the judgment ordering result.

[0240] In some embodiments, the language determination module 1002 can include a category determination unit, a candidate language set determination unit, and a translation language determination unit.

[0241] The category determination unit can be configured to determine a domain and a category of the single-language original question, the candidate language set determination unit can be configured to determine a candidate language set according to the domain and the category of the single-language original question, and the translation language determination unit can be configured to determine, from the candidate language set, a language with a top N corpus richness in the large language model as a translation language according to the corpus richness of different languages in the large language model, where N is an integer greater than or equal to 1.

[0242] In some embodiments, the large language model inference capability apparatus 1000 can further include a calculation module and a language elimination module.

[0243] The calculation module can be configured to calculate marginal benefits of each translation language on inference accuracy of the large language model, and the language elimination module can be configured to periodically eliminate a language with a marginal benefit lower than a target threshold from the candidate language set according to the marginal benefits of each translation language on inference accuracy of the large language model.

[0244] In some embodiments, the inference module 1004 can include a complexity evaluation unit, a parameter adjustment unit, and an inference unit.

[0245] The complexity evaluation unit can be configured to evaluate the complexity of the single-language original question to obtain a complexity evaluation value of the single-language original question. The parameter adjustment unit can be configured to adjust the sampling parameter of the large language model according to the complexity evaluation value of the single-language original question and perform load scheduling on the large language model. The inference unit can be configured to perform inference on the single-language original question by using the large language model after the sampling parameter is adjusted and the load scheduling is performed, to obtain a candidate answer corresponding to the single-language original question.

[0246] Since the functions of the apparatus 1000 have been described in detail in the corresponding method embodiments, the present disclosure will not be repeated here.

[0247] The modules and / or sub-modules and / or units described in the embodiments of the present disclosure can be implemented in the form of software or in the form of hardware. The described modules and / or sub-modules and / or units can also be arranged in a processor. In some cases, the names of these modules and / or sub-modules and / or units do not constitute a limitation on the modules and / or sub-modules and / or units themselves.

[0248] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module or a part of a program segment that contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from those shown in the drawings. For example, two blocks that are shown in succession can actually be executed substantially in parallel, and sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams or flowcharts, and combinations of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer program instructions.

[0249] In addition, the above-described drawings are only schematic representations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to limit the purpose. It is readily understood that the processes shown in the above-described drawings do not indicate or limit the time sequence of the processes. In addition, it is readily understood that the processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0250] Figure 11 A structural schematic diagram of an electronic device suitable for implementing the embodiments of the present disclosure is shown. It should be noted that, Figure 11 The electronic device 1100 shown is only an example and should not limit the functions and use range of the embodiments of the present disclosure. The electronic device 1100 shown is only an example and should not limit the functions and use range of the embodiments of the present disclosure.

[0251] like Figure 11 As shown, the electronic device 1100 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1102 or the program loaded from the storage part 1108 into the random access memory (RAM) 1103. Various programs and data required for the operation of the electronic device 1100 are also stored in the RAM 1103. The CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0252] The following components are connected to the I / O interface 1105: an input section 1106 including a keyboard, a mouse, and the like; an output section 1107 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1108 including a hard disk; and a communication section 1109 including a network interface card such as a LAN card or a modem. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. Removable media 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1110 as needed, so that computer programs read therefrom can be installed in the storage section 1108 as needed.

[0253] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes computer program instructions for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1109, and / or installed from a removable medium 1111. When the computer program is executed by the central processing unit (CPU) 1101, the above-mentioned functions defined in the system of the present disclosure are performed.

[0254] It should be noted that the computer-readable storage medium shown in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. In the present disclosure, the computer-readable signal medium can include a data signal carrying computer-readable computer program instructions in a baseband or as part of a carrier wave. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable storage medium other than the computer-readable storage medium, which can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, device or component. The computer program instructions contained on the computer-readable storage medium can be transmitted in any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0255] As another aspect, the present disclosure also provides a computer-readable storage medium, which can be contained in the device described in the above embodiments; or can exist separately without being assembled into the device. The above computer-readable storage medium carries one or more programs, when the one or more programs are executed by the device, the device can realize functions including: obtaining a single language original question; determining a plurality of translation languages; translating the single language original question through the plurality of translation languages to obtain a plurality of language version questions, wherein the plurality of language version questions include the single language original question; reasoning the plurality of language version questions through a large language model respectively to obtain a plurality of candidate answers, wherein one language version question corresponds to one candidate answer; and integrating the plurality of candidate answers to obtain a target answer of the single language original question.

[0256] According to one aspect of the present disclosure, a computer program product or computer program is provided, which includes computer program instructions stored in a computer readable storage medium. The computer program instructions are read from the computer readable storage medium and executed by a processor to implement the method provided in various optional implementation manners of the above-described embodiments.

[0257] Through the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of computer program instructions to make an electronic device (which can be a server or a terminal device, etc.) execute the method according to the embodiments of the present disclosure.

[0258] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the description of the embodiments herein and the disclosure contained herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including other known methods in the art of the present disclosure not specifically disclosed. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0259] It should be understood that the present disclosure is not limited to the detailed structure, drawing manner or implementation method shown herein, but rather the present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the appended claims.

Claims

1. A large language model inference capability method, characterized in that, The method comprises the following steps: acquiring a single-language original question; determining a plurality of translation languages; translating the single-language original question into a plurality of language version questions through the plurality of translation languages, wherein the plurality of language version questions comprise the single-language original question; reasoning the plurality of language version questions through a large language model respectively to obtain a plurality of candidate answers, wherein one language version question corresponds to one candidate answer; integrating the plurality of candidate answers to obtain a target answer of the single-language original question.

2. The method of claim 1, wherein, The integrating the plurality of candidate answers to obtain a target answer of the single-language original question comprises: setting a weight for each language according to corpus richness of each language in the large language model; weighting the occurrence frequency of the candidate answer corresponding to each language to obtain a weighted frequency; selecting the candidate answer with the highest weighted frequency as the target answer.

3. The method of claim 1, wherein, The integrating the plurality of candidate answers to obtain a target answer of the single-language original question comprises: vectorizing the plurality of candidate answers to obtain an answer semantic vector corresponding to each candidate answer; clustering the plurality of answer semantic vectors to obtain a plurality of answer clusters; selecting a cluster with the largest number of answer semantic vectors in the plurality of answer clusters as a target cluster; selecting a centroid of the target cluster as the target answer of the single-language original question or selecting an answer closest to the average semantic vector of the target cluster as the target answer of the single-language original question.

4. The method of claim 1, wherein, The determining a plurality of translation languages comprises: determining a domain and a category of the single-language original question; determining a candidate language set according to the domain and the category of the single-language original question; determining a language with a corpus richness in the top N according to the corpus richness of different languages in the large language model in the candidate language set as the translation language, wherein N is an integer greater than or equal to 1.

5. The method of claim 4, wherein, The method further comprises: calculating a marginal benefit of each translation language on the reasoning accuracy of the large language model; periodically removing a language with a marginal benefit lower than a target threshold from the candidate language set according to the marginal benefit of each translation language on the reasoning accuracy of the large language model.

6. The method of claim 1, wherein, The reasoning the single-language original question through the large language model to obtain a candidate answer corresponding to the single-language original question comprises: performing complexity evaluation on the single-language original question to obtain a complexity evaluation value of the single-language original question; adjusting a sampling parameter of the large language model according to the complexity evaluation value of the single-language original question and performing load scheduling on the large language model; reasoning the single-language original question through the large language model after the sampling parameter is adjusted and the load is scheduled to obtain a candidate answer corresponding to the single-language original question.

7. A large language model reasoning capability apparatus, characterized by, The method comprises the following steps: a question acquisition module configured to acquire a single-language original question; a language determination module configured to determine a plurality of translation languages; A translation module is configured to translate the single-language original question into a plurality of language version questions in the plurality of translation languages, wherein the plurality of language version questions comprises the single-language original question; A reasoning module is configured to reason the plurality of language version questions respectively by the large language model to obtain a plurality of candidate answers, wherein one language version question corresponds to one candidate answer; An answer integration module is configured to integrate the plurality of candidate answers to obtain a target answer of the single-language original question.

8. An electronic device, comprising: Comprise: A memory and a processor; The memory is configured to store computer program instructions; the processor invokes the computer program instructions stored in the memory, and is configured to implement the method according to any one of claims 1-6.

9. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the large language model reasoning method according to any one of claims 1-6.

10. A computer program product comprising computer program instructions stored in a computer readable storage medium, characterized in that, The computer program instructions are executed by the processor to implement the method according to any one of claims 1-6.

Images