A large language reasoning system and method

Abstract

The present invention discloses a large language reasoning system, including: at least one pre-filling calculation unit, at least one generating calculation unit and a cache unit; at least one pre-filling calculation unit is used to pre-fill according to the current reasoning request and historical reasoning resources, obtain the current reasoning data, and send the current reasoning data to the generating calculation unit and the cache unit; at least one generating calculation unit is used to receive the current reasoning data, and use a preset large language model to generate the current result word according to the current reasoning data, generate the current reasoning result according to the current result word and send it to the cache unit, and output the current result word to the user end; the cache unit is used to save the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request. The technical solution provided by the present invention can reduce the amount of calculation in the reasoning process and improve the fluency and continuity of the natural language of the reasoning result.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a large language reasoning system and method. Background Art

[0002] Natural language processing is an important branch of artificial intelligence, which aims to enable computers to understand and generate human language. In recent years, with the development of deep learning technology, the field of natural language processing has made significant progress. Specifically, the large language reasoning model receives questions input by users, and then uses deep learning technology to reason about the questions and finally generates reasoning answers.

[0003] Existing large language model inference methods involve repeated calculations of the same parameters. However, due to the complexity of large language inference models, a single inference may require deep learning inference calculations of hundreds of millions of parameters. Therefore, a large number of repeated calculations will lead to a large waste of computing resources, which in turn slows down the result generation process. At the same time, how to generate continuous and fluent natural language is also a difficulty in current research. Summary of the invention

[0004] The present invention provides a large language reasoning system and method. The reasoning system and method provided by the embodiments of the present invention can improve the serious waste of computing resources and the slow result generation speed. Furthermore, the fluency and continuity of the generated natural language can be improved.

[0005] In a first aspect, an embodiment of the present invention provides a large language reasoning system, including: at least one pre-filled computing unit, at least one generated computing unit and a cache unit;

[0006] The at least one pre-filling computing unit is used to pre-fill according to the current reasoning request and the historical reasoning resources to obtain the current reasoning data, and send the current reasoning data to the generating computing unit and the cache unit, wherein the historical reasoning resources include the historical reasoning requests associated with the current reasoning request and the reasoning data corresponding to the historical reasoning results;

[0007] The at least one generating and calculating unit is used to receive the current reasoning data, and use a preset large language model to generate a current result word according to the current reasoning data, generate a current reasoning result according to the current result word, send it to the cache unit, and output the current result word to the user end;

[0008] The cache unit is used to save the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request.

[0009] In a second aspect, an embodiment of the present invention provides a large language reasoning method, which is applied to the large language reasoning system described in any one of the embodiments of the present invention, including:

[0010] The at least one pre-filling computing unit performs pre-filling according to the current reasoning request and the historical reasoning resources to obtain the current reasoning data, and sends the current reasoning data to the generating computing unit and the cache unit;

[0011] The at least one generating and calculating unit receives the current reasoning data, and generates a current result word according to the current reasoning data using a preset large language model, generates a current reasoning result according to the current result word and sends it to a cache unit, and outputs the current result word to a user terminal;

[0012] The cache unit saves the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request.

[0013] An embodiment of the present invention provides a large language reasoning system and method, the system comprising: at least one pre-filling computing unit, at least one generating computing unit and a cache unit; the at least one pre-filling computing unit is used to pre-fill according to a current reasoning request and historical reasoning resources to obtain current reasoning data, and send the current reasoning data to the generating computing unit and the cache unit, wherein the historical reasoning resources include historical reasoning requests associated with the current reasoning request and reasoning data corresponding to historical reasoning results; the at least one generating computing unit is used to receive the current reasoning data, and use a preset large language model to generate a current result word according to the current reasoning data, generate a current reasoning result according to the current result word, send it to the cache unit, and output the current result word to a user end; the cache unit is used to save the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request. Specifically, by storing the reasoning resources in the process of generating historical reasoning results from historical reasoning requests through a cache unit, the historical reasoning resources can be extracted in the result reasoning process for the current reasoning request to reduce the problem of repeated calculations in the current reasoning process, thereby speeding up the reasoning speed. Furthermore, by using the generated result words to update the current reasoning data, the generation of subsequent result words can be made more accurate, which can improve the fluency and continuity of the natural language generated by the reasoning system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 A schematic diagram of the structure of a large language reasoning system provided in the first embodiment of the present invention;

[0016] Figure 2 A schematic diagram of the structure of a large language reasoning system provided in Embodiment 2 of the present invention;

[0017] Figure 2a A schematic diagram of the structure of another large language reasoning system provided by an embodiment of the present invention;

[0018] Figure 2b A schematic diagram of the structure of another large language reasoning system provided by an embodiment of the present invention;

[0019] Figure 2c A schematic diagram of the structure of another large language reasoning system provided by an embodiment of the present invention;

[0020] Figure 3 This is a flow chart of a large language reasoning method provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] Embodiment 1

[0024] Figure 1 A schematic diagram of the structure of a large language reasoning system provided in the first embodiment of the present invention is shown in FIG. Figure 1As shown, the large language reasoning system includes: at least one pre-filling calculation unit 110, at least one generating calculation unit 120 and a cache unit 130, and each unit is communicatively connected. Optionally, the large language reasoning system is deployed in a server cluster, at least one server in the server cluster is configured as the pre-filling calculation unit 110, at least one server in the server cluster is configured as the generating calculation unit 120, and at least one server in the server cluster is configured as the cache unit 130. Furthermore, the large language reasoning system is a natural language generation system, which can obtain an output of a natural language type corresponding to the input by inputting a natural language into the large language system and then performing reasoning calculations through the large language reasoning system.

[0025] At least one pre-filling computing unit 110 is used to pre-fill according to the current reasoning request and historical reasoning resources, obtain current reasoning data, and send the current reasoning data to the generation computing unit 120 and the cache unit 130, wherein the historical reasoning resources include historical reasoning requests associated with the current reasoning request and reasoning data corresponding to the historical reasoning results.

[0026] Specifically, pre-filling can be understood as the process of pre-processing input data, such as segmenting, encoding, sorting and field position marking of input data, so as to obtain data that can be accurately recognized by the large language reasoning model. In the embodiment of the present invention, it is specifically the process of obtaining the reasoning request and the required reasoning resources, and then generating the current reasoning data according to the reasoning request and the required reasoning resources. The current reasoning data contains the keywords and content information required in this reasoning operation, and can also include the historical reasoning data related to the current reasoning request in the historical reasoning resources; the historical reasoning resources are the intermediate reasoning data generated in the process of generating the historical reasoning results according to the historical reasoning requests. The historical reasoning resources can be input into the pre-filling calculation unit, and the pre-filling calculation unit will identify the parameter data in the historical reasoning resources, and should be used for the reasoning of the current reasoning request, thereby reducing the calculation amount of the current reasoning request. Furthermore, the historical reasoning resources also include the text information of the historical reasoning request and the historical reasoning result itself. Furthermore, the historical reasoning resources are stored in the cache unit to facilitate the subsequent inference and save the computing resources required for repeated reasoning; the historical reasoning resources and the current reasoning data can be data in the form of kv cache (key value cache, key value cache).

[0027] At least one generation calculation unit 120 is used to receive current reasoning data, and use a preset large language model to generate a current result word according to the current reasoning data, generate a current reasoning result according to the current result word and send it to the cache unit, and output the current result word to the user end.

[0028] Among them, the large language model can convert natural language into computing data in a preset format, and generate corresponding natural language based on the computing data in the preset format, and the computing data in the preset format includes current computing data and / or historical reasoning resources.

[0029] Specifically, the generation calculation unit 120 can generate a current result word based on the current reasoning data and send it to the user end; update the current reasoning data through the current result word, generate subsequent result words, and combine all the generated result words to form the current reasoning result, and save the current reasoning result to the cache unit.

[0030] Exemplarily, the result word generation process is as follows:

[0031] Current inference request: abc

[0032] The current reasoning request generates the current reasoning data 1 through the pre-filled computing unit 110

[0033] Generation of result word A: Current reasoning data 1→A

[0034] Generation of result word B: A+current inference data 1→current inference data 2→B

[0035] Generation of result word C: B + current reasoning data 2 → current reasoning data 3 → C

[0036] Current reasoning result: ABC

[0037] Specifically, the current reasoning data 1 can be updated by the result word A, and the updated current reasoning data 1 is the current reasoning data 2, and then the calculation unit 120 generates the result word B according to the current reasoning data 2; according to the result word B, the current reasoning data 2 is updated to the current reasoning data 3, and then the result word C is generated. Further, through the above-mentioned result word generation method, it can be ensured that the subsequent result word has a strong connection with the generated result word, thereby ensuring the fluency and accuracy of the natural language type reasoning result.

[0038] The cache unit 130 is used to save the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request.

[0039] The cache unit stores a historical reasoning field consisting of a historical reasoning request and a historical reasoning result. The historical reasoning field is associated with the address of a corresponding historical reasoning resource and is used for text matching with the current reasoning field.

[0040] The embodiment of the present invention provides a large language reasoning system and method, which includes: at least one pre-filling calculation unit, at least one generating calculation unit and a cache unit; at least one pre-filling calculation unit is used to pre-fill according to the current reasoning request and historical reasoning resources, obtain the current reasoning data, and send the current reasoning data to the generating calculation unit and the cache unit, wherein the historical reasoning resources include the historical reasoning request associated with the current reasoning request and the reasoning data corresponding to the historical reasoning result; at least one generating calculation unit is used to receive the current reasoning data, and use the preset large language model to generate the current result word according to the current reasoning data, generate the current reasoning result according to the current result word and send it to the cache unit, and output the current result word to the user end; the cache unit is used to save the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request. Specifically, the historical reasoning resources in the process of generating the historical reasoning result from the historical reasoning request are stored in the cache unit, and then the historical reasoning resources can be extracted in the result reasoning process for the current reasoning request, so as to reduce the problem of repeated calculation in the current reasoning process and speed up the reasoning speed. Further, by using the generated result word to update the current reasoning data, the generation of subsequent result words can be made more accurate, and the fluency of the natural language generated by the reasoning system can be improved.

[0041] Embodiment 2

[0042] Figure 2 A structural diagram of a large language reasoning system provided in Embodiment 2 of the present invention includes: at least one pre-filling computing unit 110, at least one generating computing unit 120, a cache unit 130, a scheduling unit 140 and an API gateway 150. The API gateway 150 is used to establish a communication connection between the user end and the large language reasoning system.

[0043] The scheduling unit 140 is used to determine the target pre-filled computing unit corresponding to the current reasoning request according to the pre-filled computing units corresponding to the historical reasoning requests.

[0044] Optionally, the scheduling unit is also used to determine the pre-filled computing unit used in the pre-filling process of the historical reasoning request as the target pre-filled computing unit corresponding to the current reasoning request. Specifically, if there is a historical reasoning request associated with the current reasoning request, the pre-filled computing unit used when the historical reasoning request is reasoned can be determined according to the historical reasoning log or the reasoning process record, and used as the target pre-filled computing unit. This method of determining the target pre-filled computing unit can facilitate the extraction of historical reasoning resources during the pre-filling process, thereby speeding up the calculation speed of the pre-filling process.

[0045] Optionally, the scheduling unit 140 is further used to determine the storage address of the historical reasoning resource in the cache unit, and forward the current reasoning request and the storage address to the target pre-filled computing unit.

[0046] Specifically, the method of determining the storage address of the historical reasoning resource in the cache unit includes:

[0047] Get the fields to be matched;

[0048] Matching the to-be-matched field with the historical inference field in the cache unit;

[0049] If the matching field contains the same subfield as the historical inference field,

[0050] Determine that the reasoning resource corresponding to the historical reasoning field is the historical reasoning resource, and send the storage address of the historical reasoning resource to the target pre-filled computing unit, wherein the historical reasoning field includes the historical reasoning request and the corresponding historical reasoning result.

[0051] Among them, the field to be matched is the input of the scheduling unit, and the field to be matched is composed of the current reasoning request, the historical reasoning request and the historical reasoning result. It should be noted that the scheduling unit can complete the matching process of the field to be matched and the historical reasoning field. The generation of the field to be matched can be performed by the upper-level encapsulation module, and the present invention is not limited or specifically introduced here.

[0052] Specifically, the field to be matched can be obtained, and the field to be matched can be text-matched with the historical reasoning field. If there is a subfield in the field to be matched that is the same as the historical reasoning field, it means that the historical reasoning request corresponding to the subfield has been reasoned in history, and the historical reasoning result has been obtained, and the corresponding historical reasoning resource has been generated. Therefore, the reasoning resource corresponding to the historical reasoning field can be used as the historical reasoning resource, and then the storage address of the historical reasoning resource in the cache unit can be sent to the target pre-filling calculation unit, so that the target pre-filling calculation unit can perform calculations in the pre-filling process.

[0053] Optionally, the target pre-filling computing unit in the pre-filling computing unit 110 is further used to extract the historical reasoning resource in the cache unit according to the storage address of the historical reasoning resource.

[0054] Specifically, after the scheduling unit obtains the field to be matched, it can perform text matching between the field to be matched and the historical reasoning field in the reasoning field table. If the text matching between the field to be matched and the historical reasoning field is successful, it means that the subfield in the field to be matched that is the same as the historical reasoning field has been calculated in history, and then the storage address of the corresponding historical reasoning resource in the cache unit can be determined based on the association information between the historical reasoning field and the corresponding historical reasoning resource. Then, the corresponding historical reasoning resource can be extracted in the pre-filled calculation unit based on the storage address, reducing repeated calculations of the same parameter and improving the reasoning speed.

[0055] For example, if the current reasoning field is AaBbCcDd and the historical reasoning field is AaBbCc, the historical reasoning resource corresponding to the successfully matched AaBbCc is determined to be the historical reasoning resource required for the current reasoning request, and the historical reasoning resource can be called to reduce the reasoning computing resources of the current reasoning request.

[0056] For example, if the field to be matched is ABCD; and the historical reasoning field stored in the cache unit is ABC, and there is a subfield ABC in the field to be matched that is the same as the historical reasoning field, then it means that the historical reasoning request corresponding to ABC has been inferred and calculated in history. The calculation of the current reasoning request D can be accelerated by extracting the historical reasoning resources corresponding to ABC in the cache unit.

[0057] For example,

[0058] User Question 1: What day is tomorrow?

[0059] Result 1: Tomorrow is June 1st.

[0060] User Question 2: What day is it?

[0061] Result 2: June 1st is a Wednesday.

[0062] In this round of questions and answers, the output of result 2 depends on question 1, result 1 and question 2. If result 1 is recalculated based on question 1 when generating result 2, and then result 2 is generated based on question 1, result 1 and question 2, a lot of repeated calculations will be performed and a lot of computing resources will be wasted. Therefore, the historical reasoning resources generated in the process of generating result 1 from question 1 can be saved, and when generating result 2, they can be called as needed to save computing resources.

[0063] Optionally, the scheduling unit 140 is further configured to determine a target generation computing unit according to remaining computing resources of each generation computing unit.

[0064] Specifically, the remaining computing resources of each generation computing unit can be determined, and then the generation computing unit with the most remaining computing resources can be determined as the target generation computing unit. This method can improve the utilization efficiency of the computing resources of each generation computing unit.

[0065] Optionally, the target generation calculation unit in the generation calculation unit is specifically used to input the current reasoning data into a preset large language model to obtain the current result word output by the large language model;

[0066] If the current result word does not meet the preset loop end condition, the current inference data is updated according to the current result word, and the current inference data is input into the preset large language model;

[0067] If the current result word meets the preset loop end condition, the current reasoning result is generated according to the current result word corresponding to the current reasoning request.

[0068] Optionally, the preset loop end condition includes that the current result word corresponding to the current reasoning request reaches a preset number of words or the current result word contains a preset end character.

[0069] Specifically, due to the limitation of current computing power, for extremely complex reasoning requests, the generated reasoning results may contain tens of thousands of reasoning result words. The generation process may be extremely complex and occupy a large amount of system resources. Therefore, it is necessary to set a preset word limit. When the total number of result words generated exceeds the upper limit, even if the answer is not finished, it must be forced to end in order to protect the system. The preset end character is a character generated by the reasoning system to determine the end of the answer. When the system believes that the current result word is the last result word and all result words can form a complete reasoning answer, it will automatically generate a reasoning end character in the last result word to end the current reasoning process.

[0070] Furthermore, the update loop of the current reasoning data will also end with the end of the reasoning process. Specifically, if the current result word contains a preset end character, it means that the current result word is the last word of the current reasoning result, and the update of the current reasoning data will end; if the generated result word reaches the preset word limit, the update of the current reasoning data will also end.

[0071] Optionally, the scheduling unit 140 is further used to receive a current inference request sent by the user end, and send it to the target pre-filled computing unit; and receive a current result word sent by the target generated computing unit, and forward it to the user end.

[0072] Exemplarily, in an achievable technical solution, the pre-filling calculation unit and the generating calculation unit may be combined, that is, the pre-filling process and the generating process may be completed by a single server. Furthermore, Figure 2aA schematic diagram of the structure of another large language reasoning system provided by an embodiment of the present invention is shown in FIG. Figure 2a As shown, the technical solution of this embodiment can complete two-stage computing operations in a single server, reducing the interaction delay in the interaction between different units. At the same time, it is used to achieve the same technical effects as the above-mentioned embodiments, such as reducing the problem of repeated calculations in multiple rounds of reasoning operations and saving computing resources.

[0073] For example, Figure 2b A schematic diagram of the structure of another large language reasoning system provided by an embodiment of the present invention is shown in FIG. Figure 2b As shown, in an achievable technical solution, the pre-filling process and the generation process can be kept separate, and the calculations of the two processes can be performed on different servers, but the cache unit is removed and a stateless architecture is adopted. Under this solution, the situation in which the pre-filling phase calculation interrupts the generation phase calculation on the same computing unit can be avoided, and there will be no pause when the user is obtaining the model generation results.

[0074] For example, Figure 2c A schematic diagram of the structure of another large language reasoning system provided by an embodiment of the present invention is shown in FIG. Figure 2c As shown, in an achievable technical solution, a cache unit can be added to the generation calculation unit, that is, a generation calculation && cache unit is formed, and the scheduling unit can perform more flexible scheduling of the current reasoning request according to the length of the current reasoning request input by the user this time. For example, when the text length of the current reasoning request of the user is short, that is, when the newly added pre-filling calculation amount is small, the request is directly sent to the generation && cache unit, and this part of the pre-filling calculation is converted into an equivalent generation calculation, and then the pre-filling of the complete context and intermediate calculation data is first realized, and finally the generation calculation is continued and the natural language result is sampled. Specifically, the pre-filling calculation unit can take the current reasoning request as a whole, perform pre-filling calculation, and generate the current calculation data. However, the generated current calculation data may have a huge amount of data, and transmitting it to the generation calculation unit will consume a lot of resources. Therefore, if the current reasoning request is short, the current reasoning request can be directly sent to the generation && cache unit, and the calculation data is generated word by word through the generation && cache unit, and finally integrated into the complete current calculation data. This solution can more flexibly implement the method of generating computing units to obtain computing data, avoiding the need to obtain full inference resources from pre-filled computing units when the amount of newly added intermediate computing data is small, thereby reducing the cost and time consumption of network transmission.

[0075] An embodiment of the present invention provides a large language reasoning system, including at least one pre-filling computing unit, at least one generating computing unit, a cache unit and a scheduling unit, wherein the scheduling unit can receive a current reasoning request sent by a user terminal, and forward the current reasoning request to a target pre-filling computing unit. Furthermore, the scheduling unit can quickly confirm the storage address of the historical reasoning resource related to the current reasoning request, and send the storage address to the target pre-filling computing unit, thereby improving the computing efficiency of the pre-filling process; at the same time, the target pre-filling computing unit and the target generating computing unit required for this round of operation can be quickly determined, thereby accelerating the reasoning speed.

[0076] Embodiment 3

[0077] Figure 3 This is a flow chart of a large language reasoning method provided in the third embodiment of the present invention. This method can be applied to any large language reasoning system in the embodiments of the present invention. Figure 3 As shown, the method includes:

[0078] Step 310: the at least one pre-filling computing unit performs pre-filling according to the current reasoning request and the historical reasoning resources to obtain current reasoning data, and sends the current reasoning data to the generating computing unit and the cache unit;

[0079] Step 320, the at least one generating and calculating unit receives the current reasoning data, and uses a preset large language model to generate a current result word according to the current reasoning data, generates a current reasoning result according to the current result word, sends it to a cache unit, and outputs the current result word to a user terminal;

[0080] Step 330: The cache unit saves the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request.

[0081] Optionally, the method of this embodiment also includes: a scheduling unit receives a current reasoning request sent by a user terminal, and forwards the current reasoning request to a target pre-filled computing unit, and is also used to receive a current result word sent by a target generation computing unit, and forward the current result word to the user terminal.

[0082] Optionally, the scheduling unit determines a storage address of the historical reasoning resource in the cache unit, and forwards the current reasoning request and the storage address to the target pre-filled computing unit.

[0083] Optionally, the scheduling unit determines the storage address of the historical reasoning resource in the cache unit specifically includes:

[0084] Get the fields to be matched;

[0085] Matching the to-be-matched field with the historical inference field in the cache unit;

[0086] If there is a subfield in the field to be matched that is the same as the historical reasoning field, determine that the reasoning resource corresponding to the historical reasoning field is a historical reasoning resource, and send the storage address of the historical reasoning resource to the target pre-filled computing unit, wherein the historical reasoning field includes a historical reasoning request and a corresponding historical reasoning result.

[0087] Optionally, the target pre-filling computing unit in the pre-filling computing unit extracts the historical reasoning resource in the cache unit according to the storage address of the historical reasoning resource.

[0088] Optionally, the scheduling unit determines the target generation computing unit according to the remaining computing resources of each generation computing unit.

[0089] Optionally, the target generation calculation unit in the generation calculation unit inputs the current reasoning data into a preset large language model to obtain a current result word output by the large language model;

[0090] If the current result word does not meet the preset loop end condition, the current reasoning data is updated according to the current result word, and the current reasoning data is input into the preset large language model;

[0091] If the current result word satisfies a preset loop end condition, a current reasoning result is generated according to the current result word corresponding to the current reasoning request.

[0092] Optionally, the preset loop end condition includes that the current result word corresponding to the current inference request reaches a preset number of words; or the current result word contains a preset end character.

[0093] Optionally, the scheduling unit receives a current inference request sent by the user end, and sends it to the target pre-filling computing unit; and receives a current result word sent by the target generation computing unit, and forwards it to the user end.

[0094] An embodiment of the present invention provides a large language reasoning method, which can be applied to any large language reasoning system of the present invention, and specifically includes: the at least one pre-filling calculation unit is pre-filled according to the current reasoning request and the historical reasoning resources to obtain the current reasoning data, and the current reasoning data is sent to the generation calculation unit and the cache unit; the at least one generation calculation unit receives the current reasoning data, and uses a preset large language model to generate a current result word according to the current reasoning data, generates a current reasoning result according to the current result word and sends it to the cache unit, and outputs the current result word to the user end; the cache unit saves the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request. Specifically, the reasoning resources in the process of generating historical reasoning results from historical reasoning requests are stored in the cache unit, and then the historical reasoning resources can be extracted in the result reasoning process for the current reasoning request, so as to reduce the problem of repeated calculation in the current reasoning process and speed up the reasoning speed. Furthermore, by using the generated result word to update the current reasoning data, the generation of subsequent result words can be made more accurate, and the accuracy of natural language generation by the reasoning system can be improved.

[0095] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0096] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0097] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0098] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A large language reasoning system, characterized in that: include: at least one pre-filling computing unit, at least one generating computing unit and a cache unit; The at least one pre-filling computing unit is used to pre-fill according to the current reasoning request and the historical reasoning resources to obtain the current reasoning data, and send the current reasoning data to the generating computing unit and the cache unit, wherein the historical reasoning resources include the historical reasoning requests associated with the current reasoning request and the reasoning data corresponding to the historical reasoning results; The at least one generating and calculating unit is used to receive the current reasoning data, and use a preset large language model to generate a current result word according to the current reasoning data, generate a current reasoning result according to the current result word, send it to the cache unit, and output the current result word to the user end; The cache unit is used to save the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request; The large language reasoning system further includes a scheduling unit, which is used to determine a target pre-filled computing unit corresponding to a current reasoning request based on pre-filled computing units corresponding to historical reasoning requests.

2. The system according to claim 1, characterized in that include: The scheduling unit is further used to determine the storage address of the historical reasoning resource in the cache unit, and forward the current reasoning request and the storage address to the target pre-filling computing unit.

3. The system according to claim 2, characterized in that The scheduling unit is specifically used to: Get the fields to be matched; Matching the to-be-matched field with the historical inference field in the cache unit; If there is a subfield in the field to be matched that is the same as the historical reasoning field, determine that the reasoning resource corresponding to the historical reasoning field is a historical reasoning resource, and send the storage address of the historical reasoning resource to the target pre-filled computing unit, wherein the historical reasoning field includes a historical reasoning request and a corresponding historical reasoning result.

4. The system according to claim 2, characterized in that include: The target pre-filling computing unit in the pre-filling computing unit is further used to extract the historical reasoning resource in the cache unit according to the storage address of the historical reasoning resource.

5. The system according to claim 1, characterized in that include: The scheduling unit is further used to determine the target generation computing unit according to the remaining computing resources of each generation computing unit; The target generation calculation unit in the generation calculation unit is specifically used to input the current reasoning data into a preset large language model to obtain the current result word output by the large language model; If the current result word does not meet the preset loop end condition, the current reasoning data is updated according to the current result word, and the current reasoning data is input into the preset large language model; If the current result word satisfies a preset loop end condition, a current reasoning result is generated according to the current result word corresponding to the current reasoning request.

6. The system according to claim 5, characterized in that The preset loop end condition includes that the current result word corresponding to the current reasoning request reaches a preset word number; Or the current result word contains a preset terminator.

7. The system according to claim 5, characterized in that include: The scheduling unit is further used to receive a current inference request sent by a user terminal and send it to a target pre-filled computing unit; And, receiving the current result word sent by the target generation calculation unit, and forwarding it to the user end.

8. A large language reasoning method, characterized in that: A large language reasoning system as claimed in any one of claims 1 to 7, comprising: The at least one pre-filling computing unit performs pre-filling according to the current reasoning request and the historical reasoning resources to obtain the current reasoning data, and sends the current reasoning data to the generating computing unit and the cache unit; The at least one generating and calculating unit receives the current reasoning data, and generates a current result word according to the current reasoning data using a preset large language model, generates a current reasoning result according to the current result word and sends it to a cache unit, and outputs the current result word to a user terminal; The cache unit stores the current reasoning data, the current reasoning request and the current reasoning result according to the current reasoning request; The scheduling unit determines a target pre-filled computing unit corresponding to a current reasoning request according to the pre-filled computing units corresponding to the historical reasoning requests.

9. The method according to claim 8, characterized in that include: The scheduling unit determines a storage address of the historical reasoning resource in the cache unit, and forwards the current reasoning request and the storage address to the target pre-filled computing unit.

10. The method according to claim 9, characterized in that include: Get the fields to be matched; Matching the to-be-matched field with the historical inference field in the cache unit; If there is a subfield in the field to be matched that is the same as the historical reasoning field, determine that the reasoning resource corresponding to the historical reasoning field is a historical reasoning resource, and send the storage address of the historical reasoning resource to the target pre-filled computing unit, wherein the historical reasoning field includes a historical reasoning request and a corresponding historical reasoning result.

11. The method according to claim 9, characterized in that include: The target pre-filling computing unit in the pre-filling computing unit extracts the historical reasoning resource in the cache unit according to the storage address of the historical reasoning resource.

12. The method according to claim 8, characterized in that include: The scheduling unit determines the target generation computing unit according to the remaining computing resources of each generation computing unit; The target generation calculation unit in the generation calculation unit inputs the current reasoning data into a preset large language model to obtain a current result word output by the large language model; If the current result word does not meet the preset loop end condition, the current reasoning data is updated according to the current result word, and the current reasoning data is input into the preset large language model; If the current result word satisfies a preset loop end condition, a current reasoning result is generated according to the current result word corresponding to the current reasoning request.

13. The method according to claim 12, characterized in that The preset loop end condition includes that the current result word corresponding to the current reasoning request reaches a preset word number; Or the current result word contains a preset terminator.

14. The method according to claim 12, characterized in that include: The scheduling unit receives a current reasoning request sent by a user terminal and sends it to a target pre-filled computing unit; And, receiving the current result word sent by the target generation calculation unit, and forwarding it to the user end.

Images