Large language model reasoning method and device, computer equipment, medium and product

Abstract

The invention relates to the technical field of artificial intelligence, and particularly provides a big language model reasoning method and device, computer equipment, a medium and a product.The method comprises the steps that in response to a reasoning request of a client side, an input text sent by the client side is received; determining a target prefix fine tuning parameter of the client for a target pre-training model based on the input text and a parameter mapping relationship; wherein the parameter mapping relationship is used for indicating the prefix fine tuning parameter associated with the target pre-training model requested to be used by the client in the prefix fine tuning parameters cached by the server; and performing text reasoning on the input text through the target prefix fine tuning parameter and the target pre-training model to obtain a text reasoning result, and sending the text reasoning result to the client.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment, medium and product for reasoning a large language model. Background Art

[0002] With the continuous development of big data and artificial intelligence technology, especially in the field of natural language processing (NLP), pre-trained models have achieved remarkable results. After obtaining the pre-trained model, it is necessary to further fine-tune it to make it more competent for various specific NLP tasks.

[0003] In the related art, for a specific data provider, the prefix fine-tuning parameters of the pre-trained model can be adjusted through its own industry data, and finally the large language model of the data provider can be obtained. Then, the large language model can be deployed to the server for the data provider. In the related art, it is necessary to deploy a corresponding large language model on the server for each data provider. When the pre-trained models required by multiple data providers are of the same type, it is also necessary to deploy a corresponding large language model for each data provider, which will consume more server resources, resulting in the inability to meet the data provider's deployment requirements for the large language model due to server resource limitations. Summary of the invention

[0004] In view of the above problems, the present disclosure is proposed. The present disclosure provides a large language model reasoning method, device, computer equipment, medium and product.

[0005] According to one aspect of the present disclosure, a large language model reasoning method is provided, which is applied to a server and includes:

[0006] Used to respond to an inference request from a client and obtain an input text sent by the client;

[0007] Based on the input text and the parameter mapping relationship, determining the target prefix fine-tuning parameters of the client for the target pre-trained model; wherein the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server;

[0008] The input text is subjected to text reasoning by using the target prefix fine-tuning parameter and the target pre-training model to obtain a text reasoning result, and the text reasoning result is sent to the client.

[0009] In addition, in another embodiment according to one aspect of the present disclosure, the method further includes:

[0010] In response to a session request from the client, searching the parameter mapping relationship for cached prefix fine-tuning parameters of the client for the target pre-trained model;

[0011] When it is determined that the cached prefix fine-tuning parameter does not exist in the parameter mapping relationship, sending a parameter upload request to the client;

[0012] When it is determined that the parameter response sent by the client carries a prefix fine-tuning parameter, the prefix fine-tuning parameter carried in the parameter response is determined as the target prefix fine-tuning parameter, and a mapping relationship between the client and the target prefix fine-tuning parameter is established in the parameter mapping relationship.

[0013] In addition, in another embodiment according to one aspect of the present disclosure, the method further includes:

[0014] In a case where it is determined that the parameter response sent by the client does not carry the prefix fine-tuning parameter, determining a preset prefix fine-tuning parameter matching the client as the target prefix fine-tuning parameter;

[0015] A mapping relationship between the client and the target prefix fine-tuning parameter is established in the parameter mapping relationship.

[0016] In addition, according to another embodiment of one aspect of the present disclosure, determining the target prefix fine-tuning parameter of the client for the target pre-trained model based on the input text and the parameter mapping relationship includes:

[0017] Parsing the inference request to obtain a user identification code of the client;

[0018] Determining a prefix fine-tuning parameter that matches the user identification code in the parameter mapping relationship;

[0019] The target prefix fine-tuning parameter is determined based on the matched prefix fine-tuning parameter.

[0020] In addition, according to another embodiment of one aspect of the present disclosure, the text reasoning of the input text is performed by using the target prefix fine-tuning parameter and the pre-trained model to obtain a text reasoning result, including:

[0021] Processing the input text based on the target prefix fine-tuning parameter to obtain a preprocessing vector representing prefix prompt information; wherein the prefix prompt information is associated with the target prefix fine-tuning parameter;

[0022] The preprocessing vector and the input text are processed by the target pre-training model to obtain a text reasoning result of the input text, and the text reasoning result is sent to the client.

[0023] In addition, in another embodiment according to one aspect of the present disclosure, the method further includes:

[0024] When the target prefix fine-tuning parameter satisfies a preset training condition, receiving training data sent by the client;

[0025] Based on the training data, the client trains the target prefix fine-tuning parameters for the target pre-trained model to obtain trained prefix fine-tuning parameters;

[0026] The target prefix fine-tuning parameter is updated to the trained prefix fine-tuning parameter in the parameter mapping relationship, and the trained prefix fine-tuning parameter is sent to the client.

[0027] According to one aspect of the present disclosure, another large language model reasoning method is provided, which is applied to a client and includes:

[0028] Send input text to the server;

[0029] Receive a text inference result sent by the server; wherein the text inference result is obtained by performing text inference on the input text through the target prefix fine-tuning parameters of the pre-trained model and the pre-trained model, the target prefix fine-tuning parameters are prefix fine-tuning parameters for the target pre-trained model determined based on the input text and the parameter mapping relationship, and the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server.

[0030] In addition, in another embodiment according to one aspect of the present disclosure, the method further includes:

[0031] Sending a session request to the server; wherein the session request is used to instruct the server to search the cached prefix fine-tuning parameters of the client for the target pre-trained model in the parameter mapping relationship;

[0032] receiving a parameter upload request sent by the server; the parameter upload request is sent when the server determines that the cached prefix fine-tuning parameter does not exist in the parameter mapping relationship;

[0033] Send a parameter response to the server.

[0034] In addition, according to another embodiment of one aspect of the present disclosure, a method for collecting training data matching the reasoning method is also included:

[0035] Based on the input text and the text inference result, training data is generated, and when the target prefix fine-tuning parameter meets a preset training condition, the training data is sent to the server.

[0036] After sending the training data to the server, the server receives the trained prefix fine-tuning parameters obtained by training the client for the target prefix fine-tuning parameters of the target pre-trained model based on the training data, and caches the trained prefix fine-tuning parameters.

[0037] In addition, according to another embodiment of one aspect of the present disclosure, generating training data based on the input text and the text reasoning result includes:

[0038] Determine a first input text and a second input text that are continuously input to the server; wherein the second input text is text that is input to the server after the first input text;

[0039] In a case where it is determined that the first input text and the second input text meet a preset combination condition, the training data is determined based on text reasoning results of the first input text and the second input text.

[0040] In addition, according to another embodiment of one aspect of the present disclosure, the determining whether the first input text and the second input text meet a preset combination condition includes:

[0041] Determining the similarity between the second input text and a preset text; wherein the preset text is a preset supplementary description text;

[0042] In the case where it is determined that the similarity between the second input text and the preset text is less than a first similarity threshold, determining the similarity between the first input text and the second input text;

[0043] In the case where it is determined that the similarity between the first input text and the second input text is greater than a second similarity threshold, it is determined that the first input text and the second input text meet the preset combination condition.

[0044] According to another aspect of the present disclosure, there is provided a large language model reasoning device, comprising:

[0045] A first receiving module, configured to obtain an input text sent by the client in response to an inference request from the client;

[0046] A determination module, configured to determine, based on the input text and the parameter mapping relationship, a target prefix fine-tuning parameter of the client for the target pre-trained model; wherein the parameter mapping relationship is used to indicate the prefix fine-tuning parameter associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server;

[0047] The text reasoning module is used to perform text reasoning on the input text through the target prefix fine-tuning parameters and the target pre-training model, obtain a text reasoning result, and send the text reasoning result to the client.

[0048] According to another aspect of the present disclosure, another inference device for a large language model is provided, comprising:

[0049] The sending module is used to send input text to the server;

[0050] A second receiving module is used to receive the text inference result sent by the server; wherein the text inference result is obtained by performing text inference on the input text through the target prefix fine-tuning parameters of the pre-trained model and the pre-trained model, the target prefix fine-tuning parameters are prefix fine-tuning parameters for the target pre-trained model determined based on the input text and the parameter mapping relationship, and the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server.

[0051] According to another aspect of the present disclosure, a computer device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement steps of a method for determining a log template.

[0052] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of a method for determining a log template are implemented.

[0053] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, which implements the steps of a method for determining a log template when the computer program / instruction is executed by a processor.

[0054] As described in detail below, according to the inference method device, computer device, medium and product of the large language model of the embodiment of the present disclosure, the target prefix fine-tuning parameter of the client for the target pre-trained model is determined through the parameter mapping relationship, and the input text is processed through the target prefix fine-tuning parameter and the target pre-trained model to obtain the text inference result.

[0055] In this embodiment, by setting the parameter mapping relationship, the prefix fine-tuning parameters set for the same target pre-trained model requested by different clients can be determined. Through this processing method, it is not necessary to set a corresponding large language model for each client. For the same type of pre-trained model requested, the prefix fine-tuning parameters of the target pre-trained model for each client can be determined through the parameter mapping relationship. For the same type of pre-trained model, there is no need to repeatedly deploy it on the server side, thereby saving the use of server resources and allowing the server to provide services to more clients.

[0056] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0058] Figure 1 A flowchart of a method for reasoning a large language model provided in an embodiment of the present disclosure.

[0059] Figure 2 An interaction graph for establishing a conversation connection in the inference method of a large language model provided in an embodiment of the present disclosure.

[0060] Figure 3 An interactive diagram of text reasoning in the reasoning method of a large language model provided in an embodiment of the present disclosure.

[0061] Figure 4 A flowchart of another large language model reasoning method provided in an embodiment of the present disclosure.

[0062] Figure 5 A flowchart of training data processing in an inference method for a large language model provided in an embodiment of the present disclosure.

[0063] Figure 6 A flowchart for determining training data in an inference method for a large language model provided in an embodiment of the present disclosure.

[0064] Figure 7 A schematic diagram of an inference system for a large language model provided in an embodiment of the present disclosure.

[0065] Figure 8 A schematic diagram of an inference device for a large language model provided in an embodiment of the present disclosure.

[0066] Fig. 9 A schematic diagram of another large language model inference device provided in an embodiment of the present disclosure.

[0067] Fig.10 A schematic diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0069] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0070] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0071] Research has found that with the continuous development of big data and artificial intelligence technology, especially in the field of natural language processing (NLP), pre-trained models have achieved remarkable results. After obtaining the pre-trained model, it is necessary to further fine-tune it to make it more competent for various specific NLP tasks.

[0072] In the related art, for a specific data provider, the prefix fine-tuning parameters of the pre-trained model can be adjusted through its own industry data, and finally the large language model of the data provider can be obtained. Then, the large language model can be deployed to the server for the data provider. In the related art, it is necessary to deploy a corresponding large language model on the server for each data provider. When the pre-trained models required by multiple data providers are of the same type, it is also necessary to deploy a corresponding large language model for each data provider, which will consume more server resources, resulting in the inability to meet the data provider's deployment requirements for the large language model due to server resource limitations.

[0073] Based on the above research, the present disclosure provides an inference method for a large language model. In an embodiment of the present disclosure, by setting a parameter mapping relationship, the prefix fine-tuning parameters set for the same target pre-trained model requested by different clients can be determined. Through this processing method, it is not necessary to set a corresponding large language model for each client. For the same type of pre-trained model requested, the prefix fine-tuning parameters of the target pre-trained model for each client can be determined through the parameter mapping relationship. For the same type of pre-trained model, there is no need to repeatedly deploy it on the server side, thereby saving the use of server resources and enabling the server to provide services to more clients.

[0074] To facilitate understanding of this embodiment, a large language model reasoning method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the large language model reasoning method provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities. In some possible implementations, the large language model reasoning method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0075] See also Figure 1 As shown, it is a flowchart of a large language model reasoning method provided by an embodiment of the present disclosure, the method includes steps S101 to S103, which are applied to the server, wherein:

[0076] S101. In response to an inference request from a client, an input text sent by the client is obtained.

[0077] In an embodiment of the present disclosure, before receiving the input text sent by the client, the server may first establish a session connection with the client. The specific session connection process will be introduced in the following implementation manner.

[0078] After the server establishes a session connection with the client, the server may notify the client that the session connection has been established. After that, the server may maintain the current session connection with the client. Based on the maintained session connection, the client may send an inference request to the server. Here, the inference request sent by the client to the server may include the question content of the question raised by the client to the server and the user identification code corresponding to the client.

[0079] Here, the user identification code is used to indicate the identity information of the user to which the client belongs. The user identification code of the client can be determined based on the user information of the client, for example, by the user name and user password of the client. In addition, the user identification code can also be determined by other user information, for example, by the user's ID number.

[0080] S102. Determine the target prefix fine-tuning parameters of the target pre-trained model for the client based on the input text and the parameter mapping relationship; wherein the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server.

[0081] In an embodiment of the present disclosure, the server may pre-cache the prefix fine-tuning parameters of the client associated with the server relative to the pre-trained model, and then determine the mapping relationship between the target pre-trained model requested by the client and the associated prefix fine-tuning parameters, and cache the association as a parameter mapping relationship.

[0082] Here, the server can determine the target pre-trained model requested by the client based on the user identification code in the input text. After that, the server can determine the parameter mapping relationship pre-cached by the server, and then determine the target prefix fine-tuning parameters of the client for the target pre-trained model based on the parameter mapping relationship and the input text.

[0083] In an embodiment of the present disclosure, the target pre-trained model requested by the client can be determined based on the input text; then, the prefix fine-tuning parameters associated with the target pre-trained model in the parameter mapping relationship can be used as the target prefix fine-tuning parameters.

[0084] The server pre-stores at least one pre-trained model that each client is allowed to use, and each client can request to use one or more pre-trained models. The target pre-trained model is the pre-trained model requested by the client at the current moment.

[0085] S103: Perform text reasoning on the input text through the target prefix fine-tuning parameters and the target pre-training model to obtain a text reasoning result, and send the text reasoning result to the client.

[0086] In an embodiment of the present disclosure, the server may process the target prefix fine-tuning parameter and the problem content in the input text to obtain a processing result (ie, a preprocessing vector carrying prefix prompt information described in the following embodiment).

[0087] After obtaining the processing results, the server can perform text reasoning on the processing results and question content through the target pre-trained model to obtain text reasoning results.

[0088] After obtaining the text inference result, the text inference result can be sent to the client. In addition, the client can also collect the text inference result, and then determine the training data through the text inference result and the corresponding input text. The target prefix fine-tuning parameter can be readjusted through the training data, so as to obtain a more accurate prefix fine-tuning parameter.

[0089] In the above implementation, by setting the parameter mapping relationship, the prefix fine-tuning parameters set for the same target pre-trained model requested by different clients can be determined. Through this processing method, it is not necessary to set up a corresponding large language model for each client. For the same type of pre-trained model requested, the prefix fine-tuning parameters of the target pre-trained model for each client can be determined through the parameter mapping relationship. For the same type of pre-trained model, there is no need to repeatedly deploy it on the server side, thereby saving the use of server resources and allowing the server to provide services to more clients.

[0090] In an optional embodiment, in the above Figure 1 Based on the described embodiment, the method further comprises the following steps:

[0091] S11, in response to a session request from the client, searching for cached prefix fine-tuning parameters of the client for the target pre-trained model in the parameter mapping relationship;

[0092] S12, when it is determined that there is no cached prefix fine-tuning parameter in the parameter mapping relationship, sending a parameter upload request to the client;

[0093] S13, when it is determined that the parameter response sent by the client carries the prefix fine-tuning parameter, determining the carried prefix fine-tuning parameter as the target prefix fine-tuning parameter, and establishing a mapping relationship between the client and the target prefix fine-tuning parameter in the parameter mapping relationship;

[0094] S14, when it is determined that the parameter response sent by the client does not carry the prefix fine-tuning parameter, determining a preset prefix fine-tuning parameter matching the client as the target prefix fine-tuning parameter;

[0095] S15, establishing a mapping relationship between the client and the target prefix fine-tuning parameter in the parameter mapping relationship.

[0096] In an embodiment of the present disclosure, the client may send a session request to the server, wherein the session request includes, but is not limited to: user information of the user to which the client belongs and all pre-trained models requested to be used.

[0097] After the server receives the session request from the client, the server may determine all pre-trained models that the client is allowed to use from among all deployed pre-trained models.

[0098] After all the pre-trained models described above are determined (referred to as pre-trained model A), the server can then search for the prefix fine-tuning parameters of the client for pre-trained model A in the parameter mapping relationship.

[0099] If the server finds the prefix fine-tuning parameter, it maintains the session connection between the client and the server and ends the session connection process. If the server fails to find the prefix fine-tuning parameter, it determines that the client does not have the cached prefix fine-tuning parameter for pre-trained model A in the parameter mapping relationship. At this time, the server can generate a parameter upload request and send the parameter upload request to the client.

[0100] After the client receives the parameter upload request, it can determine whether there are prefix fine-tuning parameters matching the pre-trained model in the client's local cache. If the client determines that there are prefix fine-tuning parameters matching the pre-trained model A in the local cache, it can generate a parameter response based on the prefix fine-tuning parameters and send the parameter response to the server.

[0101] After obtaining the prefix fine-tuning parameter, the server can determine the prefix fine-tuning parameter of the client for the pre-trained model A, thereby obtaining a mapping relationship among the client, the pre-trained model A and the prefix fine-tuning parameter, and establishing a mapping relationship among the client, the pre-trained model A and the prefix fine-tuning parameter in the parameter mapping relationship. At the same time, the prefix fine-tuning parameter can also be stored in the cache area of ​​the prefix fine-tuning parameter in the server, and the prefix fine-tuning parameter can be sent to the client for caching.

[0102] When the client determines that there are no prefix fine-tuning parameters matching the pre-trained model A in the local cache, the parameter response generated at this time does not carry the prefix fine-tuning parameters. At this time, the server can pre-select the default prefix fine-tuning parameters (i.e., the preset prefix fine-tuning parameters) set for the pre-trained model A, and store the default prefix parameters as the prefix fine-tuning parameters of the client for the pre-trained model A in the cache of the prefix fine-tuning parameters. Among them, the default prefix fine-tuning parameters can be set according to actual needs. For example, the server can set different default prefix fine-tuning parameters according to the actual needs of the client.

[0103] Here, after the server determines the preset prefix fine-tuning parameter as the prefix fine-tuning parameter of the client for pre-trained model A, it can establish an association relationship (i.e., a mapping relationship) between the client and the prefix fine-tuning parameter, and add the association relationship to the parameter mapping relationship.

[0104] See also Figure 2 As shown, it is an interaction diagram for establishing a session connection in the reasoning method of a large language model provided by an embodiment of the present disclosure, wherein the method specifically includes the following process:

[0105] First, the client initiates a session request to the server. Second, after receiving the session request sent by the client, the server searches for the prefix fine-tuning parameters of the client for the pre-trained model used in the parameter mapping relationship. Second, if the server finds the prefix fine-tuning parameters, it maintains the session connection with the client.

[0106] Secondly, if the server fails to find the prefix fine-tuning parameters, it sends a parameter upload request to the client. Secondly, after receiving the parameter upload request, the client determines whether there are prefix fine-tuning parameters matching the pre-trained model in the local cache.

[0107] Secondly, when the client determines that there are prefix fine-tuning parameters matching the pre-trained model in the local cache, it sends a parameter response carrying the prefix fine-tuning parameters to the server. Secondly, when the server receives the parameter response carrying the prefix fine-tuning parameters, it stores the prefix fine-tuning parameters carried by the parameter response in the cache of the prefix fine-tuning parameters.

[0108] Secondly, when the client determines that there are no prefix fine-tuning parameters matching the pre-trained model in the local cache, it sends a parameter response without the prefix fine-tuning parameters to the server. Finally, when the server receives the parameter response without the prefix fine-tuning parameters, it determines the default prefix fine-tuning parameters of the pre-trained model and stores the default prefix fine-tuning parameters in the cache of the prefix fine-tuning parameters.

[0109] In an optional implementation, the above steps determine the target prefix fine-tuning parameters of the target pre-trained model by the client based on the input text and the parameter mapping relationship, and specifically include the following steps:

[0110] First, the client's user identification code is parsed in the inference request;

[0111] Secondly, determining the prefix fine-tuning parameter that matches the user identification code in the parameter mapping relationship;

[0112] Finally, the target prefix fine-tuning parameter is determined based on the matched prefix fine-tuning parameters.

[0113] In the embodiments of the present disclosure, the user identification code is determined by the server based on the user information. For example, the server can assign an initial identification code to the client after the user name and user password in the user information are verified; then, the initial identification code can be matched with the pre-trained model that the client can use to obtain the user identification code; and the user identification code is returned to the client. After that, the client can send an inference request to the server based on the user identification code.

[0114] Here, the server can determine the target pre-trained model through the inference request. For example, the target pre-trained model can be determined through the user identification code of the client in the inference request. After that, the server can determine the prefix fine-tuning parameters corresponding to the target pre-trained model through the parameter mapping relationship.

[0115] In an optional implementation, the above steps perform text reasoning on the input text by fine-tuning parameters and pre-training models using target prefixes to obtain text reasoning results, including:

[0116] First, the input text is processed based on the target prefix fine-tuning parameter to obtain a preprocessing vector representing prefix prompt information; wherein the prefix prompt information is associated with the target prefix fine-tuning parameter;

[0117] Then, the preprocessing vector and the input text are processed by the target pre-trained model to obtain the text reasoning result of the input text, and the text reasoning result is sent to the client.

[0118] In an embodiment of the present disclosure, the server may use a prefix fine-tuning technology to process the target prefix fine-tuning parameter and the question input by the user of the client in the input text to obtain a preprocessing vector Prefix carrying prefix prompt information.

[0119] Here, after obtaining the preprocessing vector Prefix carrying the prefix prompt information, the server can use the preprocessing vector Prefix carrying the prefix prompt information and the question content x entered by the user of the client in the input text as the actual input of the target pre-training model, that is, the actual input of the target pre-training model is (Prefix, x).

[0120] Among them, the preprocessing vector Prefix carrying the prefix prompt information meets the following conditions:

[0121] Prefix=f(p k , x);

[0122] Among them, f() is based on p k Calculate the specific prefix fine-tuning hint calculation method of the preprocessing vector Prefix carrying the prefix hint information, p k Fine-tune parameters for target prefix.

[0123] Here, the server can use (Prefix, x) as the input of the target pre-trained model, and the target pre-trained model processes (Prefix, x) to obtain the text inference result corresponding to the client input text.

[0124] See also Figure 3 As shown, it is an interactive diagram of text reasoning in the reasoning method of a large language model provided by an embodiment of the present disclosure, wherein:

[0125] First, the client sends input text to the server. Secondly, after receiving the input text, the server determines the target prefix fine-tuning parameters of the target pre-trained model based on the input text and the parameter mapping relationship. Secondly, the server processes the input text based on the target prefix fine-tuning parameters to obtain a pre-processing vector carrying the prefix prompt information. Secondly, the server processes the pre-processing vector and the input text through the target pre-trained model to obtain the text inference result of the input text. Finally, the server sends the text inference result to the client.

[0126] In an optional embodiment, in the above Figure 1 Based on the described embodiment, the method further comprises the following steps:

[0127] First, when the target prefix fine-tuning parameters meet the preset training conditions, the training data sent by the client is received;

[0128] Secondly, based on the training data, the client trains the target prefix fine-tuning parameters of the target pre-trained model to obtain the trained prefix fine-tuning parameters;

[0129] Finally, the target prefix fine-tuning parameters are updated to the trained prefix fine-tuning parameters in the parameter mapping relationship, and the trained prefix fine-tuning parameters are sent to the client.

[0130] In an embodiment of the present disclosure, during the process of the client and server session connection, the client may generate training data for the target pre-trained model. Here, when it is determined that the amount of the generated training data is greater than or equal to the preset data amount, it is determined that the target prefix fine-tuning parameter meets the preset training condition.

[0131] In addition, during the session connection between the client and the server, when the number of times the target pre-trained model deployed in the server performs text reasoning for the client is greater than the preset number, it is determined that the target prefix fine-tuning parameters meet the preset training conditions.

[0132] Here, the client can use the data generated in the process of collecting reasoning (i.e., the questions input by the user to which the client belongs in the input text and the text reasoning results) as initial training data. Afterwards, the client can merge the initial training data to obtain training data. Among them, the specific merging process will be introduced in the following embodiment; after that, the client can send the training data to the server. After obtaining the post-training prefix fine-tuning parameters, the server can update the prefix fine-tuning parameters of the client for the target pre-training model in the parameter mapping relationship from the target fine-tuning parameters to the post-training prefix fine-tuning parameters, and at the same time send the post-training prefix fine-tuning parameters to the client. When the client receives the post-training prefix fine-tuning parameters, the post-training prefix fine-tuning parameters can be stored in the local cache area of ​​the client.

[0133] In the above implementation, during the session connection between the server and the client, training data is collected based on the client, so that the server can train the target prefix fine-tuning parameters based on the training data. This allows the entire system (client and server) to not require the user to prepare training data, but to complete the identification and collection of training data during the daily use of the client. This reduces the cost of using the client and further increases the availability of the system.

[0134] See also Figure 4 As shown, it is a flowchart of another large language model reasoning method provided by an embodiment of the present disclosure, the method is applied to a client, and specifically includes steps S201 to S202, wherein:

[0135] S201. Send input text to the server.

[0136] In an embodiment of the present disclosure, before receiving the input text sent by the client, the server may first establish a session connection with the client. The specific session connection process is as described above and will not be described in detail here.

[0137] After the server establishes a session connection with the client, the server may notify the client that the session connection has been established. After that, the server may maintain the current session connection with the client. Based on the maintained session connection, the client may send input text to the server. Here, the input text sent by the client to the server may include the content of the question raised by the client to the server.

[0138] Here, the input text includes but is not limited to: the content of the question raised by the user to which the client belongs and the user identification code corresponding to the client.

[0139] S202. Receive a text inference result sent by the server; wherein the text inference result is obtained by performing text inference on the input text through the target prefix fine-tuning parameters of the pre-trained model and the pre-trained model, the target prefix fine-tuning parameters are prefix fine-tuning parameters for the target pre-trained model determined based on the input text and the parameter mapping relationship, and the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server.

[0140] In an embodiment of the present disclosure, the server may pre-cache the prefix fine-tuning parameters of the client associated with the server relative to the pre-trained model, and then determine the mapping relationship between the target pre-trained model requested by the client and the associated prefix fine-tuning parameters, and cache the association as a parameter mapping relationship.

[0141] Here, the server can determine the target pre-trained model requested by the client based on the user identification code in the input text. After that, the server can determine the parameter mapping relationship pre-cached by the server, and then determine the target prefix fine-tuning parameters of the client for the target pre-trained model based on the parameter mapping relationship and the input text.

[0142] In an embodiment of the present disclosure, the target pre-trained model requested by the client can be determined based on the input text; then, the prefix fine-tuning parameters associated with the target pre-trained model in the parameter mapping relationship can be used as the target prefix fine-tuning parameters.

[0143] In an embodiment of the present disclosure, the server may process the target prefix fine-tuning parameter and the problem content in the input text to obtain a processing result (ie, a preprocessing vector carrying prefix prompt information described in the following embodiment).

[0144] After obtaining the processing results, the server can perform text reasoning on the processing results and question content through the target pre-trained model to obtain text reasoning results.

[0145] After obtaining the text inference result, the text inference result can be sent to the client. In addition, the client can also collect the text inference result, and then determine the training data through the text inference result and the corresponding input text. The target prefix fine-tuning parameter can be readjusted through the training data, so as to obtain a more accurate prefix fine-tuning parameter.

[0146] In the above implementation, by setting the parameter mapping relationship, the prefix fine-tuning parameters set for the same target pre-trained model requested by different clients can be determined. Through this processing method, it is not necessary to set up a corresponding large language model for each client. For the same type of pre-trained model requested, the prefix fine-tuning parameters of the target pre-trained model for each client can be determined through the parameter mapping relationship. For the same type of pre-trained model, there is no need to repeatedly deploy it on the server side, thereby saving the use of server resources and allowing the server to provide services to more clients.

[0147] In an optional embodiment, the method further comprises the following steps:

[0148] First, a session request is sent to the server; wherein the session request is used to instruct the server to search the cached prefix fine-tuning parameters of the client for the target pre-trained model in the parameter mapping relationship;

[0149] Secondly, receiving a parameter upload request sent by the server; the parameter upload request is sent when the server determines that there is no cached prefix fine-tuning parameter in the parameter mapping relationship;

[0150] Finally, send the parameter response to the server.

[0151] In an embodiment of the present disclosure, the client may send a session request to the server, wherein the session request includes, but is not limited to: user information of the user to which the client belongs and all pre-trained models requested to be used.

[0152] After the server receives the session request from the client, the server may determine all pre-trained models that the client is allowed to use from among all deployed pre-trained models.

[0153] After all the pre-trained models described above are determined (referred to as pre-trained model A), the server can then search for the prefix fine-tuning parameters of the client for pre-trained model A in the parameter mapping relationship.

[0154] If the server finds the prefix fine-tuning parameter, it maintains the session connection between the client and the server and ends the session connection process. If the server fails to find the prefix fine-tuning parameter, it determines that the client does not have the cached prefix fine-tuning parameter for pre-trained model A in the parameter mapping relationship. At this time, the server can generate a parameter upload request and send the parameter upload request to the client.

[0155] After the client receives the parameter upload request, it can determine whether there are prefix fine-tuning parameters matching the pre-trained model in the client's local cache. If the client determines that there are prefix fine-tuning parameters matching the pre-trained model A in the cache, it can generate a parameter response based on the prefix fine-tuning parameters and send the parameter response to the server.

[0156] After obtaining the prefix fine-tuning parameter, the server can determine the prefix fine-tuning parameter of the client for the pre-trained model A, thereby obtaining a mapping relationship among the client, the pre-trained model A and the prefix fine-tuning parameter, and establishing a mapping relationship among the client, the pre-trained model A and the prefix fine-tuning parameter in the parameter mapping relationship. At the same time, the prefix fine-tuning parameter can also be stored in the cache area of ​​the prefix fine-tuning parameter in the server, and the prefix fine-tuning parameter can be sent to the client for caching.

[0157] When the client determines that there are no prefix fine-tuning parameters matching the pre-trained model A in the local cache, the parameter response generated at this time does not carry the prefix fine-tuning parameters. At this time, the server can pre-select the default prefix fine-tuning parameters (i.e., the preset prefix fine-tuning parameters) set for the pre-trained model A, and store the default prefix parameters as the prefix fine-tuning parameters of the client for the pre-trained model A in the cache of the prefix fine-tuning parameters. Among them, the default prefix fine-tuning parameters can be set according to actual needs. For example, the server can set different default prefix fine-tuning parameters according to the actual needs of the client.

[0158] Here, after the server determines the preset prefix fine-tuning parameter as the prefix fine-tuning parameter of the client for pre-trained model A, it can establish an association relationship (i.e., a mapping relationship) between the client and the prefix fine-tuning parameter, and add the association relationship to the parameter mapping relationship.

[0159] In an optional implementation, a method for collecting training data matching the above-mentioned reasoning method is also included:

[0160] First, based on the input text and the text inference result, training data is generated, and when the target prefix fine-tuning parameters meet the preset training conditions, the training data is sent to the server.

[0161] Then, after sending the training data to the server, the server receives the trained prefix fine-tuning parameters obtained by training the client for the target prefix fine-tuning parameters of the target pre-trained model based on the training data, and caches the trained prefix fine-tuning parameters.

[0162] In an embodiment of the present disclosure, during the process of the client and server session connection, training data of the client for the target pre-trained model may be generated. Here, when it is determined that the amount of the generated training data is greater than or equal to the preset amount of data, it can be determined that the target prefix fine-tuning parameter meets the preset training condition.

[0163] In addition, during the session connection between the client and the server, if it is determined that the duration of the session connection between the server and the client is greater than a preset time, it is determined that the target prefix fine-tuning parameter meets the preset training condition.

[0164] Here, the client can use the data generated during the collection and reasoning process (i.e., the questions input by the user of the client in the input text and the text reasoning results) as initial training data. Afterwards, the client can merge the initial training data to obtain training data. Afterwards, the client can send the training data to the server.

[0165] In an embodiment of the present disclosure, after obtaining the post-training prefix fine-tuning parameter, the server can update the client's prefix fine-tuning parameter for the target pre-training model in the parameter mapping relationship from the target fine-tuning parameter to the post-training prefix fine-tuning parameter, and send the post-training prefix fine-tuning parameter to the client at the same time. When the client receives the post-training prefix fine-tuning parameter, the post-training prefix fine-tuning parameter can be stored in the client's local cache.

[0166] In an optional implementation, the above steps generate training data based on the input text and the text inference result, and specifically include the following steps:

[0167] First, a first input text and a second input text that are successively input to the server are determined; wherein the second input text is the text that is input to the server after the first input text;

[0168] Then, when it is determined that the first input text and the second input text meet the preset combination condition, training data is determined based on the text reasoning results of the first input text and the second input text.

[0169] In an embodiment of the present disclosure, the time interval between collecting the first input text and the second input text is less than a preset time interval, wherein the first input text and the second input text both belong to the above-mentioned input text.

[0170] Here, when the first input text and the second input text are continuously input to the server, the second input text may be a supplementary question to the first input text. For example, the first input text is "What is a large model", and the second input text is "Explain in more detail" or "What is the difference between a small model and a large model". At this time, it can be considered that the topic of the first input text and the second input text for the user of the client is the same, so the second input text can be used as a supplementary question to the first input text.

[0171] In the case where the second input text is a supplementary question of the first input text, it is determined that the first input text and the second input text meet the preset combination condition. At this time, the inference results of the first input text and the second input text can be merged to obtain a merged result of the inference results of the first input text and the second input text, and the merged result is determined as training data.

[0172] See also Figure 5 FIG. 1 is a flowchart of training data processing in the inference method of a large language model provided by an embodiment of the present disclosure, wherein:

[0173] First, obtain a first input text. Secondly, receive a text inference result of the first input text sent by the server. Secondly, obtain a second input text. Secondly, receive a text inference result of the second input text sent by the server. Secondly, determine whether the first input text and the second input text meet a preset combination condition. Finally, if the first input text and the second input text meet the preset combination condition, merge the inference results of the first input text and the second input text to obtain a merged result, and store the merged result as training data.

[0174] In an optional implementation, the above step of determining that the first input text and the second input text meet a preset combination condition specifically includes the following steps:

[0175] First, determine the similarity between the second input text and the preset text; wherein the preset text is a preset supplementary description text;

[0176] Secondly, in the case where it is determined that the similarity between the second input text and the preset text is less than the first similarity threshold, the similarity between the first input text and the second input text is determined;

[0177] Finally, when it is determined that the similarity between the first input text and the second input text is greater than the second similarity threshold, it is determined that the first input text and the second input text meet the preset combination condition.

[0178] In an embodiment of the present disclosure, the server may pre-generate a vector library, wherein the vector library includes sentence vectors corresponding to preset texts. The preset texts are supplementary description texts of preset standard supplementary questions, such as "say it in more detail", "be more refined", and "say it again".

[0179] Here, see Figure 6 As shown, it is a flowchart of determining training data in the inference method of a large language model provided by an embodiment of the present disclosure, wherein:

[0180] First, the sentence vector of the second input text can be determined. Secondly, it is determined whether the cosine distance between the sentence vector of the second input text and each sentence vector in the vector library is greater than a first preset distance threshold. Here, the similarity between the sentence vector of the second input text and each sentence vector in the vector library can be calculated by calculating the cosine distance between the sentence vector of the second input text and each sentence vector in the vector library.

[0181] Secondly, when it is determined that the cosine distance between the sentence vector of the second input text and any sentence vector in the vector library is less than or equal to the first preset distance threshold (that is, the similarity between the sentence vector of the second input text and any sentence vector in the vector library is greater than or equal to the first similarity threshold), it is determined that the first input text and the second input text meet the preset combination condition.

[0182] Secondly, when it is determined that the cosine distance between the sentence vector of the second input text and any sentence vector in the vector library is greater than a preset distance threshold, the sentence vector of the first input text is determined. Secondly, it is determined whether the cosine distance between the sentence vector of the first input text and the sentence vector of the second input text is greater than a second preset distance threshold.

[0183] Finally, when the cosine distance between the sentence vector of the first input text and the sentence vector of the second input text is less than or equal to the first preset distance threshold (i.e., the similarity between the sentence vector of the second input text and the sentence vector of the first input text is greater than or equal to the second similarity threshold), it is determined that the first input text and the second input text meet the preset combination condition.

[0184] In the above implementation, during the session connection between the client and the server, a method for collecting training data for training prefix fine-tuning parameters based on specific session content facilitates the process of collecting training data and makes it easier to perform personalized training on the client for the prefix fine-tuning parameters of the target pre-trained model.

[0185] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0186] Based on the same inventive concept, the embodiments of the present disclosure also provide an inference system of a large language model corresponding to the inference method of the large language model. Since the principle of solving the problem by the system in the embodiments of the present disclosure is similar to the inference method of the large language model in the embodiments of the present disclosure, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be repeated.

[0187] Reference Figure 7 As shown, it is a schematic diagram of a large language model reasoning system provided by an embodiment of the present disclosure, the system includes: a client and a server; wherein,

[0188] The server includes an inference system and a fine-tuning system; the inference system includes a pre-trained model module, a prefix fine-tuning parameter cache module, and a session maintenance module; the fine-tuning system includes a prefix fine-tuning parameter fine-tuning module. The client includes a session maintenance module, a training data collection module, and a prefix parameter storage module.

[0189] Here, the pre-trained model module of the server is used to store the pre-trained models deployed on the server, and when receiving the input text sent by the client, call the pre-trained model to infer the input text to obtain the text inference result.

[0190] Here, the prefix fine-tuning parameter cache module of the server is used to cache the prefix fine-tuning parameters of each pre-trained model, and cache the mapping relationship between the client, the pre-trained model and the associated prefix fine-tuning parameters (ie, the parameter mapping relationship).

[0191] Here, the session maintaining module of the server is used to maintain the session connection with the client.

[0192] Here, the prefix fine-tuning parameter fine-tuning module of the server is used to train the prefix fine-tuning parameters based on the training data.

[0193] Here, the session maintaining module of the client is used to maintain the session connection with the server.

[0194] Here, the training data collection module of the client is used to collect the training data generated during the session connection process.

[0195] Here, the client's prefix parameter saving module is used to save the client's prefix fine-tuning parameters.

[0196] Based on the same inventive concept, the embodiments of the present disclosure also provide an inference device for a large language model corresponding to the inference method for a large language model. Since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to the inference method for the large language model in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0197] Reference Figure 8 As shown, it is a schematic diagram of a large language model reasoning device provided by an embodiment of the present disclosure, the device comprises: a first receiving module 11, a determining module 12, and a text reasoning module 13; wherein,

[0198] A first receiving module, configured to obtain an input text sent by the client in response to an inference request from the client;

[0199] A determination module, configured to determine, based on the input text and the parameter mapping relationship, a target prefix fine-tuning parameter of the client for the target pre-trained model; wherein the parameter mapping relationship is used to indicate the prefix fine-tuning parameter associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server;

[0200] The text reasoning module is used to perform text reasoning on the input text through the target prefix fine-tuning parameters and the target pre-training model, obtain a text reasoning result, and send the text reasoning result to the client.

[0201] Reference Fig. 9 As shown, it is a schematic diagram of another large language model inference device provided by an embodiment of the present disclosure, and the device includes: a sending module 21 and a second receiving module 22; wherein,

[0202] The sending module is used to send input text to the server;

[0203] A second receiving module is used to receive the text inference result sent by the server; wherein the text inference result is obtained by performing text inference on the input text through the target prefix fine-tuning parameters of the pre-trained model and the pre-trained model, the target prefix fine-tuning parameters are prefix fine-tuning parameters for the target pre-trained model determined based on the input text and the parameter mapping relationship, and the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server.

[0204] The disclosed embodiment can determine the prefix fine-tuning parameters set for the same target pre-trained model requested by different clients by setting a parameter mapping relationship. With this processing method, it is not necessary to set a corresponding large language model for each client. For the same type of pre-trained model requested, the prefix fine-tuning parameters of the target pre-trained model for each client can be determined through the parameter mapping relationship. For the same type of pre-trained model, there is no need to repeatedly deploy it on the server side, thereby saving the use of server resources and allowing the server to provide services to more clients.

[0205] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0206] Corresponds to Figure 1 The embodiment of the present disclosure also provides an electronic device 1000, such as Fig.10 FIG. 1 is a schematic diagram of the structure of an electronic device 1000 provided in an embodiment of the present disclosure, including:

[0207] Processor 101, memory 102, and bus 103; memory 102 is used to store execution instructions, including internal memory 1021 and external memory 1022; the internal memory 1021 is also called internal memory, which is used to temporarily store the operation data in the processor 101 and the data exchanged with the external memory 1022 such as a hard disk. The processor 101 exchanges data with the external memory 1022 through the internal memory 1021. When the electronic device 1000 is running, the processor 101 communicates with the memory 102 through the bus 103, so that the processor 101 executes the following instructions:

[0208] Receive input text sent by the client;

[0209] Based on the input text and the parameter mapping relationship, determining the target prefix fine-tuning parameters of the client for the target pre-trained model; wherein the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server;

[0210] The input text is subjected to text reasoning by using the target prefix fine-tuning parameter and the target pre-training model to obtain a text reasoning result, and the text reasoning result is sent to the client.

[0211] The processor 101 may also execute the following instructions:

[0212] Send input text to the server;

[0213] Receive a text inference result sent by the server; wherein the text inference result is obtained by performing text inference on the input text through the target prefix fine-tuning parameters of the pre-trained model and the pre-trained model, the target prefix fine-tuning parameters are prefix fine-tuning parameters for the target pre-trained model determined based on the input text and the parameter mapping relationship, and the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server.

[0214] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limiting. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0215] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0216] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0217] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0218] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0219] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0220] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for reasoning about a large language model, characterized in that: Applied to the server, including: In response to an inference request from a client, obtaining an input text sent by the client; Based on the input text and the parameter mapping relationship, determining the target prefix fine-tuning parameters of the client for the target pre-trained model; wherein the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server; The input text is subjected to text reasoning by using the target prefix fine-tuning parameter and the target pre-training model to obtain a text reasoning result, and the text reasoning result is sent to the client.

2. The method according to claim 1, characterized in that The method further comprises: In response to a session request from the client, searching the parameter mapping relationship for cached prefix fine-tuning parameters of the client for the target pre-trained model; When it is determined that the cached prefix fine-tuning parameter does not exist in the parameter mapping relationship, sending a parameter upload request to the client; When it is determined that the parameter response sent by the client carries a prefix fine-tuning parameter, the prefix fine-tuning parameter carried in the parameter response is determined as the target prefix fine-tuning parameter, and a mapping relationship between the client and the target prefix fine-tuning parameter is established in the parameter mapping relationship.

3. The method according to claim 2, characterized in that The method further comprises: In a case where it is determined that the parameter response sent by the client does not carry the prefix fine-tuning parameter, determining a preset prefix fine-tuning parameter matching the client as the target prefix fine-tuning parameter; A mapping relationship between the client and the target prefix fine-tuning parameter is established in the parameter mapping relationship.

4. The method according to claim 1, characterized in that The determining, based on the input text and the parameter mapping relationship, a target prefix fine-tuning parameter of the target pre-trained model by the client includes: Parsing the inference request to obtain a user identification code of the client; Determining a prefix fine-tuning parameter that matches the user identification code in the parameter mapping relationship; The target prefix fine-tuning parameter is determined based on the matched prefix fine-tuning parameter.

5. The method according to claim 1, characterized in that The step of performing text reasoning on the input text by using the target prefix fine-tuning parameter and the pre-trained model to obtain a text reasoning result includes: Processing the input text based on the target prefix fine-tuning parameter to obtain a preprocessing vector representing prefix prompt information; wherein the prefix prompt information is associated with the target prefix fine-tuning parameter; The preprocessing vector and the input text are processed by the target pre-training model to obtain a text reasoning result of the input text, and the text reasoning result is sent to the client.

6. The method according to claim 1, characterized in that The method further comprises: When the target prefix fine-tuning parameter satisfies a preset training condition, receiving training data sent by the client; Based on the training data, the client trains the target prefix fine-tuning parameters for the target pre-trained model to obtain trained prefix fine-tuning parameters; The target prefix fine-tuning parameter is updated to the trained prefix fine-tuning parameter in the parameter mapping relationship, and the trained prefix fine-tuning parameter is sent to the client.

7. A method for reasoning about a large language model, characterized in that: Applied to the client, including: Send input text to the server; Receive a text inference result sent by the server; wherein the text inference result is obtained by performing text inference on the input text through the target prefix fine-tuning parameters of the pre-trained model and the pre-trained model, the target prefix fine-tuning parameters are prefix fine-tuning parameters for the target pre-trained model determined based on the input text and the parameter mapping relationship, and the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server.

8. The method according to claim 7, characterized in that The method further comprises: Sending a session request to the server; wherein the session request is used to instruct the server to search the cached prefix fine-tuning parameters of the client for the target pre-trained model in the parameter mapping relationship; receiving a parameter upload request sent by the server; the parameter upload request is sent when the server determines that the cached prefix fine-tuning parameter does not exist in the parameter mapping relationship; Send a parameter response to the server.

9. The method according to claim 7, characterized in that Also included is a method for collecting training data that matches the inference method: Generate training data based on the input text and the text inference result, and send the training data to the server when the target prefix fine-tuning parameter meets the preset training condition; After sending the training data to the server, the server receives the trained prefix fine-tuning parameters obtained by training the client for the target prefix fine-tuning parameters of the target pre-trained model based on the training data, and caches the trained prefix fine-tuning parameters.

10. The method according to claim 9, characterized in that The generating training data based on the input text and the text reasoning result includes: Determine a first input text and a second input text that are continuously input to the server; wherein the second input text is text that is input to the server after the first input text; In a case where it is determined that the first input text and the second input text meet a preset combination condition, the training data is determined based on text reasoning results of the first input text and the second input text.

11. The method according to claim 10, characterized in that The determining that the first input text and the second input text meet a preset combination condition includes: Determining the similarity between the second input text and a preset text; wherein the preset text is a preset supplementary description text; In the case where it is determined that the similarity between the second input text and the preset text is less than a first similarity threshold, determining the similarity between the first input text and the second input text; In the case where it is determined that the similarity between the first input text and the second input text is greater than a second similarity threshold, it is determined that the first input text and the second input text meet the preset combination condition.

12. A large language model reasoning device, characterized in that: include: A first receiving module, configured to obtain an input text sent by the client in response to an inference request from the client; A determination module, configured to determine a target prefix fine-tuning parameter of the client for a target pre-trained model based on the input text and the parameter mapping relationship; wherein the parameter mapping relationship is used to indicate a prefix fine-tuning parameter associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server; The text reasoning module is used to perform text reasoning on the input text through the target prefix fine-tuning parameters and the target pre-training model, obtain a text reasoning result, and send the text reasoning result to the client.

13. A large language model reasoning device, characterized in that: include: The sending module is used to send input text to the server; A second receiving module is used to receive the text inference result sent by the server; wherein the text inference result is obtained by performing text inference on the input text through the target prefix fine-tuning parameters of the pre-trained model and the pre-trained model, the target prefix fine-tuning parameters are prefix fine-tuning parameters for the target pre-trained model determined based on the input text and the parameter mapping relationship, and the parameter mapping relationship is used to indicate the prefix fine-tuning parameters associated with the target pre-trained model requested by the client among the prefix fine-tuning parameters cached by the server.

14. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 11.

15. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

16. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images