Service processing method and device based on large language model

Abstract

The embodiment of the invention provides a service processing method and device based on a large language model, and the method comprises the steps: obtaining a resource processing instruction and a plurality of data sequences, which are carried by a call request of a resource service, in a service processing process for the resource service, determining a sequence type of each data sequence according to the sequence length of each data sequence, and allocating the resource processing instruction and each data sequence to a model reasoning service corresponding to the sequence type, so that the model reasoning service inputs the resource processing instruction and each data sequence to a large language model for reasoning processing and obtains reasoning data, and finally, generating a resource processing result according to reasoning data returned by the model reasoning service, and returning the resource processing result to the resource service, thereby realizing resource service processing through the model reasoning service.

Description

Technical Field

[0001] This document relates to the field of data processing technology, and in particular to a service processing method and device based on a large language model. Background Art

[0002] With the continuous development of Internet technology, various services are provided to users online to understand and respond to user queries and needs. As users' requirements for service quality continue to increase, the application of large language models provides a feasible means for user service processing. However, with the continuous popularization of large language models and the continuous expansion of their application scope, users' requirements for the performance and processing efficiency of large language models are also increasing. Therefore, how to improve the processing efficiency of large language models in the service processing process has become a focus of attention for all parties. Summary of the Invention

[0003] One or more embodiments of the present specification provide a service processing method based on a large language model, including: obtaining resource processing instructions and multiple data sequences carried in a call request of a resource service, and the multiple data sequences are obtained by data retrieval through the resource service. The sequence type of each data sequence is determined according to the sequence length of each data sequence. The resource processing instructions and each data sequence are assigned to a model inference service corresponding to the sequence type, and the model inference service obtains inference data by inputting the resource processing instructions and each data sequence into a large language model for inference processing. A resource processing result is generated based on the inference data returned by the model inference service and returned to the resource service.

[0004] One or more embodiments of the present specification provide a service processing device based on a large language model, including: a sequence acquisition module, configured to obtain resource processing instructions and multiple data sequences carried in a call request of a resource service, and the multiple data sequences are obtained by data retrieval through the resource service. A sequence type determination module, configured to determine the sequence type of each data sequence according to the sequence length of each data sequence. A sequence allocation module, configured to allocate the resource processing instructions and each data sequence to a model inference service corresponding to the sequence type, and the model inference service obtains inference data by inputting the resource processing instructions and each data sequence into a large language model for inference processing. A result generation module, configured to generate a resource processing result based on the inference data returned by the model inference service and return it to the resource service.

[0005] One or more embodiments of the present specification provide a service processing device based on a large language model, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to: obtain resource processing instructions and multiple data sequences carried in a call request of a resource service, wherein the multiple data sequences are obtained by data retrieval through the resource service. The sequence type of each data sequence is determined according to the sequence length of each data sequence. The resource processing instructions and each data sequence are assigned to a model inference service corresponding to the sequence type, and the model inference service obtains inference data by inputting the resource processing instructions and each data sequence into a large language model for inference processing. A resource processing result is generated based on the inference data returned by the model inference service and returned to the resource service.

[0006] One or more embodiments of this specification provide a computer-readable storage medium for storing computer-executable instructions, which implement the following process when executed: obtaining resource processing instructions and multiple data sequences carried in a call request to a resource service, and the multiple data sequences are obtained by data retrieval through the resource service. Determine the sequence type of each data sequence based on the sequence length of each data sequence. Assign the resource processing instructions and each data sequence to a model inference service corresponding to the sequence type, and the model inference service obtains inference data by inputting the resource processing instructions and each data sequence into a large language model for inference processing. Generate a resource processing result based on the inference data returned by the model inference service and return it to the resource service. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate one or more embodiments of this specification or technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments described in this specification. Those skilled in the art can derive other drawings based on these drawings without inventive effort. Figure 1 A schematic diagram of an implementation environment for a service processing method based on a large language model provided in one or more embodiments of this specification; Figure 2 A processing flow chart of a service processing method based on a large language model provided in one or more embodiments of this specification; Figure 3 A processing flow chart of a service processing method based on a large language model applied to a first resource service processing scenario provided in one or more embodiments of this specification; Figure 4 A processing flow chart of a service processing method based on a large language model applied to a second resource service processing scenario provided by one or more embodiments of this specification; Figure 5 A schematic diagram of an embodiment of a service processing device based on a large language model provided in one or more embodiments of this specification; Figure 6 A schematic diagram of the structure of a service processing device based on a large language model provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0008] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this document.

[0009] The service processing method based on the large language model provided in one or more embodiments of this specification can be applied to the implementation environment of resource service processing. Figure 1 , the implementation environment at least includes: a client 101, a long-short separation module 102, a model inference service 103 and a large language model 104; The client 101 can run a resource service, which sends a call request to the long-short separation module 102, and receives the resource processing results obtained by the long-short separation module 102 based on the call request and displays them to the user; the client 101 can run on a terminal device, which can specifically be a mobile phone, personal computer, tablet computer, e-book reader, device based on VR (Virtual Reality) for information interaction, vehicle-mounted terminal, IoT device, wearable smart device, laptop portable computer and desktop computer, etc. The terminal device can install the client 101, and the client 101 can be an application or a subroutine within an application.

[0010] The long-short separation module 102 is used to obtain resource processing instructions and data sequences of the resource service running on the client 101, distribute the resource processing instructions and data sequences to the model inference service to obtain inference data, and return the resource processing results generated based on the inference data to the client 101; the model inference service 103 is used to interact with the long-short separation module 102 to input the resource processing instructions and data sequences into the large language model 104, and return the inference data obtained by the large language model 104 through inference processing to the long-short separation module 102; the large language model 104 is used to perform inference processing based on the resource processing instructions and data sequences and generate inference data, and return the inference data to the model inference service 103; The long-short separation module 102, the model inference service 103 and the large language model 104 can be deployed on the server side, and the server side can run on a server. The server can be one server or multiple servers, or a server cluster composed of several servers, or one or more cloud servers in a cloud computing platform.

[0011] In this implementation environment, after the resource service running on the client 101 obtains the resource processing instructions and multiple data sequences, it sends the resource processing instructions and multiple data sequences to the long-short separation module 102. The long-short separation module 102 obtains the resource processing instructions and multiple data sequences of the resource service, and assigns the resource processing instructions and each data sequence to the model inference service 103 corresponding to the sequence type of the data sequence, so that the model inference service 103 inputs the resource processing instructions and each data sequence into the corresponding large language model 104 for inference processing to obtain inference data and return it to the long-short separation module 102. The long-short separation module 102 generates a resource processing result based on the inference data returned by the model inference service 103 and returns it to the resource service running on the client 101. By assigning the resource processing instructions and each data sequence to different model inference services, the response efficiency of the resource processing instructions in the process of user access to the resource service is improved.

[0012] One or more embodiments of a service processing method based on a large language model provided in this specification are as follows: Reference Figure 2 The service processing method based on the large language model provided in this embodiment specifically includes steps S202 to S208.

[0013] Step S202: Obtain resource processing instructions and multiple data sequences carried in the resource service call request.

[0014] The resource service described in this embodiment refers to providing resource-related services, such as a resource service that returns resource processing results such as resource object recommendations, resource project recommendations, and resource transaction suggestions to the user based on the resource-related query content (query text) input by the user. The resource object can be an equity resource, such as a fund, stock and / or bond, and the resource object can also be a capital resource, and the resource project can be a resource management project for performing resource management.

[0015] In this embodiment, the call request is a request issued by the resource service for model reasoning. Specifically, during the model reasoning process, the call request carries resource processing instructions and data sequences, where the resource processing instructions are used to guide the model reasoning processing, and the data sequence is the data relied upon in the model reasoning process.

[0016] In specific implementation, in this embodiment, a user can access a resource service through an application or a subroutine of the application and input a resource-related query text based on the resource service. After the resource service obtains the query text input by the user, on the one hand, the resource service can call a retrieval model to search the knowledge base based on the query text to obtain a data sequence related to the query text. On the other hand, the resource service can also generate a resource processing instruction based on the query text input by the user and send a call request carrying the resource processing instruction and the data sequence. Accordingly, here, the call request for the resource service is received, along with the resource processing instruction and multiple data sequences carried in the call request. Optionally, the multiple data sequences are obtained by the resource service calling the retrieval model to search the knowledge base for data.

[0017] During the specific execution process, in order to improve the accuracy and efficiency of the resource processing process in the process of performing resource processing based on the resource processing instruction, the resource service may first call the retrieval model to obtain relevant data sequences through data retrieval before sending the call request, and determine the data sequence in the relevant data sequences for subsequent resource processing. In an optional implementation provided by this embodiment, the data retrieval includes: A query text based on the resource service is searched in the knowledge base to obtain at least one relevant data sequence; a relevance score of each relevant data sequence is calculated and a data sequence is determined according to the relevance score and a preset score threshold.

[0018] The data sequence refers to a data unit, specifically one or more data units related to the user's query text. For example, the data sequence can be an article fragment, news information, resource planning suggestions, and / or other resource-related text data, such as information about resource projects. Optionally, the data sequence is obtained by the resource service calling a retrieval model to perform data retrieval in the knowledge base.

[0019] In the process of data retrieval, the query text entered by the user in the resource service is first obtained, and then the retrieval model is called to search the knowledge base for relevant data sequences related to the query text. For the relevant data sequences retrieved, the relevance score of each relevant data sequence and the query text is calculated, and the calculated relevance score is compared with the preset score threshold. If the relevance score is lower than the score threshold, this part of the relevant data sequence is filtered. If the relevance score is higher than the score threshold, this part of the relevant data sequence is determined as the data sequence for subsequent resource processing.

[0020] In specific implementation, during the resource processing process, after the resource service obtains the resource processing instructions submitted by the user and retrieves multiple data sequences, the resource service sends a call request carrying the resource processing instructions and data sequences. Correspondingly, the resource service's call request and the resource processing instructions and multiple data sequences carried in the call request are obtained.

[0021] Step S204: Determine the sequence type of each data sequence according to the sequence length of each data sequence.

[0022] In specific implementation, after obtaining multiple data sequences as mentioned above, in order to assign data sequences of different sequence types to corresponding model inference services for inference processing, thereby improving the efficiency of subsequent calls to model inference services for inference processing, here, the sequence type of each data sequence is determined based on the sequence length of each data sequence.

[0023] During the specific execution process, in order to improve the efficiency of subsequent calls to the model inference service for inference processing, the sequence type of the data sequence can be determined according to the sequence length of the data sequence, so that data sequences of different sequence types are assigned to the corresponding model inference service for inference processing, thereby more effectively allocating computing power. In an optional implementation provided by this embodiment, the sequence type of each data sequence is determined according to the sequence length of each data sequence, including: Calculate the sequence length of each data sequence and compare the sequence length with the preset length threshold; A data sequence whose sequence length is less than a preset length threshold is determined as a short sequence type, and / or a data sequence whose sequence length is greater than a preset length threshold is determined as a long sequence type.

[0024] Specifically, in the process of determining the sequence type of each data sequence according to the sequence length of each data sequence, the sequence length of each data sequence is first calculated, and then the calculated sequence length is compared with a preset length threshold. If the length of the current data sequence is less than the preset length threshold, the sequence type of the data sequence is determined to be a short sequence type. If the length of the current data sequence is greater than or equal to the preset length threshold, the sequence type of the data sequence is determined to be a long sequence type. Based on this, the data sequence is determined to be a short sequence type and / or a long sequence type.

[0025] Furthermore, in an optional implementation provided by this embodiment, calculating the sequence length of each data sequence includes: counting the number of characters and / or bytes of each data sequence, and using the number of characters and / or bytes as the sequence length.

[0026] For example, a data sequence obtained by query text retrieval for resource services is: the rate of return of resource project A since its establishment is xx.xx%, and the investment cost-effectiveness is high; the number of characters in the data sequence is 26, so the sequence length of the data sequence is 26.

[0027] In addition, in order to ensure the efficiency and accuracy of calculating the sequence length of data sequences for different types of data sequences, such as data sequences with special structures or containing complex information, the sequence length calculation rules of the data sequences can be preset according to actual needs, and the sequence length can be calculated according to the sequence length calculation rules, that is, calculating the sequence length of each data sequence also includes: calculating the sequence length based on the preset sequence length calculation rules.

[0028] Specifically, the sequence length is calculated based on a preset sequence length calculation rule. It can be to determine the partial sequence length of the data sequence and use the partial sequence length as the sequence length of the data sequence, or to determine the length weight of the character type to which the characters contained in the data sequence belong, and determine the sequence length of the data sequence by weighted sum. It can also be to determine the sequence length of the data sequence based on the number of semantic units contained in the data sequence and / or the average length of the semantic units.

[0029] For example, the sequence length of 50% of the data in each data sequence is calculated; or, for a data sequence containing multiple character types such as ordinary text characters, numbers, and currency symbols, different weights are assigned to each character type, and the weighted sum of the lengths of each character is used as the sequence length of the data sequence; or, for a data sequence composed of multiple independent and complete sentences, the sequence length of the data sequence can be determined based on the number of semantic units (i.e., sentences) and the average length of the semantic units.

[0030] In actual applications, in addition to the need to assign data sequences to corresponding model inference services based on their sequence types, there may also be situations where other parameters need to be combined to determine how to assign data sequences to corresponding model inference services. For example, the allocation parameters of each data sequence are determined based on the sequence length of each data sequence and the load index of the model inference service. In this case, determining the sequence type of each data sequence based on the sequence length of each data sequence can be replaced by determining the allocation parameters of each data sequence based on the sequence length of each data sequence and the load index of the model inference service. Alternatively, it can also be replaced by determining the allocation parameters of each data sequence based on the sequence type of each data sequence and the load index of the model inference service.

[0031] Among them, the load index model refers to the number of data sequences to be processed in the queue of the model inference service; specifically, if the sequence type of the data sequence is a short sequence type and the load index of the model inference service does not reach the index threshold, then the allocation parameter of the data sequence is determined to be the first allocation parameter; if the sequence type of the data sequence is a long sequence type and the load index of the model inference service does not reach the index threshold, then the allocation parameter of the data sequence is determined to be the second allocation parameter; if the sequence type of the data sequence is a long sequence type and the load index of the model inference service has reached the index threshold, then the allocation parameter of the data sequence is determined to be the third allocation parameter.

[0032] Step S206: Allocate the resource processing instruction and each data sequence to a model inference service corresponding to the sequence type.

[0033] The model inference service can be an inference service for any type of model, for example, the model inference service can be an inference service for a large language model (LLM). The model inference service is used to perform online model inference. Here, the model inference service can be at least one model inference service. The specific model inference service can be a short sequence model inference service (first model inference service) and / or a long sequence model inference service. The short sequence model inference service is used to process shorter data sequences (short sequence type data sequences). Due to the small amount of data, the short sequence model inference service has a faster processing speed and higher response efficiency. The long sequence model inference service is used to process longer data sequences (long sequence type data sequences). Due to the more complex processing process, the long sequence model inference service has a larger processing time and resource consumption.

[0034] Among them, the large language model can adopt an open source large language model, or a large language model obtained by fine-tuning the open source large language model, or a large language model can be obtained by constructing a large language model and performing model training; accordingly, the large language model can be a first large language model used for inference processing based on a short sequence type data sequence, and / or a second large language model used for inference processing based on a long sequence type data sequence.

[0035] In actual applications, in order to improve the overall throughput of the model inference service and enhance the response speed of the model inference service, a long-short separation module can be deployed on the server. The long-short separation module can route data sequences and resource processing instructions to the appropriate model inference service according to the sequence type of the data sequence, thereby avoiding the overall inference processing delay caused by the long time-consuming inference processing of long sequence type data sequences. In addition, according to actual processing needs, the long-short separation module can also be replaced with a long-short separator.

[0036] In specific implementation, when determining the sequence type of the data sequence as described above, since the short sequence model inference service has a faster processing speed and higher response efficiency, while the long sequence model inference service has a larger processing time and resource consumption, the processing efficiency of the data sequence can be improved by allocating different types of data sequences to different model inference services. Here, the resource processing instructions and each data sequence are allocated to the model inference service corresponding to the sequence type.

[0037] Afterwards, after assigning the resource processing instructions and each data sequence to the model inference service corresponding to the sequence type, the multiple model inference services obtain the resource processing instructions and each data sequence, and input the resource processing instructions and each data sequence into the corresponding large language model, so that each language model performs inference processing based on the resource processing instructions and data sequence. After each language model completes the inference processing and obtains multiple inference data, it can return each inference data to the corresponding model inference service. Optionally, the model inference service obtains the inference data by inputting the resource processing instructions and each data sequence into the large language model for inference processing.

[0038] Optionally, the reasoning processing includes: search processing for information search based on the query text of the resource service, item recommendation processing for recommending resource service items based on the query text, and / or content recommendation processing for recommending preferred content based on the query text.

[0039] For example, based on the query text submitted by the user based on the resource service, search processing can be performed on the query text and the retrieved data sequence to provide the user with information search results for the query text, or item recommendation processing can be performed on the query text and the retrieved data sequence to provide the user with at least one resource service item related to the query question, or content recommendation processing can be performed on the query text and the retrieved data sequence to recommend preferred content to provide the user with resource processing results preferred by the user for the query question.

[0040] During the specific execution process, in order to improve the processing efficiency of short sequence type data sequences, multiple short sequence type data sequences can be processed concurrently. In an optional implementation provided by this embodiment, resource processing instructions and each data sequence are assigned to a model inference service corresponding to the sequence type, including: If the sequence type is a short sequence type, the data sequence is assigned to the first model inference service.

[0041] Specifically, if the sequence type of the data sequence is determined to be a short sequence type based on the sequence length, the resource processing instruction and the data sequence of the short sequence type are assigned to the first model inference service. Accordingly, after the first model inference service receives the resource processing instruction and the data sequence, it uses continuous batching to input the resource processing instruction and the data sequence into the first large language model, so that the first large language model performs inference processing based on the resource processing instruction and the data sequence to obtain the first inference data.

[0042] Optionally, the first model inference service uses a continuous batch processing method to input resource processing instructions and data sequences into the first large language model; the first large language model performs inference processing to obtain first inference data.

[0043] During the specific execution process, in another optional implementation provided by this embodiment, resource processing instructions and each data sequence are assigned to a model inference service corresponding to the sequence type, including: If the sequence type is a long sequence type, the data sequence is assigned to the second model inference service.

[0044] Specifically, if the sequence type of the data sequence is determined to be a long sequence type based on the sequence length, the resource processing instruction and the data sequence of the long sequence type are assigned to the second model inference service. Accordingly, after the second model inference service receives the resource processing instruction and the data sequence, it inputs the resource processing instruction and the data sequence into the second largest language model, so that the second largest language model uses the paged attention mechanism (Paged Attention) to perform inference processing to obtain the second inference data.

[0045] Optionally, the second model inference service inputs the resource processing instructions and data sequence into the second largest language model; the second largest language model uses the paging attention mechanism to perform inference processing to obtain second inference data.

[0046] In actual applications, after allocating short sequence type data sequences and long sequence type data sequences to corresponding model inference services, the short sequence type data sequences occupy less resources, so the processing speed of the short sequence type data sequences is faster, while the long sequence type data sequences may require more computing resources and time to process. Therefore, the second model inference service may have reached its load limit, while the first model inference service still has more idle resources. In this case, in order to improve the overall throughput and processing efficiency, the long sequence type data sequences can be processed to allocate the long sequence type data sequences to the first model inference service.

[0047] In a specific implementation process, an optional implementation provided by this embodiment allocates resource processing instructions and each data sequence to a model inference service corresponding to the sequence type, including: Detecting whether the load indicator of the second model inference service reaches the indicator threshold; If so, the data sequence of the long sequence type is compressed to obtain a compressed sequence, and the compressed sequence and the data sequence of the short sequence type are assigned to the first model inference service.

[0048] Specifically, it is possible to detect whether the load index of the second model inference service has reached the index threshold. When it is detected that the load index of the second model inference service has reached the index threshold, the data sequence of the long sequence type to be assigned can be compressed to obtain a compressed sequence, such as obtaining and retaining the summary of the data sequence and / or the key information of the data sequence, and using the summary of the data sequence and / or the key information of the data sequence as a compressed sequence, so that the sequence length of the compressed sequence is suitable for the first model inference service, and the compressed sequence is assigned to the first model inference service.

[0049] In addition, in actual applications, in order to make the time for the first model inference service and the second model inference service to complete the inference processing close, thereby achieving load balancing and improving overall throughput, the inference processing time of the two model inference services can be pre-estimated through the time estimation module. When the estimated inference time of the second model inference service exceeds the preset time threshold and the second model inference service has reached the load limit, the data sequence of the long sequence type can be divided to allocate the divided data sequence to the first model inference service.

[0050] In a specific implementation process, an optional implementation provided by this embodiment allocates resource processing instructions and each data sequence to a model inference service corresponding to the sequence type, including: Detecting whether the load indicator of the second model inference service reaches the indicator threshold; If so, the data sequence of the long sequence type to be assigned is divided into at least one subsequence, and the at least one subsequence is assigned to the first model inference service.

[0051] Specifically, when it is detected that the load index of the second model inference service has reached the index threshold, the data sequence of the long sequence type to be allocated can also be divided into at least one subsequence, and the sequence length of the subsequence should be suitable for the first model inference service, so that the at least one subsequence obtained by the division is allocated to the first model inference service.

[0052] It should be noted that the above-mentioned process of compressing the long sequence type data sequence to obtain a compressed sequence, allocating the compressed sequence and the short sequence type data sequence to the first model inference service, and / or dividing the long sequence type data sequence to be allocated into at least one subsequence, and allocating at least one subsequence to the first model inference service can be combined with other processing steps provided in this embodiment to form a new implementation method according to actual needs; for example, resource processing instructions and each data sequence are allocated to the model inference service corresponding to the sequence type, including: allocating the short sequence type data sequence to the first model inference service, and allocating the long sequence type data sequence to the second model inference service; detecting whether the load index of the second model inference service reaches the index threshold; if so, dividing the long sequence type data sequence to be allocated into at least one subsequence, and allocating at least one subsequence to the first model inference service.

[0053] Alternatively, the process provided in this embodiment of assigning resource processing instructions and each data sequence to the model inference service corresponding to the sequence type can also be replaced by the above-mentioned process of compressing the long sequence type data sequence to obtain a compressed sequence, and assigning the compressed sequence and the short sequence type data sequence to the first model inference service, and / or, dividing the long sequence type data sequence to be assigned into at least one subsequence, and assigning at least one subsequence to the first model inference service according to actual needs. This embodiment does not limit this.

[0054] It should also be noted that the above-mentioned determination of the sequence type of each data sequence based on the sequence length of each data sequence can be replaced by: determining the allocation parameters of each data sequence based on the sequence length of each data sequence and the load index of the model inference service, or, in the case of determining the allocation parameters of each data sequence based on the sequence type of each data sequence and the load index of the model inference service, here, allocating the resource processing instructions and each data sequence to the model inference service corresponding to the sequence type can be replaced by: allocating the resource processing instructions and each data sequence to the model inference service corresponding to the allocation parameter; accordingly, if the allocation parameter is the first allocation parameter, the data sequence is allocated to the first model inference service; if the allocation parameter is the second allocation parameter, the data sequence is allocated to the second model inference service; if the allocation parameter is the third allocation parameter, the data sequence is compressed to obtain a compressed sequence, and the compressed sequence is allocated to the first model inference service, or the data sequence is divided into at least one subsequence, and at least one subsequence is allocated to the first model inference service.

[0055] Step S208: Generate a resource processing result based on the inference data returned by the model inference service and return it to the resource service.

[0056] In specific implementation, after obtaining the inference data returned by the model inference service, here, the resource processing result is generated according to the inference data returned by the model inference service, and the generated resource processing result is returned to the resource service, so that the resource service receives the resource processing result and displays the resource processing result to the user.

[0057] During the specific execution process, in order to provide users with relatively accurate and comprehensive resource processing results for query questions submitted by users based on resource services, after obtaining the inference data returned by the model inference service, the inference data may be further processed to generate resource processing results. In an optional implementation provided in this embodiment, generating resource processing results based on the inference data returned by the model inference service and returning them to the resource service includes: Obtaining first inference data returned by the first model inference service, and / or obtaining second inference data returned by the second model inference service; An integration process is performed based on the first reasoning data and / or the second reasoning data to obtain a resource processing result.

[0058] Specifically, after the model inference service returns the inference data, the first inference data for inference processing by the first model inference service and / or the second inference data for inference processing by the second model inference service are first obtained, and then integration processing can be performed based on the first inference data and / or the second inference data. The integration processing can specifically be data merging, such as merging duplicate data to avoid duplicate output, or it can also be data deduplication processing, or it can also be other integration processing, such as sorting the inference data according to relevance or other criteria. Thereafter, after the first inference data and / or the second inference data are integrated, the resource processing result is obtained.

[0059] In summary, the service processing method based on the large language model provided in this embodiment, during the service processing process for the resource service, after the user accesses the resource service and submits a query question based on the resource service, the resource service can generate a resource processing instruction based on the query question input by the user and retrieve the data sequence related to the query question in the knowledge base, and send a call request carrying the resource processing instruction and the data sequence. Accordingly, the resource processing instruction and multiple data sequences carried by the call request of the resource service are first obtained, and then the data sequences of different sequence types are assigned to the corresponding model inference service for inference processing, so as to improve the efficiency of subsequent calls to the model inference service for inference processing , determine the sequence type of each data sequence according to the sequence length of each data sequence, and assign the resource processing instructions and each data sequence to the model inference service corresponding to the sequence type, so that each model inference service inputs the resource processing instructions and each data sequence into the corresponding large language model for inference processing and obtains inference data. Finally, in order to improve the accuracy and comprehensiveness of the resource processing results, generate a resource processing result based on at least one inference data returned by each model inference service, and return the generated resource processing result to the resource service, thereby improving the overall throughput of the model inference service during the resource processing process, and improving the response efficiency to the resource processing instructions when the user accesses the resource service.

[0060] The following takes the application of a service processing method based on a large language model provided in this embodiment in the first resource service processing scenario as an example, combined with Figure 3 , for further explanation of the service processing method based on the large language model provided in this embodiment, see Figure 3 , a service processing method based on a large language model applied to the first resource service processing scenario specifically includes the following steps.

[0061] Step S302: obtaining a resource processing instruction and a plurality of data sequences carried in a resource service call request, wherein the plurality of data sequences are obtained by performing data retrieval on the resource service.

[0062] Step S304: Calculate the sequence length of each data sequence.

[0063] Step S306: Determine whether each data sequence is a short sequence type or a long sequence type according to the sequence length.

[0064] Step S308 : Allocate resource processing instructions and data sequences of short sequence type to short sequence model reasoning services, and allocate resource processing instructions and data sequences of long sequence type to long sequence model reasoning services.

[0065] After step S308 is executed, the short sequence model inference service uses a continuous batch processing method to input the resource processing instructions and the short sequence type data sequence into the first large language model, so that the first large language model performs inference processing to obtain the first inference data; the long sequence model inference service inputs the resource processing instructions and the long sequence type data sequence into the second large language model, so that the second large language model uses a paged attention mechanism to perform inference processing to obtain the second inference data.

[0066] Step S310: obtaining first inference data returned by the short sequence model inference service, and obtaining second inference data returned by the long sequence model inference service.

[0067] Step S312: performing integration processing based on the first inference data and the second inference data to obtain a resource processing result.

[0068] Step S314: Send the resource processing result to the resource service; accordingly, the resource service running on the user terminal obtains the resource processing result and displays it to the user.

[0069] It should be noted that any one of steps S302 to S314 or any combination of multiple steps can be combined with any one of steps S202 to S208 to form a new implementation method according to the needs of implementation deployment; in addition, according to the needs of actual deployment, any one or multiple technical features can be selected from steps S302 to S314 and combined with any one or multiple technical features provided by steps S202 to S208 to form a new implementation method; or, any one or multiple technical features in steps S302 to S314 can be replaced with any one or multiple technical features provided by steps S202 to S208 to form a new implementation method according to the needs of actual deployment, which will not be repeated here.

[0070] The following is an example of an application of a service processing method based on a large language model provided in this embodiment in a second resource service processing scenario. Figure 4 , for further explanation of the service processing method based on the large language model provided in this embodiment, see Figure 4 , a service processing method based on a large language model applied to the second resource service processing scenario specifically includes the following steps.

[0071] Step S402: obtaining a resource processing instruction and a plurality of data sequences carried in a resource service call request, wherein the plurality of data sequences are obtained by performing data retrieval on the resource service.

[0072] Step S404: Calculate the sequence length of each data sequence.

[0073] Step S406: determining allocation parameters according to the sequence length of each data sequence and the load index of the model inference service.

[0074] Step S408: Allocate the data sequence corresponding to the resource processing instruction and the first allocation parameter to the short sequence model inference service.

[0075] Step S410: Allocate the data sequence corresponding to the resource processing instruction and the second allocation parameter to the long sequence model inference service.

[0076] Step S412: compress the data sequence corresponding to the third allocation parameter to obtain a compressed sequence.

[0077] Step S414: assign the compressed sequence to the short sequence model inference service.

[0078] After step S414 is executed, the short sequence model inference service inputs the resource processing instructions and data sequence into the first large language model in a continuous batch processing manner, so that the first large language model performs inference processing to obtain first inference data; the long sequence model inference service inputs the resource processing instructions and data sequence into the second large language model, so that the second large language model performs inference processing using a paged attention mechanism to obtain second inference data.

[0079] Step S416: Obtain the first inference data returned by the short sequence model inference service, and obtain the second inference data returned by the long sequence model inference service.

[0080] Step S418: performing integration processing based on the first inference data and the second inference data to obtain a resource processing result.

[0081] Step S420: Send the resource processing result to the resource service.

[0082] It should be noted that there is no obvious order among step S408, step S410, step S412 and step S414. Step S408, step S410, step S412 and step S414 may be executed in sequence, or may be executed simultaneously. Alternatively, the execution order of step S408, step S410, step S412 and step S414 may be determined according to actual needs, which is not limited in this embodiment.

[0083] It should also be noted that any one of the steps or any combination of multiple steps in steps S402 to S420 can be combined with any one of the steps S202 to S208 to form a new implementation method according to the needs of implementation deployment; in addition, according to the needs of actual deployment, any one or multiple technical features can be selected in steps S402 to S420 and combined with any one or multiple technical features provided in steps S202 to S208 to form a new implementation method; or, any one or multiple technical features in steps S402 to S420 can also be replaced with any one or multiple technical features provided in steps S202 to S208 to form a new implementation method according to the needs of actual deployment, which will not be repeated here.

[0084] An embodiment of a service processing device based on a large language model provided in this specification is as follows: In the above embodiment, a service processing method based on a large language model is provided. Correspondingly, a service processing device based on a large language model is also provided, which will be described below with reference to the accompanying drawings.

[0085] Reference Figure 5 , which shows a schematic diagram of an embodiment of a service processing device based on a large language model provided by this embodiment.

[0086] Since the device embodiment corresponds to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0087] This embodiment provides a service processing device based on a large language model, the device comprising: The sequence acquisition module 502 is configured to acquire a resource processing instruction and a plurality of data sequences carried in a call request of a resource service; the plurality of data sequences are acquired by performing data retrieval on the resource service; a sequence type determination module 504, configured to determine the sequence type of each data sequence according to the sequence length of each data sequence; The sequence allocation module 506 is configured to allocate the resource processing instructions and the data sequences to the model inference service corresponding to the sequence type; the model inference service inputs the resource processing instructions and the data sequences into a large language model for inference processing to obtain inference data; The result generation module 508 is configured to generate a resource processing result according to the inference data returned by the model inference service and return the result to the resource service.

[0088] An embodiment of a service processing device based on a large language model provided in this specification is as follows: Corresponding to the above-described method for processing a service based on a large language model, based on the same technical concept, one or more embodiments of this specification further provide a device for processing a service based on a large language model. The device for processing a service based on a large language model is used to execute the above-described method for processing a service based on a large language model. Figure 6 A schematic diagram of the structure of a service processing device based on a large language model provided in one or more embodiments of this specification.

[0089] This embodiment provides a service processing device based on a large language model, including: like Figure 6 As shown, a large language model-based service processing device can vary significantly depending on configuration or performance. It may include one or more processors 601 and memory 602. Memory 602 may store one or more applications or data. Memory 602 may be either ephemeral or persistent. The application stored in memory 602 may include one or more modules (not shown), each of which may include a series of computer-executable instructions for the large language model-based service processing device. Furthermore, processor 601 may be configured to communicate with memory 602 to execute the series of computer-executable instructions in memory 602 on the large language model-based service processing device. The large language model-based service processing device may also include one or more power supplies 603, one or more wired or wireless network interfaces 604, one or more input / output interfaces 605, one or more keyboards 606, and the like.

[0090] In a specific embodiment, a service processing device based on a large language model includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the service processing device based on the large language model, and the one or more programs are configured to be executed by one or more processors, including computer-executable instructions for performing the following: Obtaining a resource processing instruction and a plurality of data sequences carried in a call request of a resource service; the plurality of data sequences are obtained by performing data retrieval on the resource service; Determining a sequence type of each data sequence according to the sequence length of each data sequence; Allocating the resource processing instructions and the data sequences to a model inference service corresponding to the sequence type; the model inference service inputting the resource processing instructions and the data sequences into a large language model for inference processing to obtain inference data; Generate a resource processing result according to the inference data returned by the model inference service and return it to the resource service.

[0091] An embodiment of a computer-readable storage medium provided in this specification is as follows: Corresponding to the above-described service processing method based on a large language model, based on the same technical concept, one or more embodiments of this specification also provide a computer-readable storage medium.

[0092] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions. When the computer-executable instructions are executed, the following process is implemented: Obtaining a resource processing instruction and a plurality of data sequences carried in a call request of a resource service; the plurality of data sequences are obtained by performing data retrieval on the resource service; Determining a sequence type of each data sequence according to the sequence length of each data sequence; Allocating the resource processing instructions and the data sequences to a model inference service corresponding to the sequence type; the model inference service inputting the resource processing instructions and the data sequences into a large language model for inference processing to obtain inference data; Generate a resource processing result according to the inference data returned by the model inference service and return it to the resource service.

[0093] It should be noted that the embodiment of a computer-readable storage medium in this specification and the embodiment of a service processing method based on a large language model in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

[0094] An embodiment of a computer program product provided in this specification is as follows: Corresponding to the above-described service processing method based on a large language model, based on the same technical concept, one or more embodiments of this specification also provide a computer program product.

[0095] A computer program product comprising a computer program / instructions, which, when executed by a processor, implements the following steps: Obtaining a resource processing instruction and a plurality of data sequences carried in a call request of a resource service; the plurality of data sequences are obtained by performing data retrieval on the resource service; Determining a sequence type of each data sequence according to the sequence length of each data sequence; Allocating the resource processing instructions and the data sequences to a model inference service corresponding to the sequence type; the model inference service inputting the resource processing instructions and the data sequences into a large language model for inference processing to obtain inference data; Generate a resource processing result according to the inference data returned by the model inference service and return it to the resource service.

[0096] It should be noted that the embodiment of a computer program product in this specification and the embodiment of a service processing method based on a large language model in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

[0097] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. For example, the device embodiments, equipment embodiments, computer-readable storage medium embodiments, and computer program product embodiments are similar to the method embodiments, so the description is relatively simple. For relevant content in the device embodiments, equipment embodiments, computer-readable storage medium embodiments, and computer program product embodiments, please refer to the partial description of the method embodiments.

[0098] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] In the 1930s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using physical hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly performed using software called a "logic compiler." This is similar to the software compilers used during program development. Before compilation, the original code must be written in a specific programming language, called a Hardware Description Language (HDL). There are many types of HDL, including ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that simply by programming a method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0100] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the memory control logic. Those skilled in the art will also appreciate that, in addition to implementing the controller purely in computer-readable program code, the controller can also be implemented in the form of logic gates, switches, an application-specific integrated circuit, a programmable logic controller, an embedded microcontroller, etc. by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the means for implementing the various functions included therein can also be considered as structures within the hardware component. Alternatively, the means for implementing the various functions can be considered both a software module implementing the method and a structure within the hardware component.

[0101] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0102] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0103] Those skilled in the art will appreciate that one or more embodiments of this specification may be provided as a method, system, or computer program product. Thus, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable interactive processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable interactive processing device generate instructions for implementing the processes in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable interactive processing device in a live broadcast room to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions may also be loaded onto a computer or other programmable interactive processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0108] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0109] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising at least one ..." does not exclude the presence of additional identical elements in the process, method, commodity, or apparatus comprising the element.

[0111] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0112] The foregoing description is merely an example of the present invention and is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims herein.

Claims

1. A service processing method based on a large language model, comprising: Obtaining a resource processing instruction and a plurality of data sequences carried in a call request of a resource service; the plurality of data sequences are obtained by performing data retrieval on the resource service; Determining a sequence type of each data sequence according to the sequence length of each data sequence; Allocating the resource processing instructions and the data sequences to a model inference service corresponding to the sequence type; the model inference service inputting the resource processing instructions and the data sequences into a large language model for inference processing to obtain inference data; Generate a resource processing result according to the inference data returned by the model inference service and return it to the resource service.

2. The service processing method based on a large language model according to claim 1, wherein the assigning of the resource processing instructions and the data sequences to the model inference service corresponding to the sequence type comprises: If the sequence type is a short sequence type, allocating the data sequence to a first model inference service; The first model inference service inputs the resource processing instruction and the data sequence into a first large language model in a continuous batch processing manner; the first large language model performs inference processing to obtain first inference data.

3. The service processing method based on a large language model according to claim 1, wherein the assigning of the resource processing instructions and the data sequences to the model inference service corresponding to the sequence type comprises: If the sequence type is a long sequence type, allocating the data sequence to a second model inference service; Among them, the second model inference service inputs the resource processing instruction and the data sequence into the second largest language model; the second largest language model uses the paging attention mechanism to perform inference processing to obtain second inference data.

4. The service processing method based on a large language model according to claim 1, wherein generating a resource processing result based on the inference data returned by the model inference service and returning the result to the resource service comprises: Obtaining first inference data returned by the first model inference service, and / or obtaining second inference data returned by the second model inference service; The resource processing result is obtained by performing integration processing based on the first reasoning data and / or the second reasoning data.

5. The service processing method based on a large language model according to claim 1, wherein the data sequence is obtained by the resource service calling a retrieval model to perform data retrieval in a knowledge base; in, The data retrieval includes: searching the knowledge base based on the query text of the resource service to obtain at least one relevant data sequence; calculating the relevance score of each relevant data sequence and determining the data sequence according to the relevance score and a preset score threshold.

6. The service processing method based on a large language model according to claim 2, wherein the assigning of the resource processing instructions and the data sequences to the model inference service corresponding to the sequence type comprises: Detecting whether the load indicator of the second model inference service reaches the indicator threshold; If so, the data sequence of the long sequence type is compressed to obtain a compressed sequence, and the compressed sequence and the data sequence of the short sequence type are assigned to the first model inference service.

7. The service processing method based on a large language model according to claim 2, wherein the assigning of the resource processing instructions and the data sequences to the model inference service corresponding to the sequence type comprises: Detecting whether the load indicator of the second model inference service reaches the indicator threshold; If so, the data sequence of the long sequence type to be allocated is divided into at least one subsequence, and the at least one subsequence is allocated to the first model inference service.

8. The service processing method based on a large language model according to claim 1, wherein determining the sequence type of each data sequence according to the sequence length of each data sequence comprises: Calculating the sequence length of each data sequence and comparing the sequence length with a preset length threshold; The data sequence whose sequence length is less than the preset length threshold is determined as a short sequence type, and / or the data sequence whose sequence length is greater than the preset length threshold is determined as a long sequence type.

9. The service processing method based on a large language model according to claim 8, wherein the calculating the sequence length of each data sequence comprises: Counting the number of characters and / or bytes in each data sequence, and using the number of characters and / or bytes as the sequence length; Alternatively, the sequence length is calculated based on a preset sequence length calculation rule.

10. The service processing method based on a large language model according to claim 1, wherein the inference processing comprises: Search processing for information search based on the query text of the resource service, item recommendation processing for resource service item recommendation based on the query text, and / or content recommendation processing for preferred content recommendation based on the query text.

11. A service processing device based on a large language model, comprising: A sequence acquisition module is configured to acquire resource processing instructions and multiple data sequences carried in a resource service call request; The multiple data sequences are obtained by performing data retrieval on the resource service; a sequence type determination module, configured to determine the sequence type of each data sequence according to the sequence length of each data sequence; a sequence allocation module configured to allocate the resource processing instructions and the data sequences to a model inference service corresponding to the sequence type; the model inference service inputs the resource processing instructions and the data sequences into a large language model for inference processing to obtain inference data; The result generation module is configured to generate a resource processing result according to the inference data returned by the model inference service and return the result to the resource service.

12. A service processing device based on a large language model, comprising: processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to: Obtaining a resource processing instruction and a plurality of data sequences carried in a call request of a resource service; the plurality of data sequences are obtained by performing data retrieval on the resource service; Determining a sequence type of each data sequence according to the sequence length of each data sequence; Allocating the resource processing instructions and the data sequences to a model inference service corresponding to the sequence type; the model inference service inputting the resource processing instructions and the data sequences into a large language model for inference processing to obtain inference data; Generate a resource processing result according to the inference data returned by the model inference service and return it to the resource service.

13. A computer-readable storage medium for storing computer-executable instructions, wherein the computer-executable instructions implement the steps of the method of claim 1 when executed.

Images