Intelligent question and answer service management system

Abstract

The present invention provides an intelligent question-answering service management system, comprising a user terminal, a shunt server and an answer server, wherein the user terminal is used for users to edit question content and send the question content to the shunt server; the shunt server is used to process the question content from the user terminal, generate an answer task, specify an answer server for the user terminal, and assign the answer task to the specified answer server, wherein the answer task includes the question content; the answer server is used to run a large language model to process the question content, obtain the answer content, and send the answer content to the user terminal. The present invention reasonably utilizes all the computing resources of the system by shunting the answer task to different answer servers, avoids generating too high a load on a single server, improves the computing resources that can be called by the large language model, and further improves its computing efficiency, so that users can quickly obtain the answer content and ensure the user experience.

Description

Technical Field

[0001] The present invention relates to the field of intelligent question-answering technology, and in particular to an intelligent question-answering service management system. Background Art

[0002] Large language models (LLMs) refer to those models that are trained using a large amount of text data through deep learning technology. These models can understand and generate natural language texts and have the ability to handle a variety of natural language tasks, such as text classification, sentiment analysis, machine translation, and dialogue generation. Among them, intelligent question-answering systems are an important application direction of large language models. Large language models can deeply understand the meaning of questions, including identifying question types and extracting key information. Based on the understanding of questions, large language models can generate coherent and accurate answers. LLMs usually require a lot of computing resources to run, especially in the inference stage. Running LLMs on a single computer device is difficult to handle a large number of concurrent requests from users, causing some users to wait for a long time to get answers, affecting the user experience. How to reasonably allocate tasks to different computing nodes to meet users' question-answering needs and achieve efficient use of computing resources under the condition of limited overall computing resources is one of the hot issues in the current LLM field. Summary of the invention

[0003] The purpose of the present invention is to provide an intelligent question-answering service management system, which generates answering tasks based on user questions, and reasonably distributes the answering tasks to different answering servers for processing through a diversion server, thereby achieving efficient processing of the answering tasks while reasonably allocating computing resources, thereby improving the user experience.

[0004] To achieve the above-mentioned purpose, the present invention provides an intelligent question-answering service management system, the system comprising:

[0005] The user end is used for users to edit question content and send the question content to the distribution server;

[0006] The distribution server is used to process the question content from the user terminal, generate an answer task, designate an answer server for the user terminal, and assign the answer task to the designated answer server, wherein the answer task includes the question content;

[0007] The answer server is used to run a large language model to process the question content, obtain the answer content, and send the answer content to the user end.

[0008] Furthermore, the diversion server specifies an answer server for the user terminal, specifically: according to the number of answer servers, an answer server queue is established, all unprocessed answer tasks are sorted in order of reception time from first to last to establish a task queue, and the answer servers in the answer server queue are traversed in sequence, and each time the traversal is performed, the traversed answer server is designated as the answer server for processing the first answer task in the task queue, the answer task of the designated answer server is taken out of the task queue, and the answer server queue is traversed multiple times until the task queue is empty.

[0009] Furthermore, the distribution server specifies the answer server for the user terminal, which specifically includes the following operations:

[0010] S101, upon receiving the question content, generating a corresponding answer task, initializing a task queue, and storing the answer task in the task queue in the order of its generation time;

[0011] S102, traversing the task queue, and analyzing the computing resources required to complete each answer task in the task queue;

[0012] S103, determining the computing resources that have been occupied by each answer server;

[0013] S104, traversing the task queue according to the computing resources that have been occupied by each answer server, and calculating the impact on user experience when assigning answer tasks to different answer servers;

[0014] S105 , determining the answer server that has the lowest impact on the user experience when processing the answer task, and designating it as the answer server that processes the answer task of the user end, until the traversal of the task queue is completed.

[0015] Furthermore, the impact of assigning answer tasks to different answer servers on user experience is calculated. Specifically, the user's answer task assignment decision set x Check the delay in seeing the answer content t x , the accuracy of the answer content a x As input, the preset calculation model is used to calculate the impact on user experience under the solution task allocation decision set x. x The expression of the preset calculation model is as follows:

[0016]

[0017] In the above formula, , is the weight between delay and accuracy.

[0018] Furthermore, the distribution server specifies the answer server for the user terminal, which specifically includes the following operations:

[0019] S201, generating a corresponding answering task upon receiving the question content, and analyzing the computing resources required to complete the answering task;

[0020] S202, initializing the task queue, and storing the computing resources required to complete each task in the task queue in order from low to high;

[0021] S203, taking out the answer task from the head of the task queue, and calculating the impact on the user experience when the answer task is assigned to different answer servers;

[0022] S204: Determine the answer server that has the lowest impact on user experience when processing the answer task, and designate it as the answer server that processes the answer task of the user end, until all answer tasks in the task queue are taken out.

[0023] Furthermore, the distribution server specifies the answer server for the user terminal, which specifically includes the following operations:

[0024] S301, initializing the answer task allocation strategy of each user terminal, wherein the answer task allocation strategy is used to indicate to which answer server the answer task of the user terminal is allocated;

[0025] S302, the user terminal sends the answer task allocation strategy to the distribution server, wherein the answer task allocation strategy includes the unique identification information of the target answer server, and the target answer server is the answer server to which the user terminal wants to allocate the answer task;

[0026] S303, the user end obtains target answer server information of other user ends through the distribution server, calculates its own allocatable answer server set and uploads it to the distribution server;

[0027] S304, the distribution server determines whether the assignable answer server combination is empty, and if not, generates token (t) information;

[0028] S305, the distribution server determines the target user terminal according to the set of allocatable answer servers of each user terminal, and transmits the token (t) information to the target user terminal;

[0029] S306, after receiving the token(t) information, the target user terminal selects a target answer server from its own set of assignable answer servers, and updates the answer task assignment strategy according to the selected target answer server and the token(t) information;

[0030] S307, iteratively execute S302-S306 until the traffic distribution server sends a stop iteration message to each user terminal.

[0031] Furthermore, the answer server runs a large language model to process the question content and obtain the answer content, which specifically includes the following operations:

[0032] S401, extracting keywords from the question content, and obtaining multiple candidate documents from the knowledge base;

[0033] S402, using a pre-trained encoder to convert keywords and multiple candidate documents in the question content into vector representations;

[0034] S403, determining the most relevant candidate document based on the vector representation of the keyword and the vector representation of the plurality of candidate documents;

[0035] S404, generating structured data based on the most relevant candidate documents and question content;

[0036] S405: Using the structured data as input to a pre-trained generative model, and the generative model generates answer content based on the structured data.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention provides an intelligent question-answering service management system, in which a user edits question content through a user terminal and sends the question content to a shunt server, the shunt server generates an answer task according to the question content of the user terminal, specifies an answer server for the user terminal, and assigns the answer task to the specified answer server, the answer server processes the question content by running a large language model, obtains the answer content, and feeds back the answer content to the user terminal corresponding to the question content, the system reasonably utilizes all the computing resources of the system by shunting the answer task to different answer servers, avoids generating too high a load on a single server, improves the computing resources that can be called by the large language model, and further improves its computing efficiency, so that the user can quickly obtain the answer content, and ensures the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 It is a schematic diagram of the overall structure of an intelligent question and answer service management system provided by an embodiment of the present invention.

[0041] Figure 2 The present invention is a flowchart of a method for specifying a server for answering a question provided by an embodiment of the present invention.

[0042] Figure 3 It is a flowchart of another method for specifying a solution server provided by an embodiment of the present invention.

[0043] Figure 4 It is a flowchart of another method for specifying a solution server provided by an embodiment of the present invention.

[0044] Figure 5 It is a flowchart of a method for processing question content using a large language model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0046] Reference Figure 1 This embodiment provides an intelligent question-answering service management system, which includes a user terminal, a distribution server and an answer server. The user terminal, the distribution server and the answer server can communicate data with each other.

[0047] The user terminal is used for the user to edit the question content and send the question content to the distribution server.

[0048] The distribution server is used to process the question content from the user terminal, generate an answer task, specify an answer server for the user terminal, and assign the answer task to the specified answer server, wherein the answer task includes the question content.

[0049] The answer server is used to run the large language model to process the question content, obtain the answer content, and send the answer content to the user end.

[0050] In this embodiment, there can be multiple user terminals and multiple answer servers. The number of shunt servers is generally much less than that of user terminals and answer servers, but there is at least one. After the user completes editing the question content, the user terminal establishes a communication connection with the shunt server to send the question content to the shunt server. After the shunt server generates an answer task based on the question content and determines the answer server that processes the answer task, it establishes a communication connection with the designated answer server to transmit the answer task containing the question content to the answer server. After completing the answer task, the answer server establishes a communication connection with the user terminal that initiated the question content, sends the answer content to the user terminal, and the user terminal displays the answer content to the user.

[0051] It is understandable that the same distribution server may establish communication connections with multiple clients at the same time to receive question content, and the same answer server may process multiple answer tasks at the same time, depending on the local computing resource usage of the answer server.

[0052] As an optional implementation, the distribution server specifies the answer server for the user terminal, specifically:

[0053] First, according to the number of answer servers, an answer server queue is established, and all unprocessed answer tasks are sorted from first to last according to the time sequence of the answer tasks received by the diversion server to establish a task queue. Then, the answer servers in the answer server queue are traversed in the order of arrangement, and each time the traversed answer server is designated as the answer server for processing the first answer task in the task queue, and the answer task of the designated answer server is taken out from the task queue. The answer server queue is traversed multiple times until the task queue is empty.

[0054] In this optional implementation, at any time, the first answer task in the task queue is received by the distribution server at the earliest time, so as to avoid the user having to wait a long time to obtain the answer.

[0055] This optional implementation method implements the assignment of answer tasks by traversing the answer server list. Its advantage is that the implementation method is simple and efficient. This implementation method can continuously increase the answer tasks burdened by different answer servers from low to high in a balanced manner according to the number of answer tasks. However, since different answer tasks have different requirements for the occupation of computing resources, the occupation of computing resources of different answer servers may be different when the number of answer tasks is the same, which may cause individual answer servers to be overloaded, while the computing resources of some answer servers are not fully utilized.

[0056] The computing resources described in this embodiment can be one or more resources such as CPU, memory, storage space, GPU, etc.

[0057] As another optional implementation, refer to Figure 2 , the distribution server specifies the answer server for the user terminal, which can be achieved by performing the following operations:

[0058] S101. When receiving the question content, a corresponding answer task is generated, a task queue is initialized, and the answer tasks are stored in the task queue in the order of their generation time.

[0059] S102, traversing the task queue, and analyzing the computing resources required to complete each answer task in the task queue.

[0060] In this optional implementation, the computing resources required to complete the answering task are analyzed by inputting the answering task into a pre-trained computing resource analysis model for processing to obtain an estimated result of the computing resources required to complete the answering task. The computing resource analysis model is a deep learning model pre-trained using labeled training data, and the training data includes paired sample answering tasks and sample computing resource occupancy estimate data.

[0061] S103: Determine the computing resources that have been occupied by each answer server.

[0062] S104. According to the computing resources that have been occupied by each answer server, the task queue is traversed to calculate the impact on the user experience when the answer tasks are distributed to different answer servers.

[0063] In this embodiment, the impact on user experience when answering tasks are assigned to different answering servers is mainly reflected in the time spent from initiating a question to receiving an answer, as well as the accuracy of the answer content.

[0064] S105 , determining the answer server that has the lowest impact on the user experience when processing the answer task, and designating it as the answer server that processes the answer task of the user end, until the traversal of the task queue is completed.

[0065] In this optional implementation, the diversion server analyzes the solution task allocation method that has the least impact on the user experience based on the computing resource usage of each solution server and the estimated computing resource usage of the solution task, and selectively allocates the solution tasks to the appropriate solution server, thereby being able to reasonably allocate the solution tasks based on the load of the solution server itself.

[0066] Exemplarily, the impact of assigning answer tasks to different answer servers on user experience is calculated by: assigning the user in the answer task assignment decision set x Check the delay in seeing the answer content t x , the accuracy of the answer content a x As input, the default calculation model is used to calculate the decision set of task allocation in solving the problem. x The impact on user experience and the solution to the task allocation decision set x The expression of the preset calculation model is as follows:

[0067]

[0068] In the above formula, , is the weight between delay and accuracy.

[0069] As another optional implementation, refer to Figure 3 , the distribution server specifies the answer server for the user, which can be achieved by performing the following operations:

[0070] S201. When receiving a question, generate a corresponding answer task and analyze the computing resources required to complete the answer task.

[0071] In this optional implementation, analyzing the computing resources required to complete the answering task may also be inputting the answering task into a pre-trained computing resource analysis model for processing to obtain an estimated result of the computing resources required to complete the answering task.

[0072] S202, initializing the task queue, and storing the computing resources required to complete each task in the task queue in order from low to high.

[0073] S203: Take out the solution task from the head of the task queue, and calculate the impact on the user experience when the solution task is assigned to different solution servers.

[0074] S204: Determine the answer server that has the lowest impact on user experience when processing the answer task, and designate it as the answer server that processes the answer task of the user end, until all answer tasks in the task queue are taken out.

[0075] Compared with the previous optional implementation, the main difference between this optional implementation is that the previous optional implementation stores the answer tasks in the task queue according to the time of generation of the answer tasks, while this implementation stores the answer tasks in the task queue in order from low to high computing resources required to complete each answer task. The advantage of this implementation is that it gives priority to the allocation of answer tasks that occupy less computing resources on the answer server, so that it can give priority to task allocation decisions when the answer server load is low, reducing the impact of answer task allocation on user experience from the overall system level.

[0076] As another optional implementation, refer to Figure 4 , the distribution server specifies the answer server for the user terminal, which can also be achieved by performing the following operations:

[0077] S301, initializing the answer task allocation strategy of each user terminal, wherein the answer task allocation strategy is used to indicate to which answer server the answer task of the user terminal is allocated.

[0078] S302: The user terminal sends the answer task allocation strategy to the distribution server. The answer task allocation strategy includes unique identification information of the target answer server. The target answer server is the answer server to which the user terminal wants to allocate answer tasks.

[0079] S303: The user end obtains target answer server information of other user ends through the distribution server, calculates its own allocatable answer server set and uploads it to the distribution server.

[0080] S304. The distribution server determines whether the assignable answer server combination is empty. If it is not empty, it generates token(t) information, where t represents the generation time of the information. The token(t) information is only valid within a certain period of time after time t.

[0081] S305. The distribution server determines the target user terminal according to the set of allocatable answer servers of each user terminal, and transmits the token (t) information to the target user terminal.

[0082] S306. After receiving the token(t) information, the target user terminal selects a target answer server from its own set of assignable answer servers, and updates the answer task assignment strategy according to the selected target answer server and the token(t) information.

[0083] In this optional implementation, the solution task allocation strategy is only effective when the token(t) information is valid. If the generation time of the token(t) information exceeds the preset time, the token(t) information will become invalid, and the solution task allocation strategy with the corresponding token(t) information will no longer be valid.

[0084] S307, iteratively execute S302-S306 until the traffic distribution server sends a stop iteration message to each user terminal.

[0085] In this optional implementation, each user terminal can obtain the target answer server information of other user terminals through the diversion server, and determine its own allocable answer server set on this basis. The diversion server determines the target user terminal based on the allocable answer server set uploaded by all user terminals, and issues token(t) information to it, so that the target user terminal can mark the effective answer task allocation strategy through the token(t) information. Other user terminals need to consider the answer task allocation strategy of others when calculating the allocable answer server set, so that the diversion server can receive the answer task allocation strategies of all user terminals at the same time, and different user terminals can consider each other's global allocation strategies, thereby reaching a stable state, minimizing the impact of the answer task allocation strategy on users in a short period of time.

[0086] As an optional implementation, refer to Figure 5 The answer server runs a large language model to process the question content and obtain the answer content, which specifically includes the following operations:

[0087] S401, extracting keywords from the question content, and obtaining multiple candidate documents from the knowledge base.

[0088] Exemplarily, extracting keywords from the question content can be achieved through KeyBert keyword extraction technology, which is a minimal and easy-to-use keyword extraction technology that uses BERT embedding and simple cosine similarity to find the sub-phrases in the document that are most similar to the document itself. Specifically, first use BERT to extract document embedding to obtain a document-level representation, then use a word embedding model to extract N-gram words / phrases, and finally identify the most similar words as the words that best describe the entire document. Keywords can also be generated by methods such as Rake, TF-IDF, etc., and this embodiment does not specifically limit this.

[0089] S402: Use a pre-trained encoder to convert keywords in the question content and multiple candidate documents into vector representations.

[0090] S403: Determine the most relevant candidate document based on the vector representation of the keyword and the vector representation of the plurality of candidate documents.

[0091] For example, the correlation between vector representations can be calculated using metrics such as cosine similarity.

[0092] S404: Generate structured data based on the most relevant candidate documents and question content.

[0093] This implementation method improves the accuracy and richness of large language model processing answer tasks based on RAG retrieval enhancement technology. The core idea is to obtain relevant information from a large document collection through a retrieval module based on the generative model, and then integrate this information into the generative model to generate more accurate and rich text.

[0094] S405: Using the structured data as input to a pre-trained generative model, and the generative model generates answer content based on the structured data.

[0095] In this embodiment, the generation module generates the final text output based on the structured data. For example, a pre-trained T5 generation model or a BERT generation model can be used for text generation. The generation model uses the retrieved document fragments as additional context information and processes them together with the user's question content to generate richer and more accurate answers.

[0096] This implementation combines RAG retrieval enhancement technology and a large language model, utilizing the rich information in the knowledge base to generate answers that contain more details and accurate information. At the same time, by obtaining multiple candidate documents, it can reduce the generation of incorrect or inaccurate answers due to the lack of background knowledge related to the user's question content.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent question-answering service management system, characterized in that: The system comprises: The user end is used for users to edit question content and send the question content to the distribution server; The distribution server is used to process the question content from the user terminal, generate an answer task, designate an answer server for the user terminal, and assign the answer task to the designated answer server, wherein the answer task includes the question content; The answer server is used to run a large language model to process the question content, obtain the answer content, and send the answer content to the user end; The offload server specifies the answer server for the user, which includes the following operations: S101, upon receiving the question content, generating a corresponding answer task, initializing a task queue, and storing the answer task in the task queue in the order of its generation time; S102, traversing the task queue, and analyzing the computing resources required to complete each answer task in the task queue; S103, determining the computing resources that have been occupied by each answer server; S104, traversing the task queue according to the computing resources that have been occupied by each answer server, and calculating the impact on user experience when assigning answer tasks to different answer servers; S105, determining the answer server that has the lowest impact on the user experience when processing the answer task, and designating it as the answer server that processes the answer task of the user end until the traversal of the task queue is completed; Calculate the impact of assigning answer tasks to different answer servers on user experience, specifically: assign the user to the answer task assignment decision set x Check the delay in seeing the answer content t x , the accuracy of the answer content a x As input, the default calculation model is used to calculate the decision set of task allocation in the solution. x The impact on user experience and the solution to the task allocation decision set x The expression of the preset calculation model is as follows: In the above formula, , is the weight of delay and accuracy.

2. An intelligent question-answering service management system according to claim 1, characterized in that: The diversion server specifies the answer server for the user terminal, specifically: according to the number of answer servers, an answer server queue is established, all unprocessed answer tasks are sorted in order of reception time from first to last to establish a task queue, the answer servers in the answer server queue are traversed in sequence, and each time the traversal is performed, the traversed answer server is designated as the answer server for processing the first answer task in the task queue, the answer task of the specified answer server is taken out of the task queue, and the answer server queue is traversed multiple times until the task queue is empty.

3. An intelligent question-answering service management system according to claim 1, characterized in that: The offload server specifies the answer server for the user, which includes the following operations: S201, generating a corresponding answering task upon receiving the question content, and analyzing the computing resources required to complete the answering task; S202, initializing the task queue, and storing the computing resources required to complete each task in the task queue in order from low to high; S203, taking out the answer task from the head of the task queue, and calculating the impact on the user experience when the answer task is assigned to different answer servers; S204: Determine the answer server that has the lowest impact on the user experience when processing the answer task, and designate it as the answer server that processes the answer task of the user end, until all the answer tasks in the task queue are taken out.

4. The intelligent question-answering service management system according to claim 1, characterized in that: The offload server specifies the answer server for the user, which includes the following operations: S301, initializing the answer task allocation strategy of each user terminal, wherein the answer task allocation strategy is used to indicate to which answer server the answer task of the user terminal is allocated; S302, the user terminal sends the answer task allocation strategy to the distribution server, wherein the answer task allocation strategy includes the unique identification information of the target answer server, and the target answer server is the answer server to which the user terminal wants to allocate the answer task; S303, the user end obtains target answer server information of other user ends through the distribution server, calculates its own allocatable answer server set and uploads it to the distribution server; S304, the distribution server determines whether the assignable answer server combination is empty, and if not, generates token (t) information; S305, the distribution server determines the target user terminal according to the set of allocatable answer servers of each user terminal, and transmits the token (t) information to the target user terminal; S306, after receiving the token(t) information, the target user terminal selects a target answer server from its own set of assignable answer servers, and updates the answer task assignment strategy according to the selected target answer server and the token(t) information; S307, iteratively execute S302-S306 until the traffic distribution server sends a stop iteration message to each user terminal.

5. The intelligent question-answering service management system according to claim 1, characterized in that: The answer server runs a large language model to process the question content and obtain the answer content, which includes the following operations: S401, extracting keywords from the question content, and obtaining multiple candidate documents from the knowledge base; S402, using a pre-trained encoder to convert keywords and multiple candidate documents in the question content into vector representations; S403, determining the most relevant candidate document based on the vector representation of the keyword and the vector representation of the plurality of candidate documents; S404, generating structured data based on the most relevant candidate documents and question content; S405: Using the structured data as input to a pre-trained generative model, and the generative model generates answer content based on the structured data.

Images