Method and device for processing user request

Abstract

A method and apparatus for processing a user request are provided. In one method, a set of data chunks that match a user request are selected from a plurality of data chunks, the plurality of data chunks indicating contextual knowledge associated with the user request; on the basis of the user request and the group of data blocks, determining a cue word for being input into a machine learning model; and providing a response to the user request based on the response of the machine learning model to the cue word. By utilizing the exemplary implementation mode of the invention, a group of data blocks matched with the user request can be selected, and related background knowledge related to the user request can be acquired. The related background knowledge is included in the cue word, on one hand, a large amount of knowledge of the machine learning model can be utilized, and on the other hand, the background knowledge can be analyzed by utilizing the strong processing capability of the machine learning model, so that a more accurate response is provided.

Description

Technical Field

[0001] Exemplary implementations of the present disclosure generally relate to machine learning technology, and in particular, to methods and devices for processing user requests based on machine learning models. Background Art

[0002] Machine learning technology has been widely used in a variety of technical fields. For example, with the rapid development of artificial intelligence technology, question-answering systems have become a research direction that has attracted much attention in the field of artificial intelligence. The goal of question-answering systems is to enable computers to communicate in natural language like humans, thereby achieving intelligent dialogue and answers. However, the accuracy of existing question-answering systems is not satisfactory, and it is expected that a more effective question-answering system can be provided to answer various questions that may be included in user requests. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for processing a user request is provided. In the method, a group of data blocks matching the user request is selected from a plurality of data blocks, the plurality of data blocks indicating background knowledge associated with the user request; based on the user request and the group of data blocks, a prompt word for input to a machine learning model is determined; and based on a response of the machine learning model to the prompt word, an answer to the user request is provided.

[0004] In a second aspect of the present disclosure, a device for processing a user request is provided. The device includes: at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, and when the instructions are executed by the at least one processing unit, the electronic device: selects a group of data blocks matching the user request from a plurality of data blocks, the plurality of data blocks indicating background knowledge associated with the user request; determines a prompt word for input to a machine learning model based on the user request and the group of data blocks; and provides an answer to the user request based on the machine learning model's response to the prompt word.

[0005] According to some implementations of the second aspect, the prompt word includes: a user request and context information associated with the user request, and the instruction, when executed by at least one processing unit, further causes the electronic device to: determine the context information based on a set of data blocks.

[0006] According to some implementations of the second aspect, when the instruction is executed by at least one processing unit, the electronic device further causes: determining an order of a group of data blocks; and arranging the group of data blocks in order to determine the background information.

[0007] According to some implementations of the second aspect, when the instruction is executed by at least one processing unit, the electronic device further causes: determining a set of matching degrees between a set of data blocks and user requests respectively; and determining an order based on the set of matching degrees.

[0008] According to some implementations of the second aspect, when the instruction is executed by at least one processing unit, the electronic device further performs at least any one of the following: determining the order of a group of data blocks according to a set of matching degrees arranged in ascending order; determining the order of a group of data blocks according to a set of matching degrees arranged in descending order; and arranging the target data block at a predetermined position in a group of data blocks in response to determining that the order of the matching degrees of a target data block in a group of data blocks meets a predetermined condition.

[0009] According to some implementations of the second aspect, the instruction, when executed by at least one processing unit, further causes the electronic device to: adjust the order of a set of data blocks in the background information in the prompt word to generate another prompt word; receive another response of the machine learning model to the other prompt word; respectively determine the correlation between the user request and the response, and another correlation between the user request and the other response; and compare the correlation and another correlation to provide an answer to the user request.

[0010] According to some implementations of the second aspect, when the instruction is executed by at least one processing unit, the electronic device further causes: obtaining text data describing background knowledge associated with the user request; and dividing the text data into multiple data blocks.

[0011] According to some implementations of the second aspect, the instruction, when executed by at least one processing unit, further causes the electronic device to: determine a granularity index associated with a user request; determine a predetermined size for dividing multiple data blocks based on the granularity index; and divide the text data into multiple data blocks based on the predetermined size.

[0012] According to some implementations of the second aspect, the predetermined size is proportional to the granularity index.

[0013] According to some implementations of the second aspect, when the instruction is executed by at least one processing unit, the electronic device is further caused to: determine a group of data blocks based on at least any one of a vector search and a keyword search.

[0014] According to some implementations of the second aspect, when the instruction is executed by at least one processing unit, the electronic device further causes: in response to determining that the user request falls within the scope of background knowledge, to select a group of data blocks matching the user request from multiple data blocks.

[0015] According to some implementations of the second aspect, when the instruction is executed by at least one processing unit, the electronic device further causes: determining keywords from a user request; and selecting a group of data blocks based on a comparison of the keywords with a plurality of data blocks.

[0016] According to some implementations of the second aspect, when the instruction is executed by at least one processing unit, the electronic device further causes: based on a semantic analysis of the user request, to update the user request, wherein the updated user request has the same semantic meaning as the user request.

[0017] According to some implementations of the second aspect, the instruction, when executed by at least one processing unit, further causes the electronic device to: in response to determining that the user request involves a subsequent processing task, call an application for performing the subsequent processing task; and use the application to provide a response.

[0018] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, and when the instructions are executed by the at least one processing unit, the electronic device executes the method according to the first aspect of the present disclosure.

[0019] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor implements the method according to the first aspect of the present disclosure.

[0020] In a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method according to the first aspect of the present disclosure when executed by a processor.

[0021] It should be understood that the content described in this content section is not intended to limit the key features or important features of the implementation of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In the following, in conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages and aspects of various implementations of the present disclosure will become more apparent. In the accompanying drawings, the same or similar figure labels represent the same or similar elements, wherein:

[0023] Figure 1 A block diagram showing an application environment according to an exemplary implementation of the present disclosure is shown;

[0024] Figure 2 A block diagram for processing a user request according to some implementations of the present disclosure is shown;

[0025] Figure 3 A block diagram showing the structure of a prompt word according to some implementations of the present disclosure;

[0026] Figure 4 A block diagram illustrating ordering a set of data blocks based on a degree of match between the set of data blocks and a user request according to some implementations of the present disclosure is shown;

[0027] Figure 5 A block diagram illustrating ranking criteria according to some implementations of the present disclosure;

[0028] Figure 6 A block diagram showing the structure of another prompt word according to some implementations of the present disclosure;

[0029] Figure 7 A block diagram showing a specific process for processing a user request according to some implementations of the present disclosure;

[0030] Figure 8 A flowchart showing a method for processing a user request according to some implementations of the present disclosure; and

[0031] Fig. 9 A block diagram of a device capable of implementing multiple implementations of the present disclosure is shown. DETAILED DESCRIPTION

[0032] The implementation of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain implementations of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be interpreted as being limited to the implementations described herein. On the contrary, these implementations are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and implementations of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0033] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various implementations are described throughout this article, and any type of implementation may be included under any section / subsection. In addition, the implementation described in any section / subsection may be combined in any way with any other implementation described in the same section / subsection and / or in a different section / subsection.

[0034] In the description of the implementations of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "an implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions may also be included below.

[0035] The term "circuit" as used herein may refer to a combination of hardware circuits and / or hardware circuits and software. For example, a circuit may be a combination of analog and / or digital hardware circuits and software / firmware. As another example, a circuit may be any part of a hardware processor with software, the hardware processor including (multiple) digital signal processors, software and (multiple) memories, which work together to enable the device to work to perform various functions. In yet another example, a circuit may be a hardware circuit and / or processor, such as a microprocessor or a part of a microprocessor, which requires software / firmware for operation, but the software may not exist when it is not needed for operation. As used herein, the term "circuit" also encompasses the implementation of only a hardware circuit or processor or a part of a hardware circuit or processor and its (or their) accompanying software and / or firmware.

[0036] Example Environment

[0037] With the rapid development of artificial intelligence technology, question-answering systems have become a research direction that has attracted much attention in the field of artificial intelligence. The goal of question-answering systems is to enable computers to communicate in natural language like humans, thereby achieving intelligent dialogue and answers. This technology has broad potential in practical applications and can be used in areas such as intelligent customer service, virtual assistants, and standardized answers to provide better user experience and service quality.

[0038] See also Figure 1 Describes an overview of some implementations according to the present disclosure, Figure 1 A block diagram 100 of an application environment according to an exemplary implementation of the present disclosure is shown. Figure 1 As shown, the question-and-answer system is shown in the user page 110, and the user can enter a user request at the input box 140, and send the user request via the control 142. According to some implementations of the present disclosure, the question-and-answer system can answer questions in various fields. For the sake of ease of description, the following only takes the employee management system as an example to describe an example of processing user requests. Here, the question-and-answer system can answer questions about the employee management system. At this time, the user request 120 may include, for example, "How to initiate a leave application." At this time, the answer 130 of the question-and-answer system may be expressed, for example, as: Please log in to the employee management system and open... Here, the question-and-answer system may be implemented based on a machine learning model 150.

[0039] However, building a high-quality question-answering system is extremely challenging. It needs to process diverse user inputs, understand complex semantic structures, maintain contextual coherence, and output accurate, fluent, and logically accurate responses. Question-answering systems based on rules or knowledge base retrieval have been developed. Such systems are limited by fixed rules and knowledge bases, and are difficult to adapt to complex and changing real-world scenarios. In addition, the answering process is mechanized, which affects the user experience. For example, in a question-answering system based on a machine learning model, there may not be complete knowledge related to the user's request, which results in insufficient accuracy in the response.

[0040] In actual scenarios, questions input by users may be too concise or too lengthy. Existing systems directly vectorize user input, which may result in the inability to retrieve matching text blocks. Existing systems do not classify user input in a detailed enough manner, which will increase the number of calls to the machine learning model. Here, the machine learning model may include, for example, a language model or a large language model (also called a large model). In some cases, the accuracy of the retrieval is not high, which affects the quality of the output information of the large model. At this point, it is expected that a more effective question-and-answer system can be provided to accurately answer various questions that may be included in user requests.

[0041] Summary of processing user requests

[0042] In order to at least partially solve the deficiencies in the prior art, according to an exemplary implementation of the present disclosure, a method for processing a user request is proposed. Figure 2 Describes an overview of an exemplary implementation according to the present disclosure, Figure 2 A block diagram 200 for processing a user request according to some implementations of the present disclosure is shown.

[0043] like Figure 2 As shown, there may be a plurality of data blocks 220 (e.g., data blocks 220-1, 220-M, 220-N, etc.), where the plurality of data blocks indicate background knowledge associated with the user request. A set of data blocks 230 matching the user request 212 may be selected from the plurality of data blocks 220, for example, data blocks 220-1, ..., 220-M, etc. may be selected. Further, based on the user request 212 and the set of data blocks 230, a prompt word 240 for input to the machine learning model 250 may be determined. Further, an answer to the user request may be provided based on a response 260 of the machine learning model to the prompt word 240.

[0044] By using some implementations of the present disclosure, a set of data blocks matching the user request can be selected, and relevant background knowledge about the user request can be obtained. By including the relevant background knowledge in the prompt word, on the one hand, a large amount of knowledge of the machine learning model itself can be utilized, and on the other hand, the powerful processing power of the machine learning model can be utilized to analyze the background knowledge, thereby providing a more accurate response.

[0045] Detailed process of handling user requests

[0046] An overview of some implementations of the present disclosure has been described, and more processing details will be described below. According to some implementations of the present disclosure, text data 210 describing background knowledge associated with a user request can be obtained. Here, the text data may include one or more text data. In the application scenario of the employee management system described above, the text data may include a user manual, an administrator manual, an operation guide, and the like of the employee management system. The text data may include multiple formats, such as a plain text format, a portable document format, a hypertext link format, and the like. The text data may be divided into multiple data blocks. The data blocks may be divided according to a predetermined standard, for example, the maximum length of the data block may be specified to be 512, 256, or other numerical values, and the data blocks may be divided according to chapters, pages, or paragraphs in the text data, and the like. Alternatively and / or additionally, the data blocks may be divided according to specific symbols (e.g., commas, periods, semicolons, and the like).

[0047] According to some implementations of the present disclosure, in the process of dividing multiple data blocks, the following indicators can be considered: granularity, that is, text length (for example, every 200 (250, or other number) words can be divided into a data block); rules, such as dividing according to sentences or paragraphs. According to some implementations of the present disclosure, there can be overlap between data blocks. For example, if the overlap is 50, there can be 50 words overlapping between the previous data block and the next data block, avoiding semantic incoherence. Assuming the text length is 200, then 1-200 words are the first data block, 151-250 are the second data block, and so on.

[0048] According to some implementations of the present disclosure, in the process of dividing text data into multiple data blocks, a granularity index associated with the user request can be determined, and a predetermined size for dividing the multiple data blocks can be determined based on the granularity index, and then the text data can be divided into multiple data blocks based on the predetermined size. It should be understood that the granularity index herein can represent the level of detail of the problem involved in the user request, for example, the granularity of "large, medium, small" (or other numerical ranges) can be used to represent it. The granularity of "large" can indicate that the user request involves a generalized problem, and the granularity of "small" can indicate that the user request involves a detailed problem. For example, the question "How to use the employee management system" can correspond to the granularity of "large", and the question "How to initiate a leave application" can correspond to the granularity of "small", and so on.

[0049] According to some implementations of the present disclosure, the predetermined size may be proportional to the granularity index. Generally speaking, the answers to general questions need to be summarized from a larger text, while the answers to detailed questions may be concentrated in a specific paragraph. In this way, the size of the data block can be determined based on the granularity index requested by the user, thereby facilitating the improvement of the efficiency of obtaining background knowledge associated with the question. For example, user questions may be collected in advance, and the size of the data block may be determined according to the proportional relationship between the number of user questions of various granularities. In this way, a balance may be found between questions of different granularities, and the needs of answering general questions and detailed questions may be met at the same time.

[0050] According to some implementations of the present disclosure, multiple data blocks can be divided according to different sizes, and multiple data blocks with different sizes can be obtained. For example, set 1 can include multiple large data blocks, set 2 can include multiple medium data blocks, and set 3 can include multiple small data blocks. At this time, each set can cover all text data. When a user request is received, a set of data blocks of a size that matches the granularity index of the user request can be selected. Assuming that the user request is "How to initiate a leave application", a search can be performed in set 3 to determine a set of corresponding small data blocks. In this way, the efficiency of acquiring background knowledge can be improved, thereby improving the accuracy of obtaining responses later.

[0051] According to some implementations of the present disclosure, selecting a group of data blocks matching the user request from multiple data blocks includes: in response to determining that the user request belongs to the scope of background knowledge, selecting a group of data blocks matching the user request from multiple data blocks.

[0052] Specifically, the questions input by the user can be classified, and the questions that can be answered directly can be answered according to the categories. For example, for high-frequency questions, directly reply with the corresponding answer. Taking the question-and-answer system for employee management as an example, high-frequency questions can be sorted out from time to time, such as: "How many days of annual leave do I have in the year of joining the company?" "How is the business trip allowance calculated?", and the corresponding answers to high-frequency questions can be prepared. When a user's question is classified as a high-frequency question, the corresponding content can be directly replied. For another example, for questions that are out of scope or sensitive questions, the refusal to answer and the reason can be displayed. For another example, for questions that require manual processing or that the large model cannot answer temporarily, it is recommended that the user contact the manual directly. In the example of the employee management system, if the user asks "Please change my account to supervisor authority", because it involves authority management, it can be recommended that the user contact the manual customer service directly.

[0053] By using some implementations of the present disclosure, by classifying user questions, a portion of questions with known answers and questions in fields that cannot be processed by the machine learning model can be filtered out before calling the machine learning model. At this time, only questions in the fields that can be processed are sent to the large model, which can reduce the resource usage of the large model and improve the efficiency of the question-answering system.

[0054] According to some implementations of the present disclosure, vectorization processing may be performed on each text block to obtain a corresponding vector representation (e.g., embedding). Each vector representation may be stored in a vector database, and a set of data blocks may be selected based on vector matching, thereby improving the efficiency of determining related data blocks.

[0055] Although vector retrieval can retrieve text blocks with high similarity by calculating the semantic correlation between user input and text blocks. However, vector retrieval does not work well in some cases, such as searching for the name of a person or object, searching for abbreviations or phrases, searching for IDs, etc. Alternatively and / or additionally, each data block can be indexed based on other methods. Keyword retrieval can extract keywords from the original content and establish a mapping relationship between keywords and original corpus. For example, indexes, TF-IDF, BM25, etc. can be used. In addition to extracting keywords contained in user input, the extracted keywords can also be expanded, such as synonym expansion, professional vocabulary expansion, etc., to improve the effectiveness of the retrieval. In this way, an index can be used to select a group of data blocks based on the matching degree between keywords.

[0056] According to some implementations of the present disclosure, in the process of selecting a group of data blocks, a group of data blocks can be determined based on at least one of vector retrieval and keyword retrieval. In this way, the accuracy of obtaining background knowledge can be improved, thereby improving the accuracy of the answer. By combining the advantages of different retrieval systems, such as the combination of vector retrieval and keyword retrieval, the complementarity of multiple retrieval technologies can be achieved.

[0057] According to some implementations of the present disclosure, in the process of selecting a group of data blocks that match a user request from multiple data blocks, keywords can be determined from the user request; and a group of data blocks can be selected based on the comparison of the keywords with the multiple data blocks. It should be understood that user requests usually do not strictly conform to grammatical specifications. At this time, the user request can be updated based on the semantic analysis of the user request, and the updated user request has the same semantic meaning as the user request. Specifically, in actual scenarios, the user's input may have spelling errors, ambiguities, inaccurate expressions, etc., which will directly affect the subsequent retrieval accuracy. Therefore, before vectorizing and / or keyword searching the user input, the user input needs to be rewritten, including but not limited to: error correction, case conversion, synonym rewriting, completion, intent recognition, question splitting, sensitive word filtering, etc.

[0058] According to some implementations of the present disclosure, at least any of the following operations may be performed: error correction, case conversion, synonym rewriting based on rules, sentence rewriting and question decomposition based on a large model, intent classification based on a Bert intent classification model, etc. By rewriting the original user input, the retrieval accuracy can be improved, thereby improving the reply accuracy of the large model.

[0059] When the user's question is not expressed in a standard way, it can be rewritten by replacing synonyms. For example, the user's original question is: "My small bubble can't be opened." Since "small bubble" is a colloquial expression for the employee management system, it cannot be directly retrieved in the database. After synonym replacement, the original question is rewritten as: "My employee management system can't be opened", and the correct information can be retrieved from the database.

[0060] For another example, when a user's question is too lengthy, the big model can be used to rewrite the question. For example, a user wants to apply for sick leave, and describes various symptoms of his or her physical discomfort in the user's request, and finally asks "How to apply for sick leave." At this time, the big model can be used to rewrite the original question, and the rewritten user question is: "How to apply for sick leave?" The rewritten question is matched with the content of each data block, and the correct information can be retrieved.

[0061] According to some implementations of the present disclosure, the prompt word may include: a user request and background information associated with the user request. Figure 3Describe the structure of the prompt word. Figure 3 A block diagram 300 is shown of the structure of a prompt word according to some implementations of the present disclosure. Figure 3 As shown, the prompt word 240 may include: a user request 330, which may describe the purpose of the user's desired query. For example, "{user request}" may be replaced by the received user request.

[0062] The prompt word 240 may include: background information 320, which may describe background knowledge related to answering the question. For example, the text in the selected set of data blocks 1, 2, 3, ..., and N may be used to replace "{data block 1, data block 2, data block 3, ..., data block N}" in the background information. Therefore, in the process of determining the prompt word based on the user request and a set of data blocks, the background information 320 may be determined based on the set of data blocks. With some implementations of the present disclosure, more background information may be provided to the machine learning model, thereby facilitating the machine learning model to extract accurate answers to user questions from the background information.

[0063] Alternatively and / or additionally, the prompt may include a role definition, a task definition, and step 310 for processing the user request. For example, the role may be represented as: You are an IT customer service assistant. For example, the task definition may be represented as: According to the following steps, use the provided background information and user questions to answer the user's questions in a clear, accurate, and organized manner. Please ensure that your answer is faithful to the background information and avoid including any irrelevant or erroneous content. According to some implementations of the present disclosure, the steps may include:

[0064] Step 1: Read and understand the background information carefully.

[0065] Step 2: Make sure you understand your users’ problems.

[0066] Step 3: Based on the background information, think about and provide an accurate response.

[0067] Step 4: Your answer should not include anything that is inconsistent with the background information.

[0068] Step 5: In the process of formulating your answer, you may use internal reasoning if necessary, but make sure that your final response is accurate and clear.

[0069] It should be understood that Figure 3 The prompt word 240 in the example is only illustrative. Alternatively and / or additionally, the prompt word 240 may be represented by different structures or may include different contents. For example, some prompt words may not include steps. In this case, the user request 330 may be represented as: "Please answer the following question: {user request}".

[0070] Specifically, the prompt words can be determined by finally selecting multiple data blocks (for example, the top N ranked data blocks). The content and order of the data blocks can be optimized, for example, including but not limited to extraction, summarization, and repackaging, etc. When the matching data block is long, it may exceed the input limit of the large model and slow down the reasoning process of the large model. In addition, the data block may include redundant or unnecessary information, which will hinder the large model from generating accurate answers. Therefore, it is necessary to optimize the content of the data block, such as extracting key sentences in the data block, removing redundant information, extracting summaries of the data block, and so on.

[0071] According to some implementations of the present disclosure, in the process of determining background information based on a set of data blocks, the order of the set of data blocks can be determined, and the set of data blocks can be arranged in order to determine the background information. It should be understood that when the data blocks 1-N are arranged in different orders, the responses output by the machine learning model may be different. In this case, different prompt words can be generated in different orders, and a response that is more in line with the user's request can be found.

[0072] Experiments show that in a prompt, the order of multiple related data blocks in the prompt word may affect the accuracy of the large model's generated answer. The purpose of repackaging (i.e., sorting) is to optimize the organization order of text blocks to improve the accuracy of the large model's answer. Repackaging methods include but are not limited to: arranging in ascending or descending order by similarity, placing the text blocks with the highest similarity at the beginning and end, etc.

[0073] According to some implementations of the present disclosure, in the process of determining the order of a group of data blocks, a set of matching degrees between a group of data blocks and a user request may be determined respectively, and the order is determined based on the set of matching degrees. Figure 4 Describe in more detail the Figure 4 A block diagram 400 is shown for sorting a group of data blocks based on the degree of match between the group of data blocks and a user request according to some implementations of the present disclosure. It should be understood that the degree of match here is also the degree of match based on which a group of data blocks are selected from a plurality of data blocks, for example, including a vector-based degree of match and a keyword-based degree of match, etc. For example, the degree of match between data block 220-1 and the user request is represented as a degree of match 410, and the degree of match between data block 220-M and the user request is represented as 412. The above-mentioned degrees of match 410, ..., 412 can be sorted to further determine the order of each data block. Using some implementations of the present disclosure, each data block can be sorted in different orders to find the most suitable order.

[0074] According to some implementations of the present disclosure, determining the order based on a set of matching degrees includes at least any of the following: determining the order of a group of data blocks according to a set of matching degrees arranged in ascending order; determining the order of a group of data blocks according to a set of matching degrees arranged in descending order; and arranging the target data block at a predetermined position in a group of data blocks in response to determining that the order of the matching degrees of a target data block in a group of data blocks meets a predetermined condition.

[0075] Figure 5 A block diagram 500 of sorting criteria according to some implementations of the present disclosure is shown. Figure 5 As shown, the sorting criteria 510 shows multiple criteria. For example, the data blocks 1-N can be sorted in ascending order of matching degree. At this time, the matching degree between data block 1 and the user query is the lowest, ..., and the matching degree between data block N and the user query is the highest. Alternatively and / or additionally, the data blocks 1-N can be sorted in descending order of matching degree. At this time, the matching degree between data block 1 and the user query is the highest, ..., and the matching degree between data block N and the user query is the lowest.

[0076] Alternatively and / or additionally, the target data blocks that meet the predetermined conditions are arranged at predetermined positions in the data block queue. Specifically, the data blocks with the greatest matching degree can be arranged at both ends of the data block queue, for example, the data block with the greatest matching degree can be arranged at position 1 at the head, and the data block with the second greatest matching degree can be arranged at position N at the tail. For another example, the data block with the greatest matching degree can be arranged at position N at the tail, and the data block with the second greatest matching degree can be arranged at position 1 at the head. Further, the data block with the least matching degree can be arranged in the center of the data block queue. For example, the data blocks can be placed in position 1, position N, position 2, position N-1, position 3, position N-2, and so on in the queue in descending order of matching degree. In this way, the matching degree of the data blocks at the two end positions in the queue is high, and gradually decreases towards the center position of the queue.

[0077] According to some implementations of the present disclosure, the effects of different orders on the correctness of responses can be determined, and the order that can produce the correct response can be selected. For example, experiments have shown that arranging the data blocks with the highest matching degree at both ends of the queue helps improve the correctness, so the data blocks with the highest matching degree can be arranged at both ends of the data block queue. For another example, the data blocks can be arranged in an order from both ends to the center with the matching degree gradually decreasing. In this way, the accuracy of the response can be improved, and a response that better meets the user's expectations can be obtained.

[0078] According to some implementations of the present disclosure, in the process of providing an answer to a user request, the order of a group of data blocks in the background information in the prompt word can be adjusted to generate another prompt word. Another response of the machine learning model to another prompt word can be received, and the correlation between the user request and the response and another correlation between the user request and the other response can be determined respectively. Further, the correlation and another correlation can be compared to provide an answer to the user request.

[0079] According to some implementations of the present disclosure, prompt words may include: role definition, task definition, step requirements, background information (i.e., the top N data blocks retrieved from the database), and user questions. The order of these parts will affect the accuracy of the answers generated by the large model, so these parts need to be repackaged. The purpose of repackaging is to optimize the organization order of text blocks to improve the accuracy of the answers of the large model.

[0080] Experiments have found that in most cases, placing background information and user questions in adjacent locations is more effective. Placing the most relevant data blocks (for example, top1 and top2) at the head and tail of the background information will result in higher answer accuracy for large models. Different models may have different optimal packaging orders, and the packaging order can be determined based on actual conditions.

[0081] See also Figure 6 Describe in more detail the Figure 6 A block diagram 600 is shown of another structure of a prompt word according to some implementations of the present disclosure. Figure 6 As shown, the order of each part in the prompt word can be adjusted. For example, after the role definition and task definition, the prompt word can be determined in the order of background information, steps, and user requests; the prompt word can be determined in the order of user requests, background information, and steps; the order of each data block in the background information can be adjusted; the step part can be omitted, and so on. In this way, different prompt words can be determined, and multiple prompt words can be input into the machine learning model respectively to obtain multiple responses respectively. Further, the correlation between multiple responses and user requests can be compared respectively. Specifically, the correlation can be determined based on a variety of methods such as semantic analysis models and keyword comparisons, and the response with the highest correlation can be selected as a reply to the user request.

[0082] By using some implementations of the present disclosure, the powerful processing capabilities of the machine learning model can be fully utilized to process user queries and generate multi-faceted responses. Further, responses that are more in line with user expectations can be selected. Alternatively and / or additionally, multiple responses can be provided to the user for reference. In this way, the efficiency of solving user problems can be improved.

[0083] According to some implementations of the present disclosure, for questions that can be answered directly through text, the big model can generate text replies for the user's reference. For user questions that require the next step of operation (such as sending an email or creating a work order), the big model can automatically generate corresponding text content to facilitate the user to directly perform the next step of operation.

[0084] In the process of providing an answer to a user's request, in response to determining that the user's request involves a subsequent processing task, an application for executing the subsequent processing task is called; and the application is used to provide a response. Specifically, other applications other than the question-and-answer system can be called to reply to the user. For example, in the example of the employee management system above, assuming that the user's request is "how to initiate a leave application", a link address for initiating a leave application can be sent to the user in the employee management system in the form of an internal message or internal email. For another example, assuming that the user's request is "how to contact the system administrator", the email application can be directly started and the email address of the system administrator can be provided.

[0085] For another example, for user issues that require emails to be sent for collaboration, the big model can directly generate the email address, title, and body, and the user can directly send these contents by email. For user issues that require assistance from a third party, the big model can generate the necessary content for the work order, and the user can directly create the work order, thereby improving the convenience of user operations. In this way, user requests can be handled in a simpler and more effective way.

[0086] The various steps in processing user requests have been described separately. Figure 7 The overall workflow according to some implementations of the present disclosure is described. Figure 7 A block diagram 700 is shown of a specific process for processing a user request according to some implementations of the present disclosure. Figure 7 As shown, at box 701, initial document data associated with background knowledge can be obtained. At box 702, text data can be extracted from initial document data in various formats. At box 703, the extracted text data can be divided into multiple text blocks (i.e., data blocks). For example, it can be cut and divided according to paragraphs, or cut according to specific characters (such as commas, periods, semicolons), cut according to text length, etc. At box 704, vectorization operations can be performed on each text block. Specifically, the text block can be input into the encoder model to generate a corresponding vector representation. At box 705, the vector representation can be stored in a vector database for later comparison. At this point, the preparation stage ends.

[0087] At block 706, a user request may be received. At block 707, a question classification may be determined, that is, whether the user request belongs to a classification currently processed by the question-answering system. If the user request does not belong to a classification processed by the question-answering system, the user request may be forwarded to another processing system at block 708. If the user request belongs to a classification processed by the question-answering system, it may be determined at block 709 whether the user request needs to be rewritten, and the user request may be rewritten if necessary.

[0088] At box 710, the user request may be vectorized. For example, the same encoder may be used to determine the vectorized representation of the user request, and a search for matching text blocks may be performed in a vector database at box 712. Further, at box 711, keyword extraction may be performed for the user request and keywords of similar fields may be expanded. At box 713, a keyword search may be performed to determine matching text blocks. At box 714, a hybrid search and sorting may be performed based on the results of the vector search 712 and the keyword search 713 to select a related set of text blocks.

[0089] At box 715, a prompt word can be determined based on the method described above using a set of text blocks requested and selected by the user. Here, the individual text blocks can be arranged according to a predetermined sorting standard to determine the prompt word. Alternatively and / or additionally, the individual text blocks can be arranged in a variety of ways and multiple prompt words can be generated. At box 716, one or more prompt words can be input to model 716 to retrieve a corresponding response. At box 717, the response can be provided to the user, or subsequent processing can be provided.

[0090] Alternatively and / or additionally, at box 718, an instruction set is constructed, and model fine-tuning can be performed at box 719. Specifically, the instruction data that can be collected include, but are not limited to: (1) industry knowledge base question and answer data, (2) standard question and answer data, (3) instruction data generated based on a self-guided method, and (4) general instruction data, etc. Here, (1)-(3) involve vertical domain data, and (4) involves general domain data. A ratio between the two can be set, such as 1:5 or other ratios, to ensure that the fine-tuned model can not only answer domain questions, but also maintain the original general capabilities. Furthermore, a variety of existing models can be used as base models to fine-tune the model on the instruction set.

[0091] By utilizing some implementations of the present disclosure, user questions are classified and only a portion of the user questions are input into the large model, thus saving the resource usage of the large model. The accuracy of matching text blocks can be improved by identifying the intent of user input and rewriting it into a more standardized expression. Extracting and expanding keywords for user questions before keyword search can improve retrieval accuracy. For a set of data blocks that are finally selected, the content and order of the text blocks can be optimized when determining the prompt words, thereby improving the accuracy and efficiency of the generation of the large model. For user questions that require the next step of operation (such as sending an email or creating a work order), the large model can automatically generate the corresponding text content, making it convenient for users to directly perform the next step.

[0092] By using some implementations of the present disclosure, a set of data blocks matching the user's request can be selected, and relevant background knowledge about the user's request can be obtained. By including the relevant background knowledge in the prompt words, on the one hand, a large amount of knowledge of the machine learning model itself can be utilized, and on the other hand, the powerful processing power of the machine learning model can be utilized to analyze the background knowledge, thereby providing a more accurate response. Furthermore, the user's intention can be better identified, the retrieval accuracy can be improved, the accuracy and efficiency of the large model's answers can be improved, and the convenience of user operation can be improved, thereby improving the user satisfaction of the intelligent question-answering system.

[0093] Example Process

[0094] Figure 8 A flow chart of a method 800 for processing a user request according to some implementations of the present disclosure is shown. At block 810, a set of data blocks matching the user request is selected from a plurality of data blocks, the plurality of data blocks indicating background knowledge associated with the user request. At block 820, a prompt word for input to a machine learning model is determined based on the user request and the set of data blocks. At block 830, an answer to the user request is provided based on a response of the machine learning model to the prompt word.

[0095] According to some implementations of the present disclosure, the prompt word includes: a user request and context information associated with the user request, and determining the prompt word based on the user request and a set of data blocks includes: determining the context information based on the set of data blocks.

[0096] According to some implementations of the present disclosure, determining the context information based on a group of data blocks includes: determining an order of the group of data blocks; and arranging the group of data blocks in the order to determine the context information.

[0097] According to some implementations of the present disclosure, determining the order of a group of data blocks includes: respectively determining a set of matching degrees between the group of data blocks and user requests; and determining the order based on the set of matching degrees.

[0098] According to some implementations of the present disclosure, determining the order based on a set of matching degrees includes at least any of the following: determining the order of a group of data blocks according to a set of matching degrees arranged in ascending order; determining the order of a group of data blocks according to a set of matching degrees arranged in descending order; and arranging the target data block at a predetermined position in a group of data blocks in response to determining that the order of the matching degrees of a target data block in a group of data blocks meets a predetermined condition.

[0099] According to some implementations of the present disclosure, providing an answer to a user request further includes: adjusting the order of a set of data blocks in the background information in a prompt word to generate another prompt word; receiving another response of the machine learning model to another prompt word; respectively determining a correlation between the user request and the response, and another correlation between the user request and the other response; and comparing the correlation and another correlation to provide an answer to the user request.

[0100] According to some implementations of the present disclosure, the method further includes: acquiring text data describing background knowledge associated with the user request; and dividing the text data into a plurality of data blocks.

[0101] According to some implementations of the present disclosure, dividing text data into multiple data blocks includes: determining a granularity index associated with a user request; determining a predetermined size for dividing the multiple data blocks based on the granularity index; and dividing the text data into multiple data blocks based on the predetermined size.

[0102] According to some implementations of the present disclosure, the predetermined size is proportional to the granularity index.

[0103] According to some implementations of the present disclosure, selecting a group of data blocks includes: determining a group of data blocks based on at least one of vector retrieval and keyword retrieval.

[0104] According to some implementations of the present disclosure, selecting a group of data blocks matching the user request from multiple data blocks includes: in response to determining that the user request belongs to the scope of background knowledge, selecting a group of data blocks matching the user request from multiple data blocks.

[0105] According to some implementations of the present disclosure, selecting a group of data blocks matching a user request from a plurality of data blocks includes: determining a keyword from the user request; and selecting a group of data blocks based on a comparison of the keyword with the plurality of data blocks.

[0106] According to some implementations of the present disclosure, the method further includes: updating the user request based on semantic analysis of the user request, wherein the updated user request has the same semantic meaning as the user request.

[0107] According to some implementations of the present disclosure, providing an answer to a user request further includes: in response to determining that the user request involves a subsequent processing task, calling an application for performing the subsequent processing task; and using the application to provide a response.

[0108] Example Device

[0109] Fig. 9 9 is a block diagram of a device 900 capable of implementing multiple implementations of the present disclosure. It should be understood that Fig. 9 The computing device 900 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the implementations described herein. Fig. 9 The computing device 900 shown may be used to implement the methods described above.

[0110] like Fig. 9 As shown, computing device 900 is in the form of a general-purpose computing device. Components of computing device 900 may include, but are not limited to, one or more processors or processing units 910, memory 920, storage device 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. Processing unit 910 may be an actual or virtual processor and is capable of performing various processes according to a program stored in memory 920. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capabilities of computing device 900.

[0111] The computing device 900 typically includes a plurality of computer storage media. Such media may be any available media accessible to the computing device 900, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 920 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 930 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the computing device 900.

[0112] The computing device 900 may further include additional removable / non-removable, volatile / non-volatile storage media. Fig. 9As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. Memory 920 may include a computer program product 925 having one or more program modules that are configured to perform various methods or actions of various implementations of the present disclosure.

[0113] The communication unit 940 enables communication with other computing devices via a communication medium. Additionally, the functions of the components of the computing device 900 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 900 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0114] Input device 950 may be one or more input devices, such as a mouse, keyboard, tracking ball, etc. Output device 960 may be one or more output devices, such as a display, a speaker, a printer, etc. Computing device 900 may also communicate with one or more external devices (not shown) through communication unit 940 as needed, such as storage devices, display devices, etc., communicate with one or more devices that allow a user to interact with computing device 900, or communicate with any device that allows computing device 900 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0115] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, the computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0116] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0117] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0118] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0119] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0120] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for processing a user request, include: selecting a set of data blocks matching the user request from a plurality of data blocks, the plurality of data blocks indicating background knowledge associated with the user request; Based on the user request and the set of data chunks, determining a prompt word for input into a machine learning model; as well as Based on the machine learning model's response to the prompt word, provide an answer to the user's request.

2. The method according to claim 1, wherein the prompt word include: The user request and context information associated with the user request, and determining the prompt word based on the user request and the set of data blocks includes: determining the context information based on the set of data blocks.

3. The method according to claim 2, wherein the context information is determined based on the set of data blocks. include: determining an order of the set of data blocks; as well as The set of data blocks are arranged in the order to determine the background information.

4. The method according to claim 3, wherein determining the order of the set of data blocks include: respectively determining a set of matching degrees between the set of data blocks and the user request; as well as The order is determined based on the set of matching degrees.

5. The method according to claim 4, wherein determining the order based on the set of matching degrees comprises at least any one of the following: determining an order of the set of data blocks according to the set of matching degrees arranged in ascending order; determining an order of the set of data blocks according to the set of matching degrees arranged in descending order; and In response to determining that the ranking of the matching degree of the target data block in the group of data blocks meets a predetermined condition, the target data block is arranged at a predetermined position in the group of data blocks.

6. The method of claim 3, wherein providing an answer to the user request further comprises: include: adjusting the order of the group of data blocks in the background information in the prompt word to generate another prompt word; receiving another response from the machine learning model to the another prompt word; respectively determining a degree of association between the user request and the response, and another degree of association between the user request and the other response; as well as The degree of relevance is compared to the further degree of relevance to provide the answer to the user request.

7. The method according to claim 1, further comprising: include: Acquire text data describing the background knowledge associated with the user request; as well as The text data is divided into the plurality of data blocks.

8. The method according to claim 1, wherein the text data is divided into the plurality of data blocks include: determining a granular metric associated with the user request; determining a predetermined size for dividing the plurality of data blocks based on the granularity indicator; as well as Based on the predetermined size, the text data is divided into the plurality of data blocks.

9. The method of claim 8, wherein the predetermined size is proportional to the particle size index.

10. The method of claim 1, wherein the set of data blocks is selected include: The group of data blocks is determined based on at least one of a vector search and a keyword search.

11. The method according to claim 1, wherein a set of data blocks matching the user request is selected from a plurality of data blocks. include: In response to determining that the user request belongs to the scope of the background knowledge, a group of data blocks matching the user request are selected from a plurality of data blocks.

12. The method according to claim 1, wherein a set of data blocks matching the user request is selected from a plurality of data blocks. include: determining keywords from the user request; as well as The set of data blocks is selected based on a comparison of the keyword with the plurality of data blocks.

13. The method according to claim 1, further comprising: include: Based on the semantic analysis of the user request, the user request is updated, and the updated user request has the same semantic meaning as the user request.

14. The method of claim 1, wherein providing the answer to the user request further comprises: include: In response to determining that the user request involves a subsequent processing task, calling an application for executing the subsequent processing task; as well as The response is provided using the application.

15. An electronic device, include: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 14 when executed by the at least one processing unit.

Images