Problem processing method and device based on large language model

Abstract

The invention provides a problem processing method and device based on a large language model, and the method comprises the steps: carrying out the intention recognition of an input problem, determining the intention information of the input problem, and determining an intention processing flow corresponding to the processing intention information; when it is determined that the intention processing flow is an agent flow, determining a reply result of the input question based on an agent large language model and parameter information extracted from the target function interface; and under the condition that the intention processing flow is determined to be flow arrangement, determining a processing flow corresponding to the intention information, solving the input problem based on the processing flow corresponding to the intention information, and determining a reply result of the input problem. Through multi-level understanding, process selection, information retrieval and decomposition processing, the accuracy and correlation of the output result of the large language model are effectively improved. According to the structured processing mode, the system can generate high-quality answers better meeting user requirements, so that the user experience is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a problem processing method and device based on a large language model. Background Art

[0002] A big language model is a deep learning model trained using large amounts of text data that can understand and generate natural language.

[0003] The application of large language models in existing methods is generally based on retrieval-augmented generation (RAG) or agent-based methods. The RAG-based method only relies on the vectorization of the original document, which will lead to the loss of multi-dimensional information and reduce the accuracy of retrieval. The agent-based method has uncertainty in the call of external interfaces and is prone to exceptions. How to improve the accuracy of the output results of large language models is an important issue that needs to be solved in the industry. Summary of the invention

[0004] The present invention provides a problem processing method and device based on a large language model, so as to improve the accuracy of the output results of the large language model.

[0005] The present invention provides a problem processing method based on a large language model, comprising the following steps: Performing intent recognition on the input question, determining the intent information of the input question, and determining the intent processing flow corresponding to processing the intent information, wherein the intent processing flow at least includes an agent flow and an orchestration flow, wherein the agent flow is used to characterize that the intent processing flow is applicable to a flow for processing by an agent large language model, and the orchestration flow is used to characterize that the intent processing flow is applicable to a predefined and orchestrated processing flow; In the case where it is determined that the intention processing process is an agent process, based on the intention information, a target function interface to be called is determined from all function interfaces included in the business, and based on the agent large language model and parameter information extracted from the target function interface, a response result to the input question is determined, and the function interface is constructed based on the solution parameters corresponding to the question; When it is determined that the intention processing flow is a process orchestration, the processing flow corresponding to the intention information is determined, and based on the processing flow corresponding to the intention information, the input question is solved to determine the answer result of the input question.

[0006] According to a method for processing a question based on a large language model provided by the present invention, the processing flow corresponding to the intention information is used to solve the input question and determine the answer result of the input question, including: In the case where it is determined that the processing flow corresponding to the intention information is to determine product information, based on cosine similarity, the vector corresponding to the input question is matched in a pre-built vector library, and a matching result list is output, wherein the vector library is built based on the product information in the product library; Based on the large language model, the product information in the matching result list is supplemented to obtain a reply result.

[0007] According to a problem processing method based on a large language model provided by the present invention, the construction process of the vector library includes: Obtain product information in the product library, and perform vector processing on the product information corresponding to the input questions of each historical user, to obtain vector information corresponding to multiple product information; Based on associating each commodity information with its corresponding vector information, associated data is obtained, and a vector library is constructed based on the associated data.

[0008] According to a question processing method based on a large language model provided by the present invention, the step of performing intent recognition on an input question and determining intent information of the input question includes: Extracting intended keywords from the input question based on keyword matching; The large language model is called to perform intent recognition on the intent keywords to determine the intent information of the input question.

[0009] According to a method for processing a question based on a large language model provided by the present invention, after determining the answer result of the input question, the method further includes: Based on the intention information, determining scenario information of the input question; Based on the scenario information and the reply result, the display form of the reply result is determined, and a display tool corresponding to the display form is called to display the reply result.

[0010] A problem processing method based on a large language model provided by the present invention also includes: When it is determined that the intent processing flow for processing the input question is a simplified intent flow based on the intent information, the answer result of the input question is determined based on calling the pre-configured reply information, and the simplified intent flow is used to characterize that the intent information contains only a single intent.

[0011] The present invention also provides a problem processing device based on a large language model, comprising the following modules: An intention recognition module is used to recognize the intention of the input question, determine the intention information of the input question, and determine the intention processing flow corresponding to the intention information, wherein the intention processing flow at least includes an agent flow and an orchestration flow, wherein the agent flow is used to characterize that the intention processing flow is applicable to a flow processed by an agent large language model, and the orchestration flow is used to characterize that the intention processing flow is applicable to a predefined and orchestrated processing flow; A first reply module is used to determine, when determining that the intention processing process is an agent process, a target function interface to be called from all function interfaces included in the business based on the intention information, and to determine a reply result to the input question based on the agent large language model and parameter information extracted from the target function interface, wherein the function interface is constructed based on solution parameters corresponding to the question; The second reply module is used to determine the processing flow corresponding to the intention information when it is determined that the intention processing flow is a process arrangement, and solve the input question based on the processing flow corresponding to the intention information to determine the reply result of the input question.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the problem processing method based on a large language model as described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described problem processing methods based on a large language model.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned problem processing methods based on a large language model.

[0015] The problem processing method and device based on the large language model provided by the present invention can effectively improve the accuracy and efficiency of the system's response by identifying the intent of the input question and selecting an appropriate processing flow. The agent process is suitable for quickly processing simple queries, while the orchestration process can respond to more complex requests and combine various information to meet user needs. Through multi-level understanding, process selection, information retrieval and decomposition processing, the accuracy and relevance of the output results of the large language model are effectively improved. This structured processing method enables the system to generate high-quality answers that better meet user needs, thereby greatly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of the problem processing method based on the large language model provided by the present invention.

[0018] Figure 2 It is a schematic diagram of the intent recognition process provided by the present invention.

[0019] Figure 3 It is a schematic diagram of the intention processing flow provided by the present invention.

[0020] Figure 4 It is a schematic diagram of the intelligent agent process processing provided by the present invention.

[0021] Figure 5 It is a schematic diagram of the arrangement process processing provided by the present invention.

[0022] Figure 6 It is a schematic diagram of the vector library construction process provided by the present invention.

[0023] Figure 7 It is a schematic diagram of the vector interface query process provided by the present invention.

[0024] Figure 8 It is a schematic diagram of the matching algorithm implementation process provided by the present invention.

[0025] Fig. 9 It is a schematic diagram of the interface call encapsulation process provided by the present invention.

[0026] Fig.10 It is a schematic diagram of the text information replacement process provided by the present invention.

[0027] Fig.11 It is a schematic diagram of the array information replacement process provided by the present invention.

[0028] Fig.12 It is a flowchart of the problem processing method based on the large language model provided by the present invention.

[0029] Fig.13 It is a structural schematic diagram of the problem processing device based on the large language model provided by the present invention.

[0030] Fig.14 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The big language model is a deep learning model trained with a large amount of text data that can understand and generate natural language. The advantages of the big language model are: 1. Powerful language understanding ability: The big language model can deeply understand the structure and semantics of the language by learning a large amount of text data. This understanding ability makes the big language model perform well in the field of natural language processing, whether it is grammatical analysis, sentiment analysis or language translation. 2. Content generation ability: With a deep understanding of language, the big language model can generate coherent and logical text content. This has been widely used in automatic writing, chatbots, and various text generation applications. For example, it can be used to generate articles, reports, and even create poems and novels. 3. Cross-domain application capabilities: The big language model is not limited to applications in specific fields. After proper training and adjustment, the big language model can play a huge role in many industries such as medicine, law, and finance, providing professional content generation and information analysis services.

[0033] The application of large language models in related methods is mainly based on RAG or Agent.

[0034] The RAG architecture in the related methods aims to improve the performance of language models by integrating the two major steps of retrieval and generation. Its workflow includes retrieving relevant information from a large collection of documents (retrieval), analyzing and organizing this information (utilization), and finally generating text or answering questions based on this information (generation). The main advantages are: 1. It can effectively utilize external knowledge bases to provide more accurate and valuable answers, improve the reliability and depth of texts, and have the ability to update the knowledge base in real time, and provide the latest information without retraining the model; 2. Instant update of data: It has the ability to update the knowledge base in real time, and can provide the latest information without retraining the model. 3. Highly interpretable answers: Since the answers come directly from the retrieval library, RAG's answers are highly interpretable.

[0035] The agent in the related methods is an intelligent entity that can perceive the environment, make decisions and perform actions. It can perceive the environment, make decisions and perform actions to achieve the preset goals. Agents can be software programs, robots, virtual assistants, etc. They provide various services and functions through interaction with users or other systems. Agents provide one or more ways to call LLM, and determine the call tool or what to do next based on the expected results until the final result is obtained.

[0036] The shortcomings of the related methods are: In the related methods, the big model technology framework RAG constructs the knowledge base by sharding only the original document information and then vectorizing the data shards. When performing knowledge indexing, the corresponding knowledge shards are obtained by comparing the vectorized input question with the data shard vectors in the vector library and then handed over to the big model for summary output. This method only considers the indexing method of knowledge document information. For enterprise-level applications, such as product information, there are many different retrieval dimensions. Only document data vectors are generated for the original information, and the corresponding data dimension information will be lost during the indexing process, reducing the accuracy of the indexed data.

[0037] In the related methods, the agent calls the external interface function. There are many types of external interface formats, and the agent needs to be adapted and trained according to different interface types. In order to be compatible with the heterogeneity of different interfaces, in enterprise-level applications, compatibility needs to be achieved through code development and online release, and the overall function launch cycle is relatively long.

[0038] At the same time, the intelligent agent method used by large models to obtain external interface data has a high training cost, and the external interface call information returned by the large model has great uncertainty, which can easily cause interface call exceptions and lead to failure of the overall process.

[0039] In view of the defects of the related methods, the present invention provides a problem processing method based on a large language model. Figure 1 is a flow chart of the problem processing method based on the large language model provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 110, performing intent recognition on the input question, determining intent information of the input question, and determining an intent processing flow corresponding to processing the intent information, wherein the intent processing flow at least includes an agent flow and an orchestration flow, wherein the agent flow is used to characterize that the intent processing flow is applicable to a flow for processing by an agent large language model, and the orchestration flow is used to characterize that the intent processing flow is applicable to a predefined and orchestrated processing flow; Step 120, when it is determined that the intention processing flow is an agent flow, based on the intention information, determine the target function interface to be called from all function interfaces included in the business, and determine the answer result of the input question based on the agent large language model and parameter information extracted from the target function interface, wherein the function interface is constructed based on the solution parameters corresponding to the question; Step 130, when it is determined that the intention processing flow is a process arrangement, determine the processing flow corresponding to the intention information, and solve the input question based on the processing flow corresponding to the intention information to determine the answer result of the input question.

[0040] The execution subject of the problem processing method based on the large language model provided by the present invention can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), or a personal computer (PC), etc., which is not specifically limited by the present invention.

[0041] The technical solution of the present invention is described in detail below by taking a computer executing the problem processing method based on a large language model provided by the present invention as an example.

[0042] In step 110, intent recognition is performed on the input question to determine the intent information of the input question, and the intent processing flow corresponding to the intent information is determined. The intent processing flow includes at least an agent flow and an orchestration flow. The agent flow is used to characterize that the intent processing flow is applicable to a process for processing a large language model of an agent, and the orchestration flow is used to characterize that the intent processing flow is applicable to a predefined and orchestrated processing flow.

[0043] Intent recognition is the first step to understand user input, which aims to extract user intent information from natural language.

[0044] Optionally, the intent recognition process is as follows Figure 2 The intention recognition process provided by the present invention is shown in the schematic diagram, which can be specifically: Input question scenario-based: The main function of input question scenario-based is to define and identify the intention of input questions and provide guidance for subsequent processes.

[0045] When the input question comes in, it is filtered based on keyword rules. Based on the scene discrimination model, the input question is distinguished by scene, and the scene to which the input question belongs is determined. Among them, all intent scenes and the processing modules to which the intent belongs are confirmed through the pre-built intent management module. For example, in the field of mobile communications, there may be terminal purchase scenarios, bill query scenarios, and phone recharge scenarios. Through the AI ​​intelligent management module, it is possible to interact with the large model to identify the input questions. First, the input question and user interaction history information are used as placeholders, and the prompt word information corresponding to the intent discrimination is configured in the pre-built AI intelligent management module, and the recognized intent information is obtained by calling the large language model. The recognized intent information can also be obtained based on the relevant text classification model.

[0046] Specifically, intents can be graded into two levels: scenario intent (first-level intent) and scenario-specific intent (second-level intent, which is a sub-intent of the first-level intent). Scenario intent (first-level intent) can be divided into multiple scenario-specific intents (second-level intents). For example, in the field of mobile communications, bill inquiry intent and terminal purchase intent are scenario intents (first-level intent), and terminal details description and terminal comparison are scenario-specific intents (second-level intents) under terminal purchase intent.

[0047] Optionally, the specific intent processing flow can be as follows Figure 3 As shown in the schematic diagram of the intent processing flow provided by the present invention, the corresponding intent processing flow is obtained according to the intent information of the user's question, which may include a brief flow corresponding to the brief intent, an intelligent flow, and an orchestration flow. Through the intelligent flow and the orchestration flow, the interface encapsulated by the tool module can be used to obtain external system information in a unified manner to assist system decision-making. For the brief flow, the pre-configured reply information can be called according to the intent name.

[0048] Specifically, the AI ​​intelligent management module is mainly used to set the model type, prompt word type and related information.

[0049] Model types are divided into: generative model, text classification model (such as BERT). For the generative model, you need to set the corresponding prompt word information and access method; for the text classification model, you only need to set the access method to access the classification model.

[0050] The prompt word types are divided into scene prompt words, sub-scene prompt words, and intelligent agent prompt words. Set different tags and versions for the prompt words. You can get the corresponding latest version of the prompt word information through the tag. Configure input questions, user interaction records and other placeholder information in the scene prompt word type. Replace the corresponding placeholder with actual information during the actual interaction process to call the large model and obtain the large model output information.

[0051] Scenario prompt words are used to distinguish the corresponding scenario intention information; segmented scenario prompt words are used to obtain segmented scenario intention information and entity word information; agent prompt words are used in agent scenarios to obtain execution process information.

[0052] According to the identified intent information, the corresponding intent processing process is selected, which is mainly divided into the intelligent agent process and the orchestration process.

[0053] The agent process is used to characterize the process that the intention processing process is applicable to the agent large language model processing. Among them, the agent large language model is the Agent model in the relevant method, which is an intelligent entity that can perceive the environment, make decisions and perform actions.

[0054] The orchestration process is used to characterize that the intention processing process is applicable to a predefined and orchestrated processing process. Among them, the orchestration process is a predefined processing process used to process the input question. According to the intention information of the input question, it is determined whether it matches the predefined orchestration process. If it matches, the orchestration process is directly called to implement the processing process.

[0055] In step 120, when it is determined that the intention processing process is an agent process, based on the intention information, the target function interface to be called is determined from all function interfaces included in the business, and based on the agent large language model and the parameter information extracted from the target function interface, the answer result of the input question is determined, and the function interface is constructed based on the solution parameters corresponding to the question.

[0056] Among them, the processing flow of the intelligent agent process can be as follows Figure 4 The intelligent agent process processing schematic diagram provided by the present invention is shown in FIG.

[0057] All function interfaces corresponding to the business are pre-built. The function interface is built based on the solution parameters corresponding to the problem. According to the parsed intent information, match it in the business function interface library to find the corresponding target function interface.

[0058] The target function interface usually contains some necessary parameters, which are required to execute the function. Therefore, it is necessary to extract the information of these parameters from the input problem or context and fill them into the corresponding positions of the target function interface.

[0059] By calling the objective function interface and passing in the corresponding parameter information, and analyzing the parameter information based on the agent's large language model, the answer to the input question can be generated.

[0060] In step 130, when it is determined that the intention processing flow is a process orchestration, the processing flow corresponding to the intention information is determined, and based on the processing flow corresponding to the intention information, the input question is solved to determine the answer result of the input question.

[0061] It should be noted that process orchestration refers to the combination and coordination of a series of independent but related tasks or operations in a specific order and logic to achieve a complete business process or function.

[0062] Among them, the processing flow of the orchestration process can be as follows Figure 5 The arrangement process processing schematic diagram provided by the present invention is shown in FIG.

[0063] When the input intent is complex, involves multiple steps of processing, data retrieval, or requires integration of multiple information sources, choose the orchestration process. In this step, the original intent can be analyzed at a deeper level to identify potential sub-intents.

[0064] Based on the identified sub-intention, the processing flow corresponding to the sub-intention is determined, and the input question is solved based on the further arranged processing flow to obtain the answer result of the input question.

[0065] The problem processing method based on the large language model provided by the present invention can effectively improve the response accuracy and efficiency of the system by identifying the intent of the input question and selecting the appropriate processing flow. The agent process is suitable for quickly processing simple queries, while the orchestration process can respond to more complex requests and combine various information to meet user needs. Through multi-level understanding, process selection, information retrieval and decomposition processing, the accuracy and relevance of the output results of the large language model are effectively improved. This structured processing method enables the system to generate high-quality answers that better meet user needs, thereby greatly improving the user experience.

[0066] In one embodiment, based on the processing flow corresponding to the intent information, the input question is solved to determine the answer result of the input question, including: when it is determined that the processing flow corresponding to the intent information is to determine product information, based on cosine similarity, the vector corresponding to the input question is matched in a pre-constructed vector library, and a matching result list is output, wherein the vector library is constructed based on the product information in the product library; based on a large language model, the product information in the matching result list is supplemented to obtain a reply result.

[0067] The relevance between the input question vector and the product information vector is determined by calculating the cosine similarity between the two. According to the similarity score sorting, the system will obtain a list of matching results, including some product information closest to the input question. This matching method based on cosine similarity ensures the relevance of the matching results and can meet the actual needs of users.

[0068] In order to achieve fast matching, a vector library of product information can be built in advance. This vector library is built based on product information, and each piece of product information has been vectorized and stored in the library for efficient query.

[0069] By querying the vector library, the system can achieve efficient and accurate matching, reduce resource consumption of real-time computing, and ensure the accuracy of matching results.

[0070] After obtaining the list of matching results with the highest similarity, the large language model is used to further supplement and expand the list to ensure the richness and completeness of the reply content.

[0071] Optionally, the vector library construction process can be as follows Figure 6 The vector library construction process provided by the present invention is shown in the schematic diagram.

[0072] Obtain various types of product information to be vectorized (such as package product information and terminal product information in the mobile communications field) from heterogeneous databases (such as Redis, MySQL, and MongoDB), and convert them into a unified form required for vectorized data construction to ensure the accuracy of subsequent vectorization.

[0073] For different types of data, according to the characteristics of the data to be vectorized (such as in the mobile communications field: the name, introduction, details of the package, etc.; the packing list, name, brand, etc. of the terminal information), use pre-built dictionary information to map according to different dimensions (Redis key / MySQL table / MongoDB collection + field type + weight).

[0074] The data is sharded, the embedding engine is called, the sharded data is vectorized, and multiple vector information is generated to improve the parallel processing capability of vectorization.

[0075] The vectorized data is associated with the original data, and the associated data is stored in the vectorized database to facilitate subsequent data retrieval and analysis.

[0076] Specifically, for the user-side input construction vector: User-side information mainly includes: user query words; user enumeration attributes: user province, user number type, user package type; user numerical attributes: call balance, data traffic balance.

[0077] The user query vector is constructed as: UserQuestionVector=Bert (input question).

[0078] Among them, BERT is the language model, and the user asks the vector to take the 0th dimension vector output by BERT.

[0079] The user enumeration class vector is: ; The vectors after OneHot encoding of the enumeration attributes asked by the user are concatenated by column.

[0080] The user value class vector is: ; ; After scaling the user numerical data to the maximum and minimum values, the case sequences are concatenated and the user query vector is finally output: ; The product side vector includes product description information of different dimensions, product enumeration attributes, and product numerical attributes.

[0081] Product details dimension vector: ; The vectors of the enumerated attributes of the products are OneHot encoded and concatenated by column. The product value vectors are: ; ; The product numerical data is scaled by the maximum and minimum values ​​and then concatenated by column. The remaining positions are padded with 0 to finally obtain the product vector matrix GoodVectors.

[0082] The encoding process of product information is as follows: Step 101: After obtaining the product information vector, the product information vector is connected to a fully connected layer so that the product vector inputs the vector dimension of the product encoding module and the user query vector outputs the vector dimension of the decoding module.

[0083] ; Step 102: Perform self-attention encoding on the dimensionally unified product information, then connect it to the dropout layer of the neural network, perform residual connection on the output result and the dimensionally unified product information, and perform regularization processing: ; ; Step 103: Connect the attention output results to two fully connected layers to obtain high-dimensional features: ; ; Repeat steps 102 and 103 n times to obtain the product code output as follows: ; Step 201, connect the user information vector through a fully connected layer so that the dimension of the user vector input into the user query module is the same as the dimension of the vector of the product coding module; ; Step 202, the dimensionally unified user information is self-attention encoded, and then connected to the neural network dropout layer, the output result is residually connected with the dimensionally unified user information, and regularized: ; ; Step 203, cross-attend fusion of the encoded user information and product information to extract relevant content related to the user in the product: ; ; Step 204: Connect the attention output results to two fully connected layers to obtain high-dimensional features: ; ; Repeat steps 202, 203, and 204 n times to obtain the final output.

[0084] After constructing the vector, the query process is implemented based on the vector interface. The specific query process can be as follows: Figure 7 The vector interface query process provided by the present invention is shown in the schematic diagram.

[0085] The interface input parameters include data screening condition information, input questions, and the user's own user profile information.

[0086] Use the input question as the input parameter, call the embedding engine, obtain the vectorized input question information, concatenate the user dimensions, and obtain the user query vector.

[0087] According to the filtering conditions, the data that meets the conditions in the vectorized database is filtered.

[0088] For all the product data dimension vectors in the product library, the vectors corresponding to the input questions are output through the model, and the output results are compared with the user vector to calculate the similarity score.

[0089] Specifically, the matching process based on similarity scoring and the process of outputting the matching result list can be as follows: Figure 8 The matching algorithm provided by the present invention is implemented as shown in the flowchart. Among them, GoodVectors is the dimension vector of all commodity data in the commodity library, and UserQueryVector is the vector corresponding to the input question.

[0090] The calculation formula of similarity score is as follows: ; ; The similarity score calculation formula is Similarity, which ensures that the similarity score is between 0 and 1. x is the user vector data, and y is the model output vector information.

[0091] Return in order of rating.

[0092] In order to meet the needs of interface calls and set different json output parameter formats according to requirements, the interfaces need to be uniformly encapsulated.

[0093] The information encapsulated by the interface tool module is the request method: http (get, post), websocket, SSE; request information: request header, request body; output parameter format definition: output parameter conditions, output parameter format definition. For the request body, the system needs to configure the request body sample information. Use jsonpath to define the fields to be modified.

[0094] The interface call encapsulation process can be as follows Fig. 9 The interface call encapsulation process provided by the present invention is shown in the schematic diagram.

[0095] The function tool interface request body input parameters are the toolcode field and the params structure. toolcode is the function encoding information, and params is the tool call input parameter.

[0096] Get the defined tool interface information according to toolcode.

[0097] Replace the request header fields with params information.

[0098] Initiate encapsulation request parameters according to different request types: HTTP, SSE type get request: according to the get request parameters defined by the interface, use params to replace the get request information; HTTP, SSE type post request: Get the JSON request body information and the JSON field to be replaced defined by the function, and use params to replace the corresponding field information; Websocket type request: According to the websocket connection request parameter information defined by the interface, use params to replace, establish a ws connection, obtain the websocket json request body information and the json field to be replaced defined by the function, and use params to replace the corresponding field information Initiate an actual request and determine the return type. If it is a streaming return, process each piece of information separately. If it is a full return, process it directly: perform branch judgment (the actual call branch condition judgment can be configured using Lua, or the Python eval function can be used to dynamically execute the judgment condition), obtain the encapsulated input parameter information, encapsulate the output parameter structure, and return it.

[0099] Interface output parameter encapsulation requires text information replacement and array information replacement.

[0100] Among them, the text information replacement process can be as follows Fig.10 The text information replacement process provided by the present invention is shown in the schematic diagram.

[0101] Step 1: The method input information is the original data json and the reconstructed json template information.

[0102] Step 2: Recursively determine the type of information in the reconstructed json template: jsonarry: traverse the json array, get the json array elements, and repeat step 2; jsonObject: traverse jsonkey, get jsonValue, and repeat step 2; str / other types: Determine whether jsonpath information is included and jsonpath does not contain wildcards to obtain arrays, use jsonpath to obtain information from the original data json, and replace Step 3: Output the replaced result json.

[0103] The array information replacement process can be as follows Fig.11 The array information replacement process provided by the present invention is shown in the schematic diagram.

[0104] Receiving input information: Receive original data json, which contains array information to be processed; receive reconstructed JSON template information, which defines the rules and target structure for replacing the array information.

[0105] Recursive processing and data type identification: Recursively traverse the original data json to determine whether there is an object of data type JSONArray. For each JSONArray, determine whether its length is 1 and whether its content contains a wildcard for indicating how to obtain the array.

[0106] Array information replacement: Use the wildcard information in JSONArray to generate a new string array or JSONObject array, and replace the original JSONArray with the newly generated array to achieve structured conversion of data.

[0107] Output the replaced json information.

[0108] In one embodiment, the process of constructing a vector library includes: obtaining product information in the product library, vectorizing the product information corresponding to the input questions of each historical user, and obtaining vector information corresponding to multiple product information; associating each product information with its corresponding vector information to obtain associated data, and constructing a vector library based on the associated data.

[0109] Collect historical user input questions. These questions can come from users' online shopping records, customer service conversation records, forum discussions, or feedback surveys. For each user input question, you need to collect the relevant product information. Based on the collected product information, build a product library.

[0110] Selecting a suitable pre-trained language model for vectorization processing can convert text into a vector representation of fixed dimension and retain its semantic information.

[0111] Each product information is associated with its corresponding vector information to obtain associated data, and a vector library is constructed based on the generated associated data.

[0112] In one embodiment, intent recognition is performed on an input question to determine intent information of the input question, including: extracting intent keywords from the input question based on keyword matching; calling a large language model to perform intent recognition on the intent keywords to determine intent information of the input question.

[0113] Optionally, keyword matching can be implemented based on regular expression matching or full text matching. The reason for setting the matching rule is to reduce the model training cost and correct the model's incorrect intent recognition. If there is no subdivided secondary intent under the scenario intent, the intent rule is associated with the scenario intent; if there is a subdivided secondary intent under the scenario intent, the intent rule is associated with the secondary subdivided intent under the scenario intent.

[0114] Full-text matching returns the intent information corresponding to the rule after the input question completely matches the text configured by the system. You can use a key-value cache, such as redis, to cache the full-text matching words and corresponding intent information to speed up system processing.

[0115] Regular expression matching is to set regular expression matching rules. If the matching rules are met, the corresponding intent information can be returned. In addition, regular expressions can be used to capture groups to capture entity information in the input question for subsequent use.

[0116] When keyword rule filtering fails to identify successfully, use the pre-defined prompt word information to call the large model and obtain the intent recognition result of the large model.

[0117] In one embodiment, after determining the answer result of the input question, it also includes: based on the intention information, determining the scenario information of the input question; based on the scenario information and the answer result, determining the display form of the answer result, and calling the display tool corresponding to the display form to display the answer result.

[0118] Scenario information refers to the context or usage background of the user's question. It includes the user's purpose, environment, situation, needs and other information, which can affect the user's expectations and understanding of the answer result.

[0119] By analyzing the questions input by users and their intention information, we can extract scene-related features. For example, users may ask questions in different scenarios such as shopping, consulting, and learning.

[0120] Different scenarios may require different display formats. Simple answers or suggestions can be displayed directly in text form. If there are multiple product recommendations, they can be displayed in a list or card format, with each item containing basic information and a picture. For responses that require data visualization, such as statistical data or trend analysis, charts are more intuitive.

[0121] According to the determined display form, the corresponding display tool is called. It can be personalized according to user preferences, which can significantly improve user satisfaction and system interaction efficiency.

[0122] In one embodiment, it also includes: when, based on the intent information, it is determined that the intent processing flow for processing the input question is a simple intent flow, based on calling pre-configured reply information, determining the answer result of the input question, and the simple intent flow is used to characterize that the intent information only contains a single intent.

[0123] In common query scenarios, users often enter simple questions that can be quickly responded to through a brief intent flow. A brief intent flow refers to a question that only expresses a clear intent. This intent is usually direct, and users expect a simple and quick response.

[0124] The response information related to each single intent needs to be pre-configured during the design phase. This information can be static text, dynamically generated content, or templates to ensure that it can be quickly called when processing brief intents. Build an information library that contains preset responses for various intents.

[0125] After identifying a single intent, the system quickly retrieves the preset response information related to the intent, generates the response result, and prepares to display it to the user.

[0126] The present invention also provides a flowchart of a problem processing method based on a large language model provided by the present invention, such as Fig.12 As shown, specifically including: When the input question is entered, the scene distinction is first performed on the input question to determine the scene to which the input question belongs. The intent management module is used to confirm all intent scenarios and the processing module to which the intent belongs. For example, in the field of mobile communications, there may be terminal purchase scenarios, bill query scenarios, and phone recharge scenarios. Through the AI ​​intelligent management module, it is possible to interact with the large model to identify the input question. First, the input question and user interaction history information are used as placeholders, and the prompt word information corresponding to the intent judgment is configured in the AI ​​intelligent management module, and the recognized intent is obtained by calling the large model. If there is a related text classification model, the recognized intent can also be obtained through the related calling method of this module.

[0127] According to the identified intent information, the corresponding intent processing flow is obtained, which mainly includes simple flow, agent flow, and orchestration flow. Through the agent flow and orchestration flow, the system can use the interface encapsulated by the tool module to obtain external system information in a unified way to assist system decision-making.

[0128] For interface information, especially the interface of the external system, the returned data structure cannot meet the needs of the business system. Therefore, a tool management module is set up to manage the external interface, encapsulate the call input parameters, and set a new output parameter format.

[0129] In terms of knowledge base, industry knowledge information, especially product knowledge information, not only has strong screening properties, such as products are only available in a certain area, product price screening, etc., but also the dimensions of the products occupy different sorting weights. For example, the weight of the product name vector retrieval score is higher than the weight of the product details vector retrieval score. Therefore, in the construction of the vector engine, it is necessary to generate vectors of different dimensions for the same product information. At the same time, in terms of vector retrieval, different screening weights can be set for vectors of different dimensions to meet the needs.

[0130] The following is a description of the problem processing device based on a large language model provided by the present invention. The problem processing device based on a large language model described below and the problem processing method based on a large language model described above can be referenced to each other.

[0131] like Fig.13 As shown, the device comprises: The intention recognition module 1310 is used to recognize the intention of the input question, determine the intention information of the input question, and determine the intention processing flow corresponding to the intention information, wherein the intention processing flow at least includes an agent flow and an orchestration flow, wherein the agent flow is used to represent that the intention processing flow is applicable to the process of the agent large language model processing, and the orchestration flow is used to represent that the intention processing flow is applicable to a predefined and orchestrated processing flow; A first reply module 1320 is used to determine, when determining that the intention processing process is an agent process, a target function interface to be called from all function interfaces included in the business based on the intention information, and determine a reply result to the input question based on the agent large language model and parameter information extracted from the target function interface, wherein the function interface is constructed based on the solution parameters corresponding to the question; The second reply module 1330 is used to determine the processing flow corresponding to the intent information when it is determined that the intention processing flow is a process arrangement, and solve the input question based on the processing flow corresponding to the intent information to determine the reply result of the input question.

[0132] The problem processing device based on the large language model provided by the present invention can effectively improve the response accuracy and efficiency of the system by identifying the intent of the input question and selecting the appropriate processing flow. The intelligent process is suitable for quickly processing simple queries, while the orchestration process can cope with more complex requests and combine various information to meet user needs. Through multi-level understanding, process selection, information retrieval and decomposition processing, the accuracy and relevance of the output results of the large language model are effectively improved. This structured processing method enables the system to generate high-quality answers that better meet user needs, thereby greatly improving the user experience.

[0133] In one embodiment, the second reply module 1330 is specifically configured to: Solving the input question based on the processing flow corresponding to the intention information and determining a response result for the input question includes: In the case where it is determined that the processing flow corresponding to the intention information is to determine product information, based on cosine similarity, the vector corresponding to the input question is matched in a pre-built vector library, and a matching result list is output, wherein the vector library is built based on the product information in the product library; Based on the large language model, the product information in the matching result list is supplemented to obtain a reply result.

[0134] In one embodiment, the second reply module 1330 is further specifically configured to: The construction process of the vector library includes: Obtain product information in the product library, and perform vector processing on the product information corresponding to the input questions of each historical user, to obtain vector information corresponding to multiple product information; Based on associating each commodity information with its corresponding vector information, associated data is obtained, and a vector library is constructed based on the associated data.

[0135] In one embodiment, the intention recognition module 1310 is specifically used to: Performing intent recognition on the input question to determine the intent information of the input question includes: Extracting intended keywords from the input question based on keyword matching; The large language model is called to perform intent recognition on the intent keywords to determine the intent information of the input question.

[0136] In one embodiment, the second reply module 1330 is further specifically configured to: After determining the answer result of the input question, it also includes: Based on the intention information, determining scenario information of the input question; Based on the scenario information and the reply result, the display form of the reply result is determined, and a display tool corresponding to the display form is called to display the reply result.

[0137] In one embodiment, the intention recognition module 1310 is further specifically used for: When it is determined that the intent processing flow for processing the input question is a simplified intent flow based on the intent information, the answer result of the input question is determined based on calling the pre-configured reply information, and the simplified intent flow is used to characterize that the intent information contains only a single intent.

[0138] Fig.14 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.14 As shown, the electronic device may include: a processor 1410, a communication interface 1420, a memory 1430 and a communication bus 1440, wherein the processor 1410, the communication interface 1420 and the memory 1430 communicate with each other through the communication bus 1440. The processor 1410 may call the logic instructions in the memory 1430 to execute a problem processing method based on a large language model, the method comprising: performing intent recognition on an input question, determining intent information of the input question, and determining an intent processing flow corresponding to processing the intent information, the intent processing flow at least comprising an agent flow and an orchestration flow, the agent flow being used to characterize that the intent processing flow is applicable to a process processed by an agent large language model, and the orchestration flow being used to characterize that the intent processing flow is applicable to a predefined and orchestrated processing flow; In the case where it is determined that the intention processing process is an agent process, based on the intention information, a target function interface to be called is determined from all function interfaces included in the business, and based on the agent large language model and parameter information extracted from the target function interface, a response result to the input question is determined, and the function interface is constructed based on the solution parameters corresponding to the question; When it is determined that the intention processing flow is a process orchestration, the processing flow corresponding to the intention information is determined, and based on the processing flow corresponding to the intention information, the input question is solved to determine the answer result of the input question.

[0139] In addition, the logic instructions in the above-mentioned memory 1430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0140] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the problem processing method based on the large language model provided by the above methods, the method includes: performing intent recognition on an input question, determining intent information of the input question, and determining an intent processing flow corresponding to the intent information, the intent processing flow includes at least an agent flow and an orchestration flow, the agent flow is used to characterize that the intent processing flow is applicable to a process for agent large language model processing, and the orchestration flow is used to characterize that the intent processing flow is applicable to a predefined and orchestrated processing flow; In the case where it is determined that the intention processing process is an agent process, based on the intention information, a target function interface to be called is determined from all function interfaces included in the business, and based on the agent large language model and parameter information extracted from the target function interface, a response result to the input question is determined, and the function interface is constructed based on the solution parameters corresponding to the question; When it is determined that the intention processing flow is a process orchestration, the processing flow corresponding to the intention information is determined, and based on the processing flow corresponding to the intention information, the input question is solved to determine the answer result of the input question.

[0141] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the problem processing method based on the large language model provided by the above methods, the method comprising: performing intent recognition on an input question, determining intent information of the input question, and determining an intent processing flow corresponding to the intent information, the intent processing flow at least comprising an agent flow and an orchestration flow, the agent flow being used to characterize that the intent processing flow is applicable to a flow for agent large language model processing, and the orchestration flow being used to characterize that the intent processing flow is applicable to a predefined and orchestrated processing flow; In the case where it is determined that the intention processing process is an agent process, based on the intention information, a target function interface to be called is determined from all function interfaces included in the business, and based on the agent large language model and parameter information extracted from the target function interface, a response result to the input question is determined, and the function interface is constructed based on the solution parameters corresponding to the question; When it is determined that the intention processing flow is a process orchestration, the processing flow corresponding to the intention information is determined, and based on the processing flow corresponding to the intention information, the input question is solved to determine the answer result of the input question.

[0142] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A problem processing method based on a large language model, characterized in that: The method comprises: Performing intent recognition on the input question, determining the intent information of the input question, and determining the intent processing flow corresponding to processing the intent information, wherein the intent processing flow at least includes an agent flow and an orchestration flow, wherein the agent flow is used to characterize that the intent processing flow is applicable to a flow for processing by an agent large language model, and the orchestration flow is used to characterize that the intent processing flow is applicable to a predefined and orchestrated processing flow; In the case where it is determined that the intention processing process is an agent process, based on the intention information, a target function interface to be called is determined from all function interfaces included in the business, and based on the agent large language model and parameter information extracted from the target function interface, a response result to the input question is determined, and the function interface is constructed based on the solution parameters corresponding to the question; When it is determined that the intention processing flow is a process orchestration, the processing flow corresponding to the intention information is determined, and based on the processing flow corresponding to the intention information, the input question is solved to determine the answer result of the input question.

2. The problem processing method based on a large language model according to claim 1, characterized in that: The step of solving the input question based on the processing flow corresponding to the intention information and determining the answer result of the input question includes: In the case where it is determined that the processing flow corresponding to the intention information is to determine product information, based on cosine similarity, the vector corresponding to the input question is matched in a pre-built vector library, and a matching result list is output, wherein the vector library is built based on the product information in the product library; Based on the large language model, the product information in the matching result list is supplemented to obtain a reply result.

3. The problem processing method based on a large language model according to claim 2 is characterized in that: The construction process of the vector library includes: Obtain product information in the product library, and perform vector processing on the product information corresponding to the input questions of each historical user, to obtain vector information corresponding to multiple product information; Based on associating each commodity information with its corresponding vector information, associated data is obtained, and a vector library is constructed based on the associated data.

4. The problem processing method based on a large language model according to claim 1, characterized in that: The performing intention recognition on the input question to determine the intention information of the input question includes: Extracting intended keywords from the input question based on keyword matching; The large language model is called to perform intent recognition on the intent keywords to determine the intent information of the input question.

5. The problem processing method based on a large language model according to claim 1, characterized in that: After determining the answer result of the input question, it also includes: Based on the intention information, determining scenario information of the input question; Based on the scenario information and the reply result, the display form of the reply result is determined, and a display tool corresponding to the display form is called to display the reply result.

6. The problem processing method based on a large language model according to claim 1, characterized in that: Also includes: When it is determined that the intent processing flow for processing the input question is a simplified intent flow based on the intent information, the answer result of the input question is determined based on calling the pre-configured reply information, and the simplified intent flow is used to characterize that the intent information contains only a single intent.

7. A problem processing device based on a large language model, characterized in that: include: An intention recognition module is used to recognize the intention of the input question, determine the intention information of the input question, and determine the intention processing flow corresponding to the intention information, wherein the intention processing flow at least includes an agent flow and an orchestration flow, wherein the agent flow is used to characterize that the intention processing flow is applicable to a flow processed by an agent large language model, and the orchestration flow is used to characterize that the intention processing flow is applicable to a predefined and orchestrated processing flow; A first reply module is used to determine, when determining that the intention processing process is an agent process, a target function interface to be called from all function interfaces included in the business based on the intention information, and to determine a reply result to the input question based on the agent large language model and parameter information extracted from the target function interface, wherein the function interface is constructed based on solution parameters corresponding to the question; The second reply module is used to determine the processing flow corresponding to the intention information when it is determined that the intention processing flow is a process arrangement, and solve the input question based on the processing flow corresponding to the intention information to determine the reply result of the input question.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the problem processing method based on the large language model as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the problem processing method based on a large language model as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the problem processing method based on a large language model as described in any one of claims 1 to 6 is implemented.

Images