Service method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a service method and device, equipment, a storage medium and a program product, and relates to the field of big data. The method comprises the steps of obtaining a query text of a user and determining a corresponding first query vector; corresponding query result information is determined in a multi-modal database according to the first query vector, the multi-modal database comprises a vector database constructed according to vectors corresponding to the unstructured data and a knowledge fragment library constructed according to the structured data, and the query result information comprises result vectors and result knowledge fragments; according to the query result information and historical query information, corresponding first reply information is generated through a natural language generation model, and the historical query information comprises a query text. According to the method, the capability of processing complex interaction is improved.

Description

Technical Field

[0001] This application relates to the field of big data, and more particularly to a service method, apparatus, device, storage medium, and program product. Background Technology

[0002] In modern banking, customers increasingly need to interact with banks through mobile applications, telephone customer service, or web-based services.

[0003] In existing technologies, users navigate through preset menus using telephone keypads or simple voice commands (such as "check balance" or "transfer to a human operator"). Bank service systems, based on fixed processes and keyword matching logic, can only handle predefined, simple commands and cannot cope with complex, conversational, or multi-meaning expressions.

[0004] Therefore, there is an urgent need for a service method that can realize complex interactive services. Summary of the Invention

[0005] This application provides a service method, apparatus, device, storage medium, and product to solve technical problems related to the inability to handle complex interactive communication.

[0006] In a first aspect, this application provides a service method, comprising: acquiring a user's query text and determining a corresponding first query vector; determining corresponding query result information in a multimodal database based on the first query vector, the multimodal database including a vector database constructed based on vectors corresponding to unstructured data and a knowledge fragment database constructed based on structured data, the query result information including result vectors and result knowledge fragments; and generating corresponding first response information through a natural language generation model based on the query result information and historical query information, the historical query information including the query text.

[0007] In one possible implementation of the first aspect, determining the corresponding query result information in a multimodal database based on the first query vector includes: determining the corresponding result vector in a vector database based on the first query vector; and determining the corresponding result knowledge fragment in a knowledge fragment base based on the query text.

[0008] In one possible implementation of the first aspect, determining the corresponding result vector in the vector database based on the first query vector includes: determining the corresponding result vector in the vector database based on the similarity between the first query vector and each vector in the vector database, wherein the result vector is the K vectors with the highest similarity, or the N vectors with a similarity to the first query vector exceeding a similarity threshold, where K and N are positive integers greater than 1.

[0009] In one possible implementation of the first aspect, the method further includes: determining the corresponding user intent based on the query text; and if the user intent is to conduct business, calling the target business execution interface to perform the corresponding target business operation.

[0010] In one possible implementation of the first aspect, the method further includes: receiving sensitive information input by the user through a secure input component and verifying the user's identity through a biometric authentication trigger mechanism.

[0011] In one possible implementation of the first aspect, the method further includes: obtaining the business execution result corresponding to the target business operation; and generating the corresponding second response information based on the business execution result using a natural language generation model.

[0012] In one possible implementation of the first aspect, the method further includes: maintaining session state information, which includes user identity information, historical dialogue content, and target business progress; and restoring the business process of the target business based on the session state information when the second query text is related to the target business.

[0013] In one possible implementation of the first aspect, the method further includes: packaging the current session state into a JSON object when the confidence level of the response information generated by the natural language generation model is lower than the confidence level threshold; and pushing the JSON object to the human agent queue.

[0014] In one possible implementation of the first aspect, the method further includes: after detecting a data change in the business system, updating the vector database and / or knowledge fragment library based on the changed data.

[0015] Secondly, this application provides a service apparatus, comprising:

[0016] The acquisition module is used to acquire the user's query text and determine the corresponding first query vector;

[0017] The determination module is used to determine the corresponding query result information in the multimodal database based on the first query vector. The multimodal database includes a vector database constructed based on vectors corresponding to unstructured data and a knowledge fragment database constructed based on structured data. The query result information includes result vectors and result knowledge fragments. Based on the query result information and historical query information, the module generates the corresponding first response information through a natural language generation model. The historical query information includes query text.

[0018] Thirdly, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor;

[0019] The memory stores instructions that the computer executes;

[0020] The processor executes computer-executable instructions stored in memory to implement any of the methods of the first aspect.

[0021] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method of any one of the first aspects.

[0022] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of any one of the first aspects.

[0023] The service method provided in this application, through the combination of enhanced retrieval generation technology and a multimodal knowledge base, can utilize the bank's comprehensive and up-to-date knowledge, providing answers with a scope far exceeding that of traditional chatbots, and ensuring that all answers are factually based, thus solving the problems of knowledge limitations and inaccurate information. Furthermore, it integrates queries and transactions into a unified dialogue interface, eliminating the need for users to switch between different functions, achieving a "say it and get it done" one-stop service and greatly enhancing the user experience. Simultaneously, based on natural language generation models and context awareness capabilities, the system can conduct truly multi-turn dialogues, handling complex interactions such as interruptions and follow-up questions, further improving the user experience. Moreover, a "conversation state" packaging and transfer mechanism enables seamless human-machine interaction. Human agents can quickly intervene and resolve issues without requiring repeated user intervention, significantly improving overall service efficiency. Furthermore, knowledge updates to the knowledge base are primarily accomplished by adding and deleting documents, without modifying complex program code or rules, reducing system maintenance costs and iteration difficulty. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] Figure 1 A schematic diagram illustrating a service method provided in an embodiment of this application;

[0026] Figure 2 A flowchart illustrating a service method provided in an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of the structure of a service device provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0029] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0032] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0033] It should be noted that the methods, apparatus, equipment, storage media and products of the services provided in this application can be used in the field of big data, or in any field other than big data. The application fields of the methods, apparatus, equipment, storage media and products of the services in this application are not limited.

[0034] The following explains the relevant technical data mentioned in this application:

[0035] Large language model (LLM): refers to a deep learning model pre-trained on massive amounts of text data, possessing powerful natural language understanding, generation, reasoning, and knowledge memorization capabilities. In this solution, it specifically refers to an LLM fine-tuned and optimized for the banking business domain.

[0036] Retrieval-augmented generation (RAG) is a technical framework that combines information retrieval with the generative capabilities of an LLM (Local Management Module). It first retrieves the most relevant information fragments to the user's question from an external knowledge base, then provides these fragments as context to the LLM, guiding it to generate more accurate and factually based answers.

[0037] A multi-modal knowledge base is an integrated knowledge storage system capable of managing and processing different types of data simultaneously. It mainly includes:

[0038] Structured data: such as customer information, account balance, transaction records, product interest rates, etc. in database tables.

[0039] Unstructured data: such as bank policy documents, product manuals, operating procedures, marketing scripts, and frequently asked questions in formats such as PDF, Word, and web pages.

[0040] Context awareness: refers to the system's ability to remember and understand previous dialogue content (context) in continuous multi-turn dialogues, and to use this information to accurately understand the user's current and subsequent instructions and intentions.

[0041] Session state: A structured data packet used to fully record all key information of a single user session. It includes at least the user's identity, complete historical dialogue records, user intent identified by the system, currently processed business and its progress, and key responses generated by the system.

[0042] Figure 1 This is a schematic diagram illustrating a scenario where the risk identification method of this application is applied. For example... Figure 1 As shown, in a financial scenario, customers carry electronic devices that can interact and converse with them. These devices can execute the service methods described in this application to provide interactive dialogue services to users. The electronic devices can also interact with the bank's server to obtain relevant data needed to execute the service methods described in this application. In modern banking, the demand for customers to interact with banks through mobile applications, telephone customer service, or web pages is increasing.

[0043] In existing technologies, users navigate through preset menus using telephone keypads or simple voice commands (such as "check balance" or "transfer to a human operator"). Bank service systems, based on fixed processes and keyword matching logic, can only handle predefined, simple commands and cannot cope with complex, conversational, and multi-meaning expressions. Therefore, there is an urgent need for a service method capable of providing complex interactive services.

[0044] The service method provided in this application is intended to solve the above-mentioned technical problems of the prior art.

[0045] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0046] Figure 2 This is a flowchart illustrating a service method provided in an embodiment of this application, such as... Figure 2 As shown, the method includes:

[0047] S201. The electronic device obtains the user's query text and determines the corresponding first query vector.

[0048] After obtaining the user's query text, the electronic device can first extract the semantic feature information corresponding to the query text. For example, it can extract keywords, intent and context information from the user's query text through natural language processing technology, such as semantic feature information such as "steady profit", "rate of return" and "financial management".

[0049] The electronic device then converts these semantic feature information into a numerical vector representation, thereby determining the corresponding first query vector based on the user's query text.

[0050] S202. The electronic device determines the corresponding query result information in the multimodal database based on the first query vector.

[0051] The electronic device determines the corresponding query result information in a multimodal database based on the first query vector. This multimodal database may include a vector database and a knowledge fragment database. The vector database is constructed based on vectors corresponding to unstructured data, meaning it includes multiple vectors. The knowledge fragment database is constructed based on structured data, meaning it includes multiple knowledge fragments.

[0052] The electronic device can determine at least one corresponding vector as a result vector from a vector database based on the first query vector. The electronic device can also determine at least one corresponding knowledge fragment as a result knowledge fragment from a knowledge fragment database based on the first query vector. Then, the electronic device establishes the association between the result vector and the result knowledge fragment, determining that the result vector and the result knowledge fragment constitute the query result information; that is, the query result information includes both the result vector and the result knowledge fragment.

[0053] In a possible implementation, the electronic device determines the corresponding query result information in the database according to the first query vector. Specifically, it can first determine the result vector corresponding to the first query vector in the vector database according to the first query vector. Then, it determines the corresponding result knowledge fragment in the knowledge fragment library according to the query text. Thus, it determines that the result vector and the result knowledge fragment are the query result information.

[0054] In a possible implementation, the electronic device determines the corresponding result vector in the vector database according to the first query vector. Specifically, it can determine the top K vectors with the highest similarity in the vector database as the corresponding result vectors according to the similarity between the first query vector and each vector in the vector database, or determine the top N vectors with a similarity exceeding the similarity threshold as the corresponding result vectors. K and N are positive integers greater than 1.

[0055] The similarity threshold is a similarity demarcation value set according to business requirements and is used to filter out irrelevant vectors. For example, the threshold is set to 0.8, and only vectors with a similarity ≥ 0.8 are retained.

[0056] The vector similarity calculation in the embodiments of this application refers to quantifying the matching degree between the query vector and the vectors in the knowledge base through the vector inner product or cosine similarity algorithm. For example, when the user inputs "Robust Profit Financial Management Yield Rate", the system calculates the similarity between its vector and the vector of "Robust Profit Product Manual".

[0057] Specifically, when the electronic device calculates the similarity between the first query vector and each vector in the vector database, it can be determined by calculating the cosine angle degree between the two vectors. The smaller the cosine angle degree, the higher the similarity between the two vectors, and the larger the cosine angle degree, the lower the similarity between the two vectors. In addition, it can also be determined by other methods, such as calculating the dot product, etc. Specifically, it is not limited here.

[0058] In this possible implementation, the retrieval process is further refined through similarity screening and relevance matching, solving the problem of possible noise interference in the original retrieval. The similarity calculation ensures that only highly relevant unstructured data is retained, while the relevance matching avoids the disconnection between the retrieval results and the business scenario through the logical binding of structured data. Through the dual filtering mechanism, the accuracy and business relevance of the retrieval results are improved, ensuring the accuracy of the generated response.

[0059] S203. The electronic device generates the corresponding first response information through the natural language generation model according to the query result information and the historical query information.

[0060] Electronic devices can input query results and historical query information into a natural language generation model to generate a corresponding first response, which is a natural language response. The historical query information includes the query text.

[0061] This historical query information is contextual data of multiple rounds of user-system interactions recorded by electronic devices, such as user identification, identified intents, and system response information.

[0062] In this embodiment, the core of the electronic device's function module for generating response information is a Large Language Model (LLM) fine-tuned from a banking corpus. This module also includes a Natural Language Understanding (NLU) unit, a dialogue state tracker, and a response generator. It can receive user input and identify user intent through the NLU unit. The core workflow employs retrieval-enhanced generation technology: when a knowledge-based question needs to be answered, it first retrieves the most relevant text fragments from the vector database of the multimodal knowledge base module based on the user's question. These fragments, along with the dialogue history, are then fed as prompts to the LLM, which generates the final natural language response. The dialogue state tracker is responsible for maintaining the session state throughout the process.

[0063] The natural language generation model in this application embodiment can be a large language model (LLM) based on the Transformer architecture, or a neural network model or other models, and is not limited here.

[0064] In one possible implementation, the user queries a text message asking how to handle a related service. In this case, the electronic device can call the service execution interface to perform the service operation. Optionally, the method further includes the following steps:

[0065] S204. When the user's intention is to conduct business, the electronic device calls the target business execution interface to perform the corresponding target business operation.

[0066] The electronic device first determines the corresponding user intent based on the query text. If the user intent is to conduct business, the electronic device can call the target business execution interface to perform the corresponding target business operation.

[0067] For example, electronic devices can generate corresponding business execution requests based on user intent and business progress, call the core business system interface of the bank, and execute the corresponding target business operation.

[0068] The business execution request can be a structured instruction containing user intent and business parameters. For example, "Transfer 500 yuan to Zhang San, receiving bank is China Construction Bank." The core business system interface of the bank can be an application programming interface (API) provided by the bank's backend system (such as the account system and payment system). For example, calling the "transfer API" to complete the fund transfer.

[0069] In this embodiment, the functional module for performing business operations on the electronic device can be a secure API gateway and a series of business logic processors. This module is activated when the dialogue engine recognizes that the user's intent is to conduct business (such as transferring funds, querying transaction details, or purchasing financial products). Through the API gateway, after strict identity authentication and permission verification, it securely calls the interfaces of the bank's backend core business systems (account systems, payment systems, etc.), executes the user's instructions, and returns the results.

[0070] This possible implementation achieves seamless integration of information retrieval and business processing by combining business intent recognition with business execution interfaces. For example, after checking their credit card statement, a user can directly apply for installment payments through the chat interface. The system calls the core system interface to complete the operation without needing to navigate to the software's installment page. This solves the problems of poor user experience and long operation paths caused by fragmented interfaces in existing technologies.

[0071] In one possible implementation, the electronic device can call the target business execution interface to execute the corresponding target business operation. It can receive sensitive information input by the user through a secure input component and verify the user's identity through a biometric authentication trigger mechanism.

[0072] Secure input components refer to modules that invoke the operating system-level secure keyboard to prevent the theft of sensitive information. For example, when a user enters a bank card number, the electronic device can invoke Android's Secret Mode keyboard to prevent middleware from stealing information.

[0073] For example, before critical business nodes (such as transfer confirmation), electronic devices can force users to verify their identity via fingerprint or facial ID based on biometric authentication triggering mechanisms. For instance, after a user enters the transfer amount, the system requires fingerprint verification to ensure the legitimacy of the operation.

[0074] In addition, other secure input components or other biometric authentication triggering mechanisms can be used to verify user identity; specific details are not limited here.

[0075] This possible implementation addresses the issues of sensitive information leakage and unauthorized operations through secure input components and biometric authentication mechanisms. For example, when a user enters their bank card number, the system invokes a secure keyboard to prevent middleware from stealing information, while biometric authentication ensures the legitimacy of the operation, significantly improving the security of business transactions.

[0076] S205. The electronic device generates the corresponding second response information based on the business execution result through a natural language generation model.

[0077] Electronic devices can first obtain the business execution result corresponding to the target business operation; then, based on the business execution result, they can generate a corresponding second response message through a natural language generation model. The business execution result refers to the processing status returned by the interface of the bank's core business system. For example, "transfer successful" or "insufficient balance".

[0078] For example, an electronic device can first obtain the business execution result returned by the interface of the bank's core business system, such as "transfer successful". Then, the business execution result is converted into a natural language response using a natural language generation model and fed back to the user. For example, "Transfer of 500 yuan to Zhang San successful".

[0079] In this possible implementation, a service loop is achieved through business result feedback, ensuring users receive complete business processing information. Natural language conversion of business execution results avoids the direct output of technical jargon, improving user comprehension. The result feedback mechanism further optimizes the user experience, ensuring the integrity and transparency of the service process.

[0080] In one possible implementation, after completing a portion of the target business, the user may switch between querying information and processing business during their session with the electronic device; therefore, the electronic device needs to resume the business process of the target business. Optionally, the method further includes the following steps:

[0081] S206. The business process of electronic devices restoring the target service based on session state information.

[0082] Electronic devices can first maintain session state information, which includes user identity information, historical dialogue content, and the progress of the target service. Then, if the second query text is relevant to the target service, the business process of the target service can be restored based on the session state information.

[0083] This possible implementation addresses the issue of process fragmentation in multi-turn dialogues by leveraging session state information and business interruption recovery. The session state package ensures the system can access complete context information at any time, while the interruption recovery mechanism prevents users from repeating operations through state backtracking. The context memory mechanism further enhances the coherence of interaction, reduces the user's operational burden, and thus supports a natural dialogue experience in complex business scenarios.

[0084] In one possible implementation, if the user is dissatisfied with the session between the electronic device and the user, the call can be transferred to a human agent. Optionally, the method further includes the following steps:

[0085] S207. If the confidence level of the reply information is lower than the confidence level threshold, the electronic device will package the current session state into a JSON object and push it to the human agent queue.

[0086] If the confidence level of the response generated by the natural language generation model is lower than a confidence threshold, the electronic device can package the current session state into a JSON object and then push this JSON object to the human agent queue. The confidence level refers to the natural language generation model's assessment of the accuracy of the response content, typically represented by a value between 0 and 1. For example, when the system generates "The return rate of the stable and profitable financial product is 3.85%", the confidence level is 0.95. This human agent queue can be grouped according to question type or business area for human-to-human communication sessions with users.

[0087] In this embodiment, the electronic device status packaging and push function module may include a status packager and an intelligent routing unit. When a transfer condition is triggered (such as the LLM confidence level falling below a threshold, a user's explicit request, or the triggering of a specific risk control rule), the status packager immediately encapsulates the current complete session status into a JSON object. The intelligent routing unit, based on the user's question type or business domain, pushes this JSON object, along with the session request, to the most suitable human agent queue. The human agent's client software can directly parse this object to understand the complete context.

[0088] For example, the electronic device can assess the confidence level of the generated response information. If it is lower than a preset threshold (such as 0.7), a transfer process is triggered. For instance, if a user asks, "My credit card limit has been suddenly reduced," the system generates a response with a confidence level of 0.65, triggering a transfer.

[0089] This possible implementation addresses information gaps in human-machine collaboration through confidence level checks and intelligent transfer. Confidence level checks ensure timely transfer of low-quality responses, preventing user misunderstanding; the intelligent transfer mechanism conveys complete context through session status packets, reducing repetitive inquiries from human customer service representatives. Dynamic transfer strategies further enhance the efficiency of human-machine collaboration, reduce service time, and improve user satisfaction.

[0090] In one possible implementation, the electronic device can also update the multimodal database. Optionally, the method further includes the following steps:

[0091] S208. Update the multimodal database for electronic devices.

[0092] After detecting data changes in the business system, electronic devices can update the vector database and / or knowledge fragment library in the multimodal database based on the changed data.

[0093] Data changes in a business system can refer to data updates in the bank's core business system, triggering corresponding signals on electronic devices. For example, after adjustments to product interest rates or revisions to policy documents in the bank's core business system, electronic devices can detect these changes in real time and update the multimodal database accordingly. For instance, the changed data (such as new interest rate descriptions) can be processed into structured or unstructured data vectors and then updated in the multimodal database.

[0094] The electronic device in this application embodiment may be equipped with extraction, transformation, load (ETL) tools, which can be used to periodically or in real time synchronize the latest data from various business systems of the bank to ensure the real-time data of the multimodal database.

[0095] This possible implementation addresses the knowledge lag issue caused by manual maintenance in traditional systems through data change detection and dynamic knowledge base updates. Event detection ensures that the knowledge base content remains consistent with the bank's latest business rules, while the update mechanism avoids retrieval errors due to data delays by synchronously processing structured and unstructured data. The dynamic update mechanism further enhances the system's real-time performance and reliability, reducing the error rate caused by policy or product changes.

[0096] In this embodiment, by combining enhanced retrieval generation technology with a multimodal knowledge base, the system can utilize the bank's most up-to-date knowledge to provide answers far exceeding those of traditional robots, and all answers are based on factual evidence, solving the problems of knowledge limitations and inaccurate information. Furthermore, queries and transactions can be integrated into a unified dialogue interface, eliminating the need for users to switch between different functions, achieving a "say it and get it done" one-stop service and greatly improving the user experience. Simultaneously, based on powerful LLM and context-aware capabilities, the system can conduct truly multi-turn dialogues, handling complex interactions such as interruptions and follow-up questions, further enhancing the user experience. Moreover, a "conversation state" packaging and transfer mechanism enables seamless human-machine interaction. Human agents can quickly intervene and resolve issues without requiring repeated user intervention, significantly improving overall service efficiency. Furthermore, knowledge updates to the knowledge base are primarily accomplished by adding and deleting documents, without modifying complex program code or rules, reducing system maintenance costs and iteration difficulty.

[0097] Figure 3 This is a schematic diagram of the structure of a service device provided in an embodiment of this application, such as... Figure 3 As shown, the service device 300 provided in this embodiment includes:

[0098] The acquisition module 301 is used to acquire the user's query text and determine the corresponding first query vector;

[0099] The determination module 302 is used to determine the corresponding query result information in the multimodal database based on the first query vector. The multimodal database includes a vector database constructed based on vectors corresponding to unstructured data and a knowledge fragment database constructed based on structured data. The query result information includes result vectors and result knowledge fragments. Based on the query result information and historical query information, the corresponding first response information is generated through a natural language generation model. The historical query information includes query text.

[0100] In one possible implementation, the determining module 302 is specifically used to: determine the corresponding result vector in the vector database based on the first query vector; and determine the corresponding result knowledge fragment in the knowledge fragment library based on the query text.

[0101] In one possible implementation, the determining module 302 is specifically used to: determine the corresponding result vector in the vector database based on the similarity between the first query vector and each vector in the vector database. The result vector is either the K vectors with the highest similarity or the N vectors with a similarity exceeding the similarity threshold with the first query vector, where K and N are positive integers greater than 1.

[0102] In one possible implementation, the determining module 302 is further configured to: determine the corresponding user intent based on the query text; and if the user intent is to conduct business, call the target business execution interface to perform the corresponding target business operation.

[0103] In one possible implementation, the acquisition module 301 is further configured to: receive sensitive information input by the user through a secure input component, and the determination module 302 is further configured to: verify the user's identity through a biometric authentication trigger mechanism.

[0104] In one possible implementation, the acquisition module 301 is further configured to: acquire the business execution result corresponding to the target business operation; the determination module 302 is further configured to: generate the corresponding second response information based on the business execution result using a natural language generation model.

[0105] In one possible implementation, the determining module 302 is further configured to: maintain session state information, which includes user identity information, historical dialogue content, and target business progress; and, if the second query text is related to the target business, restore the business process of the target business based on the session state information.

[0106] In one possible implementation, the determining module 302 is further configured to: package the current session state into a JSON object if the confidence level of the response information generated by the natural language generation model is lower than the confidence level threshold; and push the JSON object to the human agent queue.

[0107] In one possible implementation, the determining module 302 is further configured to: update the vector database and / or knowledge fragment library based on the changed data after detecting a data change in the business system.

[0108] The service device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0109] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 40 may include a memory 401 and a processor 402. Optionally, the electronic device may also include a transceiver 403, wherein the memory 401 and the processor 402 communicate with each other; for example, the memory 401, the processor 402 and the transceiver 403 may communicate via a communication bus 404, the memory 401 is used to store a computer program, and the processor 402 executes the computer program to implement the method of the above embodiments.

[0110] Optionally, the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps in the method embodiments disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0111] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the methods in any of the above method embodiments.

[0112] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods in any of the above method embodiments.

[0113] All or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0114] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

[0118] In this application, the term "comprising" and its variations can refer to non-limiting inclusion; the term "or" and its variations can refer to "and / or". The terms "first", "second", etc., in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0119] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0120] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0121] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0122] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0123] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0124] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0125] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0126] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0127] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A service method, characterized in that, include: Obtain the user's query text and determine the corresponding first query vector; The corresponding query result information is determined in the multimodal database based on the first query vector. The multimodal database includes a vector database constructed based on vectors corresponding to unstructured data and a knowledge fragment database constructed based on structured data. The query result information includes result vectors and result knowledge fragments. Based on the query results and historical query information, a corresponding first response is generated using a natural language generation model, wherein the historical query information includes the query text.

2. The method according to claim 1, characterized in that, The step of determining the corresponding query result information in the multimodal database based on the first query vector includes: The corresponding result vector is determined in the vector database based on the first query vector; The corresponding result knowledge fragment is determined in the knowledge fragment library based on the query text.

3. The method according to claim 2, characterized in that, Determining the corresponding result vector in the vector database based on the first query vector includes: Based on the similarity between the first query vector and each vector in the vector database, a corresponding result vector is determined in the vector database. The result vector is either the K vectors with the highest similarity or the N vectors whose similarity to the first query vector exceeds a similarity threshold, where K and N are positive integers greater than 1.

4. The method according to claim 3, characterized in that, The method further includes: Determine the corresponding user intent based on the query text; If the user's intention is to conduct business, the target business execution interface is invoked to perform the corresponding target business operation.

5. The method according to claim 4, characterized in that, The method further includes: The system receives sensitive information input by the user through a secure input component and verifies the user's identity through a biometric authentication trigger mechanism.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the business execution result corresponding to the target business operation; Based on the business execution result, a corresponding second response message is generated using a natural language generation model.

7. The method according to claim 6, characterized in that, The method further includes: Maintain session state information, which includes user identity information, historical dialogue content, and target service progress; If the second query text is related to the target service, the business process of the target service is restored based on the session state information.

8. The method according to claim 7, characterized in that, The method further includes: If the confidence level of the response information generated by the natural language generation model is lower than the confidence threshold, the current session state is packaged into a JSON object; Push the JSON object to the human agent queue.

9. The method according to claim 8, characterized in that, The method further includes: Upon detecting data changes in the business system, the vector database and / or knowledge fragment library are updated based on the changed data.

10. A service device, characterized in that, The device includes: The acquisition module is used to acquire the user's query text and determine the corresponding first query vector; The determination module is used to determine the corresponding query result information in a multimodal database based on the first query vector. The multimodal database includes a vector database constructed based on vectors corresponding to unstructured data and a knowledge fragment database constructed based on structured data. The query result information includes result vectors and result knowledge fragments. Based on the query result information and historical query information, a corresponding first response information is generated through a natural language generation model. The historical query information includes the query text.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.

Images