Multi-party joint knowledge retrieval enhancement method and device, system, medium, and equipment

Abstract

The embodiments of this specification disclose a multi-party joint knowledge retrieval enhancement method and apparatus, system, medium, and device. First, a question request initiated by a client is obtained, and the question request includes question input data; the question input data is sent to a knowledge retrieval enhancement database of multiple knowledge providers to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database; each retrieval enhancement data is decrypted and integrated to obtain knowledge enhancement context data; the knowledge enhancement context data is sent to a server deployed with a large language model to determine the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data, and the large language model. This technical solution can effectively ensure the security of the privacy data of each knowledge provider while improving the accuracy and effectiveness of the answer feedback data, and reduce the risk of privacy leakage stored in the privacy data set.

Description

Technical Field

[0001] The present specification relates to the field of computer technology, and in particular to a method and apparatus, system, medium, and device for enhancing multi-party joint knowledge retrieval. Background Art

[0002] With the rapid development of science and technology, generative large language models (LLMs) have attracted more and more attention. Large language models are deep learning models trained based on massive text data. They can generate natural language text or understand the meaning of language text. Such models can perform a variety of natural language processing (NLP) tasks, including but not limited to text classification, question answering, and dialogue. Retrieval Augmented Generation (RAG) is currently a more commonly used LLM application solution. RAG integrates retrieval technology and LLM prompt technology. Simply put, by asking questions to LLM, RAG retrieves relevant information from various data sources, and injects the retrieved information and questions into LLM prompts. LLM finally gives the answer.

[0003] In the specific practice of building large language model applications, in order to overcome the "large language model illusion" problem of the large language model itself and make full use of the large language model's understanding and abstract ability of the language, the combination of the large language model and retrieval enhancement generation technology has received widespread attention. For example, every time a user needs to ask a question to the RAG application, the RAG application first searches the knowledge database to obtain the latest knowledge data, and then integrates the LLM prompt technology and submits the fused sample to the large language model to obtain the final result.

[0004] At present, retrieval of knowledge databases generally needs to be provided by different knowledge data providers, and different knowledge data providers need to protect their own privacy data, that is, the data is not allowed to leave the domain, and often cannot be directly retrieved from each other, which causes limitations in knowledge base retrieval and reduces the accuracy and effectiveness of the output results of the large language model. Summary of the invention

[0005] In a first aspect of the embodiments of the specification, a multi-party joint knowledge retrieval enhancement method is provided, which can effectively ensure the security of the private data of each knowledge provider while improving the accuracy and effectiveness of the answer feedback data, and reduce the risk of privacy leakage stored in the private data set. The method includes:

[0006] Obtaining a question request initiated by a client, wherein the question request includes question input data;

[0007] Sending the question input data to a knowledge retrieval enhancement database corresponding to a plurality of knowledge providers to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database;

[0008] Decrypting each of the retrieval enhancement data and integrating them to obtain knowledge enhancement context data;

[0009] The knowledge enhancement context data is sent to a pre-deployed large language model to determine answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model.

[0010] Further, in some implementations, the knowledge retrieval enhancement database is a vector database, and sending the question input data to the knowledge retrieval enhancement databases corresponding to the plurality of knowledge providers to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database includes:

[0011] Determine a question retrieval vector according to the question input data;

[0012] Sending the question retrieval vector to the knowledge retrieval enhancement database, so that each of the knowledge retrieval enhancement databases matches the retrieval enhancement data based on the question retrieval vector and encrypts it through the knowledge provider; and

[0013] Receive at least one encrypted retrieval enhancement data returned by each of the knowledge providers.

[0014] Furthermore, in some embodiments, the knowledge retrieval enhancement database matches a list of retrieval enhancement vectors whose vector similarity is greater than or equal to a first similarity threshold based on the question retrieval vector, thereby determining the retrieval enhancement data according to the retrieval enhancement vector list, and the knowledge provider encrypts the retrieval enhancement data using a stored public key.

[0015] Furthermore, in some implementations, determining the question retrieval vector according to the question input data includes:

[0016] Inputting the question input data into a target feature extraction network to obtain a question retrieval vector;

[0017] Wherein, the target feature extraction network is constructed based on the feature extraction network in the large language model deployed on the server;

[0018] The retrieval enhancement vector in the knowledge retrieval enhancement database is extracted through the target feature extraction network.

[0019] Furthermore, in some implementations, decrypting each of the retrieval enhancement data includes:

[0020] Each of the retrieval enhancement data is sent to a local hardware trusted execution environment, and the retrieval enhancement data is decrypted using a private key stored in the hardware trusted execution environment to obtain the decrypted retrieval enhancement data.

[0021] Further, in some embodiments, the integration to obtain knowledge-enhanced context data includes:

[0022] Determine the semantic similarity between the decrypted retrieval enhancement data;

[0023] The retrieval enhancement data whose semantic similarity is greater than or equal to the second similarity threshold are merged to obtain knowledge enhancement context data.

[0024] Furthermore, in some embodiments, the method further comprises:

[0025] generating at least one key pair, the key pair comprising a public key and a private key;

[0026] The public key is divided and saved in each of the knowledge retrieval enhanced databases, and the private key is saved in a local hardware trusted execution environment.

[0027] Further, in some implementations, determining the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model includes:

[0028] constructing question prompt data using the knowledge enhancement context data and the question input data;

[0029] The question prompt data is input into the large language model to obtain answer feedback data corresponding to the question input data.

[0030] Further, in some embodiments, constructing question prompt data using the knowledge enhancement context data and the question input data includes:

[0031] Get the preset model question and answer prompt template;

[0032] The knowledge enhancement context data and the question input data are filled into the model question and answer prompt template to generate the question and answer prompt data.

[0033] In a second aspect of the embodiments of the specification, a multi-party joint knowledge retrieval enhancement device is also proposed, including:

[0034] A question request acquisition module, used to acquire a question request initiated by a client, wherein the question request includes question input data;

[0035] A multi-party data joint retrieval module, used for sending the question input data to a knowledge retrieval enhancement database corresponding to a plurality of knowledge providers, so as to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database;

[0036] A context data determination module, used for decrypting each of the retrieval enhancement data and integrating them to obtain knowledge enhancement context data;

[0037] The answer feedback data generation module is used to send the knowledge enhancement context data to a server deployed with a large language model, so as to determine the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model.

[0038] According to a third aspect of the embodiments of this specification, a multi-party joint knowledge retrieval enhancement system is provided, the system comprising:

[0039] A third-party joint search terminal includes the multi-party joint knowledge search enhancement device in the second aspect, and executes the multi-party joint knowledge search enhancement method in the first aspect through the multi-party joint knowledge search enhancement device;

[0040] The client is connected to the third-party joint search end for obtaining input question input data, generating a question request according to the question input data, sending the question request to the third-party joint search end, and receiving and displaying answer feedback data returned by the third-party joint search end;

[0041] At least one knowledge providing end is communicatively connected to the third-party joint retrieval end, and is used to maintain and update the knowledge retrieval enhancement database, and obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database according to the question input data sent by the third-party joint retrieval end.

[0042] Furthermore, in some embodiments, the system further comprises:

[0043] The server is deployed with a large language model and is in communication connection with the third-party joint retrieval end, and is used to receive the knowledge enhancement context data and question input data returned by the third-party joint retrieval end, and determine the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model, and return the answer feedback data to the third-party joint retrieval end.

[0044] A fourth aspect of the embodiments of this specification further provides a computer program product, which stores at least one instruction, and the at least one instruction is suitable for being loaded by a processor and executing the method steps in the first aspect.

[0045] According to a fifth aspect of the embodiments of this specification, a storage medium is provided, wherein the storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the steps of the method in the first aspect.

[0046] According to a sixth aspect of the embodiments of the present specification, there is also provided an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method in the first aspect.

[0047] In an embodiment of the present specification, a question request initiated by a client is obtained, the question request includes question input data; the question input data is sent to a knowledge retrieval enhancement database of multiple knowledge providers to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database; each retrieval enhancement data is decrypted and integrated to obtain knowledge enhancement context data; the knowledge enhancement context data is sent to a server deployed with a large language model to determine answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model. On the one hand, by sending the question input data to the knowledge retrieval enhancement database of multiple knowledge providers, at least one encrypted retrieval enhancement data is obtained, and the knowledge enhancement context data is obtained after decryption. While ensuring the security of the privacy data in the knowledge retrieval enhancement database of each knowledge provider, the retrieval enhancement data of each knowledge provider can be fully retrieved, ensuring the integrity and validity of the large language model input data, thereby improving the accuracy and validity of the answer feedback data output by the large language model; on the other hand, the retrieved retrieval enhancement data is encrypted at each knowledge provider, and decrypted at the third-party joint retrieval terminal to obtain the knowledge enhancement context data, and the knowledge enhancement context data is used as the prompt data of the large language model for retrieval enhancement generation, which can effectively ensure the security of the privacy data in the knowledge retrieval enhancement database of each knowledge provider and reduce the privacy leakage risk caused by the centralized storage of privacy data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a multi-party joint knowledge retrieval enhancement method provided in an embodiment of this specification.

[0049] Figure 2 A schematic diagram of a process for determining retrieval enhancement data based on question input data provided in an embodiment of the present specification.

[0050] Figure 3 A flowchart of constructing question and answer prompt data based on question input data and knowledge enhancement context data is provided in an embodiment of this specification.

[0051] Figure 4A flowchart of a knowledge retrieval enhancement generation method for implementing large language model question and answer based on multiple knowledge providers is provided in an embodiment of this specification.

[0052] Figure 5 A schematic diagram of the structure of a multi-party joint knowledge retrieval enhancement device provided in an embodiment of this specification.

[0053] Figure 6 A schematic diagram of the architecture of a multi-party joint knowledge retrieval enhancement system provided in an embodiment of this specification.

[0054] Figure 7 A schematic diagram of the architecture of another multi-party joint knowledge retrieval enhancement system provided in an embodiment of this specification.

[0055] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0057] See also Figure 1 As shown, a flowchart of a multi-party joint knowledge retrieval enhancement method is provided in an embodiment of this specification. In an embodiment of this specification, the multi-party joint knowledge retrieval enhancement method can be applied to a third-party joint retrieval end, and the third-party joint retrieval end or its operating environment is independent of the client and server providing the service. Specifically, the third-party joint retrieval end can be a terminal device or a server, and this example embodiment does not make special restrictions on this. Figure 1 The process shown is described in detail. The multi-party joint knowledge retrieval enhancement method in the embodiment of this specification may specifically include the following steps:

[0058] Step S110, obtaining a question request initiated by the client, wherein the question request includes question input data;

[0059] Step S120, sending the question input data to a knowledge retrieval enhancement database corresponding to a plurality of knowledge providers, so as to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database;

[0060] Step S130, decrypting each of the search enhancement data and integrating them to obtain knowledge enhancement context data;

[0061] Step S140: sending the knowledge enhancement context data to a pre-deployed large language model, so as to determine answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model.

[0062] According to the multi-party joint knowledge retrieval enhancement method in the embodiments of the present specification, on the one hand, by sending the question input data to the knowledge retrieval enhancement database of multiple knowledge providers, at least one encrypted retrieval enhancement data is obtained, and the knowledge enhancement context data is obtained after decryption. While ensuring the security of the privacy data in the knowledge retrieval enhancement database of each knowledge provider, the retrieval enhancement data of each knowledge provider can be fully retrieved, ensuring the integrity and validity of the large language model input data, thereby improving the accuracy and validity of the answer feedback data output by the large language model; on the other hand, the retrieved retrieval enhancement data is encrypted at each knowledge provider, and decrypted at the third-party joint retrieval terminal to obtain the knowledge enhancement context data, and the knowledge enhancement context data is used as the prompt data of the large language model for retrieval enhancement generation, which can effectively ensure the security of the privacy data in the knowledge retrieval enhancement database of each knowledge provider and reduce the privacy leakage risk caused by the centralized storage of privacy data.

[0063] The following is a detailed description of the multi-party joint knowledge retrieval enhancement method in the embodiments of this specification.

[0064] In step S110, a question request initiated by the client is obtained, where the question request includes question input data.

[0065] In an example embodiment of the present specification, question input data refers to instructions input by a user through an input interface pre-provided on the client. For example, the question input data may be an instruction to query weather conditions, such as "What's the weather like today"; the question input data may also be an instruction to open path navigation, such as "Navigate to the company". Of course, the question input data may also be input instructions for other content, and this example embodiment does not specifically limit this.

[0066] A question request refers to a request instruction initiated by the client to the device deployed with a large language model based on the question input data entered by the user, which is used to request to obtain the answer feedback data corresponding to the question input data.

[0067] A user interaction interface can be provided on the client, and an input box can be provided on the user interaction interface, so that the user can manually input the question input data through the input box; a voice recognition module can also be provided on the client to obtain the question input data input by the user in voice by recognizing the user's voice command; of course, a command control can also be provided, so that the user can select and trigger the command control to input the question input data. This example embodiment does not specifically limit the input method of the current question data. After detecting that the question input data has been input, a question request can be generated based on the question input data.

[0068] In step S120, the question input data is sent to a knowledge retrieval enhancement database corresponding to a plurality of knowledge providers, so as to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database.

[0069] In an example embodiment of the present specification, the knowledge provider refers to an end side that provides various types of knowledge information for enhancing the understanding ability of the large language model and ensuring the accuracy and effectiveness of the data. The knowledge information stored in the knowledge provider generally refers to data that is not used by the large language model during deep learning training. For example, the knowledge provider can be a provider of professional databases, such as providers of literature data in the fields of medicine and life sciences, providers of academic papers in the fields of electrical engineering and computer science, etc.; the knowledge provider can also be a provider of real-time data streams such as Internet data, social media data, real-time hot discussion data, financial market data, weather forecast data, etc.; of course, the knowledge provider can also be a provider of corporate privacy data or personal-related records, such as documents, policies, manuals accumulated within the company, or providers of data such as personal health records and personal financial records. This example embodiment does not specifically limit the type of knowledge provider and the type of knowledge information it can provide.

[0070] The knowledge retrieval enhancement database refers to a database pre-established on the knowledge provider side for realizing rapid knowledge retrieval. For example, the knowledge retrieval enhancement database may be a vector database, which can convert the knowledge information that the knowledge provider side can provide into the form of knowledge vectors and store them in the knowledge retrieval enhancement database; the knowledge retrieval enhancement database may also be a scalar database, such as an SQL database, etc., which can convert the knowledge information that the knowledge provider side can provide into the form of SQL data and store them in the knowledge retrieval enhancement database; of course, the knowledge retrieval enhancement database may also be other types of databases for improving retrieval efficiency and retrieval accuracy. This example embodiment does not specifically limit the type of the knowledge retrieval enhancement database. For ease of explanation and understanding, the knowledge retrieval enhancement database is taken as a vector database as an example for subsequent explanation.

[0071] Retrieval enhancement data refers to knowledge information related to the question input data retrieved and matched from the knowledge retrieval enhancement database. For example, taking the question input data as "How is the weather today?", the retrieval enhancement data can be the target geographic area information associated with the current location obtained from the knowledge retrieval enhancement database corresponding to the provider of the map data, or it can be the weather forecast information corresponding to multiple geographic area information associated with the current location obtained from the knowledge retrieval enhancement database corresponding to the provider of the weather forecast data stream. This example embodiment does not impose any special limitation on the data type of the retrieval enhancement data.

[0072] The question input data can be sent to the knowledge retrieval enhancement databases corresponding to multiple knowledge providers, and then the knowledge providers can obtain at least one retrieval enhancement data from different knowledge retrieval enhancement databases. For example, taking the knowledge retrieval enhancement database as a vector database, the question input data can be converted into a question vector, and multiple data vectors with high similarity can be matched in the knowledge retrieval enhancement database according to the question vector, and the corresponding detection enhancement data can be determined based on these matched data vectors. After the retrieval enhancement data is matched and before the retrieval enhancement data is output from the knowledge retrieval enhancement database or the knowledge provider, the knowledge provider can encrypt and return the matched retrieval enhancement data to obtain at least one encrypted retrieval enhancement data.

[0073] In step S130, each of the retrieval enhancement data is decrypted and integrated to obtain knowledge enhancement context data.

[0074] In an example embodiment of the present specification, knowledge enhancement context data refers to data obtained by merging various retrieval enhancement data, which is used as knowledge reinforcement data for the input data of the large language model to answer questions. For example, the retrieval enhancement data from different knowledge retrieval enhancement databases can be judged for similarity, and the retrieval enhancement data with the same or similar semantics can be merged to obtain the knowledge enhancement context data. Since the retrieval enhancement data from different knowledge retrieval enhancement databases may have the same semantic expression, the amount of knowledge enhancement context data can be effectively reduced by integrating the retrieval enhancement data to obtain the knowledge enhancement context data, thereby improving the processing efficiency of the large language model to a certain extent. Of course, key information in the retrieval enhancement data can also be extracted, and the knowledge enhancement context data can be obtained by merging the key information. This example embodiment does not specifically limit the method of integrating the retrieval enhancement data to obtain the knowledge enhancement context data.

[0075] After obtaining the retrieval enhancement data encrypted by the knowledge provider, the retrieval enhancement data can be decrypted. For example, taking the method in which the knowledge provider encrypts the retrieval enhancement data by a public key, the retrieval enhancement data can be decrypted according to a private key that matches the public key, and the decrypted retrieval enhancement data can be integrated. For example, the integration processing can be a fusion of semantically identical or similar retrieval enhancement data, extracting keyword information from the retrieval enhancement data and merging them, etc., to obtain integrated knowledge enhancement context data.

[0076] In step S140, the knowledge enhancement context data is sent to a pre-deployed large language model to determine answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model.

[0077] In an example embodiment of the present specification, the pre-deployed large language model refers to a deep learning model deployed on the server side for performing different types of natural language processing (NLP) tasks. For example, the large language model can be a trained BERT (Bidirectional Encoder Representations from Transformers) model; of course, the large language model can also be a trained GPT (Generative Pre-trained Transformer) model. This example embodiment does not specifically limit the type of the large language model.

[0078] Answer feedback data refers to feedback data output by the large language model based on the knowledge-enhanced context data for answering the question input data. For example, if the question input data is "How is the weather today?" and the weather forecast data included in the knowledge-enhanced context data is "The weather is sunny," then the answer feedback data output by the large language model may be "Today's weather is sunny, suitable for going out for fun." It is understandable that this is only an illustrative example, and the present embodiment is not limited thereto.

[0079] After the knowledge enhancement context data is sent to the pre-deployed large language model, question and answer prompt data can be constructed based on the knowledge enhancement context data and the question input data, and the question and answer prompt data can be used as input data of the large language model, thereby guiding the large language model to generate answer feedback data based on the knowledge enhancement context data to improve the accuracy and effectiveness of the answer feedback data.

[0080] By sending the question input data to the knowledge retrieval enhancement databases of multiple knowledge providers, at least one encrypted retrieval enhancement data is obtained, and the knowledge enhancement context data is obtained after decryption. It is possible to ensure the security of the privacy data in the knowledge retrieval enhancement database of each knowledge provider while fully retrieving the retrieval enhancement data of each knowledge provider, thereby improving the accuracy and effectiveness of the answer feedback data output by the large language model; and it is possible to effectively ensure the security of the privacy data in the knowledge retrieval enhancement database of each knowledge provider, thereby reducing the risk of privacy leakage caused by the centralized storage of privacy data sets.

[0081] Next, step S110 to step S140 are described in detail.

[0082] In an exemplary embodiment of the present specification, the knowledge retrieval enhancement database may be a vector database, which may be obtained by Figure 2 The step in the embodiment sends the question input data to the knowledge retrieval enhancement database corresponding to the plurality of knowledge providers, so as to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database, referring to Figure 2 As shown, it may specifically include:

[0083] Step S210, determining a question retrieval vector according to the question input data;

[0084] Step S220, sending the question retrieval vector to the knowledge retrieval enhancement database, so that each of the knowledge retrieval enhancement databases matches the retrieval enhancement data based on the question retrieval vector and encrypts it through the knowledge provider; and

[0085] Step S230, receiving at least one encrypted retrieval enhancement data returned by each of the knowledge providers.

[0086] Among them, the question retrieval vector refers to a feature vector obtained by extracting features from the question input data. For example, the question retrieval vector can be a feature vector extracted by inputting the question input data into a feature extraction network of a large language model, or it can be a feature vector obtained by word embedding the question input data through a word2vec model, a bag-of-words model, etc. This example embodiment does not make any special limitations on this.

[0087] The question retrieval vector can be sent to the knowledge retrieval enhancement database of the knowledge provider end, so that each knowledge retrieval enhancement database can match the retrieval enhancement data based on the question retrieval vector, obtain at least one retrieval enhancement data, and encrypt the retrieval enhancement data through the knowledge provider end; and receive at least one encrypted retrieval enhancement data returned by each knowledge provider end.

[0088] By constructing a knowledge retrieval enhancement database in the form of a vector database at each knowledge provider end, and using the question retrieval vector corresponding to the question input data, the retrieval enhancement data related to the question input data can be quickly matched and retrieved in the knowledge retrieval enhancement database, thereby improving the retrieval efficiency and accuracy of the retrieval enhancement data; and by obtaining the retrieval enhancement data at each knowledge provider end, distributed data retrieval is realized, thereby improving the transmission efficiency of the retrieval enhancement data, and at the same time, it can effectively avoid the risk of leakage of all knowledge information due to centralized storage of all knowledge information, thereby ensuring data security; in addition, the retrieval enhancement data is encrypted at the knowledge provider end to ensure that the retrieval enhancement data is fully encrypted after leaving the database, thereby reducing the possibility of data leakage at each knowledge provider end and improving data security.

[0089] Optionally, after receiving the question retrieval vector, different knowledge providers may input the question retrieval vector into the knowledge retrieval enhancement database, so that the knowledge retrieval enhancement database can calculate the vector similarity between the question retrieval vector and each vector in the knowledge retrieval enhancement database, and sort the vector similarity, thereby retrieving a list of retrieval enhancement vectors whose matching vector similarity is greater than or equal to a first similarity threshold, and then determine the retrieval enhancement data based on the retrieval enhancement vector list; after retrieving the retrieval enhancement data, the knowledge provider can encrypt and transmit the retrieval enhancement data using the stored public key.

[0090] Among them, the first similarity threshold refers to a pre-set parameter for matching the retrieval enhancement vector related to the question retrieval vector from the knowledge retrieval enhancement database. For example, the first similarity threshold can be 99% or 98%. The specific setting can be customized according to actual usage. This example embodiment does not specifically limit the specific value of the first similarity threshold.

[0091] The retrieval enhancement vector list refers to a list consisting of multiple retrieval enhancement vectors in the knowledge retrieval enhancement database whose vector similarity with the question retrieval vector is greater than or equal to the first similarity threshold. The retrieval enhancement vector list obtained through retrieval matching can quickly obtain the retrieval enhancement data associated with the question input data in the current knowledge retrieval enhancement database.

[0092] It is understandable that the search enhancement vector obtained by searching and matching from the knowledge search enhancement database may also be directly used as the search enhancement data, but the present embodiment is not limited thereto.

[0093] Through the first similarity threshold, the retrieval enhancement data related to the question input data is quickly matched and retrieved in the knowledge retrieval enhancement database, thereby improving the retrieval efficiency and retrieval accuracy of the retrieval enhancement data; and by encrypting the retrieval enhancement data at the knowledge provider end, it is ensured that the retrieval enhancement data is fully encrypted after it is released from the database, reducing the possibility of data leakage of the retrieval enhancement data after it is released from the database, thereby improving data security.

[0094] In an optional implementation, the question input data may be input into a target feature extraction network to obtain a question retrieval vector.

[0095] Among them, the target feature extraction network refers to a neural network used to extract features from question input data. The target feature extraction network can be trained based on training sample data. For example, the target feature extraction network can be a feature extraction network built based on convolutional neural networks (CNN) and trained in combination with training sample data. Of course, the target feature extraction network can also be a feature extraction network built based on a Transformer neural network and trained. This embodiment does not specifically limit the network type of the target feature extraction network used for feature extraction tasks.

[0096] Of course, the target feature extraction network can also be constructed based on the feature extraction network in the large language model deployed on the server. For example, after the large language model is constructed, the target feature extraction network can be constructed based on the feature extraction network of the large language model. The target feature extraction network constructed based on the feature extraction network in the large language model deployed on the server can improve the accuracy of the question retrieval vector expression, thereby improving the data accuracy of the retrieval enhancement data retrieved and matched in the knowledge retrieval enhancement database.

[0097] It can be understood that the retrieval enhancement vectors in the knowledge retrieval enhancement database can also be extracted through the target feature extraction network. Using the same target feature extraction network to generate question retrieval vectors and construct the knowledge retrieval enhancement database can further improve the accuracy of the matching retrieval results, and provide good accuracy and effectiveness of the retrieval enhancement data matched by the retrieval, thereby improving the accuracy and effectiveness of the answer feedback data output by the large language model.

[0098] In an exemplary embodiment of the present specification, decryption of each retrieval enhancement data may be achieved through the following steps, which may specifically include:

[0099] Each retrieval enhancement data may be sent to a local hardware trusted execution environment, and the retrieval enhancement data may be decrypted using a private key stored in the hardware trusted execution environment to obtain the decrypted retrieval enhancement data.

[0100] Among them, the hardware trusted execution environment (TEE) is a secure area built by software and hardware methods on the computing platform, which can ensure that the code and data loaded in the secure area are protected in terms of confidentiality and integrity. Its goal is to ensure that a task is executed as expected, and to ensure the confidentiality and integrity of the initial state, as well as the confidentiality and integrity of the runtime state. The system's hardware and software resources can be divided into two execution environments, namely the hardware trusted execution environment and the ordinary execution environment. These two environments are securely isolated and have independent internal data paths and storage space required for computing. Applications in the ordinary execution environment cannot access TEE. Even within the TEE, the operation of multiple applications is independent of each other and cannot access each other without authorization.

[0101] By sending the encrypted retrieval enhancement data on the knowledge provider side to the local hardware trusted execution environment, and decrypting the retrieval enhancement data using the private key stored in the hardware trusted execution environment to obtain the decrypted retrieval enhancement data, the risk of leakage of the retrieval enhancement data after it is released from the database can be further reduced, thereby ensuring the security of the retrieval enhancement data.

[0102] It is understandable that at least one key pair can be generated during the initialization phase. The key pair may include a public key and a private key, and the public key may be divided and saved in each knowledge retrieval enhancement database, and the private key may be saved in a local hardware trusted execution environment. For example, a key pair may be generated, and the public key in the key pair may be broadcasted to each knowledge retrieval enhancement database, and the private key in the key pair may be saved in a local hardware trusted execution environment. Alternatively, a number of key pairs equal to the number of knowledge providers may be generated, and different public keys may be provided for different knowledge providers, and a set of private keys may be saved in a local hardware trusted execution environment. During decryption, the corresponding private key may be matched in the private key set according to the identification information corresponding to the knowledge provider to complete the decryption of the retrieval enhancement data. This example embodiment does not specifically limit the way in which the keys are distributed.

[0103] By generating key pairs through asymmetric encryption, distributing and saving the public keys in various knowledge retrieval enhancement databases, and saving the private keys in the local hardware trusted execution environment, the security of retrieval enhancement data can be further improved.

[0104] In an exemplary embodiment of the present specification, the knowledge enhancement context data may be integrated through the following steps, which may specifically include:

[0105] The semantic similarity between the decrypted retrieval enhancement data may be determined, and the retrieval enhancement data with a semantic similarity greater than or equal to a second similarity threshold may be merged to obtain the knowledge enhancement context data.

[0106] The second similarity threshold refers to reference data used to merge identical or similar retrieval enhancement data. For example, the second similarity threshold may be 90% or 80%. The specific setting may be customized according to actual usage, and this embodiment does not impose any special limitation on this.

[0107] The semantic similarity between the decrypted retrieval enhancement data can be determined. When retrieval enhancement data with a semantic similarity greater than or equal to a second similarity threshold is detected, it can be considered that the two retrieval enhancement data represent the same knowledge content. Therefore, the two retrieval enhancement data can be merged, either of the two retrieval enhancement data can be deleted, or the same or similar keyword information in the two retrieval enhancement data can be extracted, and the extracted same or similar keyword information can be used as the merged data to traverse all the retrieval enhancement data to obtain knowledge enhancement context data.

[0108] By merging retrieval enhancement data whose semantic similarity is greater than or equal to the second similarity threshold, the accuracy and completeness of the integrated knowledge enhancement context data can be ensured while effectively reducing the data volume of the knowledge enhancement context data, improving the efficiency of the large language model in understanding the knowledge enhancement context data, and thus improving the efficiency of the large language model in outputting answer feedback data.

[0109] In an exemplary embodiment of the present specification, the following steps may be used to determine the answer feedback data corresponding to the question input data through the knowledge enhanced context data, the question input data, and the large language model, which may specifically include:

[0110] Question prompt data can be constructed through knowledge enhancement context data and question input data, and the question prompt data can be input into a large language model to obtain answer feedback data corresponding to the question input data.

[0111] Among them, question and answer prompt data refers to prompt data constructed based on question input data and knowledge enhancement context data for input into a large language model to guide it to generate answer feedback data. For example, question and answer prompt data can be "You are a question and answer robot assistant, please use the following retrieved context data to answer the question. If you don’t know the answer, just say you don’t know; the question is {question input data}, and the context data is {knowledge enhancement context data}"; of course, the above question and answer prompt data is only a schematic example, and the question and answer prompt data only needs to include question input data and knowledge enhancement context data. This embodiment does not specifically limit the data structure of the question and answer prompt data.

[0112] Optionally, you can Figure 3The steps in this paper are to build question prompt data by enhancing the context data with knowledge and question input data, refer to Figure 3 As shown, it may specifically include:

[0113] Step S310, obtaining a preset model question and answer prompt template;

[0114] Step S320, filling the knowledge enhancement context data and the question input data into the model question and answer prompt template to generate the question and answer prompt data.

[0115] Among them, the model question and answer prompt template refers to a template used to generate answer data corresponding to the question input data that guides the large language model to fuse the knowledge-enhanced context data to output. For example, the model question and answer prompt template can be "You are a question and answer robot assistant, please use the following retrieved context data to answer the question. If you don’t know the answer, just say you don’t know; the question is {A}, and the context data is {B}", "{A}" can be filled with question input data, and "{B}" can be filled with knowledge-enhanced context data. Of course, this is only a schematic example, and this example embodiment does not specifically limit the type of model question and answer prompt template.

[0116] The model question and answer prompt template can be used to quickly generate question and answer prompt data, effectively improving the efficiency of generating question and answer prompt data. At the same time, the model question and answer prompt template can be customized according to the understanding ability of the large language model, making the generated question and answer prompt data easier to be understood by the large language model, thereby improving the accuracy and effectiveness of the answer data output by the large language model to a certain extent.

[0117] Figure 4 A flowchart of a knowledge retrieval enhancement generation method for implementing large language model question and answer based on multiple knowledge providers is provided in an embodiment of this specification.

[0118] refer to Figure 4As shown, in step S402, the client initiates a question request to the third-party joint retrieval end, and the question request includes question input data; in step S404, the third-party joint retrieval end converts the question input data into a question retrieval vector; in step S406, the third-party joint retrieval end sends the question retrieval vector to each knowledge provider; in step S408, each knowledge provider searches for a list of retrieval enhancement vectors and corresponding retrieval enhancement data that meet the similarity requirements in the local knowledge retrieval enhancement database; in step S410, each knowledge provider uses a public key to encrypt the retrieval enhancement data matched by the retrieval; in step S412, each knowledge provider returns the encrypted retrieval enhancement data to the third-party joint retrieval end; in step S414, the third-party joint retrieval end inputs the uniformly collected encrypted retrieval enhancement data into the local hardware trusted execution environment; in step S416, the decryption program in the hardware trusted execution environment is stored in the hardware The private key in the hardware trusted execution environment is used to decrypt the retrieval enhancement data; in step S418, the decryption program of the hardware trusted execution environment returns the decrypted knowledge content to the third-party joint retrieval end; in step S420, the third-party joint retrieval end integrates the decrypted retrieval enhancement data to obtain knowledge enhancement context data, and assembles it with the question input data into question prompt data, and sends it to the downstream large language model; for example, the third-party joint retrieval end can be a third-party cloud-side device independent of the server, and the large language model can be deployed in the ordinary execution environment or the hardware trusted execution environment of the third-party joint retrieval end; it can be understood that the third-party joint retrieval end can be set in the server, and the large language model can be deployed in the server, and this implementation is not limited to this; in step S422, the third-party joint retrieval end returns the answer feedback data returned by the large language model to the client; in step S424, the client displays the answer feedback data and ends the current question and answer operation process.

[0119] To summarize, the multi-party joint knowledge retrieval enhancement method provided in the embodiments of the present specification obtains a question request initiated by a client, wherein the question request includes question input data; sends the question input data to a knowledge retrieval enhancement database of multiple knowledge providers to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database; decrypts each retrieval enhancement data and integrates them to obtain knowledge enhancement context data; and sends the knowledge enhancement context data to a server deployed with a large language model to determine answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model. On the one hand, by sending the question input data to the knowledge retrieval enhancement database of multiple knowledge providers, at least one encrypted retrieval enhancement data is obtained, and the knowledge enhancement context data is obtained after decryption. While ensuring the security of the privacy data in the knowledge retrieval enhancement database of each knowledge provider, the retrieval enhancement data of each knowledge provider can be fully retrieved, ensuring the integrity and validity of the large language model input data, thereby improving the accuracy and validity of the answer feedback data output by the large language model; on the other hand, the retrieved retrieval enhancement data is encrypted at each knowledge provider, and decrypted at the third-party joint retrieval terminal to obtain the knowledge enhancement context data, and the knowledge enhancement context data is used as the prompt data of the large language model for retrieval enhancement generation, which can effectively ensure the security of the privacy data in the knowledge retrieval enhancement database of each knowledge provider and reduce the privacy leakage risk caused by the centralized storage of privacy data sets.

[0120] In addition, it should be understood that the multi-party joint knowledge retrieval enhancement method of the embodiment of the present disclosure can be configured as a software module. In some implementation scenarios, the multi-party joint knowledge retrieval enhancement scheme of the present disclosure can be deployed separately to achieve knowledge retrieval enhancement of a large language model for user questions to obtain accurate and effective answer feedback data. In other implementation scenarios, the multi-party joint knowledge retrieval enhancement scheme of the present disclosure can be deployed in other software as a functional module of the software, such as deployed in software applications in various scenarios such as search engine software, office software, online shopping software, payment software, etc. The present disclosure does not place any special restrictions on the application of the multi-party joint knowledge retrieval enhancement method.

[0121] See also Figure 5 , is a schematic diagram of the structure of a multi-party joint knowledge retrieval enhancement device provided in an embodiment of this specification. Figure 5 As shown, the multi-party joint knowledge retrieval enhancement device 500 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the multi-party joint knowledge retrieval enhancement device 500 includes a question request acquisition module 510, a multi-party data joint retrieval module 520, a context data determination module 530, and an answer feedback data generation module 540, which may specifically include:

[0122] A question request acquisition module 510 is used to acquire a question request initiated by a client, wherein the question request includes question input data;

[0123] A multi-party data joint retrieval module 520, configured to send the question input data to a knowledge retrieval enhancement database corresponding to a plurality of knowledge providers, so as to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database;

[0124] A context data determination module 530 is used to decrypt each of the search enhancement data and integrate them to obtain knowledge enhancement context data;

[0125] The answer feedback data generating module 540 is used to send the knowledge enhancement context data to a server deployed with a large language model, so as to determine the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model.

[0126] In an optional embodiment of the present specification, based on the above solution, the knowledge retrieval enhancement database is a vector database, and the multi-party data joint retrieval module 520 is configured as follows:

[0127] Determine a question retrieval vector according to the question input data;

[0128] Sending the question retrieval vector to the knowledge retrieval enhancement database, so that each of the knowledge retrieval enhancement databases matches the retrieval enhancement data based on the question retrieval vector and encrypts it through the knowledge provider; and

[0129] Receive at least one encrypted retrieval enhancement data returned by each of the knowledge providers.

[0130] In an optional embodiment of the present specification, based on the aforementioned scheme, the knowledge retrieval enhancement database matches the question retrieval vector with a vector similarity greater than or equal to a first similarity threshold, thereby determining the retrieval enhancement data according to the retrieval enhancement vector list, and the knowledge provider encrypts the retrieval enhancement data using a stored public key.

[0131] In an optional embodiment of the present specification, based on the above solution, the multi-party data joint retrieval module 520 is configured as follows:

[0132] Inputting the question input data into a target feature extraction network to obtain a question retrieval vector;

[0133] Wherein, the target feature extraction network is constructed based on the feature extraction network in the large language model deployed on the server;

[0134] The retrieval enhancement vector in the knowledge retrieval enhancement database is extracted through the target feature extraction network.

[0135] In an optional embodiment of the present specification, based on the above solution, the context data determination module 530 is configured as follows:

[0136] Each of the retrieval enhancement data is sent to a local hardware trusted execution environment, and the retrieval enhancement data is decrypted using a private key stored in the hardware trusted execution environment to obtain the decrypted retrieval enhancement data.

[0137] In an optional embodiment of the present specification, based on the above solution, the context data determination module 530 is configured as follows:

[0138] Determine the semantic similarity between the decrypted retrieval enhancement data;

[0139] The retrieval enhancement data whose semantic similarity is greater than or equal to the second similarity threshold are merged to obtain knowledge enhancement context data.

[0140] In an optional embodiment of the present specification, based on the above solution, the multi-party joint knowledge retrieval enhancement device 500 further includes a key pair distribution module, and the key pair distribution module is configured as follows:

[0141] generating at least one key pair, the key pair comprising a public key and a private key;

[0142] The public key is divided and saved in each of the knowledge retrieval enhanced databases, and the private key is saved in a local hardware trusted execution environment.

[0143] In an optional embodiment of the present specification, based on the above solution, the answer feedback data generating module 540 is configured as follows:

[0144] constructing question prompt data using the knowledge enhancement context data and the question input data;

[0145] The question prompt data is input into the large language model to obtain answer feedback data corresponding to the question input data.

[0146] In an optional embodiment of the present specification, based on the above solution, the answer feedback data generating module 540 is configured as follows:

[0147] Get the preset model question and answer prompt template;

[0148] The knowledge enhancement context data and the question input data are filled into the model question and answer prompt template to generate the question and answer prompt data.

[0149] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0150] See also Figure 6 , is a schematic diagram of the architecture of a multi-party joint knowledge retrieval enhancement system provided in an embodiment of this specification. According to some embodiments, the multi-party joint knowledge retrieval enhancement system 600 may include a third-party joint retrieval terminal 610, a client 620 and at least one knowledge providing terminal 630. Among them:

[0151] The third-party joint search terminal 610 may include a multi-party joint knowledge search enhancement device as in the embodiment of the present specification, and executes the multi-party joint knowledge search enhancement method as in the embodiment of the present specification through the multi-party joint knowledge search enhancement device. In this embodiment, the trained large language model 611 can be deployed in the third-party joint search terminal 610, and at this time, the third-party joint search terminal 610 can be considered to be a server, or the third-party joint search terminal 610 is set in the server.

[0152] The client 620 can be connected to the third-party joint search terminal 610 for communication, and is used to obtain input question input data, generate a question request based on the question input data, send the question request to the third-party joint search terminal 610, and receive and display the answer feedback data returned by the third-party joint search terminal 610.

[0153] At least one knowledge provider 630 can be connected to the third-party joint search terminal 610 for communication, and is used to maintain and update the knowledge retrieval enhancement database 631 , and obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database 631 based on the question input data sent by the third-party joint search terminal 610 .

[0154] In an alternative embodiment, reference Figure 7 As shown, the multi-party joint knowledge retrieval enhancement system 600 may also include a server 640. The server 640 may be deployed with a large language model 611, and is in communication connection with the third-party joint retrieval terminal 610, and is used to receive the knowledge enhancement context data and question input data returned by the third-party joint retrieval terminal 610, and determine the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model 611, and return the answer feedback data to the third-party joint retrieval terminal 610. In this embodiment, the third-party joint retrieval terminal 610 can be considered as a third-party security terminal independent of the client 620 and the server 640.

[0155] The embodiments of this specification also provide a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded by a processor and executing the method as described in the embodiments of this specification. The specific execution process can be found in the specific description of the embodiments of this specification, and will not be repeated here.

[0156] This specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded by the processor and executes the method as described in the embodiments of this specification. The specific execution process can be found in the specific description of the embodiments of this specification, and will not be repeated here.

[0157] The embodiments of this specification also provide Figure 8 The structural diagram of the electronic device shown in FIG. Figure 8 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned voice activity detection method.

[0158] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0159] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0160] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0161] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0162] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0163] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0167] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0168] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0169] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0170] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0171] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0172] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0173] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0174] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A multi-party joint knowledge retrieval enhancement method, the method is applied to a third-party joint retrieval end, the hardware trusted execution environment at the third-party joint retrieval end stores a private key in a key pair generated by asymmetric encryption, the method comprising: Obtaining a question request initiated by a client, wherein the question request includes question input data; Sending the question retrieval vector determined according to the question input data to a knowledge retrieval enhancement database corresponding to a plurality of knowledge providers, so as to obtain at least one retrieval enhancement data encrypted by using the public key in the key pair from the knowledge retrieval enhancement database; wherein the retrieval enhancement data is data determined according to a retrieval enhancement vector at the knowledge retrieval enhancement database whose similarity to the question retrieval vector is greater than or equal to a first similarity threshold; the question retrieval vector and the retrieval enhancement vector are vectors extracted by using the same target feature extraction network, and the target feature extraction network is constructed by using a feature extraction network in a large language model; Decrypting each of the retrieval enhancement data using the private key in the hardware trusted execution environment, and integrating them to obtain knowledge enhancement context data; The knowledge enhancement context data is sent to the pre-deployed large language model to determine answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model.

2. According to the multi-party joint knowledge retrieval enhancement method of claim 1, the knowledge retrieval enhancement database is a vector database, and the step of sending the question input data to the knowledge retrieval enhancement databases corresponding to the plurality of knowledge providers to obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database comprises: Determine a question retrieval vector according to the question input data; Sending the question retrieval vector to the knowledge retrieval enhancement database, so that each of the knowledge retrieval enhancement databases matches the retrieval enhancement data based on the question retrieval vector and encrypts it through the knowledge provider; as well as Receive at least one encrypted retrieval enhancement data returned by each of the knowledge providers.

3. According to the multi-party joint knowledge retrieval enhancement method according to claim 2, the knowledge retrieval enhancement database is based on the list of retrieval enhancement vectors whose matching vector similarity of the question retrieval vector is greater than or equal to a first similarity threshold, thereby determining the retrieval enhancement data according to the retrieval enhancement vector list, and the knowledge provider encrypts the retrieval enhancement data through a stored public key.

4. The multi-party joint knowledge retrieval enhancement method according to claim 2, wherein the step of determining the question retrieval vector according to the question input data comprises: Inputting the question input data into a target feature extraction network to obtain a question retrieval vector; Wherein, the target feature extraction network is constructed based on the feature extraction network in the large language model deployed on the server; The retrieval enhancement vector in the knowledge retrieval enhancement database is extracted through the target feature extraction network.

5. The multi-party joint knowledge retrieval enhancement method according to claim 1, wherein the decrypting of each of the retrieval enhancement data comprises: Each of the retrieval enhancement data is sent to a local hardware trusted execution environment, and the retrieval enhancement data is decrypted using a private key stored in the hardware trusted execution environment to obtain the decrypted retrieval enhancement data.

6. According to the multi-party joint knowledge retrieval enhancement method of claim 5, the integration to obtain knowledge enhancement context data comprises: Determine the semantic similarity between the decrypted retrieval enhancement data; The retrieval enhancement data whose semantic similarity is greater than or equal to the second similarity threshold are merged to obtain knowledge enhancement context data.

7. The multi-party joint knowledge retrieval enhancement method according to any one of claims 1 to 6, further comprising: generating at least one key pair, the key pair comprising a public key and a private key; The public key is divided and saved in each of the knowledge retrieval enhanced databases, and the private key is saved in a local hardware trusted execution environment.

8. The multi-party joint knowledge retrieval enhancement method according to claim 1, wherein the step of determining the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model comprises: Constructing question and answer prompt data by using the knowledge enhancement context data and the question input data; The question and answer prompt data is input into the large language model to obtain answer feedback data corresponding to the question input data.

9. The multi-party joint knowledge retrieval enhancement method according to claim 8, wherein the step of constructing question-answer prompt data using the knowledge enhancement context data and the question input data comprises: Get the preset model question and answer prompt template; The knowledge enhancement context data and the question input data are filled into the model question and answer prompt template to generate the question and answer prompt data.

10. A multi-party joint knowledge retrieval enhancement device, the device is applied to a third-party joint retrieval end, the hardware trusted execution environment at the third-party joint retrieval end stores a private key in a key pair generated by asymmetric encryption, comprising: A question request acquisition module, used to acquire a question request initiated by a client, wherein the question request includes question input data; A multi-party data joint retrieval module, used to send the question retrieval vector determined according to the question input data to the knowledge retrieval enhancement database corresponding to the multiple knowledge providers, so as to obtain at least one retrieval enhancement data encrypted by the public key in the key pair from the knowledge retrieval enhancement database; wherein the retrieval enhancement data is data determined according to the retrieval enhancement vector at the knowledge retrieval enhancement database whose similarity with the question retrieval vector is greater than or equal to a first similarity threshold; the question retrieval vector and the retrieval enhancement vector are vectors extracted by using the same target feature extraction network, and the target feature extraction network is constructed by using the feature extraction network in the large language model; A context data determination module, configured to decrypt each of the retrieval enhancement data using the private key in the hardware trusted execution environment, and integrate the data to obtain knowledge enhancement context data; The answer feedback data generating module is used to send the knowledge enhancement context data to the server where the large language model is deployed, so as to determine the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model.

11. A multi-party joint knowledge retrieval enhancement system, the system comprising: A third-party joint search terminal, comprising the multi-party joint knowledge search enhancement device as claimed in claim 10, and executing the multi-party joint knowledge search enhancement method as claimed in any one of claims 1 to 9 through the multi-party joint knowledge search enhancement device; The client is connected to the third-party joint search end for obtaining input question input data, generating a question request according to the question input data, sending the question request to the third-party joint search end, and receiving and displaying answer feedback data returned by the third-party joint search end; At least one knowledge providing end is communicatively connected to the third-party joint retrieval end, and is used to maintain and update the knowledge retrieval enhancement database, and obtain at least one encrypted retrieval enhancement data from the knowledge retrieval enhancement database according to the question input data sent by the third-party joint retrieval end.

12. The multi-party joint knowledge retrieval enhancement system according to claim 11, further comprising: The server is deployed with a large language model and is in communication connection with the third-party joint retrieval end, and is used to receive the knowledge enhancement context data and question input data returned by the third-party joint retrieval end, and determine the answer feedback data corresponding to the question input data through the knowledge enhancement context data, the question input data and the large language model, and return the answer feedback data to the third-party joint retrieval end.

13. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

14. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method as claimed in any one of claims 1 to 9.

15. A computer program product having at least one instruction stored thereon, characterized in that: When the at least one instruction is executed by the processor, the steps of the method described in any one of claims 1 to 9 are implemented.

Images