Encryption enhancement generation method, system and program product using large language model

Abstract

The present disclosure provides an encryption enhancement generation method, system, and program product using a large language model, relating to the technical field of large language models. The encryption enhancement generation method includes: obtaining a first encryption vector corresponding to the demand information through a preset encryption model based on demand information input by a user; matching the first encryption vector with an encrypted knowledge vector to determine the encrypted knowledge vector matching the first encryption vector as a second encryption vector; obtaining professional knowledge information corresponding to the second encryption vector through a decryption model corresponding to the user key based on the second encryption vector and a user key corresponding to the user; and generating feedback results corresponding to the demand information through a large language model based on the professional knowledge information and demand information.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to an encryption enhancement generation method, system, and program product using a large language model. Background Art

[0002] Large Language Models (LLMs) are deep learning-based natural language processing models trained on large amounts of text and image data. These models are capable of processing and generating natural language text and images. With the continuous development and advancement of LLM technology, they are playing an indispensable role in a growing number of specialized research fields.

[0003] Considering the vast amount of specialized knowledge and academic terminology in professional research fields, to avoid the problem of hallucination when large language models process specialized knowledge in professional research fields—that is, the content generated by the large language model appears reasonable on the surface, but is actually fictitious or inaccurate—this problem can be overcome by integrating an enhanced generation knowledge base containing specialized knowledge information into the large language model. However, the specialized knowledge information stored in the enhanced generation knowledge base is prone to leakage during the process of assisting the large language model in content generation. Furthermore, the specialized knowledge information contained in the enhanced generation knowledge base may also contain confidential content that needs to be kept confidential, making it equally susceptible to illegal access or confidential content leakage during the process of assisting the large language model in content generation.

[0004] In view of this, some embodiments of this specification provide an encryption-enhanced generation method, system, and program product using a large language model, which can ensure the confidentiality and security of professional knowledge information in the process of assisting the large language model in content generation. Summary of the Invention

[0005] One or more embodiments of the present specification provide an encryption enhancement generation method using a large language model, the method comprising: obtaining a first encryption vector corresponding to the demand information through a preset encryption model based on demand information input by a user; matching the first encryption vector with an encrypted knowledge vector for similarity to determine the encrypted knowledge vector that matches the first encryption vector as a second encryption vector; obtaining professional knowledge information corresponding to the second encryption vector through a decryption model corresponding to the user key based on the second encryption vector and a user key corresponding to the user; and generating feedback results corresponding to the demand information through a large language model based on the professional knowledge information and the demand information.

[0006] In some embodiments, the encrypted knowledge vector is obtained by performing vectorized encryption processing based on the professional knowledge information stored in the enhanced generation knowledge base associated with the large language model through the encryption model; wherein the encrypted knowledge vector is stored independently of the enhanced generation knowledge base.

[0007] In some embodiments, the method further includes: performing legitimacy verification on a large language model generation request input by a user based on a preset license key, the large language model generation request including a user key and requirement information; if the legitimacy verification passes, obtaining a first encryption vector corresponding to the requirement information through an encryption model.

[0008] In some embodiments, matching the first encrypted vector with the encrypted knowledge vector based on similarity to determine the encrypted knowledge vector that matches the first encrypted vector as the second encrypted vector includes: obtaining the cosine similarity between the first encrypted vector and the encrypted knowledge vector; and determining the encrypted knowledge vector whose cosine similarity falls within a preset range as the second encrypted vector.

[0009] In some embodiments, the preset process of the encryption model includes: performing full parameter adjustment training on the preset natural language processing model so that the input data distribution and output data distribution of the natural language processing model are consistent; and determining the part before the vector representation stage in the natural language processing model after the full parameter adjustment training as the encryption model.

[0010] In some embodiments, the preset process of the decryption model includes: introducing a randomly initialized user feature vector into the part after the vector representation stage in the natural language processing model after full parameter adjustment and performing local parameter adjustment training to make the input data distribution and output data distribution of the natural language processing model consistent; determining the part after the vector representation stage in the natural language processing model after local parameter adjustment training as the decryption model, and the decryption model corresponds one-to-one to the user feature vector; and using the user feature vector as the user key.

[0011] One or more embodiments of the present specification also provide an encryption enhancement generation system using a large language model, the system comprising: a first server, configured to obtain, based on the demand information input by the user, a first encryption vector corresponding to the demand information through a preset encryption model; and a server for matching the first encryption vector with an encrypted knowledge vector to determine the encrypted knowledge vector that matches the first encryption vector as a second encryption vector and output it; a second server, configured to obtain, based on the second encryption vector and the user key corresponding to the user, professional knowledge information corresponding to the second encryption vector through a decryption model corresponding to the user key; the large language model is mounted on the second server, configured to generate corresponding feedback results based on the professional knowledge information and the demand information.

[0012] In some embodiments, the system also includes: an enhanced generation knowledge base for storing professional knowledge information; the first server includes an encrypted knowledge vector library, the encrypted knowledge vector library is used to obtain and store encrypted knowledge vectors based on the professional knowledge information stored in the enhanced generation knowledge base through an encryption model; the first server and the enhanced generation knowledge are separated from each other and deployed independently.

[0013] One or more embodiments of the present specification also provide an encryption enhancement generation system using a large language model, the system comprising: a demand encryption module for obtaining, based on demand information input by a user, a first encryption vector corresponding to the demand information through a preset encryption model; a knowledge matching module for matching the first encryption vector with the encrypted knowledge vector to determine the encrypted knowledge vector that matches the first encryption vector as a second encryption vector; a knowledge decryption module for obtaining, based on the second encryption vector and a user key corresponding to the user, professional knowledge information corresponding to the second encryption vector through a decryption model corresponding to the user key; and a demand feedback module for generating, based on the professional knowledge information and demand information, feedback results corresponding to the demand information through a large language model.

[0014] Some embodiments of this specification also provide a computer program product, including computer instructions or a computer program. When at least part of the computer instructions or computer program is executed by a processor, it can implement the encryption enhancement generation method using a large language model provided in any embodiment of this specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. The same numbers in the drawings represent the same structures or steps.

[0016] Figure 1 This is a structural diagram of a demand feedback system using a large language model according to some embodiments of this specification.

[0017] Figure 2 This is an exemplary flowchart of an encryption enhancement generation method using a large language model according to some embodiments of this specification.

[0018] Figure 3 This is an exemplary flowchart of a preset process of an encryption model according to some embodiments of this specification.

[0019] Figure 4 This is an exemplary flowchart of a preset process of a decryption model according to some embodiments of this specification.

[0020] Figure 5This is a data flow diagram of an encryption enhancement generation system using a large language model according to some embodiments of this specification.

[0021] Figure 6 This is a functional module diagram of an encryption enhancement generation system using a large language model according to some embodiments of this specification. DETAILED DESCRIPTION

[0022] To more clearly illustrate the technical solutions of the embodiments of this specification, the embodiments will be described in detail below with reference to the accompanying drawings. Obviously, the following descriptions are some examples or embodiments of this specification, and those skilled in the art can apply the technical solutions or methods disclosed in this specification to other scenarios based on these technical contents without inventive effort.

[0023] It should be understood that the terms "system," "device," "unit," and / or "module" used in this specification are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the above terms can be replaced by other expressions.

[0024] Unless otherwise specified, technical terms used in this specification to describe components, elements, and the like do not necessarily refer to the singular but may include the plural. Generally speaking, terms such as "include" and "comprising" only indicate the inclusion of the steps, elements, or components specifically identified, and these steps, elements, and components do not constitute an exclusive list. For example, the method or device being described may also include other steps or components.

[0025] This specification uses flowcharts to illustrate the operational steps performed by the devices or systems of the relevant embodiments. However, unless otherwise specified, the order in which these steps are described should not be construed as limiting the order in which the steps are performed. A person of ordinary skill in the art may adjust the order in which these steps are performed based on the knowledge and information conveyed by the embodiments of this specification. Such adjustments include, but are not limited to, reversing the order of the steps, combining multiple steps, and splitting a step.

[0026] Figure 1 This is a schematic diagram of a demand feedback system using a large language model according to some embodiments of this specification. Figure 1 As shown, the demand feedback system 100 using a large language model may include a processing device 110, a network 120, a storage medium 130, and a terminal device 140. The processing device 110 and the terminal device 140 may transmit data via the network 120. In some embodiments, Figure 1The demand feedback system 100 using a large language model shown can be applied to clinical research, medical assistance, content generation, health care, online self-diagnosis, post-diagnosis follow-up, etc., without limitation here.

[0027] The processing device 110 can be a computer device with high computing performance, used to process the user's input demand information through a large language model and generate the feedback results required by the user. In some embodiments, the processing device 110 can be a single computer device or a computing cluster composed of multiple computer devices, thereby providing more powerful computing power and more efficient response to the user's input demand information. In some embodiments, the processing device 110 can be a server; specifically, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Distribute Network), and big data and artificial intelligence platforms.

[0028] The network 120 may be any form of wired or wireless network, or any combination thereof. By way of example only, the network 120 may be one or more combinations of a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, and the like. The network 120 may have multiple access points, through which the processing device 110 and the terminal device 140 may access the network 120.

[0029] The storage medium 130 can be used to store data and / or instructions related to the demand feedback system 100 using a large language model. In some embodiments, the storage medium 130 can store data and / or information obtained from the processing device 110, the terminal device 140, etc. For example, the storage medium 130 can store material content information used to generate feedback results, a trained large language model, etc. In some embodiments, the storage medium 130 can include one or more storage components, each of which can be a standalone device or part of another device. In some embodiments, the storage medium 130 can be located in the processing device 110. In some embodiments, the storage medium 130 can include random access memory (RAM), read-only memory (ROM), mass storage, removable storage, volatile read-write memory, etc., or any combination thereof. Exemplary mass storage devices include magnetic disks, optical disks, solid-state drives, etc. In some embodiments, the storage medium 130 can be implemented on a cloud platform.

[0030] Terminal device 140 may include, but is not limited to, a desktop computer, a smartphone, a laptop computer, a VR (Virtual Reality) device, and a tablet computer. Terminal device 140 may be a document recommendation application client, a browser client equipped with a document search program and a file recommendation program, or an instant messaging client. Users can install and run an application corresponding to a demand feedback system using a large language model on terminal device 140, display feedback results on the terminal device, and input demand information through a human-computer interface device (HID). The HID may be part of terminal device 140 or separate from terminal device 140 but signal-connected to the terminal device 140. For example, terminal device 140 may have a signal connection to processing device 110 via network 120. Users can send demand information to processing device 110 and obtain feedback corresponding to the demand information from processing device 110.

[0031] It should be noted that Figure 1 The demand feedback system 100 using a large language model is only an example. The demand feedback system and scenario using a large language model described in this specification are intended to more clearly illustrate the technical solution of this specification and do not constitute a limitation on the technical solution provided in this specification. For example, in a large language model application scenario deployed locally, Figure 1 The processing device 110, network 120, and / or storage medium 130 shown may be omitted. Those skilled in the art will appreciate that, with the emergence of new application scenarios for large language models, the technical solutions provided in the embodiments of this specification are equally applicable to similar technical problems. For example, in some application scenarios, the processing device 110, storage medium 130, and terminal device 140 may be integrated into one, in which case the network 120 may be omitted. For example, in some application scenarios, the processing device 110 and terminal device 140 may be integrated into one.

[0032] In some embodiments, considering that there is a large amount of professional knowledge and academic terms in professional research fields, in order to avoid the problem of hallucination when the large language model processes the professional knowledge in the professional research field, that is, although the content generated by the large language model looks reasonable on the surface, the actual content is fictitious or inaccurate information, the enhanced generation knowledge base containing professional knowledge information can be connected to the large language model to overcome this problem. Specifically, the demand information input by the user can be semantically matched with the professional knowledge information stored in the enhanced generation knowledge base to obtain professional knowledge information related to the demand information, and then the obtained professional knowledge information and the demand information can be input into the large language model as prompts to provide the large language model with richer reference information and help the large language model generate more accurate feedback results. However, the professional knowledge information stored in the enhanced generation knowledge base is prone to professional knowledge information leakage in the process of assisting the large language model to generate content; in addition, the professional knowledge information contained in the enhanced generation knowledge base may also contain confidential content that needs to be kept confidential, which is also easy to be illegally obtained or confidential content is leaked in the process of assisting the large language model to generate content. Malicious attackers can easily illegally obtain confidential content stored in the enhanced generation knowledge base during the process of semantic matching between demand information and professional knowledge information stored in the enhanced generation knowledge base, which has a significant impact on the data storage security of the enhanced generation knowledge base.

[0033] To overcome the above problems, this specification provides an encryption enhancement generation method, system, and program product using a large language model. Specifically, Figure 2 This is an exemplary flow chart of an encryption enhancement generation method using a large language model according to some embodiments of this specification. In some embodiments, Figure 2 The process 200 shown may be executed by the processing device 110. In some embodiments, the process 200 may be implemented by the encryption enhancement generation system 600 using a large language model deployed on the processing device 110. In some embodiments, Figure 2 As shown, the process 200 may include the following steps.

[0034] Step 220 : Based on the requirement information input by the user, obtain a first encryption vector corresponding to the requirement information through a preset encryption model. In some embodiments, step 220 may be implemented by the requirement encryption module 610 .

[0035] In some embodiments, the demand information input by the user may be content that is expected to be fed back and responded to by the large language model. The large language model can process the input demand information and generate the feedback results required by the user. For example, when the user needs to continue to pay attention to the latest diagnosis and treatment plans for a certain type of specific disease (such as gastric malignant tumors, etc.) in the field of clinical research, "check the latest diagnosis and treatment plans for gastric malignant tumors" can be input as demand information. The large language model can understand the above demand information and provide feedback to the user on the latest diagnosis and treatment plans for diseases related to gastric malignant tumors and the corresponding treatment results of each diagnosis and treatment plan. In some embodiments, the large language model (LLM) can be a type of natural language processing model based on deep learning that has been trained with a large amount of text data and has the ability to process and generate natural language text.

[0036] In some embodiments, the user can freely input the demand information according to their needs. The content of the demand information can specifically include text content, voice content, image content, etc., which is not limited here. In some embodiments, the user can input the demand information according to the input method supported by the processing device or terminal device deployed in process 200. For example, the user can use an external human-machine interface device such as a mouse and keyboard to input the demand information in text, or use an external human-machine interface device such as a microphone to input the demand information in voice, which is not limited here.

[0037] In some embodiments, the demand information input by the user can be encrypted by a preset encryption model to generate a corresponding first encryption vector. It is understandable that, considering that the demand information input by the user and the professional knowledge information stored in the enhanced generated knowledge base may involve confidential content that needs to be kept confidential, taking the clinical research field as an example, the demand information input by the user may contain confidential content such as the diagnosis and treatment plan and feasibility assessment of certain specific diseases; at the same time, in order to avoid the problem of processing hallucinations in the process of processing the demand information by the large language model, the professional knowledge information in the enhanced generated knowledge base applied in the clinical research field may also contain confidential content such as clinical trial plans and clinical guidelines for certain specific diseases. In some embodiments, in order to prevent the disclosure and / or leakage of confidential content in the process of generating feedback results for the demand information using the large language model, an encryption algorithm can be selected to encrypt and protect the demand information and professional knowledge information.

[0038] In some embodiments, symmetric and / or asymmetric algorithms, including the Advanced Encryption Standard (AES) algorithm, can be used to encrypt demand information and expertise information. In some embodiments, to further mitigate the increased inference latency of demand feedback systems using large language models due to the high computational complexity of traditional encryption and decryption algorithms, vectorized encoding can be used to encrypt demand information and / or expertise information. In some embodiments, a Transformer architecture can be used to implement vectorized encoding of demand information and / or expertise information. Because the Transformer architecture can accelerate matrix operations and has strong parallel computing capabilities, it can significantly improve the efficiency of encryption of demand information and / or expertise information. In some embodiments, considering that traditional encryption and decryption algorithms, such as the Rivest-Shamir-Adleman (RSA) algorithm and the AES algorithm, may be vulnerable to quantum computing attacks, for example, quantum computing can quickly crack large number factorization problems using Shor's algorithm, thereby deciphering traditional encryption and decryption algorithms. By using vectorized encoding, which does not rely on traditional mathematical algorithms, it can provide a certain degree of resistance to quantum computing attacks and further enhance the encryption security of expertise information. It is understood that by vectorizing the user's input requirement information, it is possible to ensure the transmission security of the requirement information while facilitating subsequent retrieval and matching with the encrypted knowledge vector corresponding to the professional knowledge information stored in the enhanced generated knowledge base associated with the large language model. The specific settings and training process of the preset encryption model will be explained in detail later and will not be detailed here.

[0039] In some embodiments, the demand information input by the user may include information with text as the content carrier, and the preset encryption model can generate a corresponding first encryption vector by vectorizing the text content. In some embodiments, when the demand information input by the user includes other content carriers besides text, the other content carriers besides text can be converted into text before vectorizing. For example, for the demand information input by the user through voice, the content of the voice input can be first recognized and converted into the corresponding text content, and then the converted text content can be vectorized using the preset encryption model. In some embodiments, when the input demand information contains two or more different forms of content carriers at the same time, the preset encryption model can be used independently for vectorizing according to the different content carriers. For example, when the demand information input by the user contains both text content and picture content, the text content and picture content can be vectorized using the preset encryption model respectively. This is not limited here.

[0040] Step 230 : Perform similarity matching between the first encrypted vector and the encrypted knowledge vector to determine the encrypted knowledge vector that matches the first encrypted vector as the second encrypted vector. In some embodiments, step 230 may be implemented by the knowledge matching module 620 .

[0041] In some embodiments, the encrypted knowledge vector is encrypted information obtained by vectorizing the professional knowledge information required by the large language model. Taking the clinical research field as an example, when the professional knowledge information is a clinical guideline, a single clinical guideline can be split into multiple text segments according to paragraphs or sections, and each text segment is vectorized to obtain a corresponding encrypted knowledge vector. In some embodiments, the encrypted knowledge vector can be obtained by vectorizing the professional knowledge information stored in the enhanced generated knowledge base associated with the large language model based on the encryption model used in the aforementioned step 220. It can be understood that when the encrypted knowledge vector and the first encrypted vector are obtained using the same encryption model based on vectorized encoding, if the encrypted knowledge vector and the first encrypted vector have a high degree of similarity, then the professional knowledge information corresponding to the encrypted knowledge vector and the demand information corresponding to the first encrypted vector also have a high degree of similarity. The professional knowledge information related to the demand information input by the user can be determined by matching the first knowledge vector with the encrypted knowledge vector for similarity.

[0042] In some embodiments, when matching the first encrypted vector with the encrypted knowledge vector for similarity, cosine similarity can be used as the similarity matching criterion. Cosine similarity is a metric used to measure the similarity between two vectors. The similarity between the two vectors is determined by calculating the cosine value of the angle between the two vectors. The value of cosine similarity ranges from [-1, 1]. In some embodiments, the cosine similarity between the first encrypted vector and all encrypted knowledge vectors can be obtained first, and then the encrypted knowledge vectors whose cosine similarity falls within a preset range can be determined as the second encrypted vector. For example, encrypted knowledge vectors whose cosine similarity is greater than 0.8 or greater than 0.9 can be determined as the second encrypted vector. Those skilled in the art can define an appropriate preset range based on actual needs, and this is not limited here. In some embodiments, the encrypted knowledge vectors can also be sorted from high to low according to cosine similarity, and the encrypted knowledge vectors with the highest ranking can be determined as the second encrypted vector. In some embodiments, the number of second encrypted vectors that are similar to and match the first encrypted vector can be one or more. When the number of second encrypted vectors is multiple, the professional knowledge information and demand information corresponding to the multiple second encrypted vectors can be combined together as input information of the large language model, which is not limited here.

[0043] In some embodiments, given the large number of encrypted knowledge vectors corresponding to the professional knowledge information, an IVF (Inverted File Index) algorithm can be employed to improve the efficiency of retrieval and matching between the first encrypted vector and the encrypted knowledge vector. Specifically, the encrypted knowledge vector can be divided into multiple clusters using a clustering algorithm, each containing similar encrypted knowledge vectors. An inverted index is created for each cluster to record the cluster to which the encrypted knowledge vector belongs. During the retrieval and matching process, the similarity between the first encrypted vector and the central vector of each cluster can be queried, and several clusters with high similarity can be selected for further in-depth retrieval and matching to optimize the retrieval and matching process between the first encrypted vector and the encrypted knowledge vector. In some embodiments, an HNSW (Hierarchical Navigable Small World) algorithm can be employed to improve the efficiency of retrieval and matching between the first encrypted vector and the encrypted knowledge vector. Specifically, a multi-level graph structure can be constructed based on the encrypted knowledge vector, where each level contains a different number of nodes. The bottom level contains the data points corresponding to all the encrypted knowledge vectors, while the upper level is a sparse graph that facilitates fast navigation. This can also optimize the retrieval and matching process between the first encrypted vector and the encrypted knowledge vector. Those skilled in the art may also select other appropriate vector search and matching algorithms according to actual needs, which is not limited here.

[0044] In some embodiments, the encrypted knowledge vector can be stored independently of the enhanced generation knowledge base associated with the large language model, that is, there is a physical isolation between the storage location of the encrypted knowledge vector and the enhanced generation knowledge base; and in the entire process 200 of implementing the encrypted enhanced generation method, the user input demand information and / or other related data streams do not involve the enhanced generation knowledge base storing professional knowledge information during the transmission and processing process, that is, the enhanced generation knowledge base and the plaintext professional knowledge information stored therein do not participate in the implementation process of process 200. In some embodiments, a malicious attacker cannot directly access the enhanced generation knowledge base or obtain the professional knowledge information stored in the enhanced generation knowledge base during the implementation of the above process 200, which can ensure that the professional knowledge information stored in the enhanced generation knowledge base is not illegally obtained. The specific settings and training process of the decryption model will be described in detail later and will not be repeated here.

[0045] Step 240 : Based on the second encryption vector and the user key corresponding to the user, obtain the professional knowledge information corresponding to the second encryption vector through a decryption model corresponding to the user key. In some embodiments, step 240 can be implemented by the knowledge decryption module 630 .

[0046] In some embodiments, the user key is a code information used to decrypt the second encrypted vector obtained in the aforementioned process step, so that the second encrypted vector can be restored to the corresponding professional knowledge information by the corresponding decryption model for use by the large language model. In some embodiments, there is a one-to-one binding relationship between the user key and the user who inputs the required information, that is, each user who is allowed to use the user key is a user who is allowed to use the user key. Figure 1 Each user of the demand feedback system 100 using the large language model shown in FIG. 1 has a corresponding user key, and there are differences between the user keys corresponding to different users. In some embodiments, when a user registers a request using Figure 1 In the process of the demand feedback system 100 that applies the large language model shown, a corresponding user key can be assigned to the user in the initial registration stage. In some embodiments, the assigned user key can be perceptible to the user. For example, when the user needs to access the demand feedback system 100, the user's identity authentication and verification of the user's access operation can be achieved by entering the user key, and the demand feedback system 100 can obtain the user key synchronously. In some embodiments, the assigned user key can be imperceptible to the user. For example, a mapping table including the correspondence between users and user keys can be stored in the demand feedback system 100. When the user needs to access the demand feedback system 100, the demand feedback system 100 can perform identity authentication based on the login information provided by the user and automatically obtain the user's corresponding user key. In some embodiments, the user key can also be entered by the user when entering the demand information. For example, the user can enter the user key additionally after completing the input of all demand information. This is not limited here. In some embodiments, the user key can be generated and obtained using different basic algorithms. This is not limited here.

[0047] In some embodiments, there is a one-to-one binding relationship between the user key and the decryption model, that is, each user key is allowed to use Figure 1Each user of the demand feedback system 100 that applies the large language model shown has a corresponding decryption model, and there are differences between the decryption models corresponding to different users. In some embodiments, if a malicious attacker attempts to obtain the professional knowledge information that needs to be encrypted corresponding to the second encryption vector by stealing the user key, even if the malicious attacker successfully steals the user key, it is only necessary to invalidate the decryption model corresponding to the stolen user key to prevent the malicious attacker from obtaining the professional knowledge information corresponding to the second encryption vector. In some embodiments, when it is determined that the first user key is stolen, the corresponding first decryption model can be obtained based on the stolen first user key, and the first decryption model can be invalidated to prevent a third party from using the stolen first user key to decrypt the encrypted knowledge vector and thereby obtain the professional knowledge information. The specific settings and training process of the decryption model will be described in detail later and will not be repeated here.

[0048] Step 250: Based on the expertise information and the requirements information, the large language model generates feedback results corresponding to the requirements information. In some embodiments, step 250 can be implemented by the requirements feedback module 640. It is understood that the introduction of expertise information can help the large language model avoid processing illusions when generating feedback results corresponding to the requirements information. Furthermore, the expertise information is encrypted and vectorized before being applied by the large language model, ensuring the privacy and security of the expertise information during the large language model's application of the requirements information and the generation of feedback results.

[0049] In some embodiments, as Figure 2The illustrated process 200 may also include step 210: validating the large language model generation request input by the user based on a preset license key, and proceeding to step 220 if the validation passes. It is understood that before vectorizing the user's inputted requirement information, the user's eligibility to access the large language model and provide the requirement information to the large language model may be verified. In some embodiments, the large language model generation request in step 210 may include a user key and the requirement information. The large language model may use its preset license key to validate the user information, such as the user key, included in the large language model generation request. In some embodiments, a decryption model may be matched against the user key included in the large language model generation request. If a decryption model matches the user key, the validation passes; if no decryption model matches the user key, the validation fails. Those skilled in the art may select an appropriate validation method, such as symmetric encryption and decryption verification, to implement validation, without limitation. In some embodiments, for large language model generation requests that have passed legality verification, the large language model generation requests can also be recorded in the form of logs, where the logs may specifically include the initiation timestamp corresponding to the large language model generation request, the initiating user information corresponding to the large language model generation request, the demand information corresponding to the large language model generation request, and other content. There is no limitation here, and the initiation and operation of the large language model generation request can be audited and traced through the logs.

[0050] Based on the relevant descriptions of the aforementioned embodiments, the security of the professional knowledge information required for the large language model in the content generation process can be achieved. The following will further elaborate and illustrate the specific settings and training process of the encryption model and decryption model involved in the above process 200 in combination with the embodiments.

[0051] Figure 3 is an exemplary flow chart of a preset process of an encryption model according to some embodiments of this specification. In some embodiments, Figure 3 As shown, the process 300 may include the following steps.

[0052] Step 310: Perform full parameter adjustment training on the preset natural language processing model so that the input data distribution and output data distribution of the natural language processing model are consistent.

[0053] It is understandable that the encryption model and decryption model involved in the above process 200 can first convert professional knowledge information into a high-dimensional vector, and then restore the corresponding professional knowledge information from the high-dimensional vector. The entire training process can be regarded as a complete self-supervised learning training process. The corresponding encryption model and decryption model can constitute a complete natural language processing (NLP) model, wherein the encryption model can be the part before the vector representation stage (Embedding) in the natural language processing model, and the decryption model is the part after the vector representation stage in the natural language processing model. In some embodiments, parameter adjustment training can be performed on a preset natural language processing model, and the part before the vector representation stage of the natural language processing model that completes the first parameter adjustment training is used as the preset encryption model, and the part after the vector representation stage of the natural language processing model that completes the second parameter adjustment training is used as the preset decryption model. In some embodiments, the preset natural language processing model can be the open source Qwen large model, which is based on the Transformer architecture and supports pre-training of large-scale text data and full parameter adjustment for specific tasks. Those skilled in the art can also choose other suitable natural language processing models, which are not limited here.

[0054] In some embodiments, the training process for the encryption model involves full parameter adjustment training of the natural language processing model, with the input training data set as For example, the output of the natural language processing model can be expressed as ,in The part before the vector representation stage preset for the natural language processing model, that is, the preset encryption model part with trainable parameters; The part after the vector representation stage preset for the natural language processing model, that is, the preset decryption model part with trainable parameters; is the output of the natural language processing model. It should be noted that since the decryption model is obtained from the second training, the parameters of the decryption model obtained in this training are In order to ensure that the encryption model and decryption model can achieve vectorized encoding and restoration of professional knowledge information, it is required that the input data distribution and output data distribution after full parameter adjustment training are consistent, that is, The distribution and In some embodiments, in order to achieve The distribution and The distribution of is consistent, and the optimization target can be set as the combination of the natural language processing model loss function and the encryption model loss function in the full parameter adjustment training, where for a single sample The natural language processing model loss function can be expressed as ,in For samples Output The rth token is in The predicted probability of For samples The input content consists of the first r tokens in ), for The length (i.e. At the same time, in order to make the output of the encryption model part quantitatively measure the semantic similarity in the feature space, it is necessary to Imposing semantic embedding constraints on a single sample The encryption model loss function can be expressed as ,in Represents the encryption model for the sample The vector represents the output, Represents a sample in a training batch Positive samples (where positive samples are randomly occluded or enhanced generated samples ), Represents a sample in a training batch Negative samples (where negative samples are samples in a training batch except other samples except , used to represent vectors and The cosine similarity of is a temperature hyperparameter that controls the smoothness of the distribution, is the total number of samples in a training batch. The total loss can be expressed as ,in For control and Hyperparameters for target balance. In some embodiments, any reasonable setting of the training loss function falls within the scope of protection of this specification. In some embodiments, a gradual unfreezing algorithm and a discriminative fine-tuning algorithm can be used in the full parameter adjustment training. Specifically, in the first five training epochs, by fixing the parameters of the first ten layers of the natural language model, only the later layers are trained to speed up the convergence of the model; from the subsequent ten training epochs, the fixed first ten layers of parameters in the natural language model are unfrozen layer by layer to ensure the stability of the model; during the unfreezing process, the learning rate of each layer decreases in a certain proportion, usually starting from the top of the model (i.e., the layer farthest from the input) to the bottom (i.e., the layer closest to the input). As an example, the learning rate of each layer is reduced by 2.6 times compared to the previous layer. In some embodiments, the Qwen large model can select the pre-trained open source large model Qwen2.5-0.5B. In some embodiments, the training data participating in the full parameter adjustment training may include professional knowledge content in the enhanced knowledge database, relevant knowledge content in related technical fields obtained through public search engines such as Wikipedia, and Chinese translations of relevant knowledge content, etc., which are not limited here.

[0055] In some embodiments, since the high-dimensional vector obtained by the encryption model needs to have a good semantic representation in order to perform retrieval and matching between the first encrypted vector and the encrypted knowledge vector, the relatedness standard can be used to evaluate the training results of the encryption model, that is, to evaluate the similarity between two segments of text after vectorization encoding through the encryption model. The similarity between two similar segments of text after vectorization encoding needs to be higher, and the similarity between two unrelated segments of text after vectorization encoding needs to be lower. Specifically, the cosine similarity can be used to judge the similarity, which is not limited here.

[0056] Step 320: Determine the portion of the natural language processing model before the vector representation stage after full parameter adjustment training as the encryption model. It is understandable that the portion of the natural language processing model before the vector representation stage after full parameter adjustment training is determined as the encryption model and fixed so that the portion before the vector representation stage of the natural language processing model can be directly used in the subsequent training process of the decryption model.

[0057] Figure 4 is an exemplary flow chart of a preset process of a decryption model according to some embodiments of this specification. In some embodiments, Figure 4 The process 400 shown is as follows Figure 3 In some embodiments, as shown in FIG300, Figure 4As shown, process 400 may include the following steps.

[0058] Step 410: Introduce randomly initialized user feature vectors to the part after the vector representation stage in the natural language processing model after full parameter adjustment and perform local parameter adjustment training to ensure that the input data distribution and output data distribution of the natural language processing model remain consistent.

[0059] In some embodiments, the training process for the decryption model involves adjusting the local parameters of the natural language processing model, with the input training data set as For example, the output of the natural language processing model can be expressed as ,in The encryption model part with fixed parameters obtained after full parameter adjustment training in the aforementioned process 300; The part after the vector representation stage preset for the natural language processing model, that is, the preset decryption model part with trainable parameters; is the output of the natural language processing model; The randomly initialized parameters of the introduced trainable user feature vector are used to ensure user diversity and vector randomness. In some embodiments, in order to achieve The distribution and The distribution of is consistent, and the optimization target can be set as the loss function of the natural language processing model in the local parameter adjustment training, that is, ,at this time Output for the sample The rth token is in The predicted probability of For samples The input content consists of the first r tokens in The length (i.e. The number of tokens in is the total number of samples in a training batch. In some embodiments, any reasonable setting of the training loss function falls within the scope of protection of this specification. In some embodiments, It can also be a series of user feature vectors preset according to the rules, which is not limited here. In some embodiments, the user feature vector Can be added to the decryption model Before any layer input in , and the user feature vector Add to decryption model The decryption model formed after fine-tuning the local parameters of any layer in the above formula falls within the scope of protection of this specification. In some broader examples, the decryption process may not be limited to the formula For example, if the user chooses to use LORA to fine-tune the decryption model, the decryption model after fine-tuning The output of the layer is ,in and The decryption model obtained after the first full-parameter fine-tuning training in the aforementioned process 300 No. The parameters of the linear layer, and The decryption model obtained after the second LORA fine-tuning training No. The low-rank update term of the layer, , at this time the low-rank update term of any layer or any combination of layers is Both can be used as user feature vectors Use, S is the decryption model obtained after the second LORA fine-tuning training The decryption model formed after the above LORA fine-tuning training falls within the scope of protection of this specification. In order to ensure that the decryption model and the decryption model can realize the vectorized encoding and restoration of professional knowledge information, it is required that the input data distribution after local parameter adjustment is consistent with the output data distribution, that is, The distribution and The distribution remains consistent.

[0060] Step 420: The portion of the natural language processing model trained after local parameter adjustment, after the vector representation stage, is determined as the decryption model. The decryption model corresponds one-to-one with the user feature vector, and the user feature vector serves as the user key. It will be appreciated that the decryption model obtained after local parameter adjustment training has a one-to-one correspondence with the user feature vector (i.e., the user key). This means that only when the user key is correct can the decryption model restore the encrypted knowledge vector to the corresponding professional knowledge information, effectively ensuring the data security of the professional knowledge information during the application of the large language model.

[0061] One or more embodiments of this specification also provide an encryption enhancement generation system using a large language model. Figure 5 This is a data flow diagram of an encryption enhancement generation system using a large language model according to some embodiments of this specification. In some embodiments, Figure 5 As shown, the encryption enhancement generation system 500 using a large language model may include a first server 510 and a second server 520 .

[0062] In some embodiments, the first server 510 can be configured to obtain a first encrypted vector corresponding to the requirement information input by the user using a preset encryption model. In some embodiments, the first server 510 can also be configured to perform a similarity match between the first encrypted vector and an encrypted knowledge vector, thereby determining and outputting the encrypted knowledge vector that matches the first encrypted vector as a second encrypted vector. In some embodiments, the first server 510 includes storage space for storing the encrypted knowledge vector. In some embodiments, the first server 510 can be a server provided by a service provider that provides professional knowledge information.

[0063] In some embodiments, the second server 520 is independent of the first server 510 and is configured to obtain the expertise information corresponding to the second encryption vector based on the second encryption vector and the user's corresponding user key using a decryption model corresponding to the user key. In some embodiments, the large language model is hosted on the second server 520 and is capable of generating corresponding feedback results based on the expertise information and demand information and providing feedback to the user. In some embodiments, the second server may be a third-party server providing large language model demand feedback services.

[0064] In some embodiments, as Figure 5 As shown, the encrypted enhancement generation system 500 using a large language model may further include an enhancement generation knowledge base 530 for storing professional knowledge information. In some embodiments, the first server 510 includes an encrypted knowledge vector library, which is used to obtain and store encrypted knowledge vectors based on the professional knowledge information stored in the enhancement generation knowledge base 530 using an encryption model installed on the first server. In some embodiments, the encrypted knowledge vector library and the enhancement generation knowledge base 530 are separated and independently deployed.

[0065] Some embodiments of this specification also provide an encryption enhancement generation system using a large language model. Figure 6 This is a functional module diagram of an encryption enhancement generation system using a large language model according to some embodiments of this specification. In some embodiments, Figure 6 As shown, the encryption enhancement generation system 600 using a large language model may include a requirement encryption module 610, a knowledge matching module 620, a knowledge decryption module 630, and a requirement feedback module 640. In some embodiments, the various modules of the encryption enhancement generation system 600 using a large language model may be deployed in the first server 510 and / or the second server 520. In some embodiments, the various modules of the encryption enhancement generation system 600 using a large language model may be configured in the processing device 110.

[0066] In some embodiments, the requirement encryption module 610 can perform vectorized encryption on the requirement information input by the user to obtain a corresponding encryption vector, ensuring the transmission security of the requirement information while facilitating subsequent retrieval and matching with the encrypted knowledge vector. In some embodiments, the requirement encryption module 610 can be configured to obtain a first encryption vector corresponding to the requirement information input by the user using a preset encryption model. In some embodiments, the requirement encryption module 610 can be deployed in the first server 510.

[0067] In some embodiments, the knowledge matching module 620 can match the encrypted vector corresponding to the user-entered requirement information with the encrypted knowledge vector corresponding to the professional knowledge information stored in the enhanced generated knowledge base associated with the large language model to obtain an encrypted knowledge vector corresponding to the professional knowledge information associated with the requirement information. In some embodiments, the knowledge matching module 620 can be configured to perform a similarity match between the first encrypted vector and the encrypted knowledge vector to determine the encrypted knowledge vector that matches the first encrypted vector as the second encrypted vector. In some embodiments, the knowledge matching module 620 can be deployed in the first server 510.

[0068] In some embodiments, the knowledge decryption module 630 can be used to decrypt the second encrypted vector obtained by the knowledge matching module 620 and matching the requirement information to obtain the professional knowledge information corresponding to the second encrypted vector. This professional knowledge information can be used by the large language model in the subsequent feedback result generation process. In some embodiments, the knowledge decryption module 630 can obtain the professional knowledge information corresponding to the second encrypted vector based on the second encrypted vector and the user's corresponding user key using a decryption model corresponding to the user key. In some embodiments, the knowledge decryption module 630 can be deployed on the second server 520.

[0069] In some embodiments, the demand feedback module 640 can generate the feedback results required by the user through the large language model based on the demand information input by the user. In some embodiments, in order to avoid the problem of a large amount of professional knowledge and / or academic terms in the demand information input by the user, which causes the large language model to have processing hallucinations during the processing process, the demand feedback module 640 can also use the professional knowledge information associated with the demand information provided by the aforementioned knowledge decryption module 630 to assist in the generation of feedback results during the process of the large language model processing the demand information. In some embodiments, the demand feedback module 640 can be used to generate feedback results corresponding to the demand information through the large language model based on the demand information and the professional knowledge information matching the demand information. In some embodiments, the demand feedback module 640 can be deployed in the second server 520.

[0070] For more information about each module, see Figures 2 to 5 The relevant description of will not be repeated here. It should be understood that Figure 6 The illustrated systems and their modules can be implemented in various ways. For example, in some embodiments, the systems and their modules can be implemented using hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic, while the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated hardware. Those skilled in the art will appreciate that the methods and systems described above can be implemented using computer-executable instructions and / or control code contained in a processor, such as provided on a carrier medium such as a disk, CD, or DVD-ROM, or in the memory of a programmable device. The systems and their modules described herein can be implemented not only using hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips or transistors, or programmable hardware devices such as field programmable gate arrays or programmable logic devices, but can also be implemented using software executed by various types of processors, or a combination of the aforementioned hardware circuits and software (e.g., firmware).

[0071] It should be noted that the above description of the system and its modules is for convenience only and does not limit this specification to the scope of the illustrated embodiments. It is understood that those skilled in the art, after understanding the principles of the system, may, without departing from these principles, arbitrarily combine the modules to form subsystems connected to other modules. Alternatively, they may split certain modules to obtain more modules or multiple units within a module. Such variations are within the scope of this specification.

[0072] Some embodiments of this specification also provide an encryption enhancement generation device using a large language model, which includes a processor and a storage medium, wherein the storage medium stores computer program instructions, and the processor is used to execute at least part of the computer program instructions to implement the encryption enhancement generation method using a large language model provided in the aforementioned embodiments of this specification.

[0073] Some embodiments of this specification also provide a computer program product, including computer instructions or a computer program. When at least part of the computer instructions or computer program is executed by a processor, it can implement the encryption enhancement generation method using a large language model provided in the aforementioned embodiments of this specification.

[0074] In some embodiments, the processor may be a combination of one or more of the following processors: a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a programmable logic controller (PLC), a reduced instruction set computer (RISC), a microprocessor, etc. In some embodiments, the processor may be the processing device 110.

[0075] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) By introducing professional knowledge information, the large language model can be helped to avoid the problem of processing illusions in the process of generating feedback results corresponding to demand information; at the same time, the professional knowledge information is in a vectorized encoded encrypted state before being applied by the large language model, which can ensure the privacy and security of the professional knowledge information during the process of applying the large language model to process demand information and generate feedback results. (2) The encryption of demand information and / or professional knowledge information is achieved by using vectorized encoding, which can significantly improve the encryption / decryption computational efficiency of demand information and / or professional knowledge information. (3) The encrypted knowledge vector corresponding to the professional knowledge information is physically isolated from the professional knowledge information in the storage location, which can avoid direct access to the storage location of the professional knowledge information during the application of the large language model, thereby improving the storage security of the professional knowledge information. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other possible beneficial effects.

[0076] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are taught in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

Claims

1. A method for generating encrypted enhanced data using a large language model, characterized in that: The method comprises: Based on the demand information input by the user, a first encryption vector corresponding to the demand information is obtained through a preset encryption model; performing similarity matching between the first encrypted vector and an encrypted knowledge vector, so as to determine the encrypted knowledge vector that matches the first encrypted vector as a second encrypted vector; Based on the second encryption vector and a user key corresponding to the user, obtaining professional knowledge information corresponding to the second encryption vector using a decryption model corresponding to the user key; Based on the professional knowledge information and the demand information, a feedback result corresponding to the demand information is generated through a large language model; wherein the preset process of the encryption model includes: Perform full parameter adjustment training on a preset natural language processing model so that the input data distribution and output data distribution of the natural language processing model are consistent; Determine the portion before the vector representation stage in the natural language processing model after the full parameter adjustment training as the encryption model; The preset process of the decryption model includes: Introducing randomly initialized user feature vectors into the part after the vector representation stage in the natural language processing model after the full parameter adjustment and performing local parameter adjustment training so that the input data distribution and output data distribution of the natural language processing model are consistent; Determining the portion after the vector representation stage in the natural language processing model after the local parameter adjustment training as the decryption model, wherein the decryption model corresponds one-to-one to the user feature vector; and The user feature vector is used as the user key.

2. The encryption enhancement generation method using a large language model according to claim 1, characterized in that: The encrypted knowledge vector is obtained by performing vectorized encryption processing on the encryption model based on the professional knowledge information stored in the enhanced generation knowledge base associated with the large language model; The encrypted knowledge vector is stored independently of the enhanced generation knowledge base.

3. The encryption enhancement generation method using a large language model according to claim 1, characterized in that: The method further comprises: Performing a legitimacy verification on a large language model generation request input by a user based on a preset license key, wherein the large language model generation request includes the user key and the requirement information; When the legality verification passes, the first encryption vector corresponding to the requirement information is obtained through the encryption model.

4. The encryption enhancement generation method using a large language model according to claim 1, characterized in that: The matching the first encrypted vector with an encrypted knowledge vector to determine the encrypted knowledge vector matching the first encrypted vector as a second encrypted vector includes: Obtaining a cosine similarity between the first encrypted vector and the encrypted knowledge vector; The encrypted knowledge vector whose cosine similarity falls within a preset range is determined as the second encrypted vector.

5. An encryption enhancement generation system using a large language model, characterized in that: The system comprises: The first server is configured to obtain, based on the requirement information input by the user, a first encryption vector corresponding to the requirement information using a preset encryption model; and to perform similarity matching between the first encryption vector and an encrypted knowledge vector, thereby determining the encrypted knowledge vector that matches the first encryption vector as a second encryption vector and outputting the second encryption vector; a second server configured to obtain, based on the second encryption vector and a user key corresponding to the user, professional knowledge information corresponding to the second encryption vector by using a decryption model corresponding to the user key; The large language model is installed on the second server and is used to generate corresponding feedback results based on the professional knowledge information and the demand information; The encryption model presetting process includes: Perform full parameter adjustment training on a preset natural language processing model so that the input data distribution and output data distribution of the natural language processing model are consistent; Determine the portion before the vector representation stage in the natural language processing model after the full parameter adjustment training as the encryption model; The preset process of the decryption model includes: Introducing randomly initialized user feature vectors into the part after the vector representation stage in the natural language processing model after the full parameter adjustment and performing local parameter adjustment training so that the input data distribution and output data distribution of the natural language processing model are consistent; Determining the portion after the vector representation stage in the natural language processing model after the local parameter adjustment training as the decryption model, wherein the decryption model corresponds one-to-one to the user feature vector; and The user feature vector is used as the user key.

6. The encryption enhancement generation system using a large language model according to claim 5, characterized in that: The system further comprises: Enhance the generated knowledge base to store the professional knowledge information; The first server includes an encrypted knowledge vector library, which is used to obtain and store the encrypted knowledge vectors through the encryption model based on the professional knowledge information stored in the enhanced generation knowledge base; The encrypted knowledge vector library and the enhanced generation knowledge library are separated from each other and deployed independently.

7. An encryption enhancement generation system using a large language model, characterized in that: The system comprises: A demand encryption module, configured to obtain a first encryption vector corresponding to the demand information input by a user through a preset encryption model; a knowledge matching module, configured to perform similarity matching between the first encrypted vector and an encrypted knowledge vector, so as to determine the encrypted knowledge vector that matches the first encrypted vector as a second encrypted vector; a knowledge decryption module, configured to obtain, based on the second encryption vector and a user key corresponding to the user, professional knowledge information corresponding to the second encryption vector by using a decryption model corresponding to the user key; A demand feedback module, configured to generate a feedback result corresponding to the demand information through a large language model based on the professional knowledge information and the demand information; The encryption model presetting process includes: Perform full parameter adjustment training on a preset natural language processing model so that the input data distribution and output data distribution of the natural language processing model are consistent; Determine the portion before the vector representation stage in the natural language processing model after the full parameter adjustment training as the encryption model; The preset process of the decryption model includes: Introducing randomly initialized user feature vectors into the part after the vector representation stage in the natural language processing model after the full parameter adjustment and performing local parameter adjustment training so that the input data distribution and output data distribution of the natural language processing model are consistent; Determining the portion after the vector representation stage in the natural language processing model after the local parameter adjustment training as the decryption model, wherein the decryption model corresponds one-to-one to the user feature vector; and The user feature vector is used as the user key.

8. A computer program product comprising computer instructions or a computer program, characterized in that When at least part of the computer instructions or the computer program is executed by a processor, the encryption enhancement generation method using a large language model as claimed in any one of claims 1 to 4 can be implemented.

Images