Search management apparatus, computer system, and search management method

Abstract

To provide a retrieval management device capable of performing safe and efficient secret similarity retrieval.SOLUTION: The prompt generation server 200 stores a DB421 for retrieval for each piece of storage data-encrypted storage data-obtained by encrypting storage data-with a shared key, and encrypted storage embedded data-obtained by encrypting storage embedded data-indicating a feature of the storage data-with a retrievable encryption key. A prompt generation server 200 generates encrypted prompt embedded data obtained by encrypting prompt embedded data indicating characteristics of a user prompt with a searchable encryption key, and transmits the encrypted prompt embedded data to a data management server 400. The prompt generation server 200 receives encrypted associated data, which is encrypted stored data corresponding to the encrypted stored embedded data that is similar to the encrypted embedded data to be embedded, and decrypts the encrypted associated data with the common key to acquire the associated data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a search management device, a computer system, and a search management method. [Background technology]

[0002] The use of AI (Artificial Intelligence) is increasing in various fields, including finance, medicine, and manufacturing, and LLM (Large Language Model) is one of the most widespread applications of AI. A technology called RAG (Retrieval Augmented Generation) has attracted attention as an efficient way to utilize LLM. When a user issues a prompt to an LLM, RAG searches a pre-prepared search database for relevant data related to the prompt, and generates a new prompt based on the prompt and the relevant data, which is then input into the LLM. Using this technology can improve the quality of the data generated by the LLM.

[0003] Security measures are also important in RAG systems, which are RAG-compliant systems. In particular, when outsourcing the management of the search database, it is desirable from a security perspective to encrypt the embedded data used to search related data in addition to encrypting the data stored in the search database.

[0004] However, encrypting the stored data and embedded data makes searching difficult and adversely affects the operation of the RAG system. In response to this, Non-Patent Document 1 discloses a private Hamming distance calculation using homomorphic encryption. Using this private Hamming distance calculation enables private similarity search, where embedded data remains encrypted. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Masaya Yasuda, 4 others, “Secure Pattern Matching using Somewhat Homomorphic Encryption”, In ACM workshop on Cloud computing security workshop-CCSW 2013, ACM, 65-76, 2013 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the technology described in Non-Patent Document 1, a trusted third party who holds a private key must decrypt the search results when performing a private similarity search, which leaves a problem in operating the RAG system efficiently.

[0007] An object of the present disclosure is to provide a search management device, a computer system, and a search management method that enable secure and efficient confidential similarity searches. [Means for solving the problem]

[0008] A search management device according to one aspect of the present disclosure is a search management device that retrieves related data related to search target data from a database, generates encrypted stored data by encrypting the stored data to be stored in the database and encrypted stored embedded data by encrypting stored embedded data indicating characteristics of the stored data with a searchable encryption key for each of the stored data, stores these in the database, generates encrypted target embedded data by encrypting target embedded data indicating characteristics of the search target data with the searchable encryption key, and transmits this to a data management device that searches for encrypted stored data corresponding to the encrypted stored embedded data that is similar to the encrypted target embedded data, retrieves the encrypted stored data searched by the data management device as encrypted related data by encrypting the related data, and decrypts the encrypted related data to retrieve the related data. [Effects of the Invention]

[0009] According to the present invention, a secure and efficient private similarity search becomes possible. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating a configuration example of a RAG system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a user computer. [Figure 3] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a prompt generation server. [Figure 4] FIG. 2 is a block diagram illustrating an example of the hardware configuration of an embedding generation server. [Figure 5] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a data management server. [Figure 6] FIG. 2 is a block diagram showing an example of the hardware configuration of a generation AI server. [Figure 7] FIG. 10 is a sequence diagram illustrating an example of processing in a data storage phase. [Figure 8] FIG. 10 is a sequence diagram illustrating an example of processing in the generation AI usage phase. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the embodiments, components having the same functions are given the same reference numerals, and descriptions thereof may be omitted. It should be noted that the embodiments are merely examples for realizing the present disclosure and do not limit the technical scope of the present invention. [Example]

[0012] Fig. 1 is a block diagram showing an example configuration of an RAG system according to an embodiment of the present disclosure. In Fig. 1, the RAG system 10 is a computer system including a user computer 100, a prompt generation server 200, an embedding generation server 300, a data management server 400, and a generation AI server 500. The prompt generation server 200 is communicatively connected to each of the user computer 100, the embedding generation server 300, the data management server 400, and the generation AI server 500 via each of communication networks 601 to 604.

[0013] The processing performed by the RAG system 10 is divided into a data storage phase in which data is stored in the data management server 400, and a generation AI utilization phase in which the generation AI is utilized using the data stored in the storage phase. The data storage phase also includes an initial setup phase in which initial setup for storing data is performed, and an operation phase in which the data is actually stored.

[0014] The user computer 100 is a computer operated by a user who uses the RAG system 10.

[0015] In the data storage phase, the user computer 100 transmits storage data to the prompt generation server 200, which is data to be stored in the data management server 400. The storage data may include, for example, information used by the organization to which the user belongs. The storage data also functions as a candidate for related data, which will be described later.

[0016] In the generation AI usage phase, the user computer 100 sends a user prompt, which is an instruction from the user to the generation AI model, to the prompt generation server 200. The user computer 100 also receives generation data generated by the generation AI model in response to the user prompt from the prompt generation server 200 and presents it to the user.

[0017] The stored data and user prompts include, for example, text data, image data, and voice data. The user prompts are search target data for performing a similarity search, which will be described later.

[0018] The prompt generation server 200 is a computer that functions as a search management device that processes user prompts from the user computer 100 .

[0019] In the initialization phase of the data storage phase, the prompt generation server 200 generates a data encryption key, a searchable encryption key, and a random shuffle key.

[0020] The data encryption key is an encryption key for encrypting data to be stored in the data management server 400.

[0021] The searchable encryption key is an encryption key for performing searchable encryption, which encrypts embedded data for similarity searches in a searchable state. Searchable encryption will be described in detail later.

[0022] The random shuffle key is an encryption key for randomly shuffling each bit of the encrypted embedded data, which is obtained by encrypting the embedded data with the searchable encryption key. Random shuffling will be described in detail later.

[0023] In the operation phase of the data storage phase, the prompt generation server 200 transmits the stored data received from the user computer 100 to the embedding generation server 300, and receives stored embedded data, which is embedded data corresponding to the stored data, from the embedding generation server 300. The prompt generation server 200 also generates encrypted stored data by encrypting the stored data with a data encryption key, and encrypted stored embedded data by encrypting the stored embedded data with a searchable encryption key, and transmits the encrypted stored data and encrypted stored embedded data to the data management server 400.

[0024] In the generation AI usage phase, the prompt generation server 200 transmits the user prompt received from the user computer 100 to the embedding generation server 300, and receives prompt-embedded data, which is embedded data corresponding to the user prompt, from the embedding generation server 300. The prompt generation server 200 generates encrypted prompt-embedded data, which is the prompt-embedded data encrypted with a searchable encryption key, as a search query for searching related data, which is stored data related to the user prompt.

[0025] The prompt generation server 200 also sends a search query to the data management server 400 and obtains encrypted stored data corresponding to the search query from the data management server 400 as encrypted related data obtained by encrypting related data. The prompt generation server 200 decrypts the encrypted related data to obtain the related data, and generates an input prompt based on the obtained related data and the user prompt and transmits it to the generation AI server 500. The prompt generation server 200 then receives the generated data from the generation AI server 500 and transmits the received generated data to the user computer 100.

[0026] The embedding generation server 300 is a computer that functions as an embedding generation device that generates embedded data.

[0027] In the data storage phase, the embedding generation server 300 receives the stored data from the prompt generation server 200 , generates embedded data corresponding to the stored data, and transmits the embedded data to the prompt generation server 200 .

[0028] In the generation AI usage phase, the embedding generation server 300 receives a user prompt from the prompt generation server 200 , generates prompt-embedded data that is embedded data corresponding to the user prompt, and transmits the generated data to the prompt generation server 200 .

[0029] The embedded data is data that indicates the characteristics of the original data (stored data and user prompts), and the similarity between the original data is reflected in the similarity between the embedded data. The embedded data is described, for example, as a numeric vector.

[0030] The data management server 400 is a computer that functions as a data management device that manages stored data. In this embodiment, the data management server 400 has a search DB 421, which is a database that stores stored data.

[0031] In the data storage phase, the data management server 400 receives the encrypted stored data and the encrypted stored embedded data from the prompt generation server 200, associates them, and stores them in the search DB 421.

[0032] In the generation AI usage phase, the data management server 400 receives a search query (encrypted prompt-embedded data) from the prompt generation server 200, performs a similarity search to search the search DB 421 for encrypted stored embedded data that is similar to the search query, and sends the encrypted stored data corresponding to the encrypted stored embedded data that is the search result to the prompt generation server 200 as encrypted related data.

[0033] The generation AI server 500 is a computer that functions as a generation AI management device that includes a pre-trained generation AI model.

[0034] In the generation AI usage phase, the generation AI server 500 inputs the input prompt received from the prompt generation server 200 into the generation AI model, and transmits the generated data output from the generation AI model to the prompt generation server 200.

[0035] The configuration of the RAG system 10 shown in Figure 1 is merely an example and is not limited to this. For example, two or more computers shown in Figure 1 may be implemented by a single computer. For example, the prompt generation server 200 and the embedding generation server 300 may be implemented by a single computer, or the prompt generation server 200, the embedding generation server 300, and the generation AI server 500 may be implemented by a single computer.

[0036] Fig. 2 is a block diagram showing an example of the hardware configuration of a user computer 100. In Fig. 2, the user computer 100 is a computer having a processor 101, an auxiliary storage device 102, a memory 103, a display device 105, an input / output interface (I / F) 106, and a communication interface (I / F) 107, and each part is connected to each other via an internal signal line 104 such as a bus.

[0037] The processor 101 is, for example, a CPU (Central Processing Unit), and is a processing device that executes programs stored in the memory 103 and controls each device. The memory 103 includes a ROM (Read Only Memory), which is a non-volatile, non-temporary storage element, and a RAM (Random Access Memory), which is a volatile storage element. The ROM stores immutable programs (e.g., BIOS). The RAM is a high-speed, volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by the processor 101 and data used when the programs are executed.

[0038] The auxiliary storage device 102 is a large-capacity, nonvolatile, non-transitory storage device such as a magnetic storage device (HDD: Hard Disk Drive) or a flash memory (SSD: Solid State Drive), and stores programs to be executed by the processor 101 and data to be used when the programs are executed. That is, the programs are read from the auxiliary storage device 102, loaded into the memory 103, and then executed by the processor 101.

[0039] The input / output interface 106 is connected to input devices such as a keyboard and a mouse, and is an interface for inputting information through operator operations. The input / output interface 106 is also connected to output devices such as a display device 105 and a printer, and is an interface for converting the results of program execution into information in a format that can be viewed by the operator and outputting the information. The display device 105 displays the results of program execution output from the input / output interface 106. The display device 105 may be connected to the internal signal line 104 without going through the input / output interface 106, and this configuration is shown in the example in the figure.

[0040] The communication interface 107 is a network interface that controls communication with other devices in accordance with a predetermined protocol, and may also include a serial interface such as a USB (Universal Serial Bus).

[0041] In addition, some or all of the programs executed by the processor 101 may be provided to the user computer 100 from a removable medium (CD-ROM, flash memory, etc.) which is a non-transitory storage medium, or from an external computer equipped with a non-transitory storage device via the communication network 601, and stored in the non-volatile auxiliary storage device 102 which is a non-transitory storage medium. For this reason, the user computer 100 can be configured to have an interface for reading data from removable media. This also applies to the prompt generation server 200, the embedding generation server 300, the data management server 400, and the generation AI server 500.

[0042] The user computer 100 is a computer system that may be configured on a single physical computer or on multiple logically or physically configured computers, and may operate in separate threads on the same computer, or may operate on a virtual computer built on multiple physical computer resources. The same applies to the prompt generation server 200, the embedding generation server 300, the data management server 400, and the generation AI server 500.

[0043] Fig. 3 is a block diagram showing an example of the hardware configuration of the prompt generation server 200. In Fig. 3, the prompt generation server 200 is a computer having a processor 201, an auxiliary storage device 202, a memory 203, a display device 205, an input / output interface 206, and a communication interface 207, and each device is connected to each other via an internal signal line 204 such as a bus.

[0044] The hardware functions of the processor 201, auxiliary storage device 202, memory 203, internal signal line 204, display device 205, input / output interface 206 and communication interface 207 are similar to the hardware functions of the processor 101, auxiliary storage device 102, memory 103, internal signal line 104, display device 105, input / output interface 106 and communication interface 107, so their explanation will be omitted.

[0045] The processor 201 executes a program stored in the memory 203 to realize the following functional components: a key generation unit 211, an encryption processing unit 212, a decryption processing unit 213, a search query generation unit 214, a random shuffle unit 215, and a prompt generation unit 216.

[0046] The key generation unit 211 generates the above-mentioned data encryption key, encrypted stored embedded data, and random shuffle key.

[0047] In this embodiment, the key generation unit 211 uses a common key encryption method such as AES (Advanced Encryption Standard) as the encryption method used for data encryption, which is encryption of stored data. In this case, the same data encryption key (common key) is used for encryption and decryption of stored data.

[0048] Furthermore, the key generation unit 211 uses a common key-based searchable encryption method as the encryption method used for the searchable encryption that encrypts the embedded data. In this embodiment, the searchable encryption is used for similarity search to obtain related data related to the user prompt, and the distance between embedded data is used as the similarity used for the similarity search. The distance between embedded data is, for example, Euclidean distance or Hamming distance. The searchable encryption method will be described in more detail later.

[0049] The encryption processing unit 212 generates encrypted stored data by encrypting the stored data received from the user computer 100 using a data encryption key. The encryption processing unit 212 also generates encrypted stored embedded data by searchable encryption using a searchable encryption key for the stored embedded data received from the embedding generation server 300.

[0050] The decryption processing unit 213 decrypts the encrypted related data received from the data management server 400 using the data encryption key to obtain the related data.

[0051] The search query generation unit 214 generates encrypted prompt-embedded data as a search query by encrypting the prompt-embedded data received from the embedding generation server 300 using a searchable encryption key.

[0052] The random shuffle unit 215 randomly shuffles, using a random shuffle key, the encrypted stored embedded data and the search query generated by the encryption processing unit 212 and the search query generation unit 214. In this embodiment, a common key is used as the random shuffle key.

[0053] The prompt generation unit 216 generates an input prompt based on the user prompt received from the user computer 100 and the related data acquired by the decryption processing unit 213 .

[0054] In the process of generating an input prompt, only one piece of related data may be used, or multiple pieces of related data may be used. Furthermore, the input prompt may reflect the related data as is, or may reflect partial data extracted from the related data by keyword search or the like.

[0055] Fig. 4 is a block diagram showing an example of the hardware configuration of the embedding generation server 300. In Fig. 4, the embedding generation server 300 is a computer having a processor 301, an auxiliary storage device 302, a memory 303, a display device 305, an input / output interface 306, and a communication interface 307, and each device is connected to each other via an internal signal line 304 such as a bus.

[0056] The hardware functions of the processor 301, auxiliary storage device 302, memory 303, internal signal line 304, display device 305, input / output interface 306 and communication interface 307 are similar to the hardware functions of the processor 101, auxiliary storage device 102, memory 103, internal signal line 104, display device 105, input / output interface 106 and communication interface 107, so their explanation will be omitted.

[0057] The processor 201 executes a program stored in the memory 203 to realize an embedding generation unit 311, which is a functional component.

[0058] The embedding generation unit 311 generates, for example, encrypted stored embedded data and encrypted prompt embedded data (search query) that are embedded data corresponding to the stored data and input prompt received from the prompt generation server 200, respectively.

[0059] Fig. 5 is a block diagram showing an example of the hardware configuration of the data management server 400. In Fig. 5, the data management server 400 is a computer having a processor 401, an auxiliary storage device 402, a memory 403, a display device 405, an input / output interface 406, and a communication interface 407, and each device is connected to each other via an internal signal line 404 such as a bus.

[0060] The hardware functions of the processor 401, auxiliary storage device 402, memory 403, internal signal line 404, display device 405, input / output interface 406 and communication interface 407 are the same as the functional configurations of the hardware of the processor 101, auxiliary storage device 102, memory 103, internal signal line 104, display device 105, input / output interface 106 and communication interface 107, so descriptions of these will be omitted.

[0061] The processor 401 executes a program stored in the memory 403 to realize a data storage unit 411 and a similarity search unit 412, which are functional components.

[0062] The auxiliary storage device 402 holds a search DB 421 that stores the encrypted stored data and encrypted stored embedded data received by the communication interface 407 from the prompt generation server 200. The search DB 421 may also be held in the memory 403.

[0063] The data storage unit 411 stores, for example, the encrypted stored data and the encrypted stored embedded data received from the prompt generation server 200 in the search DB 421 for each encrypted stored data item.

[0064] The similarity search unit 412 performs a similarity search to search the search DB 421 for encrypted stored embedded data that is similar to the search query received from the prompt generation server 200, and transmits the encrypted stored data corresponding to the encrypted stored embedded data that is the search result to the prompt generation server 200 as encrypted related data.

[0065] Fig. 6 is a block diagram showing an example of the hardware configuration of a generation AI server. In Fig. 6, the generation AI server 500 is a computer having, for example, a processor 501, an auxiliary storage device 502, a memory 503, a display device 505, an input / output interface 506, and a communication interface 507, and each device is connected to each other via an internal signal line 504 such as a bus.

[0066] The hardware functions of the processor 501, auxiliary storage device 502, memory 503, internal signal line 504, display device 505, input / output interface 506 and communication interface 507 are similar to the hardware functions of the processor 101, auxiliary storage device 102, memory 103, internal signal line 104, display device 105, input / output interface 106 and communication interface 107, so their explanation will be omitted.

[0067] The processor 501 executes a program stored in the memory 503 to realize a generation processing unit 511, which is a functional configuration.

[0068] The generation processing unit 511 has a generation AI model, inputs input prompts received by the communication interface 407 from the prompt generation server 200 into the generation AI model, and transmits generated data output from the generation AI model to the prompt generation server 200.

[0069] The following describes in more detail the private similarity search, which is a similarity search using searchable encryption. Note that the distance between embedded data is the Hamming distance.

[0070] First, searchable encryption will be described.

[0071] Let M1 and M2 be arbitrary values, k be an encryption key, Enc be an encryption function, and Search be a search function for similarity search. In this case, for example, Enc(M, k) denotes the ciphertext obtained by encrypting value M with encryption key k, and Search(X, Y) denotes the result of a similarity search of ciphertext X with ciphertext Y. In searchable encryption, when M1 = M2, formula (1) holds, and when M1 ≠ M2, formula (2) holds. Search(Enc(M1, k), Enc(M2, k))=0...(1) Search(Enc(M1, k), Enc(M2, k))=1...(2) In equation (2), the case where Enc(M1, k) coincides with Enc(M2, k) when M1≠M2 is ignored because the probability of this occurring is extremely small.

[0072] Using the above equations (1) and (2), it can be determined whether or not the values ​​M1 and M2 match from the encrypted embedded data Enc(M1, k) and Enc(M2, k) of the values ​​M1 and M2, respectively.

[0073] Hereinafter, a secure Hamming distance calculation, which is a calculation of the Hamming distance using the above-mentioned searchable encryption, will be described.

[0074] The Hamming distance is defined for multiple pieces of data (bit strings) with the same number of bits, and is defined as the number of bit positions in the multiple pieces of data that have different values. For example, the Hamming distance between the bit strings "111" and "011" is 1, and between the bit strings "111" and "010" is 2.

[0075] In the private Hamming distance calculation, first, each bit of the embedding data is encrypted by a searchable cipher. For example, if the stored embedding data A is "110" and the prompt embedding data B is "101", the encrypted stored embedding data Enc(A) and the encrypted prompt embedding data Enc(B) are respectively: Enc(A)=Enc(1)||Enc(1)||Enc(0) Enc(B)=Enc(1)||Enc(0)||Enc(1) This becomes:

[0076] The Hamming distance is the number of bits whose values ​​match in the encrypted stored embedding data Enc(A) and the encrypted prompt embedding data Enc(B), and is calculated by applying the search function Search to each corresponding bit position in the encrypted stored embedding data Enc(A) and the encrypted prompt embedding data Enc(B). For example, in the case of the above stored embedding data A "110" and the prompt embedding data B "101", when the search function Search is applied, the following equations (3) to (5) are obtained: Search(Enc(1),Enc(1))=0...(3) Search(Enc(1),Enc(0))=1...(4) Search(Enc(0),Enc(1))=1...(5) Their Hamming distance is 2.

[0077] As a result, it is possible to perform a private Hamming distance calculation using a common-key-based searchable encryption. In this case, the data management server 400, for example, calculates the Hamming distance between the search query and each of the encrypted, stored, embedded data in the search DB 421 using the private calculation described above, and sets the encrypted, stored, embedded data whose Hamming distance satisfies a predetermined standard (for example, is less than a threshold) as the encrypted, stored, embedded data that is similar to the search query, thereby realizing a private similarity search.

[0078] However, in the above-described confidential similarity search, there is a risk that bits with matching values ​​may be leaked from the data management server 400 that calculates the Hamming distance. For example, in the case of the above-described embedded data A and B, there is a risk that the first bit is the same according to equation (3), and the second and third bits are different according to equations (4) and (5). Therefore, in this embodiment, random shuffling is used to conceal bits with matching values.

[0079] Specifically, the key generation unit 211 of the prompt generation server 200 generates a random shuffle key using a common key. The random shuffle unit 215 then uses the random shuffle key to perform a random shuffle, shuffling the bit values ​​of the encrypted stored embedded data Enc(A) and the encrypted prompt embedded data Enc(B). For example, when the random shuffle key is (2, 3, 1), the random shuffle is a transformation that moves the first element of the search function calculation result to the second, the second element to the third, and the third element to the first. The Hamming distance remains unchanged during this random shuffle.

[0080] For example, if the results of random shuffling the encrypted stored embedded data Enc(A) and the encrypted prompt embedded data Enc(B) are Shuffle(A) and Shuffle(B), respectively, then: Shuffle(A)=Enc(0)||Enc(1)||Enc(1) Shuffle(B)=Enc(1)||Enc(1)||Enc(0) If we perform a secret calculation of the Hamming distance on Shuffle(A) and Shuffle(B), we get Search(Enc(0),Enc(1))=1 Search(Enc(1),Enc(1))=0 Search(Enc(1),Enc(0))=1 Therefore, the Hamming distance is 2, which is the same as when no random shuffling is performed.

[0081] 7 is a sequence diagram for explaining an example of processing in the data storage phase of the RAG system 10. As described above, the data storage phase includes an initial setting phase P701 and an operation phase P702.

[0082] First, in the initial setup phase P701, the key generation unit 211 of the prompt generation server 200 generates a data encryption key (step S701), a searchable encryption key (step S702), and a random shuffle key (step S703). After that, the operation phase P702 begins.

[0083] In the operation phase P702, first, the user computer 100 transmits the data to be stored in the data management server 400 to the prompt generation server 200 (step S704).

[0084] The encryption processing unit 212 of the prompt generation server 200 receives the stored data and transmits the stored data to the embedding generation server 300 (step S705). The embedding generation unit 311 of the embedding generation server 300 receives the stored data and generates embedded data corresponding to the stored data (step S706). The embedding generation unit 311 transmits the embedded data to the prompt generation server 200 (step S707).

[0085] When the encryption processing unit 212 of the prompt generation server 200 receives the stored embedded data, it generates encrypted stored data by encrypting the stored data with the data encryption key (step S708). The encryption processing unit 212 encrypts the stored embedded data with the searchable encryption key to generate encrypted stored embedded data (step S709). The random shuffle unit 215 shuffles the value of each bit of the encrypted stored embedded data with the random shuffle key (step S710).

[0086] The encryption processing unit 212 transmits the encrypted storage data and the shuffled encrypted storage embedded data to the data management server 400 (step S711). Upon receiving the encrypted storage data and the encrypted storage embedded data, the data storage unit 411 of the data management server 400 stores the encrypted storage data and the encrypted storage embedded data in the search DB 421 (step S712), and ends the operation phase P702.

[0087] FIG. 8 is a sequence diagram for explaining an example of processing in the generation AI utilization phase of RAG system 10.

[0088] In the generation AI usage phase P801, first, the user computer 100 transmits a user prompt to the prompt generation server 200 (step S801). The search query generation unit 214 of the prompt generation server 200 receives the user prompt and transmits the user prompt to the embedding generation server 300 (step S802).

[0089] The embedding generation unit 311 of the embedding generation server 300 receives the user prompt and generates prompt embedding data, which is embedded data corresponding to the user prompt (step S803).The embedding generation unit 311 then transmits the prompt embedding data to the prompt generation server 200 (step S804).

[0090] The search query generation unit 214 of the prompt generation server 200 receives the prompt-embedded data and encrypts the prompt-embedded data with a searchable encryption key to generate encrypted prompt-embedded data as a search query (step S805). The random shuffle unit 215 shuffles the value of each bit of the search query with a random shuffle key (step S806). The search query generation unit 214 sends the shuffled search query to the data management server 400 (step S807).

[0091] When the similarity search unit 412 of the data management server 400 receives the search query, it performs a similarity search to search the search DB 421 for encrypted stored embedded data that is similar to the search query (step S808). The similarity search unit 412 retrieves encrypted stored data that corresponds to the searched encrypted stored embedded data from the search DB 421 as encrypted associated data and transmits it to the prompt generation server 200 (step S809).

[0092] The decryption processing unit 213 of the prompt generation server 200 receives the encrypted related data and decrypts it with the data encryption key to obtain the related data (step S810). The prompt generation unit 216 generates an input prompt based on the related data and the user prompt (step S811) and sends the input prompt to the generation AI server 500 (step S812).

[0093] When the generation processing unit 511 of the generation AI server 500 receives the input prompt, it inputs the input prompt into the generation AI model and obtains the generation data output from the generation AI model (step S813).

[0094] The generation processing unit 511 transmits the generated data to the prompt generation server 200 (step S814). Upon receiving the generated data, the prompt generation unit 216 of the prompt generation server 200 transmits the generated data to the user computer 100 (step S815). Upon receiving the generated data, the user computer 100 presents the generated data to the user (step S816), thereby terminating the generation AI usage phase P801.

[0095] According to the present embodiment described above, the prompt generation server 200 stores, for each piece of stored data, encrypted stored data, which is stored data encrypted with a common key, and encrypted stored embedded data, which is stored embedded data indicating the characteristics of the stored data encrypted with a searchable encryption key, in the search DB 421. The prompt generation server 200 generates encrypted prompt embedded data, which is prompt embedded data indicating the characteristics of the user prompt encrypted with a searchable encryption key, and sends the encrypted prompt embedded data to the data management server 400. The prompt generation server 200 receives encrypted related data, which is encrypted stored data corresponding to the encrypted stored embedded data that is similar to the embedded data to be encrypted, and decrypts the encrypted related data with the common key to obtain the related data.

[0096] In this case, only encrypted data (encrypted stored data and encrypted stored embedded data) needs to be stored in the data management server 400, and there is no need for a trusted third party with a private key to decrypt the search results of the data management server 400. This enables safe and efficient confidential similarity searches. This makes it possible to prevent information leaks due to malicious intent or negligence on the part of the administrator of the data management server 400, for example.

[0097] In this embodiment, the prompt generation server 200 generates an input prompt from the acquired related data and the user prompt, inputs the generated prompt into the generation AI model, and presents the generated data output from the generation AI model to the user. This enables secure and efficient confidential similarity search to be used for similarity search for search expansion generation, thereby enabling secure search expansion generation.

[0098] Furthermore, in this embodiment, the bits of the encrypted stored embedded data and the encrypted prompt embedded data are shuffled and transmitted to the data management server 400. This makes it possible to prevent information such as the positions where the bits of the encrypted stored embedded data and the encrypted prompt embedded data match from being leaked, thereby further improving security.

[0099] In this embodiment, the data management server 400 determines that the encrypted stored embedded data is similar to the embedded data to be encrypted if the distance between the encrypted stored embedded data and the embedded data satisfies a predetermined standard. The distance may be, for example, the Hamming distance. In this case, it becomes possible to search for more appropriate encrypted stored data.

[0100] In this embodiment, the embedding generation server 300 generates stored embedding data and prompt embedding data from the stored data and the user prompt. The prompt generation server 200 encrypts the stored embedding data and prompt embedding data generated by the embedding generation server 300. By using a reliable embedding generation server 300 such as your own server, the load on the prompt generation server 200 can be reduced while maintaining security.

[0101] It should be noted that the present disclosure is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present disclosure is not necessarily limited to those having all of the described configurations.

[0102] Furthermore, the above-described configurations, functional means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the above-described configurations, functions, etc. may be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD (Digital Versatile Disc).

[0103] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0104] 10: RAG system 100: User computer 200: Prompt generation server 300: Embedding generation server 400: Data management server 500: Generation AI server

Claims

1. A search management device that acquires related data related to search target data, generating encrypted stored data for each of the stored data, the encrypted stored data being the candidate for the related data, and the encrypted stored embedded data being the stored embedded data indicating the characteristics of the stored data being encrypted with a searchable encryption key, and storing the encrypted stored embedded data in a database; generating encrypted target embedded data by encrypting target embedded data indicating characteristics of the search target data with the searchable encryption key, and transmitting the encrypted target embedded data to a data management device that searches the database for encrypted stored data that corresponds to the encrypted stored embedded data similar to the encrypted target embedded data; The encrypted stored data searched by the data management device is acquired as encrypted related data obtained by encrypting the related data; The search management device decrypts the encrypted related data to obtain the related data.

2. 2. The search management device according to claim 1, The search target data is a user prompt from a user to a generated AI model; The search management device generating an input prompt to be input to the generative AI model from the user prompt and the associated data, and inputting the input prompt to the generative AI model; A search management device that presents generated data output from the generative AI model to the user.

3. 2. The search management device according to claim 1, In storing the encrypted stored data, each bit of the encrypted stored embedded data is shuffled using a shuffle key for shuffling each bit of data and stored in the data management device; The search management device shuffles each bit of the embedded data to be encrypted using the shuffle key and transmits the data to the data management device.

4. The search management device according to claim 1; A computer system comprising the data management device.

5. 5. The computer system of claim 4, The data management device regards the encrypted stored embedded data, whose distance from the embedded data to be encrypted satisfies a predetermined standard, as the encrypted stored embedded data similar to the embedded data to be encrypted.

6. 6. The computer system according to claim 5, The distance is a Hamming distance.

7. 5. The computer system of claim 4, The method further includes an embedding generation device that generates the stored embedding data and the target embedding data from the stored data and the search target data, A computer system in which the search management device encrypts the stored embedded data and the target embedded data generated by the embedding generation device with the searchable encryption key to generate the encrypted stored embedded data and the encrypted target embedded data.

8. A search management method by a search management device that acquires related data related to search target data, generating encrypted stored data for each of the stored data, the encrypted stored data being the candidate for the related data, and the encrypted stored embedded data being the stored embedded data indicating the characteristics of the stored data being encrypted with a searchable encryption key, and storing the encrypted stored embedded data in a database; generating encrypted target embedded data by encrypting target embedded data indicating characteristics of the search target data with the searchable encryption key, and transmitting the encrypted target embedded data to a data management device that searches the database for encrypted stored data that corresponds to the encrypted stored embedded data similar to the encrypted target embedded data; The encrypted stored data searched by the data management device is acquired as encrypted related data obtained by encrypting the related data; The search management method further comprises decrypting the encrypted related data to obtain the related data.

Images