Method and device for defending against prompt word stealing attacks based on security aspects

Abstract

The embodiments of this specification disclose a method for defending against prompt stealing attacks based on security aspects. The method is implemented based on aspect technology. By directly analyzing the response content and matching it with the prompts to be protected at the semantic level, it does not rely on detection rules based on prior knowledge, and can resist various attacks on requests. Bypass, it has a higher recall rate and robustness. The device for defending against prompt stealing attacks based on security aspects described in the embodiments of this specification also has the above-mentioned beneficial effects.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for defending against message theft attacks based on security aspect. Background Technology

[0002] In large language models, prompt theft attacks refer to adding malicious instructions to user input to steal the model's original prompts, which may lead to the exposure of sensitive information and potential abuse of prompts by unauthorized individuals.

[0003] To combat message theft attacks, common defense strategies involve analyzing user request content and blocking requests that attempt to steal the message. However, interception strategies based on prior knowledge are easily bypassed by attackers. Furthermore, filtering request content requires modifying the service's infrastructure, which is a significant undertaking. Summary of the Invention

[0004] One or more embodiments of this specification provide a method and apparatus for defending against prompt theft attacks based on security aspect, which can solve the problem of sensitive information leakage caused by prompt theft attacks at a very low cost.

[0005] Firstly, a method for defending against message theft attacks based on security aspects is provided, executed by a security aspect program deployed on a large model server. The method includes:

[0006] In response to a user's query request to the large model through the large model server, the first response content of the large model to the query request is intercepted;

[0007] The first response content is vectorized to obtain the first vector;

[0008] The first vector is used to perform a search in a vector database, which stores prompt vectors that need to be protected.

[0009] If the search finds a match, the preset defense strategy is executed; otherwise, the first response content is allowed to proceed.

[0010] As an optional implementation of the method described in the first aspect, the method further includes:

[0011] If the first vector search in the vector database fails to find a match, then the historical interaction records between the user and the large model are obtained, and the second response content is obtained from the historical interaction records.

[0012] Vectorize the content of the second response to obtain the second vector;

[0013] The second vector is used to perform a search in the vector database. If the search is successful, a preset defense strategy is executed; otherwise, the first response content is allowed.

[0014] Specifically, the second response content is vectorized to obtain a second vector, which includes:

[0015] Concatenate the second response content into a single string;

[0016] The string is vectorized to obtain the second vector.

[0017] As an optional implementation of the method described in the first aspect, the method further includes constructing the vector database, specifically including:

[0018] Obtain the notification message that requires protection;

[0019] The prompt message that needs to be protected is vectorized to obtain the prompt message vector;

[0020] The prompt vector is stored in the vector database.

[0021] As an optional implementation of the method described in the first aspect, the method further includes deploying the secure aspect ratio procedure on the large model server, specifically including:

[0022] Obtain the application of the large model server and the pointcut of the application;

[0023] The safe aspecting procedure is injected at the point of cut so that the safe aspecting procedure is executed when the application is executed up to the point of cut.

[0024] Secondly, a security aspect-based message theft defense device is provided, applied to a large model server. The large model server has a security aspect program deployed on it. The device is used to execute the business logic of the security aspect program. The device includes:

[0025] The data acquisition module is configured to intercept the first response content of the large model to the question request initiated by the user through the large model server.

[0026] The processing module is configured to vectorize the first response content to obtain a first vector;

[0027] The database module is configured to build a vector database, which stores the vector prompts that need to be protected;

[0028] The retrieval module is configured to perform a retrieval in the vector database based on the first vector;

[0029] The response module is configured to execute a preset defense strategy in response to the retrieval module finding a match in the vector database based on the first vector; and to allow the first response content to proceed in response to the retrieval module not finding a match in the vector database based on the first vector.

[0030] As an alternative embodiment of the apparatus described in the second aspect, wherein...

[0031] The data acquisition module is further configured to, in response to the retrieval module's failure to find a match in the vector database based on the first vector, acquire the user's historical interaction records with the large model, and acquire the second response content from the historical interaction records;

[0032] The processing module is further configured to vectorize the second response content to obtain a second vector;

[0033] The retrieval module is also configured to perform a retrieval in the vector database based on the second vector;

[0034] The response module is further configured to execute a preset defense strategy in response to the retrieval module finding a match in the vector database based on the second vector; and to allow the first response content to proceed in response to the retrieval module not finding a match in the vector database based on the second vector.

[0035] Specifically, the processing module is used for:

[0036] Concatenate the second response content into a single string;

[0037] The string is vectorized to obtain the second vector.

[0038] As an alternative embodiment of the apparatus described in the second aspect, wherein...

[0039] The data acquisition module is also configured to acquire the prompt message that needs to be protected;

[0040] The processing module is further configured to vectorize the prompt message that needs to be protected to obtain the prompt message vector, and store the prompt message vector in the vector database.

[0041] As an optional embodiment of the apparatus described in the second aspect, the apparatus further includes:

[0042] The aspect injection module is configured to obtain the application of the large model server and the pointcut of the application, and inject the safe aspect into the pointcut so that the safe aspect is executed when the application is executed up to the pointcut.

[0043] Thirdly, a computer-readable storage medium is provided, which stores a computer program that, when run on an electronic device, causes the electronic device to execute the aforementioned security aspect-based message theft defense method.

[0044] Fourthly, an electronic device is provided, comprising:

[0045] At least one memory for storing programs;

[0046] At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the aforementioned security aspect-based message theft defense method.

[0047] The beneficial effects of the security aspect-based message theft defense method described in the embodiments of this specification are that, based on aspect-based technology, this method does not require modification of the service's infrastructure, thus offering greater convenience. This method directly analyzes the response content and performs semantic-level matching with the message to be protected, without relying on detection rules based on prior knowledge. This allows it to resist various request-based bypass attacks, resulting in higher recall and robustness.

[0048] The security-faceted message theft defense device described in the embodiments of this specification also has the above-mentioned beneficial effects. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 An exemplary flowchart illustrates a security aspect-based message theft defense method described in an embodiment of this specification.

[0051] Figure 2 An exemplary flowchart illustrates another security aspect-based message theft defense method described in an embodiment of this specification.

[0052] Figure 3 An exemplary schematic diagram of a security aspect-based message theft defense device described in an embodiment of this specification is shown.

[0053] Figure 4 A schematic diagram of the structure of an electronic device described in an embodiment of this specification is shown as an example. Detailed Implementation

[0054] First, it should be noted that the terminology used in the embodiments of this invention is for the purpose of describing specific embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0055] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments in this specification, and not all of the embodiments. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0056] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0057] Large Language Models (LLMs) are deep learning models trained on massive amounts of text data that can generate natural language text or understand the meaning of language text. These models can perform a variety of natural language processing tasks, including but not limited to text classification, question answering, and dialogue.

[0058] In this system, a prompt is a question or statement provided by the user, used to guide the generation of a relevant response. Upon receiving a prompt, the system generates the most relevant subsequent content or answer based on its internally trained knowledge and algorithms. Prompts typically consist of "user input" and "instructions," for example, the prompt: "Please translate the following content into English 'I am the class monitor,'" where "I am the class monitor" is the "user input," and "Please translate the following content into English" is the instruction.

[0059] During use, AI may be vulnerable to prompt theft attacks. Prompt theft, also known as prompt leakage, is an attempt to manipulate output to reveal part or all of the original prompts. In AI, prompt leakage typically involves adding malicious instructions to user input to steal the model's original prompts. This can lead to the exposure of sensitive information and potential misuse of prompts by unauthorized individuals. Prompt theft attacks usually refer to system prompts, which are the initial set of instructions developers use to set boundaries for AI dialogue. This set includes rules to follow, topics to avoid, and how to format responses. These instructions may be inserted into the original prompts as messages before the user interacts with the AI. If attackers obtain these system prompts, they can analyze the AI's behavioral patterns or censorship mechanisms, and then manipulate the AI ​​without authorization, posing a serious risk of privacy breaches.

[0060] To defend against message theft attacks, a common defense strategy is to analyze user request content and block message theft requests that match expert rules. However, this prior knowledge-based blocking strategy is easily bypassed by attackers, and filtering request content requires modifying the service's infrastructure, which is labor-intensive and costly.

[0061] In summary, there is an urgent need for a secure, effective, and low-cost solution to defend against message theft attacks in the existing technology.

[0062] In view of this, one or more embodiments of this specification propose a method and apparatus for defending against message theft attacks based on security aspect, which can solve the problem of data leakage caused by message theft attacks at extremely low cost.

[0063] The following will provide a more detailed description of the security aspect-based message theft defense method and apparatus described in one or more embodiments of this specification, in conjunction with the accompanying drawings and specific examples. However, this detailed description does not constitute a limitation on the embodiments of this specification.

[0064] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a security aspect-based message theft defense method proposed in one or more embodiments of this specification.

[0065] like Figure 1 As shown, in some implementations, the security aspect-based message theft defense method can be executed by a security aspect program deployed on a large model server, and the method may include steps S100 to S106:

[0066] S100: In response to a user's query request initiated by the large model server to the large model, intercept the large model's first response to the query request.

[0067] S102: Vectorize the content of the first response to obtain the first vector.

[0068] S104: Perform a search in the vector database based on the first vector.

[0069] S106: If the search is successful, execute the preset defense strategy; otherwise, allow the first response content to pass.

[0070] It can be seen that, in Figure 1 The security aspect-based prompt theft attack defense method shown determines the presence of prompt theft attacks by vectorizing the response content of each round of dialogue between the user and the large model and performing semantic-level matching between this vectorized content and the prompts to be protected. This method does not rely on detection rules based on prior knowledge, thus resisting various request-based bypass attacks. Furthermore, the aforementioned security aspect-based prompt theft attack defense method is implemented through a security aspect program, requiring no modification to the large model service's infrastructure, resulting in lower cost and greater convenience.

[0071] The following description, in conjunction with the accompanying drawings and specific embodiments, will... Figure 1 The method shown will be explained.

[0072] Before implementing the aforementioned security aspect-based message theft defense method, initialization is required. The initialization phase primarily involves injecting the security aspect protocol into the large model server and constructing the vector database.

[0073] The aforementioned security aspect is used to intercept and process the response content returned to the user, thereby implementing the aforementioned security aspect-based message theft defense method. Specifically, the method for injecting the security aspect into the large model server is as follows: obtain the application of the large model server and the pointcut of the application, and then inject the pre-built security aspect into the pointcut so that the security aspect is executed when the application is executed up to the pointcut.

[0074] The aforementioned vector database is used to store the vectors of the prompts that need to be protected. During the initialization phase, the prompts that need to be protected can be collected, and then the prompts that need to be protected can be converted into vectors and saved to the vector database.

[0075] Once initialization is complete, the above-mentioned security aspect-based message theft defense method can be executed.

[0076] First, for step S100: In response to the user's query request initiated by the large model server to the large model, intercept the first response content of the large model to the query request.

[0077] This embodiment, through the setting of the pointcut, enables the security aspect program to intercept the response text to the question request when it detects a user's question request initiated by the server, and to use the single-round response text as the first response content.

[0078] Next, for step S102: vectorize the content of the first response to obtain the first vector.

[0079] To facilitate retrieval, the first response content needs to be processed into text vectorization in step S102. Text vectorization refers to representing text as a vector that can express the semantics of the text, which is a fundamental task in the field of natural language processing.

[0080] It should be noted that this embodiment does not limit the method of text vectorization processing of the first response content, and can be adaptively selected according to requirements. For example, the text vectorization processing method may include, but is not limited to: word embedding method, one-hot encoding method, bag-of-words model, word-vector model, etc.

[0081] Next, for step S104: perform a search in the vector database based on the first vector.

[0082] The aforementioned vector database has already stored the prompt vectors that need to be protected during the initialization phase. The text vectorization processing method of these prompt vectors needs to be consistent with the text vectorization processing method of the first response content.

[0083] The search based on the first vector in the vector database essentially involves performing a similarity match between the first vector and each of the protected prompt vectors in the database. Specifically, similarity matching calculates the similarity between the first vector and each protected prompt vector in the database, and then uses a similarity threshold to filter the matching results. For example, based on the similarity matching results, the k prompt vectors with the highest similarity to the first vector are selected. Then, a second filtering is performed on these k prompt vectors using the similarity threshold. If any of these k prompt vectors has a similarity greater than the similarity threshold to the first vector, the search is considered successful; otherwise, the search is considered unsuccessful.

[0084] In some implementations, to reduce the computational cost of vector retrieval, indexes can be created for the first vector and the prompt vectors. This maps the dense first vector and prompt vectors to a low-dimensional index feature space, thereby reducing the computational cost of similarity matching. Specifically, when constructing the vector database, a prompt index can be built for each prompt vector. When retrieving the first vector, the first vector is also mapped to the same feature space to obtain its index. Then, a similarity match is performed between the first vector's index and the prompt indexes in the vector database. A similarity threshold is then used to filter out matching results, thus determining whether the retrieval was successful.

[0085] Next, for step S106: if the search is successful, execute the preset defense strategy; otherwise, allow the first response content to pass.

[0086] If the first vector search in the vector database fails to find a match, it can be determined that there is no prompt requiring protection in the corresponding response content. At this point, the application on the large model server executes the program after the security aspect segmentation point, which means that the response content is fed back to the user.

[0087] If the first vector search in the vector database is successful, it can be determined that the corresponding response content contains a prompt that needs protection. In this case, a preset defense strategy needs to be executed, that is, the response content needs to be protected. The specific method of defense can be set according to requirements, and this embodiment does not limit it. For example, it may include, but is not limited to, response operations such as content blocking, providing harmless content on behalf of others, or logging.

[0088] The above describes methods for defending against message theft attacks targeting single-turn responses. In some implementations, methods can also be applied to defend against message theft attacks targeting multi-turn dialogues. Please refer to [link / reference]. Figure 2 , Figure 2 The flowchart of another security aspect-based message theft defense method is illustrated.

[0089] like Figure 2 As shown, in some implementations, the security aspect-based message theft defense method can be executed by a security aspect program deployed on a large model server, and the method may include steps S200 to S212:

[0090] S200: In response to a user's query request initiated by the large model server to the large model, intercept the large model's first response to the query request.

[0091] S202: Vectorize the content of the first response to obtain the first vector.

[0092] S204: Perform a search in the vector database based on the first vector.

[0093] S206: If the first vector search in the vector database is successful, then execute the preset defense strategy.

[0094] S208: If the first vector search in the vector database is unsuccessful, then obtain the user's historical interaction records with the large model, and obtain the second response content from the historical interaction records.

[0095] S210: Vectorize the content of the second response to obtain the second vector.

[0096] S212: Search the vector database based on the second vector. If the search is successful, execute the preset defense strategy; otherwise, allow the first response content to pass.

[0097] It can be seen that, in Figure 2 The security aspect-based method for defending against prompt theft attacks analyzes the responses of users in single-turn and multi-turn dialogues with the large model, performing semantic-level matching with the prompts to be protected to determine the presence of prompt theft attacks. This method does not rely on detection rules based on prior knowledge, thus resisting various request-based bypass attacks. Furthermore, the aforementioned security aspect-based method for defending against prompt theft attacks is implemented through a security aspect program, requiring no modification to the infrastructure of the large model service, resulting in lower cost and greater convenience.

[0098] The following description, in conjunction with the accompanying drawings and specific embodiments, will... Figure 2 The method shown will be explained.

[0099] In execution Figure 2 Before implementing the security aspect-based message theft defense method, initialization is required. The initialization phase primarily involves injecting the security aspect protocol into the large model server and building the vector database.

[0100] The aforementioned security aspect is used to intercept and process the response content returned to the user, thereby implementing the aforementioned security aspect-based message theft defense method. Specifically, the method for injecting the security aspect into the server is as follows: obtain the application of the large model server and the pointcut of the application, and then inject the pre-built security aspect into the pointcut so that the security aspect is executed when the application is executed up to the pointcut.

[0101] The aforementioned vector database is used to store the vectors of the prompts that need to be protected. During the initialization phase, the prompts that need to be protected can be collected, and then the prompts that need to be protected can be converted into vectors and saved to the vector database.

[0102] Once initialization is complete, it can be executed. Figure 2 The method shown is a security aspect-based message theft defense method.

[0103] First, regarding step S200: In response to the user's query request initiated by the large model server to the large model, intercept the first response content of the large model to the query request.

[0104] This embodiment, through the setting of pointcuts, enables the security aspect program to intercept the large model's response text to the question request when it detects a user's question request initiated through the large model server, and to use this single-round response text as the first response content.

[0105] Next, for step S202: vectorize the content of the first response to obtain the first vector.

[0106] To facilitate retrieval, step S202 requires text vectorization of the first response content. Text vectorization refers to representing text as vectors that can express the semantics of the text, which is a fundamental task in the field of natural language processing.

[0107] It should be noted that this embodiment does not limit the method of text vectorization processing of the first response content, and can be adaptively selected according to requirements. For example, the text vectorization processing method may include, but is not limited to: word embedding method, one-hot encoding method, bag-of-words model, word-vector model, etc.

[0108] Next, for step S204: perform a search in the vector database based on the first vector.

[0109] The aforementioned vector database has already stored the prompt vectors that need to be protected during the initialization phase. The text vectorization processing method of these prompt vectors needs to be consistent with the text vectorization processing method of the first response content.

[0110] The search based on the first vector in the vector database essentially involves performing a similarity match between the first vector and each of the protected prompt vectors in the database. Specifically, similarity matching calculates the similarity between the first vector and each protected prompt vector in the database, and then uses a similarity threshold to filter the matching results. For example, based on the similarity matching results, the k prompt vectors with the highest similarity to the first vector are selected. Then, a second filtering is performed on these k prompt vectors using the similarity threshold. If any of these k prompt vectors has a similarity greater than the similarity threshold to the first vector, the search is considered successful; otherwise, the search is considered unsuccessful.

[0111] In some implementations, to reduce the computational cost of vector retrieval, indexes can be created for the first vector and the prompt vectors. This maps the dense first vector and prompt vectors to a low-dimensional index feature space, thereby reducing the computational cost of similarity matching. Specifically, when constructing the vector database, a prompt index can be built for each prompt vector. When retrieving the first vector, the first vector is also mapped to the same feature space to obtain its index. Then, a similarity match is performed between the first vector's index and the prompt indexes in the vector database. A similarity threshold is then used to filter out matching results, thus determining whether the retrieval was successful.

[0112] Next, for step S206: if the first vector search in the vector database is successful, then the preset defense strategy is executed.

[0113] If the first vector search in the vector database is successful, it can be determined that the corresponding response content contains a prompt that needs protection. In this case, a preset defense strategy needs to be executed, that is, the response content needs to be protected. The specific method of defense can be set according to requirements, and this embodiment does not limit it. For example, it may include, but is not limited to, response operations such as content blocking, providing harmless content on behalf of others, or logging.

[0114] Next, for step S208: if the first vector search in the vector database is not successful, then obtain the historical interaction records between the user and the large model, and obtain the second response content from the historical interaction records.

[0115] If the first vector search in the vector database fails to find a match, it can be determined that the corresponding response content does not contain any prompts requiring protection. However, it cannot be ruled out that the user's historical interaction records may contain prompts requiring protection. To further eliminate the risk of prompt theft attacks, the user's historical interaction records with the large model can be obtained, and then the content of the large model's replies in the historical interaction records can be extracted as the second response content mentioned above.

[0116] Next, for step S210: vectorize the content of the second response to obtain the second vector.

[0117] In this step S210, the second response content needs to be processed into text vectorization. The same processing method as that used for the first response content needs to be used to process the second response content into text vectorization.

[0118] Specifically, the content of the second response can be concatenated into a string, and then this string can be vectorized to obtain the second vector mentioned above.

[0119] The text vectorization method for the second response can be adaptively selected according to requirements. For example, text vectorization methods may include, but are not limited to: word embedding methods, one-hot encoding methods, bag-of-words models, word-vector models, etc.

[0120] Next, for step S212: search the vector database based on the second vector. If the search is successful, execute the preset defense strategy; otherwise, allow the first response content to pass.

[0121] Considering that the second vector consists of multi-round responses and carries a large amount of information, in some implementations, the second vector can be divided into multiple sub-vectors. Then, each sub-vector of the second vector is matched against the prompt vectors in the vector database for similarity, and a similarity threshold is used to filter the matching results. For example, based on the similarity matching results, the k prompt vectors with the highest similarity to the sub-vectors of the second vector are selected. Then, a similarity threshold is used to further filter these k prompt vectors. If any of these k prompt vectors has a similarity greater than the similarity threshold to a sub-vector of the second vector, the search is considered successful; otherwise, the search is considered unsuccessful.

[0122] In some implementations, to reduce the computational cost of vector retrieval, indexes can be created for the sub-vectors of the second vector and the prompt vector. This maps the dense sub-vectors and prompt vectors of the second vector to a low-dimensional index feature space, thereby reducing the computational cost of similarity matching. Specifically, when constructing the vector database, a prompt index can be built for each prompt vector. When retrieving a sub-vector of the second vector, the sub-vectors of the second vector are also mapped to the same feature space to obtain the index of the sub-vectors of the second vector. Then, a similarity match is performed between the index of the sub-vectors of the second vector and the prompt index in the vector database. A similarity threshold is then used to filter the matching results, thereby determining whether the retrieval is successful.

[0123] If the second vector search in the vector database fails to find a match, it can be determined that there are no prompts requiring protection in the user's historical interaction records with the large model. At this point, the application on the large model server will execute the program after the security aspect segmentation point, which means that the response content will be fed back to the user.

[0124] If the second vector search in the vector database yields a match, it can be determined that the user's historical interaction records with the large model contain prompts that require protection. In this case, a preset defense strategy needs to be executed, i.e., the response content needs to be protected. The specific method of defense can be set according to requirements, and this embodiment does not limit it. For example, it may include, but is not limited to, content blocking, providing harmless answers on behalf of others, or logging response operations.

[0125] Corresponding to the above-mentioned security aspect-based message theft defense method, this embodiment also provides a security aspect-based message theft defense device.

[0126] Please refer to Figure 3 , Figure 3 An exemplary embodiment of a security aspect-based message theft defense device is shown, which can be used to implement the aforementioned security aspect-based message theft defense method. It should be noted that the security aspect-based message theft defense method described in one or more embodiments of this application can rely on... Figure 3 The security aspect-based message theft defense device shown is implemented, but is not limited to this device.

[0127] like Figure 3 As shown, this security aspect-based message theft defense device is applied to a large model server. The large model server has a security aspect program deployed on it, and this device is used to execute the business logic of the security aspect program. The device includes:

[0128] The data acquisition module 301 is configured to intercept the first response content of the large model to the question request initiated by the user through the large model server.

[0129] Processing module 302 is configured to vectorize the content of the first response to obtain a first vector.

[0130] Database module 303 is configured to build a vector database that stores the vector prompts that need to be protected.

[0131] The retrieval module 304 is configured to perform a retrieval in the vector database based on the first vector.

[0132] The response module 305 is configured to execute a preset defense strategy in response to the retrieval module 304 finding a match in the vector database based on the first vector; and to allow the first response content to be released in response to the retrieval module 304 not finding a match in the vector database based on the first vector.

[0133] Regarding the aforementioned data acquisition module 301, by setting the pointcut, the data acquisition module 301 can use the security aspect program to intercept the response text to the question request when it detects that the user has initiated a question request through the server, and use the single-round response text as the first response content.

[0134] For the processing module 302 mentioned above, the processing module 302 can use methods such as word embedding, one-hot encoding, bag-of-words model, and word-vector model to vectorize the content of the first response.

[0135] Regarding the aforementioned database module 303, the database module 303 can collect the prompts that need to be protected in advance, and then use the vectorization processing method of the aforementioned vectorized first response content to convert the prompts that need to be protected into vectors and save them to the vector database.

[0136] Regarding the aforementioned retrieval module 304, this module is specifically used to perform similarity matching between the first vector and each of the protected prompt vectors in the vector database. Specifically, similarity matching involves calculating the similarity between the first vector and each protected prompt vector in the vector database, and then filtering the matching results using a similarity threshold. For example, based on the similarity matching results, the k prompt vectors with the highest similarity to the first vector are selected. Then, a second filtering is performed on these k prompt vectors using the similarity threshold. If any of these k prompt vectors has a similarity greater than the similarity threshold to the first vector, the search is considered successful; otherwise, the search is considered unsuccessful.

[0137] In some implementations, to reduce the computational cost of vector retrieval, the retrieval module 304 can also create indexes for the first vector and the prompt vector, mapping the dense first vector and prompt vectors to a low-dimensional index feature space, thereby reducing the computational cost of similarity matching. Specifically, when constructing the vector database, a prompt index can be built for each prompt vector. When the retrieval module 304 performs a retrieval on the first vector, it also maps the first vector to the same feature space to obtain the index of the first vector. Then, it performs similarity matching between the index of the first vector and the prompt index in the vector database, and then filters the matching results through a similarity threshold to determine whether the retrieval is successful.

[0138] Regarding the aforementioned response module 305, this response module 305 is specifically used to execute different response logics based on the search results of the search module 304.

[0139] If the retrieval module 304 fails to find the first vector in the vector database, it can be determined that there is no prompt that needs protection in the corresponding response content. At this time, the application on the large model server executes the program after the security aspect segmentation point, that is, it sends the response content back to the user.

[0140] If the retrieval module 304 finds a match in the vector database for the first vector search, it can determine that the corresponding response content contains a prompt that needs protection. At this point, the response module 305 needs to execute a preset defense strategy through a security aspect ratio procedure, i.e., to perform defense processing on the response content. The specific method of defense processing can be set according to requirements, and this embodiment does not limit it. For example, it may include, but is not limited to, content blocking, providing harmless answers on behalf of others, or logging responses.

[0141] The above describes the device's method for defending against prompt theft attacks on single-round response content. In some embodiments, the device can also provide a method for defending against prompt theft attacks on multi-round dialogue response content.

[0142] In some implementations, the data acquisition module 301 described above may also be configured to, in response to the retrieval module 304's failure to find a match in the vector database based on the first vector, acquire the user's historical interaction records with the large model and acquire the second response content from the historical interaction records.

[0143] The aforementioned processing module 302 can also be configured to vectorize the second response content to obtain a second vector.

[0144] Processing module 302 needs to use the same processing method as the first response content to perform text vectorization processing on the second response content. Specifically, processing module 302 can concatenate the second response content into a string, and then vectorize this string to obtain the second vector mentioned above.

[0145] The aforementioned retrieval module 304 is also configured to perform a retrieval in the vector database based on the second vector.

[0146] Considering that the second vector consists of multi-round responses and carries a large amount of information, in some implementations, the retrieval module 304 can divide the second vector into multiple sub-vectors, and then perform similarity matching between each sub-vector of the second vector and the prompt vectors in the vector database. A similarity threshold is then used to filter the matching results. For example, based on the similarity matching results, the k prompt vectors with the highest similarity to the sub-vectors of the second vector are selected. Then, a similarity threshold is used to perform a secondary filtering on these k prompt vectors. If any of these k prompt vectors has a similarity greater than the similarity threshold to a sub-vector of the second vector, the retrieval is considered successful; otherwise, the retrieval is considered unsuccessful.

[0147] In some implementations, to reduce the computational cost of vector retrieval, the retrieval module 304 can also create indexes for the sub-vectors and prompt vectors of the second vector, mapping the dense sub-vectors and prompt vectors of the second vector to a low-dimensional index feature space, thereby reducing the computational cost of similarity matching. Specifically, when constructing the vector database, a prompt index can be built for each prompt vector. When retrieving a sub-vector of the second vector, the sub-vectors of the second vector are also mapped to the same feature space to obtain the index of the sub-vectors of the second vector. Then, a similarity match is performed between the index of the sub-vectors of the second vector and the prompt index in the vector database. The matching results are then filtered out using a similarity threshold to determine whether the retrieval is successful.

[0148] The aforementioned response module 305 is further configured to execute a preset defense strategy in response to the retrieval module 304 finding a match in the vector database based on the second vector; and to allow the first response content to be released in response to the retrieval module 304 not finding a match in the vector database based on the second vector.

[0149] If the second vector search in the vector database fails to find a match, it can be determined that there are no prompts requiring protection in the user's historical interaction records with the large model. At this point, the application on the large model server will execute the program after the security aspect segmentation point, which means that the response content will be fed back to the user.

[0150] If the second vector search in the vector database is successful, it can be determined that there are prompts requiring protection in the user's historical interaction records with the large model. In this case, the response module 305 needs to execute a preset defense strategy, that is, to perform defense processing on the response content. The specific method of defense processing can be set according to requirements, and this embodiment does not limit it. For example, it may include, but is not limited to, response operations such as content blocking, answering with harmless content on behalf of the user, or logging.

[0151] For the aforementioned security aspect-based message theft defense device, taking a module as an example of a software functional unit, the data acquisition module 301 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, the data acquisition module 301 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0152] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0153] As an example of a hardware functional unit, the data acquisition module 301 may include at least one computing device, such as a server. Alternatively, the data acquisition module 301 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0154] The data acquisition module 301 includes multiple computing devices that can be distributed within the same region or in different regions. Similarly, the data acquisition module 301 can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the data acquisition module 301 can be distributed within the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0155] In other embodiments, the data acquisition module 301 can be used to execute any step in the above-described security aspect-based prompt theft attack defense method; the processing module 302 can be used to execute any step in the above-described security aspect-based prompt theft attack defense method; the database module 303 can be used to execute any step in the above-described security aspect-based prompt theft attack defense method; the retrieval module 304 can be used to execute any step in the above-described security aspect-based prompt theft attack defense method; and the response module 305 can be used to execute any step in the above-described security aspect-based prompt theft attack defense method. The steps implemented by the data acquisition module 301, processing module 302, database module 303, retrieval module 304, and response module 305 can be specified as needed. The data acquisition module 301, processing module 302, database module 303, retrieval module 304, and response module 305 respectively implement different steps in the above-described security aspect-based prompt theft attack defense method to achieve all the functions of the above-described security aspect-based prompt theft attack defense device.

[0156] In this implementation, the security aspect-based message theft defense device can also be applied to computing devices such as computers and servers, or to a computing device cluster including at least one computing device, to achieve the security aspect-based message theft defense function.

[0157] In some embodiments, an electronic device is also provided. Please refer to... Figure 4 The electronic device includes a bus 401, a processor 402, a memory 403, and a communication interface 404. The processor 402, memory 403, and communication interface 404 communicate via the bus 401. This electronic device can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the electronic device.

[0158] Bus 401 can be a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus 601 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 601 may include a path for transmitting information between various components of an electronic device (e.g., processor 402, memory 403, and communication interface 404).

[0159] Processor 402 may include any one or more processors such as CPU, graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0160] Memory 403 may include volatile memory, such as random access memory (RAM). Memory 403 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0161] The memory 403 stores executable program code, and the processor 402 executes the executable program code to implement the functions of the aforementioned data acquisition module 301, processing module 302, database module 303, retrieval module 304 and response module 305 respectively, that is, to implement the functions of the aforementioned security aspect-based prompt message theft attack defense device, thereby realizing the aforementioned security aspect-based prompt message theft attack defense method.

[0162] Communication interface 404 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between electronic devices and other devices or communication networks.

[0163] In some embodiments, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the above-described security aspect-based message theft defense method.

[0164] Computer-readable storage media can be any available medium that an electronic device can store, or a data storage device such as a data center that contains one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives). The computer-readable storage medium includes instructions that instruct the electronic device to execute the aforementioned security aspect-based message theft defense method.

[0165] It is understood that the structures illustrated in the embodiments of this specification do not constitute a specific limitation on the system of the embodiments of this specification. In other embodiments of the specification, the above system may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0166] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0167] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0168] It should be noted that the above examples are merely specific embodiments of the present invention, and the present invention is obviously not limited to the above embodiments, with many similar variations. All modifications that can be directly derived or conceived by those skilled in the art from the content disclosed in this invention should fall within the protection scope of this invention.

Claims

1. A method for defending against prompt word stealing attacks based on security aspects, which is executed by a security aspect program deployed on a large model server, and comprises: In response to a question request initiated by a user to the big model through the big model server, intercepting a first response content of the big model to the question request; Vectorizing the first response content to obtain a first vector; Searching a vector database based on the first vector, wherein the vector database stores prompt word vectors that need to be protected; If the search is successful, the preset defense strategy is executed; otherwise, the first response content is released.

2. The method of claim 1, further comprising: If the first vector is not found in the vector database, obtaining a historical interaction record between the user and the large model, and obtaining a second response content from the historical interaction record; Vectorizing the second response content to obtain a second vector; A search is performed in the vector database based on the second vector, and if a search hits, a preset defense strategy is executed; otherwise, the first response content is released.

3. The method according to claim 2, vectorizing the second response content to obtain a second vector, specifically comprising: Connect the second response content into a string; The character string is vectorized to obtain the second vector.

4. The method of claim 1, further comprising: Get the prompt words that need protection; Vectorizing the prompt word that needs to be protected to obtain the prompt word vector; The prompt vector is stored in the vector database.

5. The method of claim 1, further comprising: Obtaining an application program of the large model server and a cut-off point of the application program; The security aspect program is injected into the cut-off point, so that the security aspect program is executed when the application is executed to the cut-off point.

6. A security aspect-based prompt stealing attack defense device, applied to a large model server, the large model server is deployed with a security aspect program, the device is applied to execute the business logic of the security aspect program, the device includes: A data acquisition module, configured to respond to a question request initiated by a user to the big model through the big model server, and intercept a first response content of the big model to the question request; A processing module, configured to vectorize the first response content to obtain a first vector; A database module, configured to construct a vector database, wherein the vector database stores prompt word vectors that need to be protected; A retrieval module, configured to search the vector database based on the first vector; The response module is configured to execute a preset defense strategy in response to a search hit by the retrieval module in the vector database based on the first vector; and to release the first response content in response to a search miss by the retrieval module in the vector database based on the first vector.

7. The device according to claim 6, wherein: The data acquisition module is further configured to acquire a historical interaction record between the user and the large model in response to a search failure by the retrieval module in the vector database based on the first vector, and acquire a second response content from the historical interaction record; The processing module is further configured to vectorize the second response content to obtain a second vector; The retrieval module is further configured to search the vector database based on the second vector; The response module is also configured to execute a preset defense strategy in response to a search hit by the retrieval module in the vector database based on the second vector; and to release the first response content in response to a search miss by the retrieval module in the vector database based on the second vector.

8. The device according to claim 7, wherein the processing module is specifically used for: Connect the second response content into a string; The character string is vectorized to obtain the second vector.

9. The device according to claim 6, wherein: The data acquisition module is also configured to acquire a prompt that needs to be protected; The processing module is further configured to vectorize the prompt words that need to be protected to obtain the prompt word vector, and store the prompt word vector in the vector database.

10. The device of claim 6, further comprising: The aspect program injection module is configured to obtain the application of the large model server and the aspect point of the application, and inject the security aspect program into the aspect point so that the security aspect program is executed when the application is executed to the aspect point.

11. A computer-readable storage medium storing a computer program, wherein when the computer program is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 5.

12. An electronic device comprising: at least one memory for storing a program; At least one processor is used to execute the program stored in the memory, and when the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 5.

Images