Prompt detection resource optimization method and system, optimization terminal and medium

Abstract

The invention discloses a cue word detection resource optimization method and system, an optimization terminal and a medium, and the cue word detection resource optimization method comprises the steps: receiving a cue word sent by a user, and recognizing a cue parameter of the cue word; constructing a multi-stage dynamic Bayesian game model, and inputting the prompt parameter into the multi-stage dynamic Bayesian game model to obtain a detection probability; according to the detection probability, detecting the cue word to obtain a safety cue word; and inputting the safety cue word into a large language model for processing to obtain response information. According to the method, malicious cue word detection can be effectively identified and prevented, the LLM output safety is improved, the cue word needing to be detected can be intelligently selected to avoid unnecessary resource consumption, and then service delay of a benign user is reduced to improve the user experience.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a prompt word detection resource optimization method, system, optimization terminal and medium. Background Art

[0002] Large Language Model (LLM) is widely used. Researchers use prompt word engineering to construct prompt words to ensure that the output generated by LLM can better meet user requirements and improve the quality of response. However, attackers also use prompt word engineering to construct malicious prompt words and launch prompt word attacks, including injection attacks and jailbreak attacks, causing LLM to output unsafe content. At the same time, they can also allow unauthorized access to private data. For example, tools integrated with LLM may be compromised by prompt injection attacks, thereby exposing sensitive information. In order to protect the security of LLM output, researchers have proposed some methods to filter prompt words, but these methods do not jointly consider resource consumption and system performance optimization.

[0003] In the existing technical solutions, one is to perform synonymous replacement on the prompt words, which will unintentionally change the meaning of the original prompt words, resulting in content modification and decreased LLM response quality; the second is to use LLM to filter the prompt words, however, LLM may not be able to identify malicious attacks, and the use of LLM detection consumes a lot of resources; the third is to use a small model classifier for prompt word detection and filtering, this method can have a higher accuracy, however, in reality, the ratio of malicious prompt words is small, filtering all prompt words will waste a lot of resources.

[0004] Existing models would waste a lot of resources if they filter all the prompt words.

[0005] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0006] The main purpose of the present application is to provide a prompt word detection resource optimization method, system, optimization terminal and medium, aiming to solve the problem that the prior art uses synonymous replacement or filtering of prompt words in a large language model, resulting in low output security and high energy consumption of the large language model.

[0007] A first aspect of an embodiment of the present application provides a method for optimizing prompt word detection resources, which comprises the following steps: receiving a prompt word sent by a user and identifying prompt parameters of the prompt word; constructing a multi-stage dynamic Bayesian game model, and inputting the prompt parameters into the multi-stage dynamic Bayesian game model to obtain a detection probability; detecting the prompt word according to the detection probability to obtain a safe prompt word; and inputting the safe prompt word into a large language model for processing to obtain response information.

[0008] Optionally, in one embodiment of the present application, the constructing of a multi-stage dynamic Bayesian game model specifically includes: constructing consistency functions corresponding to benign users, malicious users and defenders; minimizing the consistency functions corresponding to the benign users, the malicious users and the defenders to obtain a multi-stage dynamic Bayesian game model.

[0009] Optionally, in one embodiment of the present application, the consistency function of the benign user is expressed as: ; The consistency function of the malicious user is expressed as: ; The defender's consistency function is expressed as: ; in, is the consistency function of benign users, m represents an edge node, represents the set of edge nodes, Indicates a good user x strategy, Is a good user x The prompt word is sent to the edge node m The total delay of the detection strategy and whether to send the prompt word to the large language model, represents the number of rounds of the game, is the total number of game rounds; is the consistency function of malicious users, Indicates malicious users y strategy, is the F1 score of the detection model, Indicates prompt words, Indicates the number of tokens in the prompt word. Indicates the number of floating point operations for each token; is the defender’s consistency function, , represents two weight parameters greater than zero, Indicates prompt words, Indicates t The sequence of prompt words received by the round defender, Indicates defender For t Each cue word of the round Whether to detect the strategy, represents the total delay for each cue word to be processed, represents the defender's belief about each cue word.

[0010] Optionally, in one embodiment of the present application, the multi-stage dynamic Bayesian game model is expressed as: ; ; ; ; in, Indicated in t Except for the defender Other good users x and malicious users y For prompt words Whether to detect the strategy, represents the set of defenders, It is t The number of malicious prompt words in the round.

[0011] Optionally, in one embodiment of the present application, the prompt parameters are input into the multi-stage dynamic Bayesian game model to obtain a detection probability, and then the method further includes: updating the beliefs of the benign users, the malicious users and the defenders on the security of the prompt words according to the detection probability to obtain updated beliefs; updating the model according to the updated beliefs to obtain an updated multi-stage dynamic Bayesian game model, and inputting the next round of prompt parameters into the updated multi-stage dynamic Bayesian game model to obtain the next round of detection probability.

[0012] Optionally, in one embodiment of the present application, the prompt word is detected according to the detection probability to obtain a safe prompt word, specifically including: when the detection probability meets a preset requirement, the prompt word is detected to obtain a detection result; if the detection result is safe, the prompt word is determined to be a safe prompt word.

[0013] Optionally, in an embodiment of the present application, when the detection probability meets a preset requirement, the prompt word is detected to obtain a detection result, and then the following step is included: if the detection result is unsafe, the prompt word is returned for the next round of prompt word detection.

[0014] The second aspect of the embodiment of the present application further provides a prompt word detection resource optimization system, wherein the prompt word detection resource optimization system includes: A prompt word receiving module, used for receiving a prompt word sent by a user and identifying prompt parameters of the prompt word; A model building and optimization module, used to build a multi-stage dynamic Bayesian game model, and input the prompt parameters into the multi-stage dynamic Bayesian game model to obtain a detection probability; A detection strategy determination module, used to detect the prompt word according to the detection probability to obtain a safety prompt word; The large model output module is used to input the safety prompt word into the large language model for processing to obtain response information.

[0015] The third aspect of the embodiment of the present application also provides a prompt word detection resource optimization terminal, wherein the prompt word detection resource optimization terminal includes: a memory, a processor, and a prompt word detection resource optimization program stored in the memory and executable on the processor, and when the prompt word detection resource optimization program is executed by the processor, the steps of the prompt word detection resource optimization method as described above are implemented.

[0016] The fourth aspect of the embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a prompt word detection resource optimization program, and when the prompt word detection resource optimization program is executed by a processor, the steps of the prompt word detection resource optimization method as described above are implemented.

[0017] Beneficial effects: The present application provides a prompt word detection resource optimization method, system, optimization terminal and medium. The multi-stage dynamic Bayesian game model of the present application makes a decision on whether to detect each incoming prompt word, and only detects the prompt words that need to be detected according to the model output, thereby reducing system resource consumption and reducing service delays for benign users. The present application optimizes prompt word security while optimizing system performance by identifying the user's prompt words and obtaining a detection strategy based on a multi-stage dynamic Bayesian game model, thereby being able to effectively identify and prevent malicious prompt word detection, improve the security of LLM output, and intelligently select prompt words that need to be detected to avoid unnecessary resource consumption, thereby reducing service delays for benign users and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of a preferred embodiment of the method for optimizing prompt word detection resources of the present application; Figure 2 It is a flowchart of specific implementation steps of multi-stage dynamic Bayesian game in a preferred embodiment of the prompt word detection resource optimization method of the present application; Figure 3 It is a structural diagram of a preferred embodiment of the prompt word detection resource optimization system of the present application; Figure 4 This is a structural diagram of a preferred embodiment of the prompt word detection resource optimization terminal of this application.

[0020] Description of reference numerals: 100. Prompt word receiving module; 200. Model building and optimization module; 300. Detection strategy determination module; 400. Large model output module. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and effect of the present application clearer and more specific, the technical scheme in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. The described embodiments are only possible technical implementations of the present application, not all possible implementations. Based on the embodiments in the present application, those skilled in the art can completely combine the embodiments of the present application to obtain other embodiments without creative work, and these embodiments are also within the scope of protection of the present application.

[0022] First, the nouns involved in the embodiments of the present application are introduced: LLM: Large Language Model, large language model, referred to as large model; VDB: Vector Database, vector database.

[0023] Secondly, the system architecture of the embodiment of the present application is introduced.

[0024] A vector database (VDB) and a prompt word anomaly detector (to detect malicious information) are deployed on each edge device (server). The prompt words sent by the user are detected on the edge device, and then the safe prompt words are sent to the LLM in the cloud for service. The VDB saves the pre-downloaded data set in the format of: {},in, is the vectorized hint word, is the label of the prompt word, Indicates that the prompt word is benign, Indicates that the prompt word is malicious. The prompt word sent by the user is first detected on the edge node, and the data set saved in the VDB is used to identify benign and malicious prompt words.

[0025] The following describes the prompt word detection resource optimization method, system, optimization terminal and medium of the embodiment of the present application with reference to the accompanying drawings. In view of the problem that the prompt words in the large language model are replaced with synonyms or filtered in the related art mentioned above, resulting in low security and high energy consumption of the large language model output, the present application provides a prompt word detection resource optimization method, in which the prompt words of the user are identified and the detection strategy is obtained based on a multi-stage dynamic Bayesian game model, the prompt word security is optimized while optimizing the system performance, so that malicious prompt word detection can be effectively identified and prevented, the security of LLM output can be improved, and the prompt words that need to be detected can be intelligently selected to avoid unnecessary resource consumption, thereby reducing the service delay of benign users to improve user experience. Thus, the technical problem that the prompt words in the large language model are replaced with synonyms or filtered in the related art, resulting in low security and high energy consumption of the large language model output is solved.

[0026] This application proposes for the first time to jointly optimize system security, resource consumption and service delay under prompt word attacks, and formalizes the joint prompt word detection, delay and resource optimization problem into a multi-stage incomplete information Bayesian game model. In order to solve the Bayesian equilibrium at each stage, this application uses a belief update method and a malicious digital prediction method to make a decision on whether to detect each incoming prompt word. The prompt word detection resource optimization method of this application can improve the security of LLM system output, reduce system resource consumption and reduce service delays for benign users.

[0027] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0028] The method for optimizing prompt word detection resources described in the preferred embodiment of the present application is as follows: Figure 1 As shown, the prompt word detection resource optimization method includes the following steps: In step S101, a prompt word sent by a user is received, and prompt parameters of the prompt word are identified.

[0029] Specifically, a prompt word input by a user is received through a user interface (such as a web page, an application program interface, a command line interface, etc.); after the prompt word is preprocessed, the prompt word is parsed to identify its structure and meaning; based on the parsed prompt word, key parameters related to subsequent processing are extracted as input for subsequent processing (such as building a multi-stage dynamic Bayesian game model).

[0030] In step S102, a multi-stage dynamic Bayesian game model is constructed, and the prompt parameters are input into the multi-stage dynamic Bayesian game model to obtain a detection probability.

[0031] In a possible implementation, a consistency function corresponding to each of the benign user, the malignant user, and the defender is constructed; and the consistency function corresponding to each of the benign user, the malignant user, and the defender is minimized to obtain a multi-stage dynamic Bayesian game model.

[0032] It can be understood that the present application uses multiple rounds of incomplete information dynamic Bayesian games to model system security and performance issues, and defines consistency functions for benign users, malicious users, and defenders.

[0033] The joint optimization system security and performance issues are modeled using multi-round incomplete information dynamic Bayesian games. x The consistency function for: ; (1) in, is the consistency function of benign users, m represents an edge node, represents the set of edge nodes, Indicates a good user x strategy (select server), Is a good user x The prompt word is sent to the edge node m The total delay of the detection strategy and whether to send the prompt word to the large language model, represents the number of rounds of the game, is the total number of game rounds.

[0034] Malicious users y The consistency function for: ; (2) in, is the consistency function of malicious users, Indicates malicious users y strategy, is the F1 score of the detection model, Indicates prompt words, Indicates the number of tokens in the prompt word. Indicates the number of floating point operations for each token.

[0035] Defender n The consistency function is: ; (3) in, is the defender’s consistency function, , represents two weight parameters greater than zero, Indicates prompt words, Indicates t The sequence of prompt words received by the round defender, Indicates defender For t Each cue word of the round Whether to detect the strategy, represents the total delay for each cue word to be processed, represents the defender's belief about each cue word. x , y , n Represents the index of a user or participant, used to distinguish different users or participants; token refers to the smallest unit into which text or data is divided.

[0036] Therefore, the multi-stage dynamic Bayesian game model representation of minimizing the defender's consistency function, that is, the game problem can be formalized as: ; ; ; ; (4) in, In the t Except for the defender Other good users x and malicious users y For prompt words Whether to detect the strategy, represents the set of defenders, It is t The number of malicious prompt words in the round. It is an abbreviation of "subject to", which means "subject to" or "satisfying conditions".

[0037] It can be understood that the three consistencies are the goals of good users, bad users and defenders, respectively. The goals correspond to the benefits of each player (i.e., good users, bad users and defenders), and the benefits of the three players are maximized through joint optimization of the game. In formula (4), the variables are ,two They represent defenders n Strategies and other players - n strategies, each player (i.e. benign users, malicious users and defenders) will have benefits, and their strategies will affect each other. The final result is that each player's change of strategy will lead to a decrease in his or her own benefits. At this time, it is an equilibrium solution, which is a Bayesian equilibrium in the Bayesian game model.

[0038] In one possible implementation, based on the detection probability, the beliefs of the benign user, the malicious user, and the defender on the security of the prompt word are updated to obtain updated beliefs; the updated beliefs are used as beliefs to obtain an updated multi-stage dynamic Bayesian game model, and the next round of prompt parameters are input into the updated multi-stage dynamic Bayesian game model to obtain the next round of detection probability.

[0039] Specifically, the present application uses the belief updating method of the incomplete information Bayesian game, adopts historical data and vector similarity to update the belief. In this step, input data preparation: collect various relevant data of the previous round, including belief, delay, detection threshold, number of tokens, expected number of tokens, safe vector similarity and unsafe vector similarity; benign user processing: determine whether the previous round of replies is benign, set the initial likelihood function value according to the reply result, calculate the difference (diff) between the ratio of the delay to the number of tokens and the detection threshold, update the likelihood function value according to the positive or negative of the diff, and ensure that the value is between 0 and 1; malicious user processing: determine whether the previous round of replies is malicious, set the initial likelihood function value according to the reply result, update the likelihood function value using the ratio of the expected number of tokens to the actual number of tokens, and ensure that the value is between 0 and 1; defender processing: determine whether the previous round detected anomalies and gave malicious replies, set the initial likelihood function value according to the detection results, and update the likelihood function value using vector similarity; belief update: use the updated likelihood function value to calculate the new round of beliefs.

[0040] The belief updating method of this application dynamically updates the belief of various players (benign users, malicious users, defenders) on the security of the prompt word by considering the response results of the previous round, processing delay, number of tokens, vector similarity and other factors. This updating mechanism helps the system to more accurately judge the security of the current prompt word and make more reasonable decisions.

[0041] Further, the belief updating method steps are as follows: Step K11, enter t -1 round of faith , cue word processing delay , detection threshold , No. t -1 round of responses Number of tokens , the number of tokens expected to be output by malicious users , security vector similarity , unsafe vector similarity .

[0042] Step K12: For benign players, if the response in the previous round is benign, define two likelihood functions: , .otherwise, , .calculate , if diff is greater than zero, update and Otherwise, update , . And ensure that the values ​​of the two likelihood functions updated in both ways are between 0 and 1.

[0043] Step K13, for malicious players, if the reply in the previous round is malicious, define two likelihood functions , .otherwise, , .renew , , and ensure that the values ​​of the two updated likelihood functions are between 0 and 1.

[0044] Step K14: For the defender, if an anomaly was detected in the previous round and a malicious response was given, two likelihood functions are defined: , ,otherwise, , . Use similarity to update the likelihood function , .

[0045] Step K15, for each player, update the belief .

[0046] The above, , , , , , is a predefined parameter used to adjust the value of the likelihood function in belief updating; , Respectively represent the likelihood function, which is used to update the belief; , Represent the safe vector similarity and unsafe vector similarity respectively; Specifically, in order to solve formula (4), the present application uses the sequential marginal analysis method to calculate the number of malicious prompt words. In this step, input data preparation: collect the t The sequence of prompt words and their length on node m; Initialization: set an initial cost parameter U ; Loop solution: traverse the prompt word sequence from 0, and for each possible number of malicious prompt words Solving the defender’s consistency function ,if If it is 0, skip the current loop and continue to the next one Solve and calculate adjacent The value corresponds to difference ,if Greater than the preset cost parameter , then stop the loop; output result: return the last one that meets the conditions The number of malicious words used as predictions .

[0047] The malicious prompt word number prediction method of the present application traverses the possible number of malicious prompt words and calculates the defender consistency function value corresponding to each number, thereby finding the number of malicious prompt words that makes the consistency function value increase the most. This number is considered to be the number of malicious prompt words that the system is most likely to encounter in the current round. The preset cost parameter , which can avoid unnecessary computational overhead while ensuring prediction accuracy.

[0048] Furthermore, the steps of the method for predicting the number of malicious prompt words are as follows: Step K21, input t The sequence of prompt words on node m , the sequence length is

[0049] Step K22, initialization U .

[0050] Step K23, .

[0051] Step K24, solve , Step K25, if , , return to step 4. Otherwise, calculateU = , , proceed to step 6.

[0052] Step K26, if , stop the algorithm and output . Otherwise, return to step K24.

[0053] The above, Represents the cost parameter, which is used as a stop condition in the malicious prompt word quantity prediction algorithm; As a loop variable, used for iterative calculations.

[0054] The multi-stage dynamic Bayesian game model of the present application models the interaction between benign users, malicious users and defenders, obtains the detection strategy through the belief updating method and the malicious digital prediction method, optimizes the prompt word security and optimizes the system performance.

[0055] In step S103, the prompt word is detected according to the detection probability to obtain a safety prompt word.

[0056] In a possible implementation, when the detection probability meets the preset requirement, the prompt word is tested to obtain a test result; if the test result is safe, the prompt word is determined to be a safe prompt word. If the test result is unsafe, the prompt word is returned for the next round of testing. That is, a decision is made based on the obtained detection probability.

[0057] It should be noted that this application adopts a mixed strategy, and the probability obtained is generally 0 or 1; if not, a random number is used to randomly generate a standard value through a computer. If the obtained value is greater than the standard value, a prompt word detection is performed, and if the obtained value is less than the standard value, the prompt word detection is not performed. It can be understood that, in general, the result of the mixed strategy is a probability, which can be any number between 0 and 1. Therefore, this application predicts the number of maliciousness so that the sum of the probabilities is equal to this number, so each probability becomes 1.

[0058] In step S104, the safety prompt word is input into the large language model for processing to obtain response information.

[0059] Specifically, the prompt words that are confirmed to be safe after detection by the edge node will be sent to the LLM (Large Language Model) in the cloud for processing.

[0060] This application jointly optimizes LLM security and system resources, uses multiple rounds of incomplete information dynamic Bayesian models to model the problem, and fits the actual problem; in order to reduce the problem of inaccurate detection caused by simply using the prompt word detection model, this application combines the Bayesian belief update method of VDB and historical data to improve the accuracy of detection decisions; in order to optimize system resources, this application adopts a method for predicting the number of malicious prompt words, and uses the sequential marginal analysis method to make the system resource consumption controllable.

[0061] like Figure 2 As shown, the above-mentioned prompt word detection resource optimization method of the present application is further explained below through a specific embodiment: in the first round of initialization belief, the prompt word is received, and the prompt word is matched with the database, and the edge device determines whether to perform prompt word detection. If it is (i.e., safe), the large model is input to respond. If not, the number of rounds is determined. t Is it less than the preset maximum number of rounds? t max If it is less than, the belief is updated and the update prompt word is tested. If the number of rounds is equal to or greater than the preset maximum number of rounds, the process ends.

[0062] Next, the prompt word detection resource optimization system proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0063] Figure 3 It is a structural diagram of the prompt word detection resource optimization system of the embodiment of the present application.

[0064] like Figure 3 As shown, the prompt word detection resource optimization system includes: a prompt word receiving module 100, a model building and optimization module 200, a detection strategy determination module 300 and a large model output module 400.

[0065] Specifically, the prompt word receiving module 100 is used to receive the prompt word sent by the user and identify the prompt parameter of the prompt word; The model building and optimization module 200 is used to build a multi-stage dynamic Bayesian game model and input the prompt parameters into the multi-stage dynamic Bayesian game model to obtain the detection probability; A detection strategy determination module 300, configured to detect the prompt word according to the detection probability to obtain a safety prompt word; The large model output module 400 is used to input the safety prompt word into the large language model for processing to obtain response information.

[0066] Figure 4 A structural diagram of a prompt word detection resource optimization terminal provided in an embodiment of the present application. The prompt word detection resource optimization terminal may include: A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0067] When the processor 502 executes the program, the prompt word detection resource optimization method provided in the above embodiment is implemented.

[0068] Furthermore, the prompt word detection resource optimization terminal also includes: The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0069] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0070] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0071] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EIS) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0072] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0073] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0074] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above prompt word detection resource optimization method is implemented.

[0075] One embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the present application Figure 1 The prompt word detection resource optimization method provided by any embodiment in the corresponding embodiment.

[0076] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0077] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0078] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0079] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary, and then storing it in a computer memory.

[0080] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0081] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0082] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0083] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0084] It should be understood that the application of the present application is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for optimizing prompt word detection resources, characterized in that: The prompt word detection resource optimization method comprises: Receiving a prompt word sent by a user, and identifying prompt parameters of the prompt word; Constructing a multi-stage dynamic Bayesian game model, and inputting the prompt parameter into the multi-stage dynamic Bayesian game model to obtain a detection probability; According to the detection probability, the prompt word is detected to obtain a safety prompt word; Inputting the safety prompt word into a large language model for processing to obtain response information; The multi-stage dynamic Bayesian game model is expressed as: ； ； ； ； in, is the defender’s consistency function, Indicates defender For t Each cue word of the round Whether to detect the strategy, Indicated in t Except for the defender Other good users x and malicious users y For prompt words Whether to detect the strategy, represents the set of defenders, Indicates prompt words, Indicates t The sequence of prompt words received by the round defender, It is t The number of malicious cue words in a round; The step of detecting the prompt word according to the detection probability to obtain a safe prompt word specifically includes: when the detection probability meets a preset requirement, detecting the prompt word to obtain a detection result; if the detection result is safe, determining that the prompt word is a safe prompt word.

2. The method for optimizing prompt word detection resources according to claim 1, characterized in that: The construction of a multi-stage dynamic Bayesian game model specifically includes: Construct the corresponding consistency functions for benign users, malicious users and defenders; The consistency functions corresponding to the benign user, the malicious user and the defender are minimized to obtain a multi-stage dynamic Bayesian game model.

3. The method for optimizing prompt word detection resources according to claim 2, characterized in that: The consistency function of the benign user is expressed as: ； The consistency function of the malicious user is expressed as: ； The defender's consistency function is expressed as: ; in, is the consistency function of benign users, m represents an edge node, represents the set of edge nodes, Indicates a good user x strategy, Is a good user x The prompt word is sent to the edge node m The total delay of the detection strategy and whether to send the prompt word to the large language model, represents the number of rounds of the game, is the total number of game rounds; is the consistency function of malicious users, Indicates malicious users y strategy, is the F1 score of the detection model, Indicates prompt words, Indicates the number of tokens in the prompt word. Indicates the number of floating point operations for each token; , represents two weight parameters greater than zero, represents the total delay for each cue word to be processed, represents the defender's belief about each cue word.

4. The method for optimizing prompt word detection resources according to claim 2, characterized in that: The prompt parameter is input into the multi-stage dynamic Bayesian game model to obtain the detection probability, and then the following steps are further included: According to the detection probability, the benign user, the malicious user and the defender's belief on the security of the prompt word are updated to obtain an updated belief; After updating the model according to the updated belief, an updated multi-stage dynamic Bayesian game model is obtained, and the next round of prompt parameters are input into the updated multi-stage dynamic Bayesian game model to obtain the next round of detection probability.

5. The method for optimizing prompt word detection resources according to claim 1, characterized in that: When the detection probability meets the preset requirement, the prompt word is detected to obtain a detection result, and then the following steps are further included: If the detection result is unsafe, the prompt word is returned for the next round of prompt word detection.

6. A prompt word detection resource optimization system, characterized in that: The prompt word detection resource optimization system is applied to the prompt word detection resource optimization method according to any one of claims 1 to 5; The prompt word detection resource optimization system comprises: A prompt word receiving module, used for receiving a prompt word sent by a user and identifying prompt parameters of the prompt word; A model building and optimization module, used to build a multi-stage dynamic Bayesian game model, and input the prompt parameters into the multi-stage dynamic Bayesian game model to obtain a detection probability; A detection strategy determination module, used to detect the prompt word according to the detection probability to obtain a safety prompt word; The large model output module is used to input the safety prompt word into the large language model for processing to obtain response information.

7. A prompt word detection resource optimization terminal, characterized in that: The prompt word detection resource optimization terminal includes: a memory, a processor, and a prompt word detection resource optimization program stored in the memory and executable on the processor. When the prompt word detection resource optimization program is executed by the processor, the steps of the prompt word detection resource optimization method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a prompt word detection resource optimization program, and when the prompt word detection resource optimization program is executed by a processor, the steps of the prompt word detection resource optimization method according to any one of claims 1 to 5 are implemented.

Images