Vision structure-based large model prompt word injection attack determination method and device

Abstract

The application discloses a large model prompt word injection attack judgment method and device based on a visual structure and belongs to the technical field of large language models. The method comprises the following steps: mapping the dialogue text of each visual angle into a visual structure to obtain a plurality of first visual structures; acquiring risk features in a long context dialogue; taking different dimensions of the risk features as different visual attributes and mapping the different visual attributes into the plurality of first visual structures to obtain a plurality of second visual structures; rendering each second visual structure into a visual image, mapping the visual features of each visual image into a text semantic space to obtain cross-modal semantic features of each visual angle; obtaining a risk score of each visual angle according to the cross-modal semantic features of each visual angle; and performing weighted fusion on the risk score of each visual angle to obtain a target risk score of the text to be judged. The method can reduce the consumption of computing resources and improve the accuracy of prompt word injection attack judgment in a long context scene.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and in particular to a method and apparatus for detecting large model cue word injection attacks based on visual structure. Background Technology

[0002] With the widespread application of large language models, systems based on natural language interaction face the security risk of prompt injection attacks. In prompt injection attacks, attackers construct specific inputs through role-playing, multi-turn guidance, etc., to guide the model to execute malicious instructions, thereby causing information leakage or loss of control.

[0003] The methods for detecting prompt word injection attacks in related technologies mostly rely on keyword matching, rule filtering, or text semantic classification models.

[0004] However, in scenarios involving long contexts and multi-turn dialogues with prompt-based attacks, the increased context length and semantic complexity reduce the attention large language models pay to early information, potentially affecting the accuracy of prompt injection attack detection. Furthermore, in processing long contexts, existing methods typically model based on text token sequences; as the context length increases, the computational complexity rises significantly, leading to high computational resource consumption. Summary of the Invention

[0005] This invention provides a method and apparatus for detecting cue word injection attacks in large models based on visual structure, which can improve the accuracy of cue word injection attack detection in long context scenarios while reducing computational resource consumption. The technical solution includes at least the following: Firstly, a method for determining large-scale language model prompt injection attacks based on visual structure is provided, comprising: rearranging long contextual dialogue text into multiple perspectives to obtain dialogue texts from multiple different perspectives; mapping the dialogue text from each perspective to a visual structure to obtain multiple first visual structures, wherein the long contextual dialogue text includes historical dialogue records of a large language model and the latest text to be determined input into the large language model; obtaining risk features in the long contextual dialogue; mapping different dimensions of the risk features as different visual attributes to the multiple first visual structures to obtain multiple second visual structures; rendering each second visual structure as a visual image, obtaining visual features of each visual image, mapping the visual features to a text semantic space to obtain cross-modal semantic features of each perspective; obtaining a risk score for each perspective based on the cross-modal semantic features of each perspective; and performing weighted fusion of the risk scores of each perspective to obtain a target risk score for the text to be determined, wherein the target risk score is used to determine the text to be determined for prompt injection attacks.

[0006] Optionally, the step of rearranging the long contextual dialogue text from multiple perspectives to obtain multiple dialogue texts from different perspectives, and mapping the dialogue text from each perspective to a visual structure to obtain multiple first visual structures, includes: extracting the statements of each character in the long contextual dialogue text and storing the statements of different characters into different groups, all of which constitute the dialogue text from the character aggregation perspective; randomly rearranging the long contextual text to obtain the dialogue text from the structure perturbation perspective; and mapping the dialogue text from the order-preserving perspective, the dialogue text from the character aggregation perspective, and the dialogue text from the structure perturbation perspective to visual structures respectively to obtain the multiple first visual structures, wherein the dialogue text from the order-preserving perspective is the long contextual dialogue text.

[0007] Optionally, mapping the dialogue text from the order-preserving perspective, the dialogue text from the role aggregation perspective, and the dialogue text from the structure perturbation perspective to visual structures to obtain the plurality of first visual structures includes: dividing the standardized canvas into multiple grid cells; for any perspective of dialogue text, using sentences or dialogue fragments as the smallest text block, dividing the dialogue text of each perspective into multiple text blocks; sequentially arranging each text block into the grid cells in a left-to-right order, with each text block occupying at least one grid cell, the number of grid cells occupied by each text block being determined according to the number of texts in the text block, until all grid cells in the k-th row are filled. After the text is entered, text blocks are arranged in the grid cells of the (k+1)th row from left to right. Multiple text blocks of the long context dialogue text are arranged in chronological order to obtain the first visual structure from the order-preserving perspective. For the dialogue text from the role aggregation perspective, the standardized canvas is pre-divided into multiple regions according to the number of roles, and the text blocks corresponding to the statements of different roles are arranged in the grid cells of different regions to obtain the first visual structure from the role aggregation perspective. For the dialogue text from the structure perturbation perspective, multiple text blocks are arranged according to the order in which the text blocks are arranged to obtain the first visual structure from the structure perturbation perspective.

[0008] Optionally, obtaining the risk features in the long context dialogue includes: calculating the risk features separately for each text block. The risk features include semantic risk, instruction strength, and role permissions. The semantic risk is used to characterize whether there is an intent to induce execution, unauthorized access, or potential prompt word injection attack in the text block. The instruction strength is used to measure the strength of the instruction expression in the text block. The role permissions are used to characterize the permission level of the role to which the text block belongs.

[0009] Optionally, the multiple different perspectives include an order-preserving perspective, a role aggregation perspective, and a structural disturbance perspective. In the process of weighted fusion of the risk scores for each perspective, the weight of the order-preserving perspective is greater than the weight of the role aggregation perspective, and the weight of the role aggregation perspective is greater than the weight of the structural disturbance perspective.

[0010] Optionally, the large language model method for determining visual structure-based cue word injection attacks further includes: if the target risk score is greater than a risk threshold, the large language model refuses to generate an answer to the text to be determined; if the target risk score is less than or equal to the risk threshold, the large language model generates an answer to the text to be determined.

[0011] Secondly, a large-scale language model prompt injection attack determination device based on visual structure is also provided, comprising: a first visual structure mapping module, used to rearrange long contextual dialogue text from multiple perspectives to obtain dialogue text from multiple different perspectives, and map the dialogue text from each perspective to a visual structure to obtain multiple first visual structures, wherein the long contextual dialogue text includes historical dialogue records of the large language model and the latest text to be determined input to the large language model; a first acquisition module, used to acquire risk features in the long contextual dialogue; a second visual structure mapping module, used to map different dimensions of the risk features as different visual attributes to the multiple first visual structures to obtain multiple second visual structures; a second acquisition module, used to render each second visual structure as a visual image, acquire the visual features of each visual image, and map the visual features to the text semantic space to obtain cross-modal semantic features of each perspective; a scoring acquisition module, used to obtain a risk score for each perspective based on the cross-modal semantic features of each perspective; and a scoring aggregation module, used to perform weighted fusion of the risk scores of each perspective to obtain a target risk score for the text to be determined, wherein the target risk score is used to determine the text to be determined for prompt injection attacks.

[0012] Optionally, the first visual structure mapping module is further configured to extract the statements of each character in the long contextual dialogue text and store the statements of different characters into different groups, all of which constitute the dialogue text from the character aggregation perspective; randomly rearrange the long contextual text to obtain the dialogue text from the structure perturbation perspective; map the dialogue text from the order-preserving perspective, the dialogue text from the character aggregation perspective, and the dialogue text from the structure perturbation perspective to visual structures respectively to obtain the plurality of first visual structures, wherein the dialogue text from the order-preserving perspective is the long contextual dialogue text.

[0013] Optionally, the first visual structure mapping module is further configured to divide the standardized canvas into multiple grid cells; for dialogue text from any perspective, using sentences or dialogue fragments as the smallest text blocks, divide the dialogue text from each perspective into multiple text blocks; arrange each text block in the grid cells in a left-to-right order, with each text block occupying at least one grid cell, and the number of grid cells occupied by each text block being determined according to the number of texts in the text block; after the k-th row of grid cells has been completely filled with text, continue arranging text blocks in the k+1-th row of grid cells in a left-to-right order; wherein, multiple text blocks of the long context dialogue text are arranged in chronological order to obtain the first visual structure of the order-preserving perspective; for dialogue text from the role aggregation perspective, the standardized canvas is pre-divided into multiple regions according to the number of roles, and the text blocks corresponding to the statements of different roles are arranged in the grid cells of different regions to obtain the first visual structure of the role aggregation perspective; for dialogue text from the structure perturbation perspective, multiple text blocks are arranged in the order in which the text blocks are arranged to obtain the first visual structure of the structure perturbation perspective.

[0014] Optionally, the first acquisition module is further configured to calculate the risk features for each text block individually. The risk features include semantic risk, instruction strength, and role permissions. The semantic risk is used to characterize whether there is an intent to induce execution, unauthorized access, or potential prompt word injection attack in the text block. The instruction strength is used to measure the strength of the instructional expression in the text block. The role permissions are used to characterize the permission level of the role to which the text block belongs.

[0015] Optionally, in the scoring aggregation module, the multiple different perspectives include an order-preserving perspective, a role aggregation perspective, and a structural disturbance perspective. In the process of weighted fusion of the risk scores for each perspective, the weight of the order-preserving perspective is greater than the weight of the role aggregation perspective, and the weight of the role aggregation perspective is greater than the weight of the structural disturbance perspective.

[0016] Optionally, the device further includes: a generation module, which is configured to refuse to generate an answer to the text to be judged when the target risk score is greater than a risk threshold, and to generate an answer to the text to be judged when the target risk score is less than or equal to the risk threshold.

[0017] Thirdly, a computer device is also provided, including: a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to perform the large model cue word injection attack determination method based on visual structure described in the above embodiments.

[0018] Fourthly, a computer-readable storage medium is also provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to perform the large model prompt word injection attack determination method based on visual structure described in the above embodiments.

[0019] Fifthly, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the large model prompt word injection attack determination method based on visual structure described in the first aspect.

[0020] The beneficial effects of the technical solution provided by this invention include at least the following: In this embodiment, by converting long contextual dialogues into first visual structures with different perspectives, and mapping risk features to the first visual structure to obtain a second visual structure, subsequent prompt injection attack judgment is actually implemented based on the visual image rendered by the second visual structure. Since the information density of a two-dimensional visual image is much greater than that of a one-dimensional text sequence, the visual image can store more long contextual dialogue text. This allows the model to process a longer range of contextual information with a limited input scale, thereby reducing computational resource consumption and avoiding the problem of false positives due to excessively long context in prompt injection attack scenarios. Furthermore, since risk features are explicitly mapped into the second visual structure through visual attributes, by introducing visual attributes based on risk, strong differences and strong features are formed. This allows risk features to receive higher attention weight during visual feature extraction, guiding the focus on features with significant risk characteristics when judging prompt injection attacks.

[0021] Finally, by fusing risk scores from multiple different perspectives to obtain the final target risk score, since different perspectives can express the same context from different structural dimensions, thus forming complementary information representations, it is possible to prevent the injection of prompt words with different tendencies. Therefore, it is possible to achieve a comprehensive risk score under different risks, thereby improving the accuracy and robustness of prompt word injection attack detection as a whole. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this embodiment, the accompanying drawings used in the description of the embodiment will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1A flowchart illustrating a method for determining large model cue word injection attacks based on visual structure, provided by an exemplary embodiment of the present invention, is shown. Figure 2 A flowchart of a large model cue word injection attack determination method based on visual structure provided by another exemplary embodiment of the present invention is shown; Figure 3 This is a data flow diagram of a large-scale cue word injection attack detection method based on visual structure. Figure 4 These are schematic diagrams of structural dialogue text from different perspectives; Figure 5 This diagram illustrates the structure of a large-scale cue word injection attack determination device based on visual structure, provided in an exemplary embodiment of the present invention. Figure 6 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of the present invention. Detailed Implementation

[0024] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” “third,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising” or “including” and similar terms mean that the elements or objects preceding “comprising” or “including” encompass the elements or objects listed following “comprising” or “including” and their equivalents, but do not exclude other elements or objects.

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0026] Figure 1 A flowchart illustrating an exemplary embodiment of the present invention is shown, illustrating a method for detecting large-model cue word injection attacks based on visual structure. This method can be executed by a computer device, such as a computer device deployed with a large language model. See also Figure 1 The method includes: In step 101, the long contextual dialogue text is rearranged from multiple perspectives to obtain dialogue text from multiple different perspectives. The dialogue text from each perspective is mapped to a visual structure to obtain multiple first visual structures.

[0027] Long-context dialogue text includes the historical dialogue records of the large language model and the latest text to be judged input into the large language model.

[0028] In step 102, risk features in long context dialogues are obtained.

[0029] In step 103, different dimensions of the risk features are mapped as different visual attributes to multiple first visual structures to obtain multiple second visual structures.

[0030] In step 104, each second visual structure is rendered as a visual image, the visual features of each visual image are obtained, and the visual features are mapped to the text semantic space to obtain the cross-modal semantic features of each viewpoint.

[0031] In step 105, a risk score for each viewpoint is obtained based on the cross-modal semantic features of each viewpoint.

[0032] In step 106, the risk scores of each perspective are weighted and fused to obtain the target risk score of the text to be judged.

[0033] The target risk score is used to determine whether the text to be judged is susceptible to prompt word injection attacks.

[0034] In this embodiment, by converting long contextual dialogues into first visual structures with different perspectives, and mapping risk features to the first visual structure to obtain a second visual structure, subsequent prompt injection attack judgment is actually implemented based on the visual image rendered by the second visual structure. Since the information density of a two-dimensional visual image is much greater than that of a one-dimensional text sequence, the visual image can store more long contextual dialogue text. This allows the model to process a longer range of contextual information with a limited input scale, thereby reducing computational resource consumption and avoiding the problem of false positives due to excessively long context in prompt injection attack scenarios. Furthermore, since risk features are explicitly mapped into the second visual structure through visual attributes, by introducing visual attributes based on risk, strong differences and strong features are formed. This allows risk features to receive higher attention weight during visual feature extraction, guiding the focus on features with significant risk characteristics when judging prompt injection attacks.

[0035] Finally, by fusing risk scores from multiple different perspectives to obtain the final target risk score, since different perspectives can express the same context from different structural dimensions, thus forming complementary information representations, it is possible to prevent the injection of prompt words with different tendencies. Therefore, it is possible to achieve a comprehensive risk score under different risks, thereby improving the accuracy and robustness of prompt word injection attack detection as a whole.

[0036] Figure 2The flowchart illustrates a method for determining large model cue word injection attacks based on visual structure according to an exemplary embodiment of the present invention. This method can be executed by a computer device, such as a computer device deployed with a large language model. Figure 3 This is a data flow diagram of a large model-based cue word injection attack detection method based on visual structure.

[0037] See Figures 2 to 3 The method includes: In step 201, the long contextual dialogue text is rearranged from multiple perspectives to obtain dialogue text from multiple different perspectives. The dialogue text from each perspective is mapped to a visual structure to obtain multiple first visual structures.

[0038] Long-context dialogue text includes the historical dialogue records of the large language model and the most recently input text to the large language model for judgment, where the large language model has not yet generated a response for the text to be judged. The text to be judged here is, for example, a prompt word, and it is the text input by the user into the large language model.

[0039] In this embodiment, the large language model is one that requires detection of prompt word injection attacks. The timing for detecting prompt word injection attacks is when the user inputs text to be judged into the large language model, and the large language model receives the text but has not yet generated the corresponding answer.

[0040] Normally, each time a user inputs text into the large language model, it can be the text to be judged. That is, every time a user inputs text into the large language model, the large language model needs to use the latest text input by the user as the text to be judged for prompt word injection attack judgment.

[0041] After a user has engaged in multiple rounds of dialogue with the large language model, the user's historical dialogue records with the large language model, along with the user's latest input text (equivalent to the text to be judged), constitute the long contextual dialogue text.

[0042] It should be noted that the visual structure-based large model prompt injection attack detection method in this embodiment not only has good results in long context dialogue scenarios, but can also be executed in dialogue text scenarios of normal length.

[0043] In this case, step 201 may optionally include steps 2011 to 2013 as follows. Figure 4 These are diagrams illustrating the structure of dialogue text from different perspectives. Below, we will combine them with... Figure 4 Steps 2011 to 2012 will be explained.

[0044] Step 2011: Extract the statements of each character in the long contextual dialogue text and store the statements of different characters into different groups. All groups are the dialogue text from the perspective of the aggregated characters.

[0045] In this embodiment, the long contextual dialogue text is the dialogue text ordered chronologically. Since existing large models have the ability for multiple people to share a single dialogue, the dialogue text from the role aggregation perspective will include at least one user role (and possibly multiple) as well as the AI ​​large model role.

[0046] When collecting long-context dialogue text, the speeches of different roles have been pre-labeled. Therefore, the speeches of the same role can be directly obtained and stored in the same group, while the speeches of different roles are stored in different groups, resulting in M ​​groups. These M groups represent the dialogue text from the perspective of each role. Furthermore, the speeches within each group are arranged in chronological order. M is a positive integer and M is greater than or equal to 2 (because there is at least one user and one large AI model, meaning there are at least two roles, corresponding to at least two groups).

[0047] like Figure 4 As shown, the original statements in the long context dialogue text are arranged in chronological order as follows: A1, B1, A2, B2, C1, B3. The same letters represent statements made by the same character, that is... Figure 4 The long contextual dialogue text contains three characters, which can be divided into three groups in the dialogue text from the perspective of the characters. The first group includes A1 and A2, the second group includes C1, and the third group includes B1, B2, and B3. The order in which each character speaks within a group is also arranged in chronological order.

[0048] Step 2012: Randomly rearrange the long context text to obtain the dialogue text from the perspective of structural perturbation.

[0049] The random rearrangement here refers to randomly shuffling and rearranging the original time order in a long context text.

[0050] For example, such as Figure 4 As shown, the original statements in the long context dialogue text are arranged in chronological order as: A1, B1, A2, B2, C1, B3. After random rearrangement, the statements in the dialogue text obtained from the perspective of structural perturbation are arranged in the order of A1, B2, C1, A2, B3, B1. That is, the order of the statements has been shuffled.

[0051] Step 2013: Map the dialogue text from the order-preserving perspective, the dialogue text from the role aggregation perspective, and the dialogue text from the structural perturbation perspective to visual structures respectively, to obtain multiple first visual structures.

[0052] In this context, the dialogue text from the order-preserving perspective is the same as the long-context dialogue text. Or, in other words, the dialogue text from the order-preserving perspective is identical to the long-context dialogue text.

[0053] In summary, the multiple perspectives in this embodiment are the order-preserving perspective, the role aggregation perspective, and the structural perturbation perspective. The order-preserving perspective maintains the original temporal order of the dialogue text, thus preserving contextual dependencies. The role aggregation perspective groups the dialogue text according to roles, ensuring that the dialogue order within different role groups remains consistent with the original order (i.e., chronological order). The structural perturbation perspective introduces randomness into the dialogue text, constructing a non-sequentially dependent representation.

[0054] Optionally, step 2013 includes the following three steps.

[0055] The first step is to divide the standardized canvas into multiple grid units.

[0056] The size of a standard canvas can be represented as Where H is the canvas length and W is the canvas width. Each grid cell is the same size, and each grid cell is preset to hold a maximum of M text characters. The size of the grid cells and the value of M are empirical values.

[0057] The second step is to divide the dialogue text from any perspective into multiple text blocks, using sentences or dialogue fragments as the smallest text blocks.

[0058] For example, when using sentences as the smallest text block, sentences in dialogue text are separated by punctuation marks such as commas or periods; that is, the text between two adjacent punctuation marks constitutes one text block. When using dialogue segments as the smallest text block, a single speech by a character is typically considered as one dialogue segment, such as... Figure 4 A1 in the text is a dialogue segment.

[0059] Dividing dialogue text into sentences or dialogue fragments as the smallest text blocks can ensure the semantic integrity of the text within each text block.

[0060] The third step is to arrange each text block into the grid cells in order from left to right.

[0061] Each text block occupies at least one grid cell. The number of grid cells occupied by each text block is determined by the number of texts in the text block. After all the grid cells in the k-th row have been filled with text, the text blocks continue to be arranged in the grid cells in the k+1-th row in order from left to right.

[0062] The first visual structure obtained in this embodiment is not a real image, but a preset image arrangement rule or image layout description, which is used to define the arrangement order, arrangement position, and visual attributes of elements (such as text to be rendered) on the image. If it is necessary to obtain the image corresponding to the first visual structure, it is also necessary to render it based on the first visual structure (for example, through an image rendering library or graphics drawing interface).

[0063] Here, when arranging text blocks into grid cells, since the number of texts that can be arranged in each grid cell is fixed, and the number of texts in a text block is also fixed, the number of grid cells occupied by each text block is also fixed, represented as follows: Where T is the number of texts in the text block, and M is the number of texts that can be arranged in a single grid cell. This indicates rounding up to the nearest integer.

[0064] The dialogue texts from the three perspectives have already been obtained. When arranging the text blocks, they can be arranged according to the order in which these text blocks are arranged in the dialogue texts of the corresponding perspectives.

[0065] For example, arranging multiple text blocks in a long contextual dialogue text according to chronological order yields the first visual structure from the perspective of maintaining order; for dialogue text from the perspective of structural perturbation, arranging multiple text blocks according to their arrangement order in the text yields the first visual structure from the perspective of structural perturbation.

[0066] In some embodiments, the first dialogue text from the structural perturbation perspective may not be required. Instead, the grid to which the text block belongs may be randomly rearranged based on the first visual structure from the order-preserving perspective to obtain the first visual structure from the structural perturbation perspective.

[0067] The first visual structure from the order-preserving perspective can maintain the original contextual dependencies of long-context dialogue text, while the first visual structure from the structural perturbation perspective can weaken the dependence on the original order and enhance robustness against potential attack patterns.

[0068] In the first visual structure from the perspective of character aggregation, the standardized canvas can be pre-divided into multiple regions according to the number of characters. The text blocks corresponding to the statements of different characters are arranged in the grid cells of different regions, thus obtaining the first visual structure from the perspective of character aggregation. Here, the number of characters is the same as the number of regions divided by the standardized canvas, that is, each region's grid cells are used to arrange the text blocks corresponding to the statements of one character, which can enhance the aggregated expression of semantics of the same character.

[0069] Through step 201 above, text from different perspectives can be mapped into a two-dimensional visual structure, which can then be converted into a two-dimensional visual image through rendering. Since the information density of a two-dimensional visual image is much greater than that of a one-dimensional text sequence, the visual image can store more long-term contextual dialogue text, enabling the model to process a longer range of contextual information with a limited input scale, thereby reducing computational complexity.

[0070] In step 202, risk features in long context dialogues are obtained.

[0071] Optionally, risk features are calculated separately for each text block. Risk features include semantic risk, instruction strength, and role permissions. Semantic risk is used to characterize whether there is an intent to induce execution, unauthorized access, or potential prompt word injection attack in the text block. Instruction strength is used to measure the strength of the instruction expression in the text block. Role permissions are used to characterize the permission level of the role to which the text block belongs.

[0072] Semantic risk, instruction strength, and role permissions all exist in the form of probability (or level, category, etc.), and the higher the value, the higher the corresponding risk. For example, the higher the probability value of the semantic risk of a text block, the stronger the semantic risk of the text block, and the more likely it is to contain intentions of induced execution, unauthorized access, or potential prompt word injection attacks.

[0073] Optionally, the methods for obtaining semantic risk include, but are not limited to: matching methods based on predefined rules (such as sensitive instruction template matching, unauthorized keyword matching), evaluation methods based on statistical features (such as statistical analysis based on the distribution characteristics of instructional expressions in the text), and semantic discrimination methods based on pre-trained models (such as small semantic classification models based on the Transformer architecture), or any combination of the above methods. For example, when there are sensitive instructions or unauthorized keywords in a text block, the semantic risk value of the text block is generated according to the number of sensitive instructions or unauthorized keywords; the more sensitive instructions or unauthorized keywords there are, the greater the semantic risk value of the text block.

[0074] Optionally, the instruction strength can be quantified based on sentence structure features (such as imperative sentences and conditional sentences), intensity words (such as "must," "immediately," etc.), or model prediction results. For example, if imperative sentences and conditional sentences exist in a text block, the instruction strength value of the text block is generated based on the number of imperative sentences and conditional sentences; the more imperative sentences and conditional sentences there are, the greater the instruction strength value of the text block.

[0075] Optionally, role permissions are used to characterize the permission level of the role to which the text block belongs in the system, and this permission level can be preset.

[0076] In step 203, different dimensions of the risk features are mapped as different visual attributes to multiple first visual structures to obtain multiple second visual structures.

[0077] Here, both the first and second visual structures are essentially preset image arrangement parameters (or structural representations), and they still require rendering to obtain the visual image. Mapping different dimensions of risk features as different visual attributes to the first visual structure is equivalent to inserting multiple visual attribute parameters into the first visual structure, thereby enhancing the first visual structure.

[0078] In implementation, the visual attribute parameters of the text block are determined according to the dimension of the risk characteristics of the text block, and then the visual attribute parameters are inserted into the corresponding positions of each text block in the first visual structure.

[0079] For example, visual attributes include color attributes, contrast attributes, density attributes, and position offset attributes.

[0080] Optionally, color attributes can be used to represent different role permissions or risk categories. For example, different colors can be used to represent different role permissions. These colors can be the text color in the grid cell or the background color of the grid cell. For instance, red can represent level 1 (20%) role permissions. If the role permission of a text block in a certain viewpoint is level 1, then when the text block is placed in the grid cell, the text color of the text block will be changed to red. In addition, contrast attributes can be used to highlight high-risk text blocks. For example, when a high-risk text block is placed in the grid cell, the brightness difference between the text and the background in that text block can be increased.

[0081] Optionally, a density attribute can be used to represent the semantic risk level of a text block. Here, density refers to the density of text arrangement, such as line spacing, character spacing, and font weight. For example, a higher semantic risk level corresponds to larger or smaller character spacing.

[0082] Optionally, a position offset attribute can be used to represent the instruction strength or contextual importance of the text block. Here, the position offset attribute refers to the degree of offset or deformation of the text within the grid cell. For example, the greater the instruction strength, the greater the degree of offset or deformation of the text within the grid cell.

[0083] When determining the visual attribute parameters of a text block based on its risk features, a mapping relationship can be established between each dimension of the risk features and its corresponding visual attribute (e.g., a mapping relationship between role permissions and color attributes, semantic risk and density attributes, and instruction strength and position offset attributes). This allows the visual attribute values ​​of the text block to be directly obtained from the values ​​of each dimension of the risk features. These visual attribute values ​​are then embedded into a first visual structure, thus mapping the risk features to the first visual structure and obtaining a second visual structure. In the second visual structure, the visual attribute values ​​indicated by the risk features and the spatial position of the text block together constitute a complete visual structural representation.

[0084] Since the second visual structure is not a rendered image, this invention essentially achieves a structured representation of risk information before image generation. In the subsequent visual image rendered based on the second visual structure, risk features are also explicitly present in the form of different visual attributes.

[0085] Compared to relying solely on the model to learn risk patterns in the implicit semantic space, step 203 can enhance the visual saliency of high-risk areas, enabling risk features to be strongly expressed at the input stage. This guides the model to prioritize potential attack information during the feature extraction stage, thereby improving the accuracy and stability of risk detection.

[0086] Embedding risk information encoding of multi-dimensional visual attributes into the second visual structure helps guide model attention and solve the problem of semantic fragmentation in long contexts. This is because subsequent visual models typically assign attention weights based on saliency features when processing the input image (the visual image rendered by the second visual structure). By introducing visual attributes such as color, contrast, and density based on risk, forming strong differences and strong features, risk features can receive higher attention weights during the visual model's extraction of visual features. Compared to the attention mechanism that relies on positional encoding and contextual propagation in plain text sequences, the above-mentioned expression based on visual saliency can simultaneously present global information, allowing the model to escape the long-distance diminishing relevance effect, thereby reducing the semantic fragmentation problem caused by attention decay in long context scenarios.

[0087] In step 204, each second visual structure is rendered as a visual image, the visual features of each visual image are obtained, and the visual features are mapped to the text semantic space to obtain the cross-modal semantic features of each viewpoint.

[0088] For example, rendering can be performed using an image rendering library or a graphics drawing interface to render each second visual structure as a visual image.

[0089] After obtaining visual images from various perspectives, visual models can be used to extract the visual features of each visual image. These visual features are the overall features of the visual image, naturally including text content, spatial layout, and risk coding information.

[0090] For example, the visual model includes a visual coding model based on the Transformer architecture or other models capable of global modeling.

[0091] This invention does not address the traditional problem of "text recognition in images," but rather focuses on parsing images actively constructed from text. Its core lies in directly modeling visual expressions containing semantic structural information, thereby avoiding the limitations of traditional sequential text-based modeling in long-context scenarios.

[0092] Compared to text sequence-based modeling, this invention uses visual images to represent long contexts, offering the following advantages: Firstly, visual images represent text through spatial layout, mapping the originally linearly growing sequence information into a two-dimensional structure and compressing the two-dimensional structure, enabling the model to process a longer range of context information with a limited input scale, thereby reducing computational complexity. Secondly, visual images explicitly express the associations between contexts through spatial adjacency relationships, allowing long-distance semantic relationships to be directly modeled through spatial location, thus avoiding semantic breaks caused by long-distance attenuation or truncation in long sequence processing.

[0093] After acquiring the visual features of each visual image, dimensionality reduction can be performed on each visual feature through feature compression or pooling operations to reduce the size of the feature representation while retaining key global structural information and risk coding information, thus achieving a balance between information compression and semantic preservation.

[0094] Since large language models typically receive input in text form or semantic space representation, it is necessary to transform visual features into a unified representation space (i.e., text semantic space) aligned with text semantics to obtain cross-modal semantic features, thereby achieving the fusion and compatibility of visual structural information and text semantic information.

[0095] In implementation, a cross-modal alignment model can be used to map the visual features of each perspective to the text semantic space, thereby obtaining the cross-modal semantic features of each perspective. For example, the cross-modal alignment model could be a visual-language alignment model or another cross-modal representation learning model. This cross-modal alignment model can map the features of the visual modality to the text semantic space. There are many related technologies regarding the implementation methods of cross-modal alignment models, which will not be detailed here.

[0096] By mapping visual features to the text semantic space, cross-modal semantic features are obtained. These cross-modal semantic features simultaneously contain text semantic information, spatial structure information, and risk coding information, thus forming a contextual representation with global structure awareness capabilities.

[0097] In step 205, a risk score for each viewpoint is obtained based on the cross-modal semantic features of each viewpoint.

[0098] Optionally, step 205 includes: inputting the cross-modal semantic features of each viewpoint into a lightweight Transformer model to obtain the risk score of each viewpoint output by the lightweight Transformer model.

[0099] Since this embodiment includes three perspectives, the risk scores can be obtained from the order preservation perspective, the role aggregation perspective, and the structural disturbance perspective.

[0100] The lightweight Transformer model takes cross-modal semantic features as input and outputs a probability value or multi-class risk score representing the risk level of the prompt word injection.

[0101] In one possible implementation, the lightweight Transformer model can be constructed through a combination of pre-training and task fine-tuning. The pre-training phase can learn basic semantic representation capabilities based on general corpora or dialogue data, while the fine-tuning phase involves supervised training based on cue word injection attack-related data to obtain the ability to discriminate risk features.

[0102] In another possible implementation, the lightweight Transformer model can also be trained directly using a labeled dataset. The samples in this labeled dataset are cross-modal semantic features, and each sample is labeled with a risk score.

[0103] In step 206, the risk scores of each perspective are weighted and fused to obtain the target risk score of the text to be judged.

[0104] The target risk score is used to determine whether the text to be judged is susceptible to prompt word injection attacks.

[0105] Optionally, when weighting and fusing the risk scores for each perspective, the weight of the order preservation perspective is greater than the weight of the role aggregation perspective, and the weight of the role aggregation perspective is greater than the weight of the structural disturbance perspective.

[0106] The order-preserving perspective has the highest weight, which is convenient for emphasizing contextual semantic dependencies and is used to detect context-based progressively induced prompt injection attacks. The role aggregation perspective is given the second highest weight to highlight the permission overstepping characteristics in cross-role interactions, which is helpful for identifying the overstepping intent of specific roles. The structural perturbation perspective is given a relatively low but non-zero weight to retain the ability to detect order-independent attack patterns, so that the model can focus on the semantics itself rather than the sequence position, while reducing the false positives that may exist in the case of out-of-order.

[0107] For example, the weight of the order preservation perspective is 0.5, the weight of the role aggregation perspective is 0.3, and the weight of the role aggregation perspective is 0.2.

[0108] Ablation experiments conducted from different perspectives revealed that the order-preserving perspective is mainly used to maintain the semantic continuity of the context, the role aggregation perspective helps to strengthen the ability to identify the authority boundaries between roles, while the structural perturbation perspective improves the adaptability to changes in attack patterns by breaking the order dependency. Multi-perspective fusion can provide complementary information under different attack modes, thereby significantly improving the overall detection performance.

[0109] By fusing risk scores from multiple different perspectives to obtain the final target risk score, since different perspectives can express the same context from different structural dimensions, thus forming complementary information representations, it is possible to prevent the injection of prompt words with different tendencies. Therefore, it is possible to achieve a comprehensive risk score under different risks, thereby improving the robustness of prompt word injection attack detection as a whole.

[0110] The resulting target risk score can accurately and comprehensively identify prompt injection attacks without the semantic breakage problem commonly encountered in long context dialogue processing.

[0111] At this point, the relationship between the target risk score and the risk threshold can be compared to determine how the large language model should process the text to be judged.

[0112] In this embodiment, a higher target risk score indicates a higher probability that the text to be judged belongs to a prompt word injection attack. Therefore, if the target risk score is greater than the risk threshold, it indicates that the text to be judged is very likely to belong to a prompt word injection attack. At this time, the large language model should not directly generate an answer to the text to be judged, but should reject it or make further judgments, that is, jump to step 208.

[0113] Therefore, if the target risk score is less than or equal to the risk threshold, it indicates that the text to be judged does not belong to the prompt word injection attack. At this time, the large language model can normally generate the answer to the text to be judged, that is, jump to step 207.

[0114] Among them, the risk threshold can be set as an empirical value based on the security level. For example, different security levels can be preset with risk thresholds, and then the risk threshold corresponding to the current security level of the large language model can be obtained.

[0115] In step 207, the large language model generates an answer to the text to be judged.

[0116] If the target risk score is less than or equal to the risk threshold, it indicates that the text to be judged does not belong to the prompt word injection attack. At this time, the cross-modal semantic features obtained from the order-preserving perspective can be directly input into the large language model, so that the large language model can generate the answer to the text to be judged based on the cross-modal semantic features.

[0117] In step 208, the large language model refuses to generate an answer to the text to be judged.

[0118] In some embodiments, steps 205 to 206 above may employ a hierarchical judgment mechanism. For example, a lightweight model may be used to perform preliminary screening of cross-modal semantic features to determine whether there is uncertainty in the cross-modal semantic features. If there is uncertainty, a model with stronger computing power (such as the lightweight Transformer model in step 205) may be called for further judgment, thereby reducing the overall consumption of computing resources while ensuring the detection effect.

[0119] The aforementioned cross-modal semantic features are used by both the lightweight Transformer model and the large language model under normal business conditions, avoiding the repetitive feature extraction process. Therefore, security detection will not lead to excessive resource consumption.

[0120] For example, a Chinese dataset is used to verify the technical effectiveness of the rule-matching-based prompt injection attack detection method, the large-model (Deepseek-R1)-based prompt injection attack detection method, and the visual structure-based large-model prompt injection attack detection method of this invention. Under the same model size and computational resources, the detection accuracy and other metrics are evaluated. This Chinese dataset contains 600 multi-turn dialogue samples (half positive examples and half negative examples), using a unified structured representation. Each data point consists of a context sequence organized chronologically and a binary tag. The context typically contains 3 to 8 turns of dialogue, involving multi-role interactions (such as User1, User2, and AI) to simulate the semantic evolution process in real human-computer or multi-person collaborative scenarios. In terms of length design, single-turn texts range from short questions to long technical descriptions, and the overall context covers a distribution from short to long texts, thus ensuring the model's robustness under different context window pressures. In terms of content types, the data covers various task formats such as daily Q&A, professional consultation, system design, academic discussions, and engineering document generation, ensuring the diversity and realism of the semantic space. The test results are shown in Table 1.

[0121] Table 1: Test metrics for different prompt word injection attack detection methods.

[0122]

[0123] As shown in Table 1, compared with the rule-matching-based and Deepseek-R1-based methods for detecting prompt injection attacks, the visual structure-based large-model method for detecting prompt injection attacks in this invention improves accuracy, precision, recall, and F1 score. Accuracy is improved by approximately 6%-10%. Furthermore, the average number of tokens in Table 1 refers to the average number of tokens input to the large model. On the same dataset, the visual structure-based large-model method for detecting prompt injection attacks in this invention has a lower average number of input tokens, indicating that the visual structure in this invention can effectively compress the number of tokens that need to be processed, reducing it by approximately 30%-60% depending on the degree of visual structure compression. By reducing the number of input tokens, the resource consumption of the large model in detecting prompt injection attacks and generating responses can be reduced.

[0124] The following are device embodiments of this application. For details not described in detail in the device embodiments, please refer to the above method embodiments.

[0125] Figure 5 This diagram illustrates the structure of a large-scale cue word injection attack detection device based on visual structure, provided in an exemplary embodiment of the present invention. See also... Figure 5 The visual structure-based large model prompt word injection attack determination device 500 includes: a first visual structure mapping module 501, a first acquisition module 502, a second visual structure mapping module 503, a second acquisition module 504, a score acquisition module 505, a score aggregation module 506, and a generation module 507.

[0126] The first visual structure mapping module 501 is used to rearrange the long contextual dialogue text in multiple perspectives to obtain multiple dialogue texts from different perspectives. The dialogue text from each perspective is mapped to a visual structure to obtain multiple first visual structures. The long contextual dialogue text includes the historical dialogue records of the large language model and the latest text to be judged input to the large language model. The first acquisition module 502 is used to acquire risk features in long context dialogues; The second visual structure mapping module 503 is used to map different dimensions of risk features as different visual attributes to multiple first visual structures to obtain multiple second visual structures. The second acquisition module 504 is used to render each second visual structure as a visual image, acquire the visual features of each visual image, map the visual features to the text semantic space, and obtain the cross-modal semantic features of each viewpoint. The scoring acquisition module 505 is used to obtain the risk score for each viewpoint based on the cross-modal semantic features of each viewpoint; The scoring aggregation module 506 is used to perform weighted fusion of the risk scores from each perspective to obtain the target risk score of the text to be judged. The target risk score is used to judge the text to be judged for prompt word injection attacks.

[0127] Optionally, the first visual structure mapping module 501 is also used to extract the statements of each character in the long contextual dialogue text and store the statements of different characters into different groups. All groups are the dialogue text from the perspective of character aggregation. The long contextual text is randomly rearranged to obtain the dialogue text from the perspective of structural perturbation. The dialogue text from the perspective of order preservation, the dialogue text from the perspective of character aggregation, and the dialogue text from the perspective of structural perturbation are mapped to visual structures respectively to obtain multiple first visual structures. The dialogue text from the perspective of order preservation is the long contextual dialogue text.

[0128] Optionally, the first visual structure mapping module 501 is further used to divide the standardized canvas into multiple grid cells; for dialogue text of any perspective, taking sentences or dialogue fragments as the smallest text blocks, the dialogue text of each perspective is divided into multiple text blocks; each text block is arranged in the grid cells in order from left to right, each text block occupies at least one grid cell, and the number of grid cells occupied by each text block is determined according to the number of texts in the text block. After the grid cells in the kth row have been completely filled with text, the text blocks are arranged in the grid cells in the k+1th row in order from left to right; wherein, multiple text blocks of long context dialogue text are arranged in chronological order to obtain the first visual structure of the order-preserving perspective; for dialogue text of the role aggregation perspective, the standardized canvas is pre-divided into multiple regions according to the number of roles, and the text blocks corresponding to the statements of different roles are arranged in the grid cells of different regions to obtain the first visual structure of the role aggregation perspective; for dialogue text of the structure perturbation perspective, multiple text blocks are arranged in the order in which the text blocks are arranged to obtain the first visual structure of the structure perturbation perspective.

[0129] Optionally, the first acquisition module 503 is also used to calculate risk features for each text block separately. The risk features include semantic risk, instruction strength, and role permissions. Semantic risk is used to characterize whether there is an intention to induce execution, unauthorized access, or potential prompt word injection attack in the text block. Instruction strength is used to measure the strength of the instruction expression in the text block. Role permissions are used to characterize the permission level of the role to which the text block belongs.

[0130] Optionally, in the scoring aggregation module 506, multiple different perspectives, including the order preservation perspective, the role aggregation perspective, and the structural disturbance perspective, are used. In the process of weighted fusion of the risk scores of each perspective, the weight of the order preservation perspective is greater than the weight of the role aggregation perspective, and the weight of the role aggregation perspective is greater than the weight of the structural disturbance perspective.

[0131] Optionally, the device further includes: a generation module 507, which is used to refuse to generate an answer to the text to be judged when the target risk score is greater than the risk threshold, and to generate an answer to the text to be judged when the target risk score is less than or equal to the risk threshold.

[0132] It should be noted that the above-described embodiment of the large model prompt injection attack determination device based on visual structure is only illustrated by the division of the functional modules described above. In practical applications, the functions described above can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the above-described embodiment of the large model prompt injection attack determination device based on visual structure and the embodiment of the method for determining large model prompt injection attacks based on visual structure belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0133] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods are possible. Furthermore, the functional modules in each embodiment of the invention can be integrated into a single processor, exist as separate physical entities, or consist of two or more modules integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0134] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a personal computer, mobile phone, or communication device, etc.) or processor to execute all or part of the steps of the method of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] Figure 6 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of the present invention. For example... Figure 6 As shown, the computer device 600 includes a processor 601 and a memory 602.

[0136] Processor 601 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0137] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 is used to store at least one instruction, which is executed by the processor 601 to implement the large model cue word injection attack determination method based on visual structure provided in this embodiment of the invention.

[0138] Those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on the computer device 600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0139] This invention also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of a computer device, enables the computer device to execute the large model cue word injection attack determination method based on visual structure provided in this invention.

[0140] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the visual structure-based large model prompt word injection attack determination method provided in this invention.

[0141] The above description is merely an optional embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting large-scale cue word injection attacks based on visual structure, characterized in that, The method for detecting large model cue word injection attacks based on visual structure includes: The long contextual dialogue text is rearranged from multiple perspectives to obtain multiple dialogue texts from different perspectives. The dialogue text from each perspective is mapped to a visual structure to obtain multiple first visual structures. The long contextual dialogue text includes the historical dialogue records of the large language model and the latest text to be judged input into the large language model. Obtain the risk characteristics in the long context dialogue; The different dimensions of the risk features are mapped as different visual attributes to the multiple first visual structures to obtain multiple second visual structures; Each of the second visual structures is rendered as a visual image, the visual features of each visual image are obtained, and the visual features are mapped to the text semantic space to obtain the cross-modal semantic features of each viewpoint. Based on the cross-modal semantic features of each perspective, a risk score is obtained for each perspective; The risk scores from each perspective are weighted and fused to obtain the target risk score of the text to be judged. The target risk score is used to judge the text to be judged for prompt word injection attack.

2. The method for determining large model cue word injection attacks based on visual structure according to claim 1, characterized in that, The process involves rearranging long contextual dialogue text from multiple perspectives to obtain dialogue text from multiple different perspectives. Each perspective's dialogue text is then mapped to a visual structure, resulting in multiple first visual structures, including: Extract the statements of each character in the long contextual dialogue text, and store the statements of different characters into different groups. All groups are the dialogue text from the perspective of the aggregated characters. The long context dialogue text is randomly rearranged to obtain dialogue text from a structural perturbation perspective; The dialogue text from the order-preserving perspective, the dialogue text from the role aggregation perspective, and the dialogue text from the structure perturbation perspective are respectively mapped to visual structures to obtain the multiple first visual structures. The dialogue text from the order-preserving perspective is the long context dialogue text.

3. The method for determining large model cue word injection attacks based on visual structure according to claim 2, characterized in that, The process of mapping the dialogue text from the order-preserving perspective, the dialogue text from the role aggregation perspective, and the dialogue text from the structure perturbation perspective to visual structures, respectively, yields the plurality of first visual structures, including: Divide the standardized canvas into multiple grid units; For dialogue text from any perspective, the smallest text block is a sentence or dialogue fragment, and the dialogue text from each perspective is divided into multiple text blocks. Arrange each text block into the grid cells in order from left to right. Each text block occupies at least one grid cell. The number of grid cells occupied by each text block is determined according to the number of texts in the text block. After the grid cells in row k have been completely filled with text, continue to arrange text blocks in the grid cells in row k+1 in order from left to right. Among them, multiple text blocks of the long context dialogue text are arranged in chronological order to obtain the first visual structure from the order-preserving perspective. For the dialogue text from the perspective of the character aggregation, the standardized canvas is pre-divided into multiple regions according to the number of characters, and the text blocks corresponding to the statements of different characters are arranged in the grid cells of different regions to obtain the first visual structure of the character aggregation perspective. For the dialogue text from the structural perturbation perspective, multiple text blocks are arranged according to the order in which the text blocks are arranged in the dialogue text from the structural perturbation perspective to obtain the first visual structure from the structural perturbation perspective.

4. The method for determining large model cue word injection attacks based on visual structure according to claim 3, characterized in that, The process of obtaining risk features from the long context dialogue includes: The risk features are calculated separately for each text block. The risk features include semantic risk, instruction strength, and role permissions. The semantic risk is used to characterize whether there is an intent to induce execution, unauthorized access, or potential prompt word injection attack in the text block. The instruction strength is used to measure the strength of the instruction expression in the text block. The role permissions are used to characterize the permission level of the role to which the text block belongs.

5. The method for determining large model cue word injection attacks based on visual structure according to any one of claims 1 to 4, characterized in that, The multiple different perspectives include an order-preserving perspective, a role aggregation perspective, and a structural disturbance perspective. In the process of weighted fusion of the risk scores for each perspective, the weight of the order-preserving perspective is greater than the weight of the role aggregation perspective, and the weight of the role aggregation perspective is greater than the weight of the structural disturbance perspective.

6. The method for determining large model cue word injection attacks based on visual structure according to any one of claims 1 to 4, characterized in that, The method for detecting large model cue word injection attacks based on visual structure also includes: If the target risk score is greater than the risk threshold, the large language model refuses to generate an answer for the text to be judged; If the target risk score is less than or equal to the risk threshold, the large language model generates an answer to the text to be judged.

7. A device for detecting large-scale cue word injection attacks based on visual structure, characterized in that, The device includes: The first visual structure mapping module is used to rearrange long contextual dialogue texts from multiple perspectives to obtain dialogue texts from multiple different perspectives. The dialogue texts from each perspective are mapped to visual structures to obtain multiple first visual structures. The long contextual dialogue texts include the historical dialogue records of the large language model and the latest text to be judged input into the large language model. The first acquisition module is used to acquire risk features in the long context dialogue; The second visual structure mapping module is used to map different dimensions of the risk features as different visual attributes to the multiple first visual structures to obtain multiple second visual structures. The second acquisition module is used to render each of the second visual structures into a visual image, acquire the visual features of each visual image, map the visual features to the text semantic space, and obtain the cross-modal semantic features of each viewpoint. The scoring acquisition module is used to obtain the risk score for each perspective based on the cross-modal semantic features of each perspective; The scoring aggregation module is used to perform weighted fusion of the risk scores from each perspective to obtain the target risk score of the text to be judged. The target risk score is used to judge the text to be judged for prompt word injection attacks.

8. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images