A large model defense method based on multi-view image-text conversion

Abstract

This invention discloses a large-scale defense method based on multi-view image-text conversion, relating to the field of artificial intelligence security technology. The method includes: acquiring the text of image-text dialogue prompts, extracting the syntactic structure parameters and semantic parameters of the statements, concatenating them into a multimodal tensor, and expanding the concatenated vector into multiple structural vectors based on semantic relation density to form a semantic structure vector group. In this invention, the semantic density decomposition of the multimodal tensor and image space mapping enhance cross-modal correlation analysis capabilities, accurately identify mixed image-text attack paths, and combine the entity mapping relationship between pixel gradients and region masks to achieve bidirectional verification of image-text semantic consistency, eliminating semantic gap vulnerabilities in single-modal detection. A path density and dynamic weight fusion mechanism for instruction words is adopted to quantify the abnormal risk level of the statement structure, overcoming the limitations of static rules in adapting to semantic variations and reducing the probability of outputting illegal content.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and in particular to a large-scale model defense method based on multi-view image-text conversion. Background Technology

[0002] The field of artificial intelligence security technology encompasses research on technologies related to the identification, defense, and control of various security threats faced by artificial intelligence systems during operation. Core aspects of this technology include protection against model adversarial attacks, detection of malicious prompts, improvement of model robustness, prevention of information injection, and privacy protection. By introducing input content monitoring, model behavior constraints, and multimodal information fusion, it achieves end-to-end security control over model input, internal reasoning processes, and output results. It focuses on solving the problem of models outputting illegal, unauthorized, or potentially dangerous content due to receiving misleading input in open-ended dialogue, text and image generation, and various task scenarios. Artificial intelligence security technology is the core technological support for ensuring the trustworthy use of large models.

[0003] Traditional AI security technologies rely on unimodal text detection and static rule matching, lacking effective mechanisms for identifying cross-modal induced attacks. Traditional malicious prompt word detection only analyzes surface-level text features, failing to address covert inducements achieved through image embedding or grammatical structure camouflage, leading to unauthorized command generation by the model. Semantic consistency verification uses one-way text comparison, ignoring potential differences in multi-perspective image-text conversion, making it difficult to detect logical misinterpretations caused by cross-modal semantic gaps. Static rule bases and keyword blacklists are updated laggingly, failing to cover new semantic variation attacks, including malicious inputs constructed through synonym substitution and grammatical recombination. The ability to analyze complex sentence structure paths is insufficient, relying on word frequency or simple dependency relationships for judgment, failing to quantify the correlation risk between path span and command word distribution, resulting in misjudgments or omissions. Information injection prevention relies on manually labeled data to train classifiers, lacking real-time response capabilities to zero-shot attacks in open-domain generation scenarios, leading to significant defense lag. Model behavior constraint mechanisms are not deeply bound to input semantic reconstruction, failing to dynamically correct abnormal reasoning paths during generation, posing a risk of uncontrolled output results. Summary of the Invention

[0004] To address the technical problems existing in the prior art, this invention provides a large-model defense method based on multi-view image-text conversion. The technical solution is as follows:

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a large-model defense method based on multi-view image-text conversion, comprising the following steps:

[0006] S1: Obtain the text of the image-text dialogue prompts, extract the syntactic structure parameters and semantic parameters of the sentences, and concatenate them into a multimodal tensor. Based on the semantic relation density, expand the concatenated multimodal tensor into multiple structural vectors to form a semantic structure vector group.

[0007] S2: Call the semantic structure vector group, analyze the semantic level value of each vector and divide it into multiple subsets according to the level interval, input the information of multiple subsets into the image generation channel respectively, perform region localization in the image space according to the vector offset angle and activation weight, construct a salient feature distribution map, and form an image generation matrix;

[0008] S3: Call the image generation matrix to extract image pixel density, edge gradient and region mask data, calculate entity mapping relationship through the intersection of pixels and semantic annotations, split and reassemble corresponding sentence fragments to generate semantic vectors, compare with image embedding vectors to calculate difference sequence, and obtain semantic reconstruction offset;

[0009] S4: Call the semantic reconstruction offset, locate the abnormal fragment in the prompt word according to the offset range, extract the position, connection node and span data of the sentence structure path, calculate the path density, and generate a risk weight value by combining the proportion of instruction phrases.

[0010] Optionally, the semantic structure vector group includes semantic level labels, location index parameters, and structural dependency markers; the image generation matrix specifically includes a region feature density map, channel index identifiers, and generation coordinate labels; the semantic reconstruction offset includes semantic segment mapping difference, sentence structure offset segment, and vector residual index; and the risk weight value specifically includes path density factor, induced component proportion coefficient, and structural span ratio.

[0011] Optionally, S1 obtains the text of the image-text dialogue prompts, extracts the syntactic structure parameters and semantic parameters of the statements, and concatenates them into a multimodal tensor. Based on the semantic relation density, the concatenated multimodal tensor is expanded into multiple structural vectors to form a semantic structure vector group, including:

[0012] S101: Obtain the text of the image-text dialogue prompts, extract the part-of-speech type, position number and grammatical dependency relationship of each word, establish a word dependency mapping index sequence based on the part-of-speech tag and word order position, and generate a syntactic structure index sequence value;

[0013] S102: Call the syntactic structure index sequence value, extract the semantic label and semantic classification number corresponding to each word in the prompt word, and perform tensor combination and concatenation according to the semantic label and syntactic mapping index to obtain the semantic combination tensor;

[0014] S103: Based on the semantic combination tensor, according to the vector density and the frequency of repetition of semantic tags, set the semantic relation density splitting benchmark value, and expand the combination tensor into multiple structural vectors according to the semantic distribution range to generate a semantic structural vector group.

[0015] Optionally, S2 calls the semantic structure vector group, analyzes the semantic level value of each vector and divides it into multiple subsets according to the level interval, inputs the information of multiple subsets into the image generation channel respectively, performs region localization in the image space according to the vector offset angle and activation weight, constructs a saliency feature distribution map, and forms an image generation matrix, including:

[0016] S201: Call the semantic structure vector group, extract the semantic level label and corresponding structure index value of each vector, analyze the semantic level of each vector, divide it into multiple subsets according to the level interval, and generate semantic level division cluster value;

[0017] S202: Divide the cluster values ​​according to the semantic level, extract the vector order and structure index number corresponding to each subset, and input them into the image generation channel. Based on the offset angle value and activation weight parameter of each vector, set the image region positioning index and obtain the image region positioning index value.

[0018] S203: Call the image region positioning index value, calculate the activation response intensity value of each region according to the distribution position of each set of location data in the image space, and partition and label the image space according to the activation response intensity value to construct the image saliency distribution coefficient matrix and generate the image generation matrix.

[0019] Optionally, S3 calls the image generation matrix to extract image pixel density, edge gradient, and region mask data. It calculates entity mapping relationships through the intersection of pixels and semantic annotations, splits and reassembles corresponding sentence fragments to generate semantic vectors, and compares these with the image embedding vectors to calculate a difference sequence, thus obtaining the semantic reconstruction offset, including:

[0020] S301: Call the image generation matrix to extract the pixel density value, edge gradient value and region mask identifier of the image, locate the semantic annotation region according to the mask identifier, compare the pixel coordinates and edge positions of the annotation region, obtain the coincident coordinate points of the image region and the semantic annotation, and generate entity mapping position parameters.

[0021] S302: Call the entity mapping position parameters to locate the corresponding sentence fragment in the original prompt word, extract the sentence unit according to the word order number, convert each sentence unit into a basic semantic vector, and construct a structured semantic sequence according to the combination to generate a semantic structured vector sequence;

[0022] S303: Call the semantic structured vector sequence, combine it with the image embedding vector, compare the semantic vectors, calculate and extract the difference sequence, and generate the semantic reconstruction offset.

[0023] Optionally, S4 invokes the semantic reconstruction offset, locates abnormal segments in the prompt words based on the offset range, extracts the position, connection nodes, and span data of the sentence structure path, calculates the path density, and generates a risk weight value based on the proportion of instruction phrases, including:

[0024] S401: Call the semantic reconstruction offset, extract the abnormal semantic vector according to the preset semantic offset critical threshold, and extract the text fragment content corresponding to the original prompt word according to the index value corresponding to the abnormal vector to generate the abnormal fragment positioning range.

[0025] S402: Based on the abnormal fragment location interval, extract the structural path of the statement within the interval, call the position number of each statement node in the structural path and the connection relationship between the nodes, combine the node connection relationship and position data, calculate the node span value of the statement structure, and generate the structural path span value.

[0026] S403: Call the structure path span value, obtain the path density coefficient based on the number of node connections and path span of each path, and generate a risk weight value by combining the proportion of instruction-type keywords in each path.

[0027] Optionally, the specific formula for calculating the node span value of the statement structure is as follows (1):

[0028] (1)

[0029] in, Represents the numerical value of the structural path span. This represents the position number of the end node of the q-th connection relationship. This represents the starting node position number of the q-th connection. The syntax connection weight represents the q-th connection. This represents the total number of connections in the structural path. Representing the Structural position offset of each node Represents the total number of nodes within the path. Represents the first in the path The node number, where q represents the number of the qth connection relationship within the path.

[0030] Optionally, the method further includes:

[0031] S5: Using the risk weight value, identify the mapping node corresponding to the risk path, detect semantically distorted paths and mark them as abnormal by the fluctuation difference between semantic vectors and residual information, stop the model output and send a prompt message, and generate path intervention parameters.

[0032] The path intervention parameters include path termination markers, abnormal node indexes, and response prompts.

[0033] Optionally, S5 uses the risk weight value to identify the mapping node corresponding to the risk path, detects semantically distorted paths and marks them as anomalies through the fluctuation difference between semantic vectors and residual information, stops model output and sends a prompt message, and generates path intervention parameters, including:

[0034] S501: Call the risk weight value, identify the risk path according to the preset path risk threshold, extract the node number and structural position of the path, establish the node mapping relationship, and generate the risk path mapping node sequence.

[0035] S502: Based on the risk path mapping node sequence, extract the semantic vector data and corresponding residual information of each node, detect semantically distorted paths by calculating the semantic offset anomaly index of each node, and generate semantically distorted path markers;

[0036] S503: Invoke the semantically distorted path marker, establish a path anomaly identifier and stop the model output, send the path anomaly response information, and generate path intervention parameters.

[0037] Optionally, the specific formula for calculating the semantic offset anomaly index of each node is as follows (2):

[0038] (2)

[0039] in, The semantic offset anomaly index represents the a-th node. This represents the semantic vector strength value of the a-th node. This represents the residual information strength value of the a-th node. This represents the deviation between the semantic vector of the b-th node and the residual. This represents the average value of the semantic vector strength of all nodes within the risk path mapping node sequence. represents the total number of nodes in the risk path mapping node sequence, b represents the number of the b-th node in the risk path mapping node sequence participating in the calculation, and a represents the number of the single node being processed when calculating the semantic offset anomaly index.

[0040] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0041] By leveraging the semantic density decomposition of multimodal tensors and image spatial mapping, cross-modal correlation analysis capabilities are enhanced, accurately identifying mixed image and text attack paths. By combining the entity mapping relationship between pixel gradients and region masks, bidirectional verification of semantic consistency between images and text is achieved, eliminating semantic gap vulnerabilities in single-modal detection. A path density and dynamic weight fusion mechanism of instruction words is adopted to quantify the abnormal risk level of sentence structure, breaking through the limitations of static rules in adapting to semantic variations and reducing the probability of outputting illegal content. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of a large-scale defense method based on multi-view image-text conversion provided by an embodiment of the present invention;

[0044] Figure 2 This is a block diagram of a large-scale defense device based on multi-view image-text conversion provided in an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of the structure of a large-scale defense device based on multi-view image-text conversion provided in an embodiment of the present invention. Detailed Implementation

[0046] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0047] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0048] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0049] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0050] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0051] Please see Figure 1 This invention provides a technical solution, a large-scale defense method based on multi-view image-text conversion, comprising the following steps:

[0052] S1: Obtain the text of the image-text dialogue prompts, extract the syntactic structure parameters and semantic parameters of the statements, and concatenate them into a multimodal tensor. Based on the semantic relation density, expand the concatenated multimodal tensor into multiple structural vectors to form a semantic structure vector group.

[0053] Optionally, the semantic structure vector group includes semantic level labels, location index parameters, and structural dependency markers; the image generation matrix specifically includes a region feature density map, channel index identifiers, and generated coordinate labels; the semantic reconstruction offset includes semantic segment mapping difference, sentence structure offset segment, and vector residual index; the risk weight value specifically includes path density factor, induced component proportion coefficient, and structural span ratio; and the path intervention parameters include path termination markers, abnormal node indexes, and response prompt content.

[0054] Optionally, the specific operation steps of S1 include S101-S103:

[0055] S101: Obtain the text of the image-text dialogue prompts, extract the part-of-speech type, position number, and grammatical dependency relationship for each word, establish a word dependency mapping index sequence based on the part-of-speech tag and word order position, and generate a syntactic structure index sequence value.

[0056] In one feasible implementation, after obtaining the text of the image-text dialogue prompts, the part-of-speech type, position number in the text, and corresponding grammatical dependency relationship of each word in the text are identified and extracted one by one. First, each word in the prompt text is classified according to language rules, and assigned part-of-speech numbers from 1 to 5 according to five categories: noun, verb, adjective, adverb, and preposition. The example text "Generate a landscape picture of blue sky and white clouds" is processed in sequence. "Generate" is identified as a verb, with part-of-speech number 2; "a" is a quantifier phrase and can be classified as an adverb, number 4; "blue sky and white clouds" as a whole is treated as a modifying modifier and classified as an adjective, number 3. "Landscape picture" is a noun, numbered 1. After completing the part-of-speech numbering, each word is assigned a position number in the text order, numbered consecutively from 1 to n, where "generate" is numbered 1, "a picture" is numbered 2, "blue sky and white clouds" is numbered 3, and "landscape picture" is numbered 4. Subsequently, based on syntactic dependency relations, the subject-predicate, verb-object, and modifier-type dependency connections between words are analyzed in sequence, clarifying that "generate" is the core predicate, governing "landscape picture", "blue sky and white clouds" modifies "landscape picture", and "a picture" modifies "landscape picture" in quantity. Based on the extracted part-of-speech number and text position number, the vocabulary index value of each word is calculated, specifically using the following calculation formula (0-1):

[0057] (0-1)

[0058] in, Let be the index value of the i-th word. Number the parts of speech (noun 1, verb 2, adjective 3, adverb 4, preposition 5). Number the words sequentially in the text.

[0059] Set the part-of-speech tag number to 2 and the position number to 1, and calculate as follows:

[0060] ;

[0061] The part-of-speech ID for "landscape image" is 1, and its position ID is 4. The calculation is as follows:

[0062] ;

[0063] The remaining words are calculated sequentially, and the index values ​​of all words are extracted step by step. The words are arranged in order and finally summarized to form the syntactic structure index sequence value.

[0064] S102: Call the syntactic structure index sequence value, extract the semantic label and semantic classification number corresponding to each word in the prompt word, and perform tensor combination and concatenation according to the semantic label and grammatical mapping index to obtain the semantic combination tensor.

[0065] In one feasible implementation, after calling the syntactic structure index sequence value, the semantic label and corresponding semantic category number of each word in the prompt are extracted one by one. Based on four categories—static objects, dynamic behaviors, spatial locations, and emotional connotations—values ​​from 1 to 4 are assigned respectively. Specifically, in the example text, "generate" corresponds to the dynamic behavior category and is numbered 2; "landscape picture" belongs to the static object category and is numbered 1; "blue sky and white clouds" is an adjective and is also classified as a static object and is numbered 1; "a picture" is a quantity modifier and does not involve a specific semantic category, so it is numbered 3 by default. After extraction, for each word, a tensor concatenation operation is performed based on the previously obtained syntactic structure index sequence value and the currently extracted semantic category number. The index value is multiplied by 10 and then the semantic category number is added according to the set rules to ensure that the index and semantics are combined. The calculation formula is as follows (0-2):

[0066] (0-2)

[0067] in, Let i be the semantic combination value of the i-th word. This is the index value for the word. Assign a semantic category number to the word (static object 1, dynamic behavior 2, spatial location 3, emotional connotation 4).

[0068] Set the "Generate" index value to 2001 and the semantic number to 2, then substituting them into the formula yields:

[0069] ;

[0070] Set the index value of "landscape image" to 1004 and the semantic number to 1, then substituting into the formula, we get:

[0071] ;

[0072] Setting the index of "blue sky and white clouds" to 3003 and the semantic number to 1, we get:

[0073] ;

[0074] Setting the index of "'one sheet'" to 4002 and the semantic number to 3, we get:

[0075] ;

[0076] By doing so, after all the words are combined, a complete semantic combination tensor is formed, providing a data foundation for subsequent analysis.

[0077] S103: Based on the semantic combination tensor, and according to the vector density and the frequency of repetition of semantic labels, set the semantic relation density splitting benchmark value, and expand the combination tensor into multiple structural vectors according to the semantic distribution range to generate a semantic structural vector group.

[0078] In one feasible implementation, based on the generated semantic combination tensor, each combination value is traversed sequentially to evaluate the semantic density in the vector. A window sliding method is used, setting each evaluation to five consecutive words, and counting the occurrences of the same semantic category number to determine the semantic aggregation features of the current segment. The semantic density calculation formula is as follows (0-3):

[0079] (0-3)

[0080] Where D represents semantic density. This represents the cumulative number of occurrences of the same semantic category ID within the window. The sliding window size is fixed at 5.

[0081] Within the specified window "20041, 15031, 20041, 15031, 20041", assuming semantic number 1 (static object) appears 4 times, substitute it into the formula:

[0082] ;

[0083] If the calculated density D is greater than or equal to the preset baseline value of 0.6, the window is determined to be a high-density area. The baseline value is derived from the static analysis. If the concentration of semantic categories within the window exceeds 60%, semantic focus is likely to form, which meets the standard for splitting into independent structural vector segments. After completing one round of window calculation, the high-density block is split separately, its start and end positions are marked, and its semantic category is recorded. The entire sentence is processed in sequence, and finally, the semantic structure vector groups are summarized and generated.

[0084] S2: Call the semantic structure vector group, analyze the semantic level value of each vector and divide it into multiple subsets according to the level interval. Input the information of multiple subsets into the image generation channel respectively. Based on the vector offset angle and activation weight, perform region localization in the image space, construct a salient feature distribution map, and form an image generation matrix.

[0085] Optionally, the specific operation steps of S2 include S201-S203:

[0086] S201: Call the semantic structure vector group, extract the semantic level label and corresponding structure index value of each vector, analyze the semantic level of each vector, divide it into multiple subsets according to the level interval, and generate semantic level division cluster value.

[0087] In one feasible implementation, after calling the semantic structure vector group, the semantic level label and structure index value corresponding to each vector are extracted one by one. The semantic level label is directly extracted from the semantic category number corresponding to the vector in the previously generated semantic structure vector group. The structure index value is the position index number of the semantic vector in the original prompt text. Then, the semantic level value of each vector is analyzed, and the semantic level value is set to a level range of 1 to 10, where the value of 1 represents the lowest semantic strength and the value of 10 represents the highest semantic strength. Taking the image-text dialogue prompt "Generate a landscape picture of blue sky and white clouds" as an example, assuming that the semantic category of "generate" is dynamic behavior, the level value is 8, "landscape picture" is a static object, the level value is 7, and "blue sky and white clouds" is an adjective, the level value is 6. After obtaining these level values, the level values ​​of all vectors are further calculated using the following formula (0-4):

[0088] (0-4)

[0089] in, The average rank value for all vector semantic rank values. For the first The semantic grade value of each vector, where n represents the total number of vectors. Assuming there are 4 vectors in the current vector group with grade values ​​of 8, 6, 7, and 5, the average grade is calculated as follows:

[0090] ;

[0091] Based on the calculated average grade value, a baseline value for dividing grade intervals is set. Specifically, a semantic grade greater than or equal to the average grade value of 6.5 is determined to be a high-grade interval, and a grade lower than 6.5 is determined to be a low-grade interval. According to this rule, "generate" with a semantic grade of 8 is divided into a high-grade subset, while "blue sky and white clouds" with a semantic grade of 6 and "a sheet" with a semantic grade of 5 are divided into a low-grade subset. Finally, the subsets to which all vectors belong are statistically obtained, and a semantic grade division cluster value is generated.

[0092] S202: Divide the cluster values ​​according to the semantic level, extract the vector order and structure index number corresponding to each subset, and input them into the image generation channel. Based on the offset angle value and activation weight parameter of each vector, set the image region positioning index and obtain the image region positioning index value.

[0093] In one feasible implementation, cluster values ​​are divided according to semantic level. The order information and structural index number of the vectors contained in each subset are extracted one by one. The vector order information is the order in which the vectors in the subset are arranged in the original prompt word text. The structural index number is the word position index value obtained in the previous step. It is set in the aforementioned image-text dialogue prompt word setting. Assuming that the high-level subset includes "generate" position index 2001 and "landscape" position index 1004, after extracting these two indices, they are sorted in ascending order of position in the original text, with "generate" 2001 first and "landscape" 1004 second, resulting in the sequence [1004, 2001]. This sequence is then input into the image generation channel. For the semantic vector corresponding to each structural index, the vector offset angle value and activation weight parameter are called. The offset angle value represents the spatial position angle of the vector relative to the center of the image space, and the activation weight parameter represents the weighted value of semantic intensity. The specific image region positioning index is calculated using the following formula (0-5):

[0094] (0-5)

[0095] in, Locate the index value for the image region. This represents the offset angle value of the vector. Let 'Generate' be the activation weight parameter for the vector. Taking "Generate" as an example, assuming the vector's offset angle is 45 degrees and the activation weight parameter is 1.2, the calculation is as follows:

[0096] ;

[0097] Similarly, assuming the "landscape image" offset angle is 30 degrees and the activation weight parameter is 1.0, the calculation is as follows:

[0098] ;

[0099] After calculating the localization index value of each vector in turn, the localization index value of the image region is obtained.

[0100] S203: Call the image region localization index value, calculate the activation response intensity value of each region according to the distribution of each set of location data in the image space, and partition and label the image space according to the activation response intensity value, construct the image saliency distribution coefficient matrix, and generate the image generation matrix.

[0101] In one feasible implementation, after calling the image region positioning index value, the image spatial position data corresponding to each set of indices is extracted. According to the image spatial coordinate system, the position corresponding to the index value is mapped to the two-dimensional image space, where the image center coordinates are set to the origin (0,0), and each index value represents the coordinate angle of the corresponding image region, with the radius R fixed as the radius of the image region range (assumed to be 100 pixels). The index value is converted into image spatial coordinates using the following formula (0-6):

[0102] (0-6)

[0103] Where (x,y) represents the image space coordinates, and R represents a fixed radius. Using the previously calculated positioning index angle value, set the positioning index value for the "generated" area to 54, and substitute it into the formula to calculate the coordinates:

[0104] ;

[0105] Subsequently, the activation response intensity value is calculated for each region. The intensity value depends on the distance between the index angle and the image center. The closer the distance, the higher the activation intensity, and vice versa. Assuming that the activation response intensity value is set with a linear proportional relationship, the specific calculation is as follows (0-7):

[0106] (0-7)

[0107] in, To activate the response intensity value, d represents the Euclidean distance between the region coordinates and the image center (0,0). For example, the Euclidean distance of the "generated" region coordinates (58.78, 80.90) is calculated as follows:

[0108] ;

[0109] Substitute into the formula:

[0110] ;

[0111] The response intensity value of each region is calculated step by step using this method. Regions with an intensity greater than or equal to 0.6 are classified as high response regions, and regions with an intensity less than 0.6 are classified as low response regions. After completing the partitioning and labeling, the image saliency distribution coefficient matrix is ​​finally constructed to generate the image generation matrix.

[0112] S3: Call the image generation matrix to extract image pixel density, edge gradient and region mask data, calculate entity mapping relationship through the intersection of pixels and semantic annotations, split and reassemble corresponding sentence fragments to generate semantic vectors, compare with image embedding vectors to calculate difference sequence, and obtain semantic reconstruction offset.

[0113] Optionally, the specific operating steps of S3 include S301-S303:

[0114] S301: Call the image generation matrix to extract the pixel density value, edge gradient value, and region mask identifier of the image. Locate the semantic annotation region based on the mask identifier. Compare the pixel coordinates and edge positions of the annotation region to obtain the overlapping coordinate points of the image region and the semantic annotation, and generate entity mapping position parameters.

[0115] In one feasible implementation, after calling the image generation matrix, the pixel density value, edge gradient value, and region mask identifier of each region in the matrix are first extracted. The pixel density value refers to the number of effective pixels in a unit image area, the edge gradient value is the grayscale change of pixels in the area, and the region mask identifier represents the existence state of each region in the semantic annotation map. Then, the corresponding semantic annotation region is extracted according to the region mask identifier, and the pixel coordinates and edge position coordinates of each region are extracted. For all extracted coordinate data, a point-by-point comparison operation is performed. During the comparison, the Euclidean distance threshold method is used to filter overlapping points. The distance threshold is set to 3 pixels. The judgment condition is that when the distance between the pixel coordinates in the image area and the edge coordinates of the semantic annotation region is less than or equal to 3 pixels, it is judged as a valid overlapping point. The distance calculation formula is as follows (0-8):

[0116] (0-8)

[0117] Where d is the distance between the two coordinate points. These are the pixel coordinates of the image region. To semantically annotate the edge point coordinates, using example values, the pixel coordinates are (120, 80), and the annotated edge point is (122, 82). The calculation is as follows:

[0118] ;

[0119] If the calculation result is less than 3, it is determined to be a valid overlapping point. All pixels and labeled edge points are processed in sequence, all overlapping coordinate points that meet the conditions are counted, and their region index and specific coordinates are extracted to finally form complete entity mapping position parameters.

[0120] S302: Call the entity mapping position parameter to locate the corresponding sentence fragment in the original prompt word, extract the sentence unit according to the word order number, convert each sentence unit into a basic semantic vector, and construct a structured semantic sequence based on the combination to generate a semantic structured vector sequence.

[0121] In one feasible implementation, after calling the entity mapping location parameters, the region indices associated with all overlapping coordinate points are extracted and located to the corresponding sentence segments in the original prompt. Sentence segment identification is accomplished through an index mapping table. The region indices in the entity mapping location parameters are matched one by one with the word order numbers in the original prompt to extract each corresponding sentence unit. After extraction, the word order numbers are arranged sequentially to form a continuous sequence of sentence units. The region indices corresponding to the entity mapping are set to 5, 8, and 10, corresponding to the 1st, 2nd, and 4th sentence units in the original prompt, respectively. These sentence units are then processed sequentially... Extract the words "generate", "a", and "landscape picture". For each sentence unit, convert it into a basic semantic vector based on part of speech and dependency relationship. The vector dimension is set to three dimensions, representing action intensity, object entity weight, and spatial attribute weight, respectively, with a value range of 0-10. "Generate" is set as a verb with an action intensity of 9, an object entity weight of 0, and a spatial attribute weight of 0. "Landscape picture" is set as a noun with an action intensity of 0, an object entity weight of 8, and a spatial attribute weight of 2. Then, all the basic semantic vectors are concatenated in sequence to construct a structured semantic sequence, and finally a semantic structured vector sequence is obtained.

[0122] S303: Call the semantic structured vector sequence, combine it with the image embedding vector, compare the semantic vectors, calculate and extract the difference sequence, and generate the semantic reconstruction offset.

[0123] In one feasible implementation, after calling the semantic structured vector sequence, each semantic vector is extracted sequentially and compared with the image embedding vector. The image embedding vector has the same dimension as the semantic vector and includes three terms: action intensity, object entity weight, and spatial attribute weight. The difference between the two is calculated for each term using the absolute difference method. The difference calculation formula is as follows (0-9):

[0124] (0-9)

[0125] in, The difference between the corresponding items. The numerical value of the corresponding item in the semantic vector. Given the numerical values ​​of the corresponding terms in the image embedding vector, and setting the "generated" semantic vector to [9,0,0] and the image embedding vector to [7,1,0], the difference in the first dimension is:

[0126] ;

[0127] The second dimension is:

[0128] ;

[0129] The third dimension is:

[0130] ;

[0131] The differences between all semantic vectors and their corresponding image embedding vectors are calculated sequentially, and the complete difference sequence is obtained by summarizing them. Finally, the semantic reconstruction offset is generated.

[0132] S4: Call the semantic reconstruction offset, locate the abnormal fragment in the prompt word according to the offset range, extract the position, connection node and span data of the statement structure path, calculate the path density, and generate a risk weight value by combining the proportion of instruction phrases.

[0133] Optionally, the specific operation steps of S4 include S401-S403:

[0134] S401: Call the semantic reconstruction offset, extract the abnormal semantic vector according to the preset semantic offset critical threshold, and extract the text fragment content corresponding to the original prompt word according to the index value of the abnormal vector to generate the abnormal fragment location range.

[0135] In one feasible implementation, after calling the semantic reconstruction offset, the difference values ​​of each dimension of each semantic vector are extracted one by one. The difference values ​​are then compared with a preset semantic offset threshold. The threshold is set by calculating the average value of the entire difference sequence and adding a deviation coefficient, expressed by the following formula (0-10):

[0136] (0-10)

[0137] in, This represents the semantic offset threshold. Represents the first in the difference sequence A number, This represents the total number of differences in the difference sequence, where k is the deviation coefficient, set to 20% of the average difference to ensure threshold stability. For example, if the difference sequence values ​​are [1.2, 1.5, 2.3, 2.0], the average difference is calculated as follows:

[0138] Then the deviation coefficient Substituting into the formula, the threshold is:

[0139] ;

[0140] Then, each vector difference is compared with the threshold to determine whether the difference exceeds the threshold of 2.10. If the difference exceeds the threshold, the corresponding semantic vector is determined to be an abnormal semantic vector. If the difference is set to 2.3, the semantic vector that significantly exceeds the threshold is determined to be an abnormal vector. The index value corresponding to the abnormal vector is extracted. Assuming that the index of the abnormal semantic vector is "2001", the text fragment content corresponding to the index in the original prompt word is called. If the text corresponding to the index "2001" is "generate", the position of this text in the original prompt word is located, and its start number and end number position are determined to obtain the complete abnormal fragment location range.

[0141] S402: Based on the abnormal fragment location interval, extract the structural path of the statement within the interval, call the position number of each statement node in the structural path and the connection relationship between the nodes, combine the node connection relationship and position data, calculate the node span value of the statement structure, and generate the structural path span value.

[0142] In one feasible implementation, the specific formula for calculating the node span value of the statement structure is as follows (1):

[0143] (1)

[0144] in, Represents the numerical value of the structural path span. This represents the position number of the end node of the q-th connection relationship. This represents the starting node position number of the q-th connection. The syntax connection weight represents the q-th connection. This represents the total number of connections in the structural path. Representing the Structural position offset of each node Represents the total number of nodes within the path. Represents the first in the path The node number, where q represents the number of the qth connection relationship within the path.

[0145] Detailed explanation and derivation of formula (1):

[0146] The formula is used to calculate the span value of the structural path. The result is used to quantify the span characteristics of the sentence structural path in the abnormal segment, providing a basis for subsequent density and risk weight calculation.

[0147] Parameter meanings and settings:

[0148] The position number of the end node of the qth connection is set to 14, 19, 25;

[0149] The starting node position number of the qth connection is set to 8, 14, or 19;

[0150] Set the syntactic connection weights for the q-th connection relationship as follows: subject-verb relationship weight is set to 2.0, verb-object relationship weight is set to 1.8, and attributive-head relationship weight is set to 1.5.

[0151] The total number of connections in the structural path is set to 3;

[0152] For the first The structural position offset of each node is set to 8, 14, 19, and 25, with offsets of 0, 6, 11, and 17 respectively.

[0153] Set the total number of nodes within the path to 4;

[0154] Substitute the parameters into the formula to calculate:

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] ;

[0160] ;

[0161] ;

[0162] ;

[0163] Result 8.12 indicates that the structural path span is relatively high, which means that the internal node connection relationship of the abnormal segment has a large span and the structural distribution is discrete. As an important basic parameter for subsequent risk density calculation, the result reflects the overall expansiveness and complexity of the current structural path.

[0164] S403: Call the structural path span value, obtain the path density coefficient based on the number of node connections and path span of each path, and generate a risk weight value by combining the proportion of instruction-type keywords in each path.

[0165] In one feasible implementation, after calling the structural path span value, the number of node connections and the node span value in each path are extracted. The number of node connections is extracted, which is the total number of node connection relationships contained in each structural path. Then, the density coefficient of each path is calculated. The density coefficient is the ratio between the number of node connections and the path span value, and the formula is expressed as follows (1-1):

[0166] (1-1)

[0167] in, This is the path density coefficient. The number of path node connections. This represents the node span value. Continuing with the example above, the number of node connections is 3 (node ​​1...). 4. Node 3 4. Node 2 4) The total node span value is (3+1+2)=6, and the density coefficient is calculated as follows:

[0168] ;

[0169] Then, the number of instruction-type keywords contained in each path is counted. Instruction-type keywords specifically include words with clear instructions, such as "generate," "make," and "display." Assuming that the current path contains 1 instruction-type keyword ("generate") and the total number of nodes in the path is 4, the proportion of instruction-type keywords is calculated as follows (1-2):

[0170] (1-2)

[0171] in, The proportion of instruction-type keywords, The number of instruction-type keywords in the path. Substitute the values ​​to represent the total number of nodes in the path for the calculation:

[0172] ;

[0173] Finally, the path density coefficient and the proportion of instruction-type keywords are used to calculate the risk weight value one by one. The risk weight value is obtained by weighting the density coefficient and the proportion of instruction values, as shown in the following formula (1-3):

[0174] (1-3)

[0175] in, This is the risk weight value. This is the path density coefficient. The weighting parameter represents the proportion of keywords in the instruction. and These represent the path density weight and instruction proportion weight, respectively, and are set to 0.6 and 0.4 by default. Substitute the values:

[0176] ;

[0177] The risk weights for all paths are calculated gradually, and the final risk weight values ​​are generated.

[0178] S5: Using risk weight values, identify the mapping nodes corresponding to risk paths. By using the fluctuation difference between semantic vectors and residual information, detect semantically distorted paths and mark them as anomalies. Stop the model output and send a prompt message to generate path intervention parameters.

[0179] Optionally, the specific operating steps of S5 include S501-S503:

[0180] S501: Call the risk weight value, identify the risk path according to the preset path risk threshold, extract the node number and structural position of the path, establish the node mapping relationship, and generate the risk path mapping node sequence.

[0181] In one feasible implementation, after calling the risk weight value, the risk weight value of each path is first extracted and compared with the preset path risk threshold. The path risk threshold is set as the average risk weight of each path plus the volatility adjustment coefficient, and the calculation formula is as follows (1-4):

[0182] (1-4)

[0183] in, The path risk threshold Let the risk weight value be that of path a. The total number of paths, The volatility adjustment factor is set to 15% of the mean. Assuming three paths exist with risk weights of 0.4, 0.6, and 0.7 respectively, the average value is calculated as follows: Then the fluctuation adjustment coefficient The final threshold is calculated as follows:

[0184] ;

[0185] Subsequently, the risk weight and threshold of each path are compared one by one. Paths with risk weights greater than the threshold are identified as risk paths. For example, the third path has a weight of 0.7, which is higher than the threshold of 0.6517, so it is determined to be a risk path. The node number and structural position of the path are extracted. The node number is the unique number of all participating nodes in the path, and the structural position is the specific number position of the node in the original text of the prompt word. The node number is set as [5,8,10], which corresponds to the three places in the original text: "generated", "blue sky and white clouds", and "scenery". The correspondence between the nodes and the text is established in sequence to form a node mapping relationship table. Finally, the risk path mapping node sequence is formed.

[0186] S502: Based on the risk path mapping node sequence, extract the semantic vector data and corresponding residual information of each node, calculate the semantic offset anomaly index of each node, detect semantically distorted paths, and generate semantically distorted path markers.

[0187] In one feasible implementation, the specific formula for calculating the semantic offset anomaly index of each is as follows (2):

[0188] (2)

[0189] Calculate the semantic offset anomaly index;

[0190] in, The semantic offset anomaly index represents the a-th node. This represents the semantic vector strength value of the a-th node. This represents the residual information strength value of the a-th node. This represents the deviation between the semantic vector of the b-th node and the residual. This represents the average value of the semantic vector strength of all nodes within the risk path mapping node sequence. represents the total number of nodes in the risk path mapping node sequence, b represents the number of the b-th node in the risk path mapping node sequence participating in the calculation, and a represents the number of the single node being processed when calculating the semantic offset anomaly index.

[0191] Detailed explanation of formula (2) and derivation of formula calculation:

[0192] The formula is used to calculate the semantic offset anomaly index, and the result is used to detect semantically distorted nodes in risky paths;

[0193] Parameter meanings and settings:

[0194] Let the semantic vector strength value of the a-th node be set. This reflects the overall semantic strength of the node;

[0195] The residual information strength value of the a-th node is set to 2.7, which reflects the semantic deviation during the generation process;

[0196] The deviation between the semantic vector and the residual of the b-th node in the risk path mapping node sequence is set as follows: node b1 is 1.8, node b2 is 2.5, node b3 is 1.2, node b4 is 3.1, and node b5 is 2.0.

[0197] The total number of nodes in the risk path mapping node sequence is set to 5;

[0198] The average semantic vector strength of all nodes within the risk path mapping node sequence is set to 5.2, 4.8, 6.1, 5.7, and 4.9 respectively. ;

[0199] Substitute the parameters into the formula to calculate:

[0200] ;

[0201] ;

[0202] ;

[0203] ;

[0204] The result of 2.05 indicates that the semantic offset anomaly index has exceeded the safety threshold of 1.5, which means that the semantic offset of node a is serious. The numerical result directly drives the confirmation of the distorted path and the subsequent output termination strategy.

[0205] S503: Invoke semantic distortion path markers, establish path anomaly identifiers and stop model output, send path anomaly response information, and generate path intervention parameters.

[0206] In one feasible implementation, after calling the semantically distorted path marker, the exception handling process is executed sequentially for all marked paths. First, a path exception identifier is established for the detected distorted path. The exception identifier format is the path number plus the "abnormal" label. For example, if the path is path 3, it is marked as "Path3 abnormal". Then, all nodes in the path are located, the subsequent large model output is forcibly stopped, and the current path is removed from the generation process. Response information is generated for the abnormal path. The response information includes the path number, the nodes involved, and the reason description text. For example, "Path 3, involving nodes generation, blue sky and white clouds, has serious semantic offset and is judged to be abnormal". At the same time, the path intervention parameter is calculated. The path intervention parameter is generated based on the number of nodes and the average abnormal index. The calculation formula is as follows (3):

[0207] (3)

[0208] in, For path intervention parameters, This represents the total number of nodes in the abnormal path. The average anomaly index for each node is set to 3, with anomaly indices of 0.667, 1, and 0.4 respectively.

[0209] ;

[0210] The final intervention parameters are:

[0211] ;

[0212] The path intervention parameter values ​​are recorded in the system log, the entire processing flow is completed, and the path intervention parameters are generated.

[0213] In this embodiment of the invention, the semantic density decomposition of multimodal tensors and image spatial mapping are used to enhance cross-modal correlation analysis capabilities, accurately identify mixed image and text attack paths, and combine the entity mapping relationship between pixel gradients and region masks to achieve bidirectional verification of semantic consistency between images and text, eliminating semantic gap vulnerabilities in single-modal detection. A path density and instruction word dynamic weight fusion mechanism is adopted to quantify the abnormal risk level of sentence structure, overcome the limitations of static rules in adapting to semantic variations, and reduce the probability of outputting illegal content.

[0214] Figure 2 This is a block diagram of a large-scale defense device based on multi-view image-text conversion provided in an embodiment of the present invention. This device is used in a large-scale defense method based on multi-view image-text conversion. (Refer to...) Figure 2 The device includes an extraction and splicing unit 210, an analysis and positioning unit 220, a splitting and comparison unit 230, a generation unit 240, and a detection unit 250. Wherein:

[0215] Extraction and splicing unit 210 is used to obtain the text of the image and text dialogue prompt words, extract the syntactic structure parameters and semantic parameters of the sentences, and splice them into a multimodal tensor. According to the semantic relation density, the spliced ​​multimodal tensor is expanded into multiple structural vectors to form a semantic structure vector group.

[0216] The analysis and positioning unit 220 is used to call the semantic structure vector group, analyze the semantic level value of each vector and divide it into multiple subsets according to the level interval, input the information of multiple subsets into the image generation channel respectively, perform region positioning in the image space according to the vector offset angle and activation weight, construct a salient feature distribution map, and form an image generation matrix.

[0217] The splitting and comparison unit 230 is used to call the image generation matrix, extract image pixel density, edge gradient and region mask data, calculate entity mapping relationship through the intersection of pixels and semantic annotations, split and reassemble corresponding sentence fragments to generate semantic vectors, compare the image embedding vectors to calculate the difference sequence, and obtain the semantic reconstruction offset.

[0218] The generation unit 240 is used to call the semantic reconstruction offset, locate the abnormal fragment in the prompt word according to the offset range, extract the position, connection node and span data of the sentence structure path, calculate the path density, and generate a risk weight value by combining the proportion of instruction phrases.

[0219] The detection unit 250 is used to identify the mapping node corresponding to the risk path using the risk weight value, detect semantically distorted paths and mark them as anomalies by the fluctuation difference between semantic vectors and residual information, stop the model output and send prompt information, and generate path intervention parameters.

[0220] In this embodiment of the invention, the semantic density decomposition of multimodal tensors and image spatial mapping are used to enhance cross-modal correlation analysis capabilities, accurately identify mixed image and text attack paths, and combine the entity mapping relationship between pixel gradients and region masks to achieve bidirectional verification of semantic consistency between images and text, eliminating semantic gap vulnerabilities in single-modal detection. A path density and instruction word dynamic weight fusion mechanism is adopted to quantify the abnormal risk level of sentence structure, overcome the limitations of static rules in adapting to semantic variations, and reduce the probability of outputting illegal content.

[0221] Figure 3 This is a schematic diagram of the structure of a large-scale defense device based on multi-view image-text conversion provided in an embodiment of the present invention, such as... Figure 3 As shown, a large-scale defense device based on multi-view image-text conversion can include the above-mentioned... Figure 2 The large-scale defense device based on multi-view image-text conversion is shown. Optionally, the large-scale defense device 310 based on multi-view image-text conversion may include a first processor 2001.

[0222] Optionally, the large-scale defense device 310 based on multi-view image-text conversion may also include a memory 2002 and a transceiver 2003.

[0223] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0224] The following is combined with Figure 3 A detailed introduction to each component of the large-scale defense device 310 based on multi-view image-text conversion:

[0225] The first processor 2001 is the control center of the large-scale defense device 310 based on multi-view image-text conversion. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0226] Optionally, the first processor 2001 can perform various functions of the large model defense device 310 based on multi-view image-text conversion by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0227] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0228] In a specific implementation, as one example, the large-scale defense device 310 based on multi-view image-text conversion may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0229] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0230] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the large-model defense device 310 based on multi-view image-text conversion. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0231] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0232] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0233] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the large-scale defense device 310 based on multi-view image-text conversion. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0234] It should be noted that, Figure 3 The structure of the large model defense device 310 based on multi-view image-text conversion shown in the figure does not constitute a limitation on the router. The actual knowledge structure identification device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0235] Furthermore, the technical effects of the large-scale defense device 310 based on multi-view image-text conversion can be referred to the technical effects of the large-scale defense method based on multi-view image-text conversion described in the above method embodiments, and will not be repeated here.

[0236] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0237] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0238] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0239] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0240] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0241] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0242] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0243] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0244] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0245] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0246] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0247] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0248] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large-scale defense method based on multi-view image-text conversion, characterized in that, The method includes: S1: Obtain the text of the image-text dialogue prompts, extract the syntactic structure parameters and semantic parameters of the sentences, and concatenate them into a multimodal tensor. Based on the semantic relation density, expand the concatenated multimodal tensor into multiple structural vectors to form a semantic structure vector group. S2: Call the semantic structure vector group, analyze the semantic level value of each vector and divide it into multiple subsets according to the level interval, input the information of multiple subsets into the image generation channel respectively, perform region localization in the image space according to the vector offset angle and activation weight, construct a salient feature distribution map, and form an image generation matrix; S3: Call the image generation matrix to extract image pixel density, edge gradient and region mask data, calculate entity mapping relationship through the intersection of pixels and semantic annotations, split and reassemble corresponding sentence fragments to generate semantic vectors, compare with image embedding vectors to calculate difference sequence, and obtain semantic reconstruction offset; S4: Call the semantic reconstruction offset, locate the abnormal fragment in the prompt word according to the offset range, extract the position, connection node and span data of the sentence structure path, calculate the path density, and generate a risk weight value by combining the proportion of instruction phrases. S4 invokes the semantic reconstruction offset, locates abnormal segments in the prompt words based on the offset range, extracts the position, connection nodes, and span data of the sentence structure path, calculates the path density, and generates a risk weight value based on the proportion of instruction phrases, including: S401: Call the semantic reconstruction offset, extract the abnormal semantic vector according to the preset semantic offset critical threshold, and extract the text fragment content corresponding to the original prompt word according to the index value corresponding to the abnormal vector to generate the abnormal fragment positioning range. S402: Based on the abnormal fragment location interval, extract the structural path of the statement within the interval, call the position number of each statement node in the structural path and the connection relationship between the nodes, combine the node connection relationship and position data, calculate the node span value of the statement structure, and generate the structural path span value. S403: Call the structure path span value, obtain the path density coefficient based on the number of node connections and path span of each path, and generate a risk weight value by combining the proportion of instruction-type keywords in each path.

2. The large-scale defense method based on multi-view image-text conversion according to claim 1, characterized in that, The semantic structure vector group includes semantic level labels, location index parameters, and structural dependency markers. The image generation matrix specifically includes a region feature density map, channel index identifiers, and generation coordinate labels. The semantic reconstruction offset includes semantic segment mapping difference, sentence structure offset segment, and vector residual index. The risk weight value specifically includes path density factor, induced component proportion coefficient, and structural span ratio.

3. The large-scale defense method based on multi-view image-text conversion according to claim 1, characterized in that, S1 obtains the text of the image-text dialogue prompts, extracts the syntactic and semantic parameters of the sentences, and concatenates them into a multimodal tensor. Based on the semantic relation density, the concatenated multimodal tensor is expanded into multiple structural vectors to form a semantic structure vector group, including: S101: Obtain the text of the image-text dialogue prompts, extract the part-of-speech type, position number and grammatical dependency relationship of each word, establish a word dependency mapping index sequence based on the part-of-speech tag and word order position, and generate a syntactic structure index sequence value; S102: Call the syntactic structure index sequence value, extract the semantic label and semantic classification number corresponding to each word in the prompt word, and perform tensor combination and concatenation according to the semantic label and syntactic mapping index to obtain the semantic combination tensor; S103: Based on the semantic combination tensor, according to the vector density and the frequency of repetition of semantic tags, set the semantic relation density splitting benchmark value, and expand the combination tensor into multiple structural vectors according to the semantic distribution range to generate a semantic structural vector group.

4. The large-scale defense method based on multi-view image-text conversion according to claim 1, characterized in that, S2 calls the semantic structure vector group, analyzes the semantic level value of each vector and divides it into multiple subsets according to the level interval. The information of each subset is input into the image generation channel. Based on the vector offset angle and activation weights, region localization is performed in the image space, a salient feature distribution map is constructed, and an image generation matrix is ​​formed, including: S201: Call the semantic structure vector group, extract the semantic level label and corresponding structure index value of each vector, analyze the semantic level of each vector, divide it into multiple subsets according to the level interval, and generate semantic level division cluster value; S202: Divide the cluster values ​​according to the semantic level, extract the vector order and structure index number corresponding to each subset, and input them into the image generation channel. Based on the offset angle value and activation weight parameter of each vector, set the image region positioning index and obtain the image region positioning index value. S203: Call the image region positioning index value, calculate the activation response intensity value of each region according to the distribution position of each set of location data in the image space, and partition and label the image space according to the activation response intensity value to construct the image saliency distribution coefficient matrix and generate the image generation matrix.

5. The large-scale defense method based on multi-view image-text conversion according to claim 1, characterized in that, S3 calls the image generation matrix to extract image pixel density, edge gradient, and region mask data. It calculates entity mapping relationships through the intersection of pixels and semantic annotations, splits and reassembles corresponding sentence fragments to generate semantic vectors, and compares the image embedding vectors to calculate the difference sequence to obtain the semantic reconstruction offset, including: S301: Call the image generation matrix to extract the pixel density value, edge gradient value and region mask identifier of the image, locate the semantic annotation region according to the mask identifier, compare the pixel coordinates and edge positions of the annotation region, obtain the coincident coordinate points of the image region and the semantic annotation, and generate entity mapping position parameters. S302: Call the entity mapping position parameters to locate the corresponding sentence fragment in the original prompt word, extract the sentence unit according to the word order number, convert each sentence unit into a basic semantic vector, and construct a structured semantic sequence according to the combination to generate a semantic structured vector sequence; S303: Call the semantic structured vector sequence, combine it with the image embedding vector, compare the semantic vectors, calculate and extract the difference sequence, and generate the semantic reconstruction offset.

6. The large-scale defense method based on multi-view image-text conversion according to claim 1, characterized in that, The specific formula for calculating the node span value of the statement structure is as follows (1): （1） in, Represents the numerical value of the structural path span. This represents the position number of the end node of the q-th connection relationship. This represents the starting node position number of the q-th connection. The syntax connection weight represents the q-th connection. This represents the total number of connections in the structural path. Representing the Structural position offset of each node Represents the total number of nodes within the path. Represents the first in the path The node number, where q represents the number of the qth connection relationship within the path.

7. The large-scale defense method based on multi-view image-text conversion according to claim 1, characterized in that, The method further includes: S5: Using the risk weight value, identify the mapping node corresponding to the risk path, detect semantically distorted paths and mark them as abnormal by the fluctuation difference between semantic vectors and residual information, stop the model output and send a prompt message, and generate path intervention parameters. The path intervention parameters include path termination markers, abnormal node indexes, and response prompts.

8. The large-scale defense method based on multi-view image-text conversion according to claim 7, characterized in that, S5 utilizes the risk weight value to identify the mapping node corresponding to the risk path, detects semantically distorted paths and marks them as anomalies through the fluctuation difference between semantic vectors and residual information, stops model output and sends a prompt message, and generates path intervention parameters, including: S501: Call the risk weight value, identify the risk path according to the preset path risk threshold, extract the node number and structural position of the path, establish the node mapping relationship, and generate the risk path mapping node sequence. S502: Based on the risk path mapping node sequence, extract the semantic vector data and corresponding residual information of each node, detect semantically distorted paths by calculating the semantic offset anomaly index of each node, and generate semantically distorted path markers; S503: Invoke the semantically distorted path marker, establish a path anomaly identifier and stop the model output, send the path anomaly response information, and generate path intervention parameters.

9. The large-scale defense method based on multi-view image-text conversion according to claim 8, characterized in that, The specific formula for calculating the semantic offset anomaly index of each node is as follows (2): （2） in, The semantic offset anomaly index represents the a-th node. This represents the semantic vector strength value of the a-th node. This represents the residual information strength value of the a-th node. This represents the deviation between the semantic vector of the b-th node and the residual. This represents the average value of the semantic vector strength of all nodes within the risk path mapping node sequence. represents the total number of nodes in the risk path mapping node sequence, b represents the number of the b-th node in the risk path mapping node sequence participating in the calculation, and a represents the number of the single node being processed when calculating the semantic offset anomaly index.

Images