Cross-regional semantic security verification method, system and device based on multi-modal deep learning

Abstract

The invention relates to the technical field of artificial intelligence security, and relates to a cross-regional semantic security verification method, system and device based on multi-modal deep learning. The method comprises the following steps: carrying out analysis, formatting and feature enhancement on multi-modal data transmitted across security domains through a preprocessing engine, and generating standardized input; the preprocessed data are input into a structure verification unit, a semantic understanding unit and a risk simulation unit in parallel; the structure verification unit detects a known attack mode through a static rule matching engine and a dynamic learning engine and dynamically updates a rule base; the semantic understanding unit generates a multi-modal fusion risk assessment result; the risk simulation unit generates a risk thermodynamic diagram; and the verification decision-making device integrates output results of all the units, and intercepts or releases cross-domain transmission data. According to the method, comprehensive semantic security verification of cross-security domain transmission data is realized, and the confidentiality and integrity of sensitive information in a transmission process are ensured.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence security technology, and to a cross-regional semantic security verification method, system and device based on multimodal deep learning. Background Art

[0002] In large-scale model training scenarios, secure data transmission and protection are crucial. With the rapid development of AI technology, the amount of data required for large-scale model training has increased dramatically, and this data often contains highly sensitive information. To effectively manage and utilize this data, it is typically classified into different security zones, such as open, restricted, and isolated. During the training process, data exchange and transmission between different security zones becomes inevitable, but this also poses serious security risks, particularly the leakage of sensitive information.

[0003] Traditional solutions to address security issues during cross-regional data transmission rely primarily on single-modal analysis methods, which have significant limitations:

[0004] Single-modality limitations: Traditional security measures typically analyze data in a single modality, such as text or images. For example, regular expression matching is used to detect sensitive information in text data, while target detection technology is used to identify specific objects in image data. However, in large-model training scenarios, cross-modal data association (such as the combination of images and text) may introduce new security risks that cannot be effectively identified by single-modality analysis methods. For example, attackers may bypass single-modality detection mechanisms by combining hidden information embedded in images with text descriptions, thereby leaking sensitive information.

[0005] Static rule failure: Existing security measures often rely on predefined keyword libraries or rule sets to detect potential security threats. These static rules may be effective against known security attacks, but they are unable to protect against new or variant semantic attacks. As attack techniques continue to evolve, attackers are able to employ increasingly sophisticated and covert means to circumvent these static rules, rendering security measures ineffective. For example, by using techniques such as synonym replacement and context obfuscation, attackers can construct text or image data that appears harmless but actually contains sensitive information.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0007] The following presents a simplified summary of some aspects of the disclosed embodiments in order to provide a basic understanding of such embodiments. This summary is not an extensive overview of the embodiments and is not intended to identify key / critical elements or to delineate the scope of the embodiments. Its sole purpose is to present some aspects of the disclosed embodiments in a simplified form as a prelude to the more detailed description that is presented later.

[0008] The embodiments of the present disclosure provide a multi-modal deep learning based cross-region semantic security verification method, system and device, which combines structure verification technology, semantic understanding technology and risk simulation technology, aims to realize comprehensive semantic security verification of cross-security domain transmission data, and ensure the confidentiality and integrity of sensitive information in the transmission process.

[0009] In some embodiments, the method comprises:

[0010] The multi-modal data transmitted across the security domain is parsed, formatted and feature enhanced by the preprocessing engine to generate standardized input;

[0011] The preprocessed data is input in parallel to the structure verification unit, the semantic understanding unit and the risk simulation unit:

[0012] The structure verification unit detects known attack patterns through a static rule matching engine and a dynamic learning engine, and dynamically updates the rule library;

[0013] The semantic understanding unit processes text, image and structured data respectively through a triple analysis architecture to generate a multi-modal fusion risk assessment result;

[0014] The risk simulation unit simulates real threats through member inference attacks, data inversion attacks and backdoor detection to generate a risk heat map;

[0015] The verification decision maker synthesizes the output results of each unit to perform interception or release operations on the cross-domain transmission data.

[0016] Preferably, the preprocessing engine includes: performing character set unification and sensitive word labeling on text data; performing EXIF metadata stripping, resolution standardization and gamut correction on image data; performing pattern extraction and relationship graph construction on structured data.

[0017] Preferably, the dynamic learning engine of the structure verification unit uses a graph neural network for anomaly detection, including: constructing an attribute graph containing field name, data type and value range; identifying abnormal associations through GNN and generating new rules to supplement the static rule matching library.

[0018] Preferably, the triple analysis architecture of the semantic understanding unit includes: using a BERT quantization model to analyze text semantic risk; using a YOLO model to detect image sensitive content; using a TabNet model to analyze potential associations of structured data.

[0019] Preferably, the three-in-one attack simulation of the risk simulation unit includes:

[0020] Membership inference attack: inferring whether data belongs to a specific training set;

[0021] Data inversion attack: reconstructing original sensitive information;

[0022] Backdoor detection: Identify trigger patterns for malicious implants.

[0023] In some embodiments, the system comprises:

[0024] Preprocessing engine: used to parse, format, and enhance multimodal input data; extract data patterns and construct relationship graphs; input graph data into the semantic understanding unit and risk simulation unit;

[0025] Structural Verification Unit: uses a dual-engine architecture to detect known attacks and dynamically update the rule base;

[0026] Semantic Understanding Unit: Achieve multimodal fusion risk assessment through a triple analysis architecture;

[0027] Risk simulation unit: Generates risk heat maps through trinity attack simulation;

[0028] Verification decision maker: intercepts or releases based on the decision data output by each unit, and sends the judgment conclusion to the disposal executor;

[0029] Executor: intercept risky data and release risk-free data.

[0030] Preferably, the structure verification unit includes:

[0031] Static rule matching engine: matches SQL injection, XSS and path traversal attacks;

[0032] Dynamic learning engine: Generates new rules and updates the rule base through GNN anomaly detection.

[0033] Preferably, the semantic understanding unit includes:

[0034] Text analysis module: quantifies text risks based on the BERT model;

[0035] Image analysis module: detects sensitive targets based on the YOLO model;

[0036] Structured data analysis module: builds a relationship graph based on the TabNet model.

[0037] Preferably, the risk simulation unit includes:

[0038] Membership inference attack module: quantify the risk of training set leakage;

[0039] Data inversion attack module: assess information reconstruction possibility;

[0040] Backdoor detection module: identify hidden malicious logic.

[0041] In some embodiments, the device includes a processor and a memory storing program instructions, the processor being configured to execute the multi-modal deep learning based cross-region semantic security verification method when running the program instructions.

[0042] The multi-modal deep learning based cross-region semantic security verification method, system and device provided by the embodiments of the present disclosure can achieve the following technical effects:

[0043] The present application combines structure verification technology, semantic understanding technology and risk simulation technology, aiming to realize comprehensive semantic security verification of cross-security domain transmission data, and ensure the confidentiality and integrity of sensitive information in the transmission process.

[0044] Structure verification technology: through a double-engine architecture (static rule matching engine and dynamic learning engine), the data is matched with rules to detect known security attacks such as SQL injection, XSS attacks, etc. At the same time, graph neural network (GNN) is used for anomaly detection to automatically identify and supplement new attack rules to the static rule library, realizing dynamic updating of the rule library.

[0045] Semantic understanding technology: a triple analysis architecture is adopted to establish deep learning models (such as BERT quantification model, YOLO sensitive detection, TabNet analysis) for text, image and structured data respectively, to deeply analyze the potential semantic risks of data and realize content understanding and protection. Through multi-modal fusion data risk assessment, the detection capability for complex semantic attacks is improved.

[0046] Risk simulation technology: simulate real attack scenarios before data cross-domain, quantify the risk of leakage, and dynamically generate protection strategies. Through the trinity attack simulation (member reasoning attack, data inversion attack, backdoor detection), a risk assessment heat map is formed to intuitively show the security risk state of the data.

[0047] The cross-region semantic security verification architecture proposed by the present application effectively solves the data security problem in the process of data cross-security domain transmission in the large model training scene, and provides a strong guarantee for the safe application of artificial intelligence technology.

[0048] The general description above and the description below are only exemplary and explanatory, and are not used to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0049] One or more embodiments are illustrated by way of example in the figures that are not intended to be limiting of the embodiments. Like numbers refer to like elements throughout the drawings, which are not necessarily to scale, and in which:

[0050] Figure 1 is a schematic diagram of the method of the present application;

[0051] Figure 2 is a general architecture diagram of the system;

[0052] Figure 3 is a block diagram of the structure verification unit;

[0053] Figure 4 is a block diagram of the semantic understanding unit;

[0054] Figure 5 is a block diagram of the risk simulation unit.

[0055] Figure 6 is a schematic diagram of the apparatus provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0056] In order to enable a more detailed understanding of the features and technical content of the embodiments of the present disclosure, the implementation of the embodiments of the present disclosure is described in detail below with reference to the accompanying drawings, which are only used for reference and do not limit the embodiments of the present disclosure. In the following technical description, in order to facilitate explanation, a plurality of details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be simplified to facilitate the drawings.

[0057] The terms "first", "second", and the like in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0058] Unless otherwise specified, the term "a plurality of" means two or more.

[0059] In the embodiments of the present disclosure, the character " / " represents an "or" relationship between the objects before and after it. For example, A / B means: A or B.

[0060] The term "and / or" is a description of the association relationship between objects, which means that there can be three relationships. For example, A and / or B means: A or B, or, A and B, the three relationships.

[0061] The term "corresponding" can refer to a kind of association or binding relationship, A corresponding to B means that there is an association or binding relationship between A and B.

[0062] Embodiment 1

[0063] As Figure 1 shown, a cross-region semantic security verification method based on multi-modal deep learning is provided. For data transmitted across security domains, the data is first preprocessed by a preprocessing engine, then analyzed and detected by a structure verification unit, a semantic understanding unit and a risk simulation unit, and finally a decision is made by a verification decision maker to determine the data processing method and realize risk isolation.

[0064] Specifically, it includes:

[0065] S1: The multi-modal data transmitted across the security domain is parsed, formatted and feature enhanced by the preprocessing engine to generate standardized input;

[0066] S2: The preprocessed data is parallelly input to the structure verification unit, the semantic understanding unit and the risk simulation unit:

[0067] The structure verification unit detects known attack patterns through a static rule matching engine and a dynamic learning engine, and dynamically updates the rule library;

[0068] The semantic understanding unit processes text, image and structured data through a triple analysis architecture to generate a multi-modal fusion risk assessment result;

[0069] The risk simulation unit simulates real threats through member reasoning attacks, data inversion attacks and backdoor detection to generate a risk heat map;

[0070] S3: The verification decision maker integrates the output results of the structure verification unit, the semantic understanding unit and the risk simulation unit to perform interception or release operations on the cross-domain transmission data.

[0071] As a refinement of the above embodiment, the preprocessing engine includes: performing character set unification and sensitive word labeling on text data; performing EXIF metadata stripping, resolution standardization and color gamut correction on image data; performing pattern extraction and relationship graph construction on structured data.

[0072] As a refinement of the above embodiment, the dynamic learning engine of the structure verification unit uses a graph neural network for anomaly detection, including: constructing an attribute graph containing field name, data type and value range; identifying abnormal associations and generating new rules through GNN and supplementing to the static rule matching library.

[0073] As a refinement of the above embodiment, the triple analysis architecture of the semantic understanding unit includes: using a BERT quantization model to analyze the semantic risk of the text; using a YOLO model to detect image sensitive content; and using a TabNet model to analyze the potential association of structured data.

[0074] As a refinement of the above embodiment, the trinity attack simulation of the risk simulation unit includes:

[0075] Member inference attack: infer whether the data belongs to a specific training set;

[0076] Data inversion attack: reconstructing the original sensitive information;

[0077] Backdoor detection: identify the trigger mode of malicious implantation.

[0078] It should be noted that: the present application proposes a cross-regional semantic security verification method based on multi-modal deep learning. This method combines structure verification technology, semantic understanding technology and risk simulation technology, aiming to realize comprehensive semantic security verification of cross-security domain transmission data, and ensure the confidentiality and integrity of sensitive information in the transmission process.

[0079] Through the rule matching of the double-engine architecture, known security attacks such as SQL injection and XSS attacks are detected. At the same time, graph neural network (GNN) is used for anomaly detection, which automatically identifies and supplements new attack rules to the static rule library, realizing the dynamic updating of the rule library.

[0080] A triple analysis architecture is adopted, and deep learning models (such as BERT quantization model, YOLO sensitive detection, TabNet analysis) are established for text, image and structured data respectively, which deeply analyzes the potential semantic risk of data, realizes content understanding and protection, and improves the detection ability of complex semantic attacks through multi-modal fusion data risk assessment.

[0081] Before data cross-domain, simulate real attack scenarios, quantify leakage risks, and dynamically generate protection strategies. Through trinity attack simulation (member inference attack, data inversion attack, backdoor detection), a risk assessment heat map is formed to intuitively display the security risk state of the data.

[0082] The cross-regional semantic security verification architecture proposed by the present application effectively solves the data security problem of data transmission across security domains in the large model training scene, and provides a strong guarantee for the safe application of artificial intelligence technology.

[0083] Embodiment 2

[0084] Figure 2The application discloses a multi-modal deep learning-based cross-region semantic security verification system and a main data processing flow thereof. Data (101) transmitted across security domains enters a preprocessing engine (102) for preprocessing, and then enters a structure verification unit (103), a semantic understanding unit (104) and a risk simulation unit (105) respectively to complete risk identification of the data. A verification decision maker (106) performs risk judgment according to the risk identification result and sends a judgment conclusion to a disposal executor (107) to intercept risk data and release non-risk data.

[0085] Specifically, the application comprises the following steps:

[0086] The preprocessing engine (102) is used for parsing, formatting and strengthening multi-modal input data, extracting data patterns and constructing a relationship graph, and inputting the graph data into the semantic understanding unit and the risk simulation unit.

[0087] The structure verification unit (103) adopts a double-engine architecture to detect known attacks and dynamically update a rule library.

[0088] The semantic understanding unit (104) realizes multi-modal fusion risk assessment through a triple analysis architecture.

[0089] The risk simulation unit (105) generates a risk heat map through a trinity attack simulation.

[0090] The verification decision maker (106) intercepts or releases data according to the output decision data of each unit and sends a judgment conclusion to the disposal executor.

[0091] The executor (107) intercepts risk data and releases non-risk data.

[0092] As a refinement of the above embodiment, the preprocessing engine preprocesses input text, image and structured data combined with text and image, unifies character sets of the text data, marks sensitive words, strips EXIF metadata of the image data, normalizes resolution, corrects color gamut, extracts patterns of the structured data, constructs a relationship graph, completes parsing, formatting and feature strengthening of the input data, and provides required input for subsequent semantic analysis.

[0093] As a refinement of the above embodiment, the structure verification unit comprises the following steps:

[0094] The static rule matching engine matches SQL injection, XSS and path traversal attacks, disposes the data according to the matching result, intercepts risk data and releases non-risk data.

[0095] The dynamic learning engine constructs an attribute graph of the input data through GNN anomaly detection, automatically identifies hidden abnormal associations through graph learning, generates new rules and updates the rule library.

[0096] Specifically, Figure 3 As a structure verification unit block diagram, after the input data is preprocessed by structure parsing, static rule matching is performed to detect known attacks such as SQL injection, XSS, path traversal, and the detection result is sent to the risk decision module. The decision execution module performs corresponding processing on the data according to the detection conclusion of the risk decision module, intercepts the risk data, and releases the non-risk data.

[0097] At the same time of static rule matching, after the input data is parsed by structure, an attribute graph is constructed. The attribute graph is marked by sensitive information recognition through field name, data type, value range and context relationship. The GNN abnormality detection processes the attribute graph data, automatically identifies hidden abnormal associations through graph learning, captures new attacks that cannot be detected by traditional rules, and notifies the rule generator of the detected new attacks. New rules are generated to supplement the static rule library, realizing dynamic updating of the rule library.

[0098] As a refinement of the above embodiment, specifically, the semantic understanding unit comprises:

[0099] Text analysis module: quantifying text risk based on BERT model;

[0100] Image analysis module: detecting sensitive targets based on YOLO model;

[0101] Structured data analysis module: constructing relationship graph based on TabNet model.

[0102] Specifically, Figure 4 As a semantic understanding unit block diagram, the semantic understanding unit processes different types of data differently. The BERT quantization model is established for text data, YOLO sensitive detection is performed for images, and TabNet analysis is performed for structured data. Finally, multi-modal fusion data is formed for risk assessment.

[0103] As a refinement of the above embodiment, specifically, the risk simulation unit comprises:

[0104] Member inference attack module: quantifying training set leakage risk;

[0105] Data inversion attack module: evaluating information reconstruction possibility;

[0106] Backdoor detection module: identifying hidden malicious logic.

[0107] Appendix Figure 5For the risk simulation unit, simulate real attacks before data crossing, quantify the risk of leakage, dynamically generate protection strategies, use a triad attack simulation, perform member inference attacks, data inversion attacks, and backdoor detection on input data, and finally form a risk heat map to identify data security risks.

[0108] In combination Figure 6 As shown in the figure, the disclosure embodiment provides a cross-region semantic security verification device 300 based on multi-modal deep learning, which includes a processor 304 and a memory 301. Optionally, the device can also include a communication interface 302 and a bus 303. Wherein the processor 304, the communication interface 302, the memory 301 can complete the mutual communication through the bus 303. The communication interface 302 can be used for information transmission. The processor 304 can call the logical instructions in the memory 301 to execute the cross-region semantic security verification method based on multi-modal deep learning of the above-mentioned embodiment.

[0109] In addition, the logical instructions in the memory 301 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium.

[0110] The memory 301 as a kind of computer readable storage medium can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the disclosure. The processor 304 executes the program instructions / modules stored in the memory 301, thereby executing function application and data processing, i.e. implementing the cross-region semantic security verification method based on multi-modal deep learning in the above-mentioned embodiment.

[0111] The memory 301 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function; The data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 301 can include a high-speed random access memory, and can also include a non-volatile memory.

[0112] The disclosure embodiment provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are set to execute the above-mentioned cross-region semantic security verification method based on multi-modal deep learning.

[0113] The above-mentioned computer readable storage medium can be a transitory computer readable storage medium, or a non-transitory computer readable storage medium.

[0114] The technical solution of the embodiments of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The aforementioned storage medium may be a non-transitory storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code, or a transient storage medium.

[0115] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to also include plural forms. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the process, method or device that includes the element. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments can be referenced to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be found in the description of the method part.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0117] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, the functional units in the embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0118] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

Claims

1. A cross-region semantic security verification method based on multimodal deep learning, characterized in that: The following steps are involved: The preprocessing engine parses, formats, and enhances the features of multimodal data transmitted across security domains to generate standardized input; The pre-processed data is input into the structure verification unit, semantic understanding unit and risk simulation unit in parallel: The structure verification unit detects known attack patterns through a static rule matching engine and a dynamic learning engine, and dynamically updates the rule base; The semantic understanding unit processes text, image and structured data separately through a triple analysis architecture to generate a multimodal fusion risk assessment result; The risk simulation unit simulates real threats through member inference attacks, data inversion attacks and backdoor detection to generate a risk heat map; The verification decision maker integrates the output results of each unit and performs interception or release operations on cross-domain transmission data.

2. The cross-region semantic security verification method based on multimodal deep learning according to claim 1 is characterized in that: The preprocessing engine includes: performing character set unification and sensitive word tagging on text data; performing EXIF ​​metadata stripping, resolution standardization and color gamut correction on image data; and performing pattern extraction and relationship graph construction on structured data.

3. The cross-region semantic security verification method based on multimodal deep learning according to claim 1 is characterized in that: The dynamic learning engine of the structure verification unit uses a graph neural network to perform anomaly detection, including: building an attribute graph containing field names, data types and value ranges; identifying abnormal associations through GNN and generating new rules, which are added to the static rule matching library.

4. The cross-region semantic security verification method based on multimodal deep learning according to claim 1 is characterized in that: The triple analysis architecture of the semantic understanding unit includes: using the BERT quantization model to parse text semantic risks; using the YOLO model to detect sensitive image content; and using the TabNet model to analyze potential associations in structured data.

5. The cross-region semantic security verification method based on multimodal deep learning according to claim 1 is characterized in that: The three-in-one attack simulation of the risk simulation unit includes: Membership inference attack: inferring whether data belongs to a specific training set; Data inversion attack: reconstructing original sensitive information; Backdoor detection: Identify trigger patterns for malicious implants.

6. A cross-regional semantic security verification system based on multimodal deep learning, characterized by: The cross-region semantic security verification method based on multimodal deep learning according to any one of claims 1 to 5 comprises: Preprocessing engine: used to parse, format, and enhance multimodal input data; extract data patterns and construct relationship graphs; input graph data into the semantic understanding unit and risk simulation unit; Structural Verification Unit: uses a dual-engine architecture to detect known attacks and dynamically update the rule base; Semantic Understanding Unit: Achieve multimodal fusion risk assessment through a triple analysis architecture; Risk simulation unit: Generates risk heat maps through trinity attack simulation; Verification decision maker: intercepts or releases based on the decision data output by each unit, and sends the judgment conclusion to the disposal executor; Executor: intercept risky data and release risk-free data.

7. The cross-region semantic security verification system based on multimodal deep learning according to claim 6 is characterized in that: The structure verification unit includes: Static rule matching engine: matches SQL injection, XSS and path traversal attacks; Dynamic learning engine: Generates new rules and updates the rule base through GNN anomaly detection.

8. The cross-regional semantic security verification system based on multimodal deep learning according to claim 6 is characterized in that: The semantic understanding unit includes: Text analysis module: quantifies text risks based on the BERT model; Image analysis module: detects sensitive targets based on the YOLO model; Structured data analysis module: builds a relationship graph based on the TabNet model.

9. The cross-region semantic security verification system based on multimodal deep learning according to claim 6 is characterized in that: The risk simulation unit includes: Membership inference attack module: quantify the risk of training set leakage; Data inversion attack module: evaluates the possibility of information reconstruction; Backdoor detection module: identifies hidden malicious logic.

10. A cross-region semantic security verification device based on multimodal deep learning, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the cross-region semantic security verification method based on multimodal deep learning as described in any one of claims 1 to 5 when running the program instructions.

Images