Science and technology sensitive data leakage prevention method and system based on vectorization and semantic retrieval

Abstract

The application discloses a scientific and technological sensitive data anti-leakage method and system based on vectorization and semantic retrieval. The system comprises a data acquisition and access layer, an intelligent analysis engine layer, a strategy and response execution layer, a management and learning layer and a data storage layer. The intelligent analysis engine layer performs semantic retrieval in a vector database based on a query vector through an embedded semantic retrieval and risk research engine, carries out multi-factor risk assessment in combination with context information, and outputs a structured risk decision package. The strategy and response execution layer generates and executes specific protection instructions connected with existing security infrastructure according to the risk level in the risk decision package. The management and learning layer is used for providing system configuration, monitoring and event investigation interfaces. The data storage layer stores different types of data. The application realizes deep understanding of technical connotation through a multi-modal unified vectorization module, can identify sensitive information, sets a semantic retrieval and risk research engine, carries out dynamic risk assessment, accurately classifies and judges risks, and triggers differentiated response strategies.

Description

Technical Field

[0001] This invention relates to the field of information security technology, specifically to a method and system for preventing the leakage of sensitive scientific and technological data based on vectorization and semantic retrieval. Background Technology

[0002] Currently, with technological innovation becoming a core competitive advantage, enterprises and research institutions generate massive amounts of unstructured sensitive data during daily research and development and collaborative communication. This sensitive data includes, but is not limited to, technical design documents, source code, experimental data, algorithm models, patent drafts, and technical discussion records; this data is the organization's most valuable asset, and its leakage would cause incalculable losses.

[0003] Existing data leakage prevention technologies suffer from the following shortcomings: rule-based or keyword-based matching methods cannot handle variations such as literal modifications and synonym substitutions, resulting in high false positive and false negative rates; traditional machine learning-based methods heavily rely on feature engineering, making it difficult to capture deep semantics and exhibiting weak generalization capabilities; and methods based on shallow topic models cannot understand fine-grained technical logic. The fundamental flaw of these methods lies in their failure to truly understand the connotations of technical content at a deep semantic level, leading to limited effectiveness in protecting against leakage behaviors presented in the form of technical summaries, explanations of ideas, pseudocode, and diagrams. Summary of the Invention

[0004] To address this issue, the present invention provides a method and system for preventing the leakage of sensitive scientific and technological data based on vectorization and semantic retrieval, in order to solve the problems in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A technology-sensitive data leakage prevention system based on vectorization and semantic retrieval includes:

[0007] The data acquisition and access layer is used to capture raw data from multiple potential leakage points in the enterprise IT environment, and to perform protocol parsing, decryption and encapsulation processing on the raw data to form a preliminary data object with basic metadata.

[0008] The intelligent analysis engine layer connects the data acquisition and access layer. It receives preliminary data objects, standardizes them through the built-in content slicing and preprocessing module to form content units to be inspected, converts the content units to be inspected into query vectors through the built-in multimodal unified vectorization module, and performs semantic retrieval in the vector database based on the query vectors through the built-in semantic retrieval and risk assessment engine. It also performs multi-factor risk assessment in combination with context information and outputs a structured risk decision package.

[0009] The strategy and response execution layer generates and executes specific protection instructions that interface with the existing security infrastructure based on the risk level in the risk decision package;

[0010] The management and learning layer provides interfaces for system configuration, monitoring, and incident investigation, and drives iterative optimization of models in the intelligent analysis engine layer based on operational feedback data.

[0011] The data storage layer is used to store vector data, system configurations, operating metrics, and copies of the original content.

[0012] Furthermore, the data acquisition and access layer includes a network traffic probe, a terminal agent, a cloud API gateway, and a storage system scanner. The data acquisition and access layer captures data from network traffic, terminal operations, cloud application interactions, and static storage systems.

[0013] Furthermore: the content slicing and preprocessing module performs normalization processing on the original data, including encoding conversion, format unification, semantic boundary recognition, and content slicing, and packages complete contextual metadata for each content unit formed after normalization processing;

[0014] The semantic boundary determination process includes: firstly, calculating the average cosine similarity between words within the sliding window and entries in the technical terminology database, using the following formula: ;

[0015] Then, determine the starting position of the slice. It satisfies:

[0016] ;

[0017] in, The average semantic similarity of the i-th window; This is the starting position for slicing; This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. L is the semantic relevance threshold; L is the length of the sliding window; N is the total number of words in the document.

[0018] If none of the positions meet the threshold conditions, the default paragraph boundaries will be used for segmentation.

[0019] The formula for cross-segment coherence verification is:

[0020] ;

[0021] in, Indicates two adjacent slices and Jaccard coefficient for keywords; For the continuity reception threshold, when When n=1, the segmentation is deemed reasonable; n is the total number of segments into which the current document is divided; when n>100, no verification is required; when n>100, a distributed verification strategy is enabled to calculate local coherence in batches.

[0022] Furthermore, the multimodal unified vectorization module provides real-time, batch vectorization computation capabilities for preprocessed content units and sensitive samples managed in the background; it supports real-time semantic encoding of multimodal content such as text, code, and images.

[0023] The cross-modal alignment loss function is:

[0024] ;

[0025] in, The feature vector output by the text / code encoder; The feature vector output by the image encoder; The temperature coefficient controls the steepness of the probability distribution; These are positive sample pairs; For the set of all possible negative samples;

[0026] Furthermore, the formula for calculating the dynamic route weight is as follows:

[0027] ;

[0028] in, Let be the activation weight of the i-th routing node in layer l; Let be the learnable weight matrix for the i-th route in layer l; Let i be the key vector of the i-th route in the l-th layer; Let i be the query vector for the i-th route at layer l; This is the bias term for the i-th route at layer l; This is the Sigmoid activation function.

[0029] Furthermore, the semantic retrieval and risk assessment engine integrates a multi-factor risk assessment algorithm and a lightweight user behavior analysis model, and comprehensively considers semantic similarity, context weight, behavior sequence, and content anonymity to generate accurate risk scores and levels.

[0030] The aggregation formula for the multi-factor risk assessment algorithm is as follows:

[0031] ;

[0032] Where R is the comprehensive risk score; S is the semantic similarity score, based on the reciprocal of the cosine distance returned by ANN retrieval; C is the contextual risk value, derived from metadata weighting; A is the behavioral anomaly score; M is the concealment index; and α, β, γ, and δ are the weight parameters of each risk factor, and ;

[0033] The formula for adversarial example detection is:

[0034] ;

[0035] in, The strength of evidence that input sample x is judged as an adversarial sample; To counteract the disturbance threshold; Let L be the gradient vector of the loss function with respect to the input x, where the loss L is calculated during forward propagation and the gradient is obtained during backpropagation. ,in Let L be the partial derivative of the loss function L with respect to the input x; y is the model prediction function; y is the true label.

[0036] Furthermore, the calculation process for the aforementioned behavioral anomalies is as follows:

[0037] 1) Constructing user historical behavior sequences ;

[0038] 2) Predicting the behavior at the next time step using LSTM ;

[0039] 3) Calculate the actual behavior x t+1 Compared with the predicted value The root mean square error is calculated using the following formula:

[0040] ,

[0041] in Let i be the actual behavioral feature value of the i-th dimension; Let m be the predicted behavioral feature value for the i-th dimension; m is the dimension of the behavioral feature vector.

[0042] Furthermore, the strategy and response execution layer includes a hierarchical response strategy library, a real-time blocking module, an approval workflow engine, and a proactive desensitization / watermarking module. The strategy and response execution layer executes immediate blocking, triggers approval processes, records alarms, or initiates proactive protection measures according to different risk levels.

[0043] Furthermore, the vector database in the data storage layer adopts a distributed architecture to store massive amounts of sensitive information reference vector indexes and provides high-concurrency, low-latency near nearest neighbor semantic retrieval services; in addition, the semantic retrieval and risk assessment engine can call the vector database.

[0044] To achieve the above objectives, the present invention also provides a method for preventing the leakage of sensitive scientific and technological data based on vectorization and semantic retrieval, comprising the following steps:

[0045] S1. Unified vectorization modeling and index construction of multimodal sensitive information; collect multi-source heterogeneous science and technology sensitive data samples, preprocess them, use deep semantic models to vectorize text, code and image content respectively, and map them to a unified semantic space through comparative learning, construct a benchmark vector index and store it in a vector database;

[0046] S2. Real-time semantic perception and vectorization of full-traffic content; capture outgoing data streams in real time from multiple touchpoints including network, terminal and cloud, perform dynamic semantic slicing on outgoing data streams to form content units to be inspected and extract context information, and then use a multimodal unified vectorization model to convert the content units to be inspected into query vectors in real time.

[0047] S3. Dynamic risk intelligent assessment based on semantic retrieval: The query vector is used to perform near nearest neighbor semantic retrieval in the vector database to obtain similar sensitive information vectors and their similarity; multiple factors such as similarity, contextual information, abnormal user behavior sequence and content concealment are combined to calculate a dynamic risk score, and the risk level is determined accordingly.

[0048] S4. Tiered and precise response and closed-loop feedback optimization; tiered response operations are executed according to the determined risk level; high-risk content is blocked immediately and proactive protection is initiated, medium-risk content is intercepted and the approval process is triggered, and low-risk content is recorded and an alarm is triggered; at the same time, the handling results and manual feedback information are collected to optimize the vectorized model, risk assessment model and update the sensitive information knowledge base.

[0049] Furthermore: newly discovered and confirmed sensitive information samples are re-entered into the process of step S1 and updated in the index of the vector database.

[0050] This invention has the following advantages: Through a multimodal unified vectorization module, it transforms text, code, images, and other content into high-dimensional vectors within a unified semantic space, enabling a deep understanding of the technical implications and effectively identifying sensitive information leaks through synonym substitution, code obfuscation, or non-original text formats. The semantic retrieval and risk assessment engine integrates multi-dimensional factors such as semantic similarity, user context, behavioral sequences, and content concealment for dynamic risk assessment, achieving accurate classification of high, medium, and low risks and triggering differentiated response strategies. This ensures security while minimizing interference with normal work.

[0051] Other features and advantages of the present invention will be set forth in the following description. Attached Figure Description

[0052] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).

[0053] Figure 1 This is a structural diagram of a technology-sensitive data leakage prevention system based on vectorization and semantic retrieval provided in an embodiment of this application.

[0054] Figure 2 This is a real-time detection timing diagram of a technology-sensitive data leakage prevention system based on vectorization and semantic retrieval in an embodiment of the present invention.

[0055] Figure 3 This is a flowchart illustrating the implementation of the technology-sensitive data leakage prevention system based on vectorization and semantic retrieval of this invention. Detailed Implementation

[0056] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these embodiments are merely for further explanation of the present invention and should not be construed as limiting the scope of protection of the present invention. Those skilled in the art can make some non-essential improvements and adjustments to the present invention based on the above-described content.

[0057] Please see Figures 1-3 A technology-sensitive data leakage prevention system based on vectorization and semantic retrieval, comprising a data acquisition and access layer, an intelligent analysis engine layer, a strategy and response execution layer, a management and learning layer, and a data storage layer.

[0058] The data acquisition and access layer can actively or passively collect raw data from various potential leakage points in the enterprise IT environment. By implementing protocol parsing, SSL decryption, application layer payload extraction, and behavior log collection, the captured raw data is encapsulated into a unified preliminary data object and attached with basic metadata, including timestamps, source IPs, user IDs, etc.

[0059] The raw data includes network outbound traffic, end-user operations, cloud application interactions, and static storage content.

[0060] Furthermore, the data acquisition and access layer integrates data compliance filtering and privacy protection mechanisms when performing data capture; specifically including:

[0061] (1) User informed consent management: Connect with the enterprise's unified identity authentication system, and show and record the signed internal "Data Security Monitoring Informed Consent Form" of the enterprise when the terminal agent is activated and monitored for the first time, to ensure that the monitoring behavior complies with the provisions of laws and regulations;

[0062] (2) Hierarchical permissions and minimal data collection: The depth of data collection is dynamically adjusted according to the user role (such as R&D, administration, senior management); for example, only the external behavior records of documents are collected for non-core sensitive positions without collecting specific content, while full content audits are conducted for core R&D positions to ensure compliance with the principle of minimum necessity.

[0063] (3) Pre-processing of desensitization and anonymization: Before the data is passed into the intelligent analysis engine layer, personal identity information (such as personal mobile phone number and non-work-related chat records) that is not related to business logic is desensitized or anonymized in real time; specifically, identification and replacement based on regular expressions (such as replacing mobile phone number with Hash value) or masking based on named entity recognition can be used.

[0064] (4) Compliance audit log: Generate an independent privacy compliance audit log to record the scope, time, data subjects and purposes of data collection. The log is tamper-proof and is used by enterprises to fulfill their legal compliance audit obligations.

[0065] The intelligent analysis engine layer receives the encapsulated raw data; transforms the raw content into understandable semantics and performs intelligent analysis, and outputs a risk decision package.

[0066] The intelligent analysis engine layer's built-in content slicing and preprocessing modules perform deep analysis and intelligent slicing of data, enriching contextual information to form standard content units to be inspected and contextual metadata data packages. The data packages are sent to the multimodal unified vectorization module, which converts them into query vectors. The query vectors are sent to the vector database in the data storage layer for retrieval, returning similar vector sets and similarity scores. The semantic retrieval and risk assessment engine receives the query vectors, retrieval results, and rich contextual packages, performs multi-factor calculations, and produces a structured risk decision package (including risk level, score, basis, and recommended actions).

[0067] The strategy and response execution layer transforms the content of the risk decision package from the intelligent analysis engine layer into specific and effective security actions. Specifically, it translates the abstract risk levels and assessment context into concrete instructions that interface with the enterprise's existing security infrastructure (firewalls, email gateways, endpoint agents, approval systems) and proactive protection modules (data masking, watermarking).

[0068] The management and learning layer provides a human-computer interaction interface, enabling security administrators to have a global view, policy configuration, incident investigation, and case review capabilities. It can also collect operational feedback and automatically trigger model training, evaluation, and deployment processes, allowing the multimodal unified vectorization module and semantic retrieval and risk assessment engine of the intelligent analysis engine layer to continuously iterate and optimize.

[0069] The data storage layer can efficiently and reliably store different types of data. Among them, the vector database is specifically designed for fast retrieval of massive high-dimensional vectors; the relational database stores structured configurations and metadata; the time-series database records metrics for monitoring; and the object storage saves copies of the original content for evidence collection and model training.

[0070] Furthermore, the vector database can store and manage massive amounts of sensitive information reference vector indexes; the vector database adopts a distributed architecture to form a vector database cluster, thereby providing highly available and highly concurrent semantic retrieval services.

[0071] The data acquisition and access layer includes network traffic probes, terminal agents, cloud API gateways, and storage system scanners;

[0072] Among them, the network traffic probe is deployed on the core switch to perform deep packet inspection at the network boundary, obtain the application layer content of protocols such as HTTP / S, SMTP, and FTP, support SSL / TLS decryption, and ensure that encrypted traffic is visible;

[0073] Terminal Agent: Installed on employee terminals as a lightweight process or driver, it monitors file operations (creation, copying, moving), clipboard usage, print jobs, peripheral connections, and sensitive operations of specific applications (such as IDEs and office software).

[0074] Cloud API Gateway: Connects to SaaS applications (such as GitHub, Jira, Slack, Office 365) via API integration or reverse proxy, and monitors cloud collaboration and sharing behavior;

[0075] Storage scanner: Regularly or in real-time scans of storage systems such as file servers, databases, and code repositories to detect the unauthorized retention of sensitive information in static storage.

[0076] The intelligent analysis engine layer includes a content slicing and preprocessing module, a multimodal unified vectorization module, and a semantic retrieval and risk assessment engine.

[0077] The content slicing and preprocessing module standardizes the raw data, including encoding conversion, format unification, semantic boundary recognition, and content slicing, and packages complete contextual metadata for each content unit.

[0078] In this embodiment, code conversion can employ a multi-level encoding adaptation mechanism, supporting automatic detection and conversion of mainstream character sets such as UTF-8 / GBK / Big5; for unstructured data, a regular expression engine is used to parse special symbol systems; for binary files (such as Office documents), a COM-OLE object parsing framework is used to achieve underlying encoding reconstruction; all data is uniformly normalized to the Unicode base encoding to ensure cross-platform compatibility.

[0079] The format unification involves establishing a multimedia format adaptation layer to transform heterogeneous data sources into standardized intermediate representations. Key processing steps include converting tabular data into nested dictionary structures, converting mathematical formulas into MathML expressions, generating Base64 encoded snapshots of image attachments, and finally outputting a set of RDF triples that conforms to the ISO / IEC 23894 standard.

[0080] Context metadata is encapsulated by attaching a five-dimensional metadata package to each content unit (granularity can be configured as sentence / paragraph / chapter).

[0081] The five-dimensional metadata data includes: a. Spatiotemporal dimension: creation / modification timestamps accurate to milliseconds, GPS coordinates (if available); b. Structural topology: parent container ID, sibling node sequence, DOM tree level depth; c. Semantic association: citation literature chain, co-occurring term network, knowledge graph node links; d. Security attributes: anonymization level markers, access control policy pointers, digital watermark hash values; e. Quality indicators: information entropy density, term coverage, sentiment score.

[0082] In addition, for content slicing, a hybrid strategy of "rules first, model second, and post-validation" is adopted to implement differentiated slicing for different data types.

[0083] Specifically, they can be divided into the following categories:

[0084] 1. For documents containing explicit formatting tags, such as Word, PDF, and Markdown, a structured document slicing method is adopted, which is tag-based segmentation. First, the hierarchical structure of the document (such as headings H1, H2, list and code block tags) is extracted using a document parsing library. The 2nd and 3rd level headings are used as the default priority segmentation boundaries, and all text, charts, and code under a heading are treated as a semantic unit. If a chapter is too long (e.g., more than 1000 words), a second fine-grained segmentation is performed at paragraph boundaries and chart titles to ensure that each slice is of appropriate size.

[0085] 2. Plain text and natural language content slicing; that is, for unformatted text streams (such as email body, technical description text), sentence boundary detection and semantic coherence analysis are performed first; specifically, high-performance natural language processing tools are used for sentence segmentation first; then, a lightweight semantic coherence model (such as cosine similarity sliding window calculation based on sentence embedding) is used for analysis.

[0086] After the analysis is completed, dynamic window segmentation is performed to maintain a dynamic "semantic window". Starting from the beginning of the text, the average semantic similarity between the continuously added sentences and the existing content in the window is calculated. When the newly added sentences cause the average similarity to drop significantly (below the set threshold), the previous semantic topic is determined to end, all sentences in the current window are output as a slice, and a new window is opened. This method can adaptively segment according to the semantic changes of the content itself.

[0087] 3. For program source code, the goal of slicing is to maintain the integrity of the code's functionality; therefore, it is based on the Abstract Syntax Tree (AST): the parser of the corresponding language is used to convert the code file into an abstract syntax tree. Independent function / method definitions, class definitions, and logical code blocks (such as the body of loops and conditional branches) are used as the core slicing units.

[0088] In addition, during the splitting process, the function signature, key comments, and the class or module name to which the function belongs are preserved as metadata to provide richer context.

[0089] 4. For interactive content such as chat logs and comment streams, interactive conversation stream slicing is adopted, that is, segmentation is based on conversation threads and topics; first, a dialogue tree is built according to conversation participants and reply relationships; then, a relatively complete "question-answer pair" or multiple rounds of discussion around a specific technical issue is taken as a slice;

[0090] In long threads, fast topic drift detection based on keyword or short text similarity is used. When a significant shift in discussion topic is detected, the thread is split even within the same thread.

[0091] Among them, semantic boundary recognition is based on NLP technology to automatically divide technical documents / code into logical units; and binds them with context, associating metadata such as user roles, operation paths, and device fingerprints; for text, it is parsed in segments / sentences to preserve the integrity of technical terms; for code, it uses syntax tree parsing to slice it at the function / class / interface granularity; and for images, it uses OCR and object detection to separate text and chart areas.

[0092] The semantic boundary determination process includes: firstly, calculating the average cosine similarity between words within the sliding window and entries in the technical terminology database, using the following formula: ;

[0093] Then, determine the starting position of the slice. It satisfies:

[0094] ;

[0095] in, The average semantic similarity of the i-th window; This is the starting position for slicing; This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. L is the semantic relevance threshold; L is the length of the sliding window; N is the total number of words in the document.

[0096] If none of the positions meet the threshold conditions, the default paragraph boundaries will be used for segmentation.

[0097] The formula for cross-segment coherence verification is:

[0098] ;

[0099] in, Indicates two adjacent slices and Jaccard coefficient for keywords; For the continuity reception threshold, when When n=1, the segmentation is deemed reasonable; n is the total number of segments into which the current document is divided; when n>100, no verification is required; when n>100, a distributed verification strategy is enabled to calculate local coherence in batches.

[0100] The multimodal unified vectorization module loads the latest unified vectorization model, providing real-time and batch vectorization computing capabilities for preprocessed content units and sensitive samples managed in the background; it supports real-time semantic encoding of multimodal content such as text, code, and images.

[0101] The cross-modal alignment loss function is:

[0102] ;

[0103] in, The feature vector output by the text / code encoder; The feature vector output by the image encoder; The temperature coefficient controls the steepness of the probability distribution; These are positive sample pairs; For the set of all possible negative samples;

[0104] The formula for calculating the dynamic route weight is as follows:

[0105] ;

[0106] in, The activation weight of the i-th routing node in layer l controls the transmission strength of information between different modalities. In this embodiment, it can be the feature fusion ratio from text to image. Let be the learnable weight matrix of the i-th route in layer l. and If both are d-dimensional, then It is a d×d matrix; Let i be the key vector of the i-th route in the l-th layer; Let i be the query vector for the i-th route at layer l; This is the bias term for the i-th route at layer l; This is the Sigmoid activation function.

[0107] The semantic retrieval and risk assessment engine integrates a multi-factor risk assessment algorithm and a lightweight user behavior analysis model. It comprehensively considers dimensions such as semantic similarity, context weight, behavior sequence, and content anonymity to generate accurate risk scores and levels.

[0108] The aggregation formula for the multi-factor risk assessment algorithm is as follows:

[0109] ;

[0110] Where R is the comprehensive risk score; S is the semantic similarity score, based on the reciprocal of the cosine distance returned by ANN retrieval; C is the contextual risk value, derived from metadata weighting, with different weights for user role, device type, transmission channel, and time window, all of which sum to 1; A is the behavioral anomaly score, based on the prediction bias of the LSTM model (i.e., the aforementioned lightweight user behavior analysis model); M is the concealment index, reflecting the degree to which content evades detection; α, β, γ, and δ are the weight parameters of each risk factor, and .

[0111] The calculation process for abnormal behavior is as follows:

[0112] 1) Constructing user historical behavior sequences ;

[0113] 2) Predicting the behavior at the next time step using LSTM ;

[0114] 3) Calculate the actual behavior x t+1 Compared with the predicted value The root mean square error is calculated using the following formula:

[0115] ,

[0116] in Let i be the actual behavioral feature value of the i-th dimension; Let m be the predicted behavioral feature value for the i-th dimension; m is the dimension of the behavioral feature vector.

[0117] In addition, when a user generates a new operation to be inspected, the frequency of the current operation (such as "attempting to send technical documents") within the recent window is calculated in real time. If the frequency significantly exceeds the user's historical baseline average by several standard deviations, an abnormal signal is generated. This process uses an efficient exponentially weighted moving average algorithm to quickly respond to changes in behavior.

[0118] The formula for adversarial example detection is:

[0119] ;

[0120] in, The strength of evidence that input sample x is judged as an adversarial sample; To counteract the disturbance threshold, an alarm is triggered if this value is exceeded; Let L be the gradient vector of the loss function with respect to the input x, where the loss L is calculated during forward propagation and the gradient is obtained during backpropagation. ,in Let L be the partial derivative of the loss function L with respect to the input x; y is the model prediction function; y is the true label.

[0121] In addition, the semantic retrieval and risk assessment engine can call vectorized services and vector databases.

[0122] The strategy and response execution layer includes a tiered response strategy library, a real-time blocking module, an approval workflow engine, and a proactive de-identification / watermarking module. The tiered response strategy library can issue execution instructions to the real-time blocking module, the approval workflow engine, and the proactive de-identification / watermarking module based on the data in the risk decision package. The real-time blocking module, the approval workflow engine, and the proactive de-identification / watermarking module can issue blocking policies to the firewall, return rejection instructions to the email gateway, pop up warning windows on the terminal, call the approval system to create work orders, and trigger the proactive de-identification / watermarking service.

[0123] The management and learning layer includes a management console, a case feedback and model optimization module, and a knowledge base and sample library management module.

[0124] The management console provides a web-based graphical interface for policy configuration, real-time risk dashboards, incident investigation and auditing, case review and annotation, and system monitoring reports.

[0125] The case feedback and model optimization module receives feedback data from the management console, automatically schedules GPU resources, regularly performs model fine-tuning and evaluation tasks, and automatically pushes new models that meet the evaluation criteria to the vectorization service and risk assessment service in the production environment.

[0126] The knowledge base and sample library management module provides an interface for security administrators to manage the sensitive information sample library (add, delete, modify, and query), and trigger sample re-vectorization and index update tasks.

[0127] See Figure 2 The real-time detection process of the system of the present invention is as follows:

[0128] Step 1. Data Trigger: The user (or application) initiates an operation that may contain sensitive data, such as sending an email or uploading a file.

[0129] Step 2. Data capture: The sensing access layer (network probe / terminal agent) captures the data stream in real time and passes it to the preprocessing module.

[0130] Step 3. Content Slicing: The preprocessing module intelligently slices long documents, conversation streams, etc., according to semantic boundaries to form content units to be inspected.

[0131] Step 4. Vectorization Preparation: The sliced ​​content units are sent to the vectorization module;

[0132] Step 5. Real-time vectorization: The vectorization module uses a unified semantic model to convert text / code and other content into high-dimensional vectors;

[0133] Step 6. Semantic Retrieval: The vectorization module initiates a query to the vector database to search for semantically similar sensitive information;

[0134] Step 7. Return Results: The vector database returns the most similar sensitive vectors and their similarity distances through an approximate nearest neighbor search;

[0135] Step 8. Risk assessment preparation: The vectorization module packages the search results with contextual information such as user role and transmission channel and passes them to the risk assessment engine;

[0136] Step 9. Dynamic Scoring: The risk assessment engine calculates a dynamic risk score by integrating multiple factors such as semantic similarity, context, and behavioral patterns to determine the risk level;

[0137] Step 10. Instruction Issuance: Based on the risk level, generate specific handling instructions (block, intercept, alarm) and send them to the policy executor;

[0138] Step 11. Execute a tiered response:

[0139] High risk: Immediately block transmission and initiate proactive protection (such as content anonymization and watermarking); Medium risk: Intercept operation, trigger approval workflow, and notify user; Low risk: Log and alert administrator, but allow normal access for user.

[0140] Step 12. User Feedback: The policy executor returns the corresponding operation result to the user (blocking prompt, approval prompt, etc.);

[0141] Step 13. Incident Reporting: Regardless of the handling method, the complete incident details are reported to the management console for auditing and analysis.

[0142] In addition, see Figure 3 In this embodiment, the processing flow of the technology-sensitive data leakage prevention system is as follows:

[0143] (1) The system captures real-time outgoing data streams from channels such as the network, terminal, and cloud, and performs intelligent semantic slicing to form content units to be detected (such as a paragraph or a piece of code).

[0144] (2) At the same time, multimodal unified vectorization processing is performed on samples of science and technology sensitive information from internal and external sources to generate high-dimensional semantic vectors, which are then used as a benchmark index and stored in a high-performance vector database.

[0145] (3) The content unit to be inspected is vectorized in real time and converted into a vector in the same semantic space; the system performs near nearest neighbor (ANN) semantic retrieval in the vector database to find the sensitive information that is most similar to the semantics of the content to be inspected, rather than performing literal matching;

[0146] (4) The search results are fed into the dynamic risk assessment model (semantic retrieval and risk assessment engine); the dynamic risk assessment model integrates multiple factors such as semantic similarity, context, and user behavior to calculate the dynamic risk score and make a three-level risk classification decision of high, medium and low.

[0147] (5) Trigger differentiated response actions based on risk level:

[0148] High risk: Implements the strictest blocking measures and can be integrated with proactive protection technologies (such as content desensitization and adding tracking watermarks).

[0149] Medium risk: Intercept and guide users into the compliance approval process, balancing security and business efficiency;

[0150] Low risk: Record and alert, and submit to the administrator for review afterward to minimize disruption to normal work;

[0151] All response results and corrective information from manual review are fed back to the system as feedback data for continuous optimization of the vectorized model and risk assessment model.

[0152] This embodiment also provides a method for preventing the leakage of sensitive scientific and technological data based on vectorization and semantic retrieval, including the following steps:

[0153] Step S1. Unified vectorization modeling and index construction of multimodal sensitive information; specifically including the collection and preprocessing of multi-source heterogeneous data, multimodal deep semantic vectorization, unified semantic space alignment and mapping, and construction of a baseline vector index.

[0154] Specifically: the collection and preprocessing of multi-source heterogeneous data involves collecting raw sample data from internal data sources (such as Git / SVN code repositories, Confluence / Wiki knowledge bases, PDM systems, internal technical forums, and mail server archives) and authorized public data sources, such as academic paper repositories, technical patent repositories, and open-source communities; preprocessing includes: denoising, segmenting, and language recognition of text data; syntax parsing and annotation extraction of code data; OCR recognition or key feature extraction of image / chart data; and converting all data into a unified intermediate representation format.

[0155] Multimodal deep semantic vectorization includes text vectorization, code vectorization, and image vectorization.

[0156] Among them, text vectorization uses a large language model that has been pre-trained and fine-tuned on a large-scale corpus of scientific and technological fields as an encoder; for long documents, a hierarchical or paragraph-level encoding strategy is adopted to finally generate a fixed-dimensional document semantic vector.

[0157] Code vectorization combines the syntactic structure and semantic information of the code. First, the code is parsed into an abstract syntax tree, and then it is encoded using a code-specific model based on the Transformer architecture (such as CodeBERT and GraphCodeBERT) to generate vectors that represent the functional logic of the code.

[0158] Image vectorization involves using the image encoder branch of a vision-language pre-trained model (such as CLIP) for technical charts, architecture diagrams, formula images, etc., or first extracting text from the image using OCR and then fusing it with image features to generate semantic vectors.

[0159] To achieve cross-modal semantic retrieval, the vectors generated by the different encoders mentioned above need to be aligned to the same semantic space. In this embodiment, a contrastive learning network with a dual-tower structure is used for mapping training.

[0160] Building a baseline vector index involves importing the semantic vectors of all sensitive information samples that have undergone alignment and mapping into a high-performance vector database to create a fast-searchable index. This index supports efficient approximate nearest neighbor search algorithms, providing millisecond-level response capabilities for subsequent real-time retrieval.

[0161] Step S2. Real-time semantic awareness and vectorization of full-traffic content, including multi-touchpoint data stream capture, dynamic content slicing and context extraction, and real-time vectorization conversion;

[0162] Among them, multi-touch data stream capture captures the content of all possible outgoing data channels non-intrusively through mirror probes deployed at the network boundary, lightweight agents on the terminal, log interfaces of cloud service API gateways, and storage system scanners.

[0163] Dynamic content slicing and context extraction involve intelligent semantic boundary recognition of captured continuous data streams (such as long emails, multi-turn chat logs, and large files), slicing them according to complete technical arguments, code functions, and logical paragraphs to form independent units of content to be examined. Simultaneously, rich contextual metadata of these data units is extracted and recorded.

[0164] Contextual metadata includes: subject information (user ID, role, department), object information (data source application, storage location), environment information (time, geographical location), behavior information (operation type: send, upload, copy, print), and channel information (destination IP / domain name, recipient, usage protocol).

[0165] Real-time vectorization transformation converts each unit of content to be inspected into a query vector in a unified semantic space in real time using a trained unified vectorization model.

[0166] Step S3. Dynamic risk intelligent assessment based on semantic retrieval; including near nearest neighbor semantic retrieval, multi-factor dynamic risk fusion calculation and risk level determination.

[0167] Nearest Neighbor Semantic Retrieval involves submitting a query vector of the content to be retrieved to a vector database. The database then uses an ANN algorithm to quickly retrieve the K most similar benchmark sensitive information vectors and returns their similarity scores and corresponding sample labels.

[0168] Multi-factor dynamic risk fusion calculation involves designing a risk scoring function to comprehensively calculate the final risk value. The scoring factors of the risk scoring function include semantic similarity factor, contextual risk factor, behavioral sequence anomaly factor, and content concealment factor.

[0169] The semantic similarity factor is based on the similarity scores returned by the retrieval, using the highest score or a weighted average score as the basis. A non-linear mapping function is set to convert the similarity scores into an initial risk baseline value;

[0170] Contextual risk factors are weighted based on extracted metadata using a predefined or machine learning-derived weight matrix; for example, if a core R&D department user sends data to a personal email address after accessing the source code server, this combination will receive an extremely high risk weight.

[0171] The behavioral sequence anomaly factor combines a user's short-term historical behavior (such as frequently trying to send fragments containing similar technical keywords in the past 5 minutes) and uses a lightweight time series model (such as simple statistics or RNN) to determine whether the current behavior deviates from the baseline, generating anomaly bonuses;

[0172] The Content Concealment Factor (Mod) detects whether content units have been intentionally concealed, such as using screenshots instead of text, text as images, character encoding obfuscation, or using inline code snippets to evade detection; identifying such behaviors will significantly improve the risk score.

[0173] Risk level determination involves comparing the calculated dynamic risk score with a preset threshold range and classifying it into three risk levels: high risk (clear leak, requires immediate blocking), medium risk (suspected leak, requires manual review and interception), and low risk (low suspicion, only alarms are recorded).

[0174] Step S4. Graded precise response and closed-loop feedback optimization; including graded response execution, closed-loop feedback and model self-optimization.

[0175] The specific execution content of the tiered response is as follows:

[0176] For high-risk events, immediate blocking is implemented, and proactive protection modules can be activated according to the strategy; including: using text generation technology to replace specific parameters, models, and algorithm names with generalized categories without changing the fluency of the sentences; embedding invisible tracking watermarks that are bound to users / time in non-sensitive parts that are allowed to be released or in the desensitized text.

[0177] For medium-risk events, the operation is blocked, and an approval workflow is automatically triggered to notify the data owner and security administrator. At the same time, the reason for the block and similar content fragments are shown to the user, allowing them to submit an appeal or resend after modification.

[0178] For low-risk events, user operations will not be affected, but the complete content, context, and risk analysis results will be recorded in the audit log, and a low-priority alarm will be generated for administrators to review later.

[0179] Closed-loop feedback and model self-optimization involve collecting all events (regardless of how they are handled) and their final human confirmation results to form a high-quality labeled data stream;

[0180] Vectorization model optimization involves periodically using new positive and negative sample pairs (e.g., confirmed leaked content as positive samples and normal business content confirmed as false alarms as difficult negative samples) to incrementally fine-tune the unified vectorization model, making its semantic boundaries clearer.

[0181] Risk assessment model optimization involves using event data as a training set and employing reinforcement learning or supervised learning methods to dynamically adjust the parameters and weights in the risk scoring function, so that the model's decisions continuously align with human expert standards.

[0182] The knowledge base is updated by re-entering the process of step S1 with newly discovered and confirmed sensitive information samples and updating the index of the vector database.

[0183] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A technology-sensitive data leakage prevention system based on vectorization and semantic retrieval, characterized in that, include: The data acquisition and access layer is used to capture raw data from multiple potential leakage points in the enterprise IT environment, and to perform protocol parsing, decryption and encapsulation processing on the raw data to form a preliminary data object with basic metadata. The intelligent analysis engine layer connects to the data acquisition and access layer. It receives the preliminary data object, standardizes the preliminary data object through the built-in content slicing and preprocessing module to form the content unit to be inspected, converts the content unit to be inspected into a query vector through the built-in multimodal unified vectorization module, and performs semantic retrieval in the vector database based on the query vector through the built-in semantic retrieval and risk assessment engine. It also performs multi-factor risk assessment in combination with context information and outputs a structured risk decision package. The strategy and response execution layer generates and executes specific protection instructions that interface with the existing security infrastructure based on the risk level in the risk decision package. The management and learning layer provides interfaces for system configuration, monitoring, and incident investigation, and drives the semantic retrieval and risk assessment engine in the intelligent analysis engine layer to perform iterative optimization based on operational feedback data. The data storage layer is used to store vector data, system configurations, operating metrics, and copies of the original content.

2. The technology-sensitive data leakage prevention system based on vectorization and semantic retrieval according to claim 1, characterized in that, The data acquisition and access layer includes a network traffic probe, a terminal agent, a cloud API gateway, and a storage system scanner. The data acquisition and access layer captures data from network traffic, terminal operations, cloud application interactions, and static storage systems.

3. The technology-sensitive data leakage prevention system based on vectorization and semantic retrieval according to claim 1, characterized in that, The content slicing and preprocessing module performs standardization processing on the initial data objects, including encoding conversion, format unification, semantic boundary recognition, and content slicing, and packages complete contextual metadata for each content unit formed after standardization. The semantic boundary determination process includes: firstly, calculating the average cosine similarity between words within the sliding window and entries in the technical terminology database, using the following formula: ； Then, determine the starting position of the slice. It satisfies: ； in, The average semantic similarity of the i-th window; This is the starting position for slicing; This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. L is the semantic relevance threshold; L is the length of the sliding window; N is the total number of words in the document. If none of the positions meet the threshold condition, the default paragraph boundaries will be used for segmentation. The formula for checking cross-segment coherence is: ； in, Indicates two adjacent slices and Jaccard coefficient of keywords; n is the total number of segments into which the current document is divided; no validation is required when n=1; when n>100, a distributed validation strategy is enabled to calculate local coherence in batches. For the continuity reception threshold, when At that time, the division was deemed reasonable.

4. The technology-sensitive data leakage prevention system based on vectorization and semantic retrieval according to claim 1, characterized in that, The multimodal unified vectorization module provides real-time, batch vectorization computation capabilities for preprocessed content units and sensitive samples managed in the background; it supports real-time semantic encoding of multimodal content, including text, code, and images. The cross-modal alignment loss function is: ； in, The feature vector output by the text / code encoder; The feature vector output by the image encoder; Temperature coefficient; These are positive sample pairs; For the set of all possible negative samples; Furthermore, the formula for calculating the dynamic route weight is as follows: ； in, Let be the activation weight of the i-th routing node in layer l; Let be the learnable weight matrix for the i-th route in layer l; Let i be the key vector of the i-th route in the l-th layer; Let i be the query vector for the i-th route at layer l; This is the bias term for the i-th route at layer l; This is the Sigmoid activation function.

5. The technology-sensitive data leakage prevention system based on vectorization and semantic retrieval according to claim 1, characterized in that, The semantic retrieval and risk assessment engine integrates a multi-factor risk assessment algorithm and a lightweight user behavior analysis model, and generates accurate risk scores and levels by comprehensively considering semantic similarity, context weight, behavior sequence, and content anonymity. The aggregation formula for the multi-factor risk assessment algorithm is as follows: ； Where R is the comprehensive risk score; S is the semantic similarity score, based on the reciprocal of the cosine distance returned by ANN retrieval; C is the contextual risk value, derived from metadata weighting; A is the behavioral anomaly score; M is the concealment index; and α, β, γ, and δ are the weight parameters of each risk factor, and ; The formula for adversarial example detection is: ； in, The strength of evidence that input sample x is judged as an adversarial sample; To counteract the disturbance threshold; Let L be the gradient vector of the loss function with respect to the input x, where the loss L is calculated during forward propagation and the gradient is obtained during backpropagation. ,in Let L be the partial derivative of the loss function L with respect to the input x; y is the model prediction function; y is the true label.

6. The technology-sensitive data leakage prevention system based on vectorization and semantic retrieval according to claim 5, characterized in that, The calculation process for the behavioral abnormality score is as follows: 1) Constructing user historical behavior sequences ; 2) Predict the behavior at the next moment using the aforementioned lightweight user behavior analysis model. The lightweight user behavior analysis model uses an LSTM model. 3) Calculate the actual behavior x t+1 Compared with the predicted value The root mean square error is calculated using the following formula: ， in Let i be the actual behavioral feature value of the i-th dimension; Let m be the predicted behavioral feature value for the i-th dimension; m is the dimension of the behavioral feature vector.

7. The technology-sensitive data leakage prevention system based on vectorization and semantic retrieval according to claim 1, characterized in that, The strategy and response execution layer includes a hierarchical response strategy library, a real-time blocking module, an approval workflow engine, and a proactive de-identification / watermarking module. The strategy and response execution layer performs immediate blocking, triggers approval processes, records alarms, or initiates proactive protection measures according to different risk levels.

8. The technology-sensitive data leakage prevention system based on vectorization and semantic retrieval according to claim 1, characterized in that, The vector database in the data storage layer adopts a distributed architecture to store massive amounts of sensitive information baseline vector indexes and provide high-concurrency, low-latency near nearest neighbor semantic retrieval services; in addition, the semantic retrieval and risk assessment engine can call the vector database.

9. A method for preventing the leakage of sensitive scientific and technological data based on vectorization and semantic retrieval, characterized in that, Includes the following steps: S1. Unified vectorized modeling and index construction of multimodal sensitive information; Collect multi-source heterogeneous science and technology sensitive data samples, preprocess them, and use deep semantic models to vectorize the text, code and image content respectively. Then, through comparative learning, they are mapped to a unified semantic space, a benchmark vector index is constructed and stored in a vector database. S2. Real-time semantic perception and vectorization of full-traffic content; capture outgoing data streams in real time from multiple touchpoints, perform dynamic semantic slicing on the outgoing data streams to form content units to be inspected and extract contextual information, and then use a multimodal unified vectorization model to convert the content units to be inspected into query vectors in real time. S3. Dynamic risk intelligent assessment based on semantic retrieval: The query vector is used to perform approximate nearest neighbor semantic retrieval in the vector database to obtain similar sensitive information vectors and their similarity; multiple factors are combined to perform fusion calculation to obtain a dynamic risk score, and the risk level is determined accordingly. S4. Hierarchical precise response and closed-loop feedback optimization; Based on the determined risk level, a tiered response operation is executed; high-risk content is blocked immediately and proactive protection is initiated, medium-risk content is intercepted and an approval process is triggered, and low-risk content is recorded and an alarm is triggered; at the same time, the handling results and manual feedback information are collected to optimize the vectorization model, risk assessment model, and update the sensitive information knowledge base.

10. The method for preventing leakage of sensitive scientific and technological data based on vectorization and semantic retrieval according to claim 9, characterized in that, Newly discovered and confirmed sensitive information samples are re-entered into step S1 and updated in the vector database index.

Images