A method and related device for leakage interference based on real-time monitoring information

Abstract

The application discloses a leakage interference method based on real-time monitoring information and related equipment, and relates to the field of plant protection. The method comprises the following steps: obtaining pixel stream data of a target display screen; performing a layout detection operation, a text recognition operation and a chart analysis operation on the pixel stream data to obtain a candidate display area; performing sensitivity determination based on a rule matching mechanism and a target deep learning model to obtain a risk score of the candidate display area; determining the candidate display area with a risk score exceeding a dynamic threshold as a sensitive area; extracting layout style features of the sensitive area, and generating alternative content irrelevant to the original semantics under the premise of maintaining consistent style by combining a generative adversarial network model; and based on optical flow alignment and timing consistency constraints, the alternative content is rendered to the corresponding sensitive area in real time to replace the original content and output to the target display screen. The method effectively overcomes the deficiencies of the prior art in real-time performance, visual consistency and cross-modal protection.

Description

Technical Field

[0001] This specification relates to the field of plant protection; more specifically, this application relates to a method and related equipment for interfering with the leakage of information based on real-time monitoring information. Background Technology

[0002] In existing technologies, the protection of sensitive information on display screens mainly relies on static desensitization and manual intervention. Common methods include masking or blurring sensitive text at the software rendering layer, or preventing unauthorized users from accessing sensitive content through access control.

[0003] However, these methods have significant shortcomings: On the one hand, static desensitization strategies typically rely on manually preset rules and fixed templates, making it difficult to cope with dynamic changes in screen content, such as real-time generated reports, scrolling data dashboards, and sensitive segments in instant messaging windows. On the other hand, traditional blurring or black-frame overlays disrupt the user's normal visual experience, easily attracting attention, and may even allow attackers to recover some information through image restoration, contextual reasoning, and other means.

[0004] In addition, existing technologies mostly focus on desensitizing text content, lacking the ability to analyze and replace complex charts and image elements as a whole, and cannot effectively prevent the inference of potential sensitive information through chart shape or layout structure.

[0005] Therefore, there is an urgent need for a more intelligent, flexible method of information leakage interference with real-time processing capabilities. Summary of the Invention

[0006] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0007] This application proposes a method and related equipment for intercepting information leaks based on real-time monitoring information.

[0008] Firstly, this application proposes a method for interfering with information leakage based on real-time monitoring information, including:

[0009] Acquire pixel stream data of the target display screen;

[0010] Perform layout detection, text recognition, and chart parsing operations on the above pixel stream data to obtain candidate display areas;

[0011] Sensitivity determination is performed based on rule matching mechanism and target deep learning model to obtain the risk score of the above candidate display areas;

[0012] Candidate display areas whose risk scores exceed the dynamic threshold are identified as sensitive areas;

[0013] Extract layout style features from the aforementioned sensitive areas, and combine them with a generative adversarial network model to generate alternative content that is unrelated to the original semantics while maintaining style consistency;

[0014] Based on optical flow alignment and timing consistency constraints, the aforementioned alternative content is rendered in real time to the corresponding sensitive area to replace the original content and output to the target display screen.

[0015] In one feasible implementation, the above-mentioned sensitivity determination based on rule matching mechanism and target deep learning model to obtain the risk score of the candidate display region includes:

[0016] Based on a preset rule base, regular expression matching and pattern validation are performed on the text content of the candidate display area to identify content that conforms to a predefined sensitive pattern and obtain the rule hit rate. The predefined sensitive patterns include keywords, phrases, data formats and contextual structures.

[0017] A semantic classification network based on a pre-trained language model is used to identify the text semantics of the above candidate display areas in order to obtain the text sensitivity probability.

[0018] An image recognition model based on a combination of convolutional neural networks and attention mechanisms analyzes the charts and / or image content of candidate display areas to obtain the chart sensitivity probability.

[0019] The risk score is output by weighting and fusing the rule hit rate, the text sensitivity probability, the chart sensitivity probability, and the application context factor to which the candidate display area belongs.

[0020] In one feasible implementation, the above image recognition model includes:

[0021] The convolutional backbone and multi-scale feature pyramid are used to extract the visual feature representations of the above candidate display regions;

[0022] A layout markup encoder is used to construct a layout feature representation by embedding the appearance, geometric position encoding, and element type embedding of the above visual feature representations;

[0023] A cross-modal attention module is used to fuse the text token obtained by optical character recognition and the above layout feature representation to generate a semantically enhanced layout feature representation;

[0024] The temporal attention module is used to temporally fuse the semantically enhanced layout feature representation of the current frame and the semantically enhanced layout feature representation of the historical frames aligned with optical flow to reduce jitter and obtain a temporally robust graph feature representation.

[0025] The sensitive prototype matching unit is used to calculate the similarity between the aforementioned time-robust chart feature representation and the preset sensitive prototype library to obtain the chart sensitivity probability of the aforementioned candidate display areas.

[0026] In one feasible implementation, the steps described above for constructing a layout feature representation from the appearance embedding, geometric position encoding, and element type embedding of the visual feature representation include:

[0027] Locate graph components on multi-scale visual feature representations to obtain geometric candidates;

[0028] Perform the RoIAlign operation on each of the above geometric candidates to extract the appearance embedding vector;

[0029] The center coordinates, width, height and orientation angle of the candidates are normalized and mapped to geometric position encoding vectors by sine and cosine position encoding and multilayer perceptron;

[0030] Learnable element type embedding vectors are obtained by looking up a table based on the component category;

[0031] The above-mentioned appearance embedding vector, the above-mentioned geometric position encoding vector and the above-mentioned element type embedding vector are concatenated and linearly projected and normalized to obtain a layout feature representation of a unified dimension, and then the above-mentioned layout feature representation is formed according to a preset spatial or reading order.

[0032] In one feasible implementation, the step of generating semantically enhanced layout feature representations by the cross-modal attention module includes:

[0033] Optical character recognition is performed on the text within the candidate display area to obtain text tokens, and the corresponding semantic vectors are obtained through a text encoder;

[0034] Using the semantic vectors mentioned above as keys and values, and the layout feature representation as queries, we input them into a multi-head attention network to calculate the semantic association weights between each layout element and the text token.

[0035] Based on the aforementioned semantic association weights, the aforementioned semantic vectors are weighted and summed, and then fused with the corresponding layout feature representations to output the semantically enhanced layout feature representations.

[0036] In one feasible implementation, the specific steps for determining the aforementioned dynamic threshold include:

[0037] The robust central value and dispersion are calculated based on the risk scores of the candidate display areas within the sliding window, and the basic threshold is obtained by combining quantile estimation.

[0038] The above-mentioned basic threshold, application scenario factor, modality factor, and user viewing factor are used to obtain the context-adjusted dynamic threshold;

[0039] When a drift in the risk score distribution is detected, the stringency of the aforementioned dynamic threshold is increased, and the aforementioned dynamic threshold is dynamically adjusted based on the target alarm rate through an online calibration mechanism.

[0040] In one feasible implementation, the above-mentioned extraction of layout style features from the sensitive areas, combined with a generative adversarial network model, generates alternative content unrelated to the original semantics while maintaining style consistency, including:

[0041] Obtain the font, color, layout, and chart element style characteristics of the aforementioned sensitive areas;

[0042] The above-mentioned layout style features are used as conditional inputs to the generative adversarial network model;

[0043] In the above generative adversarial network model, maintaining consistent layout style is a constraint, and minimizing mutual information with the original content semantics is an optimization objective. Alternative content that is unrelated to the original semantics is generated. The alternative content includes grammatically correct alternative text content that is unrelated to the original semantics, or chart content that is consistent with the original chart layout but whose data relevance is less than a preset threshold.

[0044] Secondly, the present invention also proposes a data leakage interference system based on real-time monitoring information, comprising:

[0045] The first acquisition unit is used to acquire pixel stream data of the target display screen;

[0046] The second acquisition unit is used to perform layout detection, text recognition and chart parsing operations on the above pixel stream data to obtain candidate display areas.

[0047] The third acquisition unit is used to perform sensitivity determination based on the rule matching mechanism and the target deep learning model in order to obtain the risk score of the above candidate display areas.

[0048] The determination unit is used to identify candidate display areas whose risk scores exceed the dynamic threshold as sensitive areas.

[0049] The generation unit is used to extract layout style features from the above-mentioned sensitive areas and, in conjunction with the generative adversarial network model, generate alternative content that is unrelated to the original semantics while maintaining style consistency.

[0050] The replacement unit is used to render the above-mentioned replacement content to the corresponding sensitive area in real time based on optical flow alignment and timing consistency constraints, so as to replace the original content and output it to the target display screen.

[0051] Thirdly, the present invention also proposes an electronic device comprising: a memory and a processor, wherein the processor is configured to implement the steps of the leakage interference method based on real-time monitoring information as described in any of the first aspects when executing a computer program stored in the memory.

[0052] Fourthly, the present invention also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the leakage interference method based on real-time monitoring information as described in any one of the first aspects.

[0053] In summary, this method acquires frame-by-frame pixel stream data from the screen and combines layout detection, text recognition, and chart parsing operations to quickly and automatically locate candidate display areas, avoiding the limitations of manual annotation and preset templates. This enables real-time monitoring and sensitivity identification of dynamically generated content. The method incorporates a rule-matching mechanism combined with a deep learning model in the sensitivity determination process. This ensures high recall for fixed patterns while effectively covering complex charts and contextual structures through semantic understanding and cross-modal image recognition, significantly improving the accuracy of sensitive area identification. The method utilizes an adaptive dynamic threshold mechanism, combining sliding window statistics, quantile estimation, and contextual factors. This allows for dynamic adjustment of the judgment criteria based on content fluctuations and changes in the viewing environment, avoiding the problem of static thresholds being too strict or too lenient, thus ensuring system stability in different scenarios. Finally, this method introduces a generative adversarial network-based alternative content generation strategy. Through layout style feature extraction and mutual information minimization constraints, it can generate alternative text or charts that visually maintain the style of the original content but are semantically completely decoupled. This ensures that the user's visual experience is not disrupted while completely shielding the risk of sensitive information leakage. Compared to traditional black-frame occlusion or blurring methods, the alternative content generated by this method has higher realism and naturalness, making it more difficult for attackers to detect. This method combines optical flow alignment and temporal consistency constraints to render the alternative content onto sensitive areas of the screen in real time, effectively avoiding flickering and misalignment caused by screen scrolling, refreshing, or data updates. This ensures that the information leakage interference remains smooth and stable even in dynamic video streaming scenarios. In summary, this method not only overcomes the shortcomings of existing technologies in terms of real-time performance, visual consistency, and cross-modal protection, but also provides a scalable and intelligent information leakage interference solution, demonstrating significant practical value and promising prospects for widespread application.

[0054] Other advantages, objectives and features of this application will be apparent in part from the description which follows, and in part from what those skilled in the art will understand through study and practice of this application. Attached Figure Description

[0055] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0056] Figure 1 This application provides a schematic flowchart of a method for interfering with information leakage based on real-time monitoring information.

[0057] Figure 2 A schematic diagram illustrating a risk score determination process provided in an embodiment of this application;

[0058] Figure 3 A schematic diagram illustrating a construction layout feature representation provided in an embodiment of this application;

[0059] Figure 4 A schematic diagram illustrating a process for generating semantically enhanced layout feature representations, provided in an embodiment of this application;

[0060] Figure 5 A flowchart illustrating the generation of a dynamic threshold is provided for an embodiment of this application.

[0061] Figure 6 A schematic diagram of the structure of a data leakage interference system based on real-time monitoring information is provided for an embodiment of this application.

[0062] Figure 7 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0063] The terms "first," "second," "third," "fourth," etc. (if present) in the technical solutions of this application and in the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. The technical solutions of the embodiments of this application will now be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0064] Please see Figure 1 This is a flowchart illustrating a method for interfering with information leakage based on real-time monitoring information, provided in an embodiment of this application. Specifically, it may include:

[0065] Firstly, this application proposes a method for interfering with information leakage based on real-time monitoring information, including:

[0066] S110. Obtain pixel stream data of the target display screen;

[0067] S120. Perform layout detection, text recognition, and chart parsing operations on the above pixel stream data to obtain candidate display areas;

[0068] S130. Sensitivity determination is performed based on rule matching mechanism and target deep learning model to obtain the risk score of the above candidate display area;

[0069] S140. The candidate display areas whose risk scores exceed the dynamic threshold are identified as sensitive areas.

[0070] S150. Extract layout style features from the above-mentioned sensitive areas, and combine them with a generative adversarial network model to generate alternative content that is unrelated to the original semantics while maintaining style consistency.

[0071] S160. Based on optical flow alignment and timing consistency constraints, the above-mentioned replacement content is rendered to the corresponding sensitive area in real time to replace the original content and output to the target display screen.

[0072] For example, in step S110, the system performs real-time acquisition of the target display screen to obtain its output pixel stream data, which is a frame-by-frame image sequence containing all displayed content on the screen. Subsequently, in step S120, the pixel stream data is subjected to layout detection, text recognition, and chart parsing operations to automatically segment candidate display areas containing potentially sensitive information. These candidate areas may be text paragraphs, tables, charts, or specific image elements.

[0073] In step S130, the system performs sensitivity determination on candidate display areas based on a rule-matching mechanism and a target deep learning model. The rule-matching mechanism identifies preset keywords, phrases, or data patterns, while the deep learning model utilizes semantic understanding and image recognition capabilities to perform risk analysis on text and chart content, ultimately outputting a corresponding risk score. Next, in step S140, the system identifies candidate display areas with risk scores exceeding a dynamic threshold as sensitive areas. This dynamic threshold is adaptively set through sliding window statistics, quantile estimation, and contextual factor adjustment, thereby ensuring the stability and accuracy of sensitive area identification.

[0074] In step S150, for the aforementioned sensitive areas, the system extracts layout style features such as font, color, typography, and chart elements, and inputs these features as conditions into the generative adversarial network (GAN) model. During the generation process, the GAN maintains consistent layout style as a constraint and minimizes mutual information with the original content's semantics as an optimization objective, generating alternative content unrelated to the original semantics. The alternative content can be grammatically correct text that does not contain real information, or alternative charts that maintain consistent chart layout but whose data has been perturbed.

[0075] Finally, in step S160, based on optical flow alignment and temporal consistency constraints, the system seamlessly overlays the replacement content onto the corresponding sensitive area of ​​the target display screen, thereby avoiding screen flickering or misalignment in the time sequence. After this processing, the sensitive area seen by the user when viewing the screen has been covered by the replacement content, and the overall display effect is consistent with the original content in visual style, but the key information has been effectively shielded or replaced, achieving real-time information leakage interference.

[0076] In summary, this method acquires frame-by-frame pixel stream data from the screen and combines layout detection, text recognition, and chart parsing operations to quickly and automatically locate candidate display areas, avoiding the limitations of manual annotation and preset templates. This enables real-time monitoring and sensitivity identification of dynamically generated content. The method incorporates a rule-matching mechanism combined with a deep learning model in the sensitivity determination process. This ensures high recall for fixed patterns while effectively covering complex charts and contextual structures through semantic understanding and cross-modal image recognition, significantly improving the accuracy of sensitive area identification. The method utilizes an adaptive dynamic threshold mechanism, combining sliding window statistics, quantile estimation, and contextual factors. This allows for dynamic adjustment of the judgment criteria based on content fluctuations and changes in the viewing environment, avoiding the problem of static thresholds being too strict or too lenient, thus ensuring system stability in different scenarios. Finally, this method introduces a generative adversarial network-based alternative content generation strategy. Through layout style feature extraction and mutual information minimization constraints, it can generate alternative text or charts that visually maintain the style of the original content but are semantically completely decoupled. This ensures that the user's visual experience is not disrupted while completely shielding the risk of sensitive information leakage. Compared to traditional black-frame occlusion or blurring methods, the alternative content generated by this method has higher realism and naturalness, making it more difficult for attackers to detect. This method combines optical flow alignment and temporal consistency constraints to render the alternative content onto sensitive areas of the screen in real time, effectively avoiding flickering and misalignment caused by screen scrolling, refreshing, or data updates. This ensures that the information leakage interference remains smooth and stable even in dynamic video streaming scenarios. In summary, this method not only overcomes the shortcomings of existing technologies in terms of real-time performance, visual consistency, and cross-modal protection, but also provides a scalable and intelligent information leakage interference solution, demonstrating significant practical value and promising prospects for widespread application.

[0077] In one feasible implementation, such as Figure 2 As shown, step S130 above performs sensitivity determination based on a rule matching mechanism and a target deep learning model to obtain the risk score of the candidate display region, including:

[0078] S1301. Based on a preset rule base, perform regular expression matching and pattern verification on the text content of the candidate display area to identify content that conforms to a predefined sensitive pattern and obtain the rule hit rate. The predefined sensitive patterns include keywords, phrases, data formats and contextual structures.

[0079] S1302. A semantic classification network based on a pre-trained language model identifies the text semantics of the above candidate display areas to obtain the text sensitivity probability.

[0080] S1303. An image recognition model based on a combination of convolutional neural networks and attention mechanisms analyzes the charts and / or image content of candidate display areas to obtain the chart sensitivity probability.

[0081] S1304. The above rule hit rate, the above text sensitivity probability, the above chart sensitivity probability, and the above application context factor to which the candidate display area belongs are weighted and fused to output the above risk score.

[0082] For example, regarding S1301, the system uses a preset rule base. The system maintains several configurable sensitive patterns (keywords, phrases, data formats, and contextual structures). Regular expression matching and pattern validation are performed on the text sequences of candidate regions to obtain matching indicators for each rule. With verification passed indication (e.g., whether the check bit, numerical range, and context structure are valid), and combined with the importance of the pattern. Spatial proximity weight (Measuring the geometric relationship between hit items and field names / headers / titles) and OCR confidence. Calculate rule hit rate:

[0083] ;

[0084] in To determine the number of valid words within a region, length normalization is performed to avoid bias in long texts.

[0085] In S1302, the system feeds text fragments of the same candidate region into a semantic classification network based on a pre-trained language model. To obtain the softmax probability of the sensitive category To improve robustness, temperature calibration is introduced. With quality gate coefficient (Comprehensive perplexity PPL, coverage) (and OCR confidence level)

[0086] ;

[0087] in It is Sigmoid. The learnable coefficient, This indicates the coverage ratio of the rule base or vocabulary to the regional text.

[0088] In S1303, the system uses a model that includes a convolutional backbone multi-scale pyramid, layout markers, cross-modal attention, and temporal attention to obtain the chart representation vector for the current frame. .set up If the prototype vector is a learnable sensitive prototype library (such as financial KPI reports, organizational / network topology, undisclosed roadmaps, etc.), then the sensitivity of the original chart is:

[0089]

[0090] in For temperature parameters. Considering the instantaneous jitter caused by screen scrolling or refreshing, [the following parameters are considered]. Perform exponentially weighted time-series smoothing and overlay image quality gating (Estimated from sharpness / SNR / motion blur):

[0091]

[0092] in The interval between adjacent frames, is the time constant.

[0093] In S1304, the system integrates rules, textual and graphical signals, and application context factors to output risk scores. Context factors From application prior Window title / tab prior Prior to viewing the situation (If unauthorized observers exist) This is obtained through the gating function:

[0094] ;

[0095] Simultaneously, a cross-modal consistency metric is introduced. This is used to measure alignment relationships such as "legend item-color-data series" and "axis label-unit-numerical range" (which can be obtained by thresholding the matching matrix of cross-modal attention). The final risk score is calculated using a lightweight fusion model (such as logistic regression or a single-layer MLP).

[0096] ;

[0097] in These are learnable parameters. To further suppress single-channel distortion, dynamic weighted fusion can be used:

[0098] ;

[0099] In summary, the S130... The primary criterion, and by and Context and consistency corrections are implemented, and calibration, gating, and timing smoothing are used to ensure stable and interpretable regional risk scores under different applications, visual qualities, and screen dynamics. This score will be used for subsequent comparison with a dynamic threshold to determine whether to... This area has been identified as a sensitive area.

[0100] In one feasible implementation, the above image recognition model includes:

[0101] The convolutional backbone and multi-scale feature pyramid are used to extract the visual feature representations of the above candidate display regions;

[0102] A layout markup encoder is used to construct a layout feature representation by embedding the appearance, geometric position encoding, and element type embedding of the above visual feature representations;

[0103] A cross-modal attention module is used to fuse the text token obtained by optical character recognition and the above layout feature representation to generate a semantically enhanced layout feature representation;

[0104] The temporal attention module is used to temporally fuse the semantically enhanced layout feature representation of the current frame and the semantically enhanced layout feature representation of the historical frames aligned with optical flow to reduce jitter and obtain a temporally robust graph feature representation.

[0105] The sensitive prototype matching unit is used to calculate the similarity between the aforementioned time-robust chart feature representation and the preset sensitive prototype library to obtain the chart sensitivity probability of the aforementioned candidate display areas.

[0106] For example, the image recognition model consists of multiple functional modules, which are sequentially connected to achieve sensitivity analysis of the content of charts or images in candidate display areas. First, the model uses a convolutional backbone network and a multi-scale feature pyramid to process the input candidate display areas, thereby extracting multi-level visual feature representations. These features include both fine-grained local texture information and preserve overall structural and semantic information. Subsequently, the layout tag encoder encodes the components in the candidate areas based on these visual features, mapping the appearance features, geometric position information, and element type information of each component into vector representations, and uniformly constructing a sequence of layout feature representations.

[0107] Building upon this foundation, the cross-modal attention module further incorporates text tokens obtained through optical character recognition (OCR), fusing them with the layout feature representation. Through a multi-head attention mechanism, the model establishes correspondences between chart elements and text labels, coordinate axis units, and legend descriptions, thereby outputting a semantically enhanced layout feature representation that integrates visual, structural, and semantic information. Next, the temporal attention module, targeting scenarios with dynamically changing screens, fuses the semantically enhanced layout feature representation of the current frame with the optically flow-aligned semantically enhanced layout feature representation of historical frames. Through temporal modeling, the system effectively reduces jitter caused by screen scrolling, flickering, or resolution changes, ensuring the stability of the output chart feature representation over time.

[0108] Finally, the sensitive prototype matching unit calculates the similarity between the time-robust chart feature representation and a preset sensitive prototype library. This library stores feature vectors for various types of typical sensitive charts, such as financial statements, organizational charts, or R&D progress charts. Through similarity measurement, the system can determine whether a candidate display area belongs to a sensitive category and output the corresponding chart sensitivity probability. Therefore, this embodiment achieves high-precision sensitivity identification of screen chart content through step-by-step processing including convolutional feature extraction, layout structured modeling, cross-modal semantic enhancement, time-route consistency maintenance, and prototype matching determination, providing a reliable basis for subsequent information leakage interference.

[0109] In one feasible implementation, such as Figure 3 As shown, the steps described above for constructing a layout feature representation from the appearance embedding, geometric position encoding, and element type embedding of visual feature representations include:

[0110] S210. Locate graph components on multi-scale visual feature representations to obtain geometric candidates;

[0111] S220. Perform the RoIAlign operation on each of the above geometric candidates to extract the appearance embedding vector;

[0112] S230. The center coordinates, width, height and orientation angle of the candidate are normalized and mapped to a geometric position encoding vector by sine and cosine position encoding and multilayer perceptron.

[0113] S240. Obtain the learnable element type embedding vector by looking up the table based on the component category;

[0114] S250. The above-mentioned appearance embedding vector, the above-mentioned geometric position encoding vector and the above-mentioned element type embedding vector are concatenated and linearly projected and normalized to obtain a layout feature representation with a unified dimension, and the layout feature representation is formed according to a preset spatial or reading order.

[0115] For example, the layout mark encoder takes the multi-scale visual features of the candidate display area as input and executes S210 to S250 in sequence to encapsulate information (appearance, geometry, type) from different sources into a layout feature representation that can be used for subsequent cross-modal and temporal modeling.

[0116] Specifically, in S210, the system first locates chart components on the feature maps output by the convolutional backbone and the multi-scale feature pyramid to obtain a set of geometric candidates. This localization process does not rely on fixed anchor boxes, but directly searches for the response peaks of various components (such as x / y axes, grid lines, bar / polyline points, legend items, titles / labels, etc.) on the feature maps, and decodes the center position, width, height, and (optional) orientation angle of each candidate in the original coordinate system. Through this multi-scale localization, the system can cover both fine-grained text / scales and stably capture larger structural elements.

[0117] Next, in S220, for each geometric candidate, the system performs the RoIAlign operation on its best-matching feature layer to extract a fixed-size feature block. This feature block is then further compressed into an appearance embedding vector through lightweight convolution, global average pooling, and a multilayer perceptron. The appearance embedding characterizes the candidate's texture and visual style (e.g., line thickness, color distribution, local texture). Because RoIAlign uses continuous bilinear sampling, it avoids feature shifts caused by quantization errors, thus maintaining stability at different scales and locations.

[0118] Subsequently, in S230, the system normalizes the center coordinates, width, height, and orientation angles of each geometric candidate, decoupling them from the absolute dimensions of the screen or candidate region. Simultaneously, a sine / cosine-based position encoding is introduced, mapping these scalars to vector representations with discriminative power across different frequency bands. This vector is then transformed using a multilayer perceptron to obtain the geometric position encoding vector. This encoding not only preserves position and size information but also enhances the model's sensitivity to geometric relationships such as relative orientation, shape proportions, and area percentages, facilitating subsequent understanding of the "axis-grid-data series-legend / label" structure.

[0119] In S240, the system retrieves the corresponding element type embedding vector from the learnable type embedding table based on the detection category of each geometric candidate. This vector provides a stable semantic prior for different component categories (such as coordinate axes, data points, legend items, text labels, etc.), enabling the model to distinguish between "structural" and "data" elements, as well as components with different functional roles, during feature fusion.

[0120] Finally, at S250, the system concatenates and fuses the three information streams: appearance embedding vector, geometric position encoding vector, and element type embedding vector. This is then mapped to a unified dimension via linear projection and normalization to obtain the layout feature representation of a single component. To facilitate subsequent sequence modeling and attention calculation, the system sorts the layout feature representations of all components according to a preset spatial or reading order, forming a clear and semantically consistent sequence of layout feature representations. Thus, key information from visual appearance, geometric relationships, and category semantics is organically integrated into the same representation space, providing well-aligned, dimensionally unified, and interpretable input for subsequent cross-modal and temporal attention. Through this process, the system can stably and accurately depict the layout structure of chart components in complex and dynamically changing screen scenarios, laying a solid foundation for sensitivity identification and leakage interference decisions.

[0121] In one feasible implementation, such as Figure 4 As shown, the steps of the cross-modal attention module in generating semantically enhanced layout feature representations include:

[0122] S310. Perform optical character recognition on the text within the candidate display area to obtain a text token, and obtain the corresponding semantic vector through a text encoder;

[0123] S320. Using the above semantic vectors as keys and values, and the layout feature representation as a query, input them into a multi-head attention network to calculate the semantic association weights between each layout element and the text token.

[0124] S330. Based on the above semantic association weights, the above semantic vectors are weighted and summed and then fused with the corresponding layout feature representations to output the above semantically enhanced layout feature representations.

[0125] For example, the cross-modal attention module aligns textual semantics with the chart layout structure, thereby supplementing each chart component with semantic labels indicating "what it is" and "what it represents." Specifically, the system first performs optical character recognition within the candidate display area to obtain a sequence of text tokens labeled by spatial location. Each text token is then cleaned and segmented (e.g., removing low-confidence characters, merging split units and values, and correcting common OCR obfuscation). Subsequently, the text tokens are fed into a lightweight text encoder to obtain corresponding semantic vectors, while retaining their bounding box coordinates, row and column information, and OCR confidence on the screen, which are used as constraints and quality gates for subsequent cross-modal alignment. In this way, whether it's labels like "Q1," "operating revenue," or "ten thousand yuan," or series names in the legend items, they are all converted into semantic representations with a unified dimension that can participate in attention calculation.

[0126] After obtaining the layout feature representation and text semantic vector, the module uses the layout feature representation as the query and the text semantic vector as the key and value input to the multi-head attention network to calculate the semantic association between each graph component and its surrounding text tokens. To improve the accuracy and interpretability of the alignment, two types of priors are introduced into the attention layer:

[0127] The first is relative positional bias based on spatial proximity, which makes the components more inclined to focus on text that is spatially close and located in a reasonable direction (for example, the vertical axis focuses more on the units or axis titles that are right next to the axis, and the column focuses more on the category labels below or above it).

[0128] Secondly, the component type-based mask constraint makes it easier to match "legend item-series name", "axis-unit / axis title", "data point-numerical label", etc., while suppressing combinations such as "grid line-paragraph text". The multi-head mechanism allows different attention heads to focus on different relationships, such as numerical-unit matching, color-legend item correspondence, category-column alignment, etc., thereby simultaneously estimating multiple associations in the same forward calculation.

[0129] After obtaining the semantic association weights, the module performs weighted aggregation of the relevant text semantic vectors and fuses them with the original layout feature representation in a residual manner, outputting a semantically enhanced layout feature representation. A quality gating process is also introduced during the fusion: when the overall OCR confidence near a component is low, the text is sparse, or there is occlusion, the system automatically reduces the component's dependence on text branches, preserving more of its visual and geometric information; conversely, when the spatial and type relationships between "text-component" are highly consistent, the fusion weight is increased accordingly, thus significantly enhancing the semantic discriminative power of the component. The final semantically enhanced layout feature not only indicates "this is a bar / axis / legend item," but also carries semantic cues such as "it corresponds to Q1," "the unit is ten thousand yuan," and "the series name is North Region." This result will then be fed into the temporal attention module for robust cross-frame fusion and further used for similarity measurement with a sensitive prototype library, thereby stably and accurately evaluating the chart sensitivity probability of candidate display areas in dynamic scenarios where users view the screen.

[0130] In one feasible implementation, such as Figure 5 As shown, the specific steps for determining the above dynamic threshold include:

[0131] S410. Calculate the robust central value and dispersion based on the risk score of the candidate display area within the sliding window, and obtain the basic threshold by combining quantile estimation.

[0132] S420. Combine the above-mentioned basic threshold, application scenario factor, modal factor and user viewing factor to obtain the context-adjusted dynamic threshold.

[0133] S430. When a drift in the risk score distribution is detected, the stringency of the above dynamic threshold is increased, and the above dynamic threshold is dynamically adjusted according to the target alarm rate through an online calibration mechanism.

[0134] Exemplary, exemplary, such as Figure 5 As shown, this embodiment is for each time moment. With each candidate display area Based on the risk score sequence, an adaptive threshold determination link is constructed to obtain a dynamic threshold for S140 determination. First, for the near... A sliding window is established based on the risk score of this region within the frame.

[0135] To suppress the influence of abnormal impulses, the robust center and dispersion are obtained using the median and median absolute deviation (MAD):

[0136]

[0137] in Indicates the typical risk level within the window. This reflects robust dispersion. To account for high-risk segments at the tail, the upper quantiles are estimated (exponentially weighted quantiles can be used):

[0138] ,

[0139] in To check the loss for quantiles, For quantile levels, For time decay weight, This is the attenuation coefficient. From this, the base threshold is obtained:

[0140]

[0141] in This is the dispersion amplification factor, used to adjust the sensitivity to fluctuations. The design ensures that the threshold both follows the center and reflects the general level of dispersion, while not falling below the tail quartile to cover short-term high risks.

[0142] Then, context adjustment is performed to adapt to different application environments, content modalities, and viewing behaviors. Application scenario factors are defined. Modal factor With user viewing factors Multiplicative combination yields the context-dynamic threshold:

[0143] ;

[0144] Among these, each factor can be linearly parameterized in the logarithmic field and its positive, constrained coefficients obtained through exponential / truncation, for example...

[0145]

[0146] For application categories, window titles / tabs, etc., primary / intersecting features, These are learnable weights; It can be obtained from the one-hot vectors of the content modality (text / chart / image / code, etc.) through isomorphic mapping; Features can be composed of the number of viewers, the probability of detecting unauthorized observers, line-of-sight / distance, etc. Obtained by remapping. Hyperparameters Used to ensure that the threshold adjustment range is within a safe range (e.g.) ).

[0147] To remain robust during interface switching, resolution changes, or abrupt changes in content theme, the system continuously performs distribution drift detection. The reference distribution is defined as the histogram probability of historical long windows / exponential sliding. The current window corresponds to the following distribution: Statistical distance is used:

[0148] ;

[0149] when A significant drift is considered to have occurred if the threshold is set as the drift threshold. In this case, the threshold stringency is increased, for example, by applying adaptive gains to the dispersion coefficient and quantile level.

[0150] ,

[0151] in For drift gain, (e.g., 0.99). To control alarm frequency over the long term, online calibration is also introduced: using the target alarm rate... To constrain this, the actual alarm rate near the window is statistically analyzed. Set a logarithmic domain bias for the threshold. Update in small increments:

[0152] ;

[0153] in For learning rate, This is the dynamic threshold used for final comparison. When the actual alarm value is higher than the target ( )hour, Raising the threshold increases the overall level; conversely, lowering it decreases it, thus maintaining a stable and controllable triggering frequency under different content and viewing conditions. Through the combined effects of robust statistics and quantile control, contextual multiplicative adjustment, drift perception, and online calibration, the system can effectively suppress false alarms and jitter caused by short-term noise, screen scrolling, and image quality fluctuations while maintaining high recall of truly sensitive areas. This ensures that the threshold determination process is interpretable, auditable, and easily parameterized for deployment.

[0154] In one feasible implementation, step S150 extracts layout style features from the aforementioned sensitive areas and, in conjunction with a generative adversarial network model, generates alternative content unrelated to the original semantics while maintaining style consistency, including:

[0155] S1501. Obtain the font, color, layout, and chart element style characteristics of the above-mentioned sensitive areas;

[0156] S1502. Use the above-mentioned layout style features as conditional inputs to the generative adversarial network model.

[0157] S1503. In the above generative adversarial network model, with the constraint of maintaining consistent layout style and the optimization objective of minimizing mutual information with the original content semantics, alternative content unrelated to the original semantics is generated. The alternative content includes grammatically correct alternative text content that is unrelated to the original semantics, or chart content that is consistent with the original chart layout but whose data relevance is less than a preset threshold.

[0158] For example, the system first extracts the layout style features of sensitive areas in stage S1501, including font and text style (such as font family, font size, font weight, line spacing, character spacing, etc.), color and color scheme (such as primary color, secondary color, background color, and palette histogram), layout (including the relative position, size ratio, white space, alignment, etc. of text blocks and chart elements), and chart drawing style (such as axis style, line type, bar width, grid line style, and legend). These features are then uniformly encoded to form a style code. This is provided as a conditional input to the subsequent generative adversarial network.

[0159] In stage S1502, the system employs a conditional GAN ​​structure for style-controlled generation. Generator Receive random noise With style code Output candidate replacement content Discriminator The system determines whether the input sample meets the style criteria. Through adversarial training with an adversarial loss mechanism, the generator continuously improves the style realism of its generated results. Taking Hinge-GAN as an example, the discriminator loss is...

[0160]

[0161] The generator loss is

[0162] ;

[0163] In the S1503 phase, the system introduces two key constraints: one is style consistency constraint, which utilizes a pre-trained feature extractor. Extract perceptual style features and calculate the differences between the original content and the generated content in the Gram matrix and color histogram. The loss function can be written as:

[0164] ;

[0165] in Represents the channel correlation matrix. Indicates color distribution. The system maintains weights for color matching. Optionally, it can also use edge or saliency operators. Compare the layout and edge structure of the generated image with the original image to add layout preservation items:

[0166] ;

[0167] Second, there is the constraint of minimizing semantic mutual information; the system uses a semantic encoder. Extract semantic vectors from the original content and the generated content Mutual information is calculated using InfoNCE approximation, with the following formula:

[0168] ;

[0169] in For batch size, For cosine similarity, This is a temperature parameter. This loss term reduces the pairing between the original and generated content, thereby achieving semantic decoupling.

[0170] Building upon this, the system also incorporates task-oriented branches: for text-sensitive content, a text generator generates grammatically correct but semantically unrelated alternative text based on the original text's syntactic template, while simultaneously constraining the perplexity level (PPL) to not exceed a threshold. The system uses Natural Language Inference (NLI) to ensure that the original text and the generated text are non-implied or contradictory. For sensitive content such as charts, the system uses perturbed data while maintaining the chart style (axis, grid, color scheme, etc.). Generate alternative charts and ensure the correlation coefficient between the new and old data through correlation constraints. Less than the preset threshold At the same time, maintain the coordinate axis range and scale density as the original Figure 1 To achieve this, if applied to video streaming scenarios, the system can also introduce temporal consistency constraints, that is, use optical flow alignment operations to register the content generated by adjacent frames, thereby reducing flicker and jitter. The formula is:

[0171] ;

[0172] in This is an optical flow transformation. Finally, the generator's total loss function is:

[0173] ;

[0174] Among them each These are weighting coefficients used to balance the contributions of different constraint terms. They are used in the discriminator minimization process. At the same time, the generator minimizes the above-mentioned overall loss, thereby weakening the semantic connection with the original content to the greatest extent while ensuring visual style consistency, and generating semantically irrelevant alternative text or charts.

[0175] The method provided in this embodiment enables the present invention to output anonymized alternative text or charts in real time while keeping the font, color scheme, layout and chart style unchanged in sensitive areas. This achieves seamless coverage and leakage interference of sensitive information, effectively balancing user experience and information security.

[0176] Secondly, this invention also proposes a data leakage interference system based on real-time monitoring information, such as... Figure 6 As shown, it includes:

[0177] The first acquisition unit 21 is used to acquire pixel stream data of the target display screen;

[0178] The second acquisition unit 22 is used to perform layout detection, text recognition and chart parsing operations on the above pixel stream data to obtain candidate display areas.

[0179] The third acquisition unit 23 is used to perform sensitivity determination based on the rule matching mechanism and the target deep learning model in order to obtain the risk score of the above candidate display area.

[0180] The determination unit 24 is used to determine the candidate display areas whose risk scores exceed the dynamic threshold as sensitive areas;

[0181] The generation unit 25 is used to extract layout style features from the above-mentioned sensitive areas and, in conjunction with the generative adversarial network model, generate alternative content that is unrelated to the original semantics while maintaining style consistency.

[0182] Replacement unit 26 is used to render the above-mentioned replacement content to the corresponding sensitive area in real time based on optical flow alignment and timing consistency constraints, so as to replace the original content and output it to the target display screen.

[0183] In one feasible implementation, a leak-causing system based on real-time monitoring information can also perform any step of the method proposed in the first aspect.

[0184] Thirdly, the present invention also proposes an electronic device 300, such as... Figure 7 As shown, it includes a memory 310, a processor 320, and a computer program 311 stored on the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements the steps of the leakage interference method based on real-time monitoring information as described in any of the first aspects.

[0185] Fourthly, the present invention also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the leakage interference method based on real-time monitoring information as described in any one of the first aspects.

[0186] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0187] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0188] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0189] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0190] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0191] This application also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device performs the voice-based identity recognition process in the corresponding embodiment.

[0192] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0193] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0195] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0197] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0198] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for compromising interference based on real-time monitoring information, characterized in that, include: Acquire pixel stream data of the target display screen; Perform layout detection, text recognition, and chart parsing operations on the pixel stream data to obtain candidate display areas; Sensitivity determination is performed based on a rule matching mechanism and a target deep learning model to obtain the risk score of the candidate display area; Candidate display areas whose risk scores exceed the dynamic threshold are identified as sensitive areas; The layout style features of the sensitive areas are extracted, and a generative adversarial network model is used to generate alternative content that is unrelated to the original semantics while maintaining style consistency. Based on optical flow alignment and temporal consistency constraints, the replacement content is rendered in real time to the corresponding sensitive area to replace the original content and output to the target display screen; The specific steps for determining the dynamic threshold include: The robust central value and dispersion are calculated based on the risk scores of the candidate display areas within the sliding window, and the basic threshold is obtained by combining quantile estimation. The basic threshold, application scenario factor, modal factor, and user viewing factor are combined to obtain a context-adjusted dynamic threshold. When a drift in the risk score distribution is detected, the stringency of the dynamic threshold is increased, and the dynamic threshold is dynamically adjusted according to the target alarm rate through an online calibration mechanism; The step of extracting layout style features from the sensitive areas and generating alternative content unrelated to the original semantics using a generative adversarial network model while maintaining style consistency includes: Obtain the font, color, layout, and chart element style characteristics of the sensitive area; The layout style features are used as conditional inputs to a generative adversarial network model; In the generative adversarial network model, maintaining consistent layout style is a constraint, and minimizing mutual information with the original content semantics is an optimization objective. Alternative content unrelated to the original semantics is generated. The alternative content includes grammatically correct alternative text content that is unrelated to the original semantics, or chart content that is consistent with the original chart layout but whose data relevance is less than a preset threshold.

2. The leakage interference method based on real-time monitoring information according to claim 1, characterized in that, The sensitivity determination based on the rule matching mechanism and the target deep learning model to obtain the risk score of the candidate display region includes: Based on a preset rule base, regular expression matching and pattern validation are performed on the text content of the candidate display area to identify content that conforms to a predefined sensitive pattern and obtain the rule hit rate. The predefined sensitive pattern includes keywords, phrases, data formats and contextual structures. A semantic classification network based on a pre-trained language model identifies the text semantics of the candidate display area to obtain the text sensitivity probability. An image recognition model based on a combination of convolutional neural networks and attention mechanisms analyzes the charts and / or image content of candidate display areas to obtain the chart sensitivity probability. The risk score is output by weighting and fusing the rule hit rate, the text sensitivity probability, the chart sensitivity probability, and the application context factor to which the candidate display area belongs.

3. The leakage interference method based on real-time monitoring information according to claim 2, characterized in that, The image recognition model includes: A convolutional backbone and a multi-scale feature pyramid are used to extract visual feature representations of the candidate display regions. A layout markup encoder is used to construct a layout feature representation by embedding the appearance, geometric position encoding, and element type embedding of the visual feature representation; A cross-modal attention module is used to fuse the text token obtained by optical character recognition and the layout feature representation to generate a semantically enhanced layout feature representation; The temporal attention module is used to temporally fuse the semantically enhanced layout feature representation of the current frame and the semantically enhanced layout feature representation of the historical frames aligned with optical flow to reduce jitter and obtain a temporally robust graph feature representation. The sensitive prototype matching unit is used to perform similarity calculation between the time-robust chart feature representation and the preset sensitive prototype library to obtain the chart sensitivity probability of the candidate display area.

4. The leakage interference method based on real-time monitoring information according to claim 3, characterized in that, The step of constructing a layout feature representation by embedding the appearance representation, geometric position encoding, and element type embedding of the visual feature representation includes: Locate graph components on multi-scale visual feature representations to obtain geometric candidates; Perform the RoIAlign operation on each of the geometry candidates to extract the appearance embedding vector; The center coordinates, width, height and orientation angle of the candidates are normalized and mapped to geometric position encoding vectors by sine and cosine position encoding and multilayer perceptron; Learnable element type embedding vectors are obtained by looking up a table based on the component category; The appearance embedding vector, the geometric position encoding vector, and the element type embedding vector are concatenated and linearly projected and normalized to obtain a layout feature representation with a unified dimension, which is then assembled into the layout feature representation according to a preset spatial or reading order.

5. The leakage interference method based on real-time monitoring information according to claim 3, characterized in that, The steps of generating semantically enhanced layout feature representations by the cross-modal attention module include: Optical character recognition is performed on the text within the candidate display area to obtain text tokens, and the corresponding semantic vectors are obtained through a text encoder; The semantic vector is used as the key and value, and the layout feature representation is used as the query. The result is input into a multi-head attention network to calculate the semantic association weight between each layout element and the text token. The semantic vector is weighted and summed based on the semantic association weights and then fused with the corresponding layout feature representation to output the semantically enhanced layout feature representation.

6. A data leakage interference system based on real-time monitoring information, characterized in that, include: The first acquisition unit is used to acquire pixel stream data of the target display screen; The second acquisition unit is used to perform layout detection, text recognition and chart parsing operations on the pixel stream data to obtain candidate display areas. The third acquisition unit is used to perform sensitivity determination based on the rule matching mechanism and the target deep learning model in order to obtain the risk score of the candidate display area. A determination unit is used to determine candidate display areas whose risk scores exceed a dynamic threshold as sensitive areas; The generation unit is used to extract layout style features from the sensitive area and, in conjunction with a generative adversarial network model, generate alternative content that is unrelated to the original semantics while maintaining style consistency. The replacement unit is used to render the replacement content to the corresponding sensitive area in real time based on optical flow alignment and temporal consistency constraints, so as to replace the original content and output it to the target display screen; The specific steps for determining the dynamic threshold include: The robust central value and dispersion are calculated based on the risk scores of the candidate display areas within the sliding window, and the basic threshold is obtained by combining quantile estimation. The basic threshold, application scenario factor, modal factor, and user viewing factor are combined to obtain a context-adjusted dynamic threshold. When a drift in the risk score distribution is detected, the stringency of the dynamic threshold is increased, and the dynamic threshold is dynamically adjusted according to the target alarm rate through an online calibration mechanism; The step of extracting layout style features from the sensitive areas and generating alternative content unrelated to the original semantics using a generative adversarial network model while maintaining style consistency includes: Obtain the font, color, layout, and chart element style characteristics of the sensitive area; The layout style features are used as conditional inputs to a generative adversarial network model; In the generative adversarial network model, maintaining consistent layout style is a constraint, and minimizing mutual information with the original content semantics is an optimization objective. Alternative content unrelated to the original semantics is generated. The alternative content includes grammatically correct alternative text content that is unrelated to the original semantics, or chart content that is consistent with the original chart layout but whose data relevance is less than a preset threshold.

7. An electronic device, comprising: The memory and processor are characterized in that, when the processor executes a computer program stored in the memory, it implements the steps of the leakage interference method based on real-time monitoring information as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the leakage interference method based on real-time monitoring information as described in any one of claims 1-5.

Images