Sensitive information leakage early warning method and device for bastion host and storage medium

Abstract

The invention discloses a sensitive information leakage early warning method and device for a bastion host and a storage medium, and belongs to the field of data security. The method comprises the steps that multi-modal data monitored by a bastion host are acquired, and the multi-modal data comprise operation and maintenance video data, system log data and network flow data; based on a pre-constructed deep learning model, performing feature layer weighted fusion on the multi-modal data to obtain a fusion feature vector which corresponds to the multi-modal data and is strongly associated with the sensitive information; based on a pre-constructed sensitive information rule base, matching the fusion feature vectors to identify target sensitive information in the multi-modal data; and triggering an early warning mechanism under the condition of detecting that the target sensitive information has a leakage sign. The leakage risk of sensitive information can be reduced.

Description

Technical Field

[0001] This invention relates to the field of data security, and more specifically to a method, device, and storage medium for early warning of sensitive information leakage in bastion hosts. Background Technology

[0002] In the fields of modern information technology and cybersecurity, data breaches and information security issues are becoming increasingly serious. Especially during enterprise operations and maintenance (O&M), the leakage of sensitive information can lead to severe economic losses and reputational damage. To prevent the leakage of sensitive information, many enterprises have adopted bastion hosts for auditing and monitoring O&M operations. As a security auditing tool, bastion hosts can record the entire O&M process and provide detailed audit logs. However, automatically identifying sensitive information from massive amounts of O&M data is a challenge; the accuracy of sensitive information identification is not high, leading to a high risk of sensitive information leakage. Summary of the Invention

[0003] The purpose of this invention is to provide a method, device, storage medium, and computer program product for early warning of sensitive information leakage in bastion hosts, so as to solve the problem of high risk of sensitive information leakage in the prior art.

[0004] To achieve the above objectives, the first aspect of this application provides a method for early warning of sensitive information leakage for a bastion host, comprising: Acquire multimodal data from bastion host monitoring, including operation and maintenance video data, system log data, and network traffic data; Based on a pre-built deep learning model, multimodal data is subjected to feature layer weighted fusion to obtain a fused feature vector that is strongly correlated with sensitive information corresponding to the multimodal data. Based on a pre-built sensitive information rule base, the fused feature vectors are matched to identify target sensitive information in multimodal data; An early warning mechanism is triggered if any signs of leakage of sensitive target information are detected.

[0005] In this embodiment of the application, after acquiring the multimodal data of the bastion host monitoring, the method further includes data preprocessing of the multimodal data; data preprocessing of the multimodal data includes: converting the operation and maintenance video data into operation and maintenance image frame data, associating the timestamp of the operation and maintenance video data with the system log data, and establishing a mapping relationship between the operation and maintenance image frame data and the operation and maintenance steps; and performing data cleaning and formatting processing on the system log data and network traffic data.

[0006] In this embodiment of the application, the acquisition of operation and maintenance video data includes: capturing the operation and maintenance trajectory of the monitoring video of the bastion host based on a pre-determined video directional capture rule adapted to different types of protocols, so as to obtain operation and maintenance video data, wherein the protocols include character protocols, graphics protocols and file transfer protocols.

[0007] In this embodiment of the application, the sensitive information leakage early warning method for bastion hosts further includes: using a preset screen area directional capture algorithm and a preset encoding compression algorithm to perform lightweight processing on the operation and maintenance video data in order to achieve lightweight transmission.

[0008] In this embodiment of the application, converting operation and maintenance video data into operation and maintenance image frame data includes: extracting character-type protocol image frames, graphic protocol image frames, and file transfer protocol image frames from the operation and maintenance video data at different extraction frequencies based on predetermined target extraction frequencies corresponding to different types of operation and maintenance protocol image data, so as to obtain operation and maintenance image frame data.

[0009] In this embodiment of the application, the method further includes at least one of the following steps: performing adaptive threshold binarization on the character protocol image frame; performing Gaussian filtering noise reduction and / or edge enhancement on the graphics protocol image frame; and performing target region cropping on the file transfer protocol image frame.

[0010] In this embodiment, based on a pre-built deep learning model, multimodal data is subjected to feature layer weighted fusion to obtain a fused feature vector strongly correlated with sensitive information corresponding to the multimodal data. This includes: controlling the timestamp error of the multimodal data within a preset timestamp error range based on a synchronized clock; and assigning high weights to sensitive text in the synchronized timestamped operation and maintenance video data, high-risk commands in the system log data, and abnormal transmissions in the network traffic data based on a pre-built deep learning model employing an attention mechanism, so as to obtain a fused feature vector strongly correlated with sensitive information corresponding to the multimodal data.

[0011] In this embodiment of the application, after identifying the target sensitive information in the multimodal data, the method further includes: determining the corresponding target risk level and the target early warning method corresponding to the target risk level based on the sensitive information type and behavioral context corresponding to the target sensitive information.

[0012] A second aspect of this application provides a computer device, comprising: a memory configured to store instructions; a processor configured to retrieve instructions from the memory and, when executing the instructions, to implement the aforementioned sensitive information leakage early warning method for a bastion host.

[0013] A third aspect of this application provides a machine-readable storage medium storing instructions that cause a machine to execute the aforementioned sensitive information leakage early warning method for a bastion host.

[0014] The fourth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned sensitive information leakage early warning method for a bastion host.

[0015] The aforementioned technical solution acquires multimodal data from bastion host monitoring, including operational video data, system log data, and network traffic data. Based on a pre-built deep learning model, the multimodal data is input into the deep learning model, which performs weighted feature fusion to obtain fused feature vectors strongly correlated with sensitive information. Then, based on a pre-built sensitive information rule base, an early warning mechanism is triggered when signs of sensitive information leakage are detected. This technical solution comprehensively analyzes the operational video data, system log data, and network traffic data monitored by the bastion host using a deep learning model, and combines this with a sensitive information rule base for matching and identification of sensitive information. This allows for accurate identification of sensitive information in operational data, issuing early warnings when signs of sensitive information leakage are detected, and precisely identifying potential leakage behaviors of sensitive information, thereby reducing the risk of sensitive information leakage.

[0016] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 The illustration shows a flowchart of a sensitive information leakage early warning method for a bastion host according to an embodiment of this application; Figure 2 The illustration schematically shows a framework diagram of an application system for a sensitive information leakage early warning method for a bastion host according to an embodiment of this application. Detailed Implementation

[0018] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0019] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0020] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0021] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0022] Figure 1 This illustration schematically shows a flowchart of a sensitive information leakage early warning method for a bastion host according to an embodiment of this application. Figure 1 As shown in the figure, this application provides a method for early warning of sensitive information leakage for a bastion host. Taking the application of this method to a processor as an example, the method may include the following steps: Step S102: Obtain multimodal data from the bastion host monitoring, including operation and maintenance video data, system log data, and network traffic data.

[0023] Step S104: Based on the pre-built deep learning model, perform feature layer weighted fusion on the multimodal data to obtain the fused feature vectors that are strongly correlated with sensitive information corresponding to the multimodal data.

[0024] Step S106: Based on the pre-built sensitive information rule base, the fused feature vector is matched to identify target sensitive information in multimodal data.

[0025] Step S108: If signs of leakage of sensitive target information are detected, an early warning mechanism is triggered.

[0026] It is understandable that a bastion host can monitor the entire operation and maintenance process, meaning it can record all operations in a video format to obtain a video stream, i.e., operation and maintenance video data. Simultaneously, the bastion host can also monitor and collect system log data and network traffic data related to the operation. System log data may include command execution records, account permission changes, login IPs, etc., while network traffic data may include transmission protocols, packet sizes, etc. A deep learning model is used to obtain a fused feature vector that strongly correlates with sensitive information based on multimodal data. The fused feature vector is a feature vector that integrates features from multimodal data that have a high correlation with sensitive information. The sensitive information rule base stores sensitive information related to specific rules in the bastion host operation and maintenance scenario. The construction of the sensitive information rule base can cover the following four categories of bastion host operation and maintenance scenario-specific rules: 1) Keyword rules: high-frequency operation and maintenance terms (such as password, root); 2) Regular expression rules: matching formats such as account, IP, and key; 3) Behavioral rules: such as "3 incorrect password entries within five minutes"; 4) Permission rules: such as "non-administrators use sudo to execute commands". The target sensitive information is the sensitive information in the sensitive information rule base that matches the fused feature vector.

[0027] Specifically, the processor can acquire multimodal data from bastion host monitoring, including operational video data, system log data, and network traffic data. Then, based on a pre-built deep learning model, it performs feature-layer weighted fusion of the multimodal data. For example, it can assign high weights to sensitive text such as passwords in operational video data, high-risk commands in system log data, and abnormal transmission data in network traffic data, thereby obtaining a fused feature vector output by the deep learning model that is strongly correlated with sensitive information. Furthermore, the processor can match the fused feature vector based on a pre-built sensitive information rule base to identify target sensitive information in the multimodal data. If any signs of leakage of target sensitive information are detected, the processor can immediately trigger an early warning mechanism.

[0028] The aforementioned technical solution acquires multimodal data from bastion host monitoring, including operational video data, system log data, and network traffic data. Based on a pre-built deep learning model, the multimodal data is input into the deep learning model, which performs weighted feature fusion to obtain fused feature vectors strongly correlated with sensitive information. Then, based on a pre-built sensitive information rule base, an early warning mechanism is triggered when signs of sensitive information leakage are detected. This technical solution comprehensively analyzes the operational video data, system log data, and network traffic data monitored by the bastion host using a deep learning model, and combines this with a sensitive information rule base for matching and identification of sensitive information. This allows for accurate identification of sensitive information in operational data, issuing early warnings when signs of sensitive information leakage are detected, and precisely identifying potential leakage behaviors of sensitive information, thereby reducing the risk of sensitive information leakage.

[0029] In one embodiment, after acquiring the multimodal data monitored by the bastion host, the method further includes data preprocessing of the multimodal data; data preprocessing of the multimodal data includes: converting the operation and maintenance video data into operation and maintenance image frame data, associating the timestamp of the operation and maintenance video data with the system log data, establishing a mapping relationship between the operation and maintenance image frame data and the operation and maintenance steps; and performing data cleaning and formatting processing on the system log data and network traffic data.

[0030] It is understandable that after acquiring real-time operation and maintenance video streams, targeted preprocessing transforms the video data into analyzable frame images—that is, converting operation and maintenance video data into operation and maintenance image frame data—provides high-quality input for feature analysis of deep learning models, facilitating information analysis by these models. By associating the timestamps in the video stream with the bastion host system logs, a mapping relationship between "frame images and operation and maintenance steps" can be established, ensuring the temporal accuracy of subsequent analysis.

[0031] Specifically, after acquiring the multimodal data monitored by the bastion host, the processor can perform data preprocessing on the multimodal data. The specific process of data preprocessing may include: converting the operation and maintenance video data into operation and maintenance image frame data, associating the timestamp of the operation and maintenance video data with the system log data, and establishing a mapping relationship between the operation and maintenance image frame data and the operation and maintenance steps; and performing data cleaning and formatting on the system log data and network traffic data.

[0032] In one embodiment, the acquisition of operation and maintenance video data includes: capturing the operation and maintenance trajectory of the monitoring video of the bastion host based on pre-determined video-oriented capture rules adapted to different types of protocols, so as to obtain operation and maintenance video data, wherein the protocols include character protocols, graphics protocols and file transfer protocols.

[0033] It is understandable that corresponding video capture rules can be set for common bastion host protocols such as character-based protocols (SSH / TELNET), graphical protocols (RDP / VNC), and file transfer protocols (SFTP / SCP / FTP). For example, the character-based interface can focus on recording the command input area (such as the terminal's command line window) and the output feedback area to ensure complete recording of the command execution process; the graphical interface can accurately locate sensitive control areas (such as password input boxes and configuration file editing windows) to avoid interference from irrelevant interfaces; and the file transfer interface can focus on key information such as the file list, transfer path, and progress bar to achieve complete capture of the operation and maintenance trajectory.

[0034] Specifically, based on pre-determined video-oriented capture rules adapted to different types of protocols (character protocols, graphics protocols, and file transfer protocols), the monitoring video of the bastion host is captured to obtain the operation and maintenance video data.

[0035] In one embodiment, the above-mentioned sensitive information leakage early warning method for bastion hosts may further include: using a preset screen area directional capture algorithm and a preset encoding compression algorithm to perform lightweight processing on operation and maintenance video data to achieve lightweight transmission.

[0036] It is understandable that the preset screen area directional cropping algorithm is a pre-set algorithm used to achieve screen area directional cropping, and the preset encoding compression algorithm is a pre-set algorithm used to achieve video compression.

[0037] Specifically, the processor can employ preset screen region directional cropping algorithms and preset encoding compression algorithms to perform lightweight processing on the maintenance video data, thereby achieving lightweight transmission. For example, the Python MSS library can be used to implement screen region directional cropping (supporting mainstream bastion host resolutions such as 1920×1082 / 2K), combined with OpenCV's H.264 encoding compression algorithm, to control the video stream latency to within 500ms, ensuring the temporal consistency between maintenance operations and video recording. Furthermore, in some embodiments, timestamps and protocol type tags (e.g., SSH-2025-08-21-00-00-00) can be embedded in the video stream to provide basic identifiers for subsequent multimodal data alignment.

[0038] In one embodiment, converting maintenance video data into maintenance image frame data includes: extracting character-type protocol image frames, image frames of graphic protocols, and image frames of file transfer protocols from the maintenance video data at different extraction frequencies based on predetermined target extraction frequencies corresponding to different types of protocols, to obtain maintenance image frame data.

[0039] It is understandable that the protocols include graphics protocols and file transfer protocols. The target extraction frequency is a pre-set extraction frequency for maintenance video data of different types of protocols. Understandably, the frequency of primary and secondary character input is approximately 100-300 milliseconds, and the completeness of the command directly determines the accuracy of sensitive information identification, so one frame can be extracted every 200 milliseconds. Graphical protocols: Operations such as mouse clicks and window switching are relatively slow, and important information mainly comes from the result of the operation (password input box, etc.). The interval from triggering to completion is usually 300-1000 milliseconds, so one frame can be extracted every 500 milliseconds. File transfer protocols: The focus of file transfer protocols is on file attributes (filename, path, size) and transmission status, usually at intervals of 1-3 seconds, so one frame can be extracted every 1 second. By processing video data according to different protocols, the generation of redundant data can be significantly reduced while retaining key information. Character protocol image frames are frame images of character-based protocols, graphic protocol image frames are frame images of graphic protocols, and file transfer protocol image frames are frame images of file transfer protocols.

[0040] Specifically, the processor can extract character-type protocol image frames, image frame frames, and file transfer protocol image frames from the operation and maintenance video data at different extraction frequencies based on the predetermined target extraction frequencies corresponding to different types of operation and maintenance protocol image data, so as to obtain operation and maintenance image frame data.

[0041] In one embodiment, the above method may further include at least one of the following steps: adaptive threshold binarization of character protocol image frames; Gaussian filtering for noise reduction and / or edge enhancement of graphics protocol image frames; and target region cropping of file transfer protocol image frames.

[0042] It's understandable that for frame images of character-based protocols, adaptive threshold binarization enhances the contrast between command text and background, resolving issues like blurry fonts and color interference in terminal windows. For frame images of graphics protocols, Gaussian filtering for noise reduction and / or edge enhancement can highlight the outlines of sensitive controls such as password boxes and configuration buttons. For frame images of file transfer protocols, cropping the target region facilitates information analysis by deep learning models, for example, retaining only core information like filenames and paths while removing irrelevant areas.

[0043] In one embodiment, based on a pre-built deep learning model, multimodal data is subjected to feature layer weighted fusion to obtain a fused feature vector strongly correlated with sensitive information corresponding to the multimodal data. This includes: controlling the timestamp error of the multimodal data within a preset timestamp error range based on a synchronized clock; and assigning high weights to sensitive text in the synchronized timestamped operation and maintenance video data, high-risk commands in the system log data, and abnormal transmissions in the network traffic data based on a pre-built deep learning model employing an attention mechanism, so as to obtain a fused feature vector strongly correlated with sensitive information corresponding to the multimodal data.

[0044] It is understandable that the preset timestamp error range is the pre-defined error range between the timestamps of maintenance video data, system log data, and network traffic data, for example, 100ms. The synchronization clock is used to achieve time synchronization or near-synchronization.

[0045] Specifically, the processor can construct an "operation behavior timeline" by controlling the timestamp errors of the bastion host-monitored operation and maintenance video data, system log data, and network traffic data within a preset timestamp error range (e.g., 100ms) based on a synchronous clock (e.g., an NTP synchronous clock). Furthermore, based on a deep learning model employing an attention mechanism, it achieves weighted fusion of feature layers for multimodal data, assigning high weights to sensitive text (such as passwords) in the operation and maintenance video data, high-risk commands in the system log data, and abnormal transmissions in the network traffic data. For example, it can output a fused feature vector with a dimension of 512, strengthening the correlation of sensitive information.

[0046] In one embodiment, after identifying target sensitive information in multimodal data, the method further includes: determining the corresponding target risk level and the target early warning method corresponding to the target risk level based on the sensitive information type and behavioral context corresponding to the target sensitive information.

[0047] Specifically, the processor can determine the corresponding target risk level and the corresponding warning method based on the sensitive information type and behavioral context of the target sensitive information. For example, if the target risk level is high-risk (such as plaintext password transmission or transmission of sensitive data to overseas IPs), the warning method can be a triple notification via SMS, pop-up window, and email; if the target risk level is medium-risk (such as privilege escalation commands), the warning method can be through system logs and in-application notifications; if the target risk level is low-risk (such as a single password error), the warning method can be recording audit logs. Furthermore, it can simultaneously support linkage with a bastion host, and high-risk operations can trigger session blocking.

[0048] Traditional bastion host auditing methods mainly rely on manual auditing and single-modal data analysis, which suffers from low efficiency and insufficient accuracy. Furthermore, existing technologies struggle to achieve real-time monitoring and rapid response to sensitive information leaks. In addition, automatically identifying and tagging sensitive information from massive amounts of operational data is a challenge.

[0049] In one specific embodiment, a method for early warning of sensitive information leakage for bastion hosts is provided, applied to... Figure 2 In the system architecture shown, this embodiment addresses the core requirement of sensitive information leakage detection in bastion host auditing scenarios. It constructs a real-time automatic analysis and early warning system through scenario-specific adaptation of multimodal data fusion and deep learning technology. The system architecture focuses on capturing sensitive information across the entire operation and maintenance chain, and the core module design can be as follows: 1) Real-time video capture module: This module serves as the source of system data input. Targeting the "multi-protocol operation and maintenance, high interaction frequency" scenario characteristics of bastion hosts, it achieves seamless capture and real-time transmission of operation and maintenance operations through capture adaptation of different protocols and lightweight real-time transmission.

[0050] a. Protocol Adaptation and Capture: Targeted capture rules are designed for common bastion host protocols such as character protocols (SSH / TELNET), graphical protocols (RDP / VNC), and file transfer protocols (SFTP / SCP / FTP). Character interfaces focus on recording the command input area (such as the terminal's command line window) and output feedback area to ensure complete recording of the command execution process. Graphical interfaces accurately locate sensitive control areas (such as password input boxes and configuration file editing windows) to avoid interference from irrelevant interfaces. File transfer interfaces focus on key information such as file lists, transfer paths, and progress bars to achieve complete capture of the operation and maintenance trajectory.

[0051] b. Lightweight Real-Time Transmission: Utilizing Python's MSS library for targeted screen region capture (supporting mainstream bastion host resolutions such as 1920×1082 / 2K), combined with OpenCV's H.264 encoding and compression algorithm, video stream latency is controlled within 500ms, ensuring temporal consistency between maintenance operations and video recording. Simultaneously, timestamps and protocol type tags (e.g., SSH-2025-08-21-00-00-00) are embedded in the video stream, providing basic identifiers for subsequent multimodal data alignment.

[0052] Output: This module ultimately generates a real-time video stream containing the complete operation and maintenance trajectory, which serves as the input data source for the frame processing module.

[0053] 2) Frame processing module: After acquiring the real-time operation and maintenance video stream, this module converts the video data into analyzable frame images through targeted preprocessing, providing high-quality input for deep learning analysis.

[0054] a. Contextualized Frame Extraction: The extraction frequency is dynamically adjusted according to different types of protocols. Character-based protocols: The frequency of primary and secondary character input is approximately 100-300 milliseconds, and the completeness of the command directly determines the accuracy of sensitive information identification; therefore, one frame is extracted every 200 milliseconds. Graphical protocols: Operations such as mouse clicks and window switching are relatively slow, and important information mainly comes from the result of the operation (password input box, etc.). The interval from triggering to completion is usually 300-1000 milliseconds; therefore, one frame is extracted every 500 milliseconds. File transfer protocols: The focus of file transfer protocols is on file attributes (filename, path, size) and transfer status, usually at intervals of 1-3 seconds; therefore, one frame is extracted every 1 second. By processing different protocols separately, the generation of redundant data can be significantly reduced while retaining key information.

[0055] b. Targeted Preprocessing: This operation primarily aims to facilitate information analysis of the extracted frame images. For frame images of character-based protocols, adaptive threshold binarization is used to enhance the contrast between command text and background, resolving issues such as blurry fonts and color interference in terminal windows. For frame images of graphics protocols, Gaussian filtering and edge enhancement are applied to highlight the outline features of sensitive controls such as password boxes and configuration buttons. For frame images of file transfer protocols, region cropping is performed, retaining only core information such as file names and paths while removing irrelevant areas.

[0056] c. By associating the timestamps in the video stream with the bastion host system logs, a mapping relationship of "frame image - operation and maintenance steps" is established to ensure the timing accuracy of subsequent analysis.

[0057] Output: Generate a preprocessed dataset of frame images, which is stored according to the operation scenario and used as input for the deep analysis module.

[0058] 3) Deep Learning Analysis Module: This module mainly extracts sensitive information features from frame images through customized deep learning models, providing a basis for subsequent fusion analysis.

[0059] a. Customized Deep Learning Models: For frame image datasets of character-based protocols, a combined CNN+CRNN model is employed. CNN (lightweight ResNet-18) is used to extract visual features of command text regions (such as character edges and font structure), while CRNN is used to handle sequence dependency issues in command text (such as command syntax order and long text coherence). The combination of the two accurately identifies command text (e.g., sudo). For frame images of image-based protocols, a YOLOv8 model is used, with eight customized object detection tasks for the bastion host graphical interface (password input box, sensitive information icon, permission operation, database access, file transfer, command input terminal window, network configuration, and sensitive information display). Recognition accuracy is improved by optimizing anchor box size and non-maximum suppression. For frame images of file transfer protocols, a lightweight VIT model is used, capturing global features of file icons and name formats (such as the "lock icon + .pem suffix" feature of key files) through image patch embedding and self-attention mechanisms.

[0060] Output: Each image frame outputs three types of structured features: text features, including the command text sequence identified in the character interface frame (e.g., ls -l test), and the filename and path extracted from the file transfer frame; target features, including the coordinates and type labels of sensitive controls in the graphical protocol interface frame (e.g., "High-risk control - password input box"), supporting spatial positioning of operation behaviors; and semantic features, including operation intent labels generated based on image content (e.g., privilege escalation operation, sensitive file access), providing semantic guidance for multimodal fusion.

[0061] 4) Multimodal data fusion module: To overcome the limitations of analyzing single-modal data, this module integrates multi-source data such as image frame features, system logs, and network traffic, and achieves comprehensive verification of sensitive information through deep fusion.

[0062] a. Multimodal data sources: image frames extracted by the deep learning analysis module, bastion host system logs (including command execution records, account permission changes, login IPs, etc.), and network traffic data (including transmission protocols, data packet sizes, etc.).

[0063] b. Multimodal data processing: Based on the NTP synchronization clock, the timestamp error of the above three types of data is controlled within 100ms to construct an "operation behavior timeline"; an attention mechanism is used to achieve feature layer weighted fusion - sensitive text (such as passwords) in the video, high-risk commands in the logs, and abnormal transmissions in the traffic are given high weights, and a fused feature vector with a dimension of 512 is output to strengthen the correlation of sensitive information.

[0064] Output: Generates a fused feature vector containing multi-source verification information, providing a basis for comprehensive analysis for the sensitive information identification module.

[0065] 5) Sensitive Information Identification Module: As the core judgment link of the system, this module is based on the sensitive information characteristics of the bastion host and achieves accurate identification and marking of sensitive information through "rule base + multi-source verification".

[0066] a. Sensitive Information Rule Base Construction: Covering four types of bastion host operation and maintenance scenarios with specific rules. Keyword Rules: High-frequency operation and maintenance terms (such as password, root). Regular Expression Rules: Matching formats such as account, IP, and key. Behavioral Rules: Such as "3 incorrect password entries within 5 minutes". Permission Rules: Such as "Non-administrators can use sudo to execute commands".

[0067] b. Multi-source cross-validation: Cross-compare the text recognized from the image frame with the command parameters in the log (e.g., associate the video "password input" behavior with the log "login failed"), and combine this with the transmission object verification of traffic data to reduce the false alarm rate of a single modality.

[0068] Output: The identified sensitive information is labeled with its risk level (low / medium / high), and its location (video frame coordinates), timestamp, associated operator and equipment information are recorded to form a structured audit record, triggering the real-time monitoring and early warning module.

[0069] 6) Real-time monitoring and early warning module: As the final response link of the system, this module realizes real-time early warning and audit closed loop of risks based on the results of sensitive information identification, meeting the compliance audit requirements of the bastion host.

[0070] a. Real-time inference optimization: TensorRT is used to quantize and accelerate deep learning models, ensuring that the inference time per frame is ≤20ms, supporting real-time analysis capability of 30 frames per second, and ensuring synchronous monitoring of operation and maintenance.

[0071] b. Dynamic Risk Level Assessment and Warning Triggering: Thresholds are set based on the type of sensitive information and the context of the behavior—high-risk warnings (such as plaintext password transmission, transmission of sensitive data to overseas IPs) are sent via SMS, pop-up window, and email; medium-risk warnings (such as privilege escalation commands) are sent via system logs and in-application notifications; low-risk warnings (such as a single password error) are recorded in the audit log. It also supports integration with bastion hosts, and high-risk operations can trigger session blocking.

[0072] c. Audit closed-loop implementation: All early warning information is stored in association with the original multimodal data, supporting full-link audit of "early warning-source tracing-review" and meeting compliance requirements such as the Information Security Protection 2.0.

[0073] In another specific embodiment, a method for providing early warning of sensitive information leakage for a bastion host is provided, specifically including: Suppose a large financial institution's IT department uses a bastion host audit video real-time automated analysis and sensitive information leakage early warning system based on multimodal data fusion to monitor its data center operations. The following is an example of the system's workflow in a real-world application: 1) System Deployment Financial institutions have deployed bastion hosts in their data centers to record and audit all remote operations by maintenance personnel.

[0074] The bastion host integrates a multimodal data fusion module, including video capture, log recording, and network traffic monitoring functions.

[0075] 2) Real-time video capture When the maintenance personnel begin operation, the bastion host's real-time video capture module starts working, capturing the screen activity of the operation interface and recording it as a video stream.

[0076] 3) Multimodal data collection Meanwhile, the bastion host collects system logs and network traffic data related to operations, which are used as part of multimodal data for subsequent fusion analysis.

[0077] 4) Data preprocessing The captured video stream is processed by the frame processing module and converted into a frame image dataset.

[0078] Log and network traffic data are also cleaned and formatted by the corresponding preprocessing modules.

[0079] 5) Deep learning analysis The deep learning analysis module analyzes the frame image dataset and uses object detection and OCR technology to identify sensitive information on the user interface, such as password input and configuration changes.

[0080] 6) Multimodal data fusion The multimodal data fusion module combines video frames, system logs, and network traffic data, and performs comprehensive analysis through a deep learning model to identify potential sensitive information leakage behaviors.

[0081] 7) Sensitive Information Identification The sensitive information identification module matches and analyzes the received features and key information, and uses NLP technology and rule bases to identify and label sensitive information.

[0082] 8) Real-time monitoring and early warning The real-time monitoring and early warning module continuously monitors the output of the sensitive information identification module. Once it detects any signs of sensitive information leakage, it immediately triggers an early warning mechanism.

[0083] Warning notifications can be sent to the IT security team via email, SMS, or system pop-ups.

[0084] 9) Response and Handling Upon receiving the alert, the IT security team immediately reviewed the relevant audit videos and data to confirm whether any sensitive information had been leaked.

[0085] If a breach is confirmed, the team will take appropriate response measures, such as cutting off suspicious sessions, changing exposed credentials, and launching a more in-depth investigation.

[0086] 10) Audit and Compliance All identified sensitive information and alerts are recorded in the audit log for post-audit and compliance checks.

[0087] In summary, this application's embodiments, targeting the operational characteristics of video data from character protocols, graphics protocols, and file transfer protocols, set video-oriented capture rules (such as command input areas and sensitive control areas) and dynamic frame extraction frequencies adapted to video data from different protocols. Combined with targeted preprocessing methods (such as adaptive threshold binarization of character frames and edge enhancement of sensitive controls in graphics frames), it can achieve comprehensive monitoring and accurate retention of key information in operational operations across all protocols. This solves the problems of missed detection of key information such as sensitive operations in graphical interfaces (such as password input and clicking of sensitive file icons) and video data redundancy in multi-protocol scenarios by traditional single text log analysis. Furthermore, by synchronizing clocks to control the timestamp errors of video frames, system logs, and network traffic data within a small error range, an "operational timeline" is constructed. An attention mechanism is used to assign dynamic weights to sensitive text, high-risk commands, and abnormal traffic features, outputting multi-dimensional fused feature vectors. This achieves semantic association and risk enhancement of cross-modal data, solving the one-sided problem of isolated analysis of multi-source data in bastion hosts.

[0088] This application provides a computer device, including: a memory configured to store instructions; a processor configured to retrieve instructions from the memory and, when executing the instructions, to implement the sensitive information leakage early warning method for a bastion host according to the above embodiments.

[0089] This application also provides a machine-readable storage medium storing instructions that cause a machine to execute the sensitive information leakage early warning method for a bastion host according to the above embodiments.

[0090] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the sensitive information leakage early warning method for a bastion host according to the above embodiments.

[0091] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0094] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0095] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0096] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0097] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0098] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0099] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for early warning of sensitive information leakage for bastion hosts, characterized in that, include: Acquire multimodal data from bastion host monitoring, wherein the multimodal data includes operation and maintenance video data, system log data, and network traffic data; Based on a pre-built deep learning model, the multimodal data is subjected to feature layer weighted fusion to obtain a fused feature vector that is strongly correlated with sensitive information corresponding to the multimodal data. Based on a pre-built sensitive information rule base, the fused feature vector is matched to identify target sensitive information in the multimodal data; If any signs of leakage of the target's sensitive information are detected, an early warning mechanism will be triggered.

2. The method according to claim 1, characterized in that, After acquiring the multimodal data monitored by the bastion host, the process also includes preprocessing the multimodal data. The data preprocessing of the multimodal data includes: The operation and maintenance video data is converted into operation and maintenance image frame data, and the timestamp of the operation and maintenance video data is associated with the system log data to establish a mapping relationship between the operation and maintenance image frame data and the operation and maintenance steps. The system log data and network traffic data are cleaned and formatted.

3. The method according to claim 1, characterized in that, The acquisition of the maintenance video data includes: Based on predetermined video-oriented capture rules adapted to different types of protocols, the monitoring video of the bastion host is captured to obtain the operation and maintenance video data. The protocols include character protocols, graphics protocols, and file transfer protocols.

4. The method according to claim 3, characterized in that, Also includes: The operation and maintenance video data is processed in a lightweight manner by using a preset screen area directional cropping algorithm and a preset encoding compression algorithm to achieve lightweight transmission.

5. The method according to claim 2, characterized in that, The step of converting the maintenance video data into maintenance image frame data includes: Based on the predetermined target extraction frequencies corresponding to different types of operation and maintenance video data, the operation and maintenance video data is subjected to extraction of character-type protocol image frames, graphic protocol image frames, and file transfer protocol image frames at different extraction frequencies to obtain the operation and maintenance image frame data.

6. The method according to claim 5, characterized in that, The method further includes at least one of the following steps: The character protocol image frame is subjected to adaptive threshold binarization processing; The image frames of the graphics protocol are subjected to Gaussian filtering for noise reduction and / or edge enhancement. The target region is cropped from the image frame of the file transfer protocol.

7. The method according to claim 1, characterized in that, The pre-built deep learning model performs feature-layer weighted fusion on the multimodal data to obtain a fused feature vector strongly correlated with sensitive information corresponding to the multimodal data, including: The timestamp error of the multimodal data is controlled within a preset timestamp error range based on a synchronous clock; Based on a pre-built deep learning model employing an attention mechanism, high weights are assigned to sensitive text in the operation and maintenance video data after synchronization timestamps, high-risk commands in the system log data, and abnormal transmissions in the network traffic data, in order to obtain a fusion feature vector that is strongly correlated with sensitive information corresponding to the multimodal data.

8. The method according to claim 1, characterized in that, After identifying the target-sensitive information in the multimodal data, the method further includes: Based on the sensitive information type and behavioral context corresponding to the target sensitive information, the corresponding target risk level and the target early warning method corresponding to the target risk level are determined.

9. A computer device, characterized in that, include: The memory is configured to store instructions; A processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the sensitive information leakage early warning method for a bastion host according to any one of claims 1 to 8.

10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to execute the sensitive information leakage early warning method for a bastion host according to any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the sensitive information leakage early warning method for a bastion host according to any one of claims 1 to 8.

Images