Terminal leak-proof risk quantification method and system based on multi-modal behavior data fusion

Abstract

The application relates to the technical field of the cross of intelligent security and protection, terminal security management and control, password application and access control, and particularly relates to a terminal leakage prevention risk quantification method and system based on multi-modal behavior data fusion. The method comprises the following steps: collecting multi-modal behavior data through a terminal, wherein the multi-modal behavior data comprises a video stream, a content stream and an operating system behavior event stream, and the video stream is preprocessed; sending the preprocessed multi-modal behavior data to an edge computing node; analyzing and fusing the preprocessed multi-modal behavior data at the edge computing node to obtain a risk quantification result; generating a disposal strategy based on the risk quantification result and issuing the disposal strategy to the terminal; and performing dynamic hierarchical encryption on the video stream and / or the content stream based on the disposal strategy and performing attribute-based access control, so that an unalterable evidence chain is constructed, and the end-to-end delay problem is solved.

Description

Technical Field

[0001] This application relates to the technical fields of intelligent security, terminal security management and control, cryptographic application and access control, and in particular to a terminal anti-leakage risk quantification method and system based on multimodal behavioral data fusion. Background Technology

[0002] In the field of information security, physical security and terminal behavior monitoring in confidential locations (such as classified conference rooms, military R&D bases, and core government office areas) are crucial to preventing the leakage of sensitive information through visual channels. Currently, anti-spy camera surveillance and abnormal terminal behavior monitoring mainly rely on centralized content analysis of single video streams captured by terminal cameras. However, with the evolution of attack methods and the increasing demands for security compliance, this approach faces the following problems in practical deployment:

[0003] 1. Algorithm compliance risk: Existing anti-spy camera products generally use international algorithms such as AES and RSA, and the keys are hosted on a central server, which poses risks of centralized attack, single point of failure, and key leakage.

[0004] 2. Limited Evidence Dimensions: Existing solutions rely solely on image content or single-modal sensors, lacking fusion analysis with multiple sources of corroborating evidence such as terminal operation behavior and posture movement. This makes it prone to false alarms and missed alarms, and makes it difficult to confirm a complete chain of evidence for covert filming.

[0005] 3. Evidence credibility and tampering risk: Traditional solutions cannot bind encrypted metadata to real-time video frames at the spatiotemporal level, making it impossible to verify whether the video frames have been tampered with or replaced afterward, resulting in difficulties in post-event verification and evidence collection.

[0006] 4. Insufficient centralized analysis and real-time performance: When the existing centralized detection architecture is deployed in large-scale confidential locations (conference centers, military bases), the network latency is high and the single point of bottleneck is large, making it difficult to support "millisecond-level" response.

[0007] 5. Coarse-grained authorization control: Traditional access control is mostly account or role-level authorization, which makes it difficult to combine dynamic factors such as security level, location, time, risk level, and terminal trust status to carry out fine-grained access control and immediate authorization revocation, and lacks an audit closed loop linked with the chain of evidence. Summary of the Invention

[0008] To address one of the aforementioned technical problems, this application provides a terminal anti-leakage risk quantification method and system based on multimodal behavioral data fusion, thereby constructing an immutable chain of evidence. At the same time, by collaborating with edge computing nodes, the computation is pushed down to the edge, solving the end-to-end latency problem.

[0009] In a first aspect, this application provides a terminal data leakage risk quantification method based on multimodal behavioral data fusion, comprising: collecting multimodal behavioral data through the terminal, the multimodal behavioral data including video stream, content stream, and operating system behavioral event stream, and preprocessing the video stream; sending the preprocessed multimodal behavioral data to an edge computing node; analyzing and fusing the preprocessed multimodal behavioral data at the edge computing node to obtain a risk quantification result; generating a handling strategy based on the risk quantification result and distributing the handling strategy to the terminal; and the terminal dynamically encrypting the video stream and / or content stream according to the handling strategy and performing attribute-based access control.

[0010] Optionally, in some embodiments, the preprocessing of the video stream includes: performing hash calculations on the video frames of the video stream to construct a hash chain; anchoring the digest information of the hash chain to the blockchain; and embedding metadata information associated with the video frames into the corresponding video frames through a fragile watermark.

[0011] Optionally, in some embodiments, before sending the preprocessed multimodal behavioral data to the edge computing node, the method further includes: performing identity authentication and key negotiation between the terminal and the edge computing node using the SM2 algorithm.

[0012] Optionally, in some embodiments, the operating system behavior event stream includes at least one of the following types: external device access events, data capture and output events, or network communication anomaly events.

[0013] Optionally, in some embodiments, the analysis fusion is performed through a risk assessment model, which includes a rule engine and a multimodal fusion model. The analysis fusion includes: using the rule engine to analyze the multimodal behavioral data based on predetermined rules, and outputting rule risk scores and triggering reasons; using the multimodal fusion model to analyze the cross-modal features of the multimodal behavioral data based on machine learning, and outputting a risk score; and using the decision fusion module in the risk assessment model to perform a fusion calculation on the rule risk score and the risk score, and outputting the risk quantification result.

[0014] Optionally, in some embodiments, generating a disposal strategy based on the risk quantification result includes: mapping the risk quantification result to a risk level based on a dynamic threshold, wherein the dynamic threshold is adaptively adjusted according to at least one of the terminal's historical behavior baseline, the current environmental context, or the data security level.

[0015] Optionally, in some embodiments, the dynamic hierarchical encryption of the video stream and / or content stream based on the handling strategy includes: encrypting the video stream and / or content stream by calling different combinations of national cryptographic algorithms according to the risk level. The encryption strength levels include low, medium, and high levels. The low level uses single-round SM4-GCM encryption and generates an SM3 fingerprint; the medium level uses multi-round SM4-GCM encryption and combines it with SM3 for key derivation or integrity verification; the high level uses at least three rounds of SM4-GCM encryption and combines it with multi-layer SM3 hashing for integrity verification.

[0016] Optionally, in some embodiments, the attribute-based access control includes: upon receiving an access request for the video stream or the content stream, obtaining subject attributes, object attributes, and environmental attributes associated with the access request; using the risk quantification result as a key environmental attribute, and performing a policy evaluation with the subject attribute, object attribute, and other environmental attributes; allowing the access request when each attribute satisfies a predetermined access control policy; otherwise, rejecting the access request.

[0017] Optionally, in some embodiments, the terminal data leakage risk quantification method further includes: performing federated learning on the risk assessment model, wherein the federated learning includes: each edge computing node training the risk assessment model locally, generating model parameter updates and uploading them to the central server; the central server aggregating all the model parameter updates, generating a global model update and distributing it to each edge computing node.

[0018] Secondly, this application also provides a terminal data leakage risk quantification system based on multimodal behavioral data fusion, applicable to the terminal data leakage risk quantification method based on multimodal behavioral data fusion described in the first aspect above. The terminal data leakage risk quantification system includes a terminal and edge computing nodes.

[0019] The terminal is used to collect multimodal behavior data, which includes video streams, content streams, and operating system behavior event streams, and to preprocess the video streams and send the preprocessed multimodal behavior data to the edge computing node.

[0020] The edge computing node is used to receive the preprocessed multimodal behavior data from the terminal; analyze and fuse the preprocessed multimodal behavior data to obtain risk quantification results; generate a handling strategy based on the risk quantification results; and send the handling strategy to the terminal.

[0021] The terminal is also configured to perform dynamic hierarchical encryption on the video stream and / or content stream based on the received processing strategy, and to execute attribute-based access control.

[0022] The technical solution provided in this application has the following advantages compared with the prior art:

[0023] The terminal anti-leakage risk quantification method based on multimodal behavioral data fusion provided in this application collects multimodal behavioral data such as video streams, operation content, and system behavioral events from the terminal. After performing hash calculation and watermark embedding preprocessing locally, the data is sent to the edge computing node. The edge computing node's fusion rule engine and multimodal fusion model analyze and fuse the multimodal behaviors to generate risk quantification results and handling strategies, including specific scores, triggering reasons, and evidence fragments. After the handling strategy is sent to the terminal, it drives the terminal to implement graded encryption of video content according to risk level and dynamically control access permissions based on multiple attributes. This application uses national cryptographic algorithms to achieve end-to-end security and compliance. An immutable evidence chain is constructed through terminal pre-binding, real-time edge analysis, and blockchain solidification. At the same time, the computing is pushed down to the edge through edge computing node-terminal collaboration, solving the end-to-end latency problem. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0026] Figure 1 A flowchart illustrating the terminal data leakage risk quantification method based on multimodal behavioral data fusion provided in this application embodiment;

[0027] Figure 2 A schematic diagram illustrating the workflow of the terminal data leakage risk quantification method based on multimodal behavioral data fusion provided in this application embodiment;

[0028] Figure 3 A schematic diagram of the multimodal data fusion process provided in the embodiments of this application;

[0029] Figure 4 This is a schematic diagram of the terminal anti-leakage risk quantification system based on multimodal behavioral data fusion provided in the embodiments of this application. Detailed Implementation

[0030] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0031] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.

[0032] The following description, in conjunction with the accompanying drawings, provides an exemplary description of the terminal data leakage risk quantification method and system based on multimodal behavioral data fusion provided in this application. Figure 1 A flowchart illustrating the terminal data leakage risk quantification method based on multimodal behavioral data fusion provided in this application embodiment. Figure 2 A schematic diagram illustrating the workflow of the terminal data leakage risk quantification method based on multimodal behavioral data fusion provided in this application embodiment. Figure 3 This is a schematic diagram of the multimodal data fusion process provided in an embodiment of this application. First, refer to... Figure 1 A terminal data leakage risk quantification method based on multimodal behavioral data fusion includes the following steps:

[0033] S101. Collect multimodal behavior data through the terminal. The multimodal behavior data includes video stream, content stream and operating system behavior event stream, and preprocess the video stream.

[0034] Specifically, multimodal behavioral data can be collected through the terminal. This multimodal behavioral data can include video streams (such as raw video frames captured by the terminal's camera), content streams (such as the current window name, the number of sensitive keywords, etc.), and operating system behavioral event streams (such as window / process switching, clipboard, privacy / incognito mode, peripheral device access, screenshot / screen recording calls, network access modes, etc.).

[0035] Preprocessing the video stream can involve operations such as denoising and enhancing the original video frames, extracting target detection boxes, filtering keyframes, and generating SM3 hash chains to generate lightweight feature vectors and timestamped encrypted data.

[0036] For example, the operating system behavior event stream includes at least one of the following types: external device access event, data capture and output event, or network communication anomaly event.

[0037] Specifically, external device access events such as USB or other external device access events; data capture and output events such as screenshot / screen recording events, sensitive file open / export events, clipboard operations, external channel call events; and network communication anomaly events such as network access mode mutation events.

[0038] In one embodiment, the preprocessing of the video stream may include: performing hash calculations on the video frames of the video stream to construct a hash chain; anchoring the digest information of the hash chain to a blockchain; and embedding metadata information associated with the video frames into the corresponding video frames using a fragile watermark.

[0039] Specifically, an SM3 (Chinese national cryptographic algorithm) hash value is calculated for each frame of the video stream, and the hash digest of the previous frame is used as the input parameter for the hash calculation of the current frame to construct a chain of hashes to achieve frame-level complete verification. Simultaneously, the current digest information of the hash chain is periodically (e.g., every 30 seconds) anchored to the blockchain, forming an immutable on-chain evidence record. Furthermore, metadata information associated with the video frame (including timestamps, device identification, risk level labels, and hash digests) can be embedded into a specific frequency domain region of the corresponding video frame using a fragile watermarking algorithm, making the watermark information highly sensitive to any tampering with the frame content (such as cropping, replacement, or compression).

[0040] For example, timestamps, location / location identifiers, single-frame SM3 hashes, SM4 key IDs, behavior stream summaries, content stream sensitive feature summaries, risk scores r, and trigger rule IDs can be packaged into encrypted metadata. After ASN.1 DER encoding, this metadata is encrypted using SM4-CTR mode and embedded in the 9th-15th bits of the DCT AC coefficients of the Y component of the video frame using an adaptive fragile watermarking algorithm. The embedding strength α dynamically adapts to the current macroblock visual characteristics, satisfying α=β·(L / 255), β∈[0.8,1.2], where L is the average brightness value of the current macroblock. This ensures the watermark's imperceptibility while making it highly sensitive to any frame-level tampering (such as recompression, cropping, or replacement), thereby achieving spatiotemporal binding and integrity between encrypted metadata and video content.

[0041] S102. The preprocessed multimodal behavior data is sent to the edge computing node.

[0042] Specifically, the multimodal behavior data preprocessed in step S101 can be sent to the edge computing node via a national cryptographic SSL / TLS secure channel or a lightweight identifier cryptographic authentication protocol, including lightweight feature vectors, encrypted video stream ciphertext hash values, and timestamped metadata.

[0043] Edge computing nodes may include an SM2 certificate management and CRL revocation unit for trusted terminal access and short-cycle certificate verification; they may also include a session key management unit for deriving SM4 session keys based on SM2-ECDH and combined with SM3-KDF and actively destroying them according to their lifecycle.

[0044] Edge computing nodes can build an attribute access control system that binds subject attributes (such as role, security level, organization, task, qualification), object attributes (such as video / evidence security level, time period, risk level, sidechain anchoring status), and environmental attributes (such as location, time, network domain, terminal trust status) to the handling strategies described below.

[0045] In one embodiment, before sending the preprocessed multimodal behavior data to the edge computing node, the method may further include: performing identity authentication and key negotiation between the terminal and the edge computing node using the SM2 algorithm.

[0046] Specifically, identity authentication can be achieved through the SM2 algorithm to realize two-way identity authentication between the terminal and the edge computing node. The SM2 certificate follows the GM / T 0016-2012 national cryptographic X.509 v3 format, has a validity period of less than or equal to 24 hours, and can be verified and revoked in real time in the local CRL list of the edge computing node.

[0047] Key negotiation can be based on SM2-ECDH to negotiate a shared secret, and SM4 session keys can be derived through SM3-KDF. The derived parameters include at least the terminal serial number, edge node ID, and UTC second-level timestamp. Session keys have a lifecycle and support active destruction and erasure, and support forward secrecy.

[0048] It is understood that the edge computing node can be a distributed edge computing node. A distributed edge computing node refers to an edge computing architecture that can be formed by multiple heterogeneous edge computing power units interconnected through a network or slices to achieve task scheduling, load balancing and state synchronization. This can solve the problems of single-node computing power bottleneck and network bandwidth limitation in deployment scenarios such as large-scale confidential venues (such as multi-story conference centers, cross-regional military industrial bases, etc.).

[0049] S103. At the edge computing node, the preprocessed multimodal behavior data is analyzed and fused to obtain risk quantification results.

[0050] Specifically, the preprocessed multimodal data is first time-aligned and aggregated using a sliding window to construct behavioral stream segments. The triggering reasons (such as frequent window switching, frequent clipboard activity, privacy mode activation, multi-application activity, and non-working-hour activities) and risk scores are output through these stream segments. Simultaneously, a model risk score can be output through a multimodal fusion model. Then, the risk scores, including the triggering reasons, are fused with the model risk score to obtain the risk quantification result. Figure 3 This is a schematic diagram illustrating the multimodal data fusion process provided in this embodiment of the application. In practical applications, it can be referred to... Figure 3 The process shown fuses multimodal data.

[0051] In one embodiment, the analysis fusion is performed through a risk assessment model, which includes a rule engine and a multimodal fusion model. The analysis fusion may include: using the rule engine to analyze the multimodal behavioral data based on predetermined rules, and outputting rule risk scores and triggering reasons; using the multimodal fusion model to analyze the cross-modal features of the multimodal behavioral data based on machine learning, and outputting a risk score; and using the decision fusion module in the risk assessment model to perform a fusion calculation on the rule risk scores and the risk score, and outputting the risk quantification result.

[0052] Specifically, the rule engine can analyze multimodal behavioral data based on preset, explicit expert logic rules (e.g., if three consecutive screenshot events are triggered within 10 seconds, the risk score increases by 20), and output rule risk scores with clear causal relationships and explainable triggering reasons. Explainable reasons include, for example, the type of triggering event, the time window, the number or intensity, and the corresponding fragment index.

[0053] Meanwhile, the multimodal fusion model can perform end-to-end learning and abstraction of video features, text semantic features, and behavioral sequence features in the same data stream, and output a data-driven risk score that reflects the correlation pattern. Through the decision fusion layer (such as weighted average, gated network, or Bayesian fusion algorithm), the risk score of the interpretable triggering cause output by the rule engine and the intelligent probability score output by the multimodal fusion model are comprehensively calculated to generate an interpretable risk quantification result, and finally complete the binding of frame-event-judgment.

[0054] Furthermore, edge computing nodes can construct Merkle Trees and write Merkle root hashes into blockchain sidechain transactions according to anchoring periods, forming an immutable time anchor; the corresponding sidechain transaction ID and analytical evidence can be traced through any frame hash.

[0055] In addition, edge computing nodes can perform sliding window verification on the received video and watermark. If any segment verification fails, the frame within the window is marked as untrusted and an audit event is recorded.

[0056] In one embodiment, the terminal data leakage risk quantification method may further include: performing federated learning on the risk assessment model, wherein the federated learning may include: each edge computing node training the risk assessment model locally, generating model parameter updates and uploading them to the central server; the central server aggregating all the model parameter updates, generating a global model update and distributing it to each edge computing node.

[0057] Specifically, each edge computing node can incrementally train or fine-tune its risk assessment model based on locally collected and processed multimodal behavioral data. After training, each node encrypts and uploads the resulting model parameter updates (i.e., adjustments to the model's internal weights or gradients) to the central server. The central server can securely aggregate the model parameter updates uploaded by all edge nodes, reflecting the data characteristics and risk patterns of their respective regions, using a specific federated aggregation algorithm (such as FedAvg or FedProx), thereby generating a global model update. The central server then distributes the global model update to all edge computing nodes participating in federated learning. Each edge node receives and applies this global model update and can also record version audit information, thus achieving cross-regional adaptation and continuous optimization.

[0058] S104. Generate a handling strategy based on the risk quantification results, and send the handling strategy to the terminal.

[0059] Specifically, a handling strategy is generated based on the risk quantification results obtained in step S103. The handling strategy contains a set of instructions with clearly defined parameters and execution logic. The process of generating the strategy can typically involve a pre-defined risk-strategy mapping matrix, which automatically matches and instantiates corresponding protection rules based on the risk quantification results (such as specific risk scores). These rules may include: specifying encryption levels (e.g., enabling enhanced dual-round SM4-GCM encryption for subsequent video streams), defining access control rules (e.g., marking video clips generated during the current high-risk period as accessible only to auditors), and setting response actions (e.g., triggering local audio-visual alarms or restricting peripheral interfaces). Edge computing nodes can distribute the handling strategy to the terminal that generated the risk through an established secure channel (e.g., a session based on SM4 encryption).

[0060] In one embodiment, generating a disposal strategy based on the risk quantification result may include: mapping the risk quantification result to a risk level based on a dynamic threshold, wherein the dynamic threshold is adaptively adjusted according to at least one of the terminal's historical behavior baseline, the current environmental context, or the data security level.

[0061] Specifically, a risk level mapping mechanism based on dynamic thresholds can be used. Specifically, the determination of risk levels (e.g., low, medium, high) is based on dynamically changing thresholds. The adjustment of dynamic thresholds is based on one or more of the following factors: the terminal's historical behavior baseline (e.g., for users with typically stable operations, the threshold for judging abnormal activity should be lower than for users with active behavior, to achieve personalized risk perception); the current environmental context (e.g., risk assessment in a confidential conference room should be more stringent than in a regular office area, and the threshold should be lowered accordingly); and the data's confidentiality level (e.g., when processing top-secret data, even minor anomalies should be given high priority, and the risk assessment threshold should be lowered accordingly).

[0062] S105. The terminal performs dynamic hierarchical encryption on the video stream and / or content stream based on the processing strategy, and executes attribute-based access control.

[0063] Specifically, the terminal can dynamically select and call up a combination of encryption algorithms and strength parameters that match the risk level specified in the handling strategy from a preset national cryptographic algorithm library. For example, low-risk data can be encrypted using a single round of SM4-GCM and generate an SM3 fingerprint; high-risk data can be encrypted using at least three rounds of SM4-GCM.

[0064] Attribute-based access control can be implemented by binding subject attributes, object attributes, environmental attributes, and handling strategies, as described above. Alternatively, policy decision points (PDPs) and policy execution points (PEPs) can be set up at edge computing nodes or central servers to perform real-time authentication of access to video clips, evidence packages, analysis results, and model parameters. Authentication credentials can use attribute tokens or attribute certificates signed with SM2. The token carries a set of attributes, an expiration date, and a revocation identifier, and is bound to the audit log. When the revocation condition is triggered, access is immediately denied and the key is forcibly destroyed. Furthermore, all access and handling actions can generate SM3 digests and be signed and uploaded to the blockchain or anchored to a blockchain sidechain to achieve traceable auditing.

[0065] In one embodiment, the dynamic hierarchical encryption of the video stream and / or content stream based on the handling strategy may include: encrypting the video stream and / or content stream by calling different combinations of national cryptographic algorithms according to the risk level, wherein the encryption strength levels include low, medium, and high levels, wherein the low level uses single-round SM4-GCM encryption and generates an SM3 fingerprint; the medium level uses multi-round SM4-GCM encryption and combines it with SM3 for key derivation or integrity verification; and the high level uses at least three rounds of SM4-GCM encryption and combines it with multi-layer SM3 hashing for integrity verification.

[0066] Specifically, using the risk level determined in the handling strategy as input, a predetermined combination of national cryptographic algorithms is dynamically invoked to implement hierarchical encryption protection for video streams and / or content streams. Specifically, for low-level protection (such as general risk scenarios), a single round of SM4-GCM encryption can be used (ensuring data confidentiality and integrity) and generating an SM3 fingerprint (providing basic data digest and source verification); for medium-level protection (such as significant risks), security resilience is strengthened on the basis of confidentiality, such as using multiple rounds of SM4-GCM encryption to increase the difficulty of cracking, and combining the SM3 algorithm for more complex key derivation (enhancing key security) or integrity verification (improving anti-tampering capabilities); for the highest level of protection (serious threats), at least three rounds of SM4-GCM encryption can be used to construct a high-strength confidentiality barrier, combined with multi-layer SM3 hashing for cascaded integrity verification, forming a defense-in-depth.

[0067] In one embodiment, the attribute-based access control may include: upon receiving an access request for the video stream or the content stream, obtaining subject attributes, object attributes, and environmental attributes associated with the access request; using the risk quantification result as a key environmental attribute, and performing a policy evaluation with the subject attribute, object attribute, and other environmental attributes; allowing the access request if each attribute satisfies a predetermined access control policy; otherwise, rejecting the access request.

[0068] Specifically, a three-element attribute set associated with the access request can be obtained from different dimensions: subject attributes (such as the visitor's identity, role, security level, and task), object attributes (such as the risk label, data security level, encryption level, and whether the requested video clip has been stored on the blockchain), and environmental attributes (such as the absolute time of access, terminal geographical location, and network domain status). Simultaneously, the risk quantification results calculated by upstream edge computing nodes are added to the evaluation framework as key environmental attributes of the new dynamic changes. Then, these attributes are input into a preset access control policy engine for joint logical evaluation. The policy engine can comprehensively judge all attributes according to preset rules (such as allowing read access only when the subject's security level is greater than or equal to the object's security level, the current terminal network is trustworthy, the real-time risk level of the associated video clip is less than medium, and the access time is within working hours). Only when the values ​​of all attributes fully meet the preset rules will the access request be allowed; if any attribute does not meet the requirements (such as exceeding the risk level), the access request will be rejected, thus realizing a shift from traditional static identity authentication to dynamic, context-aware, and risk-driven access control.

[0069] Figure 2 This is a schematic diagram illustrating the workflow of the terminal data leakage risk quantification method based on multimodal behavioral data fusion provided in the embodiments of this application. The following is in conjunction with... Figure 2The workflow described herein is explained. (Refer to...) Figure 2 As shown, trusted access can first be achieved through SM2 two-way authentication and key negotiation; then, multimodal data such as video streams, content streams, and behavioral event streams are collected by the terminal, and evidence binding based on SM3 hashing and fragile watermarks is completed locally; after the multimodal data is transmitted to the edge computing node, it is processed by the rule engine and the multimodal fusion model (…). Figure 2 The AI ​​model shown collaboratively performs real-time risk quantification analysis; if the risk exceeds the threshold, it triggers tiered handling—including local terminal alarms, dynamic national cryptographic encryption based on risk level (such as the combination of SM4-GCM from Basic to Maximum level), and access control that integrates real-time risk attributes; at the same time, edge computing nodes can perform watermark verification on videos, build Merkle hash chains, and anchor the root hash to the blockchain to achieve tamper-proof evidence; and the central server can aggregate model updates from each node through federated learning for optimization, and can also achieve full-process event visualization and traceability and evidence export through auditing, so as to form a complete closed loop from perception, decision-making, protection to evidence storage and evolution.

[0070] The following section will provide a detailed explanation of the terminal data leakage risk quantification method based on multimodal behavioral data fusion provided in this application embodiment, taking into account specific application scenarios.

[0071] Example 1: Typical confidential office scenario

[0072] Four edge nodes were deployed in a 60-square-meter confidential office, using wall-mounted installation to ensure full signal coverage. Before activation, all office terminals completed SM2 certificate registration, with the certificate validity period set to working hours plus a one-hour buffer period. During operation, if abnormal behavior is detected on a terminal, such as frequent window switching, high-frequency clipboard operations, or activation of privacy / incognito mode within a short period, and the content stream captures sensitive keywords while the camera video stream shows an abnormal image pattern resembling "video taken with a mobile phone," the edge computing nodes aggregate the behavioral event stream using a sliding time window and calculate rule scores. Simultaneously, they perform feature analysis on the video stream and output a risk score 'r.' After fusion, if an alarm threshold is reached, a tiered response is triggered: terminal pop-up / audio-visual alarms, encrypted alarm log generation, and reporting. The PDP issues a revocation command based on the risk level and the terminal's trusted status. The PEP immediately blocks unauthorized access to the evidence package and sensitive segments and triggers session key destruction. Post-event auditing can automatically export the evidence package for internal auditing by tracing back the triggering event, risk curve, corresponding frame hash, and sidechain transaction ID via a timeline.

[0073] Example 2: Offline R&D office area scenario

[0074] The highly confidential R&D office has network access restrictions and uses an offline caching mechanism to handle network outages. When a network outage is detected, the terminal encrypts and caches video frames, behavioral events, and metadata in a local secure area using SM4 and records consecutive timestamps; these are then uploaded in batches after the network is restored. Edge computing nodes perform watermark consistency verification and hash chain reconstruction on the re-transmitted data and rewrite sidechain transactions according to the anchor period to ensure that data can still be verified and traced during the network outage; if any GOP window watermark verification fails, the corresponding window frame is marked as "untrusted" and an audit record is generated.

[0075] Example 3: Meeting and Office Scenarios

[0076] A dedicated monitoring system is deployed in the conference room, and the central server is equipped with auditing components. During the meeting, the system continuously generates behavioral streams and anchors video evidence. When high-risk behaviors are detected (such as enabling privacy mode, abnormal clipboard export, frequent switching to external communication software, etc.) and reach a threshold, an evidence package is automatically generated and encrypted for reporting. After the meeting, access to the evidence package is subject to attribute-based access control: access is only allowed when the subject attributes meet the authorization requirements of position / security level / organization / task, the environmental attributes meet the requirements of location and valid time period, the terminal is in a normal trusted state, and the object attributes meet the requirements of "sidechain anchored and not marked as untrustworthy"; the token uses SM2 signature, has a short validity period, and is revocable. When the audit ends or the revocation conditions are triggered (timeout, certificate revocation, risk escalation), the PDP issues a revocation, and the PEP immediately blocks access, forming a traceable audit chain.

[0077] In summary, the terminal anti-leakage risk quantification method 100 based on multimodal behavioral data fusion provided in this application collects multimodal behavioral data such as video streams, operation content, and system behavioral events through the terminal, and sends the preprocessed data, including hash calculation and watermark embedding, to the edge computing node after completing hash calculation and watermark embedding locally. The edge computing node's fusion rule engine and multimodal fusion model analyze and fuse the multimodal behaviors to generate risk quantification results and handling strategies, including specific scores, triggering reasons, and evidence fragments. After the handling strategy is sent to the terminal, it drives the terminal to implement graded encryption of video content according to risk level and dynamically control access permissions based on multiple attributes. This application uses national cryptographic algorithms to achieve end-to-end security and compliance, and constructs an immutable evidence chain through terminal pre-binding, real-time edge analysis, and blockchain solidification and storage. At the same time, the edge computing node-terminal collaboration pushes the computation down to the edge, solving the end-to-end latency problem.

[0078] Based on the same inventive concept, this application also provides a terminal data leakage risk quantification system based on multimodal behavioral data fusion. Figure 4 This is a schematic diagram of the terminal data leakage risk quantification system based on multimodal behavioral data fusion, provided as an embodiment of this application. Figure 4As shown, the terminal data leakage risk quantification system includes: a terminal and an edge computing node.

[0079] The terminal is used to collect multimodal behavior data, which includes video streams, content streams, and operating system behavior event streams, and to preprocess the video streams and send the preprocessed multimodal behavior data to the edge computing node.

[0080] Specifically, the terminal is used to collect multimodal behavioral data. The terminal can be a classified office PC, a mobile work terminal, or a dedicated terminal, and it has a built-in national cryptographic module or national cryptographic library, supporting SM2, SM3, and SM4 algorithm operations. The terminal can be accessed through methods such as... Figure 4 The data acquisition module shown collects data and simultaneously acquires operating system behavior event streams (window / process switching, clipboard, privacy / incognito mode, peripheral device access, screenshot / screen recording calls, network access modes, etc.) and content streams (current window name, number of sensitive keywords, etc.). The terminal can then access data through methods such as... Figure 4 The evidence binding module shown performs SM3 hash calculation on video frames, generates metadata, and performs ASN.1 DER encoding and SM4 encryption before embedding a fragile watermark to achieve "frame-event-judgment" binding.

[0081] Furthermore, the terminal can also be accessed via, for example... Figure 4 The secure transmission and caching module shown can perform identity authentication using SM2, data upload encryption using SM4, and offline encrypted caching; and can also perform full-screen alarms, screen lock, and upload events when a risk is detected through the local response module.

[0082] The terminal can also integrate a hardware-level tamper detection unit to monitor states such as camera obstruction, system ROOT / jailbreak, and tampering of key policies; when the tampering conditions are triggered, the terminal executes a revocation response and erases the session key, while generating an auditable event.

[0083] The edge computing node is used to receive the preprocessed multimodal behavior data from the terminal; analyze and fuse the preprocessed multimodal behavior data to obtain risk quantification results; generate a handling strategy based on the risk quantification results; and send the handling strategy to the terminal.

[0084] Specifically, edge computing nodes can be used for behavior analysis, risk assessment, strategy generation, and evidence storage. Specifically, edge computing nodes can perform time alignment and sliding window aggregation on preprocessed multimodal data to construct behavior stream segments. These stream segments output triggering reasons (such as frequent window switching, frequent clipboard activity, privacy mode activation, multi-application activity, and activity outside of working hours) and risk scores. Simultaneously, a multimodal fusion model can output a model risk score. Then, the risk score, including the triggering reasons, is fused with the model risk score to obtain a risk quantification result. Based on the risk quantification result, a response strategy can be generated and distributed to the terminal.

[0085] In practical applications, edge computing nodes can be set as Figure 4 The distributed edge computing nodes shown perform certificate verification and CRL checks through trusted access and key management, negotiate keys through SM2-ECDH+SM3-KDF, and conduct SM4 encrypted communication. The behavior fusion and judgment engine can use the rule engine and multimodal model to analyze and fuse behavioral data, and output risk quantification results. Based on the risk quantification results, it generates policies and tokens, and is responsible for the issuance and revocation of tokens. It can also perform evidence verification and hash chain, such as verifying fragile watermarks and building Merkle hash chains to ensure data integrity.

[0086] The blockchain side chain receives digitally signed root hash transactions from edge computing nodes and packages the transactions onto the chain through a consensus mechanism, achieving immutability and permanent recording of evidence.

[0087] The terminal is also configured to perform dynamic hierarchical encryption on the video stream and / or content stream based on the received processing strategy, and to execute attribute-based access control.

[0088] Specifically, the terminal can dynamically select and call up a combination of encryption algorithms and strength parameters that match the risk level specified in the handling strategy from a preset national cryptographic algorithm library. For example, low-risk data can be encrypted using a single round of SM4-GCM and generate an SM3 fingerprint; high-risk data can be encrypted using at least three rounds of SM4-GCM.

[0089] Attribute-based access control can be implemented by binding subject attributes, object attributes, environmental attributes, and handling strategies. Alternatively, policy decision points (PDPs) and policy execution points (PEPs) can be set up at edge computing nodes or central servers to perform real-time authentication of access to video clips, evidence packages, analysis results, and model parameters. Authentication credentials can use attribute tokens or attribute certificates signed with SM2. The token carries a set of attributes, an expiration date, and a revocation identifier, and is bound to the audit log. When revocation conditions are triggered, access is immediately denied and the key is forcibly destroyed. Furthermore, all access and handling actions can generate SM3 digests and be signed and uploaded to the blockchain or anchored to a blockchain sidechain to achieve traceable auditing.

[0090] Furthermore, the terminal anti-leakage risk quantification system can also be configured as follows: Figure 4 The central service shown here, with the central server serving as the hub for policy management, model coordination, and audit tracing, provides backend support for distributed edge computing nodes. The server may include a federated aggregator, a policy decision (PDP) module, an audit and forensics module, etc.

[0091] The federated aggregator can aggregate trained multimodal model parameters or gradients from the behavior fusion and judgment engine of multiple edge computing nodes to generate an updated global risk identification model, and then distribute the optimized model to each edge computing node to achieve continuous optimization of the system's risk identification capability.

[0092] The Policy Decision Point (PDP) module is used for global security policies and access control rules. It is responsible for the lifecycle management of tokens (issuance, revocation, and update), and distributes the disposal policies and tokens to the distributed edge computing nodes, where they are executed by the node's PDP and PEP modules.

[0093] The auditing and evidence collection module is used to receive alarms and event logs from edge computing nodes and terminals, providing a global timeline display and comprehensive analysis; it can also export complete evidence packages containing hash chains and watermark verification information.

[0094] The terminal leakage risk quantification system based on multimodal behavioral data fusion provided in the above embodiments can execute the terminal leakage risk quantification method based on multimodal behavioral data fusion provided in the above embodiments, and has the same or corresponding beneficial effects, which will not be described in detail here.

[0095] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments.

[0096] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0097] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0098] In this application, unless otherwise stated, directional terms such as "up" and "down" are generally used in relation to the direction shown in the accompanying drawings, or in relation to the vertical, perpendicular, or gravitational direction; similarly, for ease of understanding and description, "left" and "right" are generally used in relation to the left and right shown in the accompanying drawings; "inner" and "outer" refer to the inner and outer contours of each component itself, but the above directional terms are not intended to limit this application.

[0099] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for quantifying terminal data leakage risk based on multimodal behavioral data fusion, characterized in that, include: Multimodal behavioral data is collected through the terminal, including video streams, content streams, and operating system behavior event streams, and the video streams are preprocessed. The preprocessed multimodal behavioral data is sent to the edge computing node; At the edge computing node, the preprocessed multimodal behavioral data is analyzed and fused to obtain risk quantification results; A response strategy is generated based on the risk quantification results, and the response strategy is then sent to the terminal. The terminal performs dynamic hierarchical encryption on the video stream and / or content stream based on the processing strategy, and executes attribute-based access control.

2. The terminal leakage risk quantification method according to claim 1, characterized in that, The preprocessing of the video stream includes: Hash calculations are performed on the video frames of the video stream to construct a hash chain; The hash chain's summary information is anchored to the blockchain, and metadata information associated with the video frame is embedded into the corresponding video frame using a fragile watermark.

3. The terminal leakage risk quantification method according to claim 2, characterized in that, Before sending the preprocessed multimodal behavioral data to the edge computing node, the method further includes: The SM2 algorithm is used to perform identity authentication and key negotiation between the terminal and the edge computing node.

4. The terminal leakage risk quantification method according to claim 3, characterized in that, The operating system behavior event stream includes at least one of the following types: External device access events, data capture and output events, or network communication anomaly events.

5. The terminal leakage risk quantification method according to claim 1, characterized in that, The analysis and fusion are performed through a risk assessment model, which includes a rule engine and a multimodal fusion model; the analysis and fusion include: The rule engine analyzes the multimodal behavior data based on predetermined rules, and outputs rule risk scores and triggering reasons. The multimodal fusion model is used to analyze the cross-modal features of the multimodal behavioral data based on machine learning, and a risk score is output. The decision fusion module in the risk assessment model is used to perform fusion calculation on the rule risk score and the risk rating, and output the risk quantification result.

6. The terminal leakage risk quantification method according to claim 5, characterized in that, The generation of a response strategy based on the risk quantification results includes: The risk quantification result is mapped to a risk level based on a dynamic threshold, which is adaptively adjusted according to at least one of the terminal's historical behavior baseline, the current environmental context, or the data security level.

7. The terminal leakage risk quantification method according to claim 6, characterized in that, The dynamic hierarchical encryption of the video stream and / or content stream based on the processing strategy includes: Based on the risk level, different combinations of national cryptographic algorithms are used to encrypt the video stream and / or content stream. The encryption strength levels include low, medium, and high levels. The lower level uses single-round SM4-GCM encryption and generates an SM3 fingerprint; The intermediate level uses multi-round SM4-GCM encryption, combined with SM3 for key derivation or integrity verification; The high-level encryption employs at least three rounds of SM4-GCM encryption, combined with multi-layer SM3 hashing for integrity verification.

8. The terminal leakage risk quantification method according to claim 1, characterized in that, The attribute-based access control includes: Upon receiving an access request for the video stream or the content stream, obtain the subject attributes, object attributes, and environmental attributes associated with the access request; The risk quantification results are used as key environmental attributes and are used in a strategy evaluation along with the subject attributes, object attributes, and other environmental attributes. If each of the attributes satisfies the predetermined access control policy, the access request is allowed; otherwise, the access request is denied.

9. The terminal leakage risk quantification method according to claim 1, characterized in that, Also includes: Federated learning is performed on the risk assessment model, and the federated learning includes: Each edge computing node trains the risk assessment model locally, generates model parameter updates, and uploads them to the central server; The central server aggregates all the model parameter updates, generates a global model update, and distributes it to each edge computing node.

10. A terminal data leakage risk quantification system based on multimodal behavioral data fusion, applicable to the terminal data leakage risk quantification method based on multimodal behavioral data fusion as described in any one of claims 1 to 9, wherein the terminal data leakage risk quantification system includes a terminal and an edge computing node. The terminal is used to collect multimodal behavior data, which includes video streams, content streams, and operating system behavior event streams, and to preprocess the video streams and send the preprocessed multimodal behavior data to the edge computing node. The edge computing node is used to receive the preprocessed multimodal behavior data from the terminal; The preprocessed multimodal behavioral data is analyzed and fused to obtain risk quantification results, and a handling strategy is generated based on the risk quantification results and then sent to the terminal. The terminal is also configured to perform dynamic hierarchical encryption on the video stream and / or content stream based on the received processing strategy, and to execute attribute-based access control.

Images