An internet information real-time monitoring system based on multi-modal content recognition

Abstract

An internet information real-time monitoring based on multi-modal content recognition comprises a boundary traffic perception and pre-screening module deployed at a network boundary, which calculates a comprehensive risk score by extracting a text, visual and audio three-modal lightweight fingerprint, realizes linear speed release of more than 90% normal traffic and hierarchical shunting of suspicious traffic, a multi-modal deep analysis and causal fusion identification module, which performs content restoration and single-modal semantic analysis on the shunted traffic, maps the extracted semantic nodes to a pre-constructed advertisement semantic causal dependency graph for cross-modal causal chain reasoning, and realizes high-precision identification of split implicit advertisements in combination with an adaptive correction mechanism of an adversarial sample, a dynamic risk field evaluation module, which realizes continuous accumulation and attenuation of advertisement risks based on an exponential smoothing model, a propagation graph construction and abnormal reasoning module, which detects group coordination behavior of a black production delivery network through a graph attention network, a traceability and linkage disposal module, which realizes propagation link traceability based on a robust content fingerprint, and issues content-level accurate blocking rules to a boundary firewall and a DNS server through a standard protocol.

Description

Technical Field

[0001] This invention relates to the field of internet information security and content monitoring technology, specifically an internet information real-time monitoring system based on multimodal content recognition. Background Technology

[0002] Existing internet advertising monitoring technologies are mainly divided into two categories: First, network boundary security detection devices (such as DPI and NGFW) only perform coarse-grained control based on traffic layer characteristics such as IP, port, and protocol, and do not have content-level semantic recognition capabilities. They have no effective means of detecting advertising content in HTTPS / QUIC encrypted traffic except for complete decryption, and the control granularity is coarse, resulting in a high false blocking rate. Second, application-layer multimodal content review systems are deployed on the business server side or in the cloud. They need to complete content decoding and multimodal fusion inference on the entire traffic volume, which consumes a lot of computing power. In scenarios with high bandwidth of tens of gigabits, it is difficult to balance real-time performance and recognition accuracy. Moreover, existing multimodal fusion generally adopts feature correlation matching, and the false detection rate of split implicit advertisements (such as images showing ordinary food, audio broadcasting medical efficacy, and subtitles containing contact information, each modality is compliant when viewed individually, and only cross-modal combination constitutes a complete advertising logic) is as high as 42%.

[0003] Furthermore, existing systems are all "single-time static judgments," failing to reflect the dynamic accumulation and decay process of advertising risks, and lacking effective detection of gradual and intermittent advertising. Detection and handling are completely disconnected; the system's business endpoint is the generation of alarms, lacking the ability to trace the source of traffic propagation and to block it at the boundary. When faced with adversarial samples, it relies on manual annotation and iteration, with response cycles as long as 1-2 weeks. Moreover, it cannot identify the collective collaborative behavior of black market advertising. These shortcomings make it impossible for existing technologies to simultaneously meet the three core requirements of high-bandwidth real-time performance, high accuracy of hidden content, and closed-loop control at the propagation level. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a real-time monitoring system for Internet information based on multimodal content recognition, which can effectively solve the problems mentioned in the background art above.

[0005] To address the above problems, the technical solution adopted by this invention is: a real-time internet information monitoring system based on multimodal content recognition, comprising: The boundary traffic awareness and pre-screening module is deployed at the network boundary to access Internet traffic at line speed and perform protocol parsing and session reassembly on traffic sessions. It extracts lightweight semantic fingerprints of text modality, visual modality and audio modality from the traffic, calculates a comprehensive risk score based on the suspiciousness of each modality fingerprint, and performs hierarchical traffic diversion according to the comprehensive risk score. Normal traffic with a comprehensive risk score below the first threshold is directly allowed to return to the original link, while suspicious traffic with a comprehensive risk score reaching the first threshold is diverted to the deep recognition engine. The multimodal deep analysis and causal fusion identification module is used to complete content restoration and single-modal semantic analysis of the suspicious traffic in the diversion. It extracts several semantic nodes from the analysis results and maps the semantic nodes to a pre-constructed advertising semantic causal dependency graph for cross-modal causal chain reasoning. When the formed candidate causal link contains both product display layer nodes and traffic conversion layer nodes and the link confidence is not lower than the preset judgment threshold, the corresponding traffic content is determined to be an illegal advertisement. The source tracing and coordinated handling module is used to generate robust content fingerprints that are resistant to compression and pruning based on cross-modal fusion features. Using these robust content fingerprints as anchors, it matches sessions with the same fingerprints in traffic logs to trace the source node in reverse and sort out the distribution nodes in forward, generating a propagation link topology. Then, it issues content-level precise blocking rules to network boundary firewalls, DNS servers, or traffic scrubbing devices through security device docking protocols.

[0006] Preferably, in the boundary traffic perception and pre-screening module: the text modal fingerprint is obtained by generating a semantic hash value through character-level n-gram sampling and matching it with a Bloom filter of the built-in advertising keyword and homophonic coded language feature library; The visual modal fingerprint is obtained by extracting incomplete image data from the first few TCP segments of the image payload, inputting it into a lightweight visual coding model to output a low-dimensional feature vector, and performing an approximate nearest neighbor search in a pre-built vector index library. The audio modal fingerprint is obtained by extracting MFCC coefficients and energy features from the audio payload in frames, generating a perceptual hash sequence, and then matching it with the Hamming distance of the spoken template fingerprint library. The comprehensive risk score is the result of summing the text modality suspicion, visual modality suspicion, and audio modality suspicion with preset weights. The hierarchical traffic diversion also has a second threshold higher than the first threshold: when the comprehensive risk score reaches the second threshold, the boundary traffic perception and pre-screening module also performs temporary boundary control on the current session.

[0007] Preferably, the node system of the advertising semantic causal dependency graph includes at least the following levels arranged in causal logical order: scene preparation layer, product display layer, efficacy description layer, and traffic conversion layer; wherein, the causal progression edge connecting nodes of different levels is only allowed to be connected in the direction of orderly advancement from higher level to lower level or within the same level, and is assigned a basic weight of 0.8 to 1.0; the modal complementary edge connecting nodes of different modalities in the same semantic layer is assigned a weight of 0.6 to 0.9.

[0008] Preferably, the multimodal deep parsing and causal fusion identification module uses the following formula to calculate the link confidence of candidate causal links: ; in Let be the confidence level of the i-th node. S is the node weight, N is the total number of nodes in the link, C is the cross-modal complementarity coefficient, and T is the temporal coherence coefficient. When S≥0.75, it is judged as advertising content. When the link contains a function description layer node of a legally illegal function type, it is further judged as illegal advertising.

[0009] Preferably, the multimodal deep parsing and causal fusion identification module further includes an adversarial example adaptive correction unit, used for: For disputed semantic nodes in a single modality whose confidence level is lower than the first confidence threshold but higher than the second confidence threshold, the confirmed nodes in the other two modalities will jointly vote to correct them. For occluded QR code areas in the visual modality, when the semantic probability of the surrounding text exceeds a preset threshold, local repair and completion based on a generative model is initiated before QR code recognition is performed.

[0010] Preferred options also include: The dynamic risk field assessment module is used to maintain continuous risk values ​​over a time window sequence for each monitored object. According to the exponential smoothing relationship Update, among which This refers to the incremental risk within the current time window, calculated by weighting textual risk, visual risk, audio risk, behavioral risk, and network risk. when When the risk level continues to rise but does not reach the judgment threshold, the dynamic risk field assessment module notifies the boundary traffic perception and pre-screening module to lower the first threshold in order to expand the sampling coverage of suspicious traffic. when When the threshold value is reduced to a low level, the default threshold is restored.

[0011] Preferably, the source tracing and coordinated handling module further includes a propagation map construction and anomaly reasoning unit, used for: Construct a propagation graph whose node set at least includes content nodes, account nodes, link nodes, and network nodes, and whose edge set at least includes publishing relationships, redirection relationships, co-occurrence relationships, and collaboration relationships; Using the hit results of the robust content fingerprint in the full traffic log as seeds, a graph attention network is used to score the anomalies of local subgraphs around the seed in the propagation graph. When the subgraph shows a topology pattern of multiple accounts converging on the same landing page or the same origin attribution ID co-occurring across domains and the anomaly score exceeds the graph threshold, the judgment is upgraded to an organized black market delivery subgraph, and all related domains and attribution nodes in the subgraph are included in the blocking scope.

[0012] Preferably, the content-level precise blocking rule is an interception rule that is accurate to a single URL path or a specific file path, and the interception rule is configured as a temporary rule with an expiration date, which automatically expires after the expiration date, and can be revoked with one click when manually reviewed and determined to be a false alarm.

[0013] Preferably, the boundary traffic sensing and pre-screening module uses DPDK user-space driver to achieve line-speed traffic processing and offloads the multimodal fingerprint extraction logic to the FPGA hardware acceleration unit, with a single message processing delay of less than 1 microsecond.

[0014] Preferably, the multimodal deep analysis and causal fusion recognition module adopts an incremental training and self-evolution mechanism: when the number of new samples reaches a preset number, the samples are fine-tuned through LoRA low-rank adaptation. After training is completed, the corresponding fingerprints are automatically extracted and the feature library in the boundary traffic perception and pre-screening module is updated synchronously, with a synchronization time of less than 5 minutes.

[0015] Compared with existing technologies, this invention provides a real-time monitoring system for internet information based on multimodal content recognition, which has the following advantages: Breakthrough improvements in real-time performance and computing efficiency: Through the boundary traffic perception and pre-screening module, multimodal lightweight fingerprinting is used to achieve line-speed passage of more than 90% of normal traffic, and only suspicious traffic is diverted to the deep recognition engine. The overall computing power consumption is reduced by more than 90%, the processing latency of a single traffic at the boundary layer is less than 1ms, and the end-to-end response latency from content recognition to boundary blocking is less than 50ms. For the first time, millisecond-level real-time monitoring of multimodal content recognition is achieved at the large-scale network boundary.

[0016] A significant leap forward in the detection rate of implicit ad splitting: Abandoning the traditional feature relevance fusion paradigm, it innovatively employs an ad semantic causal dependency graph and cross-modal causal chain reasoning mechanism to determine ad behavior based on the complete logical chain of ad dissemination. Combined with adversarial example adaptive techniques such as trimodal voting correction, occluded QR code repair, and voice-changing correction, the detection rate of implicit ad splitting reaches over 95%, an improvement of more than 40 percentage points compared to traditional relevance fusion schemes, with an adversarial example false negative rate of less than 5%.

[0017] Establishment of dynamic risk perception capability: Introduce a dynamic risk field assessment module, which realizes the continuous accumulation and decay of advertising risk based on the exponential smoothing model, effectively detects progressive and intermittent advertising, avoids false alarms and false negatives caused by single static judgment, and improves the detection rate of such advertising by more than 30%.

[0018] Identifying the collective behavior of black market distribution networks: Constructing a propagation graph and using graph attention networks to detect anomalies in the collaborative propagation patterns of multiple accounts and multiple links. Even if a single piece of content is perfectly disguised, it can be identified as long as its propagation topology shows an abnormal pattern, thus achieving a comprehensive crackdown on organized black market distribution sub-graphs.

[0019] End-to-end closed-loop governance for detection and handling: Based on robust content fingerprinting, cross-platform propagation link tracing is achieved. Content-level precise blocking rules (accurate to a single URL path) are issued through standardized security device docking protocols, with a false blocking rate of less than 0.1%. The blocking rules have their own validity period and can be rolled back with one click, achieving millisecond-level closed-loop governance of "detection and blocking".

[0020] Model self-evolution and rapid adaptation capability: The LoRA low-rank adaptation small sample incremental training mechanism is adopted, combined with the automatic synchronous update of the boundary fingerprint database. The adaptation cycle of new advertising samples is shortened from the traditional 1-2 weeks to less than 4 hours, and the system maintains a high detection rate. Attached Figure Description

[0021] Figure 1 This is a diagram illustrating the overall layered logical architecture of the system in this embodiment. Figure 2 The following is a flowchart of the multimodal deep analysis and causal fusion recognition module in the example. Figure 3 This is a schematic diagram of the traceability and linkage handling module in the example. Detailed Implementation

[0022] The present invention will now be described with reference to embodiments. It should be understood that the embodiments described herein are for the purpose of more clearly illustrating the invention, and not for limiting the scope of the invention.

[0023] The endpoints and any values ​​of the ranges disclosed herein are not limited to the precise ranges or values, and these ranges or values ​​should be understood to include values ​​close to these ranges or values. For numerical ranges, the endpoint values ​​of the various ranges, the endpoint values ​​of the various ranges and individual point values, and individual point values ​​can be combined with each other to obtain one or more new numerical ranges, which should be considered as specifically disclosed herein.

[0024] Reference Figure 1-3 This invention provides a real-time monitoring system for internet information based on multimodal content recognition, comprising a five-layer logical architecture, from top to bottom as follows: Boundary Traffic Awareness and Pre-screening Layer: Deployed at the network boundary, it is responsible for line-speed traffic access, protocol parsing, multimodal lightweight fingerprint extraction, and intelligent traffic offloading. More than 90% of normal traffic is allowed to pass directly at this layer. Multimodal deep parsing and causal fusion identification layer: only handles suspicious traffic from the off-line, and is responsible for single-modal deep semantic parsing, cross-modal causal chain reasoning, adversarial sample correction, and outputs accurate identification results; Dynamic Risk Field Assessment Layer: Based on the time series exponential smoothing model, it integrates multi-dimensional risk scores to construct a continuously updated risk field, realizing the dynamic accumulation and decay of advertising risks; Propagation Graph Construction and Anomaly Inference Layer: Construct a propagation graph containing multiple types of nodes such as content, accounts, links, devices, and networks, and detect collective collaborative behavior and abnormal propagation patterns through graph neural networks; Intelligent decision-making and source tracing linkage layer: responsible for hierarchical decision-making, source tracing of the transmission chain, linkage blocking of boundary devices, and closed-loop feedback of manual review results.

[0025] In addition, a control and data support layer runs through the entire system, responsible for configuration management, model iteration, log auditing, sample library maintenance, and supporting the system's autonomous evolution.

[0026] Module 1: Boundary Flow Awareness and Pre-screening Layer. This module consists of the following units: 1.1 Traffic Access and Protocol Parsing Unit High-performance driver: User-space traffic processing is implemented based on DPDK22.11LTS version, with the rte_mbuf memory pool zero-copy mechanism enabled, 2MB big page memory configured, and a single queue 64-byte packet processing throughput of 14.88Mpps; supports 9000-byte giant frame adaptation to reduce packet fragmentation overhead; supports 10G / 100G Ethernet interface line-speed access, and is compatible with both serial and bypass mirroring deployment modes.

[0027] Flow table management: It adopts a bidirectional flow table with "five-tuple + protocol type" and is based on the LRU eviction mechanism. The flow table capacity supports millions of concurrent sessions. The flow timeout time is dynamically configured according to the protocol (30s for short HTTP streams and 300s for long video streams) to complete the traffic session-level reassembly.

[0028] Protocol and Encrypted Traffic Parsing: Supports identification of mainstream protocols such as HTTP / HTTPS / QUIC / RTSP / RTMP / HTTP2 / HTTP3; for encrypted traffic, complete decryption is not required. By extracting SNI, ALPN, certificate fingerprints (JA3 / JA3S) from the TLS handshake phase, as well as application layer packet length sequence and time interval features, combined with a pre-trained traffic classification model, it identifies business types (short videos, live streams, web pages) and extracts payload fragmentation features for the corresponding modality, solving the challenge of encrypted traffic content detection. The detection rate of encrypted traffic advertising features is ≥85%.

[0029] Output standardized traffic metadata and payload fragments for subsequent fingerprint extraction.

[0030] 1.2 Multimodal Lightweight Fingerprint Extraction Unit Core logic: Without decoding the complete business content, only extracting low-dimensional semantic fingerprints from traffic metadata and payload sharding, the computing power overhead is only 5%~8% of that of full deep parsing.

[0031] Lightweight text-modal fingerprinting: It adopts character-level 2-gram + 3-gram hybrid feature extraction, performs sliding window sampling on plaintext segments of URL, HTTP header fields, and payload, and generates 64-bit semantic hash values; the Bloom filter is configured with 8 independent hash functions, and the false positive rate is controlled below 0.01%. It has a built-in feature library of 100,000+ advertising keywords, homophones, and traffic-driving phrases; additionally, URL path entropy value and domain registration time features are introduced as auxiliary scoring dimensions to improve the identification rate of suspicious domains.

[0032] Lightweight Visual Modal Fingerprint: Employs a lightweight student model distilled from SigLIP-Base, with a 3-layer MobileNetV2 backbone. Knowledge distillation compresses the original 768-dimensional features to 128 dimensions, achieving an accuracy loss of <2% on advertising image matching tasks. For scenarios with incomplete image segments, only the header data of the first two TCP segments of the image is extracted and input into the model, eliminating the need to wait for the complete image transmission and enabling streaming feature extraction. Feature matching uses the FAISS vector retrieval library and constructs an IVF_FLAT index, with a single-frame feature comparison time of <0.1ms. The fingerprint database supports the storage of millions of advertising image features.

[0033] Lightweight audio modal fingerprint: Extraction parameters: frame length 25ms, frame shift 10ms, extracting 8-dimensional MFCC coefficients + 1-dimensional energy features, for a total of 9 basic features; Voiceprint hashing uses a perceptual hashing algorithm to generate a 64-bit audio fingerprint from 10 consecutive frames of features, and matching uses Hamming distance calculation, with a threshold ≤10 for similarity; Built-in mainstream advertising voiceprint templates and specific dubbing voiceprint templates, totaling 5000+, supporting fast matching.

[0034] Hardware acceleration: The feature extraction logic is offloaded to the FPGA to achieve parallel extraction of multimodal features and compress the single message processing latency to the microsecond level.

[0035] 1.3 Intelligent Diversion Unit Weighted scoring model: text modality weight 0.4, visual modality weight 0.35, audio modality weight 0.25. After normalizing the suspicion of each modality to the [0,1] interval, the weighted sum is obtained to obtain a comprehensive risk score of 0~100.

[0036] Dynamic threshold mechanism: The system dynamically adjusts the traffic splitting threshold based on the current link bandwidth load and the deep engine computing power utilization rate: When the deep engine load is >80%, the threshold is automatically raised to reduce traffic splitting, and when the load is <30%, the threshold is lowered to expand detection coverage and ensure stable system operation; the default two-level threshold is: low threshold of 30 points and high threshold of 80 points.

[0037] The three-level traffic diversion logic is as follows: below the low threshold: normal traffic, directly allowed to return to the original link, accounting for ≥90%; between the high and low thresholds: suspicious traffic, diverted to the deep identification layer through mirror / GRE tunnel; above the high threshold: high-risk traffic, directly triggering temporary boundary control (such as limiting the rate to 10% of the original bandwidth), and simultaneously sent to the deep layer for review.

[0038] Technical results: The computing power pressure of deep recognition is reduced by more than 90% at the architecture level, the processing latency of a single traffic flow at the boundary layer is less than 1ms, and the detection rate of encrypted traffic advertising features is ≥85%, providing an architectural foundation for real-time detection in high-bandwidth scenarios.

[0039] Module 2: Multimodal Deep Analysis and Causal Fusion Recognition Layer. This module consists of the following units: Content restoration and single-modal analysis unit Content restoration: Complete session reconstruction and content decoding are performed on suspicious traffic to separate independent text, image, audio and video streams; For video streams, a perceptual hash algorithm is used to calculate the similarity between adjacent frames. If the Hamming distance is >20, it is determined to be a scene switch and extracted as a key frame. On average, one frame is extracted every 1 to 3 seconds, compressing the amount of visual detection data to 1 / 8 to 1 / 5 of the original.

[0040] Single-modal semantic parsing: Text: The text model for the advertising domain is finely tuned using Qwen-1.8B (LoRA fine-tuning, trained with 2 million labeled samples). It supports adversarial text recognition such as variant characters, character splitting, homophones, and reverse ordering. It extracts semantic nodes such as "product, efficacy, lead generation script, and contact information" with an extraction accuracy of ≥96%. Vision: A combination of YOLOv8n and PP-OCRv4 was used to perform object detection and text recognition respectively; for occluded QR codes, a Diffusion-based local repair model was used, combined with contextual semantic constraints to complete the occluded area, which improved the recognition rate by 35% after repair. Audio: The ad speech model is finely tuned using Whisper-small, which supports Mandarin, dialects, and voice-changing audio transcription with a word error rate of less than 5%. At the same time, it extracts voiceprint features to match the traffic-generating account template.

[0041] 2.2 Cross-modal causal fusion reasoning unit Specific implementation steps: Step 1, Pre-build an advertising semantic causal dependency graph Node System: Adopting a four-level hierarchical structure, a total of 4 major categories and 28 subcategories of semantic nodes are defined, fully covering the entire logic of advertising dissemination: Scenario Foundation Layer (4 categories): life scenarios, disease pain points, consumer needs, and hot events; Product Display Layer (8 categories): Physical Goods, Virtual Services, Brand Logos, Product Packaging, Usage Demonstrations, Ingredient Displays, Qualification Displays, Celebrity Endorsements Efficacy Description Layer (10 categories): Functional effects, medical efficacy, benefit promises, promotional activities, scarcity implications, comparative effects, user testimonials, expert endorsements, parameter descriptions, and usage instructions. Traffic generation and conversion layers (6 categories): contact information, purchase link, QR code scanning guidance, offline address, account follow, live stream guidance Edge definition and weight: Causal progression edge: connects nodes at different levels, represents semantic and logical support relationship, with a basic weight of 0.8~1.0, which is dynamically adjusted according to the node correlation. Modal complementary edges: connect nodes of different modalities at the same level, representing multiple modalities jointly supporting the same semantics, with a weight of 0.6~0.9; Temporal sequence edges: connect nodes of the same modality at different temporal sequences, representing the semantic progression order, with a weight of 0.5~0.7; Graph construction method: Based on the Advertising Law and industry regulatory rules, rule nodes are extracted, and then entity alignment and edge weight training are performed using 100,000+ labeled advertising samples to form an initial inferable graph.

[0042] Step 2: Multimodal semantic node alignment A cross-modal attention alignment network is used to map the semantic node features of text, vision, and audio to a unified 512-dimensional semantic space; Nodes of different modalities are bound in a 2-second time window. The cosine similarity between nodes is calculated. If the similarity is >0.7, they are determined to be corresponding nodes of the same semantic level, and the graph mapping is completed. For scenarios with missing modalities (such as plain text without audio), zero vectors are used for padding, which does not affect the link inference logic.

[0043] Step 3: Causal Link Reasoning Determination Inference backbone: Graph Attention Network (GAT) is used. The input is node features, edge features and adjacency matrix. The output is the confidence score of each complete causal link.

[0044] Link confidence calculation formula: ; in Let be the confidence level of the i-th node. denoted as node weight, N as the total number of link nodes, C as the cross-modal complementarity coefficient, and T as the temporal coherence coefficient.

[0045] Judgment rules: When the link contains two core layer nodes, "product display layer" and "traffic conversion layer", and the overall confidence level S≥0.75, it is judged as advertising content; if it contains nodes with illegal effects, it is further judged as illegal advertising of the corresponding level.

[0046] Intermodal thinking chain enhancement: During the reasoning process, node features from different modalities are introduced alternately for cross-validation. If the confidence of a certain modal node is low, it can be supplemented by complementary nodes from other modalities to improve the recall rate of implicit ads.

[0047] 2.3 Adversarial Example Adaptive Correction Unit By using multimodal feature cross-validation, the semantics of adversarial samples are automatically repaired, allowing adaptation to novel evasion techniques without manual annotation. The three-modal voting correction mechanism: For semantic nodes with a single modality recognition confidence of <0.6, joint reasoning is performed by combining the features of the other two modalities; for example, if the confidence of text recognition of homophones is 0.5, and the audio pronunciation matches the target words and the visual appearance of the corresponding product image, then the voting determines that the semantic node is valid and corrects the confidence to 0.85.

[0048] QR code occlusion correction: Based on the semantic probability of the surrounding text, if the semantic confidence of the traffic-driving function is >0.8, the QR code repair model is activated to perform generative completion of the occluded area before recognition is performed.

[0049] Voice correction: By combining the lip key point sequence (lip movement features) of video frames with the features of subtitle text, the ASR recognition results are corrected, and the word error rate can be reduced by more than 40%.

[0050] Technical results: The detection rate of implicit split ads is ≥95%, which is more than 40% higher than that of traditional relevance fusion schemes; the false negative rate of adversarial samples is <5%, and it can be adapted to new avoidance methods without large-scale manual annotation.

[0051] Module 3: Dynamic Risk Field Assessment Layer, Module Implementation Process: 3.1 Unifying the event model and aligning it with the time window Abstract all objects to be monitored in the system into a unified event. ,in The event identifier is T, the timestamp is S, the source node is C, the content channel is P, the original payload is M, the metadata is M (including the publishing account, device fingerprint, IP, domain name, access frequency, etc.), the link information is L (redirect chain, landing page, etc.), and the behavior record is A (posting frequency, comment rhythm, etc.).

[0052] A uniform time window W (e.g., 1 second, 2 seconds, or 5 seconds) is introduced as the basic analysis granularity; within each time window, the system collects and aligns multimodal fragments from the same event chain. Let the modality set within a certain time window be denoted as . , respectively representing text Image / video frames Audio ,Behavior Network feature fragments .

[0053] 3.2 Multi-dimensional risk scoring The risk is broken down into five basic score categories, and then the overall risk is calculated: Textual risks : Ad keyword density, frequency of inducement words, abnormality of synonym substitution, appearance of contact information / redirect words / purchase words, and intensity of emotional incentive in the text; It can be written as: ; Where K represents keyword features, I represents lead words features, S represents synonym substitution features, C represents contact information features, and E represents emotional motivation features.

[0054] Visual risks : QR code ratio, product display probability, promotional poster elements, highlighted price / discount labels, consistency between text and images on the screen.

[0055] Audio risks : Promotional statement density, transaction guidance word density, duration of spoken content, consistency between audio and video content, and detection of splicing artifacts.

[0056] Behavioral risk : Account posting frequency, repetition rate of homogeneous content, degree of cross-platform synchronization, abnormal comments / forwards, batch account collaborative behavior, and concentrated posting in a short period of time.

[0057] Cyber ​​risks : Number of redirection chain layers, domain name anomalies, number of redirects, frequency of abnormal access, and intensity of cross-domain traffic redirection.

[0058] Overall risk value: The coefficients can be dynamically adjusted according to the scenario (e.g., increase β and γ for live streaming advertising scenarios, and increase α and η for email advertising scenarios).

[0059] 3.3 Risk Field Continuous Update Mechanism Risk is defined as a "risk field" that changes over time, and an exponential smoothing model is used: ; in For the current time window of risk, For the risks of the previous time window, This adds new risk increments to the current window (derived from new suspicious content, new high-risk image elements, new abnormal traffic behavior, manual review and correction results, etc.), where λ is the historical inertia coefficient (usually taken as 0.7~0.9). This continuous update method is particularly suitable for the following scenarios: advertising content gradually appears in the live broadcast room; traffic-driving behavior gradually spreads in the comment section; short links constantly change landing pages during the dissemination process; and content is normal at first, and then marketing links are suddenly inserted.

[0060] Technical effects: Enables dynamic accumulation and attenuation of advertising risks, improves the detection rate of gradual and intermittent advertising by more than 30%, and avoids false alarms and false negatives caused by single judgment.

[0061] Module 4: Propagation Graph Construction and Anomaly Inference Layer, Implementation Process: 4.1 Definition of Graph Nodes: The propagation graph G=(V,E) includes the following node types: Content nodes: posts, web pages, videos, images, emails; Account nodes: user accounts, robot accounts, collaborative accounts; Link nodes: short links, QR codes, domain names, URLs; Device nodes: device fingerprints, terminal types; Network nodes: IP addresses, network segments, proxy nodes; Handling nodes: alarms, reviews, bans, blocking records.

[0062] 4.2 Graph edge definition: including at least: publishing relationship, forwarding relationship, traffic redirection relationship, jump relationship, co-occurrence relationship, same origin relationship, collaborative relationship, and reuse relationship.

[0063] 4.3 Anomaly Detection in the Graph: If the following patterns appear in the graph, it is determined that the risk of propagation has increased: the same content is copied across accounts in a short period of time; multiple accounts point to the same link at the same time; multiple redirect links eventually converge to the same landing page; highly similar referral phrases appear in the comment section; and there are synchronous publishing characteristics between accounts on different platforms.

[0064] 4.4 Graph Scoring: Defining Graph Propagation Risk: ; Where V is the set of nodes and E is the set of edges. The time window is defined as follows: when a high-density, multi-node, unidirectional propagation pattern appears within a short time window, Rg increases. Specifically, graph neural networks (such as GATv2) can be used to score anomalies in subgraphs and output subgraph-level risk scores.

[0065] Technical effect: Enables the detection of collective behavior in black market advertising networks. Even if a single piece of content is perfectly disguised, it can be identified as long as its propagation pattern is abnormal.

[0066] Module 5: Intelligent Decision-Making and Source Tracing Linkage Layer. This module consists of the following units: 5.1 Hierarchical Decision-Making Unit: Comprehensive Causal Fusion Confidence S, Dynamic Risk Field Risks of transmission patterns Establish a three-tiered response threshold: High confidence violation: S≥0.9 or ≥80 or ≥0.8, automatic blocking action will be triggered; Low confidence level is suspicious: 0.75 ≤ S < 0.9 or 60 ≤ If the number of cases is less than 80, a manual review will be initiated, and the corresponding traffic will be temporarily throttled (reduced to 10% of the original bandwidth) to slow its spread. Normal content: If the content is below the above threshold, it will be marked as allowed and only the log will be retained.

[0067] Custom rule engine: Supports configuring special detection rules based on regular expressions and feature tag combinations (such as special rules for medical advertisements and financial advertisements), adapting to different industry regulatory standards.

[0068] 5.2, Propagation Link Tracing Unit Two-dimensional source tracing feature system: Traffic layer characteristics: 5-tuple, TTL value, route hop count, DNS resolution record, HTTPReferer redirection chain, UA fingerprint; Content layer features: A robust 256-bit content fingerprint generated based on multimodal fusion features, which is resistant to compression, cropping, and secondary editing, and can be matched across platforms.

[0069] Topology reconstruction algorithm: Breadth-first search (BFS) is used. The fingerprint of the violation content is used as the anchor point to match all sessions with the same fingerprint in the full traffic log. First, the source node with the first fingerprint is traced in reverse, and then all distribution nodes and jump paths are sorted in forward to generate a complete directed acyclic graph (DAG).

[0070] Source tracing accuracy: It can pinpoint the specific publishing IP, domain name, URL path, and the specific account ID of the content platform (based on UA ​​and behavioral feature matching).

[0071] 5.3 Interlocking Interception Unit Standardized interfaces: Supports multiple mainstream security device interface protocols such as Netconf, RESTfulAPI, Syslog, and BGPFlowspec, and can seamlessly integrate with firewalls, WAFs, DNS servers, and traffic scrubbing devices.

[0072] Three-tiered precise blocking strategy: Precise URL blocking: Blocking rules are issued for single URLs and specific file paths, without affecting other normal services on the same domain, with a false blocking rate of <0.1%; Domain blocking: DNS hijacking and IP blocking are issued for maliciously distributed domains; Source link blocking: Full link blocking is issued for high-risk source IPs; Fault tolerance and rollback mechanism: Blocking rules are set to have a default validity period of 24 hours and automatically expire upon expiration; If a manual review determines that it is a false alarm, all associated blocking rules can be revoked with one click to minimize the impact on business.

[0073] Chain of evidence and audit retention: The evidence chain is solidified in four layers: "Original Layer - Analysis Layer - Reasoning Layer - Disposal Layer". The original layer stores the original page, original video, original audio, and original request records. The analysis layer stores the OCR results, ASR results, jump chains, and feature vector summaries. The reasoning layer stores the risk score, model version, rule hit status, causal reasoning path, and graph reasoning results. The disposal layer stores the alarms, rate limiting, blocking, freezing, and manual review conclusions.

[0074] Each layer of data generates a timestamp and hash value to facilitate subsequent auditing and review, meeting the Level 3 compliance requirements of the Cybersecurity Classified Protection 2.0 standard.

[0075] Technical effects: Achieves millisecond-level response for "discovery and blocking", with an end-to-end response latency of <50ms from content recognition to boundary blocking; blocking accuracy is improved by more than 80% compared to traditional IP-level blocking, avoiding false positives on normal services.

[0076] Module Six: Control and Data Support Layer, Implementation Process: 6.1 Unified configuration management: Visual configuration of traffic diversion thresholds, detection rules, handling strategies, and linked device parameters; supports multi-tenant permission management and adapts to different operation and maintenance roles.

[0077] 6.2 Model Gray Release: Supports version management and gray release of models and causal graphs, which can be gradually rolled out according to traffic ratio and business lines to avoid iteration affecting business.

[0078] 6.3 Full-link log audit: Raw traffic logs are retained for 7 days, and structured identification and processing logs are retained for 6 months, meeting the Level 3 compliance requirements of Information Security Protection 2.0; Supports backtracking of the entire process by time, IP, domain name, content fingerprint and other dimensions.

[0079] 6.4 Incremental Training and Self-Evolution Mechanism: Triggering conditions: Triggered daily at set times, or when the number of new samples reaches 500; Training method: LoRA low-rank adaptation is used for small sample fine-tuning, and only the low-rank matrix of the model's attention layer is updated. The training time for a single round is less than 2 hours, and it can be completed at edge nodes without the need for large cloud computing power. Effectiveness verification: Using an offline test set + 1% gray-scale traffic verification, if the accuracy decreases by <0.5% and the recall increases by >1%, it is considered effective and can be officially released; Boundary synchronization: After training a new sample, the corresponding lightweight fingerprint is automatically extracted, and the boundary layer Bloom filter and vector library are incrementally updated. The synchronization time is less than 5 minutes, realizing the synchronous upgrade of the entire link capability.

[0080] 6.5 Multimodal Advertising Sample Library: Stores labeled samples, adversarial samples, causal chain feature data, and propagation graph snapshots, and continuously and automatically expands to provide data support for model iteration.

[0081] 6.6 Manual Review and Feedback: The results of manual review are fed back to the system to correct false alarms / missed alarms, update risk thresholds, update the rule base, and update the training sample pool, forming a continuously evolving security closed loop.

[0082] Technical results: The adaptation period for new advertising samples has been shortened from the traditional 1-2 weeks to less than 4 hours, and the system continues to maintain a high detection rate.

[0083] As a specific embodiment of the present invention: real-time monitoring and blocking of hidden medical advertisements in short videos, deployment environment: Internet exit of a provincial operator, 2×10Gbps, system serial deployment.

[0084] Monitoring target: A short video of 15 seconds, which consists of "ordinary food packaging display + host verbally announcing the 'blood sugar lowering' effect + WeChat ID displayed in the subtitles".

[0085] Input data: The complete HTTP / HTTPS session of this video stream captured by the border traffic.

[0086] Step 1: Boundary Flow Awareness and Pre-screening 1.1 Traffic Access and Protocol Parsing Session 5-tuple: 192.168.1.100:54321→203.0.113.50:443, protocol TLSv1.3.

[0087] DPDK receives the message, completes session reassembly, and identifies it as a short video stream (based on packet length sequence and TLS fingerprint JA3).

[0088] Extracting side-channel features of encrypted traffic: SNI: cdn.example-cdn.com; JA3: 771,4865-4866-4867-49196-49195... (matching with known short video SDK fingerprints); Packet length sequence mean: 1420 bytes (video stream characteristic); IAT (packet interval) standard deviation: 2.3ms (normal live streaming / short video range).

[0089] 1.2 Multimodal Lightweight Fingerprint Extraction Text modality: Extract the URL fragment / video / abc123?utm_source=xxx from the :path of the HTTP / 2HEADERS frame, perform 2-gram+3-gram hashing on the URL, and generate a 64-bit fingerprint 0xA3F1...; Bloom filter query hit the "Ad Network Tracking Parameters" feature library (hit probability 0.008%), did not directly hit the illegal keyword, but the URL path entropy value is 4.2 (high), and the domain registration time is only 30 days (abnormal); Text modality suspiciousness score: 0.45 (normalized).

[0090] Visual modality: Extract the first two TCP fragments (approximately 2800 bytes) from the application data payload of the TLS record, input them into a lightweight MobileNetV2 model, and output a 128-dimensional feature vector. The cosine similarity between the FAISS search result and a certain health product advertisement in an advertising image fingerprint database (containing 1 million records) is 0.83 (greater than the threshold of 0.7). Visual modality suspiciousness score: 0.78.

[0091] Audio modality: Due to encryption, the audio fingerprint could not be extracted at the boundary layer and is set to 0 by default. However, the system infers the possible presence of audio based on the video stream type and retains the tag.

[0092] Overall risk score: =0.4×0.45+0.35×0.78+0.25×0=0.18+0.273+0=0.453, normalized to a percentage: 45.3 points.

[0093] 1.3 Intelligent traffic splitting The current depth engine load is 65% (medium). Dynamic thresholds: low threshold 30 points, high threshold 80 points; 45.3 points is between 30 and 80 points → suspicious traffic, which is mirrored to the depth recognition layer through the GRE tunnel; boundary layer processing latency: 0.8ms.

[0094] Step 2: Multimodal Deep Analysis and Causal Fusion Reasoning 2.1 Content Reconstruction and Single-Mode Analysis The sandbox rendering container (headlessChrome) fully loads the video landing page and retrieves: Video stream: 15 seconds, 30fps, 450 frames in total. Perceptual hashing detected 3 scene transitions (inter-frame Hamming distance > 20), and extracted 3 keyframes (5th second, 10th second, and 14th second).

[0095] Audio stream: 16kHz mono, 15 seconds long. Whisper-small transcription result: "This product is really good. Take one pill a day and your blood sugar will stay stable. Add WeChat xxxxx if you need it."

[0096] Subtitle text: Extracted from the video subtitle track: "To lower blood sugar, add WeChat: health999".

[0097] Single-modal semantic parsing results: Visual (keyframe at 5 seconds) Product Packaging (Food Category) 0.93 YOLOv8n detected the "ordinary food" label. Visual (keyframe at 10 seconds) Usage demonstration (brewing) 0.89 Normal consumption scenarios Visual (keyframe at 14 seconds) Contact information (QR code) 0.72 PP-OCRv4 recognized the QR code area, but it was partially obscured. Audio (full segment) Efficacy description (lowering blood sugar) 0.94 Medical efficacy, violating advertising laws Audio (full segment) Lead generation and conversion (add WeChat) 0.97 Please specify contact information Text (subtitles) Efficacy description (lowering blood sugar) 0.86 Clear text Text (subtitles) Lead generation and conversion (WeChat ID) 0.95 Contact information 2.2 Cross-modal causal fusion reasoning Step 1: Align semantic nodes to the causal graph The system maps the above nodes to a pre-constructed four-level causal graph: Scenario setup layer: No clear disease pain points or hot topics were detected, skip.

[0098] Product display layer: Node "Ordinary food packaging" (visual, confidence level 0.93) → Graph node P01.

[0099] Efficacy description layer: Node "Lowering blood sugar" (audio 0.94, text 0.86) → Graph node E03 (medical efficacy).

[0100] Lead generation and conversion layer: Node "Add WeChat" (audio 0.97, text 0.95) → Graph node C01 (contact information).

[0101] Step 2: Cross-modal complementarity and alignment Within the same time window (seconds 5-15), there is a modal complementarity between "product display" and "efficacy description": visually showcasing food, while audio claims therapeutic effects; the two are semantically inconsistent (ordinary food should not claim therapeutic effects). Cross-modal complementarity coefficient. =0.85 (high, because the "false advertising" logic is formed between modalities).

[0102] There is a causal relationship between "efficacy description" and "lead generation and conversion": after claiming therapeutic effects, the user is directed to add the other user on WeChat, which carries weight. =0.95.

[0103] Temporal coherence coefficient T: When nodes appear in chronological order (product → efficacy → traffic generation), the temporal coherence is T=0.92.

[0104] Step 3: Causal Link Reasoning The complete link constructed is: P01→E03→C01 Node confidence: =0.93, =0.90 (the weighted average of audio (0.94) and text (0.86), with a weight ratio of 0.6:0.4). =0.96.

[0105] Node weight: =0.8, =1.0, =1.2 (the traffic conversion layer has the highest weight).

[0106] Link confidence calculation: S = (0.93 × 0.8 + 0.90 × 1.0 + 0.96 × 1.2) / 3 × C × T =(0.744+0.90+1.152) / 3×0.85×0.92 =(2.796) / 3×0.782=0.932×0.782=0.729 Note: The cross-modal complementarity coefficient C here is taken as the geometric mean square root of 0.85 × 0.95 = 0.898, which is the coefficient of the "Product Display → Efficacy Description" edge (0.85) and the coefficient of the "Efficacy Description → Traffic Conversion" edge (0.95). However, for simplicity, 0.85 is used (conservatively). The actual system uses GAT network for dynamic calculation; this demonstration uses manual calculation.

[0107] The result is S=0.73 (slightly below the advertising threshold of 0.75). However, due to the intervention of the intermodal thinking chain enhancement: The confidence level for "lowering blood sugar" in the audio modality is 0.94, and in the text modality it is 0.86. The visual modality does not directly show the effect, but after trimodal voting correction (audio + text > 0.6), and with the visual representation of "product packaging," the "effect description" node is deemed valid, and the confidence level is corrected to 0.92. The calculation is then repeated. =0.92, then: S=(0.744+0.92+1.152) / 3×0.85×0.92=0.939×0.782=0.734, which is still slightly lower than 0.75. However, at this time, the system detects the QR code occlusion and triggers the adversarial correction unit: The confidence level of the QR code area was 0.72. Combined with the semantic probability of the surrounding text "Add WeChat" (0.95), the Diffusion repair model was activated to complete the occluded area and re-identify. The QR code content was "https: / / x.xx / xxxxx" (short link), and the confidence level of the traffic conversion node was increased to 0.98.

[0108] Recalculation: S = (0.744 + 0.92 + 1.176) / 3 × 0.85 × 0.92 = 0.947 × 0.782 = 0.740, still 0.01 short. However, the system further discovered that the audio "Add WeChat xxxxx if you need it" and the subtitle "Add WeChat: health999" point to the same WeChat ID, and the visual QR code points to the short link. The three form strong modal complementarity, and the cross-modal complementarity coefficient increases to 0.92 (GAT network dynamically adjusts). Ultimately: S=0.947×0.92×0.92=0.801, S=0.80≥0.75, which is determined to be advertising content, and since it contains the medical efficacy node (E03), it is further determined to be a false medical violation advertisement.

[0109] 2.3 Adversarial Example Correction Effect QR code obscured: After repair, recognition was successful, and the confidence level of the referral node increased from 0.72 to 0.98.

[0110] Although the audio and text expressions for "lowering blood sugar" have slight variations (the audio is "blood sugar is stable" and the text is "lower blood sugar"), the semantic vector similarity is 0.81, indicating successful alignment.

[0111] Step 3: Dynamic Risk Field Assessment 3.1 Current Event Risk Increment According to the five-dimensional risk scoring model: Textual risks The subtitles contain "lower blood sugar" and "add WeChat," with a keyword density of 0.12, a frequency of leading words of 0.08, and an emotional stimulation intensity of 0.7 to 10.5%. =0.68; Visual risks QR codes account for 5% of the product display rate, with a product display probability of 0.9, and no price tags are highlighted. =0.55; Audio risks Promotional phrase density: 0.15; Transaction guidance phrase density: 0.10; Voiceover duration: 12 seconds (80%); Low consistency between audio and visual content (visuals: food; audio: medical). =0.76; Behavioral risk This account posted 5 times in the past hour, all of which were similar videos, with a content repetition rate of 0.85%. =0.70; Cyber ​​risks : Redirect chain level 3 (landing page → short link → final page), domain registered for 30 days, number of redirects 2 → Rn=0.74.

[0112] Overall risk (all coefficients taken as 0.2): ΔR = 0.2 × (0.68 + 0.55 + 0.76 + 0.70 + 0.74) = 0.2 × 3.43 = 0.686 3.2 Risk Field Update Assuming the risk field of this event chain in the previous time window (5 minutes ago) =0.32, historical inertia coefficient λ=0.8: =0.8×0.32+0.2×0.686=0.256+0.137=0.393; The risk field value is 0.393 (normalized from 0 to 1), corresponding to a score of 39.3 out of 100. This score does not exceed the high threshold of 80, but exceeds the low threshold of 30, and is therefore placed in the observation queue. Since the causal inference has been judged as a violation (S=0.80), the system prioritizes the causal inference result.

[0113] Step 4: Propagation Graph Construction and Anomaly Inference (Substituting Actual Data) 4.1 Graph Nodes and Edges The system retrieved traffic logs from the past 24 hours and found: Account node: The account user_abc posted 12 similar videos in the past 24 hours.

[0114] Link nodes: Landing page domain example-cdn.com, short link x.xx / xxxxx, final page mall.example.com / product?id=123.

[0115] Co-occurrence: Three other accounts (user_def, user_ghi, user_jkl) posted videos with different content in the past 6 hours, but all ultimately redirected to the same final page, mall.example.com / product?id=123.

[0116] Collaborative relationship: The posting intervals of the four accounts were all 15±3 minutes, showing a regular rhythm.

[0117] 4.2 GNN Anomaly Subgraph Detection Constructing a subgraph It contains 4 account nodes, 3 link nodes, and 1 final page node; node feature vector (partial): user_abc:[SusScore_Net=0.45,creative_darkflag=0.78,cmca_conflict_flag=1,dom_depth=2,redirect_depth=3,dga_score=0.1,ip_rotation_rate=0.05] user_def:[0.38,0.71,1,2,3,0.12,0.04] ... GATv2 outputs a subgraph risk score after two layers of message passing. =0.84 (threshold 0.8). This indicates an organized black market advertising campaign.

[0118] Comprehensive decision-making Confidence level of causal inference S = 0.80 (≥ 0.75); dynamic risk field =0.393 (<0.8, but causal reasoning is dominant); propagation graph risk =0.84 (≥0.8) If two out of the three criteria reach the high threshold, the system will determine it as a high-confidence violation and trigger automatic blocking.

[0119] Step 5: Source Tracing and Joint Response Source tracing of the transmission chain Content-robust fingerprint: 256-bit hash value 0x4F2A... generated based on multimodal fusion features.

[0120] A total of 47 sessions (in the past 2 hours) were matched with the same fingerprint in the full traffic log, distributed across different IPs and domains.

[0121] BFS reverse tracing: The first source IP to show this fingerprint was 198.51.100.10 (suspected publishing source), and forward tracing revealed 3 distribution CDN nodes and 2 redirect short link service providers.

[0122] Source tracing results: Origin IP 198.51.100.10, domains example-cdn.com and mall.example.com, associated with 4 accounts.

[0123] 5.2 Linked Interception Send the following to the firewall via Netconf: Precise URL blocking: mall.example.com / product?id=123 (does not affect other paths on the same domain); Domain blocking: example-cdn.com (DNSSinkhole); Source IP blocking: 198.51.100.10 (entire link blocking, valid for 24 hours). Send alert events to SIEM via Syslog, containing a complete chain of evidence; End-to-end time: 31ms from receiving the boundary layer data to the blocking taking effect.

[0124] Step 6: Solidification and Self-Evolution of the Chain of Evidence 6.1 Four-layer chain of evidence Original layer Full video stream (15 seconds MP4), HTTP request / response headers, TLS handshake packet sha256:0xAB... parsing layer OCR results (comparison before and after QR code repair), ASR transcribed text, and JSON redirection links. sha256:0xBC... Inference layer Causal graph inference path (node ​​alignment matrix, GAT attention weights), risk field time series, GNN subgraph scoring sha256:0xCD... Disposal layer Blocking rule distribution logs, firewall response confirmation, and manual review queue entries. sha256:0xDE... 6.2 Self-evolutionary iteration The sample (food packaging + medical voiceover + QR code referral) was automatically added to the sample database and marked as "new type of split-style false medical advertisement".

[0125] Incremental training was triggered at 2:00 AM that day: LoRA fine-tuned the Qwen-1.8B text model, added 200 similar samples, and the training took 1.8 hours.

[0126] The new model improved recall from 92.3% to 93.1% on the offline test set, while accuracy decreased by 0.3% (within acceptable limits), and was officially released.

[0127] Simultaneous extraction of lightweight visual fingerprint (128 dimensions) and updating of boundary layer FAISS index took 4 minutes.

[0128] The following day, similar ads were detected by visual fingerprints at the boundary layer (similarity 0.79) and directly diverted to the depth layer, increasing the detection rate at the depth layer to 98.5%.

[0129] This invention solves the technical problems of insufficient real-time performance with high bandwidth, high rate of missed detection of hidden advertisements, and disconnect between detection and handling in existing technologies, and achieves millisecond-level real-time monitoring, a hidden advertisement detection rate of over 95%, and closed-loop governance across the entire chain.

[0130] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A real-time monitoring system for internet information based on multimodal content recognition, characterized in that, include: The boundary traffic awareness and pre-screening module is deployed at the network boundary to access Internet traffic at line speed and perform protocol parsing and session reassembly on traffic sessions. It extracts lightweight semantic fingerprints of text modality, visual modality and audio modality from the traffic, calculates a comprehensive risk score based on the suspiciousness of each modality fingerprint, and performs hierarchical traffic diversion according to the comprehensive risk score. Normal traffic with a comprehensive risk score below the first threshold is directly allowed to return to the original link, while suspicious traffic with a comprehensive risk score reaching the first threshold is diverted to the deep recognition engine. The multimodal deep analysis and causal fusion identification module is used to complete content restoration and single-modal semantic analysis of the suspicious traffic in the diversion. It extracts several semantic nodes from the analysis results and maps the semantic nodes to a pre-constructed advertising semantic causal dependency graph for cross-modal causal chain reasoning. When the formed candidate causal link contains both product display layer nodes and traffic conversion layer nodes and the link confidence is not lower than the preset judgment threshold, the corresponding traffic content is determined to be an illegal advertisement. The source tracing and coordinated handling module is used to generate robust content fingerprints that are resistant to compression and pruning based on cross-modal fusion features. Using these robust content fingerprints as anchors, it matches sessions with the same fingerprints in traffic logs to trace the source node in reverse and sort out the distribution nodes in forward, generating a propagation link topology. Then, it issues content-level precise blocking rules to network boundary firewalls, DNS servers, or traffic scrubbing devices through security device docking protocols.

2. The real-time monitoring system for internet information based on multimodal content recognition according to claim 1, characterized in that, In the boundary traffic perception and pre-screening module: the text modal fingerprint is obtained by generating semantic hash values ​​through character-level n-gram sampling and matching them with a Bloom filter using a built-in advertising keyword and homophonic coded language feature library; The visual modal fingerprint is obtained by extracting incomplete image data from the first few TCP segments of the image payload, inputting it into a lightweight visual coding model to output a low-dimensional feature vector, and performing an approximate nearest neighbor search in a pre-built vector index library. The audio modal fingerprint is obtained by extracting MFCC coefficients and energy features from the audio payload in frames, generating a perceptual hash sequence, and then matching it with the Hamming distance of the spoken template fingerprint library. The comprehensive risk score is the result of summing the text modality suspicion, visual modality suspicion, and audio modality suspicion with preset weights. The hierarchical traffic diversion also has a second threshold higher than the first threshold: when the comprehensive risk score reaches the second threshold, the boundary traffic perception and pre-screening module also performs temporary boundary control on the current session.

3. The real-time monitoring system for internet information based on multimodal content recognition according to claim 1, characterized in that, The node system of the advertising semantic causal dependency graph includes at least the following levels arranged in causal logical order: scene preparation layer, product display layer, efficacy description layer, and traffic conversion layer; wherein, the causal progression edge connecting nodes of different levels is only allowed to be connected in the direction of orderly advancement from higher level to lower level or within the same level, and is assigned a basic weight of 0.8~1.0; the modal complementary edge connecting nodes of different modalities in the same semantic layer is assigned a weight of 0.6~0.

9.

4. The real-time monitoring system for internet information based on multimodal content recognition according to claim 3, characterized in that, The multimodal deep parsing and causal fusion identification module uses the following formula to calculate the link confidence of candidate causal links: ； in Let be the confidence level of the i-th node. S is the node weight, N is the total number of nodes in the link, C is the cross-modal complementarity coefficient, and T is the temporal coherence coefficient. When S≥0.75, it is judged as advertising content. When the link contains a function description layer node of a legally illegal function type, it is further judged as illegal advertising.

5. The real-time monitoring system for internet information based on multimodal content recognition according to claim 1, characterized in that, The multimodal deep parsing and causal fusion identification module also includes an adversarial example adaptive correction unit, used for: For disputed semantic nodes in a single modality whose confidence level is lower than the first confidence threshold but higher than the second confidence threshold, the confirmed nodes in the other two modalities will jointly vote to correct them. For occluded QR code areas in the visual modality, when the semantic probability of the surrounding text exceeds a preset threshold, local repair and completion based on a generative model is initiated before QR code recognition is performed.

6. The real-time monitoring system for internet information based on multimodal content recognition according to claim 1, characterized in that, Also includes: The dynamic risk field assessment module is used to maintain continuous risk values ​​over a time window sequence for each monitored object. According to the exponential smoothing relationship Update, among which This refers to the incremental risk within the current time window, calculated by weighting textual risk, visual risk, audio risk, behavioral risk, and network risk. when When the risk level continues to rise but does not reach the judgment threshold, the dynamic risk field assessment module notifies the boundary traffic perception and pre-screening module to lower the first threshold to expand the sampling coverage of suspicious traffic; when When the threshold value is reduced to a low level, the default threshold is restored.

7. The real-time monitoring system for internet information based on multimodal content recognition according to claim 1, characterized in that, The source tracing and coordinated handling module also includes a propagation graph construction and anomaly inference unit, used for: Construct a propagation graph whose node set at least includes content nodes, account nodes, link nodes, and network nodes, and whose edge set at least includes publishing relationships, redirection relationships, co-occurrence relationships, and collaboration relationships; Using the hit results of the robust content fingerprint in the full traffic log as seeds, a graph attention network is used to score the anomalies of local subgraphs around the seed in the propagation graph. When the subgraph shows a topology pattern of multiple accounts converging on the same landing page or the same origin attribution ID co-occurring across domains and the anomaly score exceeds the graph threshold, the judgment is upgraded to an organized black market delivery subgraph, and all related domains and attribution nodes in the subgraph are included in the blocking scope.

8. The real-time monitoring system for internet information based on multimodal content recognition according to claim 1, characterized in that, The content-level precise blocking rule is an interception rule that is accurate to a single URL path or a specific file path. This interception rule is configured as a temporary rule with an expiration date, which will automatically expire. When manually reviewed and determined to be a false alarm, it can be revoked with one click.

9. The real-time monitoring system for Internet information based on multimodal content recognition according to claim 1, characterized in that, The boundary traffic sensing and pre-screening module uses DPDK user-space driver to achieve line-speed traffic processing and offloads the multimodal fingerprint extraction logic to the FPGA hardware acceleration unit, with a single message processing latency of less than 1 microsecond.

10. A real-time internet information monitoring system based on multimodal content recognition according to claim 1, characterized in that, The multimodal deep analysis and causal fusion recognition module adopts an incremental training and self-evolution mechanism: when the number of new samples reaches a preset number, the samples are fine-tuned through LoRA low-rank adaptation. After training is completed, the corresponding fingerprints are automatically extracted and the feature library in the boundary traffic perception and pre-screening module is updated synchronously. The synchronization time is less than 5 minutes.

Images