A video traffic security management and control method and system applied to a learning machine terminal

Abstract

The application provides a video traffic safety management and control method and system applied to a learning machine terminal, which can efficiently and accurately distinguish learning and entertainment video content. The method comprises: performing deep packet detection on the video traffic data packet of the video stream received by the learning machine terminal to extract the traffic characteristics of the video stream, matching the traffic characteristics with a preset list library to obtain the confidence degree of the traffic characteristics belonging to the list library; if no preset list library is matched, based on a sliding time window, frame extraction processing is performed on the video segment of the video stream located in the current sliding time window, and a neural network model is used to perform word segmentation and vectorization on the text corresponding to the video segment to obtain text features; a candidate frame sequence containing a preset number of candidate frames is obtained, a multi-modal large model based on a cross-attention mechanism is used to perform multi-modal feature fusion reasoning on the candidate frame sequence to obtain the probability distribution of each content category; and based on the probability distribution, corresponding management and control operations are performed on the video segment.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to a method and system for video traffic security management applied to learning machine terminals. Background Technology

[0002] With the rapid development of online education technology and the widespread adoption of smart terminal devices, learning machines designed specifically for student learning have become important auxiliary tools. However, the quality of video content in the internet environment varies greatly. Even within "learning" applications (APPs), videos often embed or promote entertainment videos unrelated to education or contain inappropriate content. Therefore, how to accurately and effectively manage the network video traffic of learning machine terminals to ensure the health and purity of the content students access has become a crucial issue in the current educational hardware field.

[0003] Currently, the control of video content on learning devices mainly employs blacklist filtering technology based on Uniform Resource Locators (URLs) or Internet Protocol (IP) addresses. This approach maintains a large blacklist database, recording the domain names or IP addresses of known malicious websites and video websites. When a terminal initiates a network request, the target address is parsed in real time and compared with records in the blacklist; if a match is found, the network request is blocked. However, this technology can only control the address of the accessing terminal and cannot deeply analyze the video content itself, resulting in blind spots in content recognition. For example, when a student accesses an educational platform that is on a whitelist, if the platform's page embeds entertainment video advertisements or non-educational user-generated content (UGC), the IP / URL-based firewall may allow access because the parent domain is legitimate, leading to ineffective control.

[0004] Therefore, there is an urgent need for a control scheme that can accurately distinguish between learning and entertainment video content on learning machine terminals. Summary of the Invention

[0005] This application provides a method and system for video traffic security management applied to learning machine terminals, which can efficiently and accurately distinguish between learning and entertainment video content within the learning machine terminal, thereby improving the control effect of video traffic.

[0006] Firstly, a method for secure management of video traffic applied to learning machine terminals is provided, including: Deep packet inspection is performed on the video traffic data packets of the video stream received by the learning machine terminal to extract the traffic characteristics of the video stream; By matching traffic features with a preset list database, the confidence level of whether a traffic feature belongs to the list database is obtained. The list database includes a blacklist and a whitelist. If no match is found in the preset list, the video segments within the current sliding time window are processed by frame extraction based on the sliding time window, and the corresponding text of the video segments is segmented and vectorized using a neural network model to obtain text features. Obtain a candidate frame sequence containing a preset number of candidate frames, and use a multimodal large model based on cross-attention mechanism to fuse and infer the visual and text features of the candidate frame sequence to obtain the probability distribution of each content category corresponding to the candidate frame sequence. Based on the probability distribution of each content category, corresponding control operations are performed on the video segments within the current sliding time window.

[0007] In a feasible design, the method also includes: The feature samples and decision logs stored in the corresponding tag partitions in the cloud are updated according to the control operations. The types of tag partitions include: blocking sample area, release sample area, and parent correction sample area. Each sample area is used to store multiple sets of feature samples and decision logs. Each set of feature samples and decision logs is associated with the same video stream content identifier and timestamp. The update of the feature samples and decision logs in the parent correction sample area triggers the update of the content feature matrix space. The video segment under each sliding time window in the content feature matrix space is represented by the status and the control operations of the system. The status includes traffic features, visual features, text features, feedback behavior, and current policy context. If the user provides feedback on the control operation, obtain the user's feedback behavior, which may include manually lifting the block or manually blocking the passage. Copy the feature samples and decision logs corresponding to the candidate frame sequence in the blocking sample area or the release sample area to the parent correction sample area and update the content feature matrix space. Delete the corresponding records in the original sample area where the candidate frame sequence is located. Reward signals are generated based on feedback behavior, and cumulative rewards are updated based on reward signals. Manual blocking and allowing passage corresponds to positive rewards, while manual unblocking corresponds to negative rewards. Based on the newly added states and control operations in the content feature matrix space, and combined with the cumulative reward, the policy parameters stored locally on the learning machine terminal are incrementally updated using a policy gradient-based reinforcement learning algorithm. The policy parameters include at least one of the following: the classification head weight parameters of the multimodal large model, the lightweight adapter parameters of the multimodal large model, the parameters of the policy network, and the threshold parameters.

[0008] In a feasible design, the method also includes: If the feedback behavior includes manual blocking and allowing, the traffic features, visual features, and text features corresponding to the video segments within the current sliding time window are associated and stored in the violation content baseline library stored locally on the learning machine terminal. This is so that in subsequent sliding time windows, before fusing and inferring the visual and text features of new candidate frame sequences based on a multimodal large model, the latest violation content baseline library is matched with the relevant features of the new candidate frame sequences to determine whether to perform blocking operations on the video segments corresponding to the new candidate frame sequences.

[0009] In a feasible design, before employing a multimodal large model based on a cross-attention mechanism to perform fusion inference on the visual and textual features of the candidate frame sequence, the method further includes: A visual feature extraction network is used to extract the visual features of each candidate frame in the candidate frame sequence, thereby obtaining the visual feature sequence corresponding to the candidate frame sequence. Specifically, a neural network model is used to segment and vectorize the text corresponding to the video clips to obtain text features, including: If a keyframe is identified from the candidate frame sequence, the first text in the keyframe is extracted using optical character recognition technology. The second text, which is composed of the first text and the metadata of the video stream, is then segmented and vectorized to obtain the text features of each segment corresponding to the first text and the second text. The first text and the second text are the text corresponding to the video segment. If no keyframe is identified from the candidate frame sequence, the second text is segmented and vectorized to obtain the text features of each segmented word in the second text. The second text is the text corresponding to the video segment.

[0010] In one feasible design, a multimodal large model based on a cross-attention mechanism is used to fuse and infer the visual and textual features of the candidate frame sequence, obtaining the probability distribution of each content category corresponding to the candidate frame sequence, including: Determine the attention weight of the text features of each word to each visual feature in the visual feature sequence; The visual features in the visual feature sequence are weighted and summed according to the attention weights of each visual feature corresponding to each word to obtain the visual context vector corresponding to each word. The global visual context vector is obtained by weighted summing of the visual context vectors corresponding to each word segmentation. The global visual context vector and the global text feature vector are concatenated and then input into the classifier to obtain the probability distribution of each content category corresponding to the candidate frame sequence.

[0011] In one feasible design, the visual context vectors corresponding to each word segment are weighted and summed to obtain the global visual context vector, which includes: Input a sequence of text features consisting of the text features of each word into a self-attention network that employs a self-attention mechanism, and obtain the importance score of each word; The importance score of each word is normalized to obtain the attention weight corresponding to each word. The visual context vectors corresponding to each word are weighted and summed based on the attention weights corresponding to each word to obtain the global visual context vector.

[0012] In one feasible design, the content categories include a first category, a second category, and a third category. The first category indicates that the video content is educational, the second category indicates that it is entertainment, and the third category indicates that it is illegal. Based on the probability distribution of each content category, corresponding control operations are performed on the video segments within the current sliding time window, including: If the probability of the second category is greater than the first blocking threshold, or the probability of the third category is greater than the second blocking threshold, the control operation type is determined to be blocking according to the control strategy, and the blocking operation is performed on the video segment within the current sliding time window. The control strategy is used to specify the type of control operation based on the threshold. If the probability of the third category is less than the safety threshold and the probability of the first category is greater than the release threshold, the control operation type is determined to be release according to the control strategy, and the release operation is performed on the video segment in the current sliding time window, where the safety threshold is less than the second blocking threshold. Otherwise, if the control operation type is determined to be a pop-up confirmation according to the control strategy, the pop-up confirmation operation is executed. The pop-up confirmation operation is used to display a pop-up to the user so that the user can confirm whether to block the video segment in the current sliding time window after entering the password.

[0013] In one feasible design, if a blocking operation is to be performed on a video segment within the current sliding time window, the method further includes: Update the number of blocking times corresponding to the network address based on the network address corresponding to the video stream; If the number of blocking attempts exceeds the threshold, the access control list will be adjusted based on the network address to block or restrict subsequent communication connections with the network address.

[0014] In one feasible design, corresponding control operations are performed on the video segments within the current sliding time window based on the probability distribution of each content category, including: The probability distribution of each content category is input into the policy network, which employs a multilayer perceptron mechanism. The type of control operation output by the policy network for the video segment within the current sliding time window is obtained. The types of control operations include allowing, blocking, or pop-up confirmation. Perform the corresponding control operation based on the type of control operation.

[0015] Secondly, a video traffic security management system for learning machine terminals is provided, including: The deep packet inspection preprocessing module is used to perform deep packet inspection on the video traffic data packets of the video stream received by the learning machine terminal in order to extract the traffic characteristics of the video stream. The deep packet inspection preprocessing module is also used to obtain the confidence level of traffic features belonging to the list by matching traffic features with a preset list, which includes a blacklist and a whitelist. The multimodal feature extraction module is used to extract frames of video segments within the current sliding time window if no match is found in the preset list. It then uses a neural network model to segment and vectorize the text corresponding to the video segments to obtain text features. The large model inference module is used to obtain a candidate frame sequence containing a preset number of candidate frames. It uses a multimodal large model based on cross-attention mechanism to fuse and infer the visual and text features of the candidate frame sequence to obtain the probability distribution of each content category corresponding to the candidate frame sequence. The decision-making and execution module is used to perform corresponding control operations on video segments within the current sliding time window based on the probability distribution of each content category.

[0016] This application provides a layered and progressive traffic control mechanism. Specifically, it first uses deep packet inspection to extract traffic features from the video stream and matches them with a list database. Subsequent deep analysis is only triggered when the matching confidence is insufficient, thus filtering out most obviously compliant or non-compliant traffic at the source at extremely low cost. This coarse-grained fast screening + fine-grained precise judgment architecture balances the real-time performance and accuracy of security control. For suspicious traffic, a sliding time window is used to extract frames and combine a multimodal large model to fuse and infer the visual and textual features of candidate frames, outputting a fine-grained content category probability distribution. This allows for precise control operations based on the content category probability distribution. This design breaks through the traditional static control mode based on URL / IP, enabling the system to deeply understand the semantic information of video images (such as the difference between teaching blackboard writing and entertainment scenes) and accompanying text (such as titles and text in images), effectively distinguishing between biological teaching videos and inappropriate videos, and between lectures by famous teachers and entertainment gossip, significantly reducing false alarm and false negative rates. Meanwhile, multimodal large model inference is only enabled on demand in the low-confidence gray area, while high-confidence traffic is directly decided by lightweight deep packet inspection and matching. This reduces the computational load and power consumption of the learning machine terminal while ensuring recognition accuracy, and achieves an effective balance between resource utilization and control accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of a video traffic security management method applied to a learning machine terminal provided in this application. Figure 2 This is a schematic diagram of a video traffic security management system applied to a learning machine terminal, provided by an exemplary embodiment of this application; Figure 3 This is a schematic flowchart illustrating yet another example of a video traffic security management method applied to a learning machine terminal, provided by an exemplary embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] Currently, using blacklist filtering technology to control network video traffic on learning machine terminals still has the following drawbacks: Lag: The blacklist is updated by manual collection or third-party libraries. When faced with newly generated video links or short domains (such as t.cn, bit.ly and other redirect links), it cannot be identified and blocked in real time, resulting in a high false negative rate.

[0021] High resource consumption: In order to maintain a high interception rate, the blacklist database is huge. Frequent hash comparisons will consume the limited memory and central processing unit (CPU) resources of the learning machine terminal, affecting the smoothness of the learning machine terminal.

[0022] In addition to using blacklist filtering technology to control network video traffic on learning machine terminals, some manufacturers currently use the following technologies to control network video traffic: (1) Management technology based on application layer blacklists and whitelists Specifically, parents or the system can preset a list of allowed applications (such as only allowing learning applications like "math tutoring" and "English dictionary"), and prohibit applications like "XX short video" and "XX video".

[0023] However, this technology cannot control the specific content within an application. For example, many educational applications integrate "community" and "discovery" sections, which contain a large number of non-educational videos. Simply allowing the entire application to run freely could lead to students becoming addicted to the entertainment sections within the app. Therefore, this technology suffers from the drawback of having a coarse-grained control mechanism.

[0024] Furthermore, this technology lacks the ability to deeply understand the semantics of video footage and audio text, and cannot accurately identify metaphorical or offensive content or entertainment content that skirts the line of legality. Therefore, this technology also suffers from a lack of contextual understanding.

[0025] (2) Filtering techniques based on simple image recognition Specifically, traditional computer vision algorithms (such as frame difference and skin color detection) or simple object detection models are used to determine whether inappropriate elements are present in the video.

[0026] However, the traditional models used in this technology have a weak ability to understand complex scenarios and a high false alarm rate. For example, in biology lesson videos, there may be anatomical diagrams; a simple model might misinterpret this as pornographic content and block the video, thus disrupting normal teaching.

[0027] In summary, existing technologies mainly rely on static feature matching (such as IP, URL, and application name), lacking the ability to understand the multimodal semantics of dynamic video stream content. They are unable to accurately remove entertainment or inappropriate video traffic disguised as learning applications while ensuring the smooth flow of normal learning content.

[0028] To solve the above problems, such as Figure 1 As shown, this application provides a video traffic security management method for learning machine terminals, including: S110 performs deep packet inspection on the video traffic data packets of the video stream received by the learning machine terminal to extract the traffic characteristics of the video stream.

[0029] It should be understood that video traffic data packets are discrete carrier units of video streams in network transmission. Multiple data packets are reassembled in sequence to form a continuous video stream. That is, the video traffic data of the same video stream have the same traffic characteristics.

[0030] Specifically, this involves capturing Hypertext Transfer Protocol (HTTP) or Hypertext Transfer Protocol Secure (HTTPS) video traffic data packets received by the terminal using Application Programming Interface (API) hooking technology. For example... Figure 2 As shown, this capture step can be completed in real time by the terminal video capture module located in the data acquisition layer at the network protocol stack layer of the operating system.

[0031] Subsequently, Deep Packet Inspection (DPI) technology is used to parse the captured data packets and extract the traffic characteristics (also known as DPI features) of the corresponding video stream. For example... Figure 2 As shown, this parsing step can be completed by the depth packet detection preprocessing module of the preprocessing layer.

[0032] For example, traffic characteristics include network layer 5-tuple characteristics, application layer source characteristics, and payload characteristics, wherein the network layer 5-tuple characteristics include: Source IP, destination IP, source port, destination port, transport protocol; Application layer source characteristics include, but are not limited to: Domain name, Server Name Indication (SNI), Host identifier, Certificate information, Content Delivery Network (CDN) node, Application package name, Request URL, Content identifier.

[0033] Content identifiers include, but are not limited to: video identifiers (ID) provided by the video platform, hash values ​​obtained by hashing the video URL, and traffic fingerprints.

[0034] Load characteristics include, but are not limited to: Video protocol type, request path, media segmentation characteristics, content type, and user agent. Examples of video protocol types include HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH).

[0035] S120: By matching traffic characteristics with a preset list, the confidence level of whether the traffic characteristics belong to the list is obtained.

[0036] The database includes a blacklist and a whitelist.

[0037] The blacklist includes the traffic characteristics of pre-collected non-learning websites, while the whitelist includes the traffic characteristics of pre-collected learning websites.

[0038] For example, the confidence level of a traffic feature belonging to a predefined list can be obtained by matching traffic features with the list in the following way: The traffic features are matched with the whitelist (or blacklist) in the list database to obtain the confidence level that the traffic features belong to the whitelist (or blacklist); If the confidence level is greater than the confidence level threshold, the confidence level of the traffic feature belonging to the whitelist (or blacklist) is determined as the confidence level of the traffic feature belonging to the list database; If the confidence level is less than or equal to the confidence level threshold, the traffic feature is matched with the blacklist (or whitelist) in the list database to obtain the confidence level that the traffic feature belongs to the blacklist (or whitelist). The confidence level that the traffic feature belongs to the blacklist (or whitelist) is determined as the confidence level that the traffic feature belongs to the list database.

[0039] The above example first matches traffic features with a whitelist (or blacklist). If the confidence score exceeds the threshold, it can be directly determined to belong to the whitelist (such as learning content), thus allowing it to pass quickly. Only when the confidence score of the whitelist (or blacklist) is insufficient will the blacklist (or whitelist) continue to be matched, thus avoiding the computational overhead of performing bidirectional matching on all traffic at the same time.

[0040] For example, the confidence level of a traffic feature belonging to the whitelist can be obtained by matching traffic features with a whitelist in the following way: Iterate through the traffic characteristics of each learning-related site in the whitelist, and calculate the similarity score between the video stream and each learning-related site using the following steps: The similarity of each feature dimension of the traffic characteristics of the video stream with the corresponding feature dimension of the traffic characteristics of the current site is calculated item by item to obtain the similarity of each feature dimension. For example, the similarity of the five-tuple of the video stream with the five-tuple of the current site, the similarity of the SNI of the video stream with the SNI of the current site, the similarity of the video protocol type of the video stream with the video protocol type of the current site, and so on. The similarity scores across all dimensions are weighted and summed to obtain the overall similarity score between the video stream and the current site. The weights of each dimension are dynamically allocated based on their discriminative importance in the educational context. The overall similarity score is transformed to the [0,1] interval using a linear mapping or Sigmoid function, and used as the confidence that the video stream belongs to the current site; After the traversal is complete, the maximum value among all confidence scores is taken as the final confidence score for the video stream to belong to the whitelist.

[0041] It should be understood that the confidence level of the video stream's traffic characteristics belonging to the whitelist is the same as the confidence level belonging to the list database.

[0042] The above example can comprehensively utilize multi-dimensional features such as quintuples, application layer source features, and payload features to calculate the confidence of the video stream belonging to each site item by item, and finally take the maximum value as the confidence of the video stream belonging to the whitelist. This can avoid misjudgment based on a single feature, improve the accuracy and robustness of matching, and the calculation method is simple, which can quickly identify whether the video stream belongs to the whitelist while ensuring high accuracy.

[0043] For the method of matching traffic features with blacklists using multi-dimensional similarity, please refer to the example above, and it will not be repeated here.

[0044] For example, the confidence level of a traffic feature belonging to the whitelist can be obtained by matching traffic features with a whitelist in the following way: Beforehand, positive examples of known learning-type video traffic and negative examples of known non-learning-type video traffic are collected. Traffic features are extracted from each sample, and it is marked whether it belongs to the whitelist. A classification model is trained using the labeled data mentioned above. The classification model uses algorithms such as logistic regression, random forest, decision tree or lightweight deep learning network, takes the traffic characteristics of the video stream as input, and outputs the probability value of the video stream belonging to the whitelist. The traffic characteristics of the current video stream are input into the trained classification model, and the probability values ​​in the [0,1] interval output by the classification model are used as the confidence level of the traffic characteristics belonging to the whitelist.

[0045] The above example uses a classification model to determine the confidence level of traffic features belonging to the whitelist, which can fully explore the nonlinear relationship between features and accurately obtain the confidence level of traffic features belonging to the whitelist.

[0046] For the method of using a classification model to determine the confidence level of traffic features belonging to the blacklist, please refer to the previous example, which will not be repeated here.

[0047] As a further example, if the confidence level is greater than the confidence level threshold, that is, the traffic characteristics match the preset list, the video stream is blocked or allowed according to the blacklist or whitelist matched by the traffic characteristics.

[0048] The confidence threshold can be set according to requirements.

[0049] Specifically, when traffic features are matched against a pre-defined list (including blacklists and whitelists), if the calculated confidence level is higher than a pre-set confidence threshold, it means that the current matching result has sufficient credibility, and there is no need to call a more complex multimodal model for in-depth analysis. If the confidence level of the traffic feature corresponds to the blacklist, the video stream is blocked; if the confidence level of the traffic feature corresponds to the whitelist, the video stream is allowed.

[0050] When blocking is executed, the underlying network management interface of the operating system (such as iptables in Linux or the Windows Filtering Platform application interface in Windows) can be invoked to sever the Transport Control Protocol (TCP) connection corresponding to the video stream and clear the corresponding buffer. When allowing passage is executed, all video traffic packets of the current video stream are allowed to pass through the network protocol stack normally without any blocking or rate limiting measures.

[0051] For example, a fully updated whitelist is maintained in the cloud; the terminal locally stores the traffic characteristics of learning sites in the whitelist that are frequently accessed, authorized, and recently synchronized, for quick offline assessment. This ensures both real-time performance and support for offline scenarios.

[0052] For example, the cloud maintains a full and continuously updated blacklist, which can uniformly update the traffic characteristics of non-learning sites such as malicious entertainment sites, illegal video sources, and risky CDNs; the terminal locally saves the latest version of the blacklist for low-latency blocking, so that the terminal can directly block high-risk video streams, and use subsequent multimodal recognition for low-risk video streams that are not on the whitelist.

[0053] S130, if no match is found in the preset list, based on the sliding time window, the video segments in the video stream that are within the current sliding time window are processed by frame extraction, and a neural network model is used to segment and vectorize the text corresponding to the video segments to obtain text features.

[0054] Specifically, if a video stream does not match a preset list (i.e., the confidence level is less than or equal to the confidence threshold), it may exhibit mixed or unknown features. Mixed features refer to conflicts between different features; for example, the domain name may belong to an educational platform, but the URL, title, and segment source may indicate entertainment / violation tendencies; or the five-tuple may point to a commonly used CDN, but the payload features may point to non-learning content. Unknown features refer to video streams that cannot match whitelists or blacklists, such as video streams with characteristics like new domain names, short link redirects, new CDN nodes, or encrypted traffic that cannot parse content. In this case, a sliding time window is used to logically segment the video stream, and the segmented parts are parsed by a video decoder to obtain the corresponding video segments containing image frames. Frames are dynamically extracted from these video segments, for example, 5 frames are sampled every sliding time window, and these 5 candidate frames constitute a candidate frame sequence. Then, a neural network model is used to segment and vectorize the text corresponding to the video segments to obtain text features.

[0055] S140, obtain a candidate frame sequence containing a preset number of candidate frames, and use a multimodal large model based on cross-attention mechanism to perform fusion reasoning on the visual features and text features of the candidate frame sequence to obtain the probability distribution of each content category corresponding to the candidate frame sequence.

[0056] Among them, the multimodal large model adopts a neural network structure based on the cross-attention mechanism for fusing multimodal features.

[0057] The probability distribution for each content category includes probability values ​​for multiple categories, such as learning, entertainment, and illegal content.

[0058] The duration of the sliding time window can be set according to actual needs, such as 1 second.

[0059] The preset quantity can be set according to actual needs, for example, 5.

[0060] It should be noted that the analysis task within the same sliding time window is performed on the video stream with the same ID.

[0061] In a feasible design, before employing a multimodal large model based on a cross-attention mechanism to perform fusion inference on the visual and textual features of the candidate frame sequence, the method also includes a multimodal feature extraction step, such as... Figure 2 As shown, this step can be implemented by a multimodal feature extraction module, which includes a visual feature submodule and a text feature submodule. This step includes: A visual feature extraction network is used to extract the visual features of each candidate frame in the candidate frame sequence, resulting in a visual feature sequence corresponding to the candidate frame sequence; this step can be performed by the visual feature submodule. Specifically, a neural network model is used to segment and vectorize the text corresponding to the video clips to obtain text features, including: If a keyframe is identified from the candidate frame sequence, optical character recognition technology is used to extract the first text from the keyframe. The second text, which is composed of the first text and the metadata of the video stream, is then segmented and vectorized to obtain the text features of each segment corresponding to the first and second texts. The first and second texts are the texts corresponding to the video segments. This step can be performed by the text feature submodule. If no keyframe is identified from the candidate frame sequence, the second text is segmented and vectorized to obtain the text features of each segmented word. The second text is the text corresponding to the video segment. This step can be performed by the text feature submodule.

[0062] Unless otherwise specified, the visual and textual features of this application are stored and used in computation in vector representation.

[0063] For example, the visual feature extraction network uses a 50-layer residual network (ResNet-50) or an efficient convolutional neural network (EfficientNet) as the backbone network.

[0064] For example, methods for identifying keyframes include: If the candidate frame sequence includes a candidate frame labeled as an I-frame, then that I-frame is identified as a keyframe; if the candidate frame sequence does not include a candidate frame labeled as an I-frame, then the following method is used to determine the content-related keyframe: Histogram difference detection, Structural Similarity Index Measure (SSIM) comparison, optical flow change analysis, scene transition detection, or target change detection; keyframes in terms of content are frames in which the content of the image changes significantly or contains high semantic information.

[0065] It should be noted that if the candidate frame sequence does not include candidate frames marked as I-frames, the above method may not be able to determine any keyframes in that candidate frame sequence.

[0066] For example, the metadata of a video stream includes, but is not limited to: Video title, description, and video tags.

[0067] It should be noted that for the same video stream, its metadata only needs to be extracted once after the video segment is first acquired based on the sliding time window.

[0068] The following example illustrates how to perform multimodal feature extraction: A visual feature extraction network is used to extract the visual features of each candidate frame in the candidate frame sequence, resulting in a visual feature sequence V, where V = {v1, v2, ..., v...}. n}, where n is the number of candidate frames in the candidate frame sequence, and the visual feature of the i-th candidate frame is v. i Let i = 1, 2, ..., n, where i is the candidate frame number. If a keyframe is identified from the candidate frame sequence, optical character recognition (OCR) technology is used to extract the first text from the keyframe. The second text, composed of the first text and the metadata of the video stream, is then segmented and vectorized using a neural network-based Chinese word segmentation model to obtain m segmented text features T, where T = {t1, t2, ..., tn}. m}, where m is the number of word segments, and t is the text feature of the j-th word segment. jLet j = 1, 2, ..., m, where j is the word segmentation number. If no keyframe is identified from the candidate frame sequence, there is no need to extract the first text from the keyframe; only the second text composed of metadata is segmented and vectorized to obtain text features T. The neural network-based Chinese word segmentation model can be jointly constructed from a Bidirectional Long Short-Term Memory (Bi-LSTM) network and a Conditional Random Field (CRF).

[0069] This embodiment employs a visual feature extraction network to process candidate frame sequences, efficiently obtaining sequential visual features and providing visual feature input for multimodal large-scale models to understand video image content. By applying optical character recognition (OCR) technology to the identified keyframes to extract text within the frame, and combining this with video metadata text to construct a more complete text information source, further processed through word segmentation and vectorization, semantically comprehensive text features are obtained, providing text feature input for multimodal large-scale models to understand video text information. Furthermore, the solution incorporates branching logic (merging both types of text when keyframes are identified, and processing only metadata text when keyframes are not identified), enabling reasoning based on reliable metadata text even when keyframes are unavailable, thereby improving the overall robustness of text feature extraction.

[0070] In a feasible design, this is achieved by employing a multimodal large model based on a cross-attention mechanism to fuse and infer the visual and textual features of the candidate frame sequence, thereby obtaining the probability distribution of each content category corresponding to the candidate frame sequence, such as... Figure 2 As shown, this method can be executed by the large model inference module of the intelligent processing layer: Determine the attention weight of the text features of each word to each visual feature in the visual feature sequence; The visual features in the visual feature sequence are weighted and summed according to the attention weights of each visual feature corresponding to each word to obtain the visual context vector corresponding to each word. The global visual context vector is obtained by weighted summing of the visual context vectors corresponding to each word segmentation. The global visual context vector and the global text feature vector are concatenated and then input into the classifier to obtain the probability distribution of each content category corresponding to the candidate frame sequence.

[0071] For example, the text features t of the j-th word segment can be determined using the following formula (1). j Visual feature v in the visual feature sequence i Attention weights : , formula (1); in, Used to measure visual characteristics Textual features of word segmentation The correlation is given by k, which is the index of all candidate frames traversed during normalized summation, and exp() represents the exponential function. The following formula (2) or formula (3) can be used for calculation: , formula (2); , formula (3); In formula (2), Let be the trainable mapping matrix corresponding to the visual features, () T This indicates the transpose operation. Let d be the trainable mapping matrix corresponding to the text features. Dimensions This represents the scaling factor.

[0072] In formula (3), u is the trainable weight vector, tanh is the hyperbolic tangent activation function, and b1 is the bias matrix.

[0073] The visual context vector C corresponding to the j-th word segment is determined using the following formula (4). j : , formula (4); The visual context vectors corresponding to each word segment are weighted and summed using the following formula (5) to obtain the global visual context vector C. v : , formula (5); in, is the attention weight coefficient corresponding to the j-th word segment.

[0074] The global visual context vector and the global text feature vector are concatenated and then input into a classifier (such as a fully connected layer + Softmax layer) to obtain the probability distribution of each content category corresponding to the candidate frame sequence, as shown in the following formula (6): , formula (6); Where P is the probability distribution of each content category (e.g., entertainment has a probability of 0.8, violations have a probability of 0.5, and learning has a probability of 0.2), Softmax represents the normalization function, W is the trainable mapping matrix of the classifier, b2 is the bias matrix of the classifier, T_global represents the global text feature vector, and C v T_global represents C v The concatenation operation with T_global.

[0075] For example, each content category includes a first category, a second category, and a third category, wherein the first category indicates that the video content is educational content, the second category indicates that the video content is entertainment content, and the third category indicates that the video content is illegal content.

[0076] For example, the global text feature vector can be obtained in the following ways: The target text is input into the Bidirectional Encoder Representations from Transformers (BERT) model, and the [CLS] vector of BERT is taken as the global text feature vector. Alternatively, the text features of each word segment can be averaged to obtain the global text feature vector; Alternatively, the text features of each word can be weighted and pooled to obtain a global text feature vector, where the weight corresponding to each word is the attention weight coefficient.

[0077] The above embodiments calculate the attention weight of each text segment to each video frame for each text segment, and then sum them up to obtain the visual context vector corresponding to that segment. This realizes the calculation of the correlation between video frames and text keywords, enabling each segment to focus on the most relevant visual information in the video frame sequence. By generating a unique visual context vector for each segment and further weighting and aggregating it into a global visual context, the overall intent of the text is integrated while retaining key local correlation information, enhancing the overall representation ability of complex and polysemous content. By concatenating and classifying the global visual features that integrate text intent with global text features, the reasoning process benefits from both fine-grained cross-modal alignment and global semantic fusion. This allows for the output of a more reliable and discriminative probability distribution when faced with videos that are inconsistent with the description or have ambiguous content, such as illegal or entertainment videos.

[0078] In a feasible design, the global visual context vector is obtained by weighted summing of the visual context vectors corresponding to each word segmentation: Input a sequence of text features consisting of the text features of each word into a self-attention network that employs a self-attention mechanism, and obtain the importance score of each word; The importance score of each word is normalized to obtain the attention weight corresponding to each word. The visual context vectors corresponding to each word are weighted and summed based on the attention weights corresponding to each word to obtain the global visual context vector.

[0079] This embodiment leverages the ability of self-attention networks to capture long-distance semantic interactions between word segments, more accurately identifying core word segments that play a crucial role in understanding video content and assigning them higher attention weights; while non-keywords or noise characters receive lower weights. Based on this, the visual context vectors of each word are weighted and summed using normalized importance scores, making the final global visual context vector more focused on visual information semantically related to the key word segments, while suppressing interference from irrelevant or secondary word segments, thus enhancing the multimodal large model's ability to represent the core content of the video.

[0080] S150, based on the probability distribution of each content category, performs corresponding control operations on the video segments within the current sliding time window.

[0081] For example, each content category includes a first category, a second category, and a third category, wherein the first category indicates that the video content is educational content, the second category indicates that the video content is entertainment content, and the third category indicates that the video content is illegal content.

[0082] For example, this is achieved by performing corresponding control operations on video segments within the current sliding time window based on the probability distribution of each content category: Based on the probability distribution of each content category and the preset control strategy, corresponding control operations are performed on the video segments within the current sliding time window.

[0083] The control strategy is used to specify the type of control operation based on a threshold, and can be stored in the local configuration of the learning machine terminal.

[0084] For example, control strategies include: If the probability of the second category is greater than the first blocking threshold, or the probability of the third category is greater than the second blocking threshold, the control operation in this case is blocking. If the probability of the third category is less than the second blocking threshold and the probability of the first category is greater than the release threshold, the control operation in this case is release. In other cases, the control operation type is pop-up confirmation.

[0085] Control policies can be stored in, for example Figure 2 The decision-making and execution module shown is located in the decision-making and execution layer. In one feasible design, the decision-making and execution module is implemented by performing corresponding control operations on video segments within the current sliding time window based on the probability distribution of each content category: If the probability of the second category is greater than the first blocking threshold, or the probability of the third category is greater than the second blocking threshold, the type of control operation is determined to be blocking according to the control strategy, and the blocking operation is performed on the video segment within the current sliding time window. If the probability of the third category is less than the safety threshold and the probability of the first category is greater than the release threshold, the control operation type is determined to be release according to the control strategy, and the release operation is performed on the video segment in the current sliding time window, where the safety threshold is less than the second blocking threshold. Otherwise, if the control operation type is determined to be a pop-up confirmation according to the control strategy, the pop-up confirmation operation is executed. The pop-up confirmation operation is used to display a pop-up to the user so that the user can confirm whether to block the video segment in the current sliding time window after entering the password.

[0086] The values ​​of the first blocking threshold, the second blocking threshold, the safety threshold, and the release threshold can be determined according to actual needs, and this application does not limit them.

[0087] For example, the blocking operation includes: Call the underlying operating system interface (such as iptables rules or API blocking) to cut off the TCP connection corresponding to the network address of the video stream and clear the buffer.

[0088] For example, the pop-up confirmation operation is implemented by calling the underlying interface of the operating system to trigger an operating system-level pop-up, which requires the parent to enter a password before the video can be played.

[0089] The above embodiments construct a multi-level decision-making system by setting a first blocking threshold, a second blocking threshold, and a release threshold, achieving precise control over video content. Specifically, the embodiments set independent blocking thresholds for the probability of negative content in the categories of "entertainment" and "violation," and a release threshold for the probability of positive content in the category of "education," achieving precise differentiated control based on the probability distribution of each content category. In particular, when the probability of each content category has not reached the threshold threshold, it means that this is a boundary situation with fuzzy probability. In this case, a pop-up confirmation operation gives the final decision-making power to the parents (parents can enable the permission to perform blocking or release operations with a password). Through this human-machine collaboration mechanism, both the flexibility and accuracy of control are ensured, while also taking into account the actual educational needs of parents.

[0090] For example, the "determine the type of control operation as pop-up confirmation according to the control policy and perform the pop-up confirmation operation" in the aforementioned embodiment can be replaced by the following steps: Accumulate the probability distributions of the current sliding time window and at least one subsequent consecutive sliding time window to form a probability distribution sequence; Calculate the arithmetic mean of the probabilities of the same class in a probability distribution sequence; Based on the comparison between the arithmetic mean of the probabilities of each category and the corresponding threshold, the operation of allowing passage, confirming via pop-up window, or blocking is performed.

[0091] For boundary situations with ambiguous probabilities, the above example can effectively smooth out probability fluctuations in a single window caused by sudden changes in content, encoding noise, etc., so that the probabilities of each category used for decision-making can more stably reflect the overall semantic tendency of the video over a period of time, avoid pop-ups being mistakenly triggered by a single low-confidence window due to noise, and reduce the number of unnecessary interventions by parents.

[0092] In a feasible design, the decision-making and execution module is implemented by performing corresponding control operations on video segments within the current sliding time window based on the probability distribution of each content category: The probability distribution of each content category is input into the policy network, which employs a multilayer perceptron mechanism. The type of control operation output by the policy network for the video segment within the current sliding time window is obtained. The types of control operations include allowing, blocking, or pop-up confirmation. Perform the corresponding control operation based on the type of control operation.

[0093] This embodiment employs a lightweight, multilayer perceptron-based policy network to directly map probability distributions to control operation types, achieving end-to-end adaptive decision-making. Compared to rigid rules with fixed thresholds, the policy network can learn the nonlinear interaction relationships between categories in the probability distribution, thereby adapting to more complex boundary scenarios and accurately determining the type of control operation.

[0094] In one feasible design, if a blocking operation is to be performed on a video segment within the current sliding time window, the method further includes: Update the number of blocking times corresponding to the network address based on the network address corresponding to the video stream; If the number of blocking attempts exceeds the threshold, the access control list will be adjusted based on the network address to block or restrict subsequent communication connections with the network address.

[0095] The number of times threshold is set according to actual needs, and this application does not limit it.

[0096] Specifically, a blocking count log is stored locally, containing the blocked network addresses and their corresponding blocking counts. After blocking a video segment, the log retrieves historical blocking records for that network address. If a record exists, the blocking count is incremented; otherwise, a new record for that address is created. When the blocking count reaches a threshold, the access control list is adjusted based on the network address. The methods for adjusting the access control list include: Add the network address or domain name to the blocking list by modifying the firewall / iptables rules; Alternatively, update the routing black hole to cut off traffic to that network address; Alternatively, dynamically reduce the bandwidth limit for traffic transmitted between the network address and the network address.

[0097] The above embodiments not only block the current illegal video stream, but also identify high-risk network addresses that repeatedly violate the rules through cumulative counting. Once the number of times an address is blocked exceeds a threshold, it is identified as a malicious source and proactively added to the firewall blacklist, thereby blocking all subsequent connection requests from that address at the operating system network layer, achieving proactive defense.

[0098] It should be understood that this solution uses a sliding window approach for multimodal continuous monitoring of the video stream. This means that for a video stream deemed safe and passable, it only indicates that the video segment corresponding to the current window meets the passability criteria; the system continues to sample and analyze subsequent video streams. When the probability of violation or entertainment in any subsequent video segment exceeds a preset blocking threshold, the system immediately performs a blocking operation, including disconnecting the TCP connection, stopping traffic forwarding, and clearing the terminal buffer. After the blocking operation is completed, the video session enters a terminated state, and subsequent traffic is no longer sampled or analyzed.

[0099] For example, after executing blocking, releasing, or pop-up confirmation, the feature samples and decision logs stored in the corresponding tag partition in the cloud are updated according to the control operation. The tag partition types include: blocking sample area, releasing sample area, parent correction sample area, and suspected sample area. Each sample area is used to store multiple sets of feature samples and decision logs. Each set of feature samples and decision logs is associated using the content identifier and timestamp of the same video stream. That is, the same sliding time window of the same video stream corresponds to a set of feature samples and decision logs.

[0100] The field types of feature samples include, but are not limited to: Video ID / Traffic Fingerprint: Used to record the video stream's video ID / traffic fingerprint; Traffic characteristics: Traffic characteristics used to record video streams; Visual features: used to record the sequence of visual features corresponding to the video stream; Text features: Used to record the text features of each word corresponding to the video stream; System actions: Used to record the types of control operations performed by the system; Parent feedback: Used to record parents' feedback behavior (including manual unblocking, manual blocking and allowing, or confirmation behavior without intervention); Timestamp: Used to record the time of sample collection or processing.

[0101] The field types of the decision log include, but are not limited to: Video ID / Traffic Fingerprint; Final decision: Used to record the type of control operation. If the parent does not intervene, the type of control operation recorded is the type of control operation of the system; for video streams that have been intervened by the parent, the type of control operation modified by the parent is updated. Threshold / Policy Version: Used to record the version number of each threshold or policy network in the control policy; Triggering conditions: Used to record the discrimination information (such as confidence level) that triggers this decision; Processing results: Used to record the final result, such as whether passage has been granted, blocked, or a pop-up message has been issued; Reviewer / Source: Used to record the identifier of the human reviewer or the automated system.

[0102] Log timestamp: Used to record the timestamps of the decisions made.

[0103] The blocking sample area stores feature samples and decision logs of video streams blocked by the system. It can be used to generate corresponding illegal or entertainment video samples to expand the training dataset. The expanded training dataset can be used for offline training of multimodal large models deployed in the cloud to improve the system's recognition accuracy of high-risk videos.

[0104] The release sample area is used to store feature samples and decision logs of video streams that have been released by the system. It can be used to generate corresponding security video samples to expand the training dataset. The expanded training dataset can be used to train multimodal large models deployed in the cloud offline to improve the system's recognition accuracy of security videos.

[0105] The parent correction sample area is used to store feature samples and decision logs of video streams that differ from the original system's decision after manual intervention by parents. This is used to provide positive or negative feedback for policy gradient-based reinforcement learning.

[0106] The suspected sample area is used to store feature samples and decision logs of the video stream that have been confirmed by the system through a pop-up window. Samples in the suspected sample area can be used as unlabeled samples. Combined with weak supervision signals such as manual review or parental feedback, pseudo-labels are generated and used to expand the training set for weak supervision training of the model.

[0107] It should be noted that the learning machine terminal locally stores recent (e.g., within the last 30 days) feature samples and decision logs for online incremental updates of the local policy parameters. The learning machine terminal periodically synchronizes anonymized feature samples and decision logs to the cloud. Therefore, the cloud can store the full historical feature samples and decision logs to support offline training and optimization of large multimodal models.

[0108] For example, the whitelist and blacklist in this application can be updated in the following ways: The cloud regularly sends the latest whitelist and blacklist to the learning machine terminal, triggering the learning machine terminal to incrementally update the whitelist and blacklist.

[0109] And / or, The learning machine terminal performs local incremental updates, including: periodically extracting the information required for the whitelist from newly added feature samples and decision logs in the release sample area, and updating the whitelist; periodically extracting the information required for the blacklist from newly added feature samples and decision logs in the blocking sample area, and updating the blacklist.

[0110] For example, the method further includes: A content feature matrix space is constructed based on the feature samples and corresponding decision logs in the parent correction sample area. The video segment under each sliding time window in the content feature matrix space is represented by the state and the system's control operations. The content feature matrix space is updated as the number of samples in the parent correction sample area increases.

[0111] The state includes the following characteristics: Traffic characteristics, visual characteristics, textual characteristics, feedback behavior (i.e., historical intervention characteristics), and current policy context; If a control strategy is used to determine the control operation type, the current strategy context includes: First blocking threshold, second blocking threshold, and release threshold; If a policy network is used to determine the type of control operation, the current policy context includes: The model parameters or version number of the policy network.

[0112] To allow for manual review of decisions, this application also temporarily caches the content identifier, summary information, and at least one keyframe thumbnail (e.g., keyframes from the first few seconds) of each blocked / released video stream locally. Accordingly, a client is provided to the user capable of displaying the summary information of each blocked or released video in the local cache, including the video title, source application, timestamp, and at least one keyframe thumbnail. This client also provides an entry point for manually unblocking or releasing the video, requiring the user to input credentials. When a parent inputs credentials and intervenes, the parent feedback field, the final decision field, the handling result field, and the reviewer field of the feature sample in the tag partition for that video are updated accordingly based on the video's content identifier. The updated feature sample and decision log are then re-stored in the parent correction sample area and the release or blocking sample area where they should be stored, while the corresponding records in the original blocking or release sample area are deleted.

[0113] If a user requests to allow a blocked video to play, the corresponding original video stream is retrieved from the cloud based on the video's content identifier, parsed, and displayed to the user so that the user can confirm whether to manually unblock it.

[0114] Through this client, parents can intuitively understand the system's control behavior and submit their intervention opinions, thereby improving the system's long-term accuracy and credibility.

[0115] In one feasible design, the method also includes reinforcement learning on a multimodal large model or policy network, such as... Figure 2 As shown, this can be achieved through a personalized baseline self-learning module in the self-learning layer, specifically including: The feature samples and decision logs stored in the corresponding tag partitions in the cloud are updated according to the control operations. The types of tag partitions include: blocking sample area, release sample area, and parent correction sample area. Each sample area is used to store multiple sets of feature samples and decision logs. Each set of feature samples and decision logs is associated with the same video stream content identifier and timestamp. The update of the feature samples and decision logs in the parent correction sample area triggers the update of the content feature matrix space. The video segment under each sliding time window in the content feature matrix space is represented by the status and the control operations of the system. The status includes traffic features, visual features, text features, feedback behavior, and current policy context. If the user provides feedback on the control operation, obtain the user's feedback behavior, which may include manually lifting the block or manually blocking the passage. Copy the feature samples and decision logs corresponding to the candidate frame sequence in the blocking sample area or the release sample area to the parent correction sample area and update the content feature matrix space. Delete the corresponding records in the original sample area where the candidate frame sequence is located. Reward signals are generated based on feedback behavior, and cumulative rewards are updated based on reward signals. Manual blocking and allowing passage corresponds to positive rewards, while manual unblocking corresponds to negative rewards. Based on the newly added states and control operations in the content feature matrix space, and combined with the cumulative reward, the policy parameters stored locally on the learning machine terminal are incrementally updated using a policy gradient-based reinforcement learning algorithm. The policy parameters include at least one of the following: the classification head weight parameters of the multimodal large model, the lightweight adapter parameters of the multimodal large model, the parameters of the policy network, and the threshold parameters.

[0116] The threshold parameters are the first blocking threshold, the second blocking threshold, the safety threshold, or the release threshold.

[0117] For example, the main parameters of the multimodal large model are deployed in the cloud, while the classification head weight parameters and lightweight adapter parameters are deployed locally on the learning machine terminal. The lightweight policy network is also deployed locally on the learning machine terminal. This ensures both the accuracy and generalization ability of the model's inference, while also taking into account the real-time response and personalized adaptation capabilities on the terminal side. In this case, after acquiring the visual and textual features of the candidate frame sequence, the cloud inputs them into the multimodal large model, and then uses the output of the main parameters of the multimodal large model as an intermediate feature vector, which is sent to the terminal side. The terminal side uses this vector, along with the classification head and lightweight adapter of the local multimodal large model, to complete the final inference of the probability distribution of each content category.

[0118] Specifically, if a user provides feedback on the system's control operations for the video stream through the client, the system acquires the user's feedback behavior through the operation entry point. Feedback behaviors include manually unblocking or manually blocking / allowing. A reward signal is generated based on the feedback behavior: a +1 positive reward for manually blocking / allowing and a -1 negative reward for manually unblocking. The cumulative reward is updated based on the reward signal. The feature samples and decision logs corresponding to the candidate frame sequence in the blocking or allowing sample area are copied to the parental correction sample area, and the corresponding records in the original sample area are deleted. Updates in the parental correction sample area trigger updates to the content feature matrix space, i.e., adding the state and control operation corresponding to the candidate frame sequence to the content feature matrix space. The state includes visual features, text features, traffic features, feedback behavior, and the current policy context.

[0119] Subsequently, a policy gradient-based reinforcement learning algorithm is used to incrementally update the classification head weight parameters, lightweight adapter parameters, policy network parameters, or threshold parameters deployed locally on the learning machine terminal. This ensures that the user's learned preferences are retained even after the learning machine terminal restarts. The policy gradient-based reinforcement learning algorithm can be expressed as the following formula (7): , formula (7); in, Indicates the strategy parameters, Indicates the learning rate. This indicates a control operation (blocking or allowing passage). This indicates cumulative rewards. The state is represented by the state corresponding to the video segment in the content feature matrix space, and can be denoted as . =[F_DPI, F_CV, F_NLP, F_feed, F_history], where F_DPI represents traffic features, F_CV represents visual features, F_NLP represents text features, F_feed represents feedback behavior, and F_history represents the current policy context.

[0120] In the above formula Indicates the current parameter Under the control of [the system], the system is in a state [of state]. When selecting an action The probability of. Indicates the parameter Find the gradient.

[0121] For example, for safety reasons, auditing and regression testing are required before and after each update of the multimodal large model / policy network to ensure that no new over-judgments are generated.

[0122] In the above embodiments, when parents intervene in the system's decision-making management operations, manually blocking and allowing (positive feedback) indicates that the system has missed a case for the current sample. The policy gradient update increases the probability of the policy parameters on similar state-action pairs, making the multimodal large model more inclined to output "block" when encountering video streams with similar features in the future; or it makes the policy network or control policy tend to favor the "entertainment / violation" category with higher confidence, thereby improving the system's recognition accuracy for entertainment content. Manually unblocking (negative feedback) indicates that the system has false alarms. The policy gradient update reduces the probability of the policy parameters outputting blocking for similar features, making the multimodal large model more inclined to "allow" such content, reducing false blocking of normal learning videos; or it makes the policy network or control policy tend to favor the "learning" category with higher confidence, thereby improving the system's recognition accuracy for learning content. Through the above methods, this embodiment can dynamically and directionally optimize the decision boundary according to each user's specific preferences, making the control behavior continuously approach the user's personalized expectations, realizing intelligent evolution from general rules to customized strategies.

[0123] It should be understood that offline training of multimodal large models in the cloud can also adopt reinforcement learning algorithms based on policy gradients. The implementation method can be found in the description of the aforementioned embodiments, and will not be repeated here.

[0124] For example, the personalized baseline self-learning module can also use Bayesian optimization or grid search to fine-tune the first blocking threshold, the second blocking threshold, the security threshold, or the allowance threshold.

[0125] As can be seen, this application introduces user interaction feedback and reinforcement learning, enabling the control strategy, multimodal large model and policy network to dynamically evolve and self-optimize according to the family's educational concepts and parents' behavioral habits, thus achieving intelligent adaptive control.

[0126] In a feasible design, the method also includes: If the feedback behavior includes manual blocking and allowing, the traffic features, visual features, and text features corresponding to the video segments within the current sliding time window are associated and stored in the violation content baseline library stored locally on the learning machine terminal. This is so that in subsequent sliding time windows, before fusing and inferring the visual and text features of new candidate frame sequences based on a multimodal large model, the latest violation content baseline library is matched with the relevant features (including traffic features, visual features, and text features) of the new candidate frame sequences to determine whether to perform blocking operations on the video segments corresponding to the new candidate frame sequences.

[0127] If the similarity between the relevant features of a new candidate frame sequence and the relevant features of video segments in the baseline library of illegal content exceeds a preset threshold (e.g., 0.8), it is determined to perform a blocking operation on the video segment corresponding to the new candidate frame sequence, that is, to call the underlying interface of the operating system to immediately cut off its TCP connection, clear the video buffer, etc. If the similarity between the two does not exceed a preset threshold, then it is determined not to perform blocking operations on the video segments corresponding to the new candidate frame sequence, so as to continue to perform fusion reasoning on the visual and text features of the new candidate frame sequence based on the multimodal large model, so as to obtain the probability distribution of each content category corresponding to the candidate frame sequence.

[0128] The above embodiments improve the system's real-time interception efficiency and resource utilization for known or similar illegal videos by storing the relevant features of video segments that parents manually block or allow in the illegal content baseline database. Specifically, when a user manually blocks a video that has already been allowed, the system immediately stores the relevant features of that video segment as a high-confidence negative sample in the illegal content baseline database, achieving efficient preservation of user judgments. This allows the system to prioritize rapid similarity matching of new video segment features with the baseline database before calling the multimodal large model for deep fusion inference in subsequent video stream processing. If a match is successful, blocking can be directly triggered, thus bypassing the time-consuming multimodal analysis process. This approach reduces the overhead of repeated analysis of duplicate or similar illegal content, optimizes the overall system performance while ensuring control accuracy, and makes the protection response more real-time.

[0129] For example, the local configurations of policy parameters, control policies, and violation content baseline database on the learning machine terminal can be periodically synchronized to the cloud. The violation content baseline database is uploaded to the cloud after being anonymized, allowing for the local storage of personalized family preferences while the cloud stores general violation characteristics and cross-device synchronization policies.

[0130] like Figure 3 As shown, this application also provides a method for video traffic security management applied to a learning machine terminal, including: Step S1-1: Capture the video traffic data packets of the video stream received by the terminal.

[0131] Step S1-2: Perform deep packet inspection on the video traffic data packets received by the learning machine terminal.

[0132] Step S1-3: Determine whether the traffic characteristics match the list database. If the whitelist is successfully matched, proceed to step S1-4a. If the blacklist is successfully matched, proceed to step S1-4b. If the whitelist is not matched, proceed to step S2-1.

[0133] Step S1-4a: Perform a release operation on the video stream and update the release sample area.

[0134] Step S1-4b: Perform a blocking operation on the video stream and update the blocking sample region.

[0135] Step S2-1: Based on the sliding time window, perform frame extraction on the video segments of the video stream that are located within the current sliding time window to obtain a candidate frame sequence containing 5 candidate frames.

[0136] Step S2-2: Visual feature extraction is performed on the candidate frame sequence to obtain a visual feature sequence containing the feature vector of the image semantics of each frame.

[0137] Step S2-3: Determine whether the candidate frame sequence can identify keyframes. If it can, proceed to step S2-4a; otherwise, proceed to step S2-4b.

[0138] Step S2-4a: The first text in the keyframe is extracted using optical character recognition technology. The text features of the second text, which is composed of the first text and the metadata of the video stream, are extracted to obtain the text features of each word segment.

[0139] Step S2-4b: Extract text features from the second text to obtain the text features of each word segment.

[0140] Step S3-1: Based on the multimodal large model, perform fusion reasoning on the visual features and text features of the candidate frame sequence to obtain the probability distribution of each content category corresponding to the candidate frame sequence.

[0141] Step S3-2: Determine the control operation type based on the probability distribution of each content category. If the probability of the entertainment category is greater than the first blocking threshold or the probability of the violation category is greater than the second blocking threshold, determine the control operation type as blocking and execute step S3-3a. If the probability of the learning category is greater than the allowance threshold and the probability of the violation category is less than the safety threshold, determine the control operation type as allowance and execute step S3-3b. Otherwise, determine the control operation type as pop-up confirmation and execute step S3-3c.

[0142] Step S3-3a: Perform blocking, update the local cache and the blocking sample area.

[0143] Step S3-3b: Perform release, update the local cache and release sample area.

[0144] Step S3-3c: Perform a pop-up confirmation operation. If the user confirms the blocking, proceed to step S3-3a. If the user confirms the allowing, proceed to step S3-3b.

[0145] Step S4-1: Monitor whether the user provides feedback on the control operation. If feedback is provided, proceed to step S4-2; otherwise, do not proceed with the subsequent steps.

[0146] Step S4-2: Determine the type of feedback behavior. If the feedback behavior is to manually unblock, proceed to step S4-3a. If the feedback behavior is to manually block and allow, proceed to step S4-3b.

[0147] Step S4-3a: Determine if the reward signal is a negative reward, and update the cumulative reward based on the reward signal.

[0148] Step S4-3b: Determine if the reward signal is a positive reward, update the cumulative reward based on the reward signal, and update the baseline database of violations.

[0149] Step S4-4: Periodically update the policy parameters configured locally on the learning machine terminal using a policy gradient-based reinforcement learning algorithm.

[0150] Steps S4-5: Regularly synchronize the latest strategy parameters to the cloud.

[0151] The above process achieves precise access to learned content and real-time blocking of inappropriate content through efficient filtering and deep recognition of video streams. Furthermore, a dynamic optimization mechanism continuously refines the control strategies, policy networks, and multimodal large-scale models.

[0152] This application provides a layered and progressive traffic control mechanism. Specifically, it first uses deep packet inspection to extract traffic features from the video stream and matches them with a list database. Subsequent deep analysis is only triggered when the matching confidence is insufficient, thus filtering out most obviously compliant or non-compliant traffic at the source at extremely low cost. This coarse-grained fast screening + fine-grained precise judgment architecture balances the real-time performance and accuracy of security control. For suspicious traffic, a sliding time window is used to extract frames and combine a multimodal large model to fuse and infer the visual and textual features of candidate frames, outputting a fine-grained content category probability distribution. This allows for precise control operations based on the content category probability distribution. This design breaks through the traditional static control mode based on URL / IP, enabling the system to deeply understand the semantic information of video images (such as the difference between teaching blackboard writing and entertainment scenes) and accompanying text (such as titles and text in images), effectively distinguishing between biological teaching videos and inappropriate videos, and between lectures by famous teachers and entertainment gossip, significantly reducing false alarm and false negative rates. Meanwhile, multimodal large model inference is only enabled on demand in the low-confidence gray area, while high-confidence traffic is directly decided by lightweight deep packet inspection and matching. This reduces the computational load and power consumption of the learning machine terminal while ensuring recognition accuracy, and achieves an effective balance between resource utilization and control accuracy.

[0153] like Figure 2 As shown, this application also provides a video traffic security management system for learning machine terminals, including: The deep packet inspection preprocessing module is used to perform deep packet inspection on the video traffic data packets of the video stream received by the learning machine terminal in order to extract the traffic characteristics of the video stream. The deep packet inspection preprocessing module is also used to obtain the confidence level of traffic features belonging to the list by matching traffic features with a preset list. The list includes a blacklist and a whitelist. The multimodal feature extraction module is used to extract frames of video segments within the current sliding time window if no match is found in the preset list, and to use a neural network model to segment and vectorize the text corresponding to the video segments to obtain text features. The large model inference module is used to obtain a candidate frame sequence containing a preset number of candidate frames. It uses a multimodal large model based on a cross-attention mechanism to fuse and infer the visual and text features of the candidate frame sequence to obtain the probability distribution of each content category corresponding to the candidate frame sequence. The decision-making and execution module is used to perform corresponding control operations on video segments within the current sliding time window based on the probability distribution of each content category.

[0154] In a feasible design, the system also includes a personalized baseline self-learning module, which optimizes the policy parameters in the following ways: The feature samples and decision logs stored in the corresponding tag partitions in the cloud are updated according to the control operations. The types of tag partitions include: blocking sample area, release sample area, and parent correction sample area. Each sample area is used to store multiple sets of feature samples and decision logs. Each set of feature samples and decision logs is associated with the same video stream content identifier and timestamp. The update of the feature samples and decision logs in the parent correction sample area triggers the update of the content feature matrix space. The video segment under each sliding time window in the content feature matrix space is represented by the status and the control operations of the system. The status includes traffic features, visual features, text features, feedback behavior, and current policy context. If the user provides feedback on the control operation, obtain the user's feedback behavior, which may include manually lifting the block or manually blocking the passage. Copy the feature samples and decision logs corresponding to the candidate frame sequence in the blocking sample area or the release sample area to the parent correction sample area and update the content feature matrix space. Delete the corresponding records in the original sample area where the candidate frame sequence is located. Reward signals are generated based on feedback behavior, and cumulative rewards are updated based on reward signals. Manual blocking and allowing passage corresponds to positive rewards, while manual unblocking corresponds to negative rewards. Based on the newly added states and control operations in the content feature matrix space, and combined with the cumulative reward, the policy parameters stored locally on the learning machine terminal are incrementally updated using a policy gradient-based reinforcement learning algorithm. The policy parameters include at least one of the following: the classification head weight parameters of the multimodal large model, the lightweight adapter parameters of the multimodal large model, the parameters of the policy network, and the threshold parameters.

[0155] In a feasible design, the personalized baseline self-learning module updates the violation content baseline library in the following way: If the feedback behavior includes manual blocking and allowing, the traffic features, visual features, and text features corresponding to the video segments within the current sliding time window are associated and stored in the violation content baseline library stored locally on the learning machine terminal. This is so that in subsequent sliding time windows, before fusing and inferring the visual and text features of new candidate frame sequences based on a multimodal large model, the latest violation content baseline library is matched with the relevant features of the new candidate frame sequences to determine whether to perform blocking operations on the video segments corresponding to the new candidate frame sequences.

[0156] In a feasible design, the multimodal feature extraction module is also used to extract multimodal features before fusing and inferring the visual and textual features of the candidate frame sequence using a multimodal large model based on a cross-attention mechanism, by means of the following: A visual feature extraction network is used to extract the visual features of each candidate frame in the candidate frame sequence, thereby obtaining the visual feature sequence corresponding to the candidate frame sequence. Specifically, a neural network model is used to segment and vectorize the text corresponding to the video clips to obtain text features, including: If a keyframe is identified from the candidate frame sequence, the first text in the keyframe is extracted using optical character recognition technology. The second text, which is composed of the first text and the metadata of the video stream, is then segmented and vectorized to obtain the text features of each segment corresponding to the first text and the second text. The first text and the second text are the text corresponding to the video segment. If no keyframe is identified from the candidate frame sequence, the second text is segmented and vectorized to obtain the text features of each segmented word in the second text. The second text is the text corresponding to the video segment.

[0157] In a feasible design, the large model inference module is implemented by using a multimodal large model based on a cross-attention mechanism to fuse and infer the visual and textual features of the candidate frame sequence, thereby obtaining the probability distribution of each content category corresponding to the candidate frame sequence: Determine the attention weight of the text features of each word to each visual feature in the visual feature sequence; The visual features in the visual feature sequence are weighted and summed according to the attention weights of each visual feature corresponding to each word to obtain the visual context vector corresponding to each word. The global visual context vector is obtained by weighted summing of the visual context vectors corresponding to each word segmentation. The global visual context vector and the global text feature vector are concatenated and then input into the classifier to obtain the probability distribution of each content category corresponding to the candidate frame sequence.

[0158] In a feasible design, the large model inference module is implemented by weighted summing of the visual context vectors corresponding to each word segmentation to obtain the global visual context vector: Input a sequence of text features consisting of the text features of each word into a self-attention network that employs a self-attention mechanism, and obtain the importance score of each word; The importance score of each word is normalized to obtain the attention weight corresponding to each word. The visual context vectors corresponding to each word are weighted and summed based on the attention weights corresponding to each word to obtain the global visual context vector.

[0159] In a feasible design, each content category includes a first category, a second category, and a third category. The first category indicates that the video content is educational, the second category indicates that the video content is entertainment, and the third category indicates that the video content is illegal. The decision-making and execution module performs corresponding control operations on the video segments within the current sliding time window based on the probability distribution of each content category: If the probability of the second category is greater than the first blocking threshold, or the probability of the third category is greater than the second blocking threshold, the control operation type is determined to be blocking according to the control strategy, and the blocking operation is performed on the video segment within the current sliding time window. The control strategy is used to specify the type of control operation based on the threshold. If the probability of the third category is less than the safety threshold and the probability of the first category is greater than the release threshold, the control operation type is determined to be release according to the control strategy, and the release operation is performed on the video segment in the current sliding time window, where the safety threshold is less than the second blocking threshold. Otherwise, if the control operation type is determined to be a pop-up confirmation according to the control strategy, the pop-up confirmation operation is executed. The pop-up confirmation operation is used to display a pop-up to the user so that the user can confirm whether to block the video segment in the current sliding time window after entering the password.

[0160] In a feasible design, if a blocking operation is performed on a video segment within the current sliding time window, the decision-making and execution module also achieves proactive defense through the following methods: Update the number of blocking times corresponding to the network address based on the network address corresponding to the video stream; If the number of blocking attempts exceeds the threshold, the access control list will be adjusted based on the network address to block or restrict subsequent communication connections with the network address.

[0161] In a feasible design, the decision-making and execution module is also implemented by performing corresponding control operations on video segments within the current sliding time window based on the probability distribution of each content category: The probability distribution of each content category is input into the policy network, which employs a multilayer perceptron mechanism. The type of control operation output by the policy network for the video segment within the current sliding time window is obtained. The types of control operations include allowing, blocking, or pop-up confirmation. Perform the corresponding control operation based on the type of control operation.

[0162] Other implementation methods and effects of the above system can be found in the description of the embodiments of the video traffic security management method applied to learning machine terminals, and will not be repeated here.

[0163] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0164] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0165] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0166] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0167] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0168] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for secure video traffic management applied to a learning machine terminal, characterized in that, include: Deep packet inspection is performed on the video traffic data packets of the video stream received by the learning machine terminal to extract the traffic characteristics of the video stream; By matching the traffic features with a preset list database, the confidence level of whether the traffic features belong to the list database is obtained. The list database includes a blacklist and a whitelist. If no match is found in the preset list, the video segments within the current sliding time window are processed by frame extraction based on the sliding time window, and the text corresponding to the video segments is segmented and vectorized using a neural network model to obtain text features. A candidate frame sequence containing a preset number of candidate frames is obtained. A multimodal large model based on a cross-attention mechanism is used to fuse and infer the visual and text features of the candidate frame sequence to obtain the probability distribution of each content category corresponding to the candidate frame sequence. Based on the probability distribution of each content category, corresponding control operations are performed on the video segments within the current sliding time window.

2. The method according to claim 1, characterized in that, The method further includes: The feature samples and decision logs stored in the corresponding tag partitions in the cloud are updated according to the control operation. The types of tag partitions include: blocking sample area, allowing sample area, and parent correction sample area. Each sample area is used to store multiple sets of feature samples and decision logs. Each set of feature samples and decision logs is associated with the content identifier and timestamp of the same video stream. The update of the feature samples and decision logs in the parent correction sample area triggers the update of the content feature matrix space. The video segment under each sliding time window in the content feature matrix space is represented by the status and the control operation of the system. The status includes traffic features, visual features, text features, feedback behavior and current policy context. If the user provides feedback on the control operation, obtain the user's feedback behavior on the control operation, including manually unblocking or manually blocking access; Copy the feature samples and decision logs corresponding to the candidate frame sequence in the blocking sample area or the release sample area to the parent correction sample area and update the content feature matrix space, and delete the corresponding records in the original sample area where the candidate frame sequence is located; A reward signal is generated based on the feedback behavior, and the accumulated reward is updated based on the reward signal, wherein the manual blocking and release corresponds to a positive reward, and the manual unblocking corresponds to a negative reward; Based on the newly added states and control operations in the content feature matrix space, and combined with the cumulative reward, the policy parameters stored locally on the learning machine terminal are incrementally updated using a policy gradient-based reinforcement learning algorithm. The policy parameters include at least one of the following: the classification head weight parameters of the multimodal large model, the lightweight adapter parameters of the multimodal large model, the parameters of the policy network, and the threshold parameters.

3. The method according to claim 2, characterized in that, The method further includes: If the feedback behavior includes manual blocking and allowing, the traffic features, visual features, and text features corresponding to the video segments within the current sliding time window are associated and stored in the violation content baseline library stored locally on the learning machine terminal. This is so that in subsequent sliding time windows, before fusing and inferring the visual and text features of new candidate frame sequences based on a multimodal large model, the latest violation content baseline library is matched with the relevant features of the new candidate frame sequences to determine whether to perform blocking operations on the video segments corresponding to the new candidate frame sequences.

4. The method according to any one of claims 1-3, characterized in that, Before employing a multimodal large model based on a cross-attention mechanism to perform fusion inference on the visual and textual features of the candidate frame sequence, the method further includes: A visual feature extraction network is used to extract the visual features of each candidate frame in the candidate frame sequence to obtain the visual feature sequence corresponding to the candidate frame sequence. The step of using a neural network model to segment and vectorize the text corresponding to the video clips to obtain text features includes: If a keyframe is identified from the candidate frame sequence, the first text in the keyframe is extracted using optical character recognition technology. The first text and the second text composed of the metadata of the video stream are then segmented and vectorized to obtain the text features of each segmented word corresponding to the first text and the second text. The first text and the second text are the text corresponding to the video segment. If no keyframe is identified from the candidate frame sequence, the second text is segmented and vectorized to obtain the text features of each segmented word corresponding to the second text. The second text is the text corresponding to the video segment.

5. The method according to claim 4, characterized in that, The method employs a multimodal large model based on a cross-attention mechanism to fuse and infer the visual and textual features of the candidate frame sequence, obtaining the probability distribution of each content category corresponding to the candidate frame sequence, including: Determine the attention weight of the text features of each word to each visual feature in the visual feature sequence; The visual features in the visual feature sequence are weighted and summed according to the attention weight of each visual feature corresponding to each word to obtain the visual context vector corresponding to each word. The global visual context vector is obtained by weighted summing of the visual context vectors corresponding to each word segmentation. The global visual context vector and the global text feature vector are concatenated and then input into the classifier to obtain the probability distribution of each content category corresponding to the candidate frame sequence.

6. The method according to claim 5, characterized in that, The step of weighted summing of the visual context vectors corresponding to each word segment to obtain the global visual context vector includes: Input a sequence of text features consisting of the text features of each word into a self-attention network that employs a self-attention mechanism, and obtain the importance score of each word; The importance score of each word is normalized to obtain the attention weight corresponding to each word. The visual context vectors corresponding to each word are weighted and summed based on the attention weights corresponding to each word to obtain the global visual context vector.

7. The method according to any one of claims 1-3, characterized in that, Each content category includes a first category, a second category, and a third category, where the first category indicates video content is educational, the second category indicates video content is entertainment, and the third category indicates video content is prohibited. The step of performing corresponding control operations on video segments within the current sliding time window based on the probability distribution of each content category includes: If the probability of the second category is greater than the first blocking threshold, or the probability of the third category is greater than the second blocking threshold, the type of control operation is determined to be blocking according to the control strategy, and the blocking operation is performed on the video segment within the current sliding time window. The control strategy is used to specify the type of control operation based on the threshold. If the probability of the third category is less than the safety threshold and the probability of the first category is greater than the release threshold, the control operation type is determined to be release according to the control strategy, and the release operation is performed on the video segment in the current sliding time window, where the safety threshold is less than the second blocking threshold. Otherwise, if the control operation type is determined to be a pop-up confirmation according to the control strategy, the pop-up confirmation operation is executed. The pop-up confirmation operation is used to display a pop-up to the user so that the user can confirm whether to block the video segment in the current sliding time window after entering the password.

8. The method according to claim 7, characterized in that, If a blocking operation is performed on a video segment within the current sliding time window, the method further includes: Update the number of blocking times corresponding to the network address based on the network address corresponding to the video stream; If the number of blocking attempts exceeds a threshold, the access control list is adjusted based on the network address to block or restrict subsequent communication connections with the network address.

9. The method according to claim 1 or 2, characterized in that, The control operations performed on the video segments within the current sliding time window based on the probability distribution of each content category include: The probability distribution of each content category is input into the policy network, which employs a multilayer perceptron mechanism. Obtain the type of control operation corresponding to the video segment within the current sliding time window output by the policy network. The type of control operation includes allowing, blocking, or pop-up confirmation. Execute the corresponding control operation according to the type of control operation.

10. A video traffic security management system for learning machine terminals, characterized in that, include: The deep packet inspection preprocessing module is used to perform deep packet inspection on the video traffic data packets of the video stream received by the learning machine terminal in order to extract the traffic characteristics of the video stream. The deep packet inspection preprocessing module is also used to obtain the confidence level of the traffic feature belonging to the list database by matching the traffic feature with a preset list database, wherein the list database includes a blacklist and a whitelist. The multimodal feature extraction module is used to extract frames from the video segments of the video stream that are located within the current sliding time window if no match is found in the preset list, and to use a neural network model to segment and vectorize the text corresponding to the video segments to obtain text features. The large model inference module is used to obtain a candidate frame sequence containing a preset number of candidate frames, and to use a multimodal large model based on a cross-attention mechanism to perform fusion inference on the visual features and text features of the candidate frame sequence to obtain the probability distribution of each content category corresponding to the candidate frame sequence. The decision-making and execution module is used to perform corresponding control operations on video segments within the current sliding time window based on the probability distribution of each content category.

Images