Video content auditing method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, and provides a video content auditing method and device, electronic equipment and a storage medium. For a to-be-audited video, partitioning the extracted key frame, extracting visual information of each image block, generating a visual token, and based on the association degree between each visual token and a set content query task and the attribute information of the to-be-audited video, performing hierarchical compression processing on each visual token, therefore, key information is prevented from being lost, calculation overhead is reduced, auditing efficiency and accuracy are effectively balanced, an auditing conclusion is given for the to-be-audited video, a visual basis supporting the auditing conclusion is given from the two dimensions of time and space, and the visual basis comprises a visual basis in the to-be-audited video and a visual basis in the to-be-audited video. According to the content description text of the video clip corresponding to the at least one timestamp interval associated with the content query task, the black box auditing process is well documented, and the credibility and reliability of an auditing conclusion are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and provides a method, apparatus, electronic device, and storage medium for reviewing video content. Background Technology

[0002] With the explosive growth of video content, efficient and accurate content review of long videos (such as movies, documentaries, live recordings, and edited videos) that are tens of minutes or even hours long has become a key challenge in maintaining network security and preventing the spread of risky content.

[0003] Currently, a basic review method involves first performing frame sampling on long videos at fixed intervals, converting the video stream into a series of static images. Then, traditional image recognition models (such as CNNs and Transformers) or simple rule engines are used to analyze these keyframes to identify specific objects, scenes, or symbols. While this method is simple to implement and computationally inexpensive, it ignores the temporal dimension of the video. Therefore, when risky content is not determined by a single frame but rather accumulates and evolves over time through a series of seemingly normal images, the model cannot understand the continuity of actions and the contextual relationships of events, making it difficult to identify complex risky content that is "progressive" or "contextual."

[0004] With the development of Vision Language Models (VLMs), several methods focusing on video clip review have been proposed. These methods divide long videos into several segments and then perform independent semantic understanding and risk assessment on each segment. Compared to static frame extraction, video clips can capture dynamic information within a short period of time, but they mainly suffer from two problems: 1. "Context fragmentation," which makes it impossible to effectively reason about complex events (such as premeditation, deception, etc.) that span multiple video clips and have long temporal dependencies; 2. "Information redundancy," which leads to inefficiency due to repeated calculations on a large number of risk-free background segments in the video.

[0005] Therefore, improving the accuracy of video content review has become an important aspect of maintaining network security. Summary of the Invention

[0006] This application provides a method, apparatus, electronic device, and storage medium for reviewing video content, in order to improve the accuracy and credibility of video content review.

[0007] Firstly, embodiments of this application provide a method for reviewing video content, including: Extract the set of visual tokens from the video to be reviewed; each visual token is used to describe: the visual information of an image patch in a keyframe contained in the video to be reviewed; Based on the content query task set for the video to be reviewed, and the attribute information of the video to be reviewed, and combined with the correlation between each visual token in the visual token set and the content query task, the visual tokens are compressed in layers to obtain a compressed token sequence after splicing at each layer. Based on the visual token sequence, the review conclusion of the video to be reviewed and the visual evidence supporting the review conclusion are obtained; the visual evidence includes: the content description text of the video segment corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed.

[0008] Secondly, embodiments of this application provide a video content review device, comprising: The token extraction module is used to extract the set of visual tokens from the video to be reviewed; each visual token is used to describe the visual information of an image block in a keyframe contained in the video to be reviewed. The layered compression module is used to compress each visual token in layers based on the content query task set for the video to be reviewed, the attribute information of the video to be reviewed, and the correlation between each visual token in the visual token set and the content query task, so as to obtain a compressed token sequence after splicing the layers. The conclusion and evidence module is used to obtain the review conclusion of the video to be reviewed and the visual evidence supporting the review conclusion based on the visual token sequence; the visual evidence includes: the content description text of the video segments corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed.

[0009] Optionally, the layered compression module is specifically used for: Based on the content query task set for the video to be reviewed and the attribute information of the video to be reviewed, select a token number from a preset token number range; Based on the text features and visual tokens of the content query task, an attention mechanism is used to obtain the correlation degree between each visual token and the content query task. Based on the selected number of tokens and the obtained correlation, the visual tokens are compressed hierarchically to obtain a compressed token sequence after splicing at each level.

[0010] Optionally, the layered compression module is specifically used for: The correlation degrees are sorted to obtain the sorting results; Based on the sorting results, the visual tokens corresponding to the K highest relevance are selected as the first level, and the visual tokens corresponding to the remaining N highest relevance are selected as the second level; K is the number of selected tokens, and N is greater than or equal to 1 but less than or equal to the difference between the total number of visual tokens and K. Based on the compression strategy corresponding to each level, the visual tokens in the corresponding level are compressed to obtain the corresponding token subsequence; The compressed token sequence is obtained by concatenating the token subsequences of the first level and the second level.

[0011] Optionally, the layered compression module is specifically used for: Calculate the token mean of N visual tokens in the second level, and use the token mean as the token subsequence of the second level; Iterate through the first level sequentially, performing the following operations on each iteration: Obtain the first semantic similarity between the i-th visual token and at least one candidate visual token; wherein, the candidate visual token is: a visual token located after the i-th visual token that has not been merged; For any candidate visual token whose first semantic similarity is not lower than a preset similarity threshold, obtain the associated visual token of the candidate visual token, and merge the associated visual token, the candidate visual token, and the i-th visual token to obtain a new visual token; wherein, the associated visual token is another candidate visual token whose first semantic similarity is not lower than the preset similarity threshold with the candidate visual token. The new visual token is assigned to the token subsequence of the first level, and the i-th visual token and the merged candidate visual token are removed.

[0012] Optionally, the conclusion and evidence module is specifically used for: Based on the visual token sequence, obtain at least one timestamp interval associated with the content query task in the video to be reviewed; For each timestamp interval, based on at least one visual token corresponding to the timestamp interval, the content description text of the video segment corresponding to the timestamp interval is obtained, and combined with the timestamp interval, a visual basis is generated. The review conclusion of the video to be reviewed is obtained based on at least one visual criterion.

[0013] Optionally, the review conclusion and the visual evidence supporting the review conclusion are obtained using a review model. The device is used to iteratively train the video review model based on a sample video set. Each sample video has a conclusion label and an evidence label. The evidence label includes: content description text of video segments corresponding to video frames in at least one timestamp interval associated with the conclusion label in the sample video; wherein, in each round of training: The token extraction module is also used to extract the sample visual token set of the selected sample video; The hierarchical compression module is further configured to compress each sample visual token in a hierarchical manner based on the sample query task set for the sample video and the sample attribute information of the sample video, and in combination with the sample correlation degree between each sample visual token contained in the sample visual token set and the sample query task, to obtain the sample compressed token sequence and token compression loss after each level is spliced. The evidence and strategy module is also used to obtain the sample review conclusion of the sample video and the sample visual evidence supporting the sample review conclusion based on the sample compression token sequence of the sample video. The adjustment module is used to perform model parameter tuning based on the token compression loss, the conclusion loss between the sample review conclusion and the corresponding conclusion label, and the evidence loss between the sample visual evidence and the corresponding evidence label.

[0014] Optionally, the layered compression module is specifically used for: Based on the sample query task set for the sample video and the sample attribute information of the sample video, select a number of tokens from a preset range of token numbers. Based on the selected number of tokens and the sample relevance of each sample visual token to the sample query task, the sample visual token set is compressed hierarchically. Then, based on the consistency between each sample relevance and the corresponding reference relevance, the selection evaluation value of the number of tokens is calculated. The reference relevance is the relevance between the sample visual token and the sample query task obtained using a trained teacher model. Based on the selected evaluation value, the token compression loss is obtained.

[0015] Optionally, the layered compression module is specifically used for: The text features of the sample query task and the text features of the sample attribute information are fused to obtain fused text features; Based on the fused text features, the probability distribution of the token number range is obtained; Based on the probability distribution, a number of tokens is selected from the range of token numbers.

[0016] Optionally, the adjustment module is specifically used for

[0017] For each sample video, the following steps are performed: based on the time overlap between at least one timestamp interval in the sample visual evidence and at least one tag timestamp interval in the corresponding evidence tag, and the second semantic similarity between the sample visual evidence and the corresponding evidence tag, to obtain a matching evaluation value between the sample visual evidence and the corresponding evidence tag. Based on the obtained matching evaluation values, the evidence loss between the visual evidence of the sample and the corresponding evidence label is obtained.

[0018] Optionally, the adjustment module is specifically used for: Based on the matching evaluation values, the precision and recall of the sample video set are obtained respectively; wherein, the precision is used to quantify the model's ability to output correct results, and the recall is used to quantify the model's ability to detect risks; The evidence loss is obtained based on the harmonic mean of the precision and the recall.

[0019] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any of the above-described video content review methods.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium including a computer program, which, when run on an electronic device, causes the electronic device to perform the steps of any of the above-described video content review methods.

[0021] Fifthly, embodiments of this application provide a computer program product, the program product including a computer program stored in a computer-readable storage medium, wherein a processor of an electronic device reads from and executes the computer program from the computer-readable storage medium, causing the electronic device to perform the steps of any of the above-described video content review methods.

[0022] The beneficial effects of the video content review method, apparatus, electronic device, and storage medium provided in this application embodiment are as follows: The keyframes in the video to be reviewed are divided into blocks, and the visual information of each image block is used as a visual token. The correlation between each visual token and the set content query task is calculated. Combined with the attribute information of the video to be reviewed, the visual tokens are compressed in layers. This significantly reduces the computational overhead without sacrificing key information, effectively balancing review efficiency and accuracy. At the same time, the compressed token sequence spliced ​​from each layer is sequentially linked with the visual tokens in the video to be reviewed that are related to the content query task, realizing a "progressive" and "contextual" complex review, further improving the accuracy of the review.

[0023] On the other hand, for the videos to be reviewed, not only are review conclusions given, but also visual evidence supporting the review conclusions is provided from both time and space dimensions. The visual evidence is presented in the form of content description text of video segments corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed, turning the black-box review process into visible visual facts, thereby increasing the credibility and reliability of the review conclusions.

[0024] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application; Figure 2 A schematic diagram of the training process of the audit model provided in this application embodiment; Figure 3 A schematic diagram illustrating the generation process of the sample visual token set provided in the embodiments of this application; Figure 4 A schematic diagram illustrating the compression process of the sample visual token set provided in this application embodiment; Figure 5 A schematic diagram illustrating the token number selection process provided in this application embodiment; Figure 6 A schematic diagram of the layering process of the visual token provided in this application embodiment; Figure 7 A schematic diagram of the hierarchical compression process of the visual token provided in this application embodiment; Figure 8This is a schematic diagram illustrating the output of the audit results provided in an embodiment of this application. Figure 9 Network architecture diagram of the audit model provided in this application embodiment; Figure 10 A flowchart illustrating a video content review method provided in this application embodiment; Figure 11 A flowchart illustrating the hierarchical compression process of video tokens provided in this application embodiment; Figure 12 A schematic diagram illustrating the selection of the number of tokens provided in this application embodiment; Figure 13 A schematic diagram of the review process provided for an embodiment of this application; Figure 14 A structural diagram of a video content review device provided in an embodiment of this application; Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 16 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0027] The following is an explanation of the terms used in the embodiments of this application.

[0028] Token: A feature extracted from an image or text.

[0029] Black box: External observers can only see the relationship between the model's inputs and outputs, but cannot obtain or understand the model's internal working mechanism or decision-making process.

[0030] White box: External observers can fully understand the model's internal structure, decision-making process, etc., emphasizing the model's interpretability and transparency.

[0031] The design concept of the embodiments of this application will be briefly introduced below.

[0032] Intelligent content moderation for videos is a crucial technology for maintaining network security. Currently, related technologies directly process complete long videos using visual semantic models, but this approach suffers from the following main problems: (i) It is difficult to balance the efficiency and accuracy of the review process. The number of visual tokens for long videos will result in high computational costs, and pruning or merging visual tokens will lead to the loss of crucial, transient evidence, affecting the accuracy of the review and resulting in "missed" videos.

[0033] (ii) Model reasoning is merely speculation and cannot be seen.

[0034] When processing long-term information, the Visual Model's (VLM) comprehension ability significantly declines. It is more prone to misunderstanding events far from the beginning or end of a video. Furthermore, VLM tends to make "inferences" or "associations" based on incomplete and vague memories rather than reasoning based on solid visual facts. This leads to two fatal review errors: 1. Factual errors, i.e., incorrectly judging the video content; 2. Illusions of rationality, i.e., the model generates seemingly plausible descriptions that did not occur in the video to support its erroneous conclusions.

[0035] (iii) The audit conclusions are "black box" and have poor reliability.

[0036] Related technologies typically only assess the accuracy of the conclusions given by VLM and do not verify whether the model's judgments are rooted in specific, verifiable visual facts in the video. This often results in VLM only providing a final review conclusion of risk or no risk, but it is impossible to determine whether the correct judgment of the review conclusion stems from VLM's true understanding or random conjecture. This "black box" review conclusion is not only unreliable, poorly interpretable, and lacks grounds for appeal, but also makes continuous optimization and targeted improvement of VLM extremely difficult, greatly hindering the iterative optimization and application of VLM.

[0037] In view of this, this application provides a video content review method. This method designs an adaptive visual information compression strategy and an evidence tracing mechanism, enabling efficient, accurate, and interpretable review of video content. Specifically, in the adaptive visual information compression strategy, based on the content query task set for the video to be reviewed and the attribute information of the video, the optimal number of tokens K for hierarchical compression is dynamically predicted. The top K visual tokens with high relevance to the content query task undergo low-loss refinement processing, while the remaining visual tokens with low relevance to the content query task undergo high-rate summary compression. This significantly reduces computational overhead without sacrificing key visual information, solving the bottleneck problem of review efficiency. Simultaneously, the compressed token sequence after each level of compression sequentially links the visual tokens related to the content query task in the video to be reviewed, supporting "progressive" and "contextual" complex reviews, effectively improving review accuracy. In the evidence tracing mechanism, for the video to be reviewed, the model not only outputs the final review conclusion, but also provides visual evidence to support the review conclusion from both time and space dimensions. The visual evidence is presented in the form of content description text of video segments corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed. It can accurately point out which part and which content in the video constitutes the risk facts, turning the black box process of review into verifiable visual facts, thereby increasing the credibility and reliability of the review conclusion.

[0038] It is understood that, in the specific implementation of this application, the videos to be reviewed and sample videos involved need to be licensed or agreed upon when used in the methods or products of the following embodiments of this application, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0039] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0040] See Figure 1 As shown, it is a schematic diagram of an application scenario provided by an embodiment of this application. In this scenario, at least one terminal device 100 and a server 200 may be included.

[0041] In one embodiment, the terminal device 100 is equipped with a camera and video software, used to provide the video to be reviewed or a sample video, as well as to present the review conclusion and visual evidence supporting the review conclusion. The server 200 is the backend server for the video software, mainly used for video review and model training.

[0042] In one embodiment, the terminal device 100 includes, but is not limited to, smartphones, smart TVs, laptops, desktop computers, tablets, and in-vehicle terminals. The server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, but is not limited to these.

[0043] In one implementation, the server 200 can also be configured with a database, which can be used to store data such as videos to be reviewed, sample video sets, model parameters, etc.

[0044] Terminal device 100 and server 200 can communicate directly or indirectly through one or more networks. The network can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network, or a Wireless-Fidelity (WIFI) network, or a leased line. This application embodiment does not limit this.

[0045] Of course, the methods provided in the embodiments of this application are not limited to... Figure 1 The application scenarios shown can also be used in other possible scenarios, and this application embodiment does not impose any limitations. Figure 1 The functions that each device in the application scenario shown can achieve will be described in the subsequent method embodiments, and will not be elaborated on here.

[0046] The video content review method provided in this application can first train a VLM model to obtain a review model. Then, after inputting the video to be reviewed into the review model, the review conclusion and visual evidence supporting the review conclusion can be obtained in real time.

[0047] Typically, before model training, source videos are cleaned and manually labeled to generate a sample video set. These source videos can be obtained through filming, collection from video websites, or acquisition from open-source datasets.

[0048] In some embodiments, the sample videos in the sample video set include, but are not limited to, live recordings, TV series, documentaries, movies, edited videos, and recorded videos.

[0049] In some embodiments, in the sample video set, each sample video is provided with a conclusion label and an evidence label. The conclusion label contains the result of a sample query task set for the sample video, and the evidence label contains: the content description text of the video segment corresponding to the video frame contained in at least one timestamp interval associated with the conclusion label in the sample video.

[0050] In some embodiments, the timestamp range and content description text in the conclusion label can be presented in a structured form of "timestamp range + content description text".

[0051] For example, the conclusion label of a sample video is: fighting, and the evidence labels are: {[01:00:34-01:00:56], two people are punching each other in the video}, {[01:01:56-01:02:04], the attacker is holding a metal rod-like object}.

[0052] After generating a sample video set, the video review model is iteratively trained based on this set. For example... Figure 2 The diagram shown illustrates the training process of the audit model, which mainly includes the following steps: S201: Extract the sample visual token set from the selected sample video.

[0053] Each sample visual token is used to describe the visual information of an image patch in a keyframe of the sample video.

[0054] During model training, for each sample video in a batch, multiple keyframes are extracted according to a preset sampling frame rate. For each keyframe, it is divided into blocks according to a preset size. The content (spatial angle) and the position (temporal angle) of the frame to which each image block belongs are feature extracted to obtain sample visual tokens. In this way, each sample visual token is used to describe the visual information of an image block at a specific time and a specific location.

[0055] like Figure 3 The diagram illustrates the generation process of the sample visual token set. Assuming the sample video has a total of 300 frames and a total duration of 10 seconds, two video frames are extracted per second as keyframes, and each keyframe is divided into 16 segments. Each keyframe contains 256 visual tokens, and the final generated sample visual token set contains 5120 visual tokens.

[0056] It should be noted that the embodiments of this application do not impose restrictive requirements on the frame extraction method and number, as well as the preset size, which can be set according to the actual situation.

[0057] S202: Based on the sample query task set for the sample video and the sample attribute information of the sample video, and combined with the sample correlation degree between each sample visual token contained in the sample visual token set and the sample query task, the sample visual tokens are compressed in layers to obtain the sample compressed token sequence and token compression loss after splicing at each layer.

[0058] Typically, the length window of a VLM is limited. Therefore, an adaptive visual information compression strategy is designed in the auditing model for the sample visual token set. This strategy models the decision process of "selecting the optimal number of tokens" as a reinforcement learning task. In this way, for specific tasks set by different videos, the number of tokens can be dynamically predicted. Compared with using a fixed number of tokens and compression ratio, this can effectively improve the flexibility of sample visual token set compression.

[0059] like Figure 4 The diagram shown illustrates the compression process of the sample visual token set, which mainly includes the following steps: S2021: Based on the sample query task set for the sample video and the sample attribute information of the sample video, select a token number from the preset token number range.

[0060] The sample attribute information includes metadata such as the total number of frames, total duration, and number of sampled frames of the sample video. Combined with the sample query task set for the sample video, a decision is made on the number of tokens required to complete the task.

[0061] In the auditing model, the decision network for the number of tokens One parameter can be used The lightweight multilayer perceptron (MLP) implementation for control is as follows: 1. The text features of the sample query task and the text features of the sample attribute information are fused to obtain fused text features.

[0062] like Figure 5 As shown, assuming the sample query task is "whether there is smoking behavior in public places", the sample query task and sample attribute information are input into the decision network for feature extraction. The text features extracted from the sample query task are used to express the task intent, and the text features extracted from the sample attribute information express the physical attributes of the sample video (total number of frames, total duration, number of sampling frames, etc.). The fused text features obtained by splicing the two contain all the information needed to make a decision. Therefore, the fused text features can be used as the state space of the decision model.

[0063] 2. Based on the fused text features, obtain the probability distribution of the token number range.

[0064] The token number range contains some preset, discrete token numbers, which can serve as the action space for the decision network. Based on the fused text features, the decision network outputs a probability distribution of the token number range, where each probability represents the likelihood of selecting the corresponding token number.

[0065] 3. Select a number of tokens from the range of token numbers based on the probability distribution.

[0066] Typically, probability distributions can guide decision-making models to select a specific number of tokens.

[0067] like Figure 5 As shown, assuming the number of tokens are 2048, 4096, and 8192, and the probability distribution is p1, p2, and p3, where p2>p3>p1, the final number of tokens selected is 4096. That is, more than 4096 sample visual tokens are needed to answer whether there is fighting behavior in the sample video.

[0068] During training, based on the sample query task and sample attribute information associated with the sample video, a specific number of tokens is dynamically selected from a preset range of token numbers for subsequent token compression. This ensures that the selected number of tokens meets the task requirements, reducing the token length while preserving key information in the sample video.

[0069] S2022: Based on the selected number of tokens and the correlation between each sample visual token and the sample query task, the sample visual token set is compressed in layers to obtain the sample compressed token sequence after splicing at each layer.

[0070] The greater the sample relevance, the more important the sample visual token is for accurately answering the sample query task.

[0071] During training, a cross-modal attention mechanism can be used to calculate the sample association degree between the sample visual token and the sample query task.

[0072] In some embodiments, a lightweight VLM (such as InternVL2.5-0.5B) is used as a token selector. The sample visual token set and the sample query task are input into the token selector together. The sample query task is used as the query matrix, and each sample visual token in the sample visual token set is used as the key matrix. The attention weight of each sample visual token relative to the sample query task is calculated. After processing the attention weight (such as mean, maximum, etc.), the sample correlation degree between each sample visual token and the sample query task is obtained.

[0073] like Figure 6As shown, assuming the sample query task is "whether there is smoking behavior in public places", the five words "whether", "exist", "public places", "smoking", and "behavior" are each used as a query dimension, forming a query matrix. The sample visual token set contains 10 visual tokens, each containing different visual information, representing a key matrix. Visual token 1 is the cigarette outline feature, visual token 2 is the smoke feature, visual token 3 is the hand holding the cigarette feature, visual token 4 is the mouth smoking feature, visual token 5 is the lighter flame feature, visual token 6 is the environmental scene feature, visual token 7 is the no-smoking sign feature, visual token 8 is the crowd feature, visual token 9 is the facial expression feature, and visual token 10 is the cigarette ash feature. A dot product operation is performed on each visual token and each of the five query matrices to obtain a... The attention weight matrix is ​​used to determine the maximum attention weight of each visual token in the five query matrices, which is taken as the sample association degree between the visual token and the sample query task.

[0074] After obtaining the correlation degree between each visual token in the sample visual token set and the sample query task, and combining the selected number of tokens, the visual tokens of each sample are compressed hierarchically to obtain the sample compressed token sequence after concatenation at each level. In this way, compared with the original visual token set, the data volume of the concatenated sample compressed token sequence is smaller, which can effectively improve the computational efficiency of the model. Furthermore, through hierarchical compression, different degrees of compression ratio can be supported, thereby reducing the loss of key visual information and reducing the redundancy of information above and below.

[0075] In some embodiments, based on the number of selected tokens, each sample visual token is divided into two levels, and each level is compressed using a different method. The specific compression process is as follows: First, the correlation of each sample is sorted to obtain the token ranking result.

[0076] by Figure 6 Taking the 10 sample visual tokens as an example, the token sorting results from high to low sample relevance are: [sample visual token 1, sample visual token 6, sample visual token 7, sample visual token 2, sample visual token 3, sample visual token 4, sample visual token 8, sample visual token 5, sample visual token 9, sample visual token 10].

[0077] Then, based on the token sorting results, the visual tokens corresponding to the K largest sample associations are selected as the first level, and the visual tokens corresponding to the remaining N sample associations are selected as the second level. Here, K is the number of selected tokens, and N is greater than or equal to 1 but less than or equal to the difference between the total number of sample visual tokens and K.

[0078] by Figure 6 Taking 10 sample visual tokens as an example, assuming the number of tokens K is 6 and N is 2, then sample visual token 1, sample visual token 2, sample visual token 3, sample visual token 4, sample visual token 6 and sample visual token 7 are the first level, and sample visual token 5 and sample visual token 8 are the second level.

[0079] For the sample visual token set, the lower the sample relevance, the smaller its impact on the sample query task. The number of tokens selected reflects the number of tokens required to accurately answer the sample query task. Therefore, the first K sample visual tokens carry the core visual information of the sample query task. This part should retain most of the key visual details with minimal distortion. Therefore, this part is used as the first level and compressed using the same strategy. The remaining first N sample visual tokens may contain background information useful for the sample query task. Therefore, this part is used as the second level to supplement the context information with less computation and prevent key visual information from being misunderstood due to missing context.

[0080] Secondly, based on the compression strategy corresponding to each level, the sample visual tokens in the corresponding level are compressed to obtain the corresponding token subsequence.

[0081] The first-level sample visual tokens carry the main keyframes for answering the sample query task. To reduce the loss of key visual information, a lightweight compression strategy can be used on the first-level sample visual tokens to retain most of the key visual details with minimal distortion. The second-level sample visual tokens may contain auxiliary keyframes that provide contextual information for answering the sample query task. A high-ratio, summary compression strategy can be used on the second-level sample visual tokens to supplement the contextual information with less computation and prevent key visual information from being misunderstood due to missing context.

[0082] In some embodiments, the compression strategy for the first-level sample visual tokens is as follows: Iterate through the first level in sequence. For the i-th visual token in the current iteration, perform the following operations: Obtain the first semantic similarity between the i-th sample visual token and at least one candidate visual token, wherein the candidate visual token is: the sample visual token located after the i-th sample visual token and not merged; For any candidate visual token whose first semantic similarity is not lower than a preset similarity threshold, obtain the associated visual token of the candidate visual token, and merge the associated visual token, the candidate visual token and the i-th sample visual token to obtain a new sample visual token; wherein, the associated visual token is another candidate visual token whose first semantic similarity is not lower than a preset similarity threshold with the candidate visual token. The new sample visual token is assigned to the first token subsequence of the first level, and the i-th sample visual token and the merged candidate visual token are removed.

[0083] In some embodiments, the merging of sample visual tokens is achieved by calculating the mean of each sample visual token.

[0084] Suppose the sample query task is "whether there is any behavior of damaging public property with tools", the corresponding sample visual token set contains 10 sample visual tokens, such as Figure 7 As shown, assuming the first-level sample visual tokens are: Sample Visual Token 1, Sample Visual Token 8, Sample Visual Token 2, Sample Visual Token 4, Sample Visual Token 6, and Sample Visual Token 3, with a preset similarity threshold of 0.7, the first semantic similarities between Sample Visual Token 1 and Sample Visual Token 8, Sample Visual Token 2, Sample Visual Token 4, Sample Visual Token 6, and Sample Visual Token 3 are 0.85, 0.35, 0.25, 0.15, and 0.20, respectively. The first semantic similarity between Sample Visual Token 1 and Sample Visual Token 8 is greater than 0.7, and both can serve as factual evidence of "tool-holding" behavior. Therefore, Sample Visual Token 1 and Sample Visual Token 8 are merged into a new... At this point, the first level becomes: The sample visual tokens are 2, 4, 6, and 3. The first semantic similarity scores between sample visual token 2 and samples visual tokens 4, 6, and 3 are 0.75, 0.45, and 0.60, respectively. The first semantic similarity score between sample visual token 4 and sample visual token 2 is greater than 0.7, while the first semantic similarity score between sample visual token 4 and sample visual token 3 is 0.7. All of these can be used as factual evidence of "damaging public property." Therefore, sample visual tokens 2, 4, and 3 are merged into a new sample visual token. Since sample visual token 6 does not have a semantic similarity of at least 0.7, it remains unchanged and exists independently in the first level. Therefore, the compressed first token subsequence in the first level is: , Sample visual token 6.

[0085] In some embodiments, the compression strategy for the second-level sample visual tokens is as follows: calculate the sample token mean of N sample visual tokens, and use the sample token mean as the second token subsequence of the second level.

[0086] like Figure 7As shown, assuming the second-level sample visual tokens are: sample visual token 7 and sample visual token 9, sample visual token 7 and sample visual token 9 are merged into a new one. Finally, the second token subsequence after the second level of compression is: .

[0087] Finally, the token subsequences of the first and second levels are concatenated to obtain the sample compressed token sequence.

[0088] like Figure 7 As shown, the compressed token sequence of the sample after hierarchical compression is as follows: , Sample visual token 6 .

[0089] In the first level, the first K sample visual tokens carry the main visual information in the keyframes of the task of answering sample queries. To reduce the loss of key visual information, the first semantic similarity between sample visual tokens is calculated, and sample visual tokens that exceed a preset similarity threshold are merged, thereby retaining most of the key visual details with minimal distortion. The remaining first N sample visual tokens may contain auxiliary keyframes that provide contextual information for answering sample queries. To prevent key visual information from being misunderstood due to missing context, the first N sample visual tokens are merged, thereby supplementing contextual information with less computation and improving the accuracy of the review model.

[0090] To ensure the lightweight nature of the review model, a lightweight token selector is used in the decision network. During training, to improve the accuracy of sample association calculation, the token selector can be used as a student model, and a pre-trained teacher model with large parameters is introduced to provide a reference for the sample association calculation, allowing the student model to learn the judgment of the teacher model. The process by which the teacher model calculates the reference association between the sample visual token and the sample query task is consistent with the sample association calculation process and will not be elaborated here.

[0091] S2023: Calculate the selection evaluation value of the number of tokens based on the consistency between the correlation degree of each sample and the corresponding reference correlation degree.

[0092] During model training, for the decision network, a reward function was designed to balance the accuracy of correlation ranking and computational efficiency, based on sample correlation, reference correlation, and the number of tokens selected. The output of this reward function serves as the evaluation value for the number of tokens selected, thus avoiding the pursuit of accuracy at the expense of efficiency, and vice versa. Specifically, the formula for the reward function is as follows: Formula 1 in, Indicates the degree of correlation between samples. Indicates the degree of reference relevance. The Spearman rank correlation coefficient measures the consistency between sample correlation and reference correlation, where K is the number of tokens selected. This represents the maximum number of tokens within the specified range. This is a penalty for using too many computational resources (i.e., retaining too many sample visual tokens). Hyperparameters for controlling the severity of punishment.

[0093] S2024: Based on the selected evaluation value, obtain the token compression loss.

[0094] The optimization objective during model training is to maximize the reward based on the number of tokens. To achieve this objective, the REINFORCE algorithm can be used to calculate the token compression loss. Specifically, the loss function formula for token compression loss is as follows: Formula 2 in, Indicates token compression loss. Representation Decision Network The parameters, This indicates the range of token numbers, where K represents the number of tokens selected. This means that the evaluation value R is treated as a constant during backpropagation and does not participate in gradient calculation. In this way, positive rewards increase the probability of a token being selected, while negative rewards decrease the probability of a token being selected, thereby guiding the decision network to learn to select the optimal number of tokens.

[0095] During model training, based on the number of tokens selected according to the sample query task and sample attribute information, a penalty constraint term is constructed to ensure consistency between the sample visual tokens output by the student model and the sample relevance of the sample query task, as well as the reference relevance between the sample visual tokens output by the teacher model and the sample query task. In this way, the obtained token compression loss can effectively balance the accuracy of relevance ranking and computational efficiency, thereby achieving a balance between review accuracy and review efficiency.

[0096] S203: Based on the sample compression token sequence of the sample video, obtain the sample review conclusion of the sample video and the sample visual evidence supporting the sample review conclusion.

[0097] The sample compressed token sequence, after being compressed and merged at two levels, contains key visual information and concise contextual information. Therefore, the sample compressed token sequence can be input into the decoder of the auditing model for auditing.

[0098] To prevent the review model from making arbitrary "guessing" conclusions, an evidence tracing mechanism was designed. For each sample video, the review model not only outputs a sample review conclusion based on the sample compressed token sequence, but also outputs sample visual evidence supporting the sample review conclusion from both time and space dimensions, so as to ensure that its judgment is verifiable and thus realizes the transformation from "black box" to "visible".

[0099] In some embodiments, the decoding process of the sample compressed token sequence is as follows: First, based on the sample compressed token sequence, at least one sample timestamp interval associated with the sample query task in the sample video is obtained.

[0100] The sample visual tokens in the sample compressed token sequence are obtained by feature encoding of image patches contained in keyframes extracted from the sample video. Each sample visual token carries temporal information, and the merged sample visual tokens also retain the time range of the sample visual tokens involved in the merging. Therefore, based on the temporal information of the sample visual tokens, at least one sample timestamp interval associated with the sample query task in the sample video can be obtained.

[0101] by Figure 7 Taking the sample compressed token sequence shown as an example, "holding tools" mainly focuses on It is a combination of sample visual token 1 and sample visual token 8. Assuming the time information of sample visual token 1 is 14.5 seconds and the time information of sample visual token 8 is 14.8 seconds, then... The time range is from 14.5 to 14.8 seconds, and the timestamp interval after mapping to the sample video is 00:10:21-00:10:25; "damaging public property" mainly focuses on It is a combination of sample visual token 2, sample visual token 3, and sample visual token 4, assuming The time range is from 15.0 seconds to 15.5 seconds, and the timestamp range mapped to the sample video is 00:11:05-00:11:08.

[0102] Then, for each sample timestamp interval, based on the content description text of the sample video segment corresponding to the sample timestamp interval, and combined with the sample timestamp interval, a sample visual basis is obtained.

[0103] In some embodiments, the visual basis of the sample can be presented in the form of a list of [sample timestamp range, content description text].

[0104] like Figure 8 As shown, Key visual evidence, such as the tool itself and how it is held by hand, is used to describe crucial behaviors related to "handheld tools." Provide contextual information about the person, such as facial expressions and clothing textures, to infer the person corresponding to the "tool". Therefore, the content description text of the video segment corresponding to the sample timestamp interval 00:10:21-00:10:25 is "A person in the picture is holding a tool and walking towards the fitness equipment". Sliding motions, scratches, etc., provide evidence of damage and are used to describe acts of damage. Providing evidence of tool continuity can describe the state of "tool possession." The sample visual token 6 provides information about the fitness equipment and provides the results after damage. Therefore, the content description text of the video segment corresponding to the sample timestamp interval 00:11:05-00:11:08 is "The person is sliding a tool on the fitness equipment".

[0105] Finally, based on at least one visual criterion from the sample, a review conclusion is obtained for the sample video.

[0106] like Figure 8 As shown, based on the two visual evidences {[00:10:21-00:10:25, a person holding a tool walks towards the fitness equipment], [00:11:05-00:11:08, the person slides the tool on the fitness equipment]}, the final audit conclusion is "there is an act of damaging public property with a tool".

[0107] S204: Based on token compression loss, conclusion loss between sample review conclusion and corresponding conclusion label, and evidence loss between sample visual evidence and corresponding evidence label, perform model parameter tuning.

[0108] During training, the audit model not only outputs the audit conclusions of the samples, but also outputs the visual evidence supporting the audit conclusions. Therefore, the model quality can be more refined and reliablely quantitatively evaluated from both the conclusion and evidence levels. The optimal evaluation result can be used as the optimization goal to guide the model to continuously improve the accuracy of evidence location and information extraction, thereby improving the accuracy and reliability of the conclusions.

[0109] In some embodiments, the evidence loss is used to quantify the quality of the visual evidence provided by the review model, integrating the temporal overlap and second semantic similarity between the visual evidence and the corresponding evidence label. Specifically, the evidence loss is determined as follows: for each sample video, based on the temporal overlap between at least one sample timestamp interval in the visual evidence and at least one label timestamp interval in the corresponding evidence label, and the second semantic similarity between the visual evidence and the corresponding evidence label, a matching evaluation value between the visual evidence and the corresponding evidence label is obtained. Based on the obtained matching evaluation values, the evidence loss between the visual evidence and the corresponding evidence label is obtained. The precision and recall of all visual evidence can be calculated using the matching evaluation values. Precision quantifies the model's ability to output correct results, typically the ratio of the number of correctly reviewed videos to the total number of reviewed videos. Recall quantifies the model's ability to detect risks, typically the ratio of the number of risky videos identified to the total number of actual risky videos. Finally, the evidence loss is obtained based on the harmonic mean of precision and recall, expressed by the following formula: Formula 3 in, Indicates accuracy rate. Indicates recall rate, It represents the harmonic mean.

[0110] In some embodiments, the temporal overlap can be represented by the intersection-union ratio of two timestamp intervals, and the second semantic similarity can be represented by the cosine distance between the visual evidence and the evidence label of the sample.

[0111] Thus, the loss function of the decoder part of the audit model can be expressed as: Formula 4 in, and These are the weighting coefficients for the conclusion and the evidence, respectively. The conclusion loss is represented by the sample review conclusion and conclusion label. This indicates a loss of evidence.

[0112] Finally, the audit model is globally tuned by incorporating the loss function of the decision network.

[0113] During the training of the review model, a balance between review accuracy and efficiency is achieved by dynamically predicting the number of tokens. Furthermore, by forcing the output of verifiable visual evidence, the originally "black box" review process becomes transparent and traceable, ensuring that the model's reasoning is firmly rooted in visual facts, thus significantly enhancing the credibility of the review conclusions. In addition, through a loss function, the accuracy of the conclusions and the accuracy of the evidence (through label comparison) are used as reward signals for backpropagation, guiding the model to learn how to more accurately locate and extract evidence of risky content. This forms a closed loop of "review-verification-optimization," pointing the way to building a more powerful and credible AI review system.

[0114] like Figure 9 The diagram shown illustrates the network architecture of the review model provided in this application embodiment. It mainly comprises three parts: token number prediction, hierarchical token compression, and content review. The aim is to efficiently and intelligently extract key and concise visual information from lengthy and redundant video streams, based on specific tasks, and generate verifiable review conclusions. Specifically, in the token number prediction part, key frames are extracted from the sample video, and these key frames are segmented. Visual features of each image block are extracted to generate sample visual tokens. Furthermore, the number of tokens required for the sample query task is dynamically predicted using the sample query task set in the sample video and the sample attribute information of the sample video. Furthermore, the hierarchical token compression section extracts features for the sample query task set for the sample video. It employs a cross-modal cross-attention mechanism to calculate the sample correlation degree between each sample visual token and the sample query task. Combined with the predicted number of tokens K, the extracted sample visual tokens are divided into a first level and a second level. Then, for the K sample visual tokens in the first level, those with a first semantic similarity greater than a preset similarity threshold are merged, thus retaining most of the key visual information with minimal distortion. For the second level, the mean of the remaining N sample visual tokens is calculated, supplementing contextual information with minimal computation. Finally, the two levels of compressed token subsequences are concatenated to obtain the sample compressed token sequence, which is then input into the content moderation section. Based on the sample compressed token sequence, the content moderation section generates a sample moderation conclusion for the sample video and provides sample visual evidence supporting the conclusion. These three parts aim to maximize the reward for the number of tokens and the quality of generated evidence and conclusions, thereby addressing the dual problems of fragile model moderation capabilities and a lack of an evaluation system.

[0115] In scenarios such as video platforms and cloud services that require processing massive amounts of video, the trained review model can serve as the core engine of an automated review pipeline. Through the adaptive visual information compression strategy in the review model, it can quickly and cost-effectively compress videos that are tens of minutes or even hours long, effectively solving the computing resource bottleneck caused by full-volume analysis. At the same time, through the evidence tracing mechanism in the review model, when high-risk content (such as terrorist incidents, violent incidents, indecent content, etc.) is detected, it can not only automatically block the video, but also output the blocking evidence with accuracy down to the second. This provides conclusive and traceable evidence for the automated video handling (such as video removal and account banning) of video platforms and cloud services, greatly improving the accuracy and credibility of automated review.

[0116] like Figure 10 The diagram shows the video content review process based on a review model. Taking the server as the execution entity, it mainly includes the following steps: S1001: The server extracts the visual token set of the video to be reviewed.

[0117] The received video to be reviewed is frame-segmented, and each keyframe is divided into blocks to obtain multiple image blocks. Feature extraction is performed on each image block to generate a corresponding visual token. These visual tokens constitute the visual token set of the video to be reviewed. Each visual token is used to describe the visual information of an image block in a keyframe contained in the video to be reviewed.

[0118] In some embodiments, visual information includes, but is not limited to: the ID of the keyframe to which the image patch belongs, the position of the image in the keyframe, and the time information of the image patch.

[0119] The process of extracting the visual token set is the same as the process of extracting the visual tokens from the samples during training, and will not be described again here.

[0120] S1002: Based on the content query task set for the video to be reviewed and the attribute information of the video to be reviewed, the server compresses each visual token in the visual token set according to its correlation with the content query task, and obtains a compressed token sequence after splicing each level.

[0121] Among them, the content query task and attribute information are used to dynamically predict the number of tokens required for the content query task. The top K visual tokens with the highest relevance are used as the first level, and the remaining N visual tokens with the highest relevance are used as the second level, and token compression is performed on each level.

[0122] In some embodiments, the compression process of the visual token set is as follows: Figure 11 As shown, it mainly includes the following steps: S1002_1: Based on the content query task set for the video to be reviewed and the attribute information of the video to be reviewed, select a token number from the preset token number range.

[0123] The attribute information includes the total duration, total number of frames, and number of sampled frames of the video to be reviewed. The number of tokens required for the same content query task may vary for different videos, and the number of tokens required for different content query tasks of the same video may also vary for different content query tasks of different videos.

[0124] like Figure 12 As shown, for the content review task of "whether there is a traffic violation", the sub-task requires 5,000 tokens for video 1 to be reviewed and 7,200 tokens for video 2 to be reviewed. For video 2 to be reviewed, the content review task of "whether there is speeding" requires 4,000 tokens.

[0125] Therefore, to balance review efficiency and accuracy, the number of visual tokens required for the content query task of the video to be reviewed needs to be flexibly determined. The specific process is as follows: S1002_11: The text features of the content query task and the text features of the attribute information are fused to obtain fused text features.

[0126] S1002_12: Based on the fused text features, obtain the probability distribution of the token number range.

[0127] S1002_13: Select a number of tokens from the range of token numbers based on the probability distribution.

[0128] The process of selecting the number of tokens for the videos to be reviewed is the same as the process of selecting the number of tokens for the sample videos during training, and will not be repeated here.

[0129] S1002_2: Based on the text features of the content query task and each visual token, an attention mechanism is used to obtain the correlation between each visual token and the content query task.

[0130] By using a cross-modal attention mechanism, the content query task can be associated with the video to be reviewed. Specifically, the text features extracted from the content query task are divided into multiple dimensions, with each dimension serving as a query matrix. Each visual token is used as a key matrix, and a dot product operation is performed with the query matrix to obtain the attention weight between each key matrix and the query matrix of each dimension. The maximum value of the attention weight of each dimension is taken as the degree of association between the visual token and the content query task.

[0131] The calculation process for the correlation degree is the same as that for the sample correlation degree during training, and will not be repeated here.

[0132] S1002_3: Based on the selected number of tokens and the obtained correlation, each visual token is compressed hierarchically to obtain a compressed token sequence after splicing at each level.

[0133] Generally, the more visual tokens used, the higher the accuracy of the review, but the lower the efficiency. Furthermore, the greater the relevance, the greater the impact of visual tokens on content retrieval tasks. Therefore, the visual token set can be divided into two levels based on the number of tokens available, allowing for different compression methods. The specific process is as follows: S1002_31: Sort the correlations and obtain the sorting results.

[0134] S1002_32: Based on the sorting results, select the visual tokens corresponding to the K largest correlations as the first level, and select the visual tokens corresponding to the remaining N largest correlations as the second level.

[0135] Where K is the number of selected tokens, and N is greater than or equal to 1 but less than or equal to the difference between the total number of visual tokens and K. Since the first K visual tokens contain the key visual information required for the content query task, a low compression ratio can be used to ensure the accuracy of the review. The remaining first N visual tokens can provide context for the key visual information, and a high compression ratio can be used to improve the review speed. Thus, based on the compression strategy corresponding to each level, the visual tokens within the corresponding level are compressed to obtain the corresponding token subsequence.

[0136] S1002_33: Iterate through the first level in sequence and obtain the first semantic similarity between the i-th visual token and at least one candidate visual token.

[0137] Among them, the candidate visual token is: the visual token that is located after the i-th visual token and has not been merged.

[0138] S1002_34: For any candidate visual token whose first semantic similarity is not lower than a preset similarity threshold, obtain the associated visual token of the candidate visual token, merge the associated visual token, the candidate visual token and the i-th visual token to obtain a new visual token.

[0139] Among them, the associated visual token is another candidate visual token whose first semantic similarity with the candidate visual token is not lower than a preset similarity threshold.

[0140] S1002_35: Assign the new visual token to the first-level token subsequence and remove the i-th visual token and the merged candidate visual token.

[0141] S1002_36: Calculate the token mean of N visual tokens in the second level, and use the token mean as the token subsequence of the second level.

[0142] S1002_37: Concatenate the token subsequences of the first and second levels to obtain the compressed token sequence.

[0143] In balancing review efficiency and accuracy, the system abandons static token counts and single compression strategies, and implements a hierarchical compression method based on dynamic token counts. It can intelligently and dynamically allocate an appropriate number of tokens for each task according to the semantics of the content query task and the attribute information of the video to be reviewed. It selects a corresponding number of visual tokens that are highly semantically related to the content query task, performs low-loss fine-grained merging of these visual tokens, retains key visual details, and at the same time, filters out the remaining top visual tokens to provide contextual background. It then performs high-ratio summary compression on these redundant visual tokens, achieving a compression method of "retaining the key and summarizing the secondary". This significantly reduces the amount of computation while maximizing the preservation of the key visual information required for review.

[0144] S1003: The server obtains the review conclusion of the video to be reviewed and the visual evidence supporting the review conclusion based on the visual token sequence. The visual evidence includes: the content description text of the video segment corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed.

[0145] When conducting audits based on visual token sequences, an evidence-based audit paradigm was designed to address the black-box audit process. This paradigm forces the audit model to output audit conclusions and provides structured visual evidence that supports these conclusions in terms of time and space. This mechanism fundamentally eliminates random speculation by the audit model and forces all its judgments to be verifiable, thus achieving a transformation from a "black box" to a "white box".

[0146] like Figure 13 The diagram illustrates the "white-box" review process provided in this application embodiment. Assuming the content query task is "Who does Person A believe damaged public property?", the review model performs deep reasoning based on temporal and visual-spatial cues, identifying two key timestamps: 08:23 and 22:10. The video frame at timestamp 08:23 depicts a conversation between Person A and Person B, revealing that public property C is damaged. The video frame at timestamp 22:10 depicts Person A looking down at Person D, frowning. The review model combines spatiotemporal visual content for reasoning: <08:23> Public property C should be intact, therefore someone is suspicious; thus, visual evidence 1 indicates that public property C should be on the roadside. <22:10> Person D has just driven past the road where public property C is located; therefore, visual evidence 2 indicates that public property C was damaged after Person D passed by. Finally, the review model concludes that Person D is the person responsible for the damage.

[0147] In some embodiments, for complex, ambiguous, or appeal-reviewed videos that require human intervention, the review model can serve as a powerful auxiliary tool for reviewers to improve the efficiency of human review.

[0148] In practice, after obtaining visual evidence supporting the review conclusion, video segments corresponding to at least one timestamp interval contained in the visual evidence can be marked (e.g., highlighted) on the video to be reviewed. The process of content querying, finding visual evidence, and reasoning for the review conclusion is then organized to generate a structured review summary. This summary includes key visual evidence supporting risk assessment and a reasoning process, including specific timestamps, image descriptions, and logical explanations. Thus, when a marked segment is clicked, the review summary will be clearly displayed, such as... Figure 13 As shown, this allows auditors to quickly locate problems without having to watch the entire long video from beginning to end. This shifts the focus of their work from searching for risks like "finding a needle in a haystack" to efficiently reviewing risk points with "conclusive evidence," increasing the efficiency of manual review several times over.

[0149] In some embodiments, the adaptive visual information compression strategy and evidence tracing mechanism in the review model are highly transferable. In addition to risky content review, they can be extended to a wider range of multimodal content understanding tasks. By integrating different domain knowledge and review rules, they provide an efficient, reliable and interpretable basic technical solution for various complex long video analysis scenarios.

[0150] For example, in online live streaming and real-time stream review tasks: it is used for real-time analysis of live streams, dynamically capturing violations during the live stream and issuing immediate warnings; in copyright protection and content tracing tasks: it is used to quickly detect and locate the existence of stolen or infringing video clips in long videos; in video content retrieval and highlight generation tasks: it is used to quickly locate relevant video clips in long videos based on query content and draw the located video clips into video highlights.

[0151] In some embodiments, the open-source Qwen2.5-VL-7B model is used as the base model to construct an audit model (DynamicSelect-7B) with an adaptive visual information compression strategy and evidence tracing mechanism in this application embodiment. This model is compared with the Qwen2.5-VL-7B model and the static compression strategy model. The Qwen2.5-VL-7B model directly uses the original VLM to process long videos, adapting it to context length limitations through standard frame extraction. The static compression strategy model uses a fixed, non-adaptive token compression strategy to process long videos. The comparison results of the three models are shown in Table 1.

[0152] Table 1

[0153] As shown in Table 1, while the Qwen2.5-VL-7B model achieves a certain accuracy rate (78.0%), its evidence localization quality is only 25.0%. This indicates that its judgment largely relies on the model's "speculation" or "illusion" rather than solid visual facts, leading to unreliable review conclusions. In contrast, the review model, by forcibly locating and outputting visual evidence in both time and space dimensions, achieves an evidence localization quality of 79.0%, making the review conclusions verifiable and verifiable. This fundamentally solves the "black box" problem of AI review, transforming the review process from unreliable guesswork to transparent and credible evidence-driven decision-making, significantly improving the reliability and interpretability of the review. The static compression strategy model, as a "one-size-fits-all" optimization method, while slightly improving evidence localization quality (38.0%), may lose key visual information due to its crude compression method, resulting in a decrease in review accuracy (75.0%). In comparison, the review model, through an adaptive visual information compression strategy, not only avoids losing key visual information but also improves review accuracy to 87.7% by focusing more efficiently on core content. Ultimately, the overall performance of the audit model reached 85.1%, far exceeding that of the comparison model, achieving a comprehensive lead in audit efficiency, accuracy, and reliability.

[0154] Furthermore, in this embodiment of the application, four mainstream publicly available evaluation benchmarks were selected to conduct verification experiments on the performance of the review model. Among them, the Multimodal Model Evaluation Benchmark (MME) was used to evaluate the overall score of the model, the Multi-task Long Video Understanding Benchmark (MLVU) was used to evaluate the macro-average score (M-Avg) of the model, the Long Video Benchmark (LongVB) was used to evaluate the performance of the model on the validation set, and the Long Video Benchmark was used to evaluate the performance of the model on the test set. The comparative experimental results are shown in Table 2.

[0155] Table 2

[0156] As shown in Table 2, the performance of the review model is consistently and significantly better than models such as LLaVA-Video and InternVL2.5, and also surpasses the underlying model Qwen2.5-VL. Therefore, the review model in this application embodiment is not only effective on a specific dataset, but has strong versatility and universality, and can bring substantial performance improvement to various long video understanding tasks.

[0157] The review model provided in this application, through its unique adaptive visual information compression strategy and evidence tracing mechanism, not only solves the computational bottleneck of long video processing in terms of efficiency, but also achieves breakthroughs in accuracy and interpretability. It provides a solid technical foundation for building an efficient, reliable, and trustworthy AI content review system, possessing extremely high practical application value and promising prospects.

[0158] Based on the same inventive concept, this application provides a video content review device that can implement the above-mentioned video content review method and achieve the same technical effect.

[0159] See Figure 14 The auditing device includes: a token extraction module 1401, a layered compression module 1402, and a conclusion and evidence module 1403, wherein: The token extraction module 1401 is used to extract the set of visual tokens of the video to be reviewed; each visual token is used to describe the visual information of an image patch in a keyframe contained in the video to be reviewed. The layered compression module 1402 is used to compress each visual token in layers based on the content query task set for the video to be reviewed and the attribute information of the video to be reviewed, combined with the correlation between each visual token in the visual token set and the content query task, to obtain a compressed token sequence after splicing the layers. The conclusion and evidence module 1403 is used to obtain the review conclusion of the video to be reviewed and the visual evidence supporting the review conclusion based on the visual token sequence; the visual evidence includes: the content description text of the video segment corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed.

[0160] Optionally, the layered compression module 1402 is specifically used for: Based on the content query task set for the video to be reviewed, and the attribute information of the video to be reviewed, select a token number from the preset token number range; Based on the text features and visual tokens of the content query task, an attention mechanism is used to obtain the correlation between each visual token and the content query task. Based on the selected number of tokens and the obtained correlation, each visual token is compressed hierarchically to obtain a compressed token sequence after splicing at each level.

[0161] Optionally, the layered compression module 1402 is specifically used for: Sort the correlation scores and obtain the ranking results; Based on the ranking results, the visual tokens corresponding to the K highest relevance scores are selected as the first level, and the visual tokens corresponding to the remaining N highest relevance scores are selected as the second level; K is the number of selected tokens, and N is greater than or equal to 1 but less than or equal to the difference between the total number of visual tokens and K. Based on the compression strategy corresponding to each level, the visual tokens in the corresponding level are compressed to obtain the corresponding token subsequence; The token subsequences of the first and second levels are concatenated to obtain the compressed token sequence.

[0162] Optionally, the layered compression module 1402 is specifically used for: Calculate the token mean of N visual tokens in the second level, and use the token mean as the token subsequence of the second level; Traverse the first level sequentially, performing the following operations on each traversal: Obtain the first semantic similarity between the i-th visual token and at least one candidate visual token; wherein, the candidate visual token is: the visual token located after the i-th visual token and not merged; For any candidate visual token whose first semantic similarity is not lower than a preset similarity threshold, obtain the associated visual token of the candidate visual token, and merge the associated visual token, the candidate visual token and the i-th visual token to obtain a new visual token; wherein, the associated visual token is another candidate visual token whose first semantic similarity with the candidate visual token is not lower than a preset similarity threshold. The new visual token is assigned to the first-level token subsequence, and the i-th visual token and the merged candidate visual tokens are removed.

[0163] Optionally, the Conclusions and Evidence module 1403 is specifically used for: Based on the visual token sequence, obtain at least one timestamp interval associated with the content query task in the video to be reviewed; For each timestamp interval, based on at least one visual token corresponding to the timestamp interval, obtain the content description text of the video segment corresponding to the timestamp interval, and combine it with the timestamp interval to generate a visual basis; Based on at least one visual criterion, a review conclusion is obtained for the video to be reviewed.

[0164] Optionally, the review conclusion and the visual evidence supporting the review conclusion are obtained using the review model. The device is used to iteratively train the video review model based on the sample video set. Each sample video has a conclusion label and an evidence label. The evidence label includes: content description text of the video segment corresponding to the video frame contained in at least one timestamp interval associated with the conclusion label in the sample video; wherein, in each round of training: The token extraction module 1401 is also used to extract the sample visual token set of the selected sample video; The hierarchical compression module 1402 is also used to compress each sample visual token hierarchically based on the sample query task set for the sample video and the sample attribute information of the sample video, combined with the sample correlation degree between each sample visual token contained in the sample visual token set and the sample query task, to obtain the sample compressed token sequence and token compression loss after each level is spliced. The Evidence and Strategy module 1403 is also used to obtain the sample review conclusion and the sample visual evidence supporting the sample review conclusion based on the sample video compression token sequence. The adjustment module 1404 is used to perform model parameter tuning based on token compression loss, conclusion loss between sample review conclusion and corresponding conclusion label, and evidence loss between sample visual evidence and corresponding evidence label.

[0165] Optionally, the layered compression module 1402 is specifically used for: Based on the sample query task set for the sample video and the sample attribute information of the sample video, select a token number from the preset token number range. Based on the selected number of tokens and the sample relevance of each visual token to the sample query task, the sample visual token set is compressed hierarchically. Then, based on the consistency between the sample relevance and the corresponding reference relevance, the selection evaluation value of the number of tokens is calculated. The reference relevance is the relevance between the sample visual tokens obtained by using the trained teacher model and the sample query task. Based on the selected evaluation value, the token compression loss is obtained.

[0166] Optionally, the layered compression module 1402 is specifically used for: The text features of the sample query task and the text features of the sample attribute information are fused to obtain fused text features; Based on the fused text features, the probability distribution of the token number range is obtained; Select a number of tokens from a range of token numbers based on a probability distribution.

[0167] Optionally, the adjustment module 1404 is specifically used for

[0168] For each sample video, the following steps are performed: the time overlap between at least one timestamp interval in the sample visual evidence and at least one tag timestamp interval in the corresponding evidence tag, and the second semantic similarity between the sample visual evidence and the corresponding evidence tag, to obtain the matching evaluation value between the sample visual evidence and the corresponding evidence tag. Based on the obtained matching evaluation values, the evidence loss between the visual evidence of the sample and the corresponding evidence label is obtained.

[0169] Optionally, the adjustment module 1404 is specifically used for: Based on each matching evaluation value, the precision and recall of the sample video set are obtained respectively; whereby, the precision is used to quantify the model's ability to output correct results, and the recall is used to quantify the model's ability to detect risks. The loss of evidence is obtained based on the harmonic mean of precision and recall.

[0170] The aforementioned review device divides the keyframes in the video to be reviewed into blocks, using the visual information of each image block as a visual token. It calculates the correlation between each visual token and the set content query task, and combines this with the attribute information of the video to be reviewed. The device then performs hierarchical compression processing on each visual token, significantly reducing computational overhead without sacrificing key information, effectively balancing review efficiency and accuracy. Furthermore, the compressed token sequence, spliced ​​together at each level, sequentially links visual tokens related to the content query task in the video to be reviewed, achieving a "progressive" and "contextual" approach to complex reviews, further improving review accuracy. On the other hand, for the video to be reviewed, it not only provides a review conclusion but also offers visual evidence supporting that conclusion from both temporal and spatial dimensions. This visual evidence is presented as descriptive text of the video segments corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed, transforming the black-box review process into visible visual facts, thereby increasing the credibility and reliability of the review conclusion.

[0171] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0172] Regarding the methods and apparatus in the above embodiments, the specific manner in which each unit executes the request has been described in detail in the embodiments of the method, and will not be elaborated here.

[0173] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0174] Based on the same inventive concept, embodiments of this application also provide an electronic device. In one embodiment, the electronic device may be a server.

[0175] See Figure 15 As shown, it is a schematic diagram of the structure of a possible electronic device provided in an embodiment of this application. Figure 15 The electronic device includes: a memory 1501, a communication module 1503, and one or more processors 1502.

[0176] The memory 1501 is used to store computer programs executed by the processor 1502. The memory 1501 may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system and the operating instruction set, etc.

[0177] Memory 1501 may be volatile memory, such as random-access memory (RAM); memory 1501 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1501 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1501 may be a combination of the above-described memories.

[0178] Processor 1502 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 1502 is used to implement the steps of the above-described auditing method and training method when calling the computer program stored in memory 1501.

[0179] The communication module 1503 is used to communicate with servers and terminal devices.

[0180] This application embodiment does not limit the specific connection medium between the memory 1501, communication module 1503, and processor 1502. This application embodiment... Figure 15 The memory 1501 and the processor 1502 are connected via a bus 1504, and the bus 1504 is in Figure 15 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 1504 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 15 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0181] The memory 1501 stores a computer storage medium containing computer-executable instructions for implementing the sample generation method, model training method, and dialogue generation method of the embodiments of this application. The processor 1502 is used to execute the steps of the above-described review method and training method.

[0182] In one embodiment, the electronic device may also be a terminal device.

[0183] See Figure 16 As shown, it is a schematic diagram of the structure of a possible terminal device provided in an embodiment of this application. Figure 16 The terminal equipment includes components such as a communication component 1610, a memory 1620, a display unit 1630, a camera 1640, a sensor 1650, an audio circuit 1660, a Bluetooth module 1670, and a processor 1680.

[0184] The communication component 1610 is used to communicate with the server. In some embodiments, it may include a Wireless Fidelity (WiFi) module, which is a short-range wireless transmission technology, and the electronic device can send and receive information through the WiFi module.

[0185] The memory 1620 can be used to store software programs and data. The processor 1680 executes various functions of the terminal device and performs data processing by running the software programs or data stored in the memory 1620. The memory 1620 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 1620 stores an operating system that enables the terminal device to run. In this application, the memory 1620 may store the operating system and various application programs, and may also store computer programs that execute the review methods of the embodiments of this application.

[0186] The display unit 1630 can also be used to display information input by the object or information provided to the object, as well as a graphical object interface for various menus. Specifically, the display unit 1630 may include a display screen 1632. The display screen 1632 may be configured as a liquid crystal display, a light-emitting diode, or the like. The display unit 1630 can be used to display the review interface in the embodiments of this application.

[0187] The display unit 1630 can also be used to receive input digital or character information and generate signal inputs related to object settings and function control of the terminal device. Specifically, the display unit 1630 may include a touch screen 1631 disposed on the terminal device, which can collect touch operations on or near the object, such as clicking a button, dragging a scroll bar, etc.

[0188] The touchscreen 1631 can be placed on top of the display screen 1632, or the touchscreen 1631 and the display screen 1632 can be integrated to realize the input and output functions of the terminal device. After integration, it can be referred to as a touch display screen. In this application, the display unit 1630 can display the application and the corresponding operation steps.

[0189] Camera 1640 can be used to capture images. There can be one or multiple cameras 1640. An object is projected onto a photosensitive element through a lens, generating an optical image. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to a processor 1680 to be converted into a digital image signal.

[0190] The terminal device may also include at least one sensor 1650, such as an accelerometer 1651, a proximity sensor 1652, a fingerprint sensor 1653, and a temperature sensor 1654. The terminal device may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, and motion sensor.

[0191] Audio circuitry 1660, speaker 1661, and microphone 1662 provide an audio interface between the device and the terminal equipment. Audio circuitry 1660 converts received audio data into electrical signals, which are then transmitted to speaker 1661, where they are converted into sound signals for output. The terminal equipment may also be equipped with volume buttons for adjusting the volume of the sound signal. Conversely, microphone 1662 converts collected sound signals into electrical signals, which are then received by audio circuitry 1660, converted back into audio data, and output to communication component 1610 or to memory 1620 for further processing.

[0192] The Bluetooth module 1670 is used to interact with other Bluetooth devices that also have a Bluetooth module via the Bluetooth protocol. For example, a terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smartwatch) that also has a Bluetooth module through the Bluetooth module 1670, thereby exchanging data.

[0193] The processor 1680 is the control center of the terminal device, connecting various parts of the terminal through various interfaces and lines. It executes various functions and processes data by running or executing software programs stored in the memory 1620 and calling data stored in the memory 1620. In some embodiments, the processor 1680 may include one or more processing units; the processor 1680 may also integrate an application processor and a baseband processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the baseband processor mainly handles wireless communication. It is understood that the baseband processor may not be integrated into the processor 1680. In this application, the processor 1680 can run the operating system, applications, object interface display and touch response, as well as the review method of this application embodiment. Furthermore, the processor 1680 is coupled to the display unit 1630.

[0194] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium including a computer program. When the computer program is run on an electronic device, the computer program is used to cause the electronic device to perform the steps of the above-described auditing method and training method.

[0195] In some possible implementations, aspects of the strategy model training method and video conference generation method provided in this application can also be implemented as a program product, comprising a computer program. When the program product is run on an electronic device, the computer program causes the electronic device to perform the steps of the aforementioned auditing and training methods. For example, the electronic device can perform actions such as... Figure 2 and Figure 10 The steps are shown in the figure.

[0196] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0197] The program product of the embodiments of this application may be a CD-ROM and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a computer program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0198] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a computer program for use by or in conjunction with a command execution system, apparatus, or device.

[0199] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0200] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for reviewing video content, characterized in that, include: Extract the set of visual tokens from the video to be reviewed; each visual token is used to describe: the visual information of an image patch in a keyframe contained in the video to be reviewed; Based on the content query task set for the video to be reviewed, and the attribute information of the video to be reviewed, and combined with the correlation between each visual token in the visual token set and the content query task, the visual tokens are compressed in layers to obtain a compressed token sequence after splicing at each layer. Based on the visual token sequence, the review conclusion of the video to be reviewed and the visual evidence supporting the review conclusion are obtained; the visual evidence includes: the content description text of the video segment corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed.

2. The method as described in claim 1, characterized in that, Based on the content query task set for the video to be reviewed, and the attribute information of the video to be reviewed, and combined with the correlation between each visual token in the visual token set and the content query task, the visual tokens are compressed hierarchically to obtain a compressed token sequence concatenated at each level, including: Based on the content query task set for the video to be reviewed and the attribute information of the video to be reviewed, select a token number from a preset token number range; Based on the text features and visual tokens of the content query task, an attention mechanism is used to obtain the correlation degree between each visual token and the content query task. Based on the selected number of tokens and the obtained correlation, the visual tokens are compressed hierarchically to obtain a compressed token sequence after splicing at each level.

3. The method as described in claim 2, characterized in that, Based on the selected number of tokens and the obtained correlation degrees, the visual tokens are compressed hierarchically to obtain a compressed token sequence concatenated at each level; including: The correlation degrees are sorted to obtain the sorting results; Based on the sorting results, the visual tokens corresponding to the K highest relevance are selected as the first level, and the visual tokens corresponding to the remaining N highest relevance are selected as the second level; K is the number of selected tokens, and N is greater than or equal to 1 but less than or equal to the difference between the total number of visual tokens and K. Based on the compression strategy corresponding to each level, the visual tokens in the corresponding level are compressed to obtain the corresponding token subsequence; The compressed token sequence is obtained by concatenating the token subsequences of the first level and the second level.

4. The method as described in claim 3, characterized in that, The compression strategy corresponding to each level is used to compress the visual tokens within the corresponding level to obtain the corresponding token subsequences, including: Calculate the token mean of N visual tokens in the second level, and use the token mean as the token subsequence of the second level; Iterate through the first level sequentially, performing the following operations on each iteration: Obtain the first semantic similarity between the i-th visual token and at least one candidate visual token; wherein, the candidate visual token is: a visual token located after the i-th visual token that has not been merged; For any candidate visual token whose first semantic similarity is not lower than a preset similarity threshold, obtain the associated visual token of the candidate visual token, and merge the associated visual token, the candidate visual token, and the i-th visual token to obtain a new visual token; wherein, the associated visual token is another candidate visual token whose first semantic similarity is not lower than the preset similarity threshold with the candidate visual token. The new visual token is assigned to the token subsequence of the first level, and the i-th visual token and the merged candidate visual token are removed.

5. The method as described in claim 1, characterized in that, The process of obtaining the review conclusion of the video to be reviewed and the visual evidence supporting the review conclusion based on the visual token sequence includes: Based on the visual token sequence, obtain at least one timestamp interval associated with the content query task in the video to be reviewed; For each timestamp interval, based on at least one visual token corresponding to the timestamp interval, the content description text of the video segment corresponding to the timestamp interval is obtained, and combined with the timestamp interval, a visual basis is generated. The review conclusion of the video to be reviewed is obtained based on at least one visual criterion.

6. The method according to any one of claims 1-5, characterized in that, The audit conclusions and the visual evidence supporting them are obtained using an audit model, which is obtained in the following way: Based on a sample video set, the video review model is iteratively trained. Each sample video has a conclusion label and an evidence label. The evidence label includes: content description text of the video segment corresponding to the video frame contained in at least one timestamp interval associated with the conclusion label in the sample video. Each training round includes: Extract the sample visual token set from the selected sample videos; Based on the sample query task set for the sample video and the sample attribute information of the sample video, and combined with the sample correlation degree between each sample visual token contained in the sample visual token set and the sample query task, the sample visual tokens are compressed in layers to obtain the sample compressed token sequence and token compression loss after splicing at each layer. Based on the sample compression token sequence of the sample video, obtain the sample review conclusion of the sample video and the sample visual evidence supporting the sample review conclusion; Based on the token compression loss, the conclusion loss between the sample review conclusion and the corresponding conclusion label, and the evidence loss between the sample visual evidence and the corresponding evidence label, the model parameters are tuned.

7. The method as described in claim 6, characterized in that, The method, based on the sample query task set for the sample video and the sample attribute information of the sample video, and combining the sample visual tokens contained in the sample visual token set with the sample correlation degree of each sample visual token with the sample query task, performs hierarchical compression processing on each sample visual token to obtain token compression loss, including: Based on the sample query task set for the sample video and the sample attribute information of the sample video, select a number of tokens from a preset range of token numbers. Based on the selected number of tokens and the sample relevance of each sample visual token to the sample query task, the sample visual token set is compressed hierarchically. Then, based on the consistency between each sample relevance and the corresponding reference relevance, the selection evaluation value of the number of tokens is calculated. The reference relevance is the relevance between the sample visual token and the sample query task obtained using a trained teacher model. Based on the selected evaluation value, the token compression loss is obtained.

8. The method as described in claim 7, characterized in that, The step of selecting a token count from a preset token count range based on the sample query task set for the sample video and the sample attribute information of the sample video includes: The text features of the sample query task and the text features of the sample attribute information are fused to obtain fused text features; Based on the fused text features, the probability distribution of the token number range is obtained; Based on the probability distribution, select a number of tokens from the range of token numbers.

9. The method as described in claim 6, characterized in that, The loss of evidence was obtained in the following manner: For each sample video, the following steps are performed: based on the time overlap between at least one timestamp interval in the sample visual evidence and at least one tag timestamp interval in the corresponding evidence tag, and the second semantic similarity between the sample visual evidence and the corresponding evidence tag, to obtain a matching evaluation value between the sample visual evidence and the corresponding evidence tag. Based on the obtained matching evaluation values, the evidence loss between the visual evidence of the sample and the corresponding evidence label is obtained.

10. The method as described in claim 9, characterized in that, The process of obtaining the evidence loss between the visual evidence of the sample and the corresponding evidence label based on the obtained matching evaluation values ​​includes: Based on the matching evaluation values, the precision and recall of the sample video set are obtained respectively; wherein, the precision is used to quantify the model's ability to output correct results, and the recall is used to quantify the model's ability to detect risks; The evidence loss is obtained based on the harmonic mean of the precision and the recall.

11. A video content review device, characterized in that, include: The token extraction module is used to extract the set of visual tokens from the video to be reviewed; each visual token is used to describe the visual information of an image block in a keyframe contained in the video to be reviewed. The layered compression module is used to compress each visual token in layers based on the content query task set for the video to be reviewed, the attribute information of the video to be reviewed, and the correlation between each visual token in the visual token set and the content query task, so as to obtain a compressed token sequence after splicing the layers. The conclusion and evidence module is used to obtain the review conclusion of the video to be reviewed and the visual evidence supporting the review conclusion based on the visual token sequence; the visual evidence includes: the content description text of the video segments corresponding to at least one timestamp interval associated with the content query task in the video to be reviewed.

12. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of any of the methods described in claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of any of the methods described in claims 1 to 10.

14. A computer program product, characterized in that, It includes a computer program stored in a computer-readable storage medium, and a processor of an electronic device reads from and executes the computer program, causing the electronic device to perform the steps of any of the methods described in claims 1 to 10.

Images