Video review method, device and storage medium

Abstract

The application provides a video auditing method and device and a storage medium, wherein the method comprises the following steps: playing a to-be-audited video in a playback link simulation environment, obtaining basic sampling frames and a plurality of context abstracts containing code stream structure, transcoding, playback quality and the like; generating a sampling strategy according to the context abstracts to obtain target sampling frames, and extracting multi-dimensional video features; correcting the features by using the context abstracts to obtain target video features, and finally generating an auditing result in combination with video information. The application effectively solves the misjudgment problem caused by the fact that the traditional auditing ignores the processing traces such as transcoding and superposition, and improves the accuracy and robustness of video auditing by simulating a real playback environment and introducing a context feature correction mechanism.

Description

Technical Field

[0001] This application relates to the field of video review technology, and more specifically, to a video review method, device, and storage medium. Background Technology

[0002] With the rapid development of generative artificial intelligence technology, AI-generated and deep-synthesized videos are increasingly appearing on over-the-top (OTT) television platforms. This not only brings risks to content authenticity and copyright, but also places higher demands on the content security review of these platforms. OTT content undergoes complex processing such as transcoding, encapsulation, and overlaying of subtitles or badges before being uploaded, resulting in a final presentation that differs from the original file. Therefore, the review system needs a method to accurately assess the AI ​​risks in the final playback effect.

[0003] Currently, detection technologies for AI-generated videos mainly focus on the uploading or storage stage, typically analyzing the original video files. Examples include deepfake detection based on facial regions, general forgery detection analyzing image frequency or spatial features, and content tracing schemes relying on digital watermarks or metadata. These technologies mostly detect and output classification results or risk scores through detection models, primarily identifying forgery traces within the content itself.

[0004] However, existing technologies are clearly disconnected from OTT content moderation practices. First, current solutions ignore the interference of playback processing steps such as transcoding, compression, and image overlay on detection features, easily leading to misjudgments. Second, the detection dimensions are limited, relying excessively on facial analysis and failing to cover AI-generated content, such as virtual scenes, which are not dominated by human faces. Finally, existing solutions only output simple risk scores, lacking a clear chain of evidence and specific handling recommendations that moderators can understand, thus failing to effectively support the complex OTT content moderation decision-making process. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a video review method, device, and storage medium to solve the problem that the existing video review methods are disconnected from OTT services and cannot support OTT review decisions.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows: Firstly, this application provides a video review method, the method comprising: In the playback link simulation environment, the video to be reviewed is played to obtain basic sampling frames, and multiple context summaries of the video to be reviewed are collected based on the basic sampling frames. A sampling strategy is generated based on the multiple context summaries, and the video to be reviewed is sampled based on the sampling strategy to obtain multiple target sampling frames; Feature extraction is performed on each of the target sampling frames to obtain multi-dimensional video features corresponding to each target sampling frame; Based on the multiple context summaries, the multidimensional video features corresponding to each target sampling frame are corrected to obtain the target video features corresponding to each target sampling frame; The risk assessment of the video to be reviewed is performed based on the target video features corresponding to each target sampling frame, and the risk assessment result of the video to be reviewed is obtained. The review result of the video to be reviewed is then generated based on the risk assessment result and the video information of the video to be reviewed.

[0007] Optionally, the step of obtaining a basic sampling frame and collecting multiple contextual summaries of the video to be reviewed based on the basic sampling frame includes: During the playback of the video to be reviewed, multiple video frames are collected according to a preset period, and each of the collected video frames is used as the basic sampling frame. During the playback of the video to be reviewed, the metadata and status information generated in the playback link are monitored and recorded. Information is extracted from the metadata and status information, and information is extracted from each of the basic sampling frames to obtain the context summary. The context summary includes: bitstream structure summary, transcoding summary, playback quality summary, image overlay element summary, display processing summary, and abnormal playback status summary.

[0008] Optionally, generating a sampling strategy based on the plurality of context summaries includes: Each context summary is searched in a preset policy library to obtain at least one target summary in the policy library that matches the context summary. Based on the sampling strategy corresponding to each target summary, a sampling strategy corresponding to the context summary is generated.

[0009] Optionally, the step of extracting features from each of the target sampling frames to obtain multi-dimensional video features corresponding to each target sampling frame includes: The target sampled frame is input into the feature extraction model, and the feature extraction model extracts multiple initial video features; Determine the risk level of the video to be reviewed; Based on the risk level, the multidimensional video features corresponding to the target sampling frame are determined from the plurality of initial video features.

[0010] Optionally, the step of correcting the multidimensional video features corresponding to each of the target sampling frames based on the plurality of context summaries to obtain the target video features corresponding to each of the target sampling frames includes: Based on the multiple context summaries, a correction strategy is determined, which includes correction strategies for each dimension; Determine the dimensions to which the multidimensional video features corresponding to each of the target sampling frames belong; According to the correction strategy for each dimension, the video features of each dimension in the multidimensional video features corresponding to each target sampling frame are corrected.

[0011] Optionally, the step of performing risk assessment on the video to be reviewed based on the target video features corresponding to each of the target sampling frames to obtain the risk assessment result of the video to be reviewed includes: The target video features corresponding to each target sampling frame are input into the risk assessment model. The risk assessment model searches for the corresponding target comparison video frame in the video database based on the target video features corresponding to each target sampling frame, and compares each target sampling frame with the target comparison video frame to obtain the initial risk score of each target sampling frame and the confidence level of the initial risk score. If the initial risk score is less than a preset risk threshold and the confidence level is greater than a preset confidence threshold, then a risk determination result is generated based on the initial risk score of each target sampling frame. If the initial risk score is greater than the preset risk threshold and the confidence level is less than the preset confidence level threshold, then the risk assessment result of the video to be reviewed is obtained based on the characteristics of each target video.

[0012] Optionally, the step of reviewing the video based on the features of each target video to obtain the risk assessment result of the video to be reviewed includes: Determine at least one review branch for the video to be reviewed, the review branch including: timing consistency branch, compression branch, frequency domain statistics branch, display processing traces branch, and audio consistency branch; Input the target video features corresponding to each target sampling frame into the risk judgment model corresponding to each review branch to obtain the review judgment result corresponding to each review branch. Based on the review results of each review branch, a risk assessment result for the video to be reviewed is generated.

[0013] Optionally, generating the review result of the video to be reviewed based on the risk assessment result and the video information of the video to be reviewed includes: The risk assessment result and the video information are matched in a preset mapping table to obtain the review suggestions corresponding to the risk assessment result and the video information in the mapping table; The audit results are obtained by combining the various audit suggestions.

[0014] Secondly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of a video review method as described in any one of the first aspects.

[0015] Thirdly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of a video review method as described in any one of the first aspects.

[0016] The beneficial effects of this application are as follows: By constructing a playback link simulation environment, the presentation state of the video on a real terminal can be restored, solving the problem caused by traditional review methods that only target the original file and ignore the image features after transcoding, overlay, and other processing. By collecting context summaries and generating dynamic sampling strategies accordingly, accurate capture of key content is achieved, avoiding the waste of computing power caused by invalid sampling. Using context summaries to correct multi-dimensional video features effectively eliminates interference noise introduced by the playback environment, making the extracted target video features more reflective of the true nature of the video content, thereby significantly reducing the false judgment rate. Finally, by combining video information to generate review results, the accuracy, robustness, and processing efficiency of video review are improved.

[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart of a video review method provided in an embodiment of this application is shown; Figure 2 This document illustrates a flowchart of an embodiment of the present application for obtaining a basic sampling frame and a context summary. Figure 3 A flowchart of a sampling generation strategy provided in an embodiment of this application is shown; Figure 4 This document illustrates a flowchart of an embodiment of obtaining multidimensional video features. Figure 5 This document illustrates a flowchart of an embodiment of obtaining target video features. Figure 6 This application provides a flowchart for obtaining a risk assessment result according to an embodiment of the present application; Figure 7 This invention provides a flowchart illustrating yet another method for obtaining a risk assessment result, as illustrated in an embodiment of this application. Figure 8 This application provides a flowchart for generating audit results according to an embodiment of the present application; Figure 9 This paper shows a schematic diagram of the structure of a video review device provided in an embodiment of this application; Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0022] With the increasing number of AI-generated videos, how to review and assess the risks of these videos has become a problem that needs to be solved. For example, it is necessary to determine whether there are traces of AI generation or forgery in the video, which requires reviewing the video after it has been added to the database.

[0023] Existing technologies typically involve analyzing and detecting the original video file, such as detecting faces in the original video file, and using models to detect whether the face is generated by AI or has been forged, ultimately generating an identifier for whether it is AI-generated and a risk score.

[0024] However, this approach ignores the impact of transcoding, encapsulation, image overlay, and display processing in OTT services. These processes may alter video characteristics, causing general detection models to misjudge normal playback processing traces as AI-generated traces, resulting in a large number of false alarms.

[0025] Secondly, existing technologies rely on face detection and cannot effectively identify AI-generated content that is not dominated by human faces, such as virtual scenes, partially redrawn images, or images with abnormal overall styles, making it difficult to cover the rich content types of movies, variety shows, etc. on OTT platforms.

[0026] Furthermore, existing technologies mostly output simple AI-generated labels and risk scores, lacking action suggestions that auditors can understand. This makes it difficult for auditors to quickly determine the source of risk, increasing the cost and difficulty of manual review.

[0027] Furthermore, performing highly complex deep testing on all incoming content would result in significant computational costs and review delays, making it unsuitable for scenarios involving the rapid ingestion and release of large-scale content.

[0028] Based on this, this application proposes a video review method. The method involves playing the video to be reviewed in a playback link simulation environment to obtain basic sampling frames and multiple context summaries containing information such as bitstream structure, transcoding, and playback quality. A sampling strategy is generated based on the context summaries to obtain target sampling frames, and multi-dimensional video features are extracted. The features are then corrected using the context summaries to obtain target video features, and finally, the review result is generated by combining the video information. This application, by simulating a real playback environment and introducing a context feature correction mechanism, effectively solves the problem of misjudgment caused by traditional review methods ignoring transcoding and overlay processing traces, significantly improving the accuracy and robustness of video review.

[0029] Figure 1 The diagram illustrates a flowchart of a video review method provided in an embodiment of this application. The execution subject of this method can be a server or an electronic device, such as... Figure 1 As shown, the method includes: S101. Play the video to be reviewed in the playback link simulation environment, obtain the basic sampling frames, and collect multiple context summaries of the video to be reviewed based on the basic sampling frames.

[0030] A playback link simulation environment refers to a software or hardware system that simulates the playback behavior of real terminals (such as smart TVs, set-top boxes, and mobile apps). A playback link simulation environment can not only decode video files, but also simulate the entire process from receiving network data streams, decapsulation, decoding, post-processing (such as super-resolution and noise reduction), to final rendering and display.

[0031] In one possible implementation, the basic sampling frame can be obtained through the review player, playback link simulation environment, background playback task, decoding link or general sampling interface, and multiple context summaries of the video to be reviewed can be collected based on the basic sampling frame.

[0032] When videos awaiting review are added to the database, they contain basic video information, including content identifier, media asset version, transcoded version, review task identifier, content source, target distribution channel, target operational position, content type, source tracing status, and current strategy version. The source tracing status is used to assess the compliance and authenticity of the AI-generated content's identifiers. The source tracing status in the video information can be empty or have a corresponding value. The values ​​for the source tracing status include: existence of a trustworthy AI identifier, existence but unverifiable, missing identifier, abnormal identifier, undeclared, externally supplied video declared, and listed on the platform's known whitelist.

[0033] Content identifiers are used to track the review, distribution, and audit status of videos throughout the entire process. Media asset version refers to the version record of content assets throughout their lifecycle. The platform uniformly manages video source files, transcoded versions, tags, and lifecycles to ensure that review and distribution are based on the correct material version. Transcoded version refers to the specific version of the video after transcoding. Review task identifier is a unique number assigned to a specific review task. After a video to be reviewed is added to the database, a review task can be created for that video, and a review task identifier will be generated. Current strategy version refers to the version of the platform's preset review rule library. Content source refers to the information of the video material provider or creator. Target distribution channel refers to the platform or business line where the video is planned to be distributed. Target operational position refers to the specific display position where the video is planned to go live, such as a homepage recommendation position, a regular channel page, etc. Content type refers to the subject matter or category to which the video belongs, such as news, medical, finance, etc.

[0034] A base sampling frame refers to an original image frame captured during video playback according to a certain base frequency or rule. For example, during simulated playback, a frame can be captured as a sampling frame at fixed intervals, or keyframes in the video stream can be directly extracted as sampling frames.

[0035] Context summaries refer to non-image pixel information related to the video playback process, or environmental feature information derived from image analysis. Context summaries include: stream structure summary, transcoding / packaging summary, playback quality summary, overlay element summary, display processing summary, and abnormal playback status summary.

[0036] The bitstream structure summary describes the parameters of the underlying encoded data structure of the video. The transcoding / packaging summary describes the format conversion information the video undergoes before transmission and storage. The playback quality summary reflects the smoothness and clarity of the video during actual playback. The image overlay element summary describes additional visual layer information added to the original video image, including overlaid subtitles, station logos, watermarks, and pre-roll advertisements. The display processing summary records the image enhancement or conversion processing the video undergoes before rendering to the screen. The abnormal playback state summary records abnormal states that occur within the player itself during sampling.

[0037] The context summary includes a summary generated based on the basic video information and a summary generated based on the basic sampled frames. The context summary can be collected simultaneously while the basic sampled frames are being collected. For example, when the basic sampled frame at the 10th second is captured, the playback quality summary and the abnormal playback status summary at that moment can be collected simultaneously. The basic sampled frames and the context summary are bound together in time. The basic sampled frames provide the visual pixel information of the video, while the context summary provides the non-visual environmental information of the video.

[0038] S102. Generate a sampling strategy based on multiple context summaries, and perform sampling processing on the video to be reviewed based on the sampling strategy to obtain multiple target sampling frames.

[0039] The sampling strategy guides the sampling location, sampling quantity, sampling reason, and subsequent review priority.

[0040] Different video states require different sampling densities. For example, if the context summary indicates that the video contains violent motion or complex transitions, the sampling density needs to be increased to prevent missed detections; if it is a static image, the sampling can be reduced to save computing power.

[0041] For example, if the context summary shows that the video to be reviewed has a high bitrate and a high frame rate, the sampling frequency can be increased; if the video to be reviewed has a low bitrate and a low frame rate, the sampling frequency can be decreased.

[0042] In another example, if the context summary indicates that there is a station logo in the upper right corner of the screen, the station logo area can be avoided when generating the sampling strategy, or the station logo area can be specially processed during sampling.

[0043] The target sampled frame is the image frame that, after being filtered by the strategy, is ultimately used for in-depth analysis and risk assessment. In one possible implementation, the target sampled frame can be obtained by jumping to screenshots according to the time point and sampling method indicated by the sampling strategy; or, the captured video frame can be cropped to a specific size according to the sampling area indicated by the sampling strategy, retaining the core area content, to obtain the target sampled frame.

[0044] S103. Extract features from each target sampling frame to obtain the multi-dimensional video features corresponding to each target sampling frame.

[0045] Multidimensional video features refer to vector data that can represent video frame content, texture, frequency domain information, statistical patterns, and other information from multiple perspectives. Multidimensional video features can be combinations of multiple video features, where the number of video feature combinations for each target sample frame can be the same or different.

[0046] Among them, video features may include, but are not limited to, the following features: inter-frame consistency features, compression artifact features, noise texture features, frequency domain statistical features, lighting and shadow features, display processing trace features, and optional audio consistency features.

[0047] Inter-frame consistency features describe structural changes between adjacent frames, continuity of moving regions, local texture jumps, and edge flickering. Compression artifact features describe blockiness, ringing, texture loss due to coding quality variations, and recoding artifacts. Noise texture features describe noise distribution uniformity, texture detail continuity, and abnormal smoothness in weak texture regions. Frequency domain statistical features describe high-frequency energy distribution, local spectral anomalies, and resampling or generation artifacts. Illumination and shadow features describe illumination consistency between people, objects, and backgrounds, and anomalies in shadow direction. Display processing artifact features describe edge overshoot or timing discontinuities that may occur after playback display processing. Optional audio consistency features are used to assist in judging lip-sync, audio-visual synchronization, or audio spectral consistency when resources are approved.

[0048] Optionally, features can be extracted from each target sampling frame using one or more feature extraction models to obtain multi-dimensional video features corresponding to each target sampling frame.

[0049] S104. Correct the multi-dimensional video features corresponding to each target sampling frame based on multiple context summaries to obtain the target video features corresponding to each target sampling frame.

[0050] Feature correction refers to the process of weighting, compensating, or filtering the original extracted features using contextual summarization to eliminate interference from environmental factors. Target video features refer to the combination of video features obtained after correcting multi-dimensional video features.

[0051] For example, if the context summary shows that the video underwent super-resolution processing during playback, the original multidimensional video features may contain sharpening traces unique to the super-resolution algorithm, which can easily be misjudged as AI forgery. In this case, the multidimensional video features can be corrected based on the context summary, such as reducing the weight of noise and texture features in the multidimensional video features of the target sampled frame indicated by the context summary, or performing normalization processing, thereby eliminating the impact of this feature on the final risk assessment result.

[0052] One possible approach is to use the context summary as a conditional vector and the multidimensional video features as input parameters, then apply the risk assessment to a risk assessment model to obtain the risk assessment result for the target sampled frame. Alternatively, the context summary can be used to perform parameter compensation on each video feature in the multidimensional video features, such as weight reduction, normalization, or confidence correction, to obtain the target video features.

[0053] S105. Based on the target video features corresponding to each target sampling frame, perform risk assessment on the video to be reviewed, obtain the risk assessment result of the video to be reviewed, and generate the review result of the video to be reviewed based on the risk assessment result and the video information of the video to be reviewed.

[0054] Risk assessment results refer to the quantitative score or classification label of whether a video has any violation risks (such as AI face swapping, deepfakes, or illegal content).

[0055] Optionally, the target video features of each target sampling frame can be input into a pre-built model, which will then classify and predict the data to obtain the risk assessment results for each target sampling frame. Finally, the risk assessment results of the video to be reviewed can be obtained by combining the risk assessment results of all target sampling frames.

[0056] The review results are the final decision instructions output to the business system, which include the risks of the video to be reviewed, the risk level, and corresponding handling suggestions.

[0057] Optionally, multiple risk levels can be preset, and different risk and corresponding handling strategy mapping tables can be pre-set. After obtaining the risk assessment result, the risk assessment result can be searched in the mapping table to obtain the handling suggestion of the risk assessment result. All handling suggestions are combined according to the preset format to obtain the review result of the video to be reviewed.

[0058] After obtaining the audit results, the models and audit strategies used in the previous steps can be updated based on the audit results, and logs can be generated to facilitate subsequent technical optimization and problem backtracking.

[0059] In this embodiment, by constructing a playback link simulation environment, the presentation state of the video on a real terminal can be restored, solving the problem caused by traditional review methods that only target the original file and ignore the image features after transcoding, overlay, and other processing. By collecting context summaries and generating dynamic sampling strategies accordingly, accurate capture of key content is achieved, avoiding the waste of computing power caused by invalid sampling. Using context summaries to correct multi-dimensional video features effectively eliminates interference noise introduced by the playback environment, making the extracted target video features more reflective of the true nature of the video content, thereby significantly reducing the false judgment rate. Finally, the review results are generated by combining video information, improving the accuracy, robustness, and processing efficiency of video review.

[0060] Figure 2 This document illustrates a flowchart of an embodiment of the present application for obtaining a basic sampling frame and a context summary, with reference to... Figure 2 The process of obtaining the basic sampling frame and collecting multiple contextual summaries of the video to be reviewed based on the basic sampling frame includes: S201. During the playback of the video to be reviewed, multiple video frames are collected according to a preset period, and each collected video frame is used as a basic sampling frame.

[0061] Optionally, during the playback of the video to be reviewed, the current playback timestamp can be monitored. Whenever the timestamp increment reaches the time value corresponding to the preset period, a screenshot command is triggered to capture the image of the current rendering buffer as the base sampling frame.

[0062] In another implementation, the player's frame rendering callback interface can be used to set a frame counter. Whenever the number of rendered frames reaches a preset threshold, that frame is acquired and saved as a base sample frame.

[0063] S202. During the playback of the video to be reviewed, listen to and record the metadata and status information generated in the playback link, extract information from the metadata and status information, and extract information from each basic sampling frame to obtain a context summary. The context summary includes: bitstream structure summary, transcoding summary, playback quality summary, image overlay element summary, display processing summary, and abnormal playback status summary.

[0064] The playback chain refers to the complete processing path of video data from decapsulation and decoding to final rendering and display. Metadata refers to data describing the attributes of video files, while status information refers to dynamic data during player runtime.

[0065] Optionally, a global listener can be registered in the player kernel to subscribe to system events such as decapsulation completion, decoding start, rendering frame output, and error occurrence, and write the metadata and status information carried by the events to a temporary log buffer. Alternatively, the player can map key runtime status variables (such as current bitrate and number of dropped frames) to a shared memory area for direct reading by external acquisition modules, avoiding frequent system calls.

[0066] For example, for metadata, information can be extracted by a parser using regular expressions or key-value pairs to obtain transcoding digests and bitstream structure digests. For basic sampled frames, image processing algorithms can be used to extract image overlay element digests and display processing digests from the image.

[0067] In this embodiment, by synchronously acquiring basic sampling frames and full-link context summaries during playback, not only are the visual images of the video obtained, but also environmental information such as the bitstream structure, transcoding history, playback quality, and abnormal states are recorded simultaneously, constructing a complete digital fingerprint of the video. Furthermore, by extracting overlaid elements and display processing summaries, visual interference caused by station logos, subtitles, or image enhancement can be identified, providing a data foundation for subsequently distinguishing between content violations and playback artifacts, significantly improving the accuracy of the review process.

[0068] Figure 3 A flowchart of a generation sampling strategy provided in an embodiment of this application is shown, with reference to... Figure 3 The process of generating a sampling strategy based on multiple context summaries includes: S301. Search in the preset strategy library according to each context summary to obtain at least one target summary in the strategy library that matches the context summary.

[0069] A strategy library can be a pre-configured set of rules or a knowledge graph, used to store the mapping between context summaries and sampling strategies. Different playback link states have different requirements for sampling accuracy and density. By searching a pre-built strategy library, complex context summaries can be transformed into explicit sampling plans, avoiding blind sampling.

[0070] One possible implementation is to construct a vector index for a strategy library. When determining a sampling strategy, the context summary is first vectorized, and then a similarity search is performed in the vector index of the strategy library to find the closest historical sampling strategy as the target summary. Alternatively, multiple closest historical sampling strategies can be retrieved as the target summary.

[0071] S302. Generate the sampling strategy corresponding to the context summary based on the sampling strategy corresponding to each target summary.

[0072] In the first implementation, if the number of retrieved target summaries is only one, the sampling strategy corresponding to the target summary can be directly used as the sampling strategy corresponding to the context summary.

[0073] In the second implementation, if multiple target summaries are retrieved, the sampling strategies corresponding to the target summaries can be split and combined to generate the sampling strategies corresponding to the context summaries.

[0074] For example, a priority weight can be pre-set for each strategy in the strategy library, and when multiple strategies conflict, the higher priority strategy is used to override the lower priority strategy.

[0075] In another example, the sampling strategies corresponding to each target summary can be used as different constraints. By solving the intersection of multiple constraints, a strategy that satisfies all constraints can be generated as the sampling strategy corresponding to the context summary.

[0076] In this embodiment of the application, a preset strategy library can be used to dynamically adjust the sampling density and region according to the actual playback link status of the video, thereby avoiding the waste of computing power in simple scenarios, while ensuring the coverage and accuracy of the review in complex scenarios.

[0077] Figure 4 This application provides a flowchart for obtaining multidimensional video features according to an embodiment of the present application. (Refer to...) Figure 4 The process of extracting features from each target sampling frame to obtain the multi-dimensional video features corresponding to each target sampling frame includes: S401. Input the target sampled frame into the feature extraction model, and the feature extraction model extracts multiple initial video features.

[0078] Optionally, video features of different dimensions can be extracted using multiple feature extraction models. For example, image feature extraction models can be used to extract inter-frame consistency features, compression artifact features, noise texture features, frequency domain statistical features, lighting and shadow processing features, and display processing traces. Optional audio consistency features can be extracted using audio feature extraction models.

[0079] S402. Determine the risk level of the video to be reviewed.

[0080] Risk level refers to the degree of risk pre-classified by the system based on the business attributes, historical performance, or preliminary detection results of the video to be reviewed. For example, videos to be reviewed from trusted platforms can be rated as low risk, while videos uploaded by individuals can be rated as high risk.

[0081] S403. Determine the multidimensional video features corresponding to the target sampling frame from multiple initial video features based on the risk level.

[0082] Optionally, the number of video features in a multidimensional video feature can be determined based on the risk level. Multidimensional video features with a high risk level have a larger number of video features, while multidimensional video features with a low risk level have a smaller number of video features.

[0083] In another possible implementation, different risk levels can indicate the weights of different video features. After determining the risk level, the initial video features of each target sampling frame are fused according to the weights indicated by the risk level to obtain the multi-dimensional video features corresponding to the target sampling frame.

[0084] In this embodiment of the application, by introducing risk levels for feature extraction and selection, the computational power and accuracy can be better balanced, and the review of high-risk videos can be strengthened.

[0085] Figure 5 This application provides a flowchart for obtaining target video features according to an embodiment of the present application. (Refer to...) Figure 5 The process described above, which corrects the multi-dimensional video features corresponding to each target sampling frame based on multiple context summaries to obtain the target video features corresponding to each target sampling frame, includes: S501. Based on multiple context summaries, determine the correction strategy, which includes correction strategies for each dimension.

[0086] Correction strategies are used to indicate how to eliminate interference caused by playback processing on video features. These strategies include normalization, weight reduction, or confidence correction of video features. For each dimension of video features, a correction method can be configured separately, and the correction strategy for the video features can be determined by combining the context summary.

[0087] Each context summary has pre-configured trigger conditions and corresponding affected video features, and the correction strategy corresponding to the affected video features can be pre-configured. Table 1 below shows a mapping table of the correspondence between correction strategies and context summaries.

[0088] Table 1. Examples of Correspondence Mapping Representations

[0089] S502. Determine the dimensions to which the multidimensional video features corresponding to each target sampling frame belong.

[0090] Optionally, multidimensional video features can be obtained by concatenating features of each dimension in a predefined order, and then segmented directly according to a preset index range during the correction stage to obtain video features of each dimension in the multidimensional video features, with each dimension having a corresponding correction strategy.

[0091] S503. According to the correction strategy of each dimension, the video features of each dimension in the multi-dimensional video features corresponding to each target sampling frame are corrected.

[0092] For example, when the playback quality summary shows a strong compression effect, high-frequency energy attenuation in the frequency domain may be caused by compression, thus reducing the contribution of frequency domain anomalies to the risk score. When the playback link is enabled or simulated display processing is used, edge overshoot is not directly used as a risk factor, but rather weighted and adjusted in conjunction with the display processing summary. When an anomaly is detected in the overlay element region, the impact of that region on the overall image quality evidence score is reduced. When the playback status summary shows a short-term playback switch or retry, it is marked as playback status interference, and will not be directly concluded as high-risk in subsequent risk assessments.

[0093] In this embodiment, video features are corrected through context summaries. The affected video features can be downweighted or filtered according to the specific playback environment, thereby reducing artifact interference introduced by non-content factors. Furthermore, the output target video features can more realistically reflect the video content, reducing the false alarm rate caused by the complex processing of the OTT playback link and improving the accuracy and reliability of the review system.

[0094] Figure 6 This application provides a flowchart for obtaining a risk assessment result, which is illustrated in an embodiment of the present application. Figure 6 The process of determining the risk of the video to be reviewed based on the target video features corresponding to each target sampling frame, and obtaining the risk determination result of the video to be reviewed, includes: S601. Input the target video features corresponding to each target sampling frame into the risk assessment model. The risk assessment model searches for the corresponding target comparison video frame in the video database based on the target video features corresponding to each target sampling frame, and compares each target sampling frame with the target comparison video frame to obtain the initial risk score and the confidence level of each target sampling frame.

[0095] The risk assessment model is an algorithmic model used to evaluate the risk of video forgery. The video database is a database storing a large number of known features of real videos or known features of forged videos. The target comparison video frame is a reference frame retrieved from the database that is most similar to or most relevant to the target sample frame of the video to be reviewed. The initial risk score is the risk probability or score initially calculated by the model. The confidence level is the degree of certainty the model has about this initial risk score.

[0096] Optionally, the target video features can be input into the model, which transforms them into retrieval vectors. The model then searches a video database to find the most similar target comparison video frames. The cosine similarity or Euclidean distance between the two feature vectors is calculated and mapped to an initial risk score. Simultaneously, a confidence score is calculated based on the score variance or distance threshold of the K retrieved samples, where K is a positive integer.

[0097] S602. If the initial risk score is less than the preset risk threshold and the confidence level is greater than the preset confidence level threshold, then a risk determination result is generated based on the initial risk score of each target sampling frame.

[0098] The preset risk threshold is the critical point for determining whether a video poses a risk. The preset confidence threshold is the critical point for determining whether the model results are reliable.

[0099] S603. If the initial risk score is greater than the preset risk threshold and the confidence level is less than the preset confidence level threshold, then the risk assessment result of the video to be reviewed is obtained based on the characteristics of each target video.

[0100] When the model considers the risk to be low and the confidence level to be high, there is no need for complex secondary review. The initial risk score can be directly used as the risk assessment result for the video to be reviewed, thereby greatly improving the review efficiency.

[0101] When the model indicates a high risk but the confidence level is insufficient—for example, a high score but no similar comparison frames can be found in the database, or conflicting features—it suggests the possible existence of sophisticated forgery methods or special playback link interference. In such cases, a higher-level review mechanism can be used to make the final decision, preventing false positives or missed plays.

[0102] Figure 7 This application provides a flowchart for obtaining a risk assessment result, which is illustrated in an embodiment of the present application. Figure 7 The process of reviewing each target video based on its characteristics to obtain the risk assessment result for the video to be reviewed includes: S701. Determine at least one review branch for the video to be reviewed. The review branches include: timing consistency branch, compression branch, frequency domain statistics branch, display processing traces branch, and audio consistency branch.

[0103] The review branch refers to the detection channel set up to detect forgery traces that may be exposed in AI-generated videos from different dimensions.

[0104] The temporal consistency branch is used to detect the continuity of the video on the timeline, such as inter-frame flickering and sudden changes in motion; the compression branch is used to analyze abnormal artifacts generated during the encoding and compression process of the video; the frequency domain statistics branch is used to capture the periodic patterns or abnormal distributions unique to AI-generated images in the frequency domain; the display processing traces branch is used to identify the features left by post-processing algorithms such as super-resolution and noise reduction; and the audio consistency branch is used to detect whether the audio and video are synchronized, whether the lip movements match, or whether there are synthetic traces in the audio.

[0105] Optionally, separate reviews can be conducted for audio, images, or subtitles of the video to be reviewed. After determining the type to be reviewed, the review branch corresponding to the review type can be used as the review branch for the video to be reviewed. For example, the review branches corresponding to the image review type include timing consistency branch, compression branch, frequency domain statistics branch, and display processing trace branch, while the review branches corresponding to the audio review type include audio consistency branch.

[0106] S702. Input the target video features corresponding to each target sampling frame into the risk judgment model corresponding to each review branch to obtain the review judgment result corresponding to each review branch.

[0107] The review and judgment result refers to the local detection conclusion output by each independent review branch, including the risk score, confidence level or specific anomaly label under that branch dimension.

[0108] Optionally, each review branch has a corresponding risk assessment model. The target video features can be input into the pre-trained risk assessment model to obtain the review assessment results of each review branch.

[0109] S703. Based on the review judgment results corresponding to each review branch, generate the risk judgment result of the video to be reviewed.

[0110] The risk assessment result refers to the overall audit conclusion that is finally output after synthesizing the partial conclusions of all review branches and is used to guide subsequent business handling.

[0111] In one possible implementation, weights can be assigned to different review branches. The review results of each branch are then weighted and summed, or a majority voting mechanism is used, where more than half of the branches are deemed high-risk, thus generating the final high-risk result.

[0112] In another possible implementation, the output of each review branch can be used as a new feature vector and input into a top-level meta-classifier. This classifier then comprehensively weighs the evidence strength of each branch and outputs the final risk assessment result.

[0113] In this embodiment of the application, by constructing a multi-dimensional review branch, the video can be comprehensively analyzed from multiple dimensions, thereby eliminating occasional interference from a single dimension, and ensuring that even when facing complex scenarios with high risk but insufficient confidence, it can still output extremely accurate and reliable final risk assessment results.

[0114] Figure 8 This application provides a flowchart for generating audit results, which is illustrated in an embodiment. Figure 8 The process of generating the review result for the video to be reviewed based on the risk assessment result and the video information of the video to be reviewed includes: S801. Match the risk assessment results and video information in a preset mapping table to obtain the corresponding review suggestions for the risk assessment results and video information in the mapping table.

[0115] The preset mapping table is a pre-configured rule base or decision matrix that stores the specific handling actions corresponding to the combination of technical risks and business attributes.

[0116] For example, suggestions can be made to approve low-risk content and log low-risk information can be recorded. Suggestions can be made to supplement the AI-identified but low-risk content with supplementary identification or source statements. Background review or manual review tasks can be generated for suspected risky content. Suggestions can be made to reject high-risk content, restrict its online release, restrict its entry into key operational positions, or require manual review. For inspection tasks of already online content, suggestions can be made to continue online, supplement identification, enter manual review, restrict recommendations, or remove the content for review. Suggestions can be made to submit anonymized features, low-resolution sampled fragments, or evidence summaries to the high-precision review module for low-confidence or controversial samples.

[0117] Optionally, a multidimensional decision table can be pre-built, where rows represent risk levels and columns represent business attributes. The risk assessment results and video information are matched in the multidimensional decision table to obtain corresponding review suggestions.

[0118] As another possible implementation, the risk assessment results and business attributes can be vectorized and input into a strategy recommendation model. The model can then dynamically calculate and output the most suitable review suggestion based on the current strategy version.

[0119] Table 2 shows an example of a mapping between risk assessment results and audit recommendations. The table records the audit recommendations corresponding to different risk assessment results, as well as the system tasks that can be triggered by each audit recommendation.

[0120] Table 2 Examples of Result Mapping

[0121] S802. Combine the various audit suggestions to obtain the audit results.

[0122] When there are multiple review suggestions, all the extracted review suggestions can be packaged into a list as the final review result.

[0123] Based on the same inventive concept, this application also provides a video review device corresponding to the video review method. Since the principle of the device in this application is similar to that of the video review method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0124] Figure 9 A schematic diagram of a video review device provided in an embodiment of this application is shown. The device includes an acquisition module 901, a generation module 902, an extraction module 903, a correction module 904, and a judgment module 905.

[0125] The acquisition module 901 is used to play the video to be reviewed in the playback link simulation environment, acquire basic sampling frames, and acquire multiple context summaries of the video to be reviewed based on the basic sampling frames. The generation module 902 is used to generate a sampling strategy based on multiple context summaries, and to perform sampling processing on the video to be reviewed based on the sampling strategy to obtain multiple target sampling frames; The extraction module 903 is used to extract features from each target sampling frame to obtain multi-dimensional video features corresponding to each target sampling frame; The correction module 904 is used to correct the multi-dimensional video features corresponding to each target sampling frame based on multiple context summaries, so as to obtain the target video features corresponding to each target sampling frame. The judgment module 905 is used to make risk judgments on the video to be reviewed based on the target video features corresponding to each target sampling frame, obtain the risk judgment result of the video to be reviewed, and generate the review result of the video to be reviewed based on the risk judgment result and the video information of the video to be reviewed.

[0126] Optionally, the acquisition module 901 is specifically used for: During the playback of the video to be reviewed, multiple video frames are collected according to a preset period, and each collected video frame is used as the basic sampling frame. During the playback of the video to be reviewed, the metadata and status information generated in the playback link are monitored and recorded. Information is extracted from the metadata and status information, and information is extracted from each basic sampling frame to obtain a context summary. The context summary includes: bitstream structure summary, transcoding summary, playback quality summary, image overlay element summary, display processing summary, and abnormal playback status summary.

[0127] Optionally, the generation module 902 is specifically used for: Each context summary is searched in a preset strategy library to obtain at least one target summary that matches the context summary in the strategy library. Based on the sampling strategy corresponding to each target summary, generate the sampling strategy corresponding to the context summary.

[0128] Optionally, the extraction module 903 is specifically used for: The target sampled frame is input into the feature extraction model, which extracts multiple initial video features. Determine the risk level of the videos to be reviewed; Based on the risk level, the multidimensional video features corresponding to the target sampling frame are determined from multiple initial video features.

[0129] Optionally, the calibration module 904 is specifically used for: Based on multiple context summaries, a correction strategy is determined, which includes correction strategies for each dimension. Determine the dimensions to which the multidimensional video features corresponding to each target sampling frame belong; According to the correction strategy for each dimension, the video features of each dimension in the multidimensional video features corresponding to each target sampling frame are corrected.

[0130] Optionally, the determination module 905 is specifically used for: The target video features corresponding to each target sampling frame are input into the risk assessment model. The risk assessment model searches for the corresponding target comparison video frame in the video database based on the target video features corresponding to each target sampling frame, and compares each target sampling frame with the target comparison video frame to obtain the initial risk score and the confidence level of each target sampling frame. If the initial risk score is less than the preset risk threshold and the confidence level is greater than the preset confidence threshold, then a risk determination result is generated based on the initial risk score of each target sampling frame. If the initial risk score is greater than the preset risk threshold and the confidence level is less than the preset confidence level threshold, then the risk assessment result of the video to be reviewed is obtained based on the characteristics of each target video.

[0131] Optionally, the determination module 905 is specifically used for: Identify at least one review branch for the video to be reviewed. The review branches include: timing consistency branch, compression branch, frequency domain statistics branch, display processing traces branch, and audio consistency branch. Input the target video features corresponding to each target sampling frame into the risk judgment model corresponding to each review branch to obtain the review judgment result corresponding to each review branch. Based on the review results of each review branch, a risk assessment result for the video to be reviewed is generated.

[0132] Optionally, the determination module 905 is specifically used for: The risk assessment results and video information are matched in a preset mapping table to obtain the corresponding review suggestions for the risk assessment results and video information in the mapping table; The audit results are obtained by combining the various audit suggestions.

[0133] This application constructs a playback link simulation environment, which can reproduce the presentation state of video on a real terminal, solving the problem caused by traditional review methods that only target the original file and ignore the image features after transcoding, overlay, and other processing. By collecting context summaries and generating dynamic sampling strategies based on them, it achieves accurate capture of key content and avoids the waste of computing power caused by invalid sampling. By using context summaries to correct multi-dimensional video features, it effectively eliminates interference noise introduced by the playback environment, making the extracted target video features more reflective of the true nature of the video content, thereby significantly reducing the false judgment rate. Finally, by combining video information to generate review results, it improves the accuracy, robustness, and processing efficiency of video review.

[0134] Figure 10 This illustration shows a schematic diagram of an electronic device provided in an embodiment of this application, including: a processor 1001, a storage medium 1002, and a bus 1003. The storage medium 1002 stores machine-readable instructions executable by the processor 1001. When the electronic device runs a video review method as described in the embodiment, the processor 1001 communicates with the storage medium 1002 via the bus 1003. The processor 1001 executes the machine-readable instructions, and the preamble of the method item of the processor 1001 executes the steps in the above-described video review method.

[0135] This application also provides a computer-readable storage medium storing a computer program, which is executed by a processor, and the processor performs the steps in the video review method described above.

[0136] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0137] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0138] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0140] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0142] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A video review method, characterized in that, include: In the playback link simulation environment, the video to be reviewed is played to obtain basic sampling frames, and multiple context summaries of the video to be reviewed are collected based on the basic sampling frames. A sampling strategy is generated based on the multiple context summaries, and the video to be reviewed is sampled based on the sampling strategy to obtain multiple target sampling frames; Feature extraction is performed on each of the target sampling frames to obtain multi-dimensional video features corresponding to each target sampling frame; Based on the multiple context summaries, the multidimensional video features corresponding to each target sampling frame are corrected to obtain the target video features corresponding to each target sampling frame; The risk assessment of the video to be reviewed is performed based on the target video features corresponding to each target sampling frame, and the risk assessment result of the video to be reviewed is obtained. The review result of the video to be reviewed is then generated based on the risk assessment result and the video information of the video to be reviewed.

2. The method according to claim 1, characterized in that, The process of obtaining a basic sampling frame and collecting multiple contextual summaries of the video to be reviewed based on the basic sampling frame includes: During the playback of the video to be reviewed, multiple video frames are collected according to a preset period, and each of the collected video frames is used as the basic sampling frame. During the playback of the video to be reviewed, the metadata and status information generated in the playback link are monitored and recorded. Information is extracted from the metadata and status information, and information is extracted from each of the basic sampling frames to obtain the context summary. The context summary includes: bitstream structure summary, transcoding summary, playback quality summary, image overlay element summary, display processing summary, and abnormal playback status summary.

3. The method according to claim 1, characterized in that, The step of generating a sampling strategy based on the multiple context summaries includes: Each context summary is searched in a preset policy library to obtain at least one target summary in the policy library that matches the context summary. Based on the sampling strategy corresponding to each target summary, a sampling strategy corresponding to the context summary is generated.

4. The method according to claim 1, characterized in that, The step of extracting features from each of the target sampling frames to obtain multi-dimensional video features corresponding to each target sampling frame includes: The target sampled frame is input into the feature extraction model, and the feature extraction model extracts multiple initial video features; Determine the risk level of the video to be reviewed; Based on the risk level, the multidimensional video features corresponding to the target sampling frame are determined from the plurality of initial video features.

5. The method according to claim 1, characterized in that, The step of correcting the multidimensional video features corresponding to each of the target sampling frames based on the multiple context summaries to obtain the target video features corresponding to each of the target sampling frames includes: Based on the multiple context summaries, a correction strategy is determined, which includes correction strategies for each dimension; Determine the dimensions to which the multidimensional video features corresponding to each of the target sampling frames belong; According to the correction strategy for each dimension, the video features of each dimension in the multidimensional video features corresponding to each target sampling frame are corrected.

6. The method according to claim 1, characterized in that, The step of determining the risk of the video to be reviewed based on the target video features corresponding to each of the target sampling frames, and obtaining the risk determination result of the video to be reviewed, includes: The target video features corresponding to each target sampling frame are input into the risk assessment model. The risk assessment model searches for the corresponding target comparison video frame in the video database based on the target video features corresponding to each target sampling frame, and compares each target sampling frame with the target comparison video frame to obtain the initial risk score of each target sampling frame and the confidence level of the initial risk score. If the initial risk score is less than a preset risk threshold and the confidence level is greater than a preset confidence threshold, then a risk determination result is generated based on the initial risk score of each target sampling frame. If the initial risk score is greater than the preset risk threshold and the confidence level is less than the preset confidence level threshold, then the risk assessment result of the video to be reviewed is obtained based on the characteristics of each target video.

7. The method according to claim 6, characterized in that, The step of reviewing the video based on the features of each target video to obtain the risk assessment result of the video to be reviewed includes: Determine at least one review branch for the video to be reviewed, the review branch including: timing consistency branch, compression branch, frequency domain statistics branch, display processing traces branch, and audio consistency branch; Input the target video features corresponding to each target sampling frame into the risk judgment model corresponding to each review branch to obtain the review judgment result corresponding to each review branch. Based on the review results of each review branch, a risk assessment result for the video to be reviewed is generated.

8. The method according to claim 1, characterized in that, The step of generating the review result for the video to be reviewed based on the risk assessment result and the video information of the video to be reviewed includes: The risk assessment result and the video information are matched in a preset mapping table to obtain the review suggestions corresponding to the risk assessment result and the video information in the mapping table; The audit results are obtained by combining the various audit suggestions.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of a video review method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a video review method as described in any one of claims 1 to 8.

Images