Video auditing system and method based on artificial intelligence
Through the video auditing system based on artificial intelligence, the video content is automatically identified and analyzed, and the problems of low efficiency and insufficient accuracy of traditional manual audits are solved, efficient and accurate video content review is achieved, and content quality control is improved.
Patent Information
- Application Number
- CN202411131177.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-17
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional video content management relies on manual review, which is inefficient and difficult to ensure quality standards, and there are problems with audit efficiency and accuracy.
Adopt a video auditing system based on artificial intelligence, including video data acquisition module, policy configuration module, AI audit module and review module, automatically review video content through AI, identify violations and quality problems, reduce false alarms and missed reports, and provide visual operations Environmental and timing analysis results.
It improves the efficiency and accuracy of video audits, reduces false alarms and missed reports, helps quickly fix abnormal problems, and improves the effectiveness of content quality control.
Smart Images

Figure CN119094843A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and more specifically to a video review system and method based on artificial intelligence. Background Art
[0002] With the booming development of Internet education, the number of online teaching videos has grown exponentially, aiming to meet the diverse learning needs of users. However, the widespread nature of this knowledge dissemination channel also exposes the challenge of content quality control: video materials vary significantly in terms of healthiness, suitability, and viewing friendliness. Traditional video content management relies on a large-scale manual operation team to perform one-by-one review. This method is not only limited by huge time and resource consumption, but also difficult to avoid omissions due to the heavy review tasks, resulting in a discount in both review efficiency and accuracy, and unable to fully guarantee the quality standards of platform content.
[0003] Therefore, how to improve the efficiency and accuracy of video review is an issue that technical personnel in this field urgently need to solve. Summary of the invention
[0004] In view of this, the present invention provides a video review system and method based on artificial intelligence, which can realize automatic review through AI and improve the review efficiency of the video. At the same time, it can configure the recognition model according to different review items, and can more accurately identify illegal content and quality issues, and reduce false alarms and missed alarms.
[0005] In order to achieve the above object, the present invention adopts the following technical solution:
[0006] An artificial intelligence-based video review system, including a video data acquisition module, a policy configuration module, an AI review module and a review module;
[0007] The video data acquisition module is used to receive video files in at least one format;
[0008] The policy configuration module is used for users to configure the detection parameters of AI audit;
[0009] The AI audit module is used to configure audit items and identify each selected audit item according to the detection parameters. For the identified abnormal points, the timestamp, screenshot and confidence level in the video file are marked and a link is generated based on the video file.
[0010] The review module is used to perform statistical sorting according to the abnormality categories and timestamps of the abnormal points to generate a visual operating environment; and is used to receive user instructions to generate timing analysis results for video files.
[0011] Preferably, the AI review module includes a first video preprocessing submodule, a video content review submodule, a second video preprocessing submodule and a video quality review submodule.
[0012] The first video preprocessing submodule is used to extract frames and audio from video files according to first configuration information; the video content review submodule is used to perform detection on a preset first detection item based on the output result of the first video preprocessing submodule; the video quality review submodule is used to extract frames and audio from video files according to second configuration information; and is used to slice the extracted audio; the second video processing submodule is used to perform detection on a preset second detection item based on the output result of the second video preprocessing submodule.
[0013] Preferably, the video content review submodule includes one or more of a pornography detection unit, a politics detection unit, a sensitive information detection unit, an advertisement detection unit and a picture and text detection unit.
[0014] Preferably, the video quality review submodule includes a black screen detection unit, a white screen detection unit, a blur detection unit, a mosaic detection unit, a silence detection unit, a sound interruption detection unit, a low volume detection unit and a high volume detection unit.
[0015] Preferably, the AI review module also includes a sensitive word management submodule, which is used to collect sensitive word blacklist text offline for the video content review submodule to perform content anomaly detection.
[0016] Preferably, the review module includes an abnormality statistics submodule, a video projection submodule and an abnormality handling submodule.
[0017] The abnormal statistics submodule is used to classify abnormal items according to abnormal points and sort them according to the timestamps of the abnormal points; the video projection submodule is used to play video files and receive abnormal point selection signals to locate the corresponding timestamps; the abnormal processing submodule is used to pay attention to or ignore each abnormal point.
[0018] Preferably, it also includes an audit details module and a report generation module; the audit details module is used to store and manage the review results of the review module; and the report generation module is used to generate an audit report based on the review results.
[0019] Preferably, it also includes an abnormality correction module for mosaic processing, deletion, image quality enhancement, image repair and color correction of video files.
[0020] A video review method based on artificial intelligence, comprising the following steps:
[0021] S1: Get the video to be reviewed.
[0022] S2: AI Audit:
[0023] Select the audit project and pre-process the audited video according to the audit requirements of each audit project; perform anomaly recognition on the pre-processed results to obtain the anomaly points in the video; mark each anomaly point with the timestamp, screenshot and confidence level of the anomaly category in the video file, and generate a playback link accordingly;
[0024] S3: Review:
[0025] Count the abnormal points, classify and sort them, and generate a visual operation environment; receive external operation signals and generate timing analysis results for video files.
[0026] Preferably, the steps further include:
[0027] S4: Display the results of S3 in detail, showing the specific information of each anomaly point, including screenshots, timestamps, confidence level of anomaly type, review and disposal methods, and corresponding video jump links.
[0028] S5: Configure the report format and generate the audit report.
[0029] It can be seen from the above technical solution that compared with the prior art, the present invention discloses a video review system and method based on artificial intelligence, which can realize separate review of different review items through AI to improve efficiency and accuracy; the timing analysis results obtained through review can intuitively reflect abnormal situations in video files, which helps to help uploaders quickly fix abnormal problems. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0031] Figure 1 A schematic diagram of a video review system based on artificial intelligence provided by an embodiment of the present invention;
[0032] Figure 2 This is a schematic diagram of the structure of the AI audit module provided in an embodiment of the present invention;
[0033] Figure 3 A schematic diagram of the structure of a review module provided in an embodiment of the present invention;
[0034] Figure 4A schematic diagram of an artificial intelligence-based video review method provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] Example 1
[0037] like Figure 1 An embodiment of the present invention discloses a video review system based on artificial intelligence, including a video data acquisition module, a policy configuration module, an AI review module and a review module.
[0038] The video data acquisition module is used to receive video files in at least one format; the policy configuration module is used for users to configure the detection parameters of AI audits; the AI audit module is used to configure audit items and identify each selected audit item based on the detection parameters. For the identified anomalies, the timestamp, screenshot and confidence level in the video file are marked and a link is generated based on the video file; the review module is used to perform statistical sorting based on the anomaly category and timestamp of the anomaly points and generate a visual operating environment; and the module is used to receive user instructions to generate timing analysis results for video files.
[0039] Next, each structure in this embodiment is described in detail:
[0040] 1. Video data acquisition module
[0041] The video data acquisition module is responsible for receiving and processing video files of different formats, ensuring data compatibility through an efficient transcoding process. It supports users to manually upload video files, whether uploading a single file or multiple files at the same time, or uploading in pieces, to meet a variety of usage scenarios.
[0042] In addition, this module also supports data docking and collection functions between systems, allowing users to automatically synchronize videos to the video intelligent review system after uploading videos in the designated system, thereby achieving seamless video data processing.
[0043] In terms of video formats, the module supports common video formats, including but not limited to mp4, flv, mpeg, mov, etc. For unsupported video formats, the module will automatically transcode them to mp4 format to ensure the smooth progress of subsequent processes. The video size is limited to a specified size range, such as within 2GB, which fully meets the upload and processing requirements of up to 3 hours of teaching videos.
[0044] During the uploading process, users can upload multiple video files at the same time, and the module provides a retry mechanism. For videos that fail to analyze or have unsatisfactory results, users can directly retry or re-upload for analysis.
[0045] In addition, the audit strategy replacement function allows users to select different audit standards as needed, further improving the flexibility and applicability of the system.
[0046] 2. Policy configuration module
[0047] The policy configuration module enables users to select different audit standards according to their needs, further improving the flexibility and applicability of the system.
[0048] Among them, the configurable detection parameters include: detection interval configuration, detection duration and detection threshold; the interval configuration of each detection model supports separate configuration; the detection duration of each detection model supports separate configuration; the threshold of each detection model supports separate setting.
[0049] 3. AI audit module
[0050] like Figure 2 The AI review module includes a first video preprocessing submodule, a video content review submodule, a second video preprocessing submodule and a video quality review submodule; the first video preprocessing submodule is used to extract frames and audio from the video file according to the first configuration information;
[0051] The video content review submodule is used to perform detection on the preset first detection item according to the output result of the first video preprocessing submodule; the video quality review submodule is used to extract frames and audio from the video file according to the second configuration information; and is used to slice the extracted audio; the second video processing submodule is used to perform detection on the preset second detection item according to the output result of the second video preprocessing submodule.
[0052] Among them, the first configuration information and the second configuration information both include the total length of the video and the set number of frames required for extraction; the frame extraction frequency is calculated according to the total length of the video and the number of frames desired to be extracted, and the first video preprocessing submodule and the second video processing submodule can extract frames according to different frame extraction frequencies. The optimization of the frame extraction strategy can take into account the complexity of the video content. For example, when there are many dynamic scenes, the frame extraction frequency can be appropriately increased. Ensure that both the key pictures can be captured and the computing resources are not over-consumed. By performing frame extraction detection on the video and setting different frame extraction frequencies, the efficiency of video content review can be effectively improved. Generally, we can configure the frame extraction strategy to extract one frame every 5 seconds, so that the number of frame extraction pictures for a normal 1-hour video is 720. If it is found that the detection is more intensive, it can be set to 3 seconds or more frequent intervals. But correspondingly, the more frequent the frame extraction, the more performance required for AI review, and the review speed will be relatively slower. Therefore, when configuring the strategy, it can be adjusted repeatedly to finally find a balance between accuracy and economy.
[0053] In addition, video preprocessing also includes audio extraction. Specifically, the audio file is separated from the input video, and the audio file is transcribed into the full video text through ASR. ASR can be transcribed through self-developed algorithms, or by calling the common audio file transcription API on the market. Sensitive words in the full text of the video are detected.
[0054] In order to further implement the above technical solution, the AI review module also includes a sensitive word management sub-module, which collects sensitive word blacklist text offline and imports it into the system so that the system can read the blacklist text content. If the same text is detected in the OCR or ASR text, the specific timestamp and hit text information will be output.
[0055] In order to further implement the above technical solution, the first detection item, as a detection item about video content, can be divided into three categories: pornographic, political and sensitive; in addition, advertising detection items and graphic detection items can also be added.
[0056] Each item is processed using a corresponding recognition model, which performs classification and calculates category confidence; it determines whether the item has reached an abnormal level based on a pre-confirmed audit threshold.
[0057] The video content review submodule includes pornographic detection unit, political detection unit, sensitive detection unit, advertising detection unit and image and text detection unit. The corresponding detection unit is called to perform detection according to the first review item selected by the user. The specific detection method is as follows:
[0058] Pornography detection: Based on the pornography type, pornography items are divided into multiple small categories. During the detection, a single item can be detected separately, or multiple items can be selected for simultaneous detection to detect whether pornography appears on the screen;
[0059] Political detection: Detect whether there are people or symbols related to political factors in the picture.
[0060] Sensitive word detection: Check whether sensitive words appear through image detection, OCR recognition and audio ASR transcription results.
[0061] Advertisement detection: Use image detection or CV visual detection to detect whether a mobile phone number, QR code, barcode, watermark, etc. appear in the screen;
[0062] Image and text detection: OCR recognizes the text in the book, and the system inputs the custom blacklist text. After OCR recognizes the text in the picture, it compares it with the blacklist text. If there is a similarity, it outputs the abnormal item.
[0063] In order to further implement the above technical solution, the second audit project is a detection project about video quality, which may include black screen detection, white screen detection, blur detection, mosaic detection, silence detection, sound interruption detection, volume too low detection and volume too high detection. Each project is processed by a corresponding recognition model, which is classified and calculates the category confidence; and the degree of abnormality is distinguished according to the pre-confirmed audit threshold.
[0064] The video quality review submodule includes a black screen detection unit, a white screen detection unit, a blur detection unit, a mosaic detection unit, a silence detection unit, a sound interruption detection unit, a low volume detection unit, and a high volume detection unit. The corresponding detection unit is called to perform detection according to the second review item selected by the user. The specific detection method is as follows:
[0065] Black screen detection: detect whether there is a black screen in the area or the entire screen in the video image.
[0066] White screen detection: Detect whether there is a white screen area or a completely black screen in the video image.
[0067] Blur detection: Detects blurry (lens blur, Gaussian blur) segments in video images.
[0068] Mosaic detection: Detect whether mosaics are artificially added in the video.
[0069] Silence detection: detect when the volume of the audio is too low or even no sound.
[0070] Sound intermittent detection: Detects intermittent sound volume in audio clips.
[0071] Low volume detection: Detects audio clips that are too low in volume and exceed the comfortable range for human ears.
[0072] Loud volume detection: Detects audio clips that are too loud and exceed the comfortable range of human ears.
[0073] In this embodiment, after completing the review, the video content review submodule and the video quality review submodule output their respective abnormal item situations, including the abnormal item content (picture clip or audio clip), abnormal item category (large item_small item), abnormal item time, and video online playback and jump links.
[0074] Generally speaking, AI review can be divided into image anomaly detection and audio anomaly detection.
[0075] The formula for image anomaly detection can be described as follows: Let F_abnormal(Vi) be the probability that the i-th frame has an abnormality. The pixel features, such as gray value distribution, texture features, etc., can be analyzed by computer vision algorithms to give an abnormality score S_abnormal(Vi). Then set the threshold T. If S_abnormal(Vi)>T, then F_abnormal(Vi)=1, otherwise F_abnormal(Vi)=0.
[0076] The formula for audio anomaly detection can be described as:
[0077] For audio frame A = {a_1, a_2, ..., a_m}, let Q_audio(a_j) be the abnormal probability of the j-th audio segment. By analyzing the audio spectrum, energy distribution and other features, the abnormal score E_audio(a_j) of indicators such as volume and clarity is calculated. The threshold T' is also set. When E_audio(a_j)>T', Q_audio(a_j) = 1, otherwise Q_audio(a_j) = 0.
[0078] According to the above method, the result of the abnormal probability of the picture and audio detection is output as 1, and the corresponding timestamp is output at the same time.
[0079] In order to further implement the above technical solutions, the results of AI audit are filtered and merged.
[0080] Regular rules: When the duration of abnormal problem E_dark1 is T_contin1, the duration of abnormal problem E_dark2 is T_contin2; if T_contin1<=N_darkcontin, filter this time period and do not output; when T_contin1>N_darkcontin, output the abnormal item normally.
[0081] Merge rule: If a certain type of abnormal problem, such as mosaic E_mosaic, appears in time period T1, and the same type of abnormal problem E_mosaic appears in time period T2, and T_contin1 time is less than N_procontin, then these two abnormal items can be merged into one abnormal item during post-processing, and the merged time display method is T1+T2. N_procontin supports independent setting for each type of abnormal item.
[0082] 4. Review module
[0083] The review module can form a visual operation interface based on the AI audit results for operators to conduct manual review.
[0084] like Figure 3 ,The review module includes an exception statistics submodule, a video projection submodule and an exception handling submodule.
[0085] The anomaly statistics submodule is used to classify the anomaly items according to the anomaly points and sort them according to the timestamps of the anomaly points; the video projection submodule is used to play the video file and receive the anomaly point selection signal to locate the corresponding timestamp; the anomaly processing submodule is used to mark each anomaly point as a key attention mark, a medium attention mark, or ignore it.
[0086] In this embodiment, the user can intuitively see the number of exceptions corresponding to each exception type in the visual operation interface. By clicking on each exception item, all exception points of the corresponding exception type can be obtained. Each exception point can be sorted according to timestamp or confidence level; the user can further click on the exception point to play and jump the video.
[0087] In addition, the coordinates of abnormal points can be marked.
[0088] In order to further implement the above technical solution, it also includes an audit details module and a report generation module; the audit details module stores and manages the review results of the review module; the report generation module is used to generate an audit report based on the review results.
[0089] Specifically, the audit details module can view the result details, which refers to the result after AI audit and manual review. At this time, the problem marked as "ignored" will no longer appear in the result details page; it can display the specific information of each abnormal item, including screenshots, timestamps, confidence / scores, etc. In this module, you can view the audit abnormal item results, view timing labels, view attention level marks, view abnormal screenshots, video online playback and jump; you can view the distribution of abnormal problems through timing labels, and you can click on the abnormal timing block to quickly view the abnormal time point;
[0090] The audit report generation module can summarize all audit results and automatically generate a PDF report for easy viewing and archiving. The format of the generated report can be customized by the user, and the default audit results include the audit video name and the number of audit anomalies. The audit details include the specific type of video quality audit, whether it passed the audit, and the number of anomalies; the specific type of video content audit, whether it passed the audit, and the number of anomalies; and specific anomaly details: anomaly type, number of anomalies, anomaly time, and specific anomaly cause.
[0091] Example 2
[0092] like Figure 4 Based on the same inventive concept, the embodiment of the present invention discloses a video review method based on artificial intelligence. The method can be implemented based on the system in embodiment 1. The specific steps of the method include:
[0093] S1: Get the video to be reviewed.
[0094] S2: AI review. Select the review project and pre-process the video to be reviewed according to the review requirements of each review project; perform anomaly recognition on the pre-processed results to obtain the abnormal points in the video; mark each abnormal point with the timestamp, screenshot and abnormal item category confidence in the video file, and generate a playback link accordingly;
[0095] S3: Review. Count the abnormal points, classify and sort them, and generate a visual operation environment; receive external operation signals and generate timing analysis results for video files.
[0096] S4: Display the results of S3 in detail, showing the specific information of each anomaly point, including screenshots, timestamps, confidence level of anomaly type, review and disposal methods, and corresponding video jump links.
[0097] S5: Configure the report format and generate the audit report.
[0098] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0099] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A video review system based on artificial intelligence, characterized in that: It includes video data acquisition module, policy configuration module, AI audit module and review module; The video data acquisition module is used to receive video files in at least one format; The policy configuration module is used for users to configure the detection parameters of AI audit; The AI audit module is used to configure audit items and identify each selected audit item according to the detection parameters. For the identified abnormal points, the timestamp, screenshot and confidence level in the video file are marked and a link is generated based on the video file. The review module is used to perform statistical sorting according to the abnormality categories and timestamps of the abnormal points and generate a visual operating environment; and is used to receive user instructions to generate timing analysis results for video files.
2. According to claim 1, a video review system based on artificial intelligence is characterized in that: The AI review module includes a first video preprocessing submodule, a video content review submodule, a second video preprocessing submodule and a video quality review submodule; The first video preprocessing submodule is used to extract frames and audio from the video file according to the first configuration information; The video content review submodule is used to perform detection on a preset first detection item according to the output result of the first video preprocessing submodule; The video quality audit submodule is used to extract frames and audio from the video file according to the second configuration information; Used to slice the extracted audio; The second video processing submodule is used to perform detection on a preset second detection item according to the output result of the second video preprocessing submodule.
3. The video review system based on artificial intelligence according to claim 2 is characterized in that: The video content review submodule includes one or more of a pornographic detection unit, a political detection unit, a sensitive detection unit, an advertisement detection unit, and a graphic detection unit.
4. The video review system based on artificial intelligence according to claim 2 is characterized in that: The video quality review submodule includes a black screen detection unit, a white screen detection unit, a blur detection unit, a mosaic detection unit, a silence detection unit, a sound interruption detection unit, a low volume detection unit and a high volume detection unit.
5. The video review system based on artificial intelligence according to claim 2 is characterized in that: The AI review module also includes a sensitive word management submodule, which is used to collect sensitive word blacklist text offline for the video content review submodule to perform content anomaly detection.
6. An artificial intelligence-based video review system according to claim 1, characterized in that: The review module includes an abnormality statistics submodule, a video projection submodule and an abnormality handling submodule; The abnormal statistics submodule is used to classify abnormal items according to abnormal points and sort them according to the timestamps of the abnormal points; The video projection submodule is used to play the video file and receive the abnormal point selection signal to locate the corresponding timestamp; The exception handling submodule is used to pay attention to or ignore each abnormal point.
7. An artificial intelligence-based video review system according to claim 1, characterized in that: It also includes an audit details module and a report generation module; The audit details module is used to store and manage the review results of the review module; The report generation module is used to generate an audit report based on the review results.
8. The video review system based on artificial intelligence according to claim 1 is characterized in that: It also includes an abnormality correction module for mosaic processing, deletion, image quality enhancement, image repair and color correction of video files.
9. A video review method based on artificial intelligence, characterized in that: The following steps are involved: S1: Get the video to be reviewed; S2: AI audit; Select the audit project and pre-process the audited video according to the audit requirements of each audit project; perform anomaly recognition on the pre-processed results to obtain the anomaly points in the video; mark each anomaly point with the timestamp, screenshot and confidence level of the anomaly category in the video file, and generate a playback link accordingly; S3: Review; Count the abnormal points, classify and sort them, and generate a visual operation environment; receive external operation signals and generate timing analysis results for video files.
10. The video review method based on artificial intelligence according to claim 9, characterized in that: The steps also include: S4: Display the results of S3 in detail, showing the specific information of each abnormal point, including screenshots, timestamps, confidence of abnormal type, review and disposal methods, and corresponding video jump links; S5: Configure the report format and generate the audit report.
Citation Information
Cited By
System and method for automatically rechecking video AI recognition result
CN119296010A
Early warning management method of video intelligent application system
CN120279691A
Video content automatic auditing method and system based on AI drive
CN120726547A
Video playback method and control system, electronic equipment and computer readable storage medium
CN120812357A
