Left and right brain collaborative artificial intelligence network audio and video management method
By employing an AI governance approach that integrates left and right brain functions, and combining visual and text detection, this approach addresses the problem of existing technologies that rely solely on visual detection, achieving more efficient and accurate network audio and video detection.
Patent Information
- Application Number
- CN202410232003.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-01
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing technologies for network audio and video detection only perform detection from a visual perspective, resulting in poor efficiency and effectiveness.
We adopt an AI governance approach that coordinates the left and right hemispheres of the brain, and perform collaborative detection from both visual and textual perspectives. This involves using deep learning models for audio and video recognition and deep learning models for text generation for collaborative detection.
It improves the efficiency and effectiveness of online audio and video detection, enabling more accurate identification and location of inappropriate content, and providing text interpretation and modification suggestions for producers.
Smart Images

Figure CN118939819A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a left-brain-and-right-brain collaborative artificial intelligence method for managing network audio and video. Background Art
[0002] Inappropriate content in online audio and video needs to be detected intelligently.
[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art: the prior art often only detects audio and video from a visual perspective.
[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention
[0005] Based on this, it is necessary to provide an artificial intelligence method for managing network audio and video that collaborates between the left and right brains to address the defects or shortcomings of existing technologies, and to perform collaborative detection from both visual and text perspectives to improve the efficiency and effectiveness of network audio and video detection.
[0006] In a first aspect, an embodiment of the present invention provides an artificial intelligence governance method, the method comprising:
[0007] Bad annotation interpretation step: input the audio and video clip into the audio and video recognition deep learning model, and calculate the output as the bad content annotation of the audio and video clip; for each bad content annotation of the audio and video clip, obtain the audio and video slice corresponding to the said each bad content annotation, input the audio and video slice text to generate the output calculated by the deep learning model as the text corresponding to the audio and video slice; use the text corresponding to the audio and video slice as input, and use the output calculated by the text bad content recognition deep learning model as the bad content existing in the text corresponding to the audio and video slice, and use the bad content existing in the text corresponding to the audio and video slice as the text meaning of each bad content annotation of the audio and video clip.
[0008] Preferably, the method further comprises:
[0009] Steps for managing bad audio and video types: sending each bad content label and its textual meaning of the audio and video clip to the producer of the audio and video clip, and sending information to remind the producer to delete or modify the audio and video slice corresponding to each bad content label.
[0010] Preferably, the method further comprises:
[0011] The first step of detecting the inappropriate content in the text corresponding to the audio and video clip is: inputting the audio and video clip as the audio and video slice into the audio and video slice text generation deep learning model, and calculating the output as the text corresponding to the audio and video clip, and inputting the text into the text inappropriate content recognition deep learning model, and calculating the output as the inappropriate content of the text corresponding to the audio and video clip.
[0012] Preferably, the method further comprises:
[0013] The second detection step of the inappropriate content in the text corresponding to the audio and video clips: obtain the keyword set corresponding to the inappropriate type to be detected from the knowledge base, search for each keyword in the keyword set from the text corresponding to the audio and video clip, and extract each sentence containing any keyword in the keyword set from the text corresponding to the audio and video clip; if each sentence is not marked as inappropriate content in the text corresponding to the audio and video clip, send the sentence to the auditor for review; if the review result is that each sentence belongs to inappropriate content, add each sentence and its location and the inappropriate type to which it belongs to the inappropriate content mark in the text corresponding to the audio and video clip.
[0014] In a second aspect, an embodiment of the present invention provides an artificial intelligence governance system, the system comprising:
[0015] Bad content annotation interpretation module: input the audio and video clip into the audio and video recognition deep learning model, and calculate the output as the bad content annotation of the audio and video clip; for each bad content annotation of the audio and video clip, obtain the audio and video slice corresponding to the said each bad content annotation, input the audio and video slice text to generate the output calculated by the deep learning model as the text corresponding to the audio and video slice; use the text corresponding to the audio and video slice as input, and use the output calculated by the text bad content recognition deep learning model as the bad content existing in the text corresponding to the audio and video slice, and use the bad content existing in the text corresponding to the audio and video slice as the text meaning of each bad content annotation of the audio and video clip.
[0016] Preferably, the system further comprises:
[0017] Audio and video bad type management module: each bad content label and its textual meaning of the audio and video clip are sent to the producer of the audio and video clip, and information is sent to remind the producer to delete or modify the audio and video slice corresponding to each bad content label.
[0018] Preferably, the system further comprises:
[0019] The first detection module for bad text content corresponding to audio and video clips: the audio and video clips are input as audio and video slices into the audio and video slice text generation deep learning model, the calculated output is used as the text corresponding to the audio and video clips, the text is input into the text bad content recognition deep learning model, and the calculated output is used as the bad content of the text corresponding to the audio and video clips.
[0020] Preferably, the system further comprises:
[0021] The second detection module for bad content in the text corresponding to the audio and video clips: obtain the keyword set corresponding to the bad type to be detected from the knowledge base, search for each keyword in the keyword set from the text corresponding to the audio and video clip, and extract each sentence containing any keyword in the keyword set from the text corresponding to the audio and video clip; if each sentence is not marked as bad content in the text corresponding to the audio and video clip, send the sentence to the auditor for review; if the review result is that each sentence belongs to bad content, add each sentence and its location and bad type to the bad content mark in the text corresponding to the audio and video clip.
[0022] In a third aspect, an embodiment of the present invention provides an artificial intelligence device, which includes a module of the system described in any one of the embodiments of the second aspect.
[0023] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method described in any one of the embodiments of the first aspect.
[0024] In a fifth aspect, an embodiment of the present invention provides a robot, comprising a memory, a processor, and an artificial intelligence robot program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the method described in the first aspect when executing the program.
[0025] The present embodiment provides an artificial intelligence method for managing network audio and video with the collaboration of the left and right brains. It manages network audio and video through artificial intelligence based on the idea of left and right brain collaboration. It quickly detects inappropriate content directly from audio and video through an artificial intelligence model, which is equivalent to the intuitive function of the right brain. It converts audio and video into text and combines it with knowledge of inappropriate types to detect the annotation of inappropriate content in the text, which is equivalent to the analysis function of the left brain. By combining the left and right brains, it can improve both the speed and accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 A flowchart of an artificial intelligence governance method provided for one embodiment of the present invention;
[0027] Figure 2A framework diagram of an artificial intelligence governance system provided for one embodiment of the present invention. DETAILED DESCRIPTION
[0028] The technical solutions in the embodiments of the present invention are described in detail below in conjunction with the embodiments of the present invention.
[0029] Basic Embodiments of the Invention
[0030] First, as Figure 1 As shown, an embodiment of the present invention provides an artificial intelligence governance method, the method comprising a bad annotation definition step. In a preferred embodiment, the method further comprises a bad audio and video type governance step. In a preferred embodiment, the method further comprises a first detection step of bad text content corresponding to the audio and video clip. In a preferred embodiment, the method further comprises a second detection step of bad text content corresponding to the audio and video clip.
[0031] Second, as Figure 2 As shown, an embodiment of the present invention provides an artificial intelligence governance system, the system comprising a bad annotation definition module. In a preferred embodiment, the system also comprises an audio and video bad type governance module. In a preferred embodiment, the system also comprises a first detection module for bad text content corresponding to audio and video clips. In a preferred embodiment, the system also comprises a second detection module for bad text content corresponding to audio and video clips.
[0032] Preferred embodiments of the present invention
[0033] Identifying online audio and video is like clearing mines. Mines are buried underground and are hidden. Bad audio and video may occasionally appear among legal and compliant audio and video, making it difficult to guard against. To accurately detect and locate bad audio and video, coarse-grained label-based identification often results in misjudgment and missed judgments.
[0034] 1. Training and testing steps of the audio and video recognition deep learning model: take the audio and video clips as input (the audio and video clips are generally formed by slicing the audio and video files or intercepting the audio and video streams), annotate the bad content of the audio and video clips (the bad content annotation of the audio and video clips includes the bad content slices, the locations of the bad content slices, and the bad types of the bad content slices, for example, circle the pornographic parts in the clips and label them "pornographic") as the expected output, train and test the deep learning model to obtain the audio and video recognition deep learning model;
[0035] 2. Training and testing steps of the deep learning model for generating text from audio and video slices: In the training and testing phase, the audio and video slices are taken as input, and the text corresponding to the audio and video slices (audio converted to text, or video subtitles converted to text) is taken as the expected output. The deep learning model is trained and tested to obtain the deep learning model for generating text from audio and video slices.
[0036] 3. Training and testing steps of the deep learning model for identifying bad content in text: In the training and testing phase, the text is used as input, and the bad content annotations in the text (the bad content annotations in the text include bad content sentences, the locations of the bad content sentences, and the bad types of the bad content sentences) are used as expected outputs. The deep learning model is trained and tested to obtain a deep learning model for identifying bad content in text;
[0037] Technical effect: The audio and video recognition deep learning model can only recognize fixed and limited types of labels, because at this time the deep learning model recognizes based on its own impression, that is, it uses the right brain to recognize, and at this time the deep learning model does not understand the semantics of the audio and video. If the left brain recognizes the audio and video, it uses the text-based content recognition deep learning model to recognize the text semantics of the audio and video.
[0038] 4. Bad content annotation interpretation step: input the audio and video clip into the audio and video recognition deep learning model, and use the calculated output as the bad content annotation of the audio and video clip; for each bad content annotation of the audio and video clip, obtain the audio and video slice corresponding to the said each bad content annotation, input the audio and video slice text to generate the output calculated by the deep learning model as the text corresponding to the audio and video slice; use the text corresponding to the audio and video slice as input, and use the output calculated by the text bad content recognition deep learning model as the bad content existing in the text corresponding to the audio and video slice, and use the bad content existing in the text corresponding to the audio and video slice as the text meaning of each bad content annotation of the audio and video clip; Technical effect: by interpreting each bad content annotation of the audio and video clip in a textual manner, the producer of the audio and video clip can understand the review results of the artificial intelligence model on bad content.
[0039] After step 4,
[0040] 5. Steps for managing bad audio and video types: Send each bad content label and its textual meaning of the audio and video clip to the producer of the audio and video clip, and send information to remind the producer to delete or modify the audio and video slice corresponding to each bad content label; Technical effect: When bad audio and video clips are found, two methods can be used. One is to directly delete the audio and video clip. If the bad content is relatively serious, for example, the label identified is political or violent, etc., then this method can be used to directly delete the entire audio and video clip to eliminate the hidden danger. The other is to directly delete the audio and video clip. If the bad content is relatively serious, then this method can be used to directly delete the entire audio and video clip to eliminate the hidden danger. If the bad content is not serious, for example, the label identified is mild pornography, etc., then there is no need to delete the entire audio and video, and the bad content can be blocked by mosaic. At this time, it is necessary to locate the specific location of the bad content in the audio and video clip, and delete, modify or replace the audio and video content at that location.
[0041] After step 4,
[0042] 6. The first step of detecting the inappropriate content of the text corresponding to the audio and video clip is as follows: the audio and video clip is input into the audio and video slice text generation deep learning model, the calculated output is used as the text corresponding to the audio and video clip, the text is input into the text inappropriate content recognition deep learning model, and the calculated output is used as the inappropriate content of the text corresponding to the audio and video clip;
[0043] 7 The second detection step of the bad content of the text corresponding to the audio and video clip: obtain the keyword set corresponding to the bad type to be detected from the knowledge base, search each keyword in the keyword set from the text corresponding to the audio and video clip, and extract each sentence containing any keyword in the keyword set from the text corresponding to the audio and video clip; if each sentence is not in the bad content annotation in the text corresponding to the audio and video clip, send the sentence to the auditor for review, and if the review result is that each sentence belongs to bad content, then each sentence and its location and bad type are added to the bad content annotation in the text corresponding to the audio and video clip; Technical effect: Taking the audio and video text as input and the bad text in the audio and video clip as output is actually limited, because if some bad text does not exist in the training data set, it is difficult to identify it at this stage. Deep learning is mainly based on historical experience data, which lacks common sense knowledge, so it needs to be supplemented with keyword knowledge of the type to be detected. This solution is not only an innovation in audio and video recognition technology, but also an innovation in deep learning technology. The second detection step can make full use of the knowledge of the keyword set corresponding to the bad type to be detected, overcoming the defect that the first detection step can only rely on deep learning model learning experience.
[0044] After step 7,
[0045] 8. Incremental training steps for the deep learning model for identifying bad text content: taking the text corresponding to the audio and video clip as input, taking the bad text obtained by the second detection step of the bad text corresponding to the audio and video clip as the expected output, and performing incremental training on the deep learning model for identifying bad text content; Technical effect: After adding the bad sentences detected by the bad type knowledge (keyword set) to the bad text, the deep learning model for identifying bad text content is incrementally trained, so that the model can learn knowledge that it did not originally have.
[0046] After step 4,
[0047] 9. Training and testing steps of the audio and video modification deep learning model: taking the audio and video clips and the annotations of the bad content of the audio and video clips as input, taking the modified audio and video clips without bad content as the expected output, training and testing the deep learning model to obtain the audio and video modification deep learning model;
[0048] 10. Audio and video clip correction step: taking the audio and video clip and the bad content annotation of the audio and video clip as input, and using the output calculated by the audio and video modification deep learning model as the modified audio and video clip without bad content; Technical effect: realizing the modification of audio and video through the deep learning model;
[0049] After step 4,
[0050] 11. Training and testing steps of the deep learning model for modifying bad content in text: In the training and testing phase, the text and the bad content annotation in the text are taken as input, and the modified text without bad content is taken as the expected output. The deep learning model is trained and tested to obtain the deep learning model for modifying bad content in text;
[0051] 12. The first correction step of the text corresponding to the audio and video clips: taking the text corresponding to the audio and video clips and the annotation of the inappropriate content in the text as input, and using the output calculated by the deep learning model to modify the inappropriate content in the text as the modified text; Technical effect: The corresponding text of the audio and video is modified through the deep learning model to eliminate the inappropriate content.
[0052] 13 The second correction step of the text corresponding to the audio and video clips: obtain the keyword set corresponding to the bad type to be modified from the knowledge base, search for each keyword in the keyword set from the modified text, and extract each sentence containing any keyword in the keyword set from the modified text; send each sentence to the auditor for review. If the review result is that each sentence belongs to bad content, send each sentence to the expert for modification to obtain the modified text; Technical effect: The second correction step supplements the results of the deep learning model detection through bad type keywords, which can improve the modification effect.
[0053] 14. Incremental training steps for the deep learning model for correcting text with bad content: taking the text corresponding to the audio and video clip as input, taking the modified text obtained by the second correction step of the text corresponding to the audio and video clip as the expected output, and performing incremental training on the deep learning model for correcting text with bad content; Technical effect: performing incremental training on the deep learning model for correcting text with bad content using the text that has been further modified by the knowledge of bad types (keyword set), can improve the modification ability of the model.
[0054] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.
Claims
1. An artificial intelligence governance method, characterized in that: The method comprises: Bad annotation interpretation step: input the audio and video clip into the audio and video recognition deep learning model, and calculate the output as the bad content annotation of the audio and video clip; for each bad content annotation of the audio and video clip, obtain the audio and video slice corresponding to the said each bad content annotation, input the audio and video slice text to generate the output calculated by the deep learning model as the text corresponding to the audio and video slice; use the text corresponding to the audio and video slice as input, and use the output calculated by the text bad content recognition deep learning model as the bad content existing in the text corresponding to the audio and video slice, and use the bad content existing in the text corresponding to the audio and video slice as the text meaning of each bad content annotation of the audio and video clip.
2. The artificial intelligence governance method according to claim 1, characterized in that: The method further comprises: Steps for managing bad audio and video types: sending each bad content label and its textual meaning of the audio and video clip to the producer of the audio and video clip, and sending information to remind the producer to delete or modify the audio and video slice corresponding to each bad content label.
3. The artificial intelligence governance method according to claim 1, characterized in that: The method further comprises: The first step of detecting the inappropriate content in the text corresponding to the audio and video clip is: inputting the audio and video clip as the audio and video slice into the audio and video slice text generation deep learning model, and calculating the output as the text corresponding to the audio and video clip, and inputting the text into the text inappropriate content recognition deep learning model, and calculating the output as the inappropriate content of the text corresponding to the audio and video clip.
4. The artificial intelligence governance method according to claim 1, characterized in that: The method further comprises: The second detection step of the inappropriate content in the text corresponding to the audio and video clips: obtain the keyword set corresponding to the inappropriate type to be detected from the knowledge base, search for each keyword in the keyword set from the text corresponding to the audio and video clip, and extract each sentence containing any keyword in the keyword set from the text corresponding to the audio and video clip; if each sentence is not marked as inappropriate content in the text corresponding to the audio and video clip, send the sentence to the auditor for review; if the review result is that each sentence belongs to inappropriate content, add each sentence and its location and the inappropriate type to which it belongs to the inappropriate content mark in the text corresponding to the audio and video clip.
5. An artificial intelligence governance system, characterized in that: The system comprises: Bad content annotation interpretation module: input the audio and video clip into the audio and video recognition deep learning model, and calculate the output as the bad content annotation of the audio and video clip; for each bad content annotation of the audio and video clip, obtain the audio and video slice corresponding to the said each bad content annotation, input the audio and video slice text to generate the output calculated by the deep learning model as the text corresponding to the audio and video slice; use the text corresponding to the audio and video slice as input, and use the output calculated by the text bad content recognition deep learning model as the bad content existing in the text corresponding to the audio and video slice, and use the bad content existing in the text corresponding to the audio and video slice as the text meaning of each bad content annotation of the audio and video clip.
6. The artificial intelligence governance system according to claim 5, characterized in that: The system further comprises: Audio and video bad type management module: each bad content label and its textual meaning of the audio and video clip are sent to the producer of the audio and video clip, and information is sent to remind the producer to delete or modify the audio and video slice corresponding to each bad content label.
7. The artificial intelligence governance system according to claim 5, characterized in that: The system further comprises: The first detection module for bad content in the text corresponding to the audio and video clip: the audio and video clip is input as the audio and video slice into the audio and video slice text generation deep learning model, the calculated output is used as the text corresponding to the audio and video clip, the text is input into the text bad content recognition deep learning model, and the calculated output is used as the bad content of the text corresponding to the audio and video clip.
8. The artificial intelligence governance system according to claim 5, characterized in that: The system further comprises: The second detection module for bad content in the text corresponding to the audio and video clips: obtain the keyword set corresponding to the bad type to be detected from the knowledge base, search for each keyword in the keyword set from the text corresponding to the audio and video clip, and extract each sentence containing any keyword in the keyword set from the text corresponding to the audio and video clip; if each sentence is not marked as bad content in the text corresponding to the audio and video clip, send the sentence to the auditor for review; if the review result is that each sentence belongs to bad content, add each sentence and its location and bad type to the bad content mark in the text corresponding to the audio and video clip.
9. A robot comprising a memory, a processor, and an artificial intelligence robot program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 4 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.
