Video comment auditing method and device, computer equipment and storage medium

Abstract

The embodiment of the invention provides a video comment auditing method and device, computer equipment and a storage medium. The method comprises the steps that a target video and comment data corresponding to the target video are acquired; marking the comment data corresponding to the target video based on the relevance between the target video and the comment data corresponding to the target video, and constructing a comment data set corresponding to the target video; taking the target video and the comment data set corresponding to the target video as a training set, and performing fine tuning training on a preset multi-modal model to obtain a trained first multi-modal model; the method comprises the steps of receiving a specified video and comment data corresponding to the specified video, inputting the specified video and the comment data corresponding to the specified video into a first multi-modal model, and obtaining an audit result of the comment data of the specified video. And the auditing purpose that the trained first multi-modal model is used for identifying the implicit information of the video comments is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of radio and television media content review, and specifically, to a video comment review method, apparatus, computer equipment and storage medium. Background Art

[0002] In the process of dissemination of information such as text, sound, images and videos in the field of audio-visual media, content review can help filter out illegal, malicious and other bad content, and ensure that the disseminated content is legal and compliant and has correct value orientation.

[0003] At present, content review is mainly a combination of manual content review and intelligent content review. In the field of intelligent review, different intelligent review classification methods are used according to different data types. In the text field, natural processing technology is used to automatically analyze and understand text; in the image field, computer vision technology is used to automatically identify and filter illegal images by using image segmentation, target detection, etc.; in the audio field, audio is converted into text for identification; in the video field, video is decomposed into video frames and audio for review.

[0004] In the related technology, a semantic recognition method and system for automatic review of text and pictures by AI is provided. The system receives text data and picture data to be published, cleans all the data, performs semantic understanding and sentiment analysis on the text data, and performs image processing on the picture data, including feature extraction, target detection and semantic segmentation, and finally performs content review on the processed data. A multimodal data fusion classification method based on a large model and an attention mechanism is also provided. The method uses a cross-attention fusion model to fuse image and text data, and directly fuses different image features, which not only fuses the feature information between different images, but also avoids the risk of overfitting caused by excessive fusion, reduces information redundancy and noise, and can better balance the text modality and image modality, and improve the accuracy of the classification results.

[0005] However, related technologies all use natural language processing technology for text analysis and computer vision technology for image analysis, which cannot directly identify the negative information content of multimodal data.

[0006] At present, with the development of the Internet, people's way of obtaining information is becoming more and more popular. Short videos and other content carriers have become the main body of people's discussion and sharing of life. The emergence of online comments allows users to discuss any topic and share their views on video content. At the same time, Internet slang has become an interesting way for young people to express their views on the Internet.

[0007] The implicit information in video comments refers to the emotions, attitudes and opinions that are not explicitly expressed in the comments. In the scenario where videos and comments are combined, these comments usually do not have sarcastic meanings as words themselves, but they will have sarcastic meanings when combined with the image or video context, which will lead to the spread of negative energy. Therefore, it is necessary to review the implicit meaning of such comments in advance to avoid the risk of public opinion dissemination.

[0008] In the related technologies, there is no method directly used to review the sarcastic meaning of video comments. In academia, sentiment analysis technology is usually used to detect the implicit information of comments. At present, there are mainly three methods: dictionary-based technology, machine learning-based technology, and hybrid methods. Dictionary-based technology generally uses statistical analysis, such as nearest neighbors and conditional random fields, which are easier to detect when emotional words are explicitly expressed; machine learning-based technology is divided into traditional machine learning and deep learning technology, which can understand lexical features using recurrent neural networks, etc. These methods are usually converted to process document and sentence-level classification problems; hybrid methods are to mix dictionary-based and deep learning methods to enhance the accuracy of text sentiment analysis. At present, sentiment analysis mainly analyzes the sentiment intensity and polarity of the text itself, and there is no implicit information review of text comments combined with pictures and comments.

[0009] In the related art, a multi-dimensional comment review method is provided, which reviews comments from multiple dimensions, wherein repetition is used to detect the novelty of the comments to be reviewed; text richness is used to detect whether the content of the comments to be reviewed is single; emotion recognition is used to detect the emotional positivity of the comments to be reviewed; and timeliness is used to measure the timeliness of the release of the comments to be reviewed; thus, the comments are reviewed from the above four dimensions, and the quality of the comments is quantified, thereby ensuring the stability and quality of the quality of the comments. A news comment review method is also provided, which includes obtaining comments initiated by the user end, identifying the text and pictures in the comments; extracting the text and elements in the pictures, identifying and judging whether the elements contain illegal elements through matching, and if so, removing the pictures; performing semantic monitoring on the text and picture text, and if the monitoring result is a sensitive comment, a flooding comment or an extreme comment, the comment is judged to be an illegal comment, the comment is removed, and the comment is recorded in the database; if it is impossible to judge whether the comment is an illegal comment, the user situation of the commenting user is obtained, and the comment is further judged according to the user situation to determine whether the comment is illegal. This method has the effect of increasing the review efficiency and reducing the passing rate of improper comments.

[0010] Therefore, in the related technology, the review is mainly conducted on text and images, and there is no method for reviewing videos. In addition, it is impossible to judge the implicit meaning of the comments, that is, the comments themselves are risk-free, and whether the comments generate content risks in the video scenario. Summary of the invention

[0011] The embodiments of the present application provide a video comment review method, apparatus, computer equipment, and storage medium.

[0012] A first aspect of an embodiment of the present application provides a video comment review method, comprising:

[0013] Get the target video and its corresponding comment data;

[0014] Based on the correlation between the target video and its corresponding comment data, the comment data corresponding to the target video is annotated to construct a comment dataset corresponding to the target video;

[0015] The target video and its corresponding comment dataset are used as training sets, and the preset multimodal model is fine-tuned to obtain a trained first multimodal model;

[0016] A designated video and its corresponding comment data are received, and the designated video and its corresponding comment data are input into a first multimodal model to obtain a review result of the comment data of the designated video, and the comment data whose review result is a negative comment is processed.

[0017] In an optional embodiment of the present application, the tagging of the comment data corresponding to the target video based on the correlation between the target video and the comment data corresponding to the target video includes:

[0018] Extract key information and sentiment information from the comment data corresponding to the target video, and preliminarily determine whether the comment data is a negative comment based on the sentiment information;

[0019] If it is preliminarily determined that the comment data is a negative comment, marking the comment data as a negative comment;

[0020] In the case where it is preliminarily determined that the comment data is a positive comment, further determining whether the comment data is a negative comment based on the correlation between the target video and its corresponding comment data;

[0021] If it is further determined that the comment data is a negative comment, marking the comment data as a negative comment;

[0022] When it is further determined that the comment data is a positive comment, the comment data is marked as a positive comment.

[0023] In an optional embodiment of the present application, further determining whether the comment data is a negative comment based on the correlation between the target video and its corresponding comment data includes:

[0024] Extract key information and sentiment information from the target video, and calculate the similarity between the key information and sentiment information in the target video and its corresponding comment data;

[0025] Determine the degree of conflict between the target video and the text content in the corresponding comment data;

[0026] Determine the conflict score between the target video and its corresponding comment data according to the similarity between the key information and emotional information in the target video and its corresponding comment data and the degree of conflict between the text contents;

[0027] The review data whose conflict score exceeds a preset conflict score threshold is determined as a negative review.

[0028] In an optional embodiment of the present application, the similarity between the key information and the sentiment information in the target video and its corresponding comment data is calculated by the following expression:

[0029]

[0030] Among them, A is the first text vector composed of key information and emotional information in the target video, B is the second text vector composed of key information and emotional information in the comment data corresponding to the target video, a i is the i-th vector element in the first text vector, b i is the i-th vector element in the second text vector, and w1 and w2 are both weight factors.

[0031] In an optional embodiment of the present application, constructing a comment dataset corresponding to the target video includes:

[0032] Each comment data corresponding to the target video and the annotation information of each comment data are collected into a comment data set corresponding to the target video.

[0033] In an optional embodiment of the present application, the target video and its corresponding comment data set are used as training sets to fine-tune the preset multimodal model to obtain a trained first multimodal model, including:

[0034] The video frame of the target video and each comment data are taken as input, and the annotation information of the current comment data is taken as output. The network structure used to process the video frames in the preset multimodal model is trained using the lora fine-tuning method of supervised tasks, and the network structure used to process the comment data is trained using the qlora fine-tuning method of supervised tasks to obtain the first multimodal model after training.

[0035] In an optional embodiment of the present application, inputting the specified video and its corresponding comment data into the first multimodal model to obtain the review result of the comment data of the specified video includes:

[0036] Inputting a video frame of a specified video into a network structure for processing the video frame in a first multimodal model to obtain a video frame feature;

[0037] Inputting comment data corresponding to the specified video into a network structure for processing comment data in the first multimodal model to obtain comment data features;

[0038] The video frame features and the comment data features are input into the fusion network structure in the first multimodal model to obtain the review result of the comment data of the designated video.

[0039] A second aspect of the embodiment of the present application provides a video comment review device, comprising:

[0040] An acquisition module is used to acquire a target video and its corresponding comment data;

[0041] A construction module is used to annotate the comment data corresponding to the target video based on the correlation between the target video and the corresponding comment data, and to construct a comment data set corresponding to the target video;

[0042] A training module, used to use the target video and its corresponding comment data set as a training set, to perform fine-tuning training on the preset multimodal model, and obtain a trained first multimodal model;

[0043] The input module is used to receive a specified video and its corresponding comment data, and input the specified video and its corresponding comment data into the first multimodal model to obtain the review results of the comment data of the specified video, and process the comment data whose review results are negative comments.

[0044] According to a third aspect of an embodiment of the present application, a computer device is provided, including: a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0045] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above video comment review methods.

[0046] The above technical solution provided by the embodiment of the present application has at least some or all of the following advantages compared with the prior art:

[0047] The video comment review method described in the embodiment of the present application obtains a target video and its corresponding comment data; based on the correlation between the target video and its corresponding comment data, the comment data corresponding to the target video is annotated to construct a comment data set corresponding to the target video; the target video and its corresponding comment data set are used as a training set to fine-tune the preset multimodal model to obtain a trained first multimodal model; a specified video and its corresponding comment data are received, and the specified video and its corresponding comment data are input into the first multimodal model to obtain the review result of the comment data of the specified video, and the comment data with a negative review result is processed. The present application understands the video frame of the target video and annotates the comments of the target video, and fine-tunes the training of the preset multimodal model in combination with the video frame, its comments and the annotations of the comments, so as to achieve the review purpose of using the trained first multimodal model to identify implicit information in video comments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0049] Figure 1 A flowchart of a video review review method provided for one embodiment of the present application;

[0050] Figure 2 A flowchart of a video comment review method provided for another embodiment of the present application;

[0051] Figure 3 A detailed flow chart of the steps of annotating video comment data provided by one embodiment of the present application;

[0052] Figure 4 A flowchart of a content review method provided for one embodiment of the present application;

[0053] Figure 5 A schematic diagram of a content review method based on a large model provided for an embodiment of the present application;

[0054] Figure 6 A schematic diagram of a text content review method provided for one embodiment of the present application;

[0055] Figure 7 A schematic diagram of an image scene content review method provided by an embodiment of the present application;

[0056] Figure 8 A schematic diagram of the structure of a video comment review device provided for one embodiment of the present application;

[0057] Fig. 9A schematic diagram of the computer device structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0059] See also Figure 1 and Figure 2 The video review method provided in the embodiment of the present application includes the following steps 100 to 400:

[0060] Step 100: Obtain the target video and its corresponding comment data;

[0061] Step 200: Based on the correlation between the target video and its corresponding comment data, the comment data corresponding to the target video is annotated to construct a comment data set corresponding to the target video;

[0062] Step 300: taking the target video and its corresponding comment data set as a training set, fine-tuning the preset multimodal model to obtain a trained first multimodal model;

[0063] Step 400: Receive a designated video and its corresponding comment data, and input the designated video and its corresponding comment data into a first multimodal model to obtain a review result of the comment data of the designated video, and process the comment data whose review result is a negative comment.

[0064] In an optional embodiment of the present application, see Figure 3 In step 200, the comment data corresponding to the target video is labeled based on the correlation between the target video and the comment data corresponding to the target video, including:

[0065] Extract key information and sentiment information from the comment data corresponding to the target video, and preliminarily determine whether the comment data is a negative comment based on the sentiment information;

[0066] If it is preliminarily determined that the comment data is a negative comment, marking the comment data as a negative comment;

[0067] In the case where it is preliminarily determined that the comment data is a positive comment, further determining whether the comment data is a negative comment based on the correlation between the target video and its corresponding comment data;

[0068] If it is further determined that the comment data is a negative comment, marking the comment data as a negative comment;

[0069] When it is further determined that the comment data is a positive comment, the comment data is marked as a positive comment.

[0070] In an optional embodiment of the present application, extracting key information and sentiment information from comment data corresponding to a target video may be achieved through a large multimodal model or a pre-trained second multimodal model.

[0071] In the present application, when it is preliminarily determined that the comment data is a positive comment, it is further determined whether the comment data is a negative comment based on the correlation between the target video and its corresponding comment data, so that implicit information in the comment data can be identified.

[0072] In an optional embodiment of the present application, further determining whether the comment data is a negative comment based on the correlation between the target video and its corresponding comment data includes:

[0073] Extract key information and sentiment information from the target video, and calculate the similarity between the key information and sentiment information in the target video and its corresponding comment data;

[0074] Determine the degree of conflict between the target video and the text content in the corresponding comment data;

[0075] Determine the conflict score between the target video and its corresponding comment data according to the similarity between the key information and emotional information in the target video and its corresponding comment data and the degree of conflict between the text contents;

[0076] The review data whose conflict score exceeds a preset conflict score threshold is determined as a negative review.

[0077] In an optional embodiment of the present application, extracting key information and emotional information from the target video can be achieved by a multimodal large model or a pre-trained second multimodal model. For example, a transport truck loaded with cars on the left side of the video overturned, causing traffic interruption. Key information includes: snowy weather, highway, vehicle queues, traffic accidents, and emotional information or emotional color is negative.

[0078] In an optional embodiment of the present application, the similarity between the key information and the emotional information in the target video and its corresponding comment data is calculated using the cosine similarity and Euclidean distance (normalization required) algorithms.

[0079] In an optional embodiment of the present application, the similarity between the key information and the sentiment information in the target video and its corresponding comment data is calculated by the following expression:

[0080]

[0081] Among them, A is the first text vector composed of key information and emotional information in the target video, B is the second text vector composed of key information and emotional information in the comment data corresponding to the target video, a i is the i-th vector element in the first text vector, b i is the i-th vector element in the second text vector, and w1 and w2 are both weight factors.

[0082] In an optional embodiment of the present application, the degree of conflict between the target video and the text content in the corresponding comment data is determined, and the natural language processing Bert model is used to implement it, and logical consistency detection and semantic analysis are used to perform conflict detection, and a score of the degree of conflict is given. No conflict at all is 0, and complete opposition is 1. For example, the dialectical unity of economic development and environmental protection conflicts with the view that economic development is more important, and the degree of conflict is about 0.5; advocating "hard work and emphasis on reading" conflicts with "reading is not the focus", and the degree of conflict is about 0.9.

[0083] In an optional embodiment of the present application, the conflict score between the target video and its corresponding comment data is determined by using a weighted score, which assigns weights to the similarity and the degree of conflict respectively, and calculates a comprehensive conflict score.

[0084] In this application, since the implicit information is caused by the conflict between the comment content and the video frame content, the extracted video frame content and the comment meaning content are compared. If they are completely contradictory or conflicting, they are negatively labeled according to the text content, and thus a negatively labeled graphic data set is obtained, namely: the image features of the video are extracted using a video understanding algorithm; the comments are analyzed using a text analysis method to obtain text features; and the two features are aligned in feature space, and the feature similarity is calculated to evaluate the consistency or opposition of the features; then the rule model and a text analysis model similar to BERT are applied for conflict detection, to identify opposing viewpoints and give a conflict degree score; the weighted scoring method is used to evaluate the conflict level based on the similarity and conflict degree, and it is divided into three levels: low, medium and high. For comments with medium and high conflicts, it is believed that there may be implied irony, so negative labeling is performed. Finally, it is reviewed by professional auditors to ensure data accuracy and quality. After the labeled data set is completed, it can be used for testing and training of large models for video comment detection scenarios. After repeated verification, the reasoning of the large model is used to determine whether the new data comments have implicit meanings, thereby achieving the purpose of reviewing the content of the comments.

[0085] In an optional embodiment of the present application, in step 200, constructing a comment dataset corresponding to the target video includes:

[0086] Each comment data corresponding to the target video and the annotation information of each comment data are collected into a comment data set corresponding to the target video.

[0087] In an optional embodiment of the present application, in step 300, the target video and its corresponding comment data set are used as training sets to fine-tune the preset multimodal model to obtain a trained first multimodal model, including:

[0088] The video frame of the target video and each comment data are taken as input, and the annotation information of the current comment data is taken as output. The network structure used to process the video frames in the preset multimodal model is trained using the lora fine-tuning method of supervised tasks, and the network structure used to process the comment data is trained using the qlora fine-tuning method of supervised tasks to obtain the first multimodal model after training.

[0089] In an optional embodiment of the present application, for text data such as comment data, supervised task training fine-tuning is performed using the Qlora fine-tuning method. The experiment is completed on an 8-card NVIDIA HGX H800 GPU on the Ubuntu 22.04 operating system. All experiments use the Pytorch framework. The batch size is set to 4 during training, and a total of 20 training cycles are performed. The number of gradient updates for each batch is 1, and the learning rate is set to 2e -4 , the learning rate is constant with warm up strategy, that is, the learning rate is 0 at the beginning, and then gradually increases to the learning rate setting value during the warm-up phase, and the learning rate setting value is used during formal learning. The maximum length sequence length accepted by the model is 128. For video frame data, the training and fine-tuning of supervised tasks are carried out using lora fine-tuning. The experiment was completed on 8 NVIDIA HGX H800 GPUs on the Ubuntu22.04 operating system. All experiments used the Pytorch framework. The batch size was set to 4 during training, and a total of 20 training cycles were conducted. The learning rate was set to 0.00001, and the learning rate was the Cosine Annealing LR strategy. The learning rate is adjusted based on the curve shape of the cosine function. When the training starts, the learning rate is large, which can help the model converge quickly. As the training progresses, the learning rate will gradually decrease to ensure that the model can accurately search the parameter space. The model accepts input image resolution of 1024×1024.

[0090] In an optional embodiment of the present application, in step 400, the step of inputting the specified video and its corresponding comment data into the first multimodal model to obtain the review result of the comment data of the specified video includes:

[0091] Inputting a video frame of a specified video into a network structure for processing the video frame in a first multimodal model to obtain a video frame feature;

[0092] Inputting comment data corresponding to the specified video into a network structure for processing comment data in the first multimodal model to obtain comment data features;

[0093] The video frame features and the comment data features are input into the fusion network structure in the first multimodal model to obtain the review result of the comment data of the designated video.

[0094] In an optional embodiment of the present application, the preset multimodal model is a pre-trained second multimodal model or a multimodal large model. In the case where the preset multimodal model is a pre-trained second multimodal model, the target video and its corresponding comment data set are used as a training set, and the pre-trained second multimodal model is fine-tuned to obtain a trained first multimodal model, wherein the training and use process of the pre-trained second multimodal model is as follows: Figure 4 See Figure 4 The training and use process of the pre-trained second multimodal model includes the following steps: the first step is to construct a news image and text data set, and the data set is annotated by labeling the image and text; the second step is to use the multimodal large model to convert the image and text into feature vectors and train to learn the joint distribution characteristics of the image and text vectors; the third step is to retrieve and match the image and text data input by the user, and use the trained multimodal large model for prediction; the fourth step is to review and evaluate the actual image and text content; the fifth step is to establish a feedback mechanism between the evaluation results and the manual review results, and continuously optimize the accuracy of the model evaluation.

[0095] In an optional embodiment of the present application, in the first step, a news image and text data set is constructed, including two parts: news image and text data collection and news image and text data annotation. From the perspective of correct value orientation, social current affairs news and images with positive and negative information are selected as the data source, and a news image and text data set is constructed. The data set is classified into four scenarios according to the scenario: Scenario 1 negative article review scenario, Scenario 2 negative article and comment review scenario, Scenario 3 vilification of character image review scenario, and Scenario 4 bad image and text implication scenario. The news image and text data set constructed for the above scenarios is divided into a news text data set and a news image data set according to text and images, and then data sets are constructed for the four subdivided scenarios for content review model training.

[0096] For text data, we choose the public social and political news titles, news content and news comments from authoritative sources as the main data sources. The main reasons are: 1) Social and political news covers positive and negative events, and public news has the advantage of being easy to obtain. 2) Social news involves social events, social problems, social outlook, etc. in the daily life of the people, and has the characteristics of authenticity, timeliness and accuracy. Modeling is carried out for the positive and negative discrimination task in the Chinese context. The collected news texts include domestic and foreign news, and the JAVA natural language processing toolkit is used to screen out social news with positive and negative meanings, such as THULAC (THU Lexical Analyzer for Chinese, Chinese lexical analysis toolkit), which can accurately perform Chinese word segmentation and part-of-speech tagging, and can realize user-defined text classification tasks. In the content review task, in order for the model to review negative news texts, the sampled news texts are kept as negative as possible. The collected news text data set is used to train the model's ability to distinguish between positive and negative, screen the above collected news, retain news texts related to negative information, and treat texts without negative information as positive news texts. At the same time, texts with incorrect formats are deleted. After data processing, 4519 news titles and news contents were used to construct the news text dataset required for scenario 1. Table 1 shows an example of the dataset for scenario 1. 4627 news titles, contents, and comments were used to construct the news text dataset required for scenario 2. Table 2 shows an example of scenario 2.

[0097] Table 1

[0098]

[0099]

[0100] Table 2

[0101]

[0102]

[0103] For image data, the negative side of images is usually seen in the tampering of public figures’ images, and the use of positive characters in satirical scenes. Therefore, the images selected are mainly about public figures. Since there are more positive images of people published in China than negative images, the automatically collected image data has a sample imbalance problem. In order to solve this problem, we further crawled the main characters’ images and expanded the negative image data.

[0104] The news image dataset constructed above is used to train the model's ability to distinguish between positive and negative. Therefore, the above collected images are screened, and images with positive and negative information and related image comments are retained. At the same time, neutral images that do not contain positive and negative information, images with incorrect formats, etc. are deleted. After data processing, 9243 images are used to construct the news image dataset required for scenario three. Table 3 is an example display of the scenario three dataset. 4252 images and image comments are used to construct the news image dataset required for scenario four. Table 4 is an example display of scenario four.

[0105] Table 3

[0106]

[0107] Table 4

[0108]

[0109]

[0110] In the process of labeling, positive data refers to the positive and positive reports and comments formed on the Internet and social media. The opposite of positive data is negative data, which includes negative comments, malicious criticism and even defamatory reports. The labeling of audit scenarios is mainly aimed at negative data. Therefore, negative data is mainly labeled, and the rest of the data is positive data, which does not include neutral data. Specifically, four scenarios are mainly labeled. For positive scenarios, they are labeled as positive; for negative scenarios, they are labeled as negative, and the reasons for the negative are also labeled. According to the definition of positive and negative, the data in the scenario is labeled as 1 (positive) and 2 (negative and negative reasons).

[0111] In an optional embodiment of the present application, in the second step, the multimodal large model and the news image and text dataset are used to perform fine-tuning training on the four scenarios, learn the joint distribution of image and text vectors, and obtain the content review large model. Figure 5 , mainly using the large language model of ChatGLM (a generative language model) and the multimodal large model of CogVLM (an open source visual language basic model). The input of the model is two major categories: text and image. Scene one is shown in ①, and the text content is input; scene two is shown in ②, and the text and text comments need to be input; scene three is shown in ③, and the image needs to be input; scene four is shown in ④, and the image and the comments about the image need to be input. GLM mainly uses the decoder part of Transformer (a neural network model based on the self-attention mechanism). In natural language understanding and natural language generation tasks, GLM uses the generation principle for reasoning. The main principle is to generate possible fill-in-the-blank content based on the known partial text content. GLM can be used in multiple natural language processing tasks such as automatic text completion, question-answering systems, semantic understanding and generation. Figure 6 As shown in the figure, it is the principle part of text content understanding. After the text content is input, it is embedded into words to form text features. Assuming that a piece of text content t = {t1,...,tn} is given, the original input text is segmented, and multiple texts x = [x 1,...,xn] are selected. Each x represents a continuous text token (the smallest text unit), which represents the word embedding output with semantic features. The GLM pre-training structure is called autoregressive blank filling. The original text token is [x 1,...,xn]. The original text is randomly sampled. Assume that [x3], [x 5,x 6] are the sampled texts. The sampled part of the original text is replaced by [M] as Part A. At the same time, GLM autoregressively generates Part B. Each sampled text is preceded by [S] as input and followed by [E] to represent output. Part A and Part B are added with two-dimensional position encoding, where the position encoding Pos1 represents the position of the original text, and the position encoding Pos2 represents the position of the sampled text itself. After adding the position encoding, it is input into the sub-attention mask part of GLM. Each text in Part A of this part can be noticed, and the text in Part B can be noticed in the text that has been generated in Part A and Part B. Finally, the output text is obtained, and its output text is as close to the original text as possible. GLM understands the meaning of the text through this autoregressive blank filling method. After understanding the meaning of the text, the model can be fine-tuned in combination with specific tasks. The text review scenario is to understand the meaning of the text content and judge whether the detailed text content of the text scene is positive or negative. In the natural language understanding task of GLM fine-tuning, given a labeled example (x, y), the input text x is converted into a cloze problem through a pattern containing a single mask label. Given x, predict the conditional probability of y. Then the positive and negative of the label are mapped to the words good and bad, and the cross entropy loss function is used for back propagation to achieve fine-tuning. In this application, given a piece of text content x, the annotated positive and negative meanings are used as y labels, and each time the model uses an autoregressive blank filling method to generate self-judged positive and negative meanings. The CogVLM model uses the task form of visual question answering and consists of four basic components: the visual encoder ViT (Vision Transformer) encoder, the MLP (Multilayer Perceptron) adapter, the pre-trained large language model LLM and the visual expert module. Figure 7As shown in the figure, the understanding principle of image and question-answer text is shown. Specifically, the text question-answer description of the image is first converted into text features through word embedding; the image is converted into feature representation through the ViT encoder, and then mapped to the same space as the text features through the MLP adapter. The two features are combined through concat (a function for connecting multiple strings) and combined with position encoding to input into the pre-trained large language model structure. The structure is mainly based on the Vicuna-7B model, which is a pre-trained architecture of GPT. Each layer in the structure adds a visual expert module to achieve deep alignment of vision and language, so as to achieve the model's deep understanding of image data. Finally, similar to the text review scenario, the image review scenario also needs to fine-tune the model in combination with specific tasks. In the process of training the news graphic data set with the large model, different questioning methods are adopted as prompt words for obtaining the review results. The questioning methods are as follows: Scenario 1 Questioning method: Please judge whether the following text contains negative remarks. Negative remarks usually involve slanderous or misleading remarks, which cause social dissatisfaction or undermine social stability. If the text contains negative remarks, please return "negative" and give reasons, otherwise return "positive". Scenario 2 Questioning method: Based on the title and content of the text, please judge whether the following comments contain negative remarks. Negative remarks usually involve slanderous or misleading remarks, which may cause social dissatisfaction or undermine social stability. If the article content contains negative information or the comments contain negative remarks, please return "negative" and give a reason, otherwise return "positive". Scenario 3 Questioning method: Does the picture vilify the image of a person? If the picture vilifies the image of a person, please return "negative" and give a reason, otherwise return "positive". Scenario 4 Questioning method: Do the pictures and comments involve sensitive remarks? If the pictures and comments involve sensitive remarks, please return "negative" and give a reason, otherwise return "positive".

[0112] During fine-tuning training, each time the text and image to be detected are input, the system will automatically combine the questioning methods of each scenario and input them to the model to obtain the final audit result. In the text scenario, the GLM model is used to understand the meaning of the text through autoregressive blank filling to determine whether the text content is positive or negative. In the image audit scenario, using the principle of understanding images and texts, in CogVLM, the text question and answer description of the image is first converted into text features through word embedding; the image is converted into feature representation through the ViT encoder, and then mapped to the same space as the text features through the MLP adapter. The two features are combined through concat and combined with position encoding to input into the pre-trained large language model structure. This structure is mainly based on the Vicuna-7B model, which is a pre-trained architecture of GPT. Each layer in the structure adds a visual expert module to achieve deep alignment of vision and language, thereby achieving a deep understanding of the model for image data. Finally, similar to the text audit scenario, the image audit scenario also needs to fine-tune the model in combination with specific tasks.

[0113] In an optional embodiment of the present application, in the third step, the trained big model is used to predict the results. Microservices are established through the big model, and a content review system based on the multimodal big model is formed. The content review system architecture based on the multimodal big model includes hardware, software, database and functional modules. The system hardware is a high-computing-power server. The system database includes a text database and an image database that are uploaded to the system for review. The system functional modules include a system data conversion module, a content review algorithm encapsulation module, a system output module, an unstructured data processing module, a multimodal data integration module, a content analysis module, a content anomaly quantification module and an intelligent control module.

[0114] The data conversion module is used to determine the type of data input by the user into the system, determine the scene category from the input data, and after preprocessing the data, combine it with different questioning methods for each scene, and then transmit it to the multimodal large model for reasoning;

[0115] The unstructured data processing module is used to perform semantic analysis on unstructured data including news graphics, audio, and video using a multimodal large model, generate a semantic association graph, extract the core context and information relevance in text and images through in-depth analysis, and construct a preliminary review data set;

[0116] The multimodal data integration module is used to integrate the graphic data, audio data and video data in a unified context, convert them into a graphic-text association vector space, and transmit them to the content analysis module for in-depth analysis of content context and emotion;

[0117] The content analysis module is used to preliminarily determine the degree of matching between the text and the image content based on the semantic association graph generated by the unstructured data processing module, generate a context factor Qycs, and analyze the potential sensitive content in each modal data to generate a sensitivity factor Gmyz; then, based on the context factor Qycs and the sensitivity factor Gmyz, perform hierarchical analysis on the potential emotional factors, social sensitive information and potential misleading factors in the image and text content to generate a multimodal association factor Mmgyz;

[0118] The content anomaly quantification module calculates the audit weight value Hqwz based on the situational factor Qycs, the sensitivity factor Gmyz and the multimodal association factor Mmgyz; then obtains the system audit risk factor Rxz by calculation and associates it with the audit weight value Hqwz, thereby generating the final content anomaly index Cxzs;

[0119] Among them, the calculation formula of the audit weight value Hqwz is:

[0120] Hqwz=b1×Qycs+b2×Gmyz+b3×Mmgyz+B;

[0121] Among them, b1, b2 and b3 represent the preset weight coefficients of the situational factor Qycs, the sensitive factor Gmyz and the multimodal association factor Mmgyz respectively, and B represents the audit correction coefficient;

[0122] The intelligent control module is used to set the compliance threshold T and the audit risk threshold R, and compare the audit weight value Hqwz with the compliance threshold T to determine whether the current content meets the audit standards, and compare the audit risk threshold R with the content anomaly index Cxzs to obtain the final audit result, and selectively trigger manual review or automatic release control plans based on the audit results.

[0123] In an optional embodiment of the present application, the unstructured data processing module is first used to receive the unstructured data of news graphics, audio and video to be reviewed, and perform format standardization processing on the unstructured data, including image resolution adjustment, text language standardization processing and audio and video transcoding;

[0124] Then, the pre-trained network of the multimodal large model is called to extract multi-level features of news pictures, audio and video, including visual features, semantic features and emotional features, and the extracted multi-level features are normalized to generate a unimodal feature matrix; based on the unimodal feature matrix, the multi-level features of news pictures, audio and video are mapped in the multimodal association vector space, and a cross-modal semantic association graph is generated through the semantic embedding mechanism of the multimodal large model;

[0125] Secondly, we use the cross-modal semantic association graph to conduct a detailed analysis of the deep-level association between text and images, extract the core contextual elements in the text and images, and simultaneously analyze the emotional state and expression trends in the audio and video, so as to identify potential sensitive information, misleading information and social influencing factors in the news content in real time, and form multi-modal association semantic indicators;

[0126] Finally, a preliminary review dataset is generated.

[0127] In an optional embodiment of the present application, the multimodal data integration module is used to receive the preliminary review data set generated by the unstructured data processing module, and perform uniform alignment processing of the timing and content of the graphic data, audio data and video data therein through a frame synchronization algorithm;

[0128] Then, a multimodal feature mapping algorithm is applied to fuse the context, emotion and semantic features of text, audio and video into a multimodal feature vector, and a correlation matrix is ​​generated based on the correlation between each modality to perform hierarchical mapping of information density, context relevance and semantic consistency.

[0129] Subsequently, the image-text association vector space is constructed based on the association matrix, and the multimodal data is quantified in a unified semantic space to generate an integrated semantic association representation vector. At the same time, the semantic deviation caused by modality differences is corrected through the context adjustment algorithm.

[0130] Finally, the integrated image-text association vector space is transmitted to the content analysis module.

[0131] In an optional embodiment of the present application, the content analysis module includes a preliminary discrimination unit, a multi-layer sensitivity analysis unit and a correlation factor generation unit;

[0132] The preliminary discrimination unit identifies semantic deviation related data and content matching related data by analyzing the semantic similarity and context consistency between the image and the text; extracts the semantic similarity Ycy, context consistency Sjy, context association deviation Qjl and image and text emotion consistency Twq from the semantic deviation related data and the content matching related data and performs dimensionless processing, and then calculates the context factor Qycs through the following formula:

[0133]

[0134] The preset situation threshold Q is compared with the situation factor Qycs to evaluate the situational relevance of text and images in the audit scenario. The specific comparison and evaluation contents are as follows:

[0135] If the context factor Qycs ≥ the context threshold Q, it indicates that the contexts of the text and the image match, and the semantics, emotions, and contexts of the two are consistent. The review is passed and marked as "normal context association";

[0136] If the situation factor Qycs < the situation threshold Q, it indicates that the situation between the text and the image does not match, and the semantics, emotions and context of the two are inconsistent. The review fails and further analysis of the specific deviation causes, including potential sensitive content, is performed. The semantic similarity Ycy is used to measure the similarity between the text content and the content expressed by the image at the semantic level, and the text vector representation is obtained by inputting the text into the pre-trained multimodal model; the context consistency Sjy mainly measures the consistency of multidimensional information such as time, place, person or event between the text context and the image context, and is obtained by comparing the context feature set obtained from the text with the context feature set identified in the image or video; the situation association deviation Qjl is used to quantify the potential situation mismatch between the text and the image or video, and is obtained by aligning the text and the image, identifying the inconsistent parts of the "core scene or main object", and counting the inconsistent categories or quantities; the text-image emotional consistency Twq is a measure of the degree of consistency between the emotion conveyed by the text and the emotion conveyed by the image, and is obtained by calculating the similarity between the text emotional distribution and the image emotional distribution.

[0137] In an optional embodiment of the present application, the multi-layer sensitivity analysis unit is used to further perform in-depth analysis on the potential sensitive content of the graphic data, including analyzing the emotion-related data, social sensitivity-related data and potential misleading-related data in the text and image, and constructing a sensitive content data set after summarizing and dimensionlessly processing the emotion-related data, social sensitivity-related data and potential misleading-related data in the analyzed text and image; extracting the sensitive content data set and generating the sensitivity factor Gmyz through the following formula:

[0138]

[0139] In the formula, Qmq represents the emotion intensity coefficient in the sensitive content data set, Mgc represents the sensitive word index in the sensitive content data set, and Xwg represents the possibility of information misleading in the sensitive content data set. The emotion intensity coefficient Qmq is used to reflect the emotion or emotion intensity in the news content, which is obtained through text emotion intensity analysis; the sensitive word index Mgc is mainly used to measure the frequency and intensity of sensitive information, sensitive terms, banned words or sensitive symbols in the text, which is obtained through sensitive dictionary and rule matching; the possibility of information misleading Xwg is used to evaluate whether there is a potential tendency in the news content to "confuse the audience", "falsify information" or "mislead the public", which is obtained through cross-modal comparison.

[0140] In an optional embodiment of the present application, the correlation factor generation unit is used to quantify the degree of correlation between emotion, context and sensitive content based on the context factor Qycs and the sensitivity factor Gmyz by fusing multi-level information in the image-text correlation vector space, and generate a multimodal correlation factor Mmgyz by calculating through the following formula;

[0141]

[0142] In an optional embodiment of the present application, the content anomaly quantification module includes an audit calculation unit and an anomaly index acquisition unit;

[0143] The content evaluation unit is used to construct a system audit risk factor Rxz, obtain the semantic deviation degree Ycp and the content consistency index Nry by extracting semantic deviation related data and content matching related data, and obtain the high sensitivity trigger rate Gmg by extracting the sensitive content data set, and calculate the system audit risk factor Rxz in combination with the following formula:

[0144]

[0145] In an optional embodiment of the present application, the abnormality index obtaining unit is used to calculate and obtain the final content abnormality index Cxzs, and its specific calculation formula is as follows:

[0146]

[0147] In an optional embodiment of the present application, the intelligent control module compares and evaluates the audit weight value Hqwz with the compliance threshold T, and compares and evaluates the audit risk threshold R with the content anomaly index Cxzs, and specifically generates the following evaluation content:

[0148] Compliance standards comparison:

[0149] If the audit weight value Hqwz ≥ the compliance threshold T, it means that the content meets the audit standards and the system marks it as "compliant".

[0150] If the audit weight value Hqwz < the compliance threshold T, it means that the content does not meet the audit standards and is insufficiently compliant, requiring further review or adjustment.

[0151] Abnormal risk assessment:

[0152] If the content anomaly index Cxzs ≥ the audit risk threshold R, it indicates that the content has an abnormal risk, and the system generates an "abnormal content" mark, triggering the manual review process;

[0153] If the content anomaly index Cxzs < the audit risk threshold R, it means that there is no abnormal risk in the content, and the system generates a "normal content" mark, and then enters the automatic publishing process;

[0154] When the evaluation results of the audit weight value Hqwz and the content anomaly index Cxzs are marked as "compliant" and "normal content" at the same time, the content will be automatically marked as "passed" and published directly;

[0155] When the evaluation result of the audit weight value Hqwz and the content anomaly index Cxzs is that the content does not meet the audit standards or has abnormal risks, the content will be automatically marked as "pending review" and the selective review process will be triggered.

[0156] It is used to set the compliance threshold T and the audit risk threshold R, and to determine whether the current content meets the audit standards, obtain the final audit results, and selectively trigger manual review or automatic release control plans based on the audit results.

[0157] In an optional embodiment of the present application, a content audit algorithm encapsulation module is used to encapsulate all algorithms in the system.

[0158] In an optional embodiment of the present application, the system output module is used to extract key information of the results obtained based on the large model and output the final content review category.

[0159] In an optional embodiment of the present application, the first sub-model of the first multimodal model is used to receive and obtain the review result of the comment data of the specified video based on the specified video and its corresponding comment data.

[0160] In an optional embodiment of the present application, the second sub-model of the first multimodal model is also used to receive and determine the target application system failure scenario according to the target application system failure graph.

[0161] In an optional embodiment of the present application, the target application system failure scenario is used to determine the query prompt words of the target application system failure graph, and the target application system failure graph text is extracted according to the query prompt words, so as to decompose the target application system failure text description and the target application system failure graph text into multiple subtasks based on the intelligent agent and the large language model, and determine the target application system failure factors according to the information returned by the subtasks.

[0162] In an optional embodiment of the present application, the second sub-model of the first multimodal model is trained by the following steps:

[0163] Collect historical data of application system failure graphs under each application system failure scenario, wherein the application system failures include video playback errors, sequence errors, data missing, and content errors;

[0164] The application system failure diagram in each application system failure scenario is used as input, and the application system failure scenario type is used as output, and the neural network model is trained to obtain the second sub-model of the first multimodal model.

[0165] In an optional embodiment of the present application, the target application system failure scenario is used to determine a query prompt word of the target application system failure graph, including:

[0166] Determine the query prompt word corresponding to the target application system failure scenario based on the correspondence between the preset application system failure scenario and the query prompt word;

[0167] The query prompt word is used as the query prompt word of the target application system fault diagram.

[0168] In an optional embodiment of the present application, the step of extracting target application system fault graph text according to the query prompt word includes:

[0169] Inputting the query prompt words and the target application system fault diagram into the preset multi-modal large model;

[0170] The text extracted from the target application system fault diagram according to the query prompt words is output from the preset multimodal large model as the target application system fault diagram text.

[0171] In an optional embodiment of the present application, the target application system fault text description and the target application system fault graph text are decomposed into multiple subtasks based on the agent and the large language model, and the target application system fault factor is determined according to the information returned by the subtasks, including:

[0172] Inputting the target application system fault text description and the target application system fault graph text into the scheduling agent;

[0173] The scheduling agent determines the fault handling state according to the target application system fault text description and the target application system fault graph text, and sends the target application system fault text description and the target application system fault graph text to the task decomposition agent if the fault is not handled;

[0174] The task decomposition agent decomposes the fault into multiple subtasks according to the preset prompt word template based on the functions of the search engine, interface and database in the preset knowledge base, and sends the multiple subtasks to the task execution agent, wherein each subtask corresponds to at least one of the search engine, interface and database in the preset knowledge base;

[0175] The task execution agent generates executable programs for each subtask according to the prompt word template by using the preset large language model;

[0176] The task execution agent executes each executable program using at least one of the search engine, interface and database according to the corresponding relationship between each executable program and the search engine, interface and database, and uses the execution result of each executable program as the failure factor of the target application system.

[0177] In an optional embodiment of the present application, before taking the execution result of each executable program as a target application system failure factor, the method further includes:

[0178] Send the execution result of each executable program to the task decomposition agent;

[0179] The task decomposition agent determines whether the execution result of each executable program meets the preset conditions. If it does not meet the preset conditions, it adjusts the preset prompt word template, and re-decomposes the fault into multiple subtasks according to the adjusted prompt word template to re-execute multiple executable programs until the execution result of each executable program meets the preset conditions.

[0180] In an optional embodiment of the present application, the third sub-model of the first multimodal model is also used to determine whether the specified file is a target file to be deleted based on the file title, time and text information of the specified file.

[0181] In an optional embodiment of the present application, the third sub-model of the first multimodal model is trained by the following steps:

[0182] The file title, time and text information of the historical file are taken as input, and whether the file is a file to be deleted is taken as output for training to obtain the third sub-model of the first multimodal model.

[0183] In an optional embodiment of the present application, the file title, time and text information of the specified file are obtained by the following steps:

[0184] Receive the target file by using the file transfer module, send the received target file to the content distribution network module, and send the information of the target file to the message server;

[0185] The file parsing module is used to obtain the information of the target file on the message server, obtain the target file, parse the target file, and push the parsed file title, time and text information to the index module.

[0186] In an optional embodiment of the present application, a file parsing module is used to obtain information of a target file on a message server, obtain the target file, parse the target file, and push the parsed file title, time, and text information to an indexing module, including:

[0187] Get the file type of the target file;

[0188] According to the file type of the target file, determine whether the target file needs to be preprocessed;

[0189] When the target file needs to be preprocessed, the target file is preprocessed, and then the preprocessed target file is parsed;

[0190] Without preprocessing the target file, the target file is directly parsed;

[0191] Push the parsed file title, time and text information to the index module.

[0192] In an optional embodiment of the present application, determining whether the target file needs to be preprocessed according to the file type of the target file includes:

[0193] When the file type of the target file is a text file, there is no need to preprocess the target file;

[0194] When the file type of the target file is a JSON file or an HTML file, the target file needs to be preprocessed.

[0195] In an optional embodiment of the present application, when the target file needs to be preprocessed, the target file is preprocessed, including:

[0196] When the file type of the target file is a JSON file, delete the JSON tag in the JSON file;

[0197] When the file type of the target file is an HTML file, and the HTML file contains at least one of audio and video, obtain the globally unique identifiers of the audio and video, obtain the identification text of the audio and video in the HTML file through the globally unique identifiers of the audio and video, and then use the identification text as the HTML file to extract the largest body text block of the HTML file; when the HTML file does not contain at least one of audio and video, extract the largest body text block of the HTML file.

[0198] In an optional embodiment of the present application, extracting the largest body text block of the HTML file includes:

[0199] Parse the tags in the HTML file into a DOM tree and obtain the total number of tags in the HTML file and the location information of each tag;

[0200] According to the number of line break tags and paragraph tags in each node of the HTML file, all nodes are sorted to get the top N nodes;

[0201] For each of the top N nodes, perform secondary sorting based on the number of links, the number of Chinese punctuation marks, the total number of words and the number of words with links, the total number of tags in the HTML file, and the position information of each tag, and take the first node as the largest text block;

[0202] Move forward from the node position of the largest text block, extract the title of the HTML file, extract the release time of the HTML file through a regular expression, and if the extraction is successful, use the text of the first node as the text of the HTML file; if the extraction is unsuccessful, use the second node in the secondary sorting as the largest text block, and extract the title and release time of the HTML file until the extraction is successful;

[0203] When the first node is taken as the largest body text block, if the body of the first node cannot be extracted or the number of extracted body words is less than a preset threshold, the html tag in the HTML file is directly deleted, and the HTML file with the html tag deleted is taken as the body of the HTML file.

[0204] In an optional embodiment of the present application, all nodes are sorted according to the number of line break tags and paragraph tags in each node on the DOM tree to obtain the top N nodes, including:

[0205] The first weight value of each node is determined by the following expression:

[0206] W1=brCount+2*pCount

[0207] Where W1 is the first weight value of each node, brCount is the number of line break tags of each node, and pCount is the number of paragraph tags of each node.

[0208] All nodes are sorted from largest to smallest according to their first weight values ​​to obtain the top N nodes.

[0209] In an optional embodiment of the present application, for each of the top N nodes, a secondary sorting is performed on all the top N nodes according to the number of links, the number of Chinese punctuation marks, the total number of words and the number of words with links of the current node, the total number of tags of the HTML file, and the position information of each tag, including:

[0210] The second weight value of each node is determined by the following expression:

[0211] W2=(WN-LWL)×dl×f

[0212] Wherein, W2 is the second weight value of each node, WN is the total number of words of each node, LWL is the number of linked words of each node, dl is the location information of each node, and f is the weight factor of each node.

[0213] According to the second weight values ​​of all the nodes in the top N, sort them again from large to small.

[0214] Among them, the position information of each node is obtained by the following expression:

[0215]

[0216] Among them, dl is the location information of each node, NodeCount and NodePos are the total number of tags in the HTML file and the location information of each tag, respectively.

[0217] The weight factor of each node is obtained by the following expression:

[0218]

[0219] Among them, f is the weight factor of each node, dLinkPower is the number of links of each node, and nCDotNum is the number of Chinese punctuation marks of each node.

[0220] In an optional embodiment of the present application, the functions implemented by the system include the main functions of the system, which are text content review, image content review, and historical record management. Text content review is mainly divided into article content review and article comment content review. In the main function page of the system, the content display area on the right side of the text screening page is the overall content. The text screening page consists of two parts, the left side is the content input area, which includes two switch pages involving article screening and text review screening, and the right side is the screening result display area. Enter the title of the article and the content of the article, click the "Screening" button, and the article can be screened. The screening results will be displayed in the "Screening Results" area on the right. If the user does not agree with the review result, you can click the "Manual Review" button, and a pop-up window will pop up after clicking. Enter the review opinion to correct the platform analysis, enter the review result that the user agrees with, and click OK. Click Cancel to close the pop-up window directly. Image content review is similar to text content review. The picture content review is mainly divided into the review of the image of vilified characters and the review of the bad connotations of pictures and texts. The picture and text identification page consists of two parts. The left side is the content upload and input area, which includes two switching pages for the identification of vilified leaders pictures and the identification of bad connotations of pictures and texts. The right side is the identification result display area. On the picture identification page, click the "Upload Picture" button, select the picture to be identified, and the picture area will display the picture content to be identified. Click the "Identify" button to identify the picture. The identification result will be displayed in the "Identification Result" area on the right. If the user does not agree with the audit result, he can click the "Manual Review" button. The history record management includes text review and picture review content. The risk identification record page is in the form of a table. The table header has the identification date, material content, material type, scene classification, identification result, manual identification result and operation. Click the date, material type, identification result, and manual identification result in the table to filter. Click the "Edit" button to enter the manual review pop-up page, and click the "Delete" button to delete the identification record. On the record page, the table header contains date, material content, material type, summary, keywords and tags. Click the date and material type in the table to filter. Provide history management, users can view all uploaded historical images and texts, view the uploaded content and browse, to avoid repeated processing of the same file.

[0221] In an optional embodiment of the present application, in the fourth step, the prediction results are manually reviewed and evaluated. An audit feedback mechanism is established. If you are not satisfied with the prediction (screening) results, you can click the manual review button to re-evaluate the results. The background will record this result, re-label the data set, and add it to the fine-tuning of the model.

[0222] In an optional embodiment of the present application, in the fifth step, a feedback mechanism between the evaluation results and the manual review results is established to continuously optimize the accuracy of the model evaluation. The data with inconsistent results will be marked in the historical records, and the re-marked data will be put into the model for fine-tuning to continuously improve the accuracy of the model evaluation.

[0223] The present application provides a content review method based on a multimodal large model, which targets the four major scenarios that need to be reviewed in the news field, namely, the negative article review scenario, the negative article comment review scenario, the character image vilification review scenario, and the image and text bad implication review scenario. It solves the problem of negative information that traditional intelligent technology cannot identify, improves the accuracy of content review, realizes the automation of image and text review, and detects multimodal review problems that traditional intelligent review cannot solve through large models, thereby improving review efficiency.

[0224] In an optional embodiment of the present application, in the model training in the second step, for text data, according to the experimental parameter settings, the data levels of scene one and scene two are consistent, and there is no need to consider the mixed ratio of different task data. Therefore, all data is used in combination with the questioning methods of different scenes for supervised training fine-tuning. For image data, since the data of scene four is less than that of scene three, the data set of scene four is fixed, and the data volume of data three is increased. When the data ratio of scene three to scene four is 2:1, the model performance is optimal. Therefore, the data mixing ratio of the image scene is 2:1, and the training fine-tuning is combined with the questioning methods of different scenes. In terms of training strategies, there are mainly three types, namely, multi-task mixing with the same training, multi-task sequential training, and mixing with general corpus and then training with data sets. During the experiment, sequential learning in the experiments of the two scenes will lead to learning forgetting, and not mixing with general corpus will lose general ability, resulting in catastrophic forgetting. Combining general corpus training will alleviate the occurrence of this problem. Therefore, the general corpus CogVLM-SFT-311K bilingual visual instruction data set is combined with the data set of specific scenes for fine-tuning. The trained model will make result predictions on the validation set, which contains 910 text data and 1,350 image data.

[0225] Experimental results First, we need to calculate TP, TN, FP, and FN for each scenario; TP represents the total number of data that are accurately detected to contain negative information; TN represents the total number of data that are accurately detected to have positive meanings; FP represents the total number of data that are incorrectly detected as negative information; and FN represents the total number of data that are incorrectly detected as positive information. Then, we can calculate the accuracy, precision, recall, and F1 of each scenario according to the following formulas. The calculation results are shown in Table 5:

[0226]

[0227] Table 5

[0228] Scenario Accuracy Accuracy Recall <![CDATA[F1]]> 1 93.8％ 94.9％ 92.5％ 93.1％ 2 94.2％ 97.8％ 93.2％ 93.7％ 3 97.9％ 99.3％ 97.3％ 97.6％ 4 96.8％ 100％ 96.1％ 96.4％

[0229] As shown in Table 5, the scene accuracy, precision and recall are all above 90%. F1 is above 90%, indicating that the experimental method is relatively ideal. Among them, the graphic scene has a relatively high technical difficulty in the overall task, and there is still room for further improvement. However, the overall situation basically meets the needs of auditors, and can reduce the audit burden of auditors in practical applications.

[0230] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0231] See also Figure 8 An embodiment of the present application provides a video comment review device 800, comprising:

[0232] An acquisition module 810 is used to acquire a target video and its corresponding comment data;

[0233] A construction module 820 is used to annotate the comment data corresponding to the target video based on the correlation between the target video and the comment data corresponding to the target video, and to construct a comment data set corresponding to the target video;

[0234] A training module 830 is used to use the target video and its corresponding comment data set as a training set to perform fine-tuning training on the preset multimodal model to obtain a trained first multimodal model;

[0235] The input module 840 is used to receive a specified video and its corresponding comment data, and input the specified video and its corresponding comment data into the first multimodal model to obtain the review result of the comment data of the specified video, and process the comment data whose review result is a negative comment.

[0236] For the specific limitations of the above-mentioned device 800, please refer to the limitations of the video comment review method above, which will not be repeated here. The various modules in the above-mentioned device 800 can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0237] In one embodiment, a computer device is provided, the internal structure diagram of which can be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a video comment review method as described above is implemented. It includes: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, any step in the video comment review method as described above is implemented.

[0238] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above-mentioned video comment review method can be implemented.

[0239] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0240] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0241] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0242] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0243] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0244] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A video comment review method, characterized in that: include: Get the target video and its corresponding comment data; Based on the correlation between the target video and its corresponding comment data, the comment data corresponding to the target video is annotated to construct a comment dataset corresponding to the target video; The target video and its corresponding comment dataset are used as training sets, and the preset multimodal model is fine-tuned to obtain a trained first multimodal model; A designated video and its corresponding comment data are received, and the designated video and its corresponding comment data are input into a first multimodal model to obtain a review result of the comment data of the designated video, and the comment data whose review result is a negative comment is processed.

2. The method according to claim 1, characterized in that The step of labeling the comment data corresponding to the target video based on the correlation between the target video and the comment data corresponding to the target video includes: Extract key information and sentiment information from the comment data corresponding to the target video, and preliminarily determine whether the comment data is a negative comment based on the sentiment information; If it is preliminarily determined that the comment data is a negative comment, marking the comment data as a negative comment; In the case where it is preliminarily determined that the comment data is a positive comment, further determining whether the comment data is a negative comment based on the correlation between the target video and its corresponding comment data; If it is further determined that the comment data is a negative comment, marking the comment data as a negative comment; When it is further determined that the comment data is a positive comment, the comment data is marked as a positive comment.

3. The method according to claim 2, characterized in that The further determining whether the comment data is a negative comment based on the correlation between the target video and the corresponding comment data includes: Extract key information and sentiment information from the target video, and calculate the similarity between the key information and sentiment information in the target video and its corresponding comment data; Determine the degree of conflict between the target video and the text content in the corresponding comment data; Determine the conflict score between the target video and its corresponding comment data according to the similarity between the key information and emotional information in the target video and its corresponding comment data and the degree of conflict between the text contents; The review data whose conflict score exceeds a preset conflict score threshold is determined as a negative review.

4. The method according to claim 3, characterized in that The similarity between the key information and sentiment information in the target video and its corresponding comment data is calculated by the following expression: Among them, A is the first text vector composed of key information and emotional information in the target video, B is the second text vector composed of key information and emotional information in the comment data corresponding to the target video, a i is the i-th vector element in the first text vector, b i is the i-th vector element in the second text vector, and w1 and w2 are both weight factors.

5. The method according to claim 1, characterized in that: The constructing of the comment dataset corresponding to the target video includes: Each comment data corresponding to the target video and the annotation information of each comment data are collected into a comment data set corresponding to the target video.

6. The method according to claim 1, characterized in that The target video and its corresponding comment data set are used as training sets to fine-tune the preset multimodal model to obtain a trained first multimodal model, including: The video frame of the target video and each comment data are taken as input, and the annotation information of the current comment data is taken as output. The network structure used to process the video frames in the preset multimodal model is trained using the lora fine-tuning method of supervised tasks, and the network structure used to process the comment data is trained using the qlora fine-tuning method of supervised tasks to obtain the first multimodal model after training.

7. The method according to claim 6, characterized in that The step of inputting the designated video and its corresponding comment data into the first multimodal model to obtain the review result of the comment data of the designated video includes: Inputting a video frame of a specified video into a network structure for processing the video frame in a first multimodal model to obtain a video frame feature; Inputting comment data corresponding to the specified video into a network structure for processing comment data in the first multimodal model to obtain comment data features; The video frame features and the comment data features are input into the fusion network structure in the first multimodal model to obtain the review result of the comment data of the designated video.

8. A video comment review device, characterized in that: include: An acquisition module is used to acquire a target video and its corresponding comment data; A construction module is used to annotate the comment data corresponding to the target video based on the correlation between the target video and the corresponding comment data, and to construct a comment data set corresponding to the target video; A training module, used to use the target video and its corresponding comment data set as a training set, to perform fine-tuning training on the preset multimodal model, and obtain a trained first multimodal model; The input module is used to receive a specified video and its corresponding comment data, and input the specified video and its corresponding comment data into the first multimodal model to obtain the review results of the comment data of the specified video, and process the comment data whose review results are negative comments.

9. A computer device comprising: A memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the video comment review method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video comment review method described in any one of claims 1 to 7 are implemented.

Images