Task auditing method and device based on artificial intelligence technology

Abstract

The embodiment of the invention discloses a task auditing method and device based on an artificial intelligence technology. The task auditing method comprises the steps that multi-modal content is acquired and preprocessed; extracting text features, image features and video features respectively, and generating a multi-modal feature vector; inputting the feature vector into a pre-trained deep learning model for fusion analysis, and outputting an auditing result; based on the confidence of the auditing result, different preset strategies are adopted to generate a final auditing decision; performing interpretability analysis on the final auditing decision to generate an auditing basis report; collecting auditing feedback data, and updating the deep learning model. According to the method, the efficiency, accuracy and interpretability of content auditing are effectively improved, and the problems that a traditional auditing method is low in efficiency, high in cost and insufficient in accuracy are solved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a task review method and device based on artificial intelligence technology. Background Art

[0002] With the rapid development of the internet and mobile applications, user-generated content (UGC) on various online platforms and applications has exploded. This content includes, but is not limited to, text, images, videos, and other multimedia formats. To ensure the quality and compliance of platform content, large-scale content review has become crucial.

[0003] Traditional content review relies primarily on manual methods, with professional reviewers individually reviewing user-generated content according to platform rules and relevant regulations. While this method can ensure a certain level of review quality, it is inefficient due to the sheer volume of content and cannot meet the demands of real-time review. Furthermore, manual review is costly, and different reviewers may have different criteria for judging content, leading to inconsistent review results.

[0004] To address these issues, the industry has begun experimenting with automated technology to assist with content review. Automated review systems primarily rely on simple techniques like keyword filtering and rule matching. While effective in certain scenarios, they have significant limitations. These systems typically only process structured text and have limited processing capabilities for complex media formats like images and videos. Furthermore, simple rules are unable to address the various distorted expressions and obscure content that users use to circumvent review, resulting in high rates of false positives and missed detections. Summary of the Invention

[0005] In view of this, the embodiments of the present disclosure provide a task review method and device based on artificial intelligence technology to solve the problems existing in the prior art.

[0006] In a first aspect, the present disclosure provides a task review method based on artificial intelligence technology, which adopts the following technical solutions: The task review method based on artificial intelligence technology includes: Acquire multimodal content to be reviewed in the current task, preprocess the multimodal content, and obtain preprocessed multimodal data; Respectively extracting text features, image features, and video features from the preprocessed multimodal data to generate a multimodal feature vector; Inputting the multimodal feature vector into a pre-trained deep learning model, performing fusion analysis on the multimodal feature vector, and outputting a task review result; Based on the confidence level of the task audit results, different preset strategies are used to generate the final audit decision; Conduct an explainability analysis on the final audit decision and generate an audit basis report; Based on the audit basis report and audit process data, audit feedback data is collected, and the deep learning model is updated using the audit feedback data.

[0007] Optionally, respectively extracting text features, image features, and video features from the preprocessed multimodal data to generate a multimodal feature vector includes: Inputting the text data in the preprocessed multimodal data into a pre-trained bidirectional encoder representation model, extracting the text encoding, and obtaining a text feature vector; Inputting the image data in the preprocessed multimodal data into a pre-trained residual network model to extract image features to obtain an image feature vector; Segmenting the video data in the preprocessed multimodal data into segments of fixed length, extracting spatiotemporal features of each segment using a three-dimensional convolutional neural network, and inputting the extracted spatiotemporal feature sequence into a long short-term memory network to obtain a video feature vector; The text feature vector, the image feature vector, and the video feature vector are combined to generate the multimodal feature vector.

[0008] Optionally, the pre-trained deep learning model includes a multi-input and multi-output deep neural network, and the deep neural network includes an input layer, a feature fusion layer, a multi-layer perceptron and an output layer; Input the multimodal feature vector into a pre-trained deep learning model, perform fusion analysis on the multimodal feature vector, and output the task review results, including: In the feature fusion layer, the multimodal feature vectors are fused using an attention mechanism to generate fused features; In the multi-layer perceptron, the fused features are processed to extract high-level semantic information; In the output layer, high-level semantic information is classified through multi-label classification to obtain content categories and corresponding confidence values to form the task review results.

[0009] Optionally, based on the confidence level of the task review result, different preset strategies are used to generate a final review decision, including: Setting a confidence threshold to classify the task review result into a high confidence result, a medium confidence result, and a low confidence result; Processing the high-confidence results directly through a fast filtering layer to generate an automatic review decision; Perform secondary analysis on the medium confidence results through the deep analysis layer to generate a deep audit decision; Transferring the low-confidence results or the controversial content in the deep review decision to the manual review layer for manual review and generating a manual review decision; The automatic review decision, the in-depth review decision, and the manual review decision are integrated to form the final review decision.

[0010] Optionally, an explainability analysis is performed on the final audit decision to generate an audit basis report, including: Use feature importance analysis to calculate the contribution of each feature to the final review decision and identify key influencing factors; Generate a heat map of the text features, image features, and video features to highlight key areas that influence decision-making; Record the activation status of each layer within the deep learning model, build decision paths, and explain the reasoning process of the deep learning model; Based on the text features, image features, and video features, retrieve historical audit cases similar to the current task as a reference for decision-making; The key influencing factors, the heat map, the decision path and the historical audit cases are integrated to generate the audit basis report.

[0011] Optionally, before inputting the multimodal feature vector into the pre-trained deep learning model, the method further includes: Construct a cross-modal attention interaction network; The feature vector of each modality is used as the source of query, key and value to build a multi-head cross attention structure; Compute image features guided by text features, text features guided by image features, and interactive features between text-video and image-video; Design an adaptive weight network to integrate the outputs of each cross-attention head and dynamically adjust the importance weight of the modality relationship based on the content type; The interacted features are used as input to the deep learning model.

[0012] Optionally, the updating the deep learning model using the review feedback data includes: The audit feedback data includes task audit results, and different model update strategies are adopted for task audit results with different confidence levels; The audit feedback data is divided into three types: manual audit results, user reporting data, and platform operation data. Differentiated processing methods are adopted according to different types of audit feedback data to update the deep learning model; Different model updating strategies are adopted for task review results with different confidence levels, including: For high-confidence task review results, their automatic review decisions are included as standard samples in the training set for regular incremental training of the model; For the task review results with medium confidence, the in-depth review decision and the feature analysis results of the decision-making process are used to adjust the feature fusion layer parameters of the model; For low-confidence task review results, their manual review decisions are used as key samples and prioritized for the model's active learning process; The updating of the deep learning model by adopting differentiated processing methods according to different types of review feedback data includes: For manual review results, the deep learning model is fine-tuned using transfer learning methods, focusing on optimizing the model's classification boundaries; For user-reported data, we analyze it as a case of model misjudgment and generate adversarial samples to enhance the robustness of the model. For platform operation data, extract content distribution and user interaction patterns, and adjust the model's attention mechanism weight distribution; Maintain the model version library, support model rollback and A / B testing, and ensure the stability of model updates.

[0013] Optionally, the updating the deep learning model using the review feedback data includes: Perform cluster analysis on historical audit data, including historical task audit results and corresponding audit feedback data, to identify data features of different types of content patterns or time windows; Train the corresponding expert model for each type of data feature to obtain an expert model pool; Design and train a meta-learner that receives the output of all expert models as input and generates the final decision through an ensemble learning algorithm; Evaluate the processing capabilities of existing expert models based on new data, and selectively update the models or train new expert models; Combining the statistical characteristics of real user data and the decision boundary of the expert model, adversarial samples close to real scenarios are generated.

[0014] Optionally, the task review method further includes: when updating the deep learning model using the review feedback data, constructing a cross-modal knowledge fusion network based on decentralized Bayesian reasoning, including: Construct a probabilistic feature representation layer and convert the feature vector into a probability distribution representation by adding a variational encoder, so that each feature point is represented as a Gaussian distribution parameterized by mean and variance; Implement a decentralized Bayesian interaction mechanism, transforming the attention calculation process into a Bayesian update process. The probability distribution of each modal feature is used as prior knowledge, and the distribution of each other is updated by calculating the conditional probability to form the posterior distribution. Develop a distributed expert knowledge integration system, transform each expert model into a Bayesian neural network, and integrate the probability outputs of each expert model through the Bayesian model averaging method; Establish a knowledge graph-assisted prior construction system to encode domain rules, regulatory requirements, and historical audit experience into conditional probability tables in Bayesian networks; Design an uncertainty-aware decision-making mechanism, calculate the entropy or variance of the decision as an uncertainty measure based on the probability distribution of the reasoning results, and automatically transfer cases with high uncertainty to manual review.

[0015] In a second aspect, the present disclosure also provides a task review device based on artificial intelligence technology, which adopts the following technical solutions: The task review device based on artificial intelligence technology includes: A multimodal content acquisition unit, configured to acquire multimodal content to be reviewed in a current task, preprocess the multimodal content, and obtain preprocessed multimodal data; A feature extraction unit, configured to extract text features, image features, and video features from the preprocessed multimodal data, respectively, to generate a multimodal feature vector; A deep learning unit, configured to input the multimodal feature vector into a pre-trained deep learning model, perform fusion analysis on the multimodal feature vector, and output a task review result; A hierarchical review unit, configured to generate a final review decision using different preset strategies based on the confidence level of the task review result; An explainability analysis unit, configured to perform explainability analysis on the final audit decision and generate an audit basis report; A model updating unit is used to collect audit feedback data based on the audit basis report and audit process data, and use the audit feedback data to update the deep learning model.

[0016] In a third aspect, the embodiments of the present disclosure further provide a computer device that adopts the following technical solution: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any of the above-mentioned task review methods based on artificial intelligence technology.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned task review methods based on artificial intelligence technology.

[0018] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of any of the above-mentioned task review methods based on artificial intelligence technology.

[0019] By integrating advanced technologies such as deep learning, natural language processing, and computer vision, we achieve efficient and accurate review of multimodal content, significantly improving the performance of automated review. We implement a tiered review process, applying different processing strategies to content with different confidence levels, reducing manual intervention, improving resource utilization, and lowering the overall operating costs of content review.

[0020] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A flowchart of a task review method based on artificial intelligence technology provided by an embodiment of the present invention; Figure 2 A schematic diagram of a specific flow chart of step S2 provided in an embodiment of the present invention; Figure 3 A schematic diagram of a specific flow chart of step S3 provided in an embodiment of the present invention; Figure 4 A schematic diagram of a specific process of step S4 provided in an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a task review device based on artificial intelligence technology provided by an embodiment of the present invention; Figure 6 A schematic diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0024] It should be clear that the following embodiments of the present disclosure are described through specific concrete examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0025] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0026] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0027] Additionally, in the following description, specific details are provided to provide a thorough understanding of the examples. However, one skilled in the art will appreciate that the aspects described can be practiced without these specific details.

[0028] With the development of artificial intelligence (AI) technology, some platforms have begun experimenting with machine learning methods for content moderation. These systems typically utilize specialized classification models designed for a single modality (such as text or images). While this improves automation, it still suffers from significant shortcomings when dealing with complex, multimodal content in the real world. First, these systems often fail to effectively integrate information across modalities, resulting in inadequate identification of cross-modal violations (such as misleading content with inconsistent text and images). Second, traditional machine learning models are typically "black box" designs, lacking explainability in their decision-making processes and providing insufficient evidence for review results. Third, once deployed, these models struggle to adapt to evolving violation patterns and lack mechanisms for continuous optimization.

[0029] The latest research has begun to explore the application of technologies such as deep learning and multimodal fusion in the field of content review, but still faces many challenges: how to effectively process feature extraction and fusion of different types of data such as text, images, and videos; how to balance model complexity and inference efficiency; how to provide explainable support for review decisions; and how to design dynamic learning mechanisms to adapt to the ever-changing content environment.

[0030] Therefore, there is an urgent need for a content review technology solution that can efficiently process multimodal content and possesses explainability and adaptability to meet the growing demand for content governance on current internet platforms. This paper addresses these issues by proposing a task review method and device based on artificial intelligence technology, aiming to improve the efficiency, accuracy, and scalability of content review and provide strong technical support for content governance on various online platforms.

[0031] like Figure 1 As shown, an embodiment of the present invention provides a task review method based on artificial intelligence technology, comprising the following steps: Step S1: Acquire multimodal content to be reviewed in the current task, preprocess the multimodal content, and obtain preprocessed multimodal data.

[0032] In this embodiment, multimodal content may include, but is not limited to, data in various forms, such as text, images, and videos. Preprocessing operations utilize appropriate processing methods for different types of data. For text data, preprocessing includes steps such as text cleaning, word segmentation, stop word removal, and standardization. For image data, preprocessing includes scaling, cropping, normalization, and enhancement. For video data, preprocessing includes frame extraction, resolution adjustment, and format conversion. The purpose of preprocessing is to remove noise and standardize the data format to make it suitable for subsequent feature extraction and deep learning model processing.

[0033] Step S2: extracting text features, image features, and video features from the preprocessed multimodal data respectively to generate a multimodal feature vector.

[0034] like Figure 2 As shown, in this embodiment, multimodal feature extraction specifically includes: Step S2.1: Input the text data in the preprocessed multimodal data into a pre-trained bidirectional encoder representation model, extract the text encoding, and obtain a text feature vector.

[0035] In this example, the BERT (Bidirectional Encoder Representations from Transformers) model is used for text encoding. BERT is a pre-trained language model that captures the contextual semantics of text. In implementation, the input text is first segmented and tokenized. The pre-trained BERT model is then used to encode the text. Finally, the output of the BERT model's last layer is extracted as a text feature vector, typically with 768 dimensions.

[0036] Step S2.2: Input the image data in the preprocessed multimodal data into the pre-trained residual network model, extract image features, and obtain an image feature vector.

[0037] In this example, a Residual Network (ResNet) deep convolutional neural network is used to extract image features. ResNet solves the vanishing gradient problem in deep networks through residual connections, effectively extracting high-level image features. In implementation, the input image is first preprocessed (scaling, normalization, etc.), then image features are extracted using a pretrained ResNet model. Finally, the feature map before the last fully connected layer of the ResNet is extracted as the image feature vector, typically with a dimension of 2048.

[0038] Step S2.3: Segment the video data in the preprocessed multimodal data into segments of fixed length, extract the spatiotemporal features of each segment using a three-dimensional convolutional neural network, input the extracted spatiotemporal feature sequence into a long short-term memory network, and obtain a video feature vector.

[0039] In this example, a 3D convolutional neural network (C3D) and a long short-term memory (LSTM) network are combined to extract spatiotemporal features from videos. Specifically, the video is first segmented into fixed-length segments. The C3D model is then used to extract spatiotemporal features from each segment. The feature sequence extracted by the C3D is then fed into an LSTM network to capture long-term temporal dependencies. Finally, the output of the last time step of the LSTM network is extracted as the video feature vector.

[0040] Step S2.4: combining the text feature vector, the image feature vector, and the video feature vector to generate the multimodal feature vector.

[0041] Step S3: Input the multimodal feature vector into a pre-trained deep learning model, perform fusion analysis on the multimodal feature vector, and output the task review result.

[0042] Optionally, in this embodiment, the pre-trained deep learning model includes a multi-input multi-output deep neural network, and the deep neural network includes an input layer, a feature fusion layer, a multi-layer perceptron and an output layer. Figure 3 As shown, step S3 specifically includes: Step S3.1: In the feature fusion layer, the multimodal feature vectors are fused using an attention mechanism to generate fused features.

[0043] In this embodiment, the feature fusion layer primarily relies on the attention mechanism to achieve effective fusion of features from different modalities. The attention mechanism automatically learns the importance weights of features from different modalities, allowing the model to focus on more critical information. In specific implementations, a multi-head self-attention mechanism can be used to first enhance features within the modality, and then achieve cross-attention to enable information interaction and fusion between modalities.

[0044] Step S3.2: In the multi-layer perceptron, the fused features are processed to extract high-level semantic information.

[0045] In this embodiment, the multilayer perceptron (MLP) consists of multiple fully connected layers, each followed by a nonlinear activation function (such as ReLU) and a regularization layer (such as Dropout). The MLP further processes the fused features, extracting higher-level abstract representations and improving the model's expressiveness and generalization capabilities.

[0046] Step S3.3: In the output layer, high-level semantic information is classified through multi-label classification to obtain content categories and corresponding confidence values to form the task review results.

[0047] In this embodiment, the output layer typically uses a Softmax or Sigmoid activation function to map high-level semantic features to different category probabilities. In content review scenarios, categories may include multiple labels such as "normal," "violation," and "sensitive." Therefore, a multi-label classification approach is adopted, meaning that a piece of content may belong to multiple categories simultaneously, each with a corresponding confidence value.

[0048] Step S4: Based on the confidence level of the task audit result, different preset strategies are used to generate a final audit decision.

[0049] like Figure 4As shown, in this embodiment, based on the confidence level of the task audit results, a hierarchical audit strategy is adopted to generate the final audit decision, specifically including: Step S4.1: Set a confidence threshold and divide the task review result into high confidence results, medium confidence results and low confidence results.

[0050] In this embodiment, two confidence thresholds can be set, for example, 0.8 and 0.6, to classify results with a confidence greater than 0.8 as high confidence results, results with a confidence between 0.6 and 0.8 as medium confidence results, and results with a confidence less than 0.6 as low confidence results. These thresholds can be adjusted according to actual application scenarios and requirements.

[0051] Step S4.2: The high confidence results are directly processed through a fast filtering layer to generate an automatic review decision.

[0052] In this embodiment, the fast filtering layer directly uses the deep learning model's judgment results for high-confidence results without additional analysis, thereby achieving efficient processing. These results are usually very certain judgments made by the model, such as clearly normal content or clearly illegal content.

[0053] Step S4.3: The medium confidence result is subjected to secondary analysis through the deep analysis layer to generate a deep audit decision.

[0054] In this embodiment, the deep analysis layer uses more complex algorithms and rules to perform secondary analysis on medium confidence results, which may include more detailed feature analysis, contextual association analysis, historical data comparison, etc., to generate more reliable audit decisions.

[0055] Step S4.4: Transfer the low-confidence results or the controversial content in the deep review decision to the manual review layer for manual review to generate a manual review decision.

[0056] In this example, the manual review layer is the final decision-making stage, primarily handling two types of content: content with low model confidence and content that remains controversial after in-depth analysis. Manual review is conducted by professional reviewers based on platform rules and relevant regulations to ensure quality.

[0057] Step S4.5: Integrate the automatic review decision, the in-depth review decision, and the manual review decision to form the final review decision.

[0058] Step S5: Perform an explainability analysis on the final audit decision and generate an audit basis report.

[0059] In this embodiment, an explainability analysis is performed on the final audit decision to generate an audit basis report, which specifically includes: Step S5.1: Use the feature importance analysis method to calculate the contribution of each feature to the final audit decision and identify key influencing factors.

[0060] In this example, SHAP (SHapley Additive exPlanations) values are used to calculate the contribution of each feature to the audit decision. SHAP, a game-theory-based method, assigns an importance value to each feature, indicating its influence on the model's predictions. By analyzing these importance values, the key factors that influence the final decision can be identified.

[0061] Step S5.2: Generate a heat map for the text features, image features, and video features to highlight key areas that affect decision-making.

[0062] In this embodiment, for text content, attention weights can be used to generate text heat maps to intuitively show which words or sentences have the greatest impact on decision-making; for image content, Grad-CAM and other technologies can be used to generate heat maps to identify key areas in the image; for video content, heat maps can be generated in both time and space dimensions to identify key frames and key areas.

[0063] Step S5.3: Record the activation states of each layer within the deep learning model, construct the decision path, and explain the reasoning process of the deep learning model.

[0064] In this embodiment, by recording and analyzing the activation states of each layer within a deep learning model, a visual decision path can be constructed to track the model's reasoning process. This approach can transform the complex "black box" neural network decision-making process into a series of understandable steps, enhancing the model's transparency.

[0065] Step S5.4: Based on the text features, image features, and video features, retrieve historical audit cases similar to the current task as a reference for decision making.

[0066] In this embodiment, content feature vectors are used to retrieve similar cases from a historical audit database, providing a "case similarity" analysis. This case-based reasoning approach can provide historical references for current audit decisions, and is particularly valuable for complex or borderline content judgments.

[0067] Step S5.5: Integrate the key influencing factors, the heat map, the decision path and the historical audit cases to generate the audit basis report.

[0068] Step S6: Based on the audit basis report and audit process data, collect audit feedback data, and use the audit feedback data to update the deep learning model.

[0069] In this embodiment, different model update strategies are adopted for task review results with different confidence levels, and differentiated processing methods are adopted according to different types of review feedback data to update the deep learning model, specifically including: On the one hand, different model update strategies are adopted for task review results with different confidence levels: (1) For high-confidence task review results, their automatic review decisions are included as standard samples in the training set for regular incremental training of the model; (2) For the task review results with medium confidence, the deep review decision and the feature analysis results of the decision-making process are used to adjust the parameters of the feature fusion layer of the model; (3) For low-confidence task review results, their manual review decisions are used as key samples and prioritized for the model’s active learning process.

[0070] On the other hand, audit feedback data is divided into three types: manual audit results, user reporting data, and platform operation data. Different processing methods are adopted according to different types of audit feedback data: (1) For the manual review results, the deep learning model is fine-tuned using transfer learning methods, focusing on optimizing the classification boundaries of the model; (2) For user-reported data, we analyze it as a case of model misjudgment and generate adversarial samples to enhance the robustness of the model; (3) For platform operation data, extract content distribution and user interaction patterns and adjust the weight distribution of the model’s attention mechanism; (4) Maintain the model version library, support model rollback and A / B testing, and ensure the stability of model updates.

[0071] Through the above steps, the task review method based on artificial intelligence technology provided by the present invention can efficiently and accurately review multimodal content, while providing explainable review basis and continuously optimized dynamic learning mechanism, providing strong technical support for content governance of various online platforms.

[0072] Based on the above, the embodiment of the present invention also provides several improvement solutions to further enhance the performance and adaptability of the audit system.

[0073] Optionally, in this embodiment, before inputting the multimodal feature vector into the pre-trained deep learning model, the following steps are introduced: Step S6.1: Construct a cross-modal attention interaction network.

[0074] In this embodiment, the application of the Cross-Modal Attention Interaction Network (CMAIN) addresses the inefficient multimodal feature fusion problem in existing technologies. Traditional attention mechanisms for modal fusion cannot fully capture complex cross-modal semantic associations, and are particularly ineffective in identifying subtle and illegal content. CMAIN utilizes a bidirectional interaction mechanism to enable features from different modalities to guide and enhance each other, thereby improving fusion efficiency and accuracy.

[0075] Step S6.2: Take the feature vector of each modality as the source of query, key and value, and build a multi-head cross attention structure.

[0076] In this embodiment, the existing BERT text feature extraction, ResNet image feature extraction, and C3D+LSTM video feature extraction modules remain unchanged, but an inter-modal interaction layer is added after feature extraction. In specific implementation, the feature vectors of each modality are used as the source of queries, keys, and values to construct a multi-head cross-attention structure. For example, text features can be used as queries, and image features as keys and values, and "text-guided image features" can be generated through attention calculation; vice versa, "image-guided text features" can be generated. The same method is applied to text-video and image-video pairs.

[0077] Step S6.3: Calculate image features guided by text features and text features guided by image features, as well as interactive features between text and video and between image and video.

[0078] In this embodiment, for any two modalities A and B, two interactive features can be calculated: A-guided B features and B-guided A features. A-guided B features indicate which parts of modal B are more important under the guidance of A modal information; B-guided A features reflect the important information in modal A under the guidance of B modal information. This two-way interaction enables the system to fully capture cross-modal semantic associations.

[0079] Step S6.4: Design an adaptive weight network to integrate the outputs of each cross-attention head and dynamically adjust the importance weight of the modal relationship according to the content type.

[0080] In this embodiment, the adaptive weight network is a small neural network that receives content type encoding or features as input and outputs weight coefficients for different cross-attention heads. For example, for content that combines text and images, the system may increase the weight of text-image interaction; while for content such as video comments, it may increase the weight of video-text interaction. This adaptive weight design allows the system to optimize processing for different content scenarios.

[0081] Step S6.5: Use the interacted features as input to the deep learning model.

[0082] Through the above steps, CMAIN can more accurately capture cross-modal semantic associations, especially significantly improving the identification of obscure and illegal content. At the same time, the utilization of computing resources is more efficient because the system can focus on truly relevant modal features.

[0083] In this embodiment, there are many specific ways to update the deep learning model using the review feedback data, which are illustrated below with examples.

[0084] Example 1: When using audit feedback data to update a deep learning model, the following method is used: Step S7.1: Perform cluster analysis on the historical audit data including historical task audit results and corresponding audit feedback data to identify data features of different types of content patterns or time windows.

[0085] In this example, we first perform cluster analysis on historical audit data. K-means, hierarchical clustering, or more advanced deep clustering methods can be used to identify natural clusters within the data. Clustering can be based on content type (e.g., primarily text, primarily images, primarily videos, etc.) or time windows (e.g., hot topics or violation patterns over different periods). The purpose of clustering is to group similar data, facilitating subsequent targeted training of expert models.

[0086] Step S7.2: Train the corresponding expert model for each type of data feature to obtain an expert model pool.

[0087] In this embodiment, a dedicated expert model is trained for each clustering result. Each expert model utilizes the same multimodal deep learning architecture as the main system, but with parameters and optimization targets tailored to the specific data type. For example, an expert model processing primarily textual content may have a more complex structure for text feature extraction and processing, while an expert model processing primarily image-based content may have stronger capabilities in image processing.

[0088] Step S7.3: Design and train a meta-learner that receives the outputs of all expert models as input and generates the final decision through an ensemble learning algorithm.

[0089] In this embodiment, the meta-learner is a high-level model responsible for integrating the judgment results of all expert models. This can be done using a simple weighted voting mechanism or more complex methods such as stacking or Bayesian model averaging. The meta-learner is trained on the validation set, with the goal of maximizing overall audit accuracy. The meta-learner not only learns the relative credibility of each expert model but also captures combined patterns in the outputs of these expert models, uncovering information that individual models might overlook.

[0090] Step S7.4: Evaluate the processing capability of the existing expert model based on the new data, and selectively update the model or train a new expert model.

[0091] In this embodiment, when new data arrives, the system first evaluates the processing capabilities of existing expert models for that data. If a particular expert model performs well for the new data (i.e., has high prediction accuracy), only that model is updated. If all models perform poorly, a new expert model is trained and added to the model pool. Simultaneously, the performance and contribution of each expert model in the model pool are regularly evaluated, and redundant or outdated models are eliminated to maintain a streamlined and efficient system.

[0092] Step S7.5: Combine the statistical characteristics of real user data and the decision boundary of the expert model to generate adversarial samples close to the real scene.

[0093] In this embodiment, adversarial example generation is no longer based solely on model gradient information, but instead incorporates the statistical characteristics of real user data. Specifically, the system analyzes the distribution characteristics of real illegal content (such as word frequency distribution and image feature distribution), then applies these statistical constraints when generating adversarial examples to ensure that the generated examples conform to real user behavior patterns. Furthermore, the system analyzes the decision boundaries of each expert model, generating more valuable adversarial examples near these boundaries to improve model robustness.

[0094] Through the above steps, MEELS can effectively solve the stability problem in the dynamic learning process, realize the accumulation rather than replacement of knowledge, and generate more realistic adversarial samples to enhance the generalization performance of the model in real scenarios.

[0095] Example 2: When using audit feedback data to update the deep learning model, the following method is used: Step S8.1: Construct a probabilistic feature representation layer and convert the feature vector into a probability distribution representation by adding a variational encoder so that each feature point is represented as a Gaussian distribution parameterized by mean and variance.

[0096] In this embodiment, the probabilistic feature representation layer is a fundamental component of DBI-CMKFN. Feature vectors extracted by existing BERT, ResNet, and C3D+LSTM models are converted into probabilistic distribution representations by adding a variational encoder. Specifically, each feature point is no longer a single value, but rather a Gaussian distribution parameterized by its mean and variance. This representation captures the uncertainty and range of features, distinguishing between "certain features" and "uncertain features," and provides a foundation for subsequent Bayesian reasoning.

[0097] Step S8.2: Implement a decentralized Bayesian interaction mechanism, transform the attention calculation process into a Bayesian update process, use the probability distribution of each modal feature as prior knowledge, and update each other's distribution by calculating the conditional probability to form the posterior distribution.

[0098] In this embodiment, the decentralized Bayesian interaction mechanism is an improvement to the CMAIN multi-head cross-attention structure. In the traditional attention mechanism, the attention weight is obtained by calculating the similarity between the query and the key, and then the values are weighted and summed. In this solution, this process is transformed into a Bayesian update: the probability distribution of each modal feature is used as prior knowledge, and after observing the features of other modalities, its own distribution is updated to form a posterior distribution. This probability-based interaction method is more consistent with human cognitive processes and can make reasonable inferences when features are incomplete or noisy. In practice, this complex Bayesian update process can be implemented through methods such as Monte Carlo sampling or variational inference.

[0099] Step S8.3: Develop a distributed expert knowledge integration system, transform each expert model into a Bayesian neural network, and integrate the probability outputs of each expert model through the Bayesian model averaging method.

[0100] In this embodiment, the distributed expert knowledge integration system improves upon the MEELS expert model pool architecture. Each expert model is transformed into a Bayesian neural network, outputting not only deterministic judgments but also the probability distribution of those judgments (typically expressed as the mean and variance of a normal distribution). The meta-learner integrates the probabilistic outputs of each expert model using Bayesian model averaging to generate a more reliable final judgment. This integration approach automatically resolves conflicts between models and provides an estimate of decision uncertainty.

[0101] Step S8.4: Establish a knowledge graph-assisted prior construction system to encode domain rules, regulatory requirements, and historical audit experience into a conditional probability table in the Bayesian network.

[0102] In this embodiment, the knowledge graph-assisted prior construction system is an innovative component of DBI-CMKFN. It constructs a knowledge graph for content moderation, encoding domain rules (such as the relationship between sensitive words and violation types), regulatory requirements (such as review standards for different content categories), and historical review experience as conditional probability tables within a Bayesian network. This prior knowledge is incorporated into the model through rule injection or semi-supervised learning, guiding the inference process, especially in data-sparse areas. This step addresses the shortcomings of purely data-driven approaches and enhances the system's ability to identify emerging violations.

[0103] Step S8.5: Design an uncertainty-aware decision-making mechanism. Based on the probability distribution of the inference results, calculate the entropy or variance of the decision as an uncertainty measure, and automatically transfer cases with high uncertainty to manual review.

[0104] In this embodiment, the uncertainty-aware decision-making mechanism further enhances the tiered review process. Based on the probability distribution of the inference results, the system calculates the entropy (a measure of uncertainty in information theory) or variance of the decision as an uncertainty measure. The system sets an uncertainty threshold and automatically transfers cases with high uncertainty to human review, achieving more intelligent human-machine collaboration. This mechanism seamlessly integrates with the existing tiered review process, but with more objective and quantifiable decision criteria.

[0105] Through the above steps, DBI-CMKFN not only inherits the advantages of CMAIN and MEELS, but also introduces a probabilistic reasoning framework, which enables the system to make a qualitative leap in handling uncertainty, knowledge fusion, and decision explanation. It is particularly suitable for key content areas that require high-precision review and clear decision-making basis.

[0106] like Figure 5 As shown, the present invention also provides a task review device based on artificial intelligence technology, comprising: A multimodal content acquisition unit, configured to acquire multimodal content to be reviewed in a current task, preprocess the multimodal content, and obtain preprocessed multimodal data; A feature extraction unit, configured to extract text features, image features, and video features from the preprocessed multimodal data, respectively, to generate a multimodal feature vector; A deep learning unit, configured to input the multimodal feature vector into a pre-trained deep learning model, perform fusion analysis on the multimodal feature vector, and output a task review result; A hierarchical review unit, configured to generate a final review decision using different preset strategies based on the confidence level of the task review result; An explainability analysis unit, configured to perform explainability analysis on the final audit decision and generate an audit basis report; A model updating unit is used to collect audit feedback data based on the audit basis report and audit process data, and use the audit feedback data to update the deep learning model.

[0107] The functions of each unit correspond to the corresponding steps in the above method, and will not be repeated here. The device can be implemented as a software system, or a system combining software and hardware, for performing content review tasks in various online platforms.

[0108] It should be noted that those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. If these changes and modifications fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these changes and modifications.

[0109] A computer device according to an embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc.

[0110] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is configured to execute the computer-readable instructions stored in the memory, causing the computer device to execute all or part of the steps of the task review method based on artificial intelligence technology described in each embodiment of the present disclosure.

[0111] Those skilled in the art should understand that in order to solve the technical problem of how to obtain a good user experience, this embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the scope of protection of this disclosure.

[0112] like Figure 6 The present invention provides a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Figure 6 The computer device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0113] like Figure 6 As shown, a computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) or programs loaded from a storage device into a random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0114] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes and hard disks; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 6A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0115] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by the processor, all or part of the steps of the task review method based on artificial intelligence technology of the embodiment of the present disclosure are executed.

[0116] For detailed description of this embodiment, please refer to the corresponding description in the aforementioned embodiments, which will not be repeated here.

[0117] According to a computer-readable storage medium of an embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the task review method based on artificial intelligence technology of each embodiment of the present disclosure are executed.

[0118] The above-mentioned computer-readable storage media include, but are not limited to, optical storage media (e.g., CD-ROMs and DVDs), magneto-optical storage media (e.g., MOs), magnetic storage media (e.g., magnetic tapes or mobile hard disks), media with built-in rewritable non-volatile memory (e.g., memory cards), and media with built-in ROM (e.g., ROM cartridges).

[0119] For detailed description of this embodiment, please refer to the corresponding description in the aforementioned embodiments, which will not be repeated here.

[0120] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0121] In the present disclosure, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "including," "comprising," "having," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0122] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0123] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0124] Various changes, substitutions, and modifications may be made to the technology described herein without departing from the teachings defined by the appended claims. Moreover, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of things, means, methods, and actions described above. Currently existing or later developed processes, machines, manufactures, compositions of things, means, methods, or actions that perform substantially the same function or achieve substantially the same results as the corresponding aspects described herein may be utilized. Accordingly, the appended claims include within their scope such processes, machines, manufactures, compositions of things, means, methods, or actions.

[0125] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0126] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A task review method based on artificial intelligence technology, characterized in that: The task review method includes: Acquire multimodal content to be reviewed in the current task, preprocess the multimodal content, and obtain preprocessed multimodal data; Respectively extracting text features, image features, and video features from the preprocessed multimodal data to generate a multimodal feature vector; Inputting the multimodal feature vector into a pre-trained deep learning model, performing fusion analysis on the multimodal feature vector, and outputting a task review result; Based on the confidence level of the task audit results, different preset strategies are used to generate the final audit decision; Conduct an explainability analysis on the final audit decision and generate an audit basis report; Based on the audit basis report and audit process data, audit feedback data is collected, and the deep learning model is updated using the audit feedback data.

2. The task review method according to claim 1, characterized in that: The extracting text features, image features, and video features from the preprocessed multimodal data to generate a multimodal feature vector includes: Inputting the text data in the preprocessed multimodal data into a pre-trained bidirectional encoder representation model, extracting the text encoding, and obtaining a text feature vector; Inputting the image data in the preprocessed multimodal data into a pre-trained residual network model to extract image features to obtain an image feature vector; Segmenting the video data in the preprocessed multimodal data into segments of fixed length, extracting spatiotemporal features of each segment using a three-dimensional convolutional neural network, and inputting the extracted spatiotemporal feature sequence into a long short-term memory network to obtain a video feature vector; The text feature vector, the image feature vector, and the video feature vector are combined to generate the multimodal feature vector.

3. The task review method according to claim 1, characterized in that: The pre-trained deep learning model includes a multi-input and multi-output deep neural network, and the deep neural network includes an input layer, a feature fusion layer, a multi-layer perceptron and an output layer; Input the multimodal feature vector into a pre-trained deep learning model, perform fusion analysis on the multimodal feature vector, and output the task review results, including: In the feature fusion layer, the multimodal feature vectors are fused using an attention mechanism to generate fused features; In the multi-layer perceptron, the fused features are processed to extract high-level semantic information; In the output layer, high-level semantic information is classified through multi-label classification to obtain content categories and corresponding confidence values to form the task review results.

4. The task review method according to claim 1, characterized in that: Based on the confidence level of the task audit results, different preset strategies are used to generate the final audit decision, including: Setting a confidence threshold to classify the task review result into a high confidence result, a medium confidence result, and a low confidence result; Processing the high-confidence results directly through a fast filtering layer to generate an automatic review decision; Perform secondary analysis on the medium confidence results through the deep analysis layer to generate a deep audit decision; Transferring the low-confidence results or the controversial content in the deep review decision to the manual review layer for manual review and generating a manual review decision; The automatic review decision, the in-depth review decision, and the manual review decision are integrated to form the final review decision.

5. The task review method according to claim 1, characterized in that: Conduct an explainability analysis on the final audit decision and generate an audit basis report, including: Use feature importance analysis to calculate the contribution of each feature to the final review decision and identify key influencing factors; Generate a heat map of the text features, image features, and video features to highlight key areas that influence decision-making; Record the activation status of each layer within the deep learning model, build decision paths, and explain the reasoning process of the deep learning model; Based on the text features, image features, and video features, retrieve historical audit cases similar to the current task as a reference for decision-making; The key influencing factors, the heat map, the decision path and the historical audit cases are integrated to generate the audit basis report.

6. The task review method according to claim 1, characterized in that: Before inputting the multimodal feature vector into the pre-trained deep learning model, the method further includes: Construct a cross-modal attention interaction network; The feature vector of each modality is used as the source of query, key and value to build a multi-head cross attention structure; Compute image features guided by text features, text features guided by image features, and interactive features between text-video and image-video; Design an adaptive weight network to integrate the outputs of each cross-attention head and dynamically adjust the importance weight of the modality relationship based on the content type; The interacted features are used as input to the deep learning model.

7. The task review method according to claim 1, characterized in that: The updating of the deep learning model using the review feedback data includes: The audit feedback data includes task audit results, and different model update strategies are adopted for task audit results with different confidence levels; The audit feedback data is divided into three types: manual audit results, user reporting data, and platform operation data. Differentiated processing methods are adopted according to different types of audit feedback data to update the deep learning model; Different model updating strategies are adopted for task review results with different confidence levels, including: For high-confidence task review results, their automatic review decisions are included as standard samples in the training set for regular incremental training of the model; For the task review results with medium confidence, the in-depth review decision and the feature analysis results of the decision-making process are used to adjust the feature fusion layer parameters of the model; For low-confidence task review results, their manual review decisions are used as key samples and prioritized for the model's active learning process; The updating of the deep learning model by adopting differentiated processing methods according to different types of review feedback data includes: For manual review results, the deep learning model is fine-tuned using transfer learning methods, focusing on optimizing the model's classification boundaries; For user-reported data, we analyze it as a case of model misjudgment and generate adversarial samples to enhance the robustness of the model. For platform operation data, extract content distribution and user interaction patterns, and adjust the model's attention mechanism weight distribution; Maintain the model version library, support model rollback and A / B testing, and ensure the stability of model updates.

8. The task review method according to claim 1, characterized in that: The updating of the deep learning model using the review feedback data includes: Perform cluster analysis on historical audit data, including historical task audit results and corresponding audit feedback data, to identify data features of different types of content patterns or time windows; Train the corresponding expert model for each type of data feature to obtain an expert model pool; Design and train a meta-learner that receives the output of all expert models as input and generates the final decision through an ensemble learning algorithm; Evaluate the processing capabilities of existing expert models based on new data, and selectively update the models or train new expert models; Combining the statistical characteristics of real user data and the decision boundary of the expert model, adversarial samples close to real scenarios are generated.

9. The task review method according to claim 1, characterized in that: Also includes: When the deep learning model is updated using the audit feedback data, a cross-modal knowledge fusion network based on decentralized Bayesian reasoning is constructed, including: Construct a probabilistic feature representation layer and convert the feature vector into a probability distribution representation by adding a variational encoder, so that each feature point is represented as a Gaussian distribution parameterized by mean and variance; Implement a decentralized Bayesian interaction mechanism, transforming the attention calculation process into a Bayesian update process. The probability distribution of each modal feature is used as prior knowledge, and the distribution of each other is updated by calculating the conditional probability to form the posterior distribution. Develop a distributed expert knowledge integration system, transform each expert model into a Bayesian neural network, and integrate the probability outputs of each expert model through the Bayesian model averaging method; Establish a knowledge graph-assisted prior construction system to encode domain rules, regulatory requirements, and historical audit experience into conditional probability tables in Bayesian networks; Design an uncertainty-aware decision-making mechanism, calculate the entropy or variance of the decision as an uncertainty measure based on the probability distribution of the reasoning results, and automatically transfer cases with high uncertainty to manual review.

10. A task review device based on artificial intelligence technology, characterized in that: include: A multimodal content acquisition unit, configured to acquire multimodal content to be reviewed in a current task, preprocess the multimodal content, and obtain preprocessed multimodal data; A feature extraction unit, configured to extract text features, image features, and video features from the preprocessed multimodal data, respectively, to generate a multimodal feature vector; A deep learning unit, configured to input the multimodal feature vector into a pre-trained deep learning model, perform fusion analysis on the multimodal feature vector, and output a task review result; A hierarchical review unit, configured to generate a final review decision using different preset strategies based on the confidence level of the task review result; An explainability analysis unit, configured to perform explainability analysis on the final audit decision and generate an audit basis report; A model updating unit is used to collect audit feedback data based on the audit basis report and audit process data, and use the audit feedback data to update the deep learning model.

Images