Content auditing method and system and electronic equipment
By obtaining the content to be reviewed and retrieving the matching audit strategy set, using multiple audit matters and models to form an audit strategy tree, the problem of insufficient flexibility and timeliness of audit strategy adjustment in the existing technology is solved, and efficient and accurate content review.
Patent Information
- Application Number
- CN202310777775.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-28
- Publication Date
- 2024-12-31
AI Technical Summary
The existing content auditing platform has poor flexibility and timeliness when adjusting audit strategies, and cannot meet the flexible adjustment needs of specific time nodes or event nodes, resulting in inefficiency and insufficient accuracy.
By obtaining the content to be reviewed and retrieving the matching audit strategy set, multiple audit matters and corresponding audit models are used for flexible configuration to form an audit strategy tree to quickly adjust and retrieve audit strategies and improve audit efficiency and accuracy.
The ability to flexibly adjust the audit strategy is realized, the accuracy and efficiency of the audit results are improved, the cost of model training is reduced, and the need for flexible adjustment is met.
Smart Images
Figure CN119226748A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of content review, and in particular to a content review method, system and electronic device. Background Art
[0002] Traditionally, video content review is mainly done manually, which requires checking every frame of the video one by one. This is inefficient and cannot meet the timeliness of media review. On the other hand, manual review relies on the subjective consciousness of the reviewer, and different decisions will be made for some borderline videos. The current trend of media review is to use machine learning models to identify review issues in video content. A large number of service providers have launched content review platforms based on machine learning models. These content review platforms based on machine learning models can improve review efficiency.
[0003] In actual application, the audit strategy of the content audit platform usually changes with factors such as time, scene, audit object and playback object. In particular, for certain time nodes or event nodes such as certain festivals and events, it is usually necessary to increase the corresponding audit items in a targeted manner. Once these time nodes or event nodes have passed, the corresponding audit items need to be revoked. However, if a conventional content audit platform needs to increase or adjust the audit strategy, it usually needs to perform reinforcement learning on the machine learning model so that the machine learning model can identify the corresponding audit items. The flexibility and timeliness are poor, and it cannot meet the needs of flexible adjustment of the audit strategy. Summary of the invention
[0004] In view of this, the embodiments of the present application propose a content review method, system and electronic device, which enable the review strategy to be flexibly adjusted and the review results to be more accurate.
[0005] The first aspect of the present application provides a content review method, comprising:
[0006] Get the content to be reviewed;
[0007] Retrieving an audit policy set that matches the content to be audited, wherein the audit policy set includes one or more audit items and audit policies set for the audit items, and each of the audit policies has a corresponding audit model;
[0008] Based on each of the audit strategies, the corresponding audit model is used to audit the content to be audited, and an audit result is obtained.
[0009] In some embodiments, obtaining the content to be reviewed includes:
[0010] Get the documents to be reviewed;
[0011] One or more contents to be reviewed are extracted from the file to be reviewed according to the data type.
[0012] In some embodiments, the audit items have multiple associated data audit policies, and each of the data audit policies has a one-to-one corresponding data audit model;
[0013] Based on each of the audit strategies, the content to be audited is audited using the corresponding audit model to obtain the audit result, including:
[0014] Based on each of the data audit strategies, the corresponding data audit model is utilized to audit the to-be-audited content of the corresponding data type to obtain the audit result.
[0015] In some embodiments, the audit policy set includes an audit policy tree, the audit items form a second-level node of the audit policy tree, and the data audit policy forms a leaf node that is a child node of the second-level node;
[0016] Based on each of the data audit strategies, using the corresponding data audit model, auditing the to-be-audited content of the corresponding data type, and obtaining the audit result, including:
[0017] The data audit strategies of the leaf nodes are traversed, and the corresponding data audit model is used to audit the contents to be audited of the corresponding data types to obtain the audit results.
[0018] In some embodiments, obtaining the audit results includes:
[0019] Obtaining the audit object identification results output by each of the data audit models that correspond one-to-one to the leaf nodes;
[0020] Determine the item score of the audit items of each of the second-layer nodes based on the data audit strategy and the object identification result;
[0021] Based on the matter scores of each of the second-layer nodes, the audit result corresponding to the root node of the audit policy tree is determined.
[0022] In some embodiments, the content to be reviewed includes image data, the review model includes a feature extraction network and an object recognition network, and the feature extraction network includes a plurality of convolutional layers arranged in sequence; and reviewing the content to be reviewed using the corresponding review model includes:
[0023] Performing convolution processing on the image data through the convolution layer at the head end to output a feature matrix, and performing convolution processing on the feature matrix output by the previous convolution layer through each of the convolution layers after the convolution layer at the head end to output a feature matrix;
[0024] The feature matrix output by the convolutional layer at the end is used as the feature information extracted by the convolutional layer;
[0025] The feature matrix output by each convolution layer before the convolution layer at the end is superimposed with the feature information extracted by the next convolution layer as the feature information extracted by each convolution layer;
[0026] The feature information extracted by each convolutional layer of the feature extraction network is input into the object recognition network, and the object recognition result is output through the object recognition network.
[0027] In some embodiments, obtaining the content to be reviewed includes:
[0028] Acquire a file to be reviewed, metadata for describing the file to be reviewed, and identification information for identifying the file to be reviewed; wherein the metadata at least includes address information of the file to be reviewed;
[0029] Based on the priority of the files to be reviewed, adding identification information of each of the files to be reviewed to a message queue;
[0030] Based on the identification information in the message queue, sequentially obtain the address information of each of the files to be reviewed, and retrieve the corresponding files to be reviewed based on the address information;
[0031] According to the data type, one or more contents to be reviewed are extracted from the file to be reviewed.
[0032] In some embodiments, extracting one or more contents to be reviewed from the file to be reviewed according to the data type includes:
[0033] Extracting text data, image data, audio data and / or video data from the file to be reviewed;
[0034] In the case where the file to be reviewed contains audio data, obtaining corresponding text data based on the audio data;
[0035] In the case that the file to be reviewed contains video data, one or more frames of image data of the video data are acquired.
[0036] The second aspect of the present application provides a content review system, including:
[0037] The acquisition module is used to obtain the content to be reviewed;
[0038] A calling module, used to call an audit policy set matching the content to be audited, wherein the audit policy set includes one or more audit items and an audit policy set for the audit items, and each of the audit policies has a corresponding audit model;
[0039] The audit module is used to audit the content to be audited based on each of the audit strategies using the corresponding audit model to obtain the audit result.
[0040] A third aspect of the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the above method when executing the computer program on the memory.
[0041] The content audit method of the embodiment of the present application obtains the content to be audited and the audit policy set that matches the audit content, and audits the content to be audited using the corresponding audit model based on each audit policy to obtain the audit result. Since each audit item has a corresponding audit policy and a corresponding audit model, the audit items and audit policies contained in the audit policy set can be flexibly configured according to actual needs. Moreover, since each audit model targets fewer audit items, not only is the training time shorter, but the accuracy of the test results is also higher. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0043] Figure 1 A flowchart of a content review method according to an embodiment of the present application;
[0044] Figure 2 A flowchart of a specific embodiment of the content review method of the embodiment of the present application;
[0045] Figure 3 A schematic diagram of a scenario of a candidate set to be reviewed and a message queue in an embodiment of the present application;
[0046] Figure 4 A data structure diagram of the audit policy tree of an embodiment of the present application;
[0047] Figure 5 A schematic diagram of the structure of a feature extraction network according to an embodiment of the present application;
[0048] Figure 6A structural block diagram of a content review system according to an embodiment of the present application;
[0049] Figure 7 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0051] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0052] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of well-known functions and well-known components.
[0053] The present application embodiment provides a content review method, Figure 1 See the flowchart of the content review method of the embodiment of the present application. Figure 1 As shown, the content review method of the embodiment of the present application may specifically include the following steps.
[0054] S110, obtaining the content to be reviewed.
[0055] Optionally, the content to be reviewed may include various contents that need to be made public or released. For example, the content to be reviewed may include various types of data such as video, audio, image, text, web page, application interface, etc.
[0056] Optionally, the content to be reviewed may include the entire content of the release document that needs to be publicly released, or may include part of the content of the release document that needs to be publicly released. For example, when the release document that needs to be publicly released is a text, the text may be used as the content to be reviewed. For another example, when an advertisement needs to be released, characteristic content that can characterize the advertisement file may be extracted from the advertisement file as the content to be reviewed.
[0057] Optionally, the content to be reviewed can be directly obtained from the client or from a storage device of the content review system. For example, the client can upload the content to be reviewed to the content review system, and the content review system can temporarily store the content to be reviewed in a database. When it is determined that the content to be reviewed needs to be reviewed based on the system resource situation and / or task situation of the content review system, the content to be reviewed can be retrieved from the database.
[0058] S120, retrieve an audit policy set that matches the content to be audited. The audit policy set includes one or more audit items and audit policies set for the audit items, and each audit policy has a corresponding audit model.
[0059] Optionally, the audit policy set may be a general audit policy set. For example, the audit policy set may be a general audit policy set set for ordinary users by the content audit system according to the current time node or event node. Alternatively, the audit policy set may also be an audit policy set related to the client. For example, the client may have the authority to configure its own audit policy set. The client may send a policy setting request to the content audit system based on the sensitivity of the content it publishes, the company's own positioning and development needs, etc., and the content request system sets the audit policy set corresponding to the client. Alternatively, the audit policy set may also match the technical field, data type or publishing method to which the content to be audited belongs.
[0060] Optionally, the audit strategy may include a data processing method, audit process, assessment criteria, etc. for auditing whether the audit content has corresponding audit items using a corresponding audit model. The audit strategy may have one corresponding audit model or multiple corresponding audit models.
[0061] The audit model is set for the corresponding audit items, and is used to audit whether there are corresponding audit items for the content to be audited. Optionally, the audit model can be used to audit whether the content to be audited has a single audit item, and can also be used to audit whether the content to be audited has multiple audit items corresponding to it. For example, in order to improve the flexibility of the audit policy configuration, each audit model can be set to audit a single audit item. In this way, the service provider or user can flexibly combine audit policies according to needs. For example, a conventional audit model and an unconventional audit model can be set, and the conventional audit model can be used to audit multiple common and conventional audit items, and the unconventional audit model can be used to audit one or more unconventional audit items. In this way, the flexibility of the audit policy configuration can be maintained, and it is beneficial to reduce the cost of model training.
[0062] Optionally, the audit model may adopt various types of machine learning models, including but not limited to regional convolutional neural network (R-CNN), fast regional convolutional neural network (Fast R-CNN), Faster R-CNN network (Faster Region-based Convolutional Network), etc. The type of machine learning model adopted by the audit model is not limited here. As long as it can audit whether there are audit items in the audited content, various types of machine learning models can be selected according to actual needs.
[0063] S130, based on each of the audit strategies, using the corresponding audit model to audit the content to be audited, and obtaining an audit result.
[0064] When the content to be audited and the audit policy set are obtained, the audit model can be used to audit the content to be audited based on the corresponding audit policy, obtain the audit item identification results output by each audit model, and obtain the final audit result based on one or more audit item identification results.
[0065] Optionally, the audit item identification result may include the audit rating, audit number, etc. of the audit item that triggers the corresponding audit item. When the audit item identification result is obtained, the audit score of the audit item can be determined based on the scoring criteria in the audit strategy. Then, the final audit result can be determined based on the weight of each audit item and the audit score of each audit item. Of course, in actual application, other methods can also be used to determine the audit result, not limited to the weight-based calculation method.
[0066] The content audit method of the embodiment of the present application obtains the content to be audited and the audit policy set that matches the audit content, and audits the content to be audited using the corresponding audit model based on each audit policy to obtain the audit result. Since each audit item has a corresponding audit policy and a corresponding audit model, the audit items and audit policies contained in the audit policy set can be flexibly configured according to actual needs. Moreover, since each audit model targets fewer audit items, not only is the training time shorter, but the accuracy of the test results is also higher.
[0067] In some embodiments, step S110, obtaining the content to be reviewed, may include: obtaining a file to be reviewed; and extracting one or more contents to be reviewed from the file to be reviewed according to the data type.
[0068] Optionally, the file to be reviewed may include a source file of a published file that needs to be publicly released, or a file containing the main content of the published file. The file to be reviewed may be uploaded to the content review system through a client. For example, the file to be reviewed may be uploaded to the content review system through a web page or an application.
[0069] Optionally, the data types include but are not limited to text data, image data, audio data, video data, etc. For example, when an advertisement file is acquired, video data, audio data, text data, image data, etc. may be extracted from the advertisement file.
[0070] In the case that the to-be-audited file contains audio data, corresponding text data is obtained based on the audio data. For example, the audio data may be converted into text data using a speech recognition model.
[0071] In the case where the to-be-audited file contains video data, one or more frames of image data of the video data are obtained. Optionally, uniform frames can be extracted from the video data, that is, frame image data can be sequentially extracted from the video data at specific time intervals.
[0072] Optionally, key frame image data may also be extracted from the video data. Key frames may include complete frames (I frames), single prediction frames (P frames) and bidirectional prediction frames (B frames). A complete frame (I frame) refers to an independent frame in the video data with a complete image and all information, and may be a frame of a static picture in the video data. A single prediction frame (P frame) is represented by defining the difference between the current frame and the previous frame. When decoding, the previous frame needs to be superimposed with the difference defined by the current frame. A bidirectional prediction frame (B frame) is represented by defining the difference between the current frame and the two frames before and after. When decoding, the previous and next frame pictures need to be decoded, and the difference is determined by the difference between the previous and next frame pictures and the current frame.
[0073] Optionally, a scene transition frame in the video data may also be extracted. For example, a scene difference threshold may be set to determine whether the difference between two adjacent frames of image data is greater than the scene difference threshold. If so, the latter of the two adjacent frames is determined to be a scene transition frame.
[0074] It can be understood that in actual applications, more than two items of frame image data from uniform frames, complete frames, single-prediction frames, double-prediction frames, and scene transition frames can be extracted from the video data, or other frame image data from the video data can be extracted, as long as the acquired frame image data can represent the characteristic content of the video data.
[0075] In some embodiments, step S110, obtaining the content to be reviewed, may include:
[0076] Acquire a file to be reviewed, metadata for describing the file to be reviewed, and identification information for identifying the file to be reviewed; wherein the metadata at least includes address information of the file to be reviewed;
[0077] Based on the priority of the files to be reviewed, adding identification information of each of the files to be reviewed to a message queue;
[0078] Based on the identification information in the message queue, sequentially obtain the address information of each of the files to be reviewed, and retrieve the corresponding files to be reviewed based on the address information;
[0079] According to the data type, one or more contents to be reviewed are extracted from the file to be reviewed.
[0080] Optionally, the metadata may include various data used to describe the file to be reviewed. For example, when the file to be reviewed is a video file, the metadata may include the name, resolution, frame rate, video encoding, audio encoding, and duration of the video file. When the file to be reviewed contains image data, the metadata may include image size, shooting time, aperture and shutter information, address information, and the like.
[0081] Optionally, the identification information may include coding information capable of shift-identifying the file to be reviewed. For example, when the content review system obtains the file to be reviewed, it may assign a unique code to each file to be reviewed as the identification information of the file to be reviewed.
[0082] Optionally, the priority of the file to be reviewed can be determined based on multiple factors, such as the release time, field, data type of the file to be reviewed, and the level of the client requesting the review of the file to be reviewed.
[0083] Exemplary, with Figure 2As shown in FIG. 1 , when the to-be-reviewed file and the identification information are obtained, the to-be-reviewed file can be uploaded to the database. The address information of the to-be-reviewed file can be obtained, for example, the uniform resource locator (URL) of the to-be-reviewed file can be obtained and the URL can be used as part of the metadata. The identification information can also be added to the to-be-reviewed candidate set, such as Figure 3 shown.
[0084] After that, the priority of the pending files can be determined based on the release time and file type of the pending files. For example, the pending files with earlier release time can have higher priority, with file type 1 having the highest priority, followed by file type 2, and file type 3 having the third highest priority. Figure 3 Then, the identification information in the candidate set to be reviewed can be added to the message queue.
[0085] According to the system resource situation and task situation, the identification information is taken out from the message queue one by one, the URL is obtained based on the identification information, and the file to be reviewed is obtained based on the URL. Then, video data, audio data, image data, text data, etc. are extracted from the file to be reviewed. In this way, each file to be reviewed can be scheduled in an orderly manner, which can avoid system congestion and review each file to be reviewed in time.
[0086] In some embodiments, the audit items have multiple associated data audit policies, and each of the data audit policies has a one-to-one corresponding data audit model.
[0087] Step S130, based on each of the audit strategies, using the corresponding audit model to audit the content to be audited, and obtaining the audit result, may include:
[0088] S131, based on each of the data audit strategies, using the corresponding data audit model, audit the to-be-audited content of the corresponding data type to obtain the audit result.
[0089] Setting corresponding data audit models for different data types of content to be audited can not only improve the accuracy of the test results. Moreover, each data audit model targets a single input data type, making the data audit model simpler and more efficient, which is beneficial to reducing service costs.
[0090] Exemplarily, each of the audit items may include an image audit model and a text audit model. After obtaining the file to be audited, images, text, video and audio may be extracted from the file to be audited, and frame images of the video file may be obtained, and the audio may be converted into text. Afterwards, the image is audited based on the image audit model, and the text is audited based on the text audit model.
[0091] In some embodiments, the audit policy set includes an audit policy tree, the audit items form a second-level node of the audit policy tree, and the data audit policy forms a leaf node that is a child node of the second-level node, such as Figure 4 shown.
[0092] Step S131, based on each of the data audit strategies, using the corresponding data audit model, auditing the content to be audited of the corresponding data type, and obtaining the audit result, may include: traversing the data audit strategies of each of the leaf nodes, and using the corresponding data audit model, auditing the content to be audited of the corresponding data type, and obtaining the audit result.
[0093] The audit strategy tree is highly flexible. When adding or reducing audit items, you only need to add or remove the corresponding branches from the corresponding second-level nodes, without adjusting other branches. Moreover, the audit strategy tree itself is highly logical, which is conducive to the rapid retrieval of various data audit strategies and helps improve audit efficiency.
[0094] In some embodiments, step S130, obtaining the audit result, may include:
[0095] Obtaining object recognition results output by each of the data audit models that correspond one-to-one to the leaf nodes;
[0096] Determine the item score of the audit items of each of the second-layer nodes based on the data audit strategy and the object identification result;
[0097] Based on the matter scores of each of the second-layer nodes, the audit result corresponding to the root node of the audit policy tree is determined.
[0098] Exemplary, with Figure 4 As shown, the first text audit model corresponding to the first audit strategy can audit the text data to determine the vocabulary, rating, and number of occurrences of the first audit item in the text data. The first image audit model corresponding to the first audit strategy can audit the image data to determine the pattern, rating, pattern size, and number of occurrences of the pattern in the image data that are related to the first audit item. Similarly, the object recognition results output by the second text audit model and the second pattern audit model are also obtained.
[0099] Each of the data audit strategies may record a scoring standard and the weight of the data audit strategy in the second-level node of the audit strategy tree. The item score of the audit item of the second-level node may be determined based on the scoring standard and the object identification result. For example, the item score involving the first audit item and the item score involving the second audit item may be determined separately.
[0100] Then, the audit result at the root node of the audit strategy tree can be calculated by weighted calculation based on the weights of each second-layer node. Of course, the audit result can also include the audit object identification result. For example, the vocabulary, rating, and number of occurrences of the first audit item are summarized. It can be seen that configuring the audit strategy set to include an audit strategy tree is not only conducive to the audit process, but also conducive to the calculation and aggregation of audit results.
[0101] It should be noted that the above-mentioned scoring and audit result aggregation methods are only exemplary. In specific implementations, the audit results are not limited to the above-mentioned methods.
[0102] In some embodiments, the content to be reviewed includes image data, the review model includes a feature extraction network and a target recognition network, and the feature extraction network includes a plurality of convolutional layers arranged in sequence.
[0103] Step S130, reviewing the content to be reviewed using the corresponding review model, may include:
[0104] Performing convolution processing on the image data through the convolution layer at the head end to output a feature matrix, and performing convolution processing on the feature matrix output by the previous convolution layer through each of the convolution layers after the convolution layer at the head end to output a feature matrix;
[0105] The feature matrix output by the convolutional layer at the end is used as the feature information extracted by the convolutional layer;
[0106] The feature matrix output by each convolution layer before the convolution layer at the end is superimposed with the feature information extracted by the next convolution layer as the feature information extracted by each convolution layer;
[0107] The feature information extracted by each convolutional layer of the feature extraction network is input into the target recognition network, and the object recognition result is output through the target recognition network.
[0108] Exemplary, with Figure 5 As shown, the feature extraction network may include a first convolution layer C1, a second convolution layer C2, a third convolution layer C3, a fourth convolution layer C4 and a fifth convolution layer C5, a total of five convolution layers arranged in sequence. The first convolution layer C1 is a 1×1 convolution layer, which is used to adjust the channel data of the image data to form a first feature matrix X1. Afterwards, the feature matrix output by the previous convolution layer is convolved through C2, C3, C4 and C5 in sequence to form a second feature matrix X2, a third feature matrix X3, a fourth feature matrix X4 and a fifth feature matrix X5.
[0109] The fifth feature matrix X5 output by the fifth convolutional layer C5 is used as the fifth feature map P5, and the fifth feature map P5 is upsampled so that the upsampled fifth feature map P5 has the same size as the fourth feature matrix X4, and P5 and X4 are superimposed to form the fourth feature map P4. Similarly, P4 and X3 are superimposed to form the third feature map P3, and P3 and X2 are superimposed to form the second feature map P2.
[0110] In order to achieve multi-scale detection effects, the fused features {P2, P3, P4, P5} are used as input data of the target recognition network. The target recognition network uses the feature map of the last layer of the network to classify and regress the k anchor frames of preset sizes mapped from the original image to the feature map. These k anchor frames are pre-set to a fixed aspect ratio, and the feature map of the same level is used for calculation in RPN. In order to fuse multiple feature scales, the anchor frame matches the corresponding anchor frame on different feature maps while maintaining the original preset size and scale. The anchor frames are distributed on the four feature layers P2 to P5, and the anchor frames assigned to each feature map have three scales, namely 1:1, 1:2, and 2:1.
[0111] After changing the region proposal generated by the target recognition network, the anchor frame generated by the target recognition network needs to be projected onto feature maps of different levels. The smaller the size of the anchor frame, the more it needs to be projected onto the lower-level feature map in order to prevent information loss on the smaller-scale high-level feature map. The mapping relationship between the feature map and the target recognition network can be expressed as the following formula:
[0112]
[0113] Among them, P identifies the position of the anchor box, k0 represents the input level; k is the input image scale, w and h represent the width and height of the region of interest, that is, the width and height of the RoI (Region of Interest) area. The target recognition network identifies whether there is an audit object in the image data based on P2, P3, P4 and P5 respectively. Only when the audit object is detected based on any one of the feature maps of P2, P3, P4 and P5, it can be determined that the image data contains an audit object. In this way, the problem of missing small targets due to only using the feature map output by the last layer for target recognition can be avoided.
[0114] The present application also provides a content review system. Figure 6 For a structural diagram of the content review system of the present application embodiment, see Figure 6 As shown, the content review system of the embodiment of the present application may include:
[0115] The acquisition module 201 is used to acquire the content to be reviewed;
[0116] The retrieving module 202 is used to retrieve the audit policy set that matches the content to be audited, wherein the audit policy set includes one or more audit items and the audit policies set for the audit items, and each of the audit policies has a corresponding audit model;
[0117] The audit module 203 is used to audit the content to be audited based on each of the audit strategies using the corresponding audit model to obtain the audit result.
[0118] In some embodiments, the acquisition module 201 is specifically used for:
[0119] Get the documents to be reviewed;
[0120] One or more contents to be reviewed are extracted from the file to be reviewed according to the data type.
[0121] In some embodiments, the audit items have multiple associated data audit strategies, and each of the data audit strategies has a one-to-one corresponding data audit model; the audit module 203 is specifically used for:
[0122] Based on each of the data audit strategies, the corresponding data audit model is utilized to audit the to-be-audited content of the corresponding data type to obtain the audit result.
[0123] In some embodiments, the audit policy set includes an audit policy tree, the audit items form a second-layer node of the audit policy tree, and the data audit policy forms a leaf node as a child node of the second-layer node; the audit module 203 is specifically used to:
[0124] The data audit strategies of the leaf nodes are traversed, and the corresponding data audit model is used to audit the contents to be audited of the corresponding data types to obtain the audit results.
[0125] In some embodiments, the audit module 203 is specifically used to:
[0126] Obtaining object recognition results output by each of the data audit models that correspond one-to-one to the leaf nodes;
[0127] Determine the item score of the audit items of each of the second-layer nodes based on the data audit strategy and the object identification result;
[0128] Based on the matter scores of each of the second-layer nodes, the audit result corresponding to the root node of the audit policy tree is determined.
[0129] In some embodiments, the content to be reviewed includes image data, the review model includes a feature extraction network and an object recognition network, the feature extraction network includes a plurality of convolutional layers arranged in sequence; the review module 203 is specifically used for:
[0130] Performing convolution processing on the image data through the convolution layer at the head end to output a feature matrix, and performing convolution processing on the feature matrix output by the previous convolution layer through each of the convolution layers after the convolution layer at the head end to output a feature matrix;
[0131] The feature matrix output by the convolutional layer at the end is used as the feature information extracted by the convolutional layer;
[0132] The feature matrix output by each convolution layer before the convolution layer at the end is superimposed with the feature information extracted by the next convolution layer as the feature information extracted by each convolution layer;
[0133] The feature information extracted by each convolutional layer of the feature extraction network is input into the object recognition network, and the object recognition result is output through the object recognition network.
[0134] In some embodiments, the acquisition module 201 is specifically used for:
[0135] Acquire a file to be reviewed, metadata for describing the file to be reviewed, and identification information for identifying the file to be reviewed; wherein the metadata at least includes address information of the file to be reviewed;
[0136] Based on the priority of the files to be reviewed, adding identification information of each of the files to be reviewed to a message queue;
[0137] Based on the identification information in the message queue, sequentially obtain the address information of each of the files to be reviewed, and retrieve the corresponding files to be reviewed based on the address information;
[0138] According to the data type, one or more contents to be reviewed are extracted from the file to be reviewed.
[0139] In some embodiments, the acquisition module 201 is specifically used for:
[0140] Extracting text data, image data, audio data and / or video data from the file to be reviewed;
[0141] In the case where the file to be reviewed contains audio data, obtaining corresponding text data based on the audio data;
[0142] In the case that the file to be reviewed contains video data, one or more frames of image data of the video data are acquired.
[0143] See also Figure 7 As shown, an embodiment of the present application further provides an electronic device, comprising at least a memory 301 and a processor 302, wherein the memory 301 stores a program, and the processor 302 implements the method described in any of the above embodiments when executing the program on the memory 301.
[0144] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions in the computer-readable storage medium are executed, the method described in any of the above embodiments is implemented.
[0145] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, electronic equipment, computer-readable storage media or computer program products. Therefore, the present application may adopt the form of complete hardware embodiments, complete software embodiments or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code. When implemented by software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.
[0146] The processor may be a general purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. The general purpose processor may be a microprocessor or any conventional processor.
[0147] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
[0148] The above-mentioned readable storage medium can be a disk, an optical disk, a DVD, a USB, a read-only memory (ROM) or a random access memory (RAM), etc. This application does not limit the specific form of the storage medium.
[0149] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.
Claims
1. A content review method, characterized in that: include: Get the content to be reviewed; Retrieving an audit policy set that matches the content to be audited, wherein the audit policy set includes one or more audit items and audit policies set for the audit items, and each of the audit policies has a corresponding audit model; Based on each of the audit strategies, the corresponding audit model is used to audit the content to be audited, and an audit result is obtained.
2. The method according to claim 1, characterized in that The obtaining of the content to be reviewed includes: Get the documents to be reviewed; One or more contents to be reviewed are extracted from the file to be reviewed according to the data type.
3. The method according to claim 2, characterized in that The audit items have multiple associated data audit strategies, and each of the data audit strategies has a one-to-one corresponding data audit model; Based on each of the audit strategies, the content to be audited is audited using the corresponding audit model to obtain the audit result, including: Based on each of the data audit strategies, the corresponding data audit model is utilized to audit the to-be-audited content of the corresponding data type to obtain the audit result.
4. The method according to claim 3, characterized in that The audit policy set includes an audit policy tree, the audit items form a second-layer node of the audit policy tree, and the data audit policy forms a leaf node that is a child node of the second-layer node; Based on each of the data audit strategies, using the corresponding data audit model, auditing the to-be-audited content of the corresponding data type, and obtaining the audit result, including: The data audit strategies of the leaf nodes are traversed, and the corresponding data audit model is used to audit the contents to be audited of the corresponding data types to obtain the audit results.
5. The method according to claim 4, characterized in that Get audit results, including: Obtaining object recognition results output by each of the data audit models that correspond one-to-one to the leaf nodes; Determine the item score of the audit items of each of the second-layer nodes based on the data audit strategy and the object identification result; Based on the matter scores of each of the second-layer nodes, the audit result corresponding to the root node of the audit policy tree is determined.
6. The method according to claim 1, characterized in that The content to be reviewed includes image data, the review model includes a feature extraction network and an object recognition network, and the feature extraction network includes a plurality of convolutional layers arranged in sequence; and reviewing the content to be reviewed using the corresponding review model includes: Performing convolution processing on the image data through the convolution layer at the head end to output a feature matrix, and performing convolution processing on the feature matrix output by the previous convolution layer through each of the convolution layers after the convolution layer at the head end to output a feature matrix; The feature matrix output by the convolutional layer at the end is used as the feature information extracted by the convolutional layer; The feature matrix output by each convolution layer before the convolution layer at the end is superimposed with the feature information extracted by the next convolution layer as the feature information extracted by each convolution layer; The feature information extracted by each convolutional layer of the feature extraction network is input into the object recognition network, and the object recognition result is output through the object recognition network.
7. The method according to claim 6, characterized in that The feature extraction network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer which are arranged in sequence; The first convolutional layer is used to adjust the number of channels of the image data and output a first feature matrix; The second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer sequentially perform convolution processing on the feature matrix output by the previous convolutional layer, output a second feature matrix, a third feature matrix, a fourth feature matrix and a fifth feature matrix, use the fifth feature matrix as a fifth feature map, superimpose the fifth feature map with the fourth feature matrix to form a fourth feature map, superimpose the fourth feature map with the third feature matrix to form a third feature map, and superimpose the third feature map with the second feature matrix to form a second feature map.
8. The method according to claim 1, characterized in that The obtaining of the content to be reviewed includes: Acquire a file to be reviewed, metadata for describing the file to be reviewed, and identification information for identifying the file to be reviewed; wherein the metadata at least includes address information of the file to be reviewed; Based on the priority of the files to be reviewed, adding identification information of each of the files to be reviewed to a message queue; Based on the identification information in the message queue, sequentially obtain the address information of each of the files to be reviewed, and retrieve the corresponding files to be reviewed based on the address information; According to the data type, one or more contents to be reviewed are extracted from the file to be reviewed.
9. The method according to claim 8, characterized in that The step of extracting one or more contents to be reviewed from the file to be reviewed according to the data type includes: Extracting text data, image data, audio data and / or video data from the file to be reviewed; In the case where the file to be reviewed contains audio data, obtaining corresponding text data based on the audio data; In the case that the file to be reviewed contains video data, one or more frames of image data of the video data are acquired.
10. A content review system, characterized in that: include: The acquisition module is used to obtain the content to be reviewed; A calling module, used to call an audit policy set matching the content to be audited, wherein the audit policy set includes one or more audit items and an audit policy set for the audit items, and each of the audit policies has a corresponding audit model; The audit module is used to audit the content to be audited based on each of the audit strategies using the corresponding audit model to obtain the audit result.
11. An electronic device comprising at least a memory and a processor, wherein a computer program is stored in the memory, wherein: The processor implements the method of any one of claims 1 to 9 when executing the computer program on the memory.