Video review model training method, video review method and related device

Abstract

This application discloses a video review model training method, a video review method, and related apparatus, including: acquiring a first training image set belonging to a total violation category and a second training image set belonging to a sub-violation category; training a first video review model using the first training image set and training a second video review model using the second training image set; during video review, the first video review model outputs a first violation score indicating that the input video image belongs to the total violation category, and the second video review model outputs a second violation score indicating that the input video image belongs to a sub-violation category when the first violation score is within a preset score range. This method can identify violations with the same violation standards across different countries or regions using the first video review model, and can also identify violations with different standards across different countries or regions using the second video review model, enabling regionalized video push for manual review and improving review efficiency.

Description

Technical Field

[0001] This application relates to the field of content moderation technology, and in particular to a video moderation model training method, a video moderation method, and related apparatus. Background Technology

[0002] With the development of internet technology, various video platforms have entered people's daily lives. People can watch or upload videos through short video, live streaming and other video platforms. However, due to the diversity of videos, in order to maintain a healthy online environment, it is necessary to review whether there is any illegal content in the videos.

[0003] Currently, video review is mainly conducted through neural networks. The process involves inputting a video into the neural network to obtain a score indicating whether it violates regulations. Videos with scores exceeding a preset threshold are then pushed to a human review terminal for further manual review. However, different countries and regions have different standards for identifying violations. Setting the preset threshold too high, while ensuring that videos with high violation standards are pushed to human review to determine the major violation category, will miss low-scoring videos from countries or regions with lower violation standards. Conversely, setting the preset threshold too low, while covering countries or regions with low violation standards, undoubtedly increases the number of videos requiring manual review for countries or regions with high violation standards, leading to a decrease in review efficiency. Summary of the Invention

[0004] This application provides a video review model training method, a video review method, and related apparatus to solve the problem in the prior art that it is impossible to push videos for manual review based on the violation standards of different countries or regions.

[0005] In a first aspect, embodiments of this application provide a video review model training method, including:

[0006] Obtain a first training image set and a second training image set, wherein the first training image set is the image set belonging to the total violation category, and the second training image set is the image set belonging to the sub-violation category;

[0007] Initialize the first video review model and the second video review model;

[0008] The first video review model is trained using the first training image set, and the second video review model is trained using the second training image set;

[0009] Specifically, when the video review model is used for video review, the first video review model is used to output a first violation score of the video image belonging to the total violation category after inputting a video image, and the second video review model is used to output a second violation score of the video image belonging to a sub-violation category when inputting a video image whose first violation score is within a preset score range.

[0010] Secondly, embodiments of this application provide a video review method, including:

[0011] Extract video images from the video data to be reviewed;

[0012] The video image is input into a pre-trained first video review model to obtain the first violation score of the total violation category of the video image;

[0013] Determine whether the first violation score is greater than the first threshold;

[0014] If so, send the video data to be reviewed to the review client;

[0015] If not, when the first violation score is greater than the second threshold, the video image is input into a pre-trained second video review model to obtain the second violation score of the video image belonging to the sub-violation category;

[0016] When the second violation score is greater than the third threshold, the video data to be reviewed is sent to the review client;

[0017] The first video review model and the second video review model are trained using the video review model training method described in the first aspect.

[0018] Thirdly, embodiments of this application provide a video review model training device, including:

[0019] The training image acquisition module is used to acquire a first training image set and a second training image set, wherein the first training image set is an image set belonging to the total violation category, and the second training image set is an image set belonging to the sub-violation category;

[0020] The model initialization module is used to initialize the first video review model and the second video review model.

[0021] The model training module is used to train the first video review model using the first training image set, and to train the second video review model using the second training image set.

[0022] Specifically, when the video review model is used for video review, the first video review model is used to output a first violation score of the video image belonging to the total violation category after inputting a video image, and the second video review model is used to output a second violation score of the video image belonging to a sub-violation category when inputting a video image whose first violation score is within a preset score range.

[0023] Fourthly, embodiments of this application provide a video review device, including:

[0024] The video image extraction module is used to extract video images from the video data to be reviewed;

[0025] The total violation category prediction module is used to input the video image into a pre-trained first video review model to obtain the first violation score of the video image belonging to the total violation category;

[0026] The first violation score determination module is used to determine whether the first violation score is greater than the first threshold.

[0027] The first submission module is used to send the video data to be reviewed to the review client;

[0028] The sub-violation category prediction module inputs the video image into a pre-trained second video review model when the first violation score is greater than the second threshold to obtain the second violation score of the video image belonging to the sub-violation category;

[0029] The second submission module is used to send the video data to be reviewed to the review client when the second violation score is greater than the third threshold.

[0030] The first video review model and the second video review model are trained using the video review model training method described in the first aspect.

[0031] Fifthly, embodiments of this application provide an electronic device, the electronic device comprising:

[0032] One or more processors;

[0033] Storage device for storing one or more computer programs.

[0034] When the one or more computer programs are executed by the one or more processors, the one or more processors implement the video review model training method of the first aspect of this application, and / or the video review method of the second aspect.

[0035] Sixthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video review model training method described in the first aspect of embodiments of this application, and / or the video review method described in the second aspect.

[0036] In a seventh aspect, embodiments of this application provide a computer program product that, when instructions in the computer program product are executed by a processor, implements the video review model training method described in the first aspect, and / or the video review method described in the second aspect.

[0037] In implementation, this application, for each type of violation, categorizes violations with the same violation standards across different countries or regions into violations belonging to the overall violation category, and categorizes violations with different violation standards across different countries or regions into violations belonging to sub-violation categories. After training a first video review model using a first training image set belonging to the overall violation category and a second training image set belonging to the sub-violation categories, the first video review model learns to identify violations with the same violation standards across different countries or regions in the images, and the second video review model learns to identify violations with different violation standards across different countries or regions in the images. During video review, after the first video review model outputs the first violation score for the video image belonging to the overall violation category, videos with higher first violation scores can be pushed to various platforms. The system uses a national or regional manual review terminal to conduct manual reviews. Videos with a first violation score within a preset range are input into a second video review model to obtain a second violation score for the sub-violation category. Videos with higher second violation scores are then sent to manual review terminals in countries or regions with different violation standards for further review. This approach can identify violations with the same violation standards in different countries or regions through the first video review model, and also identify violations with different standards in different countries or regions through the second video review model. This avoids the problem of a surge in the number of submitted videos and low review efficiency caused by pushing all videos with a first violation score greater than a certain low value to all countries or regions for manual review in order to achieve full coverage. It achieves regionalized push of videos for manual review, thus improving the efficiency of video review. Attached Figure Description

[0038] Figure 1 This is a flowchart of a video review model training method provided in Embodiment 1 of this application;

[0039] Figure 2A This is a flowchart of a video review model training method provided in Embodiment 2 of this application;

[0040] Figure 2B This is a schematic diagram of the network structure of the first video review model and the second video review model in the embodiments of this application;

[0041] Figure 3 This is a flowchart of a video review method provided in Embodiment 3 of this application;

[0042] Figure 4 This is a structural block diagram of a video review model training device provided in Embodiment 4 of this application;

[0043] Figure 5 This is a structural block diagram of a video review device provided in Embodiment 5 of this application;

[0044] Figure 6 This is a structural block diagram of an electronic device provided in Embodiment Six of this application. Detailed Implementation

[0045] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit its scope. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0046] Video review checks whether videos contain prohibited content. Prohibited content can be anything banned in different countries or regions. Different countries or regions have different standards for prohibited content. In one example, for each type of violation, content with the same violation standard in different countries or regions can be defined as a violation belonging to the overall violation category, while content with different violation standards in different countries or regions can be defined as a violation belonging to a sub-violation category. That is, a violation belonging to a sub-violation category must belong to the overall violation category. Prohibited content in a video can include the appearance of people under a specified age, the appearance of prohibited items, the appearance of sensitive symbols, the appearance of prohibited actions, etc.

[0047] Taking the appearance of individuals under a specified age (14, 16, 18, etc.) in a video as an example of a violation, for most countries or regions, the appearance of individuals under the specified age in a video is considered a violation, and such violations are generally punished, such as account bans, permanent bans, bans for one week, or bans for one month. Therefore, the general violation category can be defined as: the appearance of individuals under the specified age in a video. However, for some countries or regions, the appearance of individuals under the specified age in a video is considered a violation, so the sub-violation category can be defined as: the appearance of individuals under the specified age in a video. Furthermore, for some other countries or regions, the appearance of both individuals under the specified age and individuals over the specified age in a video is also considered a violation, so the sub-violation category can be defined as: the appearance of both individuals under the specified age and individuals over the specified age in a video. Other violations, such as the appearance of prohibited items, sensitive symbols, or prohibited actions, can be classified into general and sub-violation categories according to the specific country or region. This application embodiment does not impose any restrictions on this.

[0048] In existing technologies, taking the detection of violations involving individuals under a specified age in videos as an example, the typical approach is to input the video image into a model and obtain a violation score for each such individual. If only individuals under the specified age appear, the violation score is high; if both individuals under and over the specified age appear, the violation score decreases. If only videos with high violation scores are pushed to human review terminals for manual review—for example, pushing videos with scores greater than 0.8 out of 1 to human review terminals in all countries or regions—while these videos are highly likely to be violations, the approach of considering videos containing both individuals under and over the specified age as violations by certain countries or regions is flawed. For certain regions, the current system overlooks videos that simultaneously feature individuals under a specified age and those over a specified age (e.g., videos with a score of 0.5-0.8). While pushing videos with scores greater than 0.5 to all countries or regions ensures accurate manual review for those regions where both are considered violations, it necessitates reviewing videos featuring only individuals under a specified age for those regions where only such individuals are considered violations. This results in inaccurate submissions, increases the number of videos requiring manual review, and reduces review efficiency. To address these issues, this application provides the following embodiments.

[0049] Example 1

[0050] Figure 1This is a flowchart illustrating a video review model training method provided in Embodiment 1 of this application. This embodiment is applicable to training a video review model to review video data for violations. The method can be executed by the video review model training device implemented in this application. This device can be implemented in hardware or software and integrated into the electronic device provided in this embodiment. Specifically, as shown... Figure 1 As shown, the video review model training method of this application embodiment may include the following steps:

[0051] S101. Obtain the first training image set and the second training image set. The first training image set is the image set belonging to the total violation category, and the second training image set is the image set belonging to the sub-violation category.

[0052] Video violations can be caused by the presence of prohibited content in the video. Prohibited content can be content that is prohibited in various countries or regions. In one example, prohibited content could be the presence of people under a specified age, prohibited items, sensitive symbols, or sensitive actions in the video.

[0053] For each type of violation, different countries or regions have different standards for identifying the violation. Violations with the same standards in different countries or regions can be classified into general violation categories, while violations with different standards in different countries or regions can be classified into sub-violation categories. That is, violations belonging to sub-violation categories must belong to general violation categories.

[0054] Taking the appearance of individuals under a specified age (14, 16, 18, etc.) in a video as an example of a violation, the overall violation category can be defined as: the appearance of individuals under the specified age in the video. Sub-violation categories include: the appearance of both individuals under and over the specified age in the video, or the appearance of only individuals under the specified age in the video. Other types of violations can be defined according to the standards of each country or region, and this application does not impose such limitations.

[0055] In practical applications, a first training image set and a second training image set can be obtained from existing manually reviewed video clips. Specifically, for manually reviewed video clips, if the video clip contains illegal content, all video images containing illegal content in the video clip can be classified into the total illegal category, and the video images are added to the first training image set. Then, images are selected from the first training image set and added to the second training image set according to the penalty level of the video data. For example, the penalty level for only people under a specified age is the first penalty level, and the penalty level for both people under a specified age and people under a specified age is the second penalty level. Images with the penalty level of the first penalty level can be added to the second training image set, or images with both people under a specified age and people under a specified age can be added to the second training image set. Different video review models can be trained for different countries or regions using different second training image sets.

[0056] Of course, in practical applications, the first training image set belonging to the total violation category and the second training image set belonging to the sub-violation category can also be manually selected directly from a large amount of video data. For example, images in which people under a specified age appear can be manually labeled as the first training images, images in which people under a specified age and people over a specified age appear simultaneously can be labeled as the second training images, and images in which only people under a specified age appear can be labeled as the second training images, etc. The embodiments of this application do not limit the method of obtaining the first training image set and the second training image set.

[0057] S102. Initialize the first video review model and the second video review model.

[0058] In this embodiment of the application, the first video review model is used to identify whether content belonging to the overall violation category appears in the image, and the second video review model is used to identify whether content belonging to the sub-violation category appears in the image. The first video review model and the second video review model can be various neural networks such as CNN, RNN, and DNN. The structures of the first video review model and the second video review model can be the same or different.

[0059] Initializing the first and second video review models can involve building the network structures of the first and second video review models, and initializing the model parameters, etc.

[0060] It should be noted that different second training image sets can be obtained based on the differentiated violation identification standards of different countries or regions, so as to train different video review models for these different countries or regions respectively. That is, for a violation, there can be one first video review model, which is applicable to all countries or regions, so as to identify the violation content with the same violation identification standards in all countries or regions. There can be multiple second video review models, each of which is applicable to countries or regions with different violation identification standards, so as to identify the differentiated violation content identified by the country or region with different violation identification standards.

[0061] S103. Train the first video review model using the first training image set, and train the second video review model using the second training image set.

[0062] The images in the first and second training image sets are labeled. The images and labels can be used to train the first and second video review models. The first video review model learns to identify violations of the same violation standard in different countries or regions, while the second video review model learns to identify violations of different violation standards in different countries or regions. The training methods for the first and second video review models can refer to existing training methods and will not be detailed here.

[0063] After the first and second video review models are trained, during video review, the first video review model is used to output the first violation score of the video image belonging to the total violation category after inputting the video image, and the second video review model is used to output the second violation score of the video image belonging to the sub-violation category when inputting the video image with the first violation score within the preset score range.

[0064] Taking the review of whether a person under a specified age appears in the video as an example, the higher the score for the first violation, the higher the probability that a person under a specified age appears in the video image. Conversely, the lower the score for the first violation, the lower the probability that a person under a specified age appears in the video image. For example, it is possible that a person under a specified age appears in the video image along with another person under a specified age, or even that no person under a specified age appears. As previously stated, if all countries or regions generally consider the presence of individuals under a specified age in a video to be a violation, then video data with a first violation score greater than a higher threshold can be pushed to the manual review terminals of all countries or regions. For example, video data with a first violation score greater than 0.8 (out of a total of 1 point) can be pushed to the manual review terminals of all countries or regions. For video data with a first violation score between 0.5 and 0.8, the video image in the video data can be input into the second video review model to obtain the second violation score of the video image belonging to the sub-violation category. Video data with a second violation score greater than a preset value can be pushed to the manual review terminals of individual countries or regions. For example, if a certain country or region considers the simultaneous presence of individuals under a specified age as a violation, and the second video review model outputs that the second violation score of the video image belonging to the sub-violation category is greater than 0.8, then the video data can be pushed to the manual review terminal of that country or region.

[0065] In this embodiment, for each type of violation, violations with the same violation standards across different countries or regions can be categorized into violations belonging to the overall violation category, while violations with different violation standards across different countries or regions can be categorized into violations belonging to sub-violation categories. After training a first video review model using a first training image set belonging to the overall violation category and a second video review model using a second training image set belonging to the sub-violation categories, the first video review model learns to identify violations with the same violation standards across different countries or regions in the image, and the second video review model learns to identify violations with different violation standards across different countries or regions in the image. During video review, after the first video review model outputs the first violation score for the video image belonging to the overall violation category, videos with higher first violation scores can be pushed to various countries / regions. The system uses a manual review terminal in a home or region to conduct manual reviews. Videos with a first violation score within a preset range are input into a second video review model to obtain a second violation score for the sub-violation category. Videos with higher second violation scores are then sent to manual review terminals in countries or regions with different violation standards for further review. This approach can identify violations with the same violation standards in different countries or regions through the first video review model, and also identify violations with different standards in different countries or regions through the second video review model. This avoids the problem of a surge in the number of submitted videos and low review efficiency caused by pushing all videos with a first violation score greater than a certain low value to all countries or regions for manual review in order to achieve full coverage. It achieves regionalized push of videos for manual review, thus improving the efficiency of video review.

[0066] Example 2

[0067] Figure 2A This is a flowchart of a video review model training method provided in Embodiment 2 of this application. This embodiment is an optimization based on Embodiment 1 above. Specifically, as follows: Figure 2A As shown, the video review model training method of this application embodiment may include the following steps:

[0068] S201. Obtain the illegal video clip, which includes multiple frames of video images.

[0069] In an optional embodiment of this application, video segments can be randomly extracted from video data, and the video segments can be input into a pre-trained violation identification model to obtain a third violation score indicating that the video segment belongs to a violation video segment. Video segments with a third violation score greater than a preset first threshold are regarded as violation video segments.

[0070] The violation identification model can be an existing trained model. In the process of reviewing videos using this violation identification model, video clips are randomly extracted from short video platforms and live streaming platforms, and video images are sampled from these video clips and input into the violation identification model to obtain a violation score indicating whether the video clip belongs to a violation video clip. If the violation score is greater than a preset threshold, it indicates that the video clip is a violation video clip.

[0071] In another optional embodiment, video clips with violation scores greater than a preset threshold can also be sent to a manual review terminal for manual review. After receiving the review results from the manual review terminal, if the review result is a violation, the video clip is confirmed as a violation video clip, thereby improving the accuracy of reviewing violation video clips.

[0072] S202. Extract the illegal images from the multi-frame video images and add them to the pre-established first training image set.

[0073] The illegal video clips consist of multiple video images. Each video image can be input into a pre-trained illegal object detection model to obtain a fourth violation score for the video image containing the illegal object. Video images with a fourth violation score greater than a preset second threshold are then cropped into the first training image set.

[0074] Taking the appearance of a person under a specified age in a video as an example of a violation, the violation detection model can be a pre-trained model that detects people under a specified age in video images. Each frame of the video clip violating the violation can be input into the violation detection model. If the violation score output by the violation detection model is greater than a preset threshold, it indicates that a person under the specified age appears in the video image, and the video image is added to the first training image set; otherwise, the video image is discarded. This avoids the problem of inaccurate training data caused by labeling video images without violation content as violation data when all video images of a video clip are used as training data. Therefore, images without violation content and background images can be removed from the video clip, retaining only the video images containing violation content as training data, thus improving the accuracy of the training data.

[0075] S203. Select images from the first training image set that are classified as sub-violation categories to obtain the second training image set.

[0076] In one optional embodiment, after the video clip is sent to the manual review terminal for manual review, the reviewer provides feedback on the review results on the manual review terminal. The review results may be penalty data for the user of the violating video clip, such as the penalty level. Different violations are usually given different penalty levels.

[0077] Taking the appearance of individuals under a specified age in a video as an example of a violation, if an individual under the specified age appears alone in a video clip, the reviewer can impose the first level of penalty: permanent account ban, permanent ban from broadcasting, etc. If an individual under the specified age appears in a video clip along with another individual under the specified age, the reviewer can impose the second level of penalty: a ban from broadcasting for one week, one month, etc. As in the first training image set obtained in S202, video images containing violators are added to the first training image set. That is, the first training image set can include images belonging to the overall violation category and sub-violation categories. For example, if all images in the first training image set contain individuals under the specified age, this can include images of individuals under the specified age appearing alone, as well as images of individuals under the specified age appearing together with another individual under the specified age. Then, images with the specified penalty level can be selected from the first training image set. The specified penalty level is the penalty level for images belonging to a sub-violation category, and these images with the specified penalty level are added to the pre-established second training image set.

[0078] For example, the first training image set includes images of individuals appearing alone below a specified age, and also includes images of individuals appearing simultaneously below a specified age and images of individuals appearing simultaneously below a specified age. The penalty level for images of individuals appearing alone below a specified age is the first penalty level, and the penalty level for images of individuals appearing simultaneously below a specified age and images of individuals appearing simultaneously below a specified age is the second penalty level. The national A violation identification standard is: a video containing an individual appearing below a specified age. The national B violation identification standard is: in addition to a video containing an individual appearing below a specified age being considered a violation, a video containing an individual appearing below a specified age and images of individuals appearing simultaneously below a specified age being considered a violation. Therefore, images of the first penalty level are added to a second training image set to train the national A review model using the second training image set, and images of the second penalty level are added to another second training image set to train the national B review model using the second training image set.

[0079] This application embodiment extracts illegal video images from historically reviewed illegal video clips and adds them to a first training image set. Images with a specified penalty level are then selected from the first training image set and added to a second training image set. On the one hand, this eliminates the need for manual image annotation, saving training data annotation costs. On the other hand, by extracting only illegal images from video clips and removing images without illegal content and background images from illegal video clips, the video images containing illegal content are retained as training data, improving the accuracy of the training data.

[0080] S204. Construct a first network and a second network. Both the first network and the second network include two or more feature extraction modules and an attention mechanism module located between two adjacent feature extraction modules. The attention mechanism module is used to extract an attention map when inputting the feature map output by the feature extraction module, and to concatenate the attention map with the feature map to output to the next feature extraction module.

[0081] In one embodiment of this application, the first video review model and the second video review model have the same network structure. In this network structure, an attention mechanism module is set between the two feature extraction modules. The feature extraction module can be a convolution module. After inputting a video image or feature map, the convolution module performs a convolution operation to output a feature map. The feature map is then input to the attention mechanism module, which extracts an attention map from the feature map and concatenates the attention map with the feature map to input it into the next feature extraction module.

[0082] like Figure 2B The diagram shows the network structure of the first and second networks. An attention mechanism module is set between the two feature extraction modules, and the feature map output by the feature extraction module is the input of the attention mechanism module.

[0083] Optionally, the first and second networks are DenseNet, and the feature extraction module is DenseBlock. In each DenseBlock network layer, each network layer concatenates the output features of all previous network layers, making the transfer of features and gradients more efficient. Furthermore, setting multiple DenseBlocks in the network can avoid the problem of inconsistent feature map sizes during concatenation.

[0084] The attention mechanism module can employ a multi-head attention mechanism based on non-local information. This module calculates the correlation between features at any two positions in the feature map output by each DenseBlock, resulting in an attention map that preserves global dependency information. It then calculates the response of this attention map to each position in the feature map and applies it to the feature map through weighted summation, thereby introducing global information to aid in the extraction of key features. The multi-head attention mechanism defines multiple sets of attention, mapping features to multiple subspaces, and then merges the outputs of different subspaces as the final result. Because the attention mechanism is applied to the feature map in parallel across multiple subspaces, each subspace focuses on different parts of the features, thus allowing for the inclusion of diverse contextual information.

[0085] Specifically, the attention mechanism module uses a Gaussian function as the relation function between features. It calculates the dependency information between different locations in the neighboring regions of the input feature map through the Gaussian function to extract non-local relation features, and defines multiple sets of non-local attention parallel computations. In one example, the attention mechanism module transposes (deconvolutions) the input feature map, performs matrix multiplication on the feature map and the transposed result to calculate the relationship between feature values ​​at different locations in the global feature map, and then uses softmax for normalization to obtain an attention map carrying the normalized correlation between each feature at a location in the feature map and other features at other locations. Simultaneously, the feature map is first processed through a 1×1 convolution operation to extract information, and then multiplied with the attention map. This applies non-local information to the corresponding locations in the feature map using the attention mechanism, and then another 1×1 convolution operation is used to obtain a non-local relation feature map. During the extraction of the non-local relation feature map, different weight matrices are used to define multiple attention heads, projecting features onto different representation subspaces to emphasize different information. Multiple features obtained by concatenating and combining these features are used as the final output, improving the model's ability to extract image features and helping the model focus on relevant feature information during the learning process, thus optimizing the model's prediction performance.

[0086] S205. Initialize the parameters of the first network and the second network to obtain the initialized first video review model and second video review model.

[0087] Specifically, the parameters for initializing the first network and the second network can be the parameters for initializing the neurons in the network, such as initializing weights and values.

[0088] S206. Extract the first image from the first training image set and input it into the first video review model to train the first video review model until the first video review model converges, thus obtaining the trained first video review model.

[0089] In an optional embodiment, a first image can be extracted from the first training image set and input into the first video review model to obtain a first violation score for the first image belonging to the total violation category. A loss rate is calculated based on the first violation score and the label of the first image. It is determined whether the loss rate is less than a preset loss threshold. If so, the adjustment of the model parameters of the first video review model is stopped, and a trained first video review model is obtained. If not, the gradient is calculated using the loss rate, and gradient descent is performed on the model parameters of the first video review model using the gradient. The process then returns to the step of extracting the first image from the first training image set and inputting it into the first video review model.

[0090] The loss rate can be calculated using a preset loss function, such as the mean squared error loss function (MSELoss) or the L1 norm loss function. Gradient descent can be achieved using methods such as stochastic gradient descent, batch gradient descent, or mini-batch gradient descent. This application does not limit the method of calculating the loss rate and gradient descent.

[0091] If the loss rate is less than the preset loss threshold after each iteration, it means that the first video review model has converged, and training can be terminated. The model parameters can be saved to obtain the trained first video review model. Of course, in practical applications, a preset number of iterations can also be set. When the number of iterations equals the preset number of iterations, training can be stopped to obtain the trained first video review model.

[0092] S207. Pass the trained model parameters of the first video review model to the second video review model.

[0093] Since the first video review model and the second video review model have the same model structure, the model parameters of the trained first video review model can be copied to the second video review model to reduce the number of iterations of the second video review model and speed up its convergence.

[0094] S208. Extract the second image from the second training image set and input it into the second video review model to train the second video review model until the second video review model converges, thus obtaining the trained second video review model.

[0095] The model parameters of the trained first video review model are passed to the second video review model. The second image in the second training image set can be used to train the second video review model. For specific training methods, please refer to the training method of the first video review model in S206, which will not be described in detail here.

[0096] In this embodiment, after training a first video review model using a first training image set belonging to the overall violation category and a second video review model using a second training image set belonging to a sub-violation category, the first video review model learns to identify violations with the same violation standards in different countries or regions. The second video review model learns to identify violations with different violation standards in different countries or regions. During video review, after the first video review model outputs the first violation score of the video image belonging to the overall violation category, videos with higher first violation scores can be pushed to manual review terminals in different countries or regions for manual review. Video images with first violation scores within a preset range are input into the second video review model to obtain the second violation score of the video image belonging to a sub-violation category. Videos with higher second violation scores are then sent to manual review terminals in different countries or regions for manual review. This approach can identify violations with the same violation standards in different countries or regions through the first video review model and violations with different standards in different countries or regions through the second video review model. This avoids the problem of a surge in the number of submitted videos and low review efficiency caused by pushing all videos with first violation scores greater than a certain low value to all countries or regions for manual review in order to achieve full coverage. It realizes regionalized push of videos for manual review and improves video review efficiency.

[0097] Furthermore, violation images are extracted from multiple frames of previously reviewed violation video clips and added to a pre-established first training image set. Images with a specified penalty level are then selected from the first training image set and added to a second training image set. On the one hand, this eliminates the need for manual image annotation, saving on training data annotation costs. On the other hand, by extracting only violation images from video clips and removing images without violation content and background images from violation video clips, the accuracy of the training data is improved.

[0098] Furthermore, the first video review model and the second video review model have the same model structure. The first video review model is trained first, and the model parameters of the trained first video review model are passed to the second video review model before the second video review model is trained. This reduces the number of iterations of the second video review model and speeds up its convergence.

[0099] Furthermore, an attention mechanism module is added to both the first and second video review models, which can extract non-local relation feature maps, improve the model's ability to extract image features, help the model focus on review-related feature information during the learning process, and optimize the model's prediction performance.

[0100] Example 3

[0101] Figure 3 This is a flowchart illustrating a video review method provided in Embodiment 3 of this application. This embodiment is applicable to reviewing video data for infringing content. The method can be executed by the video review device implemented in this application. This video review device can be implemented in hardware or software and integrated into the electronic device provided in this embodiment. Specifically, as shown... Figure 3 As shown, the video review method in this application embodiment may include the following steps:

[0102] S301. Extract video images from the video data to be reviewed.

[0103] In this embodiment, the video data to be reviewed can be video clips from platforms such as live streaming platforms, short video platforms, and video playback platforms. For example, to review video data A, a video clip of a preset playback duration can be extracted from video data A as the video data to be reviewed. If the video data to be reviewed includes multiple frames of video images, then one or more video images can be extracted from the video data to be reviewed. For example, video images can be extracted randomly, video images can be extracted according to a preset period, I-frames can be extracted from video images, or one frame can be extracted from a group of frames (GOP). This embodiment does not limit the method of extracting video images from video data.

[0104] S302. Input the video image into the pre-trained first video review model to obtain the first violation score of the video image belonging to the total violation category.

[0105] In this embodiment, the first video review model and the second video review model are trained using the video review model training method provided in Embodiment 1 or Embodiment 2. The total violation category refers to the category of violation content for which different countries or regions have the same violation identification standards. After inputting the video image into the first video review model, a first violation score can be obtained for the violation content in the video image that contains the total violation category.

[0106] S303. Determine whether the score for the first violation is greater than the first threshold.

[0107] Taking the violation of having people under a specified age appear in the video as an example, the total violation category is defined as: having people under a specified age appear in the video. The higher the first violation score, the higher the probability that people under a specified age appear in the video image. For example, the first violation score is high when only people under a specified age appear in the video image, the first violation score is low when both people under a specified age and people over a specified age appear in the video image, and the first violation score is 0 when no people under a specified age appear in the video image.

[0108] To ensure that videos submitted for manual review meet the violation criteria of all countries or regions—for example, if a video image only contains individuals under a specified age, it will be considered a violation in all countries or regions and generally subject to a higher level of punishment—a higher first threshold can be set, such as 0.8. The system then checks whether the score for the first violation is greater than 0.8. If yes, S304 is executed; otherwise, S305 is executed.

[0109] S304. Send the video data to be reviewed to the review client.

[0110] If the first violation score is greater than the first threshold, it indicates that the video data to be reviewed contains content that meets the violation standards of most countries or regions. As in the example above, most countries or regions consider it a violation when a person under a specified age appears alone in a video. When the first violation score is greater than 0.8, it can be determined that the video data to be reviewed contains a person under a specified age alone. The video data to be reviewed is then sent to the manual review client of the country or region that needs to conduct violation review. After receiving the video data to be reviewed, the manual review client plays the video data on the display interface. The reviewers manually review the video content played on the playback interface and provide feedback on the review results on the review client, such as the suggested penalty level.

[0111] S305. When the first violation score is greater than the second threshold, the video image is input into the pre-trained second video review model to obtain the second violation score of the video image belonging to the sub-violation category.

[0112] When the first violation score is less than the second threshold but greater than the second threshold, the region code associated with the video data to be reviewed can be obtained first. The second video review model that matches the region code can be found in the region code-second video review model table. The video image can be input into the second video review model that matches the region code to obtain the second violation score of the video image belonging to the sub-violation category.

[0113] The second video review model is a video review model trained for violation standards that differ from those of other countries or regions. For example, if a country judges a video as a violation if it contains both individuals under a specified age and individuals over a specified age, then a second video review model is trained for that country to predict the second violation score for the video containing both individuals under the specified age and individuals over the specified age. The country's regional code can be associated with the second video review model to form a regional code-second video review model table. The regional code-second video review model table can be used to find the second video review model that matches the country's regional name. Video images of videos to be reviewed with a first violation score less than a second threshold and greater than the second threshold can be input into the second video review model to obtain the second violation score.

[0114] S306. When the second violation score is greater than the third threshold, the video data to be reviewed will be sent to the review client.

[0115] When the second violation score is greater than the third threshold, it means that the content contained in the video data to be reviewed meets the violation standards of a certain country. The video data to be reviewed is then sent to the manual review terminal of that country for manual review, thus realizing regional video submission.

[0116] To enable those skilled in the art to more clearly understand the video review method implemented in this application, the video review method is explained below with examples:

[0117] Assuming there are two countries, A and B, and taking the appearance of individuals under a specified age in a video as a violation as an example, the violation criteria for country A (country code 01) and country B (country code 02) are as follows:

[0118] Country A: Videos featuring only individuals under a specified age are considered to be in violation.

[0119] Country B: Videos containing both individuals under the specified age and individuals over the specified age are also considered violations.

[0120] Then, a first video review model can be trained to detect whether a person under the specified age appears in the video. The higher the violation score of the first video review model, the more likely a person under the specified age will appear alone in the video. Otherwise, it is possible that a person under the specified age and a person over the specified age appear at the same time, or no person under the specified age appears. A second video review model can be trained to detect whether a person under the specified age and a person over the specified age appear at the same time in the video. The country code 02 in the country code-second video review model table is matched with the second video review model.

[0121] Video images 1 and 2 are extracted from video data 1 and 2 respectively. These images are then input into the first video review model. The first violation scores are 0.9 and 0.6, indicating a high probability of individuals under a specified age appearing alone in video data 1 (first violation score greater than the threshold 0.8). Therefore, video data 1 is directly sent to the manual review terminals of countries A and B for manual review. Video data 2 has a score of 0.6, suggesting the possibility of both individuals under and over a specified age appearing simultaneously. In this case, country B also considers it a violation. Therefore, video data 2 (first violation score less than the threshold 0.8 and greater than the threshold 0.6) is considered a violation. If the second violation score output by the second video review model is 0.9 (threshold 0.8) when the threshold is 0.5, it means that there is a high probability that the video data to be reviewed 2 contains both people under the specified age and people over the specified age. The video data to be reviewed 2 can be sent to the manual review terminal of country B. This avoids the problem that when the first violation score threshold is set to 0.8, only the video data to be reviewed 1 is pushed to the manual review terminal of country B and the video data to be reviewed 2 is missed. Or, when the first violation score threshold is set to 0.5, both the video data to be reviewed 1 and the video data to be reviewed 2 are pushed to country A at the same time. However, the video data to be reviewed 2 is not considered a violation by country A, which leads to an increase in the number of videos to be reviewed.

[0122] In the video review method of this application embodiment, the video review model includes a first video review model and a second video review model. After extracting video images from the video data to be reviewed, the video images are input into the pre-trained first video review model to obtain a first violation score for the video image belonging to the total violation category. It is then determined whether the first violation score is greater than a first threshold. If so, the video data to be reviewed is sent to the review client. If not, when the first violation score is greater than a second threshold, the video images are input into the pre-trained second video review model to obtain a second violation score for the video image belonging to a sub-violation category. When the second violation score is greater than a third threshold, the video data to be reviewed is sent to the review client. This method can identify violations with the same violation standards in different countries or regions through the first video review model, and it can also identify violations with different standards in different countries or regions through the second video review model. This avoids the problem of a surge in the number of submitted videos and low review efficiency caused by pushing all videos with a first violation score greater than a certain low value to various countries or regions for manual review in order to achieve full coverage. It realizes regional push of videos for manual review and improves the efficiency of video review.

[0123] Example 4

[0124] Figure 4 This is a structural block diagram of a video review model training device provided in Embodiment 4 of this application, as shown below. Figure 4 As shown, the video review model training device in this application embodiment may specifically include the following modules:

[0125] The training image acquisition module 401 is used to acquire a first training image set and a second training image set, wherein the first training image set is an image set belonging to the total violation category, and the second training image set is an image set belonging to the sub-violation category;

[0126] The model initialization module 402 is used to initialize the first video review model and the second video review model.

[0127] The model training module 403 is used to train the first video review model using the first training image set and to train the second video review model using the second training image set.

[0128] Specifically, when the video review model is used for video review, the first video review model is used to output a first violation score of the video image belonging to the total violation category after inputting a video image, and the second video review model is used to output a second violation score of the video image belonging to a sub-violation category when inputting a video image whose first violation score is within a preset score range.

[0129] The video review model training device provided in this application can execute the video review model training method provided in Embodiment 1 and Embodiment 2 of this application, and has the corresponding functional modules and beneficial effects of the execution method.

[0130] Example 5

[0131] Figure 5 This is a structural block diagram of a video review device provided in Embodiment 5 of this application, as shown below. Figure 5 As shown, the video review device in this application embodiment may specifically include the following modules:

[0132] The video image capture module 501 is used to capture video images from the video data to be reviewed;

[0133] The total violation category prediction module 502 is used to input the video image into a pre-trained first video review model to obtain a first violation score of the total violation category of the video image;

[0134] The first violation score judgment module 503 is used to determine whether the first violation score is greater than the first threshold.

[0135] The first submission module 504 is used to send the video data to be reviewed to the review client;

[0136] The sub-violation category prediction module 505, when the first violation score is greater than the second threshold, inputs the video image into the pre-trained second video review model to obtain the second violation score of the video image belonging to the sub-violation category;

[0137] The second submission module 506 is used to send the video data to be reviewed to the review client when the second violation score is greater than the third threshold.

[0138] The first video review model and the second video review model are trained using the video review model training method described in Example 1 or Example 2.

[0139] Example 6

[0140] Reference Figure 6 The diagram illustrates the structure of an electronic device as an example of this application. Figure 6 As shown, the electronic device may specifically include: a processor 601, a storage device 602, a touch-enabled display screen 603, an input device 604, an output device 605, and a communication device 606. The electronic device may contain one or more processors 601. Figure 6 Taking a processor 601 as an example, the processor 601, storage device 602, display screen 603, input device 604, output device 605, and communication device 606 of this electronic device can be connected via a bus or other means. Figure 6 Taking a bus connection as an example, the electronic device is used to execute the video review model training method and / or the video review method provided in the embodiments of this application.

[0141] Example 7

[0142] This application provides a computer-readable storage medium, wherein the instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the video review model training method and / or the video review method as described in the above method embodiments.

[0143] Example 8

[0144] This application provides a computer program product in which instructions are executed by a processor to implement the video review model training method and / or the video review method described in the above method embodiments.

[0145] For the purposes of this application, a computer-readable storage medium can be any means that can contain, store, communicate, propagate, or transmit a program for use in or in conjunction with an instruction execution system, apparatus, or device. More specific examples of computer-readable media (a non-exhaustive list) include: an electrical connection (electronic device) having one or more wires, a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Additionally, a computer-readable medium can even be paper or other suitable media on which the program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory. A computer program product can be a product containing a computer-readable storage medium such that, when the instructions in the computer-readable storage medium are executed by a processor, they implement the video review model training method and / or the video review method described in the above-described method embodiments.

[0146] It should be noted that the embodiments of the device, electronic device, storage medium, and computer program are basically similar to the method embodiments, so the description is relatively simple. For relevant parts, please refer to the description of the method embodiments.

Claims

1. A method for training a video review model, characterized in that, include: Obtain illegal video clips, which include multiple video frames and are associated with a penalty level; Violation images are extracted from multiple frames of the video images and added to a pre-established first training image set; Images with a specified penalty level are selected from the first training image set, where the specified penalty level is the penalty level when the image belongs to a sub-violation category; The images with the specified penalty level are added to a pre-established second training image set; Initialize the first video review model and the second video review model; The first video review model is trained using the first training image set, and the second video review model is trained using the second training image set; Specifically, when the video review model is used for video review, the first video review model is used to output a first violation score of the video image belonging to the total violation category after inputting a video image, and the second video review model is used to output a second violation score of the video image belonging to a sub-violation category when inputting a video image whose first violation score is within a preset score range.

2. The video review model training method as described in claim 1, characterized in that, The acquisition of the illegal video clips includes: Extract video segments to be reviewed from the video data; The video segment to be reviewed is input into a pre-trained violation identification model to obtain a third violation score indicating that the video segment to be reviewed belongs to a violation video segment; The video segments to be reviewed that have a third violation score greater than a preset first threshold are considered as violation video segments.

3. The video review model training method as described in claim 1, characterized in that, The step of extracting the violation images from multiple frames of the video images and adding them to a pre-established first training image set includes: Each frame of video image is input into a pre-trained violation object detection model to obtain a fourth violation score for the video image containing a violation object; The video image with the fourth violation score greater than the preset second threshold is cropped into the first training image set.

4. The video review model training method as described in any one of claims 1-3, characterized in that, The initialization of the first video review model and the second video review model includes: A first network and a second network are constructed. Both the first network and the second network include two or more feature extraction modules and an attention mechanism module located between two adjacent feature extraction modules. The attention mechanism module is used to extract an attention map when the feature map output by the input feature extraction module is input, and to concatenate the attention map with the feature map to output to the next feature extraction module. The parameters of the first network and the second network are initialized to obtain the initialized first video review model and second video review model.

5. The video review model training method as described in any one of claims 1-3, characterized in that, The first video review model and the second video review model have the same model structure. The step of training the first video review model using the first training image set and training the second video review model using the second training image set includes: The first image is extracted from the first training image set and input into the first video review model to train the first video review model until the first video review model converges, thus obtaining the trained first video review model. The model parameters of the trained first video review model are passed to the second video review model; The second image is extracted from the second training image set and input into the second video review model to train the second video review model until the second video review model converges, thus obtaining the trained second video review model.

6. The video review model training method as described in claim 5, characterized in that, The step of extracting a first image from the first training image set and inputting it into the first video review model to train the first video review model until it converges, thereby obtaining a trained first video review model, includes: The first image is extracted from the first training image set and input into the first video review model to obtain the first violation score of the first image belonging to the total violation category; The loss rate is calculated based on the first violation score and the label of the first image; Determine whether the loss rate is less than a preset loss threshold; If so, stop adjusting the model parameters of the first video review model to obtain the trained first video review model; If not, the gradient is calculated using the aforementioned loss rate; The gradient is used to perform gradient descent on the model parameters of the first video review model, and the steps of extracting the first image from the first training image set and inputting it into the first video review model are returned.

7. A video review method, characterized in that, include: Extract video images from the video data to be reviewed; The video image is input into a pre-trained first video review model to obtain the first violation score of the total violation category of the video image; Determine whether the first violation score is greater than the first threshold; If so, send the video data to be reviewed to the review client; If not, when the first violation score is greater than the second threshold, the video image is input into a pre-trained second video review model to obtain the second violation score of the video image belonging to the sub-violation category; When the second violation score is greater than the third threshold, the video data to be reviewed is sent to the review client; The first video review model and the second video review model are trained using the video review model training method described in any one of claims 1-6.

8. The video review method as described in claim 7, characterized in that, The step of inputting the video image into a pre-trained second video review model to obtain a second violation score for the video image belonging to a sub-violation category includes: Obtain the region code associated with the video data to be reviewed; Find the second video review model that matches the region code in the region code-second video review model table; The video image is input into a second video review model that matches the region code to obtain a second violation score for the video image belonging to a sub-violation category.

9. A video review model training device, characterized in that, include: The training image acquisition module is used to acquire a first training image set and a second training image set, wherein the first training image set is an image set belonging to the total violation category, and the second training image set is an image set belonging to the sub-violation category; The model initialization module is used to initialize the first video review model and the second video review model. The model training module is used to train the first video review model using the first training image set, and to train the second video review model using the second training image set. Wherein, when the video review model is used for video review, the first video review model is used to output a first violation score of the video image belonging to the total violation category after inputting a video image, and the second video review model is used to output a second violation score of the video image belonging to a sub-violation category when inputting a video image whose first violation score is within a preset score range; The training image acquisition module is specifically used for: Obtain illegal video clips, which include multiple video frames and are associated with a penalty level; Violation images are extracted from multiple frames of the video images and added to a pre-established first training image set; Images with a specified penalty level are selected from the first training image set, where the specified penalty level is the penalty level when the image belongs to a sub-violation category; The images with the specified penalty level are added to a pre-established second training image set.

10. A video review device, characterized in that, include: The video image extraction module is used to extract video images from the video data to be reviewed; The total violation category prediction module is used to input the video image into a pre-trained first video review model to obtain the first violation score of the video image belonging to the total violation category; The first violation score determination module is used to determine whether the first violation score is greater than the first threshold. The first submission module is used to send the video data to be reviewed to the review client; The sub-violation category prediction module inputs the video image into a pre-trained second video review model when the first violation score is greater than the second threshold to obtain the second violation score of the video image belonging to the sub-violation category; The second submission module is used to send the video data to be reviewed to the review client when the second violation score is greater than the third threshold. The first video review model and the second video review model are trained using the video review model training method described in any one of claims 1-6.

11. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more computer programs. When the one or more computer programs are executed by the one or more processors, the one or more processors implement the video review model training method as described in any one of claims 1-6, and / or the video review method as described in any one of claims 7-8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the video review model training method as described in any one of claims 1-6, and / or the video review method as described in any one of claims 7-8.

13. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor, the video review model training method of any one of claims 1-6 is implemented, and / or the video review method of any one of claims 7-8 is implemented.

Images