Method for live streaming judging, device for live streaming judging, server and storage medium

A cascade-connected inspection model with a behavioral review model improves the accuracy and recall of live streaming content detection by analyzing visual and behavioral features, addressing the inaccuracies in existing neural network-based methods.

JP7584054B2Active Publication Date: 2024-11-15BIGO TECH PTE LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023540184
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-12-30
Filing Date
2021-12-27
Publication Date
2024-11-15
Estimated Expiration
2041-12-27

AI Technical Summary

Technical Problem

Existing live streaming content examination methods using neural networks suffer from inaccurate and incomplete detection of unauthorized content, leading to erroneous screenings and omissions, particularly in live streaming platforms where topics like terrorism, violence, and pornography are present.

Method used

A cascade-connected high-accuracy and high-recall inspection model is employed, followed by a behavioral review model, to ensure comprehensive and accurate detection of violating content by analyzing both visual and behavioral features of live streaming frames.

Benefits of technology

This approach significantly reduces erroneous screenings and omissions, enhancing the accuracy and recall rates of live streaming content examination while minimizing additional equipment costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584054000004
    Figure 0007584054000004
  • Figure 0007584054000005
    Figure 0007584054000005
  • Figure 0007584054000006
    Figure 0007584054000006
Patent Text Reader

Abstract

A method, an apparatus, a server and a storage medium for live streaming screening are disclosed, which includes: performing an initial violation screening for a current live frame in a live room to be screened by a cascaded high-precision screening model and a high-recall screening model; if the current live frame passes the initial violation screening, inputting a first violation score of the current live frame in the high-precision screening model, a second violation score of the current live frame in the high-recall screening model and a multi-dimensional behavior feature of the current live frame in the live room to be screened into a pre-constructed behavior screening model to obtain a target violation score for the current live frame.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] This application claims priority to a Chinese patent application with Application No. 202011613190.6 filed with the China Patent Office on December 30, 2020, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of violating content screening on Internet resources, such as a live streaming screening method, a live streaming screening device, a server and a storage medium. [Background technology]

[0003] With the rapid development of Internet technology, there are more and more live streaming platforms on the Internet, and the spread of Internet resources is becoming more and more widespread. At the same time, the live streaming content of some live rooms may touch on topics such as terrorism, violence, pornography, political secrets, etc., which may lead to a large amount of illegal live streaming content being rapidly spread on the Internet. Therefore, it is necessary to detect whether there is illegal live streaming content in such live rooms in real time, so as to prevent the spread of illegal live streaming content.

[0004] Typically, a single feature trained neural network model is used to perform a violation content analysis on the live visual features reflected in multiple live video frames during live streaming and the live audio features in the live audio, and to determine whether there is a violation screen in multiple live video frames or whether there is a violation audio related to topics such as terrorism, violence, pornography, political secrecy, etc., to screen suspiciously violating live rooms and push them to a manual screening platform for manual screening. In this case, when extracting live video frames and live audio during live streaming, there is a problem that the live streaming network is unstable and the extraction result is inaccurate, so when the extracted live video frames and live audio are used to screen whether there is a violation content in the live room, the accuracy rate and recall rate of the violation screening for the live room cannot be guaranteed, and problems of erroneous screening and screening omissions are likely to occur. Summary of the Invention [Means for solving the problem]

[0005] The present application relates to a method, device, server and storage medium for live streaming screening, which avoids erroneous screening and omissions in live streaming screening and improves the accuracy rate and recall rate of live streaming screening.

[0006] The present application relates to a method for live streaming screening, the method comprising: By using the cascaded high-precision screening model and the high-recall screening model, an initial violation screening is performed on a current live frame in a live room to be screened; If the current live frame passes the initial violation screening, input the first violation score of the current live frame in the high-precision screening model, the second violation score of the current live frame in the high-recall screening model, and the multi-dimensional behavior features of the current live frame in the live room to be screened into a pre-constructed behavior screening model to obtain a target violation score of the current live frame.,of Contains wherein performing initial violation screening on a current live frame in a live room to be screened by the cascaded high accuracy screening model and high recall screening model includes: inputting the current live frame in the live room to be screened into a pre-constructed high accuracy screening model to obtain a first violation score of the current live frame; and if the first violation score does not exceed a preset accurate violation threshold, continuing to input the current live frame into a pre-constructed high recall screening model to obtain a second violation score of the current live frame, wherein the current live frame passes the initial violation screening in response to the first violation score of the current live frame not exceeding the preset accurate violation threshold and the second violation score of the current live frame not exceeding a preset recall violation threshold. .

[0007] The present application relates to an apparatus for live streaming review, the apparatus comprising: An initial screening module configured to perform initial violation screening on a current live frame in a live room to be screened by a cascaded high-precision screening model and a high-recall screening model; a violation score determination module configured to obtain a target violation score for the current live frame by inputting a first violation score for the current live frame in the high-precision screening model, a second violation score for the current live frame in the high-recall screening model, and multi-dimensional behavior features of the current live frame in the live room to be screened into a pre-constructed behavior screening model if the current live frame passes an initial violation screening; ,of Contains wherein the initial screening module is specifically used to input a current live frame in the live room to be screened into a pre-built high-precision screening model to obtain a first violation score of the current live frame; if the first violation score does not exceed a preset accurate violation threshold, continue to input the current live frame into a pre-built high-recall screening model to obtain a second violation score of the current live frame; wherein, according to the first violation score of the current live frame does not exceed the preset accurate violation threshold and the second violation score of the current live frame does not exceed the preset recall violation threshold, the current live frame passes the initial violation screening. .

[0008] This application is Live streaming review Regarding the server, the server: one or more processors; a storage device configured to store one or more programs; Including, When the one or more programs are executed by the one or more processors, they cause the one or more processors to implement the live streaming review method described in any embodiment of the present application.

[0009] This application is Non-transient With respect to a computer-readable storage medium, a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the live streaming review method described in any embodiment of the present application is realized. [Brief description of the drawings]

[0010] [Figure 1A] 1 is a flowchart of a live streaming review method according to Example 1 of the present application. [Figure 1B] FIG. 1 is a principle schematic diagram of a live streaming review process according to Example 1 of the present application. [Figure 2A] 1 is a flowchart of a live streaming review method according to Example 2 of the present application. [Figure 2B] FIG. 13 is a principle schematic diagram of a live streaming review process according to Example 2 of the present application. [Figure 3A] 11 is a flowchart of a live streaming review method according to Example 3 of the present application. [Figure 3B] FIG. 11 is a principle schematic diagram of a live streaming review process according to Example 3 of the present application. [Figure 4] FIG. 11 is a structural schematic diagram of a live streaming review device according to Example 4 of the present application. [Diagram 5] FIG. 11 is a structural schematic diagram of a server according to Example 5 of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present application will be described below with reference to the drawings and examples. The examples described in this specification are not intended to limit the present application, but are intended to illustrate the present application. For ease of explanation, the drawings show only parts related to the present application, not the entire configuration. In addition, the examples in the present application and the features in the examples can be combined with each other, if not inconsistent.

[0012] Example 1

[0013] 1A is a flowchart of a live streaming screening method according to a first embodiment of the present application, which can be applied to a case where a violation screening is performed on a live streaming screen in a live room. The live streaming screening method according to this embodiment can be performed by a live streaming screening device according to an embodiment of the present application, which can be realized by software and / or hardware and integrated into a server that executes the method, and the server can be a background server equipped with a video screening function.

[0014] Referring to FIG. 1A, the method may include the following steps.

[0015] In S110, an initial violation screening is performed on a current live frame in a live room to be screened by the cascaded high-precision screening model and high-recall screening model.

[0016] Because the types of audiences targeted by live rooms are wide, when the live streaming content of some live rooms covers topics such as terrorism, violence, pornography, political sensitivities, etc., the violating content will spread rapidly on the Internet. Therefore, in order to prevent the violating content from having a negative impact on users' lives, it is necessary to detect whether there is any illegal live streaming content in real time during live streaming.

[0017] In this embodiment, in order to ensure the comprehensiveness of the live streaming violation screening, the entire screening process of the live streaming screening is divided into two parts: initial violation screening and re-violation screening. In this case, in order to avoid the problem of erroneous screening or screening omission in the live streaming screening, the initial violation screening process can quickly screen relatively obvious violating live streaming that can easily distinguish whether or not there is a violation, and the re-violation screening process adopts a screening power higher than the initial violation screening to perform a violation screening again on the live frame that cannot be accurately judged whether or not there is a violating content by the initial violation screening, thereby ensuring the comprehensive violation screening of the current live frame in the live room to be screened.

[0018] In this embodiment, in the initial violation screening process, two network models, a high-precision screening model and a high-recall screening model, are pre-constructed, where the high-precision screening model has a high-precision violation screening ability, and when a violation screening is performed on a current live frame in a live room to be screened by the high-precision screening model, it can ensure that the screened violation live frame is highly likely to be a real violation, and the problem of erroneous audit of violations occurs. At the same time, the high-recall screening model has a high-recall violation screening ability, and when a violation screening is performed on a current live frame in a live room to be screened by the high-recall screening model, it can ensure that even if there is a small amount of violation content in the current live frame, it can be screened as much as possible, and the situation of missed violation screening occurs is reduced.

[0019] In this embodiment, when training the high-precision screening model and the high-recall screening model, a training sample set is constructed in advance, and the training sample set includes historical live frames in historical live videos in multiple types of live rooms. At the same time, according to whether each historical live frame actually violates the rules and the difficulty of accurately screening for violations, each training sample in the training sample set can be divided into four types, namely, easy positive samples, easy negative samples, hard positive samples, and hard negative samples. The easy positive samples and the easy negative samples respectively refer to positive samples and negative samples that are easy to identify whether they violate the rules by the model, and the hard positive samples and hard negative samples respectively refer to positive samples and negative samples that are difficult to identify whether they violate the rules by the model. In this case, the high-precision screening model can accurately detect violations. Because it has high requirements for accuracy, the training samples of the high-precision screening model can be composed of simple positive samples and simple negative samples in the training sample set, and the model parameters and network structure of the high-precision screening model can be constantly optimized by checking whether there is violation content in the simple positive samples and simple negative samples. Because the high-recall screening model has high requirements for the recall of violation screening, the training samples of the high-recall screening model can be composed of hard positive samples and hard negative samples in the training sample set, and by adopting training samples that are difficult to check for violations to perform screening training, the high-recall screening model can screen even when faced with a violation live frame that is difficult to screen, thereby ensuring the high recall ability of the high-recall screening model for violation screening.

[0020] At the same time, the high-precision screening model only trains the accuracy of violation screening, and there will be situations where violation screening is missed, but the high-recall screening model has high-recall violation screening ability and can reduce the situations where violation screening is missed as much as possible. Therefore, in order to ensure the high precision and high recall of initial violation screening, in this embodiment, as shown in Figure 1B, the high-precision screening model and the high-recall screening model can be cascaded, so that first, an initial violation screening is performed for each current live frame in the live room to be screened using the high-precision screening model, and for the current live frames that have not been screened for violations by the high-precision screening model, violation screening is continued using the high-recall screening model, and violation screening is performed sequentially using the cascaded two-layer screening model, and the initial violation screening process in this embodiment is completed, ensuring the high precision and high recall of initial violation screening.

[0021] In this embodiment, during live streaming of the video of the live room to be reviewed, one live image can be captured from the live room to be reviewed at every preset time (for example, every 2 s), as the current live frame in this embodiment, and the current live frame is input into a high-precision screening model in a two-layer cascade network, and the high-precision screening model determines whether there is any violating content in the current live frame. If the current live frame passes the violation screening of the high-precision screening model, the high-precision screening model has determined that there is no violating content in the current live frame. However, since there is a situation where the high-precision screening model has missed the violation screening, the current live frame needs to be input again into a high-recall screening model in a two-layer cascade network, and the high-recall screening model needs to continue to determine whether there is any violating content in the current live frame, thereby completing the initial violation screening for the current live frame in the live room to be reviewed.

[0022] In this case, there are two cases for the violation screening results of the high recall screening model for the current live frame. In case 1, if the violation screening by the high recall screening model is not passed, it indicates that the high recall screening model can confirm that the current live frame has violating content, and it is directly confirmed that the live room to be screened in which the current live frame exists is a violating live room. In case 2, if the violation screening by the high recall screening model is not passed, it indicates that the high recall screening model can determine that the current live frame does not have violating content, and then it is necessary to continue the violation screening for the current live frame using the pre-constructed behavior screening model.

[0023] In S120, if the current live frame passes the initial violation screening, a target violation score for the current live frame is obtained by inputting the first violation score of the current live frame in the high-precision screening model, the second violation score of the current live frame in the high-recall screening model, and the multi-dimensional behavioral features of the current live frame in the live room to be screened into a pre-constructed behavioral screening model.

[0024] As one option, if the current live frame in the live room to be audited passes both the violation audit by the high-precision audit model and the violation audit by the high-recall audit model, it indicates that the current live frame has passed the initial violation audit, and it is necessary to continue the re-violation audit to ensure the accuracy and comprehensiveness of the violation audit.

[0025] When an initial violation screening is performed on a current live frame using a cascaded high-precision screening model and a high-recall screening model, a violation analysis is usually performed on the information in the live vision screen of the current live frame, in which case the high-precision screening model outputs a first violation score for the current live frame, and the high-recall screening model outputs a second violation score for the current live frame, and both the first violation score and the second violation score can represent the vision screen features of the current live frame.

[0026] However, since the live streaming content in the live room is diverse, and the influencer and the viewers in the live room may have interactive interactions regarding the live streaming content, such as gifting, public comments on the content, and the influencer's past violations, the interactive behavior in the live room can also be used as a reference basis for determining whether there is a violation content in the current live frame. Therefore, in this embodiment, after it is determined that the current live frame has passed the initial violation screening, when a re-violation screening is performed on the current live frame, the vision screen features represented by the first violation score of the current live frame in the high-precision screening model and the second violation score of the current live frame in the high-recall screening model can be referenced, and multiple live streaming interactive behaviors between the influencer and the viewers in the live room to be reviewed at the time of the live streaming where the current live frame is located in the live room to be reviewed, as shown in FIG. 1B, are analyzed. By this, a multi-dimensional behavior feature of the current live frame in the live room to be reviewed is determined, and the vision screen features represented by the first violation score and the second violation score of the current live frame and the multi-dimensional behavior feature determined from the interactive behaviors of multiple live broadcasts in the live room to be reviewed are combined and input into a pre-constructed behavior screening model, and the behavior screening model fuses and analyzes the vision screen features represented by the first violation score and the second violation score and the multi-dimensional behavior feature determined from the interactive behaviors of multiple live broadcasts in the live room to be reviewed, and collectively determines whether there is a violation content in the current live frame, and outputs a target violation score of the current live frame. In this case, by adopting multi-angle feature fusion that combines the multi-dimensional behavior features of the vision screen features to analyze the target violation score, re-violation screening is realized, there is no need to adopt additional screening equipment, the violation screening cost is reduced, and the accuracy and recall of violation screening for the live room is guaranteed.

[0027] In the technical proposal of this embodiment, the high-precision screening model and the high-recall screening model are cascaded to perform initial violation screening on the current live frame in the live room to be screened, so that the violating live frame which is easy to distinguish and has obvious violation can be quickly screened; for the current live frame which is not easy to distinguish or has obvious violation but has passed the initial violation screening, in addition to the first violation score of the current live frame in the high-precision screening model and the second violation score of the current live frame in the high-recall screening model, the multi-dimensional behavior feature of the current live frame in the live room to be screened is referred to, and the behavior screening model again performs a unified feature analysis on the current live frame by combining the vision screen feature and the behavior feature, so as to obtain a target violation score of the current live frame, thereby realizing a comprehensive violation screening of the current live frame in the live room to be screened, thereby avoiding the problem of erroneous screening or screening omission in live streaming screening, reducing the cost of violation screening, and improving the accuracy rate and recall rate of live streaming screening.

[0028] Example 2

[0029] FIG. 2A is a flow chart of a live streaming screening method according to embodiment 2 of the present application, and FIG. 2B is a principle schematic diagram of a live streaming screening process according to embodiment 2 of the present application. Based on the above embodiment, this embodiment will be described. As shown in FIG. 2A, this embodiment describes the screening process of the re-violation screening of the current live frame in the live room to be screened and the modeling process of the behavior screening model.

[0030] As an option, as shown in FIG. 2A, this embodiment may include the following steps.

[0031] In S210, an initial violation screening is performed on a current live frame in a live room to be screened by the cascaded high-precision screening model and high-recall screening model.

[0032] In S220, if the current live frame passes the initial violation screening, a live streaming behavior feature of the current live frame is obtained by merging a first violation score of the current live frame in the merged high-precision screening model, a second violation score of the current live frame in the high-recall screening model, and a multi-dimensional behavior feature of the current live frame in the live room to be screened.

[0033] As an option, when the current live frame passes the initial violation screening, the high precision screening model and the high recall screening model are used to determine that there is no violation content in the current live frame. Therefore, in order to ensure the accuracy of the live streaming screening and avoid the situation of missing violation screening, the vision screen features and behavior features of the current live frame are used to continue the re-violation screening for the current live frame. In this case, the first violation score of the current live frame in the high precision screening model and the second violation score of the current live frame in the high recall screening model are confirmed, and the first violation score and the second violation score are taken as the vision screen features corresponding to the current live frame. At the same time, according to the interactive behaviors of multiple live streams in the live room to be screened at the time of the live streaming of the current live frame, a multi-dimensional behavior feature of the current live frame in the live room to be screened is determined, and the first violation score, the second violation score and the multi-dimensional behavior feature are merged to obtain the live streaming behavior feature of the current live frame as the multi-angle fusion feature in the re-violation screening. Then, the live streaming behavior feature is used to determine whether there is violation content in the current live frame, so as to ensure the comprehensiveness and accuracy of the live streaming screening.

[0034] Illustratively, the live streaming behavior feature in this embodiment may be X=(x1, x2, ..., x11), where x1 is the first violation score of the current live frame in the high-precision screening model, x2 is the second violation score of the current live frame in the high-recall screening model, x3 is the number of real-time views of the live streaming in the live room to be screened among the multi-dimensional behavior features, x4 is the number of comments sent by users in the live room to be screened among the multi-dimensional behavior features, x5 is the number of violation comments in the live room to be screened among the multi-dimensional behavior features, and x6 is the number of violation comments in the live room to be screened among the multi-dimensional behavior features. x is the percentage of violating comments in the live room to be reviewed among all comments, x7 is the number of giftings in the live room to be reviewed among multidimensional behavioral characteristics, x8 is the number of past violations by the influencer in the live room to be reviewed among multidimensional behavioral characteristics, x9 is the percentage of past violations by the influencer in the live room to be reviewed among multidimensional behavioral characteristics, x10 is the number of violations in past audio of the influencer in the live room to be reviewed among multidimensional behavioral characteristics, and x11 is the percentage of violations in past audio of the influencer in the live room to be reviewed among multidimensional behavioral characteristics.

[0035] In order to ensure the accuracy of live streaming violation screening, the multi-dimensional behavior features in this embodiment can be dynamically configured, and are not limited thereto.

[0036] At S230, the live broadcast behavior features are input into a behavior screening model to obtain a target violation score for the current live frame.

[0037] As one option, after obtaining the live streaming behavior feature, the live streaming behavior feature can be input into a pre-constructed behavior screening model as a violation fusion feature in multiple angles, and the behavior screening model can fuse and analyze the feature values ​​of the live streaming behavior feature in each dimension to determine whether it carries violation content information, and further output a target violation score of the current live frame, and determine whether there is violation content in the current live frame based on the target violation score, thereby improving the accuracy of live streaming screening.

[0038] Illustratively, in this embodiment, the behavior screening model is modeled using the following steps: respectively extracting corresponding historical live frames from each historical live video in the historical live video set, and determining the multi-dimensional historical behavior features of each historical live frame in the live room in which it is located and the violation label of the historical live frame; for each historical live frame, inputting the first historical violation score of the historical live frame in the high-precision screening model, the second historical violation score of the historical live frame in the high-recall screening model, and the multi-dimensional historical behavior features of the historical live frame in the live room in which it is located into the pre-constructed behavior screening model to obtain the historical violation score of the historical live frame; and constantly optimizing the behavior screening model with the goal of minimizing the difference between the historical violation score and the violation label of each historical live frame.

[0039] By searching historical live videos of multiple live rooms, a historical live video set for modeling is constructed, and when performing violation screening for live rooms, violation screening is mainly performed for live frames in live broadcasts, so video frames need to be extracted for each historical live video in the historical live video set. For example, for each preset time, one historical live image is captured from each historical live video as a historical live frame extracted from the historical live video. In this case, all the extracted historical live frames can constitute a modeling sample for the behavior screening model.

[0040] In this case, in order to successfully model the behavior screening model, it is necessary to set a number of modeling features that need to be referred to during the modeling process and a modeling optimization goal of the behavior screening model, and according to the modeling optimization goal, constantly optimize the actual modeling situation in the number of modeling features. For example, the modeling feature in this embodiment may be X=(x1, x2, ..., x11), where x1 is the first historical violation score of the historical live frame in the high-precision screening model, x2 is the second historical violation score of the historical live frame in the high-recall screening model, x3 is the number of real-time views of the live broadcast in the live room where the historical live frame of the set multi-dimensional historical behavior feature is located, x4 is the number of comments sent by users in the live room where the historical live frame of the multi-dimensional historical behavior feature is located, and x5 is the number of comments sent by users in the live room where the historical live frame of the multi-dimensional historical behavior feature is located. x5 is the number of violating comments in the live room in which the influencer is located, x6 is the proportion of violating comments in the live room in which the influencer is located in the historical live frame of the multidimensional historical behavior features to all comments, x7 is the number of giftings in the live room in which the influencer is located in the historical live frame of the multidimensional historical behavior features, x8 is the number of past violations by the influencer in the live room in which the influencer is located in the historical live frame of the multidimensional historical behavior features, x9 is the proportion of past violations by the influencer in the live room in which the influencer is located in the historical live frame of the multidimensional historical behavior features, x10 is the number of violations in past audio of the influencer in the live room in which the influencer is located in the historical live frame of the multidimensional historical behavior features, and x11 is the proportion of violations in past audio of the influencer in the live room in which the influencer is located in the historical live frame of the multidimensional historical behavior features.

[0041] Since the multi-dimensional historical behavior features of the historical live frame in the live room in which it is located, among the multiple modeling features, can be determined during the historical live streaming, in order to perform continuous modeling optimization according to the modeling optimization goal, as shown in FIG. 2B, after extracting corresponding historical live frames from each historical live video, the interactive behavior of the historical live streaming in the live room in which the historical live frame is located can be determined, and the violation label of each historical live frame can be determined. For each historical live frame, the first historical violation score and the second historical violation score of each historical live frame can be obtained by inputting the historical live frame into the high precision screening model and the high recall screening model of this embodiment respectively, so that the first historical violation score of each historical live frame, the second historical violation score of each historical live frame, and the multi-dimensional historical behavior features of each historical live frame in the live room in which it is located jointly constitute the modeling features corresponding to each historical live frame, and the modeling features corresponding to each historical live frame can be constantly input into the behavior screening model to obtain the historical violation score of each historical live frame.

[0042] In this case, the modeling optimization goal of the behavior screening model in this embodiment may be to minimize the difference between the historical violation score and the violation label of each historical live frame, and therefore, in order to achieve the goal of minimizing the difference between the historical violation score and the violation label of each historical live frame, the difference between the historical violation score and the violation label of each historical live frame is constantly analyzed during modeling optimization, and the difference is constantly reduced by optimizing the mathematical parameters in the behavior screening model, thereby obtaining a final optimized behavior screening model, and in this case, the final optimized behavior screening model has a highly accurate violation screening ability.

[0043] Illustratively, the modeling optimization goal of the behavior screening model in this embodiment is:

number

number

number

[0044] In the technical proposal of this embodiment, the modeling optimization goal is to minimize the difference between the historical violation score and the violation label of each historical live frame, a behavior screening model is modeled, and the accuracy of violation screening by the behavior screening model is ensured. Then, the first violation score of the current live frame in the live room to be screened in the high-precision screening model, the second violation score of the current live frame in the high-recall screening model, and the multi-dimensional behavior feature of the current live frame in the live room to be screened are merged to obtain the live streaming behavior feature of the current live frame, and a multi-angle violation feature analysis is performed on the live streaming behavior feature by the modeled behavior screening model, thereby improving the accuracy rate and recall rate of live streaming screening.

[0045] Example 3

[0046] FIG. 3A is a flow chart of a live streaming review method according to the third embodiment of the present application, and FIG. 3B is a principle schematic diagram of a live streaming review process according to the third embodiment of the present application. Based on the above embodiment, the present embodiment will be described. As shown in FIG. 3A, the present embodiment describes the entire violation review process of the live room to be reviewed.

[0047] As an option, as shown in FIG. 3A, this embodiment may include the following steps.

[0048] In S310, a current live frame in a live room to be judged is inputted into a pre-constructed high-precision judgment model to obtain a first violation score for the current live frame.

[0049] As one option, when a violation screening is performed on a live room to be screened, whether or not there is a violation content on the live streaming screen is detected in real time during live streaming, so that a violation screening needs to be performed on a current live frame in the live streaming. The current live frame is input into a pre-constructed high-precision screening model, and an initial violation screening is performed on the live streaming screen in the current live frame by the high-precision screening model, thereby outputting a first violation score for the current live frame, and then it can be determined whether or not there is a violation content in the current live frame by determining whether the first violation score exceeds a preset high-precision violation threshold.

[0050] In S320, it is determined whether the first violation score exceeds the preset accurate violation threshold, and if YES, execute S380; if NO, execute S330.

[0051] At S330, a second violation score for the current live frame is obtained by continuing to input the current live frame into the pre-constructed high-recall screening model.

[0052] As one option, if the first violation score does not exceed the preset accurate violation threshold, the high precision screening model will determine that there is no violating content in the current live frame, but since there is a situation where the high precision screening model misses the violation screening, the current live frame needs to continue to be input into the pre-constructed high recall screening model, and as shown in FIG. 3B, the high recall screening model continues to screen for violations corresponding to the live streaming screen in the current live frame, thereby outputting a second violation score for the current live frame, and then it can be determined again whether there is violating content in the current live frame by determining whether the second violation score exceeds the preset recall violation threshold.

[0053] In S340, it is determined whether the second violation score exceeds a preset recall violation threshold, and if YES, S380 is executed, and if NO, S350 is executed.

[0054] If the second violation score does not exceed the preset recall violation threshold, the high recall screening model determines that the current live frame does not have any violation content, that is, the current live frame simultaneously passes the violation screening by the high accuracy screening model and the violation screening by the high recall screening model, that is, passes the initial violation screening, and continues to perform re-violation screening on the current live frame using the behavior screening model. If the first violation score of the current live frame does not exceed the preset accurate violation threshold and the second violation score of the current live frame does not exceed the preset recall violation threshold, the current live frame passes the initial violation screening.

[0055] In S350, a target violation score for the current live frame is obtained by inputting the first violation score of the current live frame in the high-precision screening model, the second violation score of the current live frame in the high-recall screening model, and the multi-dimensional behavioral features of the current live frame in the live room to be screened into a pre-constructed behavioral screening model.

[0056] In S360, it is determined whether the target violation score exceeds a preset recall violation threshold, and if YES, S380 is executed, and if NO, S370 is executed.

[0057] As an option, the behavior screening model can be used to output a target violation score for the current live frame, and then, in order to perform different live streaming processing processes, it can be finally confirmed whether there is a violation content in the current live frame by determining whether the target violation score exceeds a preset recall violation threshold.

[0058] In S370, after a preset live distribution time, a new current live frame is collected from the live room to be judged, and violation judgment is continued for the new current live frame.

[0059] As one option, if the target violation score does not exceed the preset behavior violation threshold, the behavior screening model determines that there is no violating content in the current live frame. In this case, the three models in this embodiment determine that there is no violating content in the current live frame, and it can be determined that the current live frame is a non-violating frame, indicating that there is no violating content in the live room to be screened currently. In this case, in order to ensure the real-time nature of the violation screening for the live room, after a preset live streaming time (for example, every 2 s), one new live image can be captured from the live room to be screened at the current live streaming time point as the new current live frame collected, and violation screening can be continued for the new current live frame according to steps S310 to S380 of this embodiment, thereby performing violation screening in real time for the live room to be screened until the live streaming ends or the violating live frame is screened.

[0060] In S380, live streaming information of the live rooms to be reviewed is pushed to the manual review platform.

[0061] As one option, if the first violation score exceeds the preset accurate violation threshold, or if the second violation score exceeds the preset recall violation threshold, or if the target violation score exceeds the preset behavior violation threshold, in either case, the model used for the current review will determine that there is violating content in the current live frame. In order to avoid a situation in which a false audit of a violation occurs, as shown in FIG. 3B, in this embodiment, by adding a manual review platform, when it is confirmed that there is violating content in the current live frame, the live streaming information of the live room to be reviewed in which the current live frame is located is pushed to the manual review platform, and further a manual review is performed on the live streaming content of the live room to be reviewed, thereby ensuring the accuracy of the live streaming review.

[0062] In this embodiment, the live streaming information of the live room to be reviewed may be the live streaming destination of the live room to be reviewed or a video screen during live streaming, and is not limited thereto.

[0063] In S390, the live streaming interactive items in the live room to be reviewed are monitored in real time, and when the live streaming interactive items exceed a preset interactive threshold, the live streaming information of the live room to be reviewed is pushed to a manual review platform, and the violation review for the current live frame in the live room to be reviewed is stopped.

[0064] As an option, in order to avoid the occurrence of abnormal situations in violation screening and ensure a high recall rate of live streaming screening, in this embodiment, a model is constructed to realize live streaming violation screening, and then the live streaming interactive items in the live room to be screened are monitored in real time to determine whether they need to be directly pushed to the manual screening platform for manual screening, and manual screening is the minimum guarantee plan for live streaming screening abnormality. In this case, the live streaming interactive items include the live streaming viewing number and gifting number in the live room to be screened, and the breadth of the audience of the live room to be screened can be analyzed based on this. If the live streaming interactive items exceed the preset interactive threshold, it means that the live streaming viewing rate of the live room to be screened is high, and once violation content occurs, it will spread quickly. Therefore, in order to ensure the efficiency of live streaming violation screening, it is necessary to directly push the live streaming information of the live room to be screened to the manual screening platform for manual screening, and stop using the multiple models constructed to screen the current live frame in the live room to be screened for violation screening.

[0065] In the technical solution of this embodiment, after it is determined that there is a violation content in the current live frame, the live streaming information of the live room to be reviewed is directly pushed to the manual review platform for manual review, so as to avoid the occurrence of false audit of violations and improve the accuracy of live streaming violation review; At the same time, the live streaming interactive item in the live room to be reviewed is used to additionally set a minimum guarantee plan for abnormality of live streaming review, so as to ensure the comprehensiveness of live streaming violation review.

[0066] Example 4

[0067] FIG. 4 is a structural schematic diagram of a live streaming review device according to the fourth embodiment of the present application. As shown in FIG. 4, the device includes: an initial screening module 410 configured to perform initial violation screening on a current live frame in a live room to be screened by a cascaded high accuracy screening model and a high recall screening model; a violation score determination module 420 configured to obtain a target violation score for the current live frame by inputting a first violation score for the current live frame in the high-precision screening model, a second violation score for the current live frame in the high-recall screening model, and multi-dimensional behavior features of the current live frame in the live room to be screened into a pre-constructed behavior screening model if the current live frame passes the initial violation screening; may include.

[0068] In the technical proposal of this embodiment, the high-precision screening model and the high-recall screening model are cascaded to perform initial violation screening on the current live frame in the live room to be screened, so that the violating live frame which is easy to distinguish and has obvious violation can be quickly screened; for the current live frame which is not easy to distinguish or has obvious violation but has passed the initial violation screening, in addition to the first violation score of the current live frame in the high-precision screening model and the second violation score of the current live frame in the high-recall screening model, the multi-dimensional behavior feature of the current live frame in the live room to be screened is referred to, and the behavior screening model again performs a unified feature analysis on the current live frame by combining the vision screen feature and the behavior feature, so as to obtain a target violation score of the current live frame, thereby realizing a comprehensive violation screening of the current live frame in the live room to be screened, thereby avoiding the problem of erroneous screening or screening omission in live streaming screening, reducing the cost of violation screening, and improving the accuracy rate and recall rate of live streaming screening.

[0069] The distribution examination device according to this embodiment is applicable to the live distribution examination method according to any of the above embodiments, and has corresponding functions and effects.

[0070] Example 5

[0071] Fig. 5 is a structural schematic diagram of a server according to a fifth embodiment of the present application, and as shown in Fig. 5, the server includes a processor 50, a storage device 51, and a communication device 52. The number of processors 50 included in the server may be one or more, and Fig. 5 takes one processor 50 as an example. The processor 50, storage device 51, and communication device 52 included in the server may be connected by a bus or other method, and Fig. 5 takes connection by a bus as an example.

[0072] The server according to this embodiment may be configured to execute the live streaming review method according to any of the above embodiments and has corresponding functions and effects.

[0073] Example 6

[0074] In Example 6 of the present application, Non-transient A computer-readable storage medium is further provided, in which a computer program is stored, and when the computer program is executed by a processor, the method for live streaming review according to any of the above embodiments can be realized. The method includes: By using the cascaded high-precision screening model and the high-recall screening model, an initial violation screening is performed on a current live frame in a live room to be screened; If the current live frame passes the initial violation screening, input the first violation score of the current live frame in the high-precision screening model, the second violation score of the current live frame in the high-recall screening model, and the multi-dimensional behavior features of the current live frame in the live room to be screened into a pre-constructed behavior screening model to obtain a target violation score of the current live frame; may include.

[0075] In an embodiment of the present application, a storage medium including computer-executable instructions is further provided, the computer-executable instructions being capable of performing operations in the above-described method as well as related operations in the live streaming review method according to any embodiment of the present application. The storage medium may be a non-transitory storage medium.

[0076] The present application may be realized by software and necessary general-purpose hardware, or may be realized by hardware. The technical solution of the present application may be embodied as a software product, which may be stored in a computer-readable storage medium such as a computer floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash (FLASH), a hard disk or an optical disk, and includes a plurality of instructions for making a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in the embodiments of the present application.

[0077] In the above embodiment of the live streaming review device, the included multiple units and modules are divided according to functional logic, but are not limited to the above divisions as long as they can realize the corresponding functions. Note that the names of the multiple functional units are intended to facilitate mutual distinction and do not limit the scope of protection of the present application. [Explanation of symbols]

[0078] 50 processors 51 Storage device 52 Communication Equipment 410 Initial Screening Module 420 Violation Score Determination Module

Claims

1. A method for live streaming judging carried out by a live streaming judging device, comprising: By using the cascaded high-precision screening model and the high-recall screening model, an initial violation screening is performed on a current live frame in a live room to be screened; If the current live frame passes the initial violation screening, input a first violation score of the current live frame in the high-precision screening model, a second violation score of the current live frame in the high-recall screening model, and multi-dimensional behavior features of the current live frame in the live room to be screened into a pre-constructed behavior screening model to obtain a target violation score of the current live frame; Including, The first violation score and the second violation score represent vision screen features of the current live frame, the multi-dimensional behavior feature represents a plurality of live streaming interaction behaviors in the live room to be judged, and the target violation score represents a combined feature of the vision screen feature and the multi-dimensional behavior feature; The cascaded high-precision screening model and the high-recall screening model are used to perform an initial violation screening on a current live frame in a live room to be screened. Inputting a current live frame in the live room to be judged into a pre-constructed high-precision judgement model to obtain a first violation score for the current live frame; if the first violation score does not exceed a preset accurate violation threshold, continue inputting the current live frame into a pre-built high-recall screening model to obtain a second violation score for the current live frame; Including, the current live frame passes the initial violation check in response to a first violation score of the current live frame not exceeding the preset precise violation threshold and a second violation score of the current live frame not exceeding a preset recall violation threshold.

2. inputting a first violation score of the current live frame in the high-precision screening model, a second violation score of the current live frame in the high-recall screening model, and multi-dimensional behavior features of the current live frame in the live room to be screened into a pre-constructed behavior screening model to obtain a target violation score of the current live frame; Obtaining a live streaming behavior feature of the current live frame by merging a first violation score of the current live frame in the high-precision screening model, a second violation score of the current live frame in the high-recall screening model, and a multi-dimensional behavior feature of the current live frame in the live room to be screened; inputting the live streaming behavior features into the behavior screening model to obtain a target violation score for the current live frame; The method of claim 1 , comprising:

3. The behavior screening model includes the following steps: Extracting corresponding historical live frames from each historical live video in the historical live video set, and determining multi-dimensional historical behavior features of each historical live frame in a live room where the historical live frame is located and violation labels of the historical live frames; For each historical live frame, inputting a first historical violation score of the historical live frame in the high-precision screening model, a second historical violation score of the historical live frame in the high-recall screening model, and a multi-dimensional historical behavior feature of the historical live frame in a live room in which the historical live frame is located into a pre-constructed behavior screening model to obtain a historical violation score of the historical live frame; continually optimizing the behavior screening model with the goal of minimizing the difference between the historical violation score and the violation label of each historical live frame; The method of claim 1 , wherein the model is performed by executing:

4. inputting the first violation score of the current live frame in the high-precision screening model, the second violation score of the current live frame in the high-recall screening model, and the multi-dimensional behavior features of the current live frame in the live room to be screened into a pre-constructed behavior screening model to obtain a target violation score of the current live frame; When the target violation score exceeds a preset behavior violation threshold, push the live streaming information of the live room to be reviewed to a manual review platform; 2. The method of claim 1, further comprising: if the target violation score does not exceed the preset behavior violation threshold, after a preset live broadcast time, collecting a new current live frame from the live room to be reviewed, and continuing violation review for the new current live frame.

5. Real-time monitoring of live streaming interactive items in the live room to be reviewed; when the live streaming interactive items exceed a preset interactive threshold, pushing live streaming information of the live room to be reviewed to the manual review platform, and stopping violation review of the current live frame in the live room to be reviewed. The method of claim 4 further comprising:

6. When the first violation score exceeds the preset accurate violation threshold or the second violation score exceeds the preset recall violation threshold, push the live streaming information of the live room to be reviewed to a manual review platform. The method of claim 1 further comprising:

7. The method of any one of claims 1 to 6, wherein the training samples of the high-precision screening model are composed of easy positive samples and easy negative samples in a training sample set, and the training samples of the high-recall screening model are composed of hard positive samples and hard negative samples in a training sample set.

8. An initial screening module configured to perform initial violation screening on a current live frame in a live room to be screened by a cascaded high-precision screening model and a high-recall screening model; a violation score determination module configured to obtain a target violation score for the current live frame by inputting a first violation score for the current live frame in the high-precision screening model, a second violation score for the current live frame in the high-recall screening model, and multi-dimensional behavior features of the current live frame in the live room to be screened into a pre-constructed behavior screening model when the current live frame passes the initial violation screening; Including, The first violation score and the second violation score represent vision screen features of the current live frame, the multi-dimensional behavior feature represents a plurality of live streaming interaction behaviors in the live room to be judged, and the target violation score represents a combined feature of the vision screen feature and the multi-dimensional behavior feature; The initial screening module further comprises: A current live frame in the live room to be judged is inputted into a pre-built high-precision judgment model to obtain a first violation score of the current live frame, and if the first violation score does not exceed a preset precision violation threshold, the current live frame is continued to be inputted into a pre-built high-recall judgment model to obtain a second violation score of the current live frame; The apparatus for live streaming screening, wherein the current live frame passes the initial violation screening in response to a first violation score of the current live frame not exceeding the preset precise violation threshold and a second violation score of the current live frame not exceeding a preset recall violation threshold.

9. one or more processors; a storage device configured to store one or more programs; Including, A server for live streaming review, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, the computer program implementing the method according to any one of claims 1 to 6 when executed by a processor.

Citation Information

Patent Citations

  • Machine auditing processing method and device, electronic device and storage medium

    CN109766807A

  • Giving device, giving method, and giving program

    JP2020154840A

  • Compliance model training to classify landing page content that violates content item distribution guidelines

    US8788442B1