Model training method, illegal live broadcast identification method, device, equipment and storage medium

Abstract

The present invention relates to a model training method, a method for identifying illegal live broadcasts, an apparatus, a device and a storage medium, the method comprising: inputting a positive sample set containing multiple positive samples and a first negative sample set containing multiple negative samples into a model, training a first branch of the model, wherein the first branch is used to distinguish positive samples from negative samples; judging whether the distinction result output by the first branch is true, so as to divide the distinction result output by the first branch into a TP sample set, a TN sample set, an FP sample set and an FN sample set; fixing the network structure and parameters of the first branch; inputting the positive sample set and the FP sample set in the distinction result output by the first branch as a second negative sample set into the model, training a second branch of the model; wherein the second branch is used to distinguish positive samples from FP samples. The model training method or the illegal live broadcast identification method of the present invention can improve the model recognition effect.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a model training method, an illegal live broadcast identification method, a device, a device and a storage medium. Background Art

[0002] The statements herein merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] On live streaming platforms, when the host is broadcasting live, there may be various unhealthy, illegal, and unlawful behaviors such as pornography, violence, terrorism, drug abuse, gambling, etc.

[0004] The existing violation identification model trained by the neural network model will have a long tail effect, that is, as the push ratio increases, the accuracy will drop rapidly. The push ratio refers to the ratio of the number of images pushed for human review to the total number of images. For example, when using some existing violation identification models, an average of 300,000 work orders are pushed per day, and 110,000 violation work orders are pushed. Among them, 100,000 violation work orders are pushed when 150,000 work orders are pushed. This shows that the efficiency of the first 150,000 orders pushed for review is relatively high, while the efficiency of the last 150,000 orders from 150,000 to 300,000 is very low. The last 150,000 orders with low efficiency are the objects that need to be distinguished.

[0005] Moreover, the live broadcast platform needs to limit the exposure time of illegal live broadcast content, that is, it needs to judge the violation in real time based on the short-term live broadcast content.

[0006] Therefore, the content security and review of live streaming platforms usually face the following problems: the push of pornographic models and non-pornographic violation models is relatively high, the violation ratio is very low, the efficiency of human review is low, the FP sample (false positive) sample confusion is high, and it is difficult to distinguish between positive samples and FP samples. Summary of the invention

[0007] The purpose of the present invention is to provide a new model training method, illegal live broadcast identification method, device, equipment and storage medium to improve the model recognition effect.

[0008] The purpose of the present invention is achieved by the following technical solutions. A model training method proposed in the present invention includes the following steps: inputting a positive sample set containing multiple positive samples and a first negative sample set containing multiple negative samples into a model, training a first branch of the model, wherein the first branch is used to distinguish positive samples from negative samples; judging whether the distinction result output by the first branch is true, so as to divide the distinction result output by the first branch into a TP sample set, a TN sample set, a FP sample set and a FN sample set; fixing the network structure and parameters of the first branch; inputting the positive sample set and the FP sample set in the distinction result output by the first branch as a second negative sample set into the model, training a second branch of the model; wherein the second branch is used to distinguish positive samples from FP samples.

[0009] The purpose of the present invention is also achieved by the following technical solutions. A method for identifying illegal live broadcasts proposed in accordance with the present invention includes the following steps: obtaining media data of live broadcasts; inputting the media data into a model trained by the aforementioned model training method, and obtaining a first score of the media data by using the first branch of the model; wherein the first branch is used to distinguish between positive samples and negative samples; if the first score is greater than or equal to a first threshold, the live broadcast content is determined to be illegal; if the first score is less than the first threshold, the second score of the media data is obtained by using the second branch of the model; wherein the second branch is used to distinguish between positive samples and FP samples of the first branch; if the second score is greater than or equal to the second threshold, the live broadcast content is determined to be illegal; otherwise, the live broadcast content is determined not to be illegal.

[0010] The purpose of the present invention is also achieved by the following technical solutions. A model training device proposed in accordance with the present disclosure includes: a first branch training module, used to input a positive sample set containing multiple positive samples and a first negative sample set containing multiple negative samples into a model to train the first branch of the model, wherein the first branch is used to distinguish positive samples from negative samples; a first result division module, used to determine whether the distinction result output by the first branch is true, so as to divide the distinction result output by the first branch into a TP sample set, a TN sample set, a FP sample set and a FN sample set; a second branch training module, used to fix the network structure and parameters of the first branch; the positive sample set and the FP sample set in the distinction result output by the first branch are input as a second negative sample set into the model to train the second branch of the model; wherein the second branch is used to distinguish positive samples from FP samples.

[0011] The purpose of the present invention is also achieved by the following technical solutions. A device for identifying illegal live broadcasts proposed in accordance with the present disclosure includes: an acquisition module for acquiring media data of live broadcasts; a first scoring module for inputting the media data into a model trained by the aforementioned model training method, and obtaining a first score of the media data by using the first branch of the model; wherein the first branch is used to distinguish positive samples from negative samples; a first judgment module for determining that the live broadcast content is illegal if the first score is greater than or equal to a first threshold; a second scoring module for obtaining a second score of the media data by using the second branch of the model if the first score is less than the first threshold; wherein the second branch is used to distinguish positive samples from FP samples of the first branch; a second judgment module for determining that the live broadcast content is illegal if the second score is greater than or equal to a second threshold, otherwise, determining that the live broadcast content is not illegal.

[0012] The purpose of the present invention is also achieved by the following technical solution. A model training device proposed in accordance with the present disclosure includes: a memory for storing non-temporary computer-readable instructions; and a processor for running the computer-readable instructions, so that the processor implements any of the aforementioned model training methods when executing.

[0013] The purpose of the present invention is also achieved by the following technical solutions. A device for identifying illegal live broadcasts proposed in the present disclosure includes: a memory for storing non-temporary computer-readable instructions; and a processor for running the computer-readable instructions, so that the processor implements any of the aforementioned illegal live broadcast identification methods when executing the instructions.

[0014] The purpose of the present invention is also achieved by the following technical solutions: A computer-readable storage medium proposed in the present disclosure is used to store non-temporary computer-readable instructions, and when the non-temporary computer-readable instructions are executed by a computer, the computer executes any one of the aforementioned model training methods or illegal live broadcast identification methods.

[0015] Compared with the prior art, the present invention has obvious advantages and beneficial effects. By means of the above technical scheme, the model training method, illegal live broadcast identification method, device, equipment and storage medium proposed in the present invention have at least the following advantages and beneficial effects:

[0016] 1. The present invention proposes a new dual-branch network integration architecture, one branch is used to distinguish positive samples from negative samples, and the other branch is used to distinguish positive samples from FP samples, which can effectively distinguish positive samples from FP samples;

[0017] 2. Furthermore, in the network structure proposed by the present invention, the two branches share at least part of the feature extractor, which can optimize the network structure;

[0018] 3. The model proposed by the present invention has a mechanism for selecting shared features between two branches, which can improve the model training effect;

[0019] 4. When training the model, the present invention adopts a serial training method, which can improve the model training effect.

[0020] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites a preferred embodiment and describes it in detail with the accompanying drawings as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a model training method according to an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of a dual-branch model provided by an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a model output result provided by an embodiment of the present invention;

[0024] Figure 4 It is a flowchart of a method for identifying illegal live broadcasts according to an embodiment of the present invention;

[0025] Figure 5 is a schematic diagram of a model training device according to an embodiment of the present invention;

[0026] Figure 6 It is a schematic diagram of a device for identifying illegal live broadcasts according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, characteristics and effects of the model training method, illegal live broadcast identification method, device, equipment and storage medium proposed in the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0028] It should be noted that, in this article, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In addition, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0029] The TP (True positives) samples mentioned in this article refer to samples that are predicted to be positive and are actually positive samples; the TN (True negatives) samples refer to samples that are predicted to be negative and are actually negative samples; the FP (False positives) samples refer to samples that are predicted to be positive and are actually negative samples; and the FN (False negatives) samples refer to samples that are predicted to be negative and are actually positive samples.

[0030] The existing violation processing model uses sample-driven neural networks. For FP samples, additional training data is used, such as adding FP samples to the negative samples of the training set. Since FP samples account for a low proportion of the total samples and the distribution of training samples does not change significantly, this solution has little effect on improving accuracy.

[0031] Figure 1 is a schematic flow chart of an embodiment of the model training method of the present invention. In some embodiments of the present invention, please refer to Figure 1 The model training method of the present invention mainly includes the following steps:

[0032] Step S11, inputting a positive sample set including multiple positive samples and a negative sample set including multiple negative samples (may be referred to as a first negative sample set) into a model, and training a first branch of the model (may be referred to as model1). The first branch is used to distinguish positive samples from negative samples.

[0033] Step S12, determining whether the distinction result output by the first branch (may be referred to as the first distinction result) is true, so as to divide the distinction result output by the first branch into a TP sample set, a TN sample set, a FP sample set and a FN sample set.

[0034] Step S13, fix the network structure and parameters (such as convolution parameters) of the first branch; input the original positive sample set and the FP sample set in the distinction result output by the first branch as the second negative sample set into the model, and train the second branch of the model (which may be recorded as model2). The second branch is used to distinguish positive samples from FP samples. Thus, the training of the dual-branch model is completed.

[0035] Optionally, in some examples of the present invention, the model is a neural network model.

[0036] This model has a dual-branch network integration architecture, which may be called a dual-branch model.

[0037] In some examples, the model distinguishes positive samples from negative samples by generating scores for the extracted features. Generally speaking, samples with high scores are judged as positive samples, and samples with low scores are judged as negative samples. Therefore, in this case, the presence of FP samples in the distinction results output by the first branch of the model is due to the presence of high-scoring negative samples. The new dual-branch network integration architecture proposed in the present invention uses one branch to distinguish positive samples from negative samples, and one branch to distinguish positive samples from FP samples (high-scoring negative samples), which can effectively distinguish positive samples from FP samples (high-scoring negative samples).

[0038] In some embodiments of the present invention, the second branch of the model includes a first partial structure and a second partial structure. The first partial structure of the second branch is at least a partial structure of the first branch, that is, the second branch shares at least a partial structure of the first branch; and the second partial structure of the second branch is a structure independent of the first branch, that is, the second partial structure of the second branch is a structure exclusively belonging to the second branch.

[0039] Figure 2 A schematic structural diagram of a dual-branch model provided by an embodiment of the present invention. Figure 2 In some embodiments of the present invention, the first branch of the model ( Figure 2 The left branch in the first branch) contains multiple convolution blocks to distinguish positive samples from negative samples. Among them, at least some of the convolution blocks in the first branch are used to extract features, that is, the output of at least some of the convolution blocks is the extracted features. Let's assume that the first branch contains m convolution blocks, denoted as convolution-1, convolution-2, ..., convolution-m (abbreviated as cov-1, cov-2, ..., cov-m). The second branch of the model ( Figure 2 The second branch (the right branch in the figure) shares at least some convolution blocks of the first branch, and the second branch is used to distinguish positive samples from FP samples of the first branch. It is assumed that the second branch shares k convolution blocks of the first branch, that is, the first partial structure of the second branch may include convolution-1, convolution-2, ..., convolution-k.

[0040] In some embodiments of the present invention, the first branch further includes at least one first fully connected (FC) structure, for example Figure 2 The first fully connected structure is the fully connected-1 (which may be abbreviated as fc-1) in the convolution-m. The first score (which may be abbreviated as score1) can be obtained by passing the features output by convolution-m through the first fully connected structure. The first fully connected structure is used to roughly distinguish positive samples from negative samples, and can be a high recall classification. Specifically, samples whose first scores are greater than or equal to a first threshold (which may be abbreviated as s1) can be judged as positive samples, and samples whose first scores are less than the first threshold can be judged as negative samples. Optionally, the first fully connected structure is located after the multiple convolution blocks cov-1, cov-2, ..., cov-m of the first branch.

[0041] In some embodiments of the present invention, the second branch further includes one or more convolution blocks dedicated to the second branch, for example Figure 2 The convolution blocks in the second branch may include convolution blocks-m+1, etc. (may be abbreviated as cov-m+1). That is, the second part of the structure of the second branch may include convolution blocks-m+1. At least some of the convolution blocks in the second branch are used to extract features, that is, the outputs of at least some of the convolution blocks are the extracted features.

[0042] In some embodiments of the present invention, the second branch further comprises at least one second fully connected structure, e.g. Figure 2 The second fully connected structure is used to distinguish positive samples from high-scoring negative samples corresponding to the first branch. Specifically, samples whose second scores are greater than or equal to a second threshold (which may be denoted as s2) may be determined as positive samples, and samples whose second scores are less than the second threshold may be determined as negative samples (such as the aforementioned high-scoring negative samples).

[0043] In some embodiments of the present invention, the second branch of the training model in the aforementioned step S13 includes: using the aforementioned FP samples as negative samples and the original positive samples as training sets to train the convolution blocks (such as cov-m+1, etc.) and the second fully connected structure (such as fc-2) dedicated to the second branch.

[0044] It should be noted that the present invention does not limit the structure of the convolutional blocks in the model. Optionally, in some optional embodiments of the present invention, the convolutional blocks in the model are composed of multiple layers of convolutional blocks, that is, Figure 2Convolution-1, convolution-2, ..., convolution-m, or convolution-m+1 in the first branch can all be composed of multiple layers of convolution blocks. In some optional embodiments of the present invention, at least one of the multiple convolution blocks in the first branch is a separate convolution block, or all are separate convolution blocks. In some optional embodiments of the present invention, one or more convolution blocks in the second branch that are exclusive to the second branch are separate convolution blocks, or all are separate convolution blocks. Among them, the separate convolution block refers to a depthwise separable convolution, and each separate convolution block can be composed of multiple sub-convolution blocks. Compared with the inseparable convolution block, the separate convolution block has a better effect. It should be noted that the present invention does not limit the first branch or the second branch of the model to have a separate convolution block. In fact, even if a model based on an inseparable convolution block is used, a better distinction effect can be achieved.

[0045] In some embodiments of the present invention, the dual-branch model adopts a feature sharing mechanism.

[0046] Specifically, the first branch of the training model in the aforementioned step S11 includes: in the process of training the first branch, saving the output vectors of multiple convolution blocks in the first branch, denoted as x ijp , where i is the number of the sample, for example, i can represent the i-th sample; j is the number of the convolution block and corresponds to the number of layers in the model where the convolution block is located, for example, j can represent the j-th convolution block, p=0 is a negative sample, and p=1 is a positive sample.

[0047] Furthermore, the aforementioned second branch sharing at least some of the convolution blocks of the first branch means: selecting the discriminative convolution block output features in the first branch as the input features of the unique part of the second branch (for example, the convolution block cov-m+1 that is exclusive to the second branch). Specifically, after fixing the network structure and parameters of the first branch in the aforementioned step S13 and before performing the step of training the second branch of the model in step S13, the model training method of the present invention also includes: judging whether the convolution block in the first branch can distinguish between positive samples and FP samples of the first branch, and selecting the convolution block with the largest number of layers from the convolution blocks in the first branch that can be distinguished as the kth convolution block. Furthermore, the aforementioned second branch sharing at least some of the convolution blocks of the first branch includes: the second branch sharing the 1st to kth convolution blocks in the first branch, for example Figure 2 The convolution-1 (abbreviated as cov-1) to convolution-k (abbreviated as cov-k) in . The value range of k is 1 to m, that is, the second branch may share at least one convolution block with the first branch, and at most all convolution blocks of the first branch.

[0048] As a specific example, the aforementioned determination of whether the convolution block in the first branch can distinguish between the positive sample and the FP sample of the first branch specifically includes determining whether the j-th convolution block meets the following requirements:

[0049] There exists a binary function f(x) such that

[0050] Where l represents the total number of samples and α is a constant less than 1. Optionally, α can be 0.85. Optionally, the binary function f(x) is a binary function of 0 and 1. If there exists a binary function such that the output vector x of the jth convolutional block ijp If the above inequality is satisfied, it means that the features of the convolution block have the ability to distinguish positive samples from FP samples of the first branch.

[0051] In view of the above problems, in a specific embodiment of the present invention, t-SNE (t-distributed stochastic neighbor embedding) is used to test whether the output features of cov-i can distinguish positive samples and FP samples of model1, and the largest i is selected. Among them, t-SNE is a dimensionality reduction method that converts a multidimensional data set into a low-dimensional data set.

[0052] In a specific example, distinguishing situations such as Figure 3 As shown in the figure, (0, 1, 2, 3, 4) are different types of high-scoring negative samples, and (6, 7, 8, 9) are different types of positive samples. Taking the use of the model to distinguish illegal live broadcasts as an example, Figure 3 The "0" in is the 2D feature corresponding to the normal image of the FP sample belonging to model1 on the Internet; Figure 3 The “1” in the figure is the 2D feature corresponding to the picture with rope (the picture of the tool used to commit the illegal behavior) of the FP sample of model1 on the Internet; Figure 3 The “2” in is the 2D feature corresponding to the rope picture of the FP sample of model1 in the first application (APP1); Figure 3 The “3” in is the 2D feature corresponding to the normal image of the FP sample of model1 in the first application (APP1); Figure 3 The “4” in is the 2D feature corresponding to the normal image of the FP sample of model1 in the second application (APP2); Figure 3 The “6” in the figure is the 2D feature corresponding to the picture of a person preparing to hang himself (picture of a person preparing to commit a violation); Figure 3 The “7” in the figure is the 2D feature corresponding to the online picture of a person hanging himself (picture of a person committing an illegal act); Figure 3 The “8” in is the 2D feature corresponding to the picture of a person preparing to hang himself in the first application (APP1); Figure 3The "9" in the figure is the 2D feature corresponding to the picture of a person hanging in the first application (APP1). (0, 1, 2, 3, 4) are different types of FP samples, and (6, 7, 8, 9) are different types of positive samples. Their score1 obtained by the first branch is difficult to distinguish, but the features corresponding to the second-to-last convolution layer are separable, and the distinction effect is very good after training the second branch.

[0053] In some embodiments of the present invention, a serial training method is adopted: first, the neural network model1 is trained with 800,000 positive samples and 1.2 million negative samples; the feature sharing mechanism is used to fix the network structure and convolution parameters of model1, and 500,000 FP samples are used as negative samples and the original 800,000 positive samples are used as training sets to train cov-m+1 and fc-2 in the neural network model2, and the training of the dual-branch neural network is completed.

[0054] It should be noted that the present invention does not limit the specific structure of the first branch in the double-branch model, and the present invention does not limit the specific structure of the second branch in the double-branch model. Figure 2 Instead of the specific structure shown, a more complex neural network structure is adopted.

[0055] It should be noted that the present invention does not limit the specific method of determining whether a feature can distinguish between positive samples and FP samples. For example, other clustering methods can be used to determine whether a feature can distinguish between positive samples and FP samples.

[0056] The dual-branch model trained by the aforementioned embodiment of the model training method of the present invention can be used for model inference (also called reasoning) in specific businesses in different fields, including but not limited to the identification of illegal live broadcasts.

[0057] Figure 4 FIG. 1 is a schematic flow chart of an embodiment of a method for identifying illegal live broadcasts of the present invention. Figure 4 The illegal live broadcast identification method of the present invention mainly includes the following steps:

[0058] Step S21, obtaining live media data. Optionally, the media data includes picture data of the live video; optionally, obtaining a screenshot of the live room from the live video stream. Optionally, obtaining a real-time media stream of the live broadcast in real time, and obtaining a real-time picture of the live broadcast from the media stream.

[0059] Step S22, input the media data into a pre-trained dual-branch model, and use the first branch (model1) of the model to obtain a first score (which may be recorded as score1) of the media data.

[0060] The first branch of the model is used to distinguish positive samples from negative samples, that is, to distinguish whether the media data is illegal.

[0061] Step S23: If the first score is greater than or equal to the first threshold (which may be denoted as s1), the live broadcast content is determined to be in violation of the rules. For live broadcasts determined by the system to be in violation of the rules, they can be pushed to manual review to further determine whether the live broadcast is in violation of the rules through manual review. If it is in violation of the rules, punishment measures such as suspension and blocking will be taken.

[0062] Step S24: if the first score is less than the first threshold, the second branch of the model (model2) is used to obtain a second score of the media data (may be recorded as score2).

[0063] The second branch of the model is used to distinguish positive samples from FP samples of the first branch.

[0064] Step S25: If the second score is greater than or equal to the second threshold (which may be recorded as s2), the live broadcast content is determined to be in violation of the rules. For live broadcasts determined by the system to be in violation of the rules, they can be pushed to manual review to further determine whether the live broadcast is in violation of the rules through manual review. If it is in violation of the rules, punishment measures such as suspension and blocking will be taken.

[0065] Step S26: Otherwise, it is determined that the live broadcast content does not violate the regulations.

[0066] The present invention has at least the following advantages:

[0067] 1. The present invention proposes a neural network model integration and training method and system. The model has a two-branch structure, one branch is used to distinguish positive samples from negative samples, and the other branch is used to distinguish positive samples from FP samples. It can effectively distinguish positive samples from FP samples. On the basis of the original recognition of positive and negative samples, the ability to distinguish positive samples from FP samples is improved, and the accuracy is significantly improved.

[0068] 2. The present invention does not modify the parameters of the original network, but only adds a new branch to distinguish positive samples from FP samples. At this time, FP samples account for a large proportion. It is equivalent to adding new parameters only to distinguish the distribution of positive samples and FP samples. It can effectively reduce the push and improve the accuracy without basically increasing the computing resources.

[0069] 3. The first branch of the model, model1, can perform coarse filtering. Model1 calls 400 million images. After filtering, the second branch, model2, only needs to be called 300,000 times (server resource consumption is minimal), but model2 can effectively reduce push and improve accuracy. Compared with two completely independent models, this solution consumes less resources and has a good effect of reducing push and improving accuracy.

[0070] 4. Therefore, the solution proposed in the present invention improves the ability to distinguish positive samples from FP samples on the basis of the original identification of positive and negative samples, and has a significant effect on improving the accuracy. In specific business, it can reduce the push volume by more than 20% while maintaining the illegal recall, greatly saving the human resources for review.

[0071] 5. According to a specific example of the method for identifying illegal live broadcasts of the present invention, recalling the same number of illegal work orders in the pornographic category can reduce the number of work orders pushed by 20%; recalling the same number of illegal work orders in the non-pornographic category can reduce the number of work orders pushed by 50%.

[0072] An embodiment of the present invention also provides a model training device, which mainly includes: a first branch training module, a first result division module, and a second branch training module.

[0073] The first branch training module is used to input a positive sample set including multiple positive samples and a first negative sample set including multiple negative samples into the model to train the first branch of the model. The first branch is used to distinguish positive samples from negative samples.

[0074] The first result division module is used to determine whether the distinction result output by the first branch is true, so as to divide the distinction result output by the first branch into a TP sample set, a TN sample set, a FP sample set and a FN sample set.

[0075] The second branch training module is used to: fix the network structure and parameters of the first branch; input the positive sample set and the FP sample set in the distinction result output by the first branch as the second negative sample set into the model to train the second branch of the model. The second branch is used to distinguish positive samples from FP samples.

[0076] An embodiment of the present invention also provides a device for identifying illegal live broadcasts, which mainly includes: an acquisition module, a first scoring module, a first judgment module, a second scoring module, and a second judgment module.

[0077] The acquisition module is used to acquire live broadcast media data.

[0078] The first scoring module is used to: input the media data into a pre-trained dual-branch model proposed in any of the aforementioned embodiments, and use the first branch of the model to obtain a first score of the media data; wherein the first branch is used to distinguish positive samples from negative samples.

[0079] The first judgment module is used to: if the first score is greater than or equal to a first threshold, determine that the live broadcast content violates the regulations.

[0080] The second scoring module is used to: if the first score is less than the first threshold, obtain a second score of the media data using a second branch of the model; wherein the second branch is used to distinguish between positive samples and FP samples of the first branch.

[0081] The second judgment module is used to: if the second score is greater than or equal to the second threshold, determine that the live broadcast content violates the rules; otherwise, determine that the live broadcast content does not violate the rules.

[0082] In addition, the various model training devices and various illegal live broadcast identification devices shown in the embodiments of the present invention include modules and units corresponding to the methods of the aforementioned embodiments, and their detailed descriptions and technical effects can be referred to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0083] Figure 5 is a schematic block diagram illustrating a model training device according to an embodiment of the present invention. Figure 5 As shown, the model training device 100 according to an embodiment of the present disclosure includes a memory 101 and a processor 102.

[0084] The memory 101 is used to store non-temporary computer-readable instructions. Specifically, the memory 101 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0085] The processor 102 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the model training device 100 to perform desired functions. In one embodiment of the present disclosure, the processor 102 is used to execute the computer-readable instructions stored in the memory 101, so that the model training device 100 performs all or part of the steps of the model training method of each embodiment of the present disclosure.

[0086] For detailed description and technical effects of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0087] Figure 6 2 is a schematic block diagram illustrating a device for identifying illegal live broadcasts according to an embodiment of the present invention. Figure 6 As shown, the illegal live broadcast identification device 200 according to an embodiment of the present disclosure includes a memory 201 and a processor 202.

[0088] The memory 201 is used to store non-temporary computer-readable instructions. Specifically, the memory 201 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0089] The processor 202 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the illegal live broadcast identification device 200 to perform desired functions. In one embodiment of the present disclosure, the processor 202 is used to run the computer-readable instructions stored in the memory 201, so that the illegal live broadcast identification device 200 performs all or part of the steps of the illegal live broadcast identification method of each embodiment of the present disclosure.

[0090] For detailed description and technical effects of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0091] An embodiment of the present invention also provides a computer storage medium, which stores computer instructions. When the computer instructions are executed on a device, the device executes the above-mentioned related method steps to implement the model training method or illegal live broadcast identification method in the above-mentioned embodiment.

[0092] An embodiment of the present invention also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement the model training method or illegal live broadcast identification method in the above-mentioned embodiment.

[0093] In addition, an embodiment of the present invention further provides a device, which may specifically be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory, so that the chip executes the model training method or the illegal live broadcast identification method in the above-mentioned method embodiments.

[0094] Among them, the device, computer storage medium, computer program product or chip provided by the present invention are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0095] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with the profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A model training method, characterized in that: The method comprises the following steps: Inputting a positive sample set including a plurality of positive samples and a first negative sample set including a plurality of negative samples into a model, and training a first branch of the model, wherein the first branch is used to distinguish positive samples from negative samples, and the first branch includes a plurality of convolution blocks; Determine whether the distinction result output by the first branch is true, so as to divide the distinction result output by the first branch into a TP sample set, a TN sample set, a FP sample set, and a FN sample set; Fixing the network structure and parameters of the first branch; Input the positive sample set and the FP sample set in the distinction result output by the first branch as a second negative sample set into the model to train the second branch of the model; wherein the second branch is used to distinguish positive samples from FP samples; Wherein, after fixing the network structure and parameters of the first branch and before training the second branch of the model, the method further includes: determining a convolution block for sharing with the second branch from a plurality of convolution blocks included in the first branch; The positive samples, the negative samples and the FP samples are media picture samples.

2. The model training method according to claim 1, characterized in that: The second branch includes a first partial structure and a second partial structure; wherein the first partial structure of the second branch is at least a partial structure of the first branch, and the second partial structure of the second branch is a structure independent of the first branch.

3. The model training method according to claim 2, characterized in that: The second branch also includes one or more convolution blocks dedicated to the second branch, and the second branch is used to distinguish positive samples from FP samples of the first branch.

4. The model training method according to claim 3, characterized in that: The first branch of training the model includes: in the process of training the first branch, saving the output vectors of the multiple convolution blocks in the first branch, denoted as x ijp , where i represents the i-th sample, j represents the j-th convolutional block, p=0 is a negative sample, and p=1 is a positive sample; The determining, from the plurality of convolution blocks included in the first branch, a convolution block for sharing with the second branch, comprises: judging whether the convolution block in the first branch can distinguish between positive samples and FP samples of the first branch, and selecting a convolution block with the largest number of layers from the convolution blocks in the first branch that can distinguish between the positive samples and the FP samples of the first branch as the kth convolution block; The second branch shares the 1st to kth convolution blocks in the first branch.

5. The model training method according to claim 4, characterized in that: The determining whether the convolution block in the first branch can distinguish between positive samples and FP samples of the first branch specifically includes: Determine whether the jth convolution block meets the following requirements, There exists a binary function f(x) such that Among them, l represents the total number of samples, and α is a constant less than 1.

6. The model training method according to claim 3, characterized in that: The first branch further includes at least one first fully connected structure, wherein the first fully connected structure is used to distinguish positive samples from negative samples; The second branch further includes at least one second fully connected structure; wherein the second fully connected structure is used to distinguish positive samples from high-scoring negative samples corresponding to the first branch; The training of the second branch of the model includes: training the convolution block dedicated to the second branch and the second fully connected structure.

7. A method for identifying illegal live broadcasts, characterized in that: The method comprises the following steps: Get live media data; Inputting the media data into a model trained by the model training method according to any one of claims 1 to 6, and obtaining a first score of the media data by using a first branch of the model; wherein the first branch is used to distinguish positive samples from negative samples; If the first score is greater than or equal to a first threshold, the live broadcast content is determined to be in violation of the regulations; If the first score is less than a first threshold, obtaining a second score of the media data using a second branch of the model; wherein the second branch is used to distinguish positive samples from FP samples of the first branch; If the second score is greater than or equal to a second threshold, the live broadcast content is determined to be in violation of the regulations; Otherwise, the live broadcast content is judged not to be in violation of regulations.

8. A model training device, characterized in that: The device comprises: A first branch training module, used for inputting a positive sample set including a plurality of positive samples and a first negative sample set including a plurality of negative samples into a model, and training a first branch of the model, wherein the first branch is used for distinguishing positive samples from negative samples, and the first branch includes a plurality of convolution blocks; a first result division module, configured to determine whether the distinction result output by the first branch is true, so as to divide the distinction result output by the first branch into a TP sample set, a TN sample set, a FP sample set, and a FN sample set; A second branch training module is used to fix the network structure and parameters of the first branch; input the positive sample set and the FP sample set in the distinction result output by the first branch as a second negative sample set into the model to train the second branch of the model; wherein the second branch is used to distinguish between positive samples and FP samples; Wherein, after fixing the network structure and parameters of the first branch and before training the second branch of the model, the device is further used to: determine a convolution block shared by the second branch from a plurality of convolution blocks included in the first branch; The positive samples, the negative samples and the FP samples are media picture samples.

9. A device for identifying illegal live broadcasts, characterized in that: The device comprises: An acquisition module, used to acquire live broadcast media data, wherein the media data includes picture data of the live broadcast video, screenshots of the live broadcast room acquired from the live broadcast video stream, and real-time pictures acquired from the real-time media stream of the live broadcast; A first scoring module, configured to input the media data into a model trained by the model training method according to any one of claims 1 to 6, and obtain a first score of the media data by using a first branch of the model; wherein the first branch is used to distinguish between positive samples and negative samples; A first judgment module, configured to determine that the live broadcast content violates the regulations if the first score is greater than or equal to a first threshold; A second scoring module, configured to obtain a second score of the media data using a second branch of the model if the first score is less than a first threshold; wherein the second branch is configured to distinguish between positive samples and FP samples of the first branch; The second judgment module is used to determine that the live broadcast content violates the regulations if the second score is greater than or equal to a second threshold, otherwise, determine that the live broadcast content does not violate the regulations.

10. A model training device, comprising: a memory for storing non-transitory computer-readable instructions; as well as A processor, configured to run the computer-readable instructions, so that when the computer-readable instructions are executed by the processor, the model training method described in any one of claims 1 to 6 is implemented.

11. A device for identifying illegal live broadcasts, comprising: a memory for storing non-transitory computer-readable instructions; as well as A processor is used to run the computer-readable instructions so that when the computer-readable instructions are executed by the processor, the method for identifying illegal live broadcasts according to claim 7 is implemented.

12. A computer storage medium, characterized in that: It includes computer instructions, which, when executed on a device, enable the device to execute the model training method described in any one of claims 1 to 6, or the illegal live broadcast identification method described in claim 7.

Images