A risk control method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a risk control method, device, electronic device and storage medium, the method comprising: obtaining interactive information of at least two first users on output content of a second user within a set period according to a preset frequency; and performing risk control on the output content of the second user according to the interactive information. Through the technical solution of the embodiment of the present invention, the purpose of performing risk control on the output content of the second user based on the interactive information of the first user on the output content of the second user is achieved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a risk control method, device, electronic device, and storage medium. Background Art

[0002] Live broadcast refers to a network interaction method that uses live broadcast technology to communicate closely, display products, or answer inquiries through some Internet platforms. It is a new type of shopping guide service on shopping platforms, where professional anchors promote and introduce products. Live broadcast has become a very popular interactive method in various fields such as e-commerce, games, and entertainment.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] The current mainstream risk control method focuses on manual review and video image recognition. It samples live video images, then automatically recognizes the sampled video images through algorithms, and then manually checks the video images that may be risky. This method has a high cost of human review, and because live video images are constantly changing, the sampling recognition method is prone to miss risky content, resulting in low risk control accuracy. Recognizing all video images requires a huge computing cost, which cannot be achieved by general algorithms. Summary of the invention

[0005] Embodiments of the present invention provide a risk control method, device, electronic device and storage medium, which achieve the purpose of risk controlling the output content of a second user based on the interaction information of a first user with respect to the output content of the second user.

[0006] In a first aspect, an embodiment of the present invention provides a risk control method, the method comprising:

[0007] Acquiring, at a preset frequency, interaction information between at least two first users with respect to output content of a second user within a set period;

[0008] The output content of the second user is risk controlled according to the interaction information.

[0009] In a second aspect, an embodiment of the present invention further provides a wind control device, including:

[0010] An acquisition module, configured to acquire, at a preset frequency, interaction information of at least two first users with respect to output content of a second user within a set period;

[0011] A risk control module is used to perform risk control on the output content of the second user according to the interaction information.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0013] one or more processors;

[0014] A memory for storing one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the risk control method steps provided in any embodiment of the present invention.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the risk control method provided in any embodiment of the present invention.

[0017] The embodiments of the above invention have the following advantages or beneficial effects:

[0018] By acquiring the interaction information of at least two first users on the output content of the second user within a set period at a preset frequency; and performing risk control on the output content of the second user according to the interaction information, the purpose of risk control on the output content of the second user based on the interaction information of the first user on the output content of the second user is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flow chart of a risk control method provided by Embodiment 1 of the present invention;

[0020] Figure 2 This is a flow chart of a risk control method provided by Embodiment 2 of the present invention;

[0021] Figure 3 This is a flow chart of a risk control method provided by Embodiment 3 of the present invention;

[0022] Figure 4 is a flow chart of another risk control method provided by Embodiment 3 of the present invention;

[0023] Figure 5 is a schematic diagram of the structure of a wind control device provided in Embodiment 4 of the present invention;

[0024] Figure 6 It is a structural schematic diagram of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0026] Embodiment 1

[0027] Figure 1 This is a flow chart of a risk control method provided in Embodiment 1 of the present invention. The technical solution of this embodiment can be applied to risk control of live broadcast content of anchors, and can also be applied to risk control of group chat content and risk control of content published or released by users. The method can be executed by a risk control device, which can be implemented by software and / or hardware and integrated into an electronic device, such as a server.

[0028] like Figure 1 As shown, the method specifically comprises the following steps:

[0029] Step 110: Obtain, at a preset frequency, interaction information between at least two first users with respect to output content of a second user within a set period.

[0030] Among them, the first user includes the audience watching the live broadcast room, the members participating in the group chat, or the users participating in the comments of the social forum; the second user includes the anchor of the live broadcast room, the group chat member, or the blogger who publishes articles and videos in the social forum. The technical solution of the embodiment of the present invention is to perform risk control on the output content of the target user based on the group interaction behavior of the audience by changing the angle, rather than performing risk control from the output content of the target user itself. For example, the risk control of the live broadcast content of the anchor in the live broadcast room is not performed by identifying and monitoring the live broadcast content itself, but by risk control identification and monitoring of the live broadcast content of the anchor through the interactive messages between the audience watching the live broadcast and the anchor. For example, the risk control of the articles published by the blogger of the social forum is not based on the direct identification and monitoring of the articles published by the blogger itself, but through the group behaviors such as comment information, likes, and forwarding behaviors of other users on the articles published by the blogger to identify the articles published by the blogger.

[0031] The following description will be given by taking the first user as a viewer of a live broadcast room, the second user as a host of the live broadcast room, and performing risk control on the host's live broadcast content through interactive information between the viewer and the host as an example.

[0032] The preset frequency may be, for example, collecting data once every one minute, and collecting interaction information of viewers on the live broadcast content of the target anchor within one minute each time, that is, the set period is 1 minute.

[0033] The interactive information includes at least one of the following: barrage information, like information, gift information and login information.

[0034] Specifically, the interaction information of the audience with respect to the live broadcast content of the target anchor can be obtained through technologies such as embedding points and logs.

[0035] Step 120: Perform risk control on the output content of the second user according to the interaction information.

[0036] Exemplarily, when the first user includes a viewer watching a live broadcast room, and the second user includes a host of the live broadcast room, the interactive information includes at least one of the following: barrage information, like information, gift information, and login information, and the output content includes the host's live broadcast content.

[0037] The performing risk control on the live broadcast content of the target anchor according to the interaction information includes:

[0038] Determining risk text features according to the barrage information in the interactive information;

[0039] The live broadcast content of the target anchor is risk controlled according to the risk text feature and other information in the interactive information.

[0040] Furthermore, determining risk text features according to the bullet screen information in the interactive information includes:

[0041] According to the barrage information in the interaction information, determining whether the increase rate of the total amount of barrage information sent by each audience to the live broadcast content of the target anchor in the current set period exceeds a set threshold;

[0042] If yes, perform text clustering on the bullet message information within the current set period to obtain clustered text;

[0043] The risk text feature is determined according to the clustered text.

[0044] The determination of the increase rate of the total amount of barrage information sent by each audience to the live broadcast content of the target anchor within the current set period can be: assuming that the set period is 1 minute, the total amount of barrage information sent by each audience who logged into the live broadcast room to the live broadcast content of the target anchor within the current 1 minute is 15,000, and the total amount of barrage information sent by each audience who logged into the live broadcast room to the live broadcast content of the target anchor within the previous 1 minute is 10,000, then the increase rate can be calculated as: 50%. The set threshold can be determined according to the preset frequency and the length of the set period. For example, if the set period is longer, the set threshold can be set higher, and if the set period is shorter, the set threshold can be set lower.

[0045] The method of text clustering the barrage information within the current set period can adopt the TF-IDF (Term Frequency Inverse Document Frequency) algorithm or the neural network model. Among them, TF-IDF is a commonly used weighted technology for information retrieval and text mining. It is a statistical method used to evaluate the importance of a word to a file set or one of the files in a corpus. The importance of a word increases in direct proportion to the number of times it appears in the file, but at the same time it decreases in inverse proportion to the frequency of its appearance in the corpus. The main idea is that if a word appears in an article with a high TF frequency and rarely appears in other articles, it is considered that this word or phrase has a good ability to distinguish categories and is suitable for classification.

[0046] Furthermore, determining the risk text feature according to the clustered text includes:

[0047] Determine whether the bullet message sent by each audience in the current set period has convergence based on the clustered text;

[0048] If the bullet-screen messages sent by the viewers in the current set period have similarity, then the clustered text is subjected to sensitive word recognition;

[0049] If the clustered text contains sensitive words, the risk text feature is determined according to the number of bullet screen messages containing sensitive words and the total number of bullet screen messages in the current set period. Specifically, the bullet screen messages containing sensitive words, the number of bullet screen messages containing sensitive words, and the total number of bullet screen messages in the current set period can all be used as the risk text feature.

[0050] The step of determining whether the bullet screen messages sent by each audience in the current set period have similarity based on the clustered texts includes:

[0051] Determine the ratio between the number of text categories after clustering and the total number of barrage information;

[0052] If the ratio is less than the ratio threshold, it is determined that the barrage information sent by each audience within the current set period is similar.

[0053] It can be understood that if the number of text categories after clustering is smaller, it means that the convergence between the barrage information is stronger, that is, the more similar barrage information there is, it means that the popularity of the live broadcast room is abnormal to a certain extent.

[0054] The technical solution of this embodiment proposes a method for risk control of live broadcast content based on the interactive information between the audience and the anchor. It analyzes the risk behavior of the audience in an automated and intelligent real-time manner, and uses the feedback information from the audience on the anchor's live broadcast content to reversely infer the risks of the live broadcast content. It provides a method for quickly locating risky live broadcast content, which can effectively solve the problem of difficulty in identifying risky behaviors on the anchor side; it avoids the problem of missed detection that exists in the method of sampling live video images and then detecting the sampled video images, and saves the cost of manual review.

[0055] Embodiment 2

[0056] Figure 2 This is a flow chart of a risk control method provided in the second embodiment of the present invention. Based on the above embodiment, this embodiment further optimizes the above step of "risk control of the live broadcast content of the target anchor according to the risk text features and other information in the interactive information". The benefit of the optimization is that the accuracy of risk control can be further improved. The explanations of the terms that are the same or corresponding to the above embodiment are not repeated here.

[0057] See also Figure 2 The risk control method provided in this embodiment specifically includes the following steps:

[0058] Step 210: Obtain the interaction information of the audience with respect to the live broadcast content of the target anchor within a set period according to a preset frequency.

[0059] Step 220: Determine risk text features based on the barrage information in the interactive information.

[0060] Step 230: Determine the audience's behavioral consistency characteristics based on the like information and / or gift information in the interactive information.

[0061] Exemplarily, determining the audience's behavior consistency characteristics based on the like information and / or gift information in the interaction information includes:

[0062] Determine a ratio between the number of like information and / or gift information in the interaction information and the total number of the interaction information;

[0063] The number of the like information and / or gift information, the total number of the interaction information, and the ratio are determined as the behavior consistency characteristics of the audience.

[0064] For example, the ratio between the number of likes information and the total number of interaction information is determined as the behavior consistency feature of the audience. The number of likes information represents the number of viewers who like it. If all the online viewers in the current time period have liked it, it means that the audience's behavior consistency is high, and there is suspicion of being driven by the anchor. For example, the anchor can use the audience's likes as a condition. If all online viewers like it, or the number of likes reaches a threshold, certain live content will be broadcast.

[0065] The ratio between the number of gift-sending messages and the total number of interactive messages can also be determined as a behavioral consistency feature of the audience. The number of gift-sending messages represents the number of viewers who send gifts. If all online viewers in the current period have sent gifts, it means that the audience's behavior is highly consistent and there is suspicion of being driven by the host. For example, the host can make the sending of gifts by the audience a condition. If all online viewers send gifts to the host, or the number of gifts reaches a threshold, certain live broadcast content will be performed.

[0066] The ratio of the number of likes information to the number of gift-giving information and the total number of interactive information can also be determined as the behavioral consistency feature of the audience. The number of gift-giving information and the number of likes information represent the number of viewers who send gifts and like them. If all the online viewers in the current time period have gift-giving behavior or have sent gifts, it means that the audience's behavior consistency is high and there is suspicion of being driven by the host. For example, the host can make the sending of gifts or likes by the audience a condition. If all online viewers send gifts or likes to the host, certain live content will be broadcast.

[0067] Step 240: Determine the login increase characteristics of the target anchor's live broadcast room within the current set period based on the login information in the interactive information.

[0068] Exemplarily, determining the login increase feature of the target anchor's live broadcast room in the current set period according to the login information in the interaction information includes:

[0069] The login increase feature is determined based on the number of logins to the target anchor's live broadcast room in the previous set period adjacent to the current set period and the number of logins to the target anchor's live broadcast room in the current set period.

[0070] The login increase feature may specifically be the increase rate of the number of people logging into the live broadcast room. For example, assuming that the set period is 1 minute, the number of viewers logging into the live broadcast room within the current 1 minute is 15,000, and the number of viewers logging into the live broadcast room within the previous 1 minute is 5,000. The increase rate of the number of people logging into the live broadcast room can be calculated as: 200%, and this value can be determined as the login increase feature. If the number of people logging into the live broadcast room suddenly changes within a certain period of time, the probability that the live broadcast room is at risk is relatively high, so this can be used as a risk identification feature to assist in risk identification of the live broadcast room.

[0071] Step 250: Perform risk control on the live broadcast content of the target anchor based on the risk text feature, the consistency feature, and the login increase feature.

[0072] Specifically, the risk control of the live broadcast content of the target anchor based on the risk text feature, the consistency feature, and the login increase feature includes:

[0073] The risk text feature, the consistency feature and the login increase feature are input into a preset risk identification model, and the probability that the live broadcast content of the target anchor contains risky content is output through the preset risk identification model.

[0074] Among them, the preset risk identification model can specifically be a convolutional neural network CNN, a decision tree algorithm or a lightGBM algorithm. LightGBM is a gradient boosting framework that uses a tree-based learning algorithm.

[0075] The technical solution of this embodiment proposes a method for risk control of live broadcast content based on the interactive information between the audience and the anchor. Specifically, the risk text features are determined based on the barrage information sent by the audience to the anchor's live broadcast content, the audience's behavior consistency features are determined based on the audience's likes and gift-giving behaviors to the anchor, and the login increase features are determined based on the increase in the number of people logging into the live broadcast room within a set period. Then, the live broadcast content of the live broadcast room is identified for risk control based on the risk text features, the behavior consistency features and the login increase features. Through automatic and intelligent real-time analysis of the audience's risk behavior, the feedback information from the audience on the anchor's live content is used to reversely infer the risks of the live broadcast content. A method for quickly locating risky live broadcast content is provided, which can effectively solve the problem of difficulty in identifying risky behaviors on the anchor side.

[0076] Embodiment 3

[0077] Figure 3This is a flow chart of a risk control method provided in the third embodiment of the present invention. Based on the above embodiments, this embodiment adds a manual review step to further improve the risk control accuracy of the live broadcast room. At the same time, if it is manually confirmed that the live broadcast content of the live broadcast room does contain risky content, the identified risky content will be used as a new annotation sample, and the preset risk identification model will be trained online to update the preset risk identification model and continuously improve the recognition performance of the preset risk identification model. At the same time, operations for managing and controlling risky live broadcast rooms are added to reduce the negative impact of risky live broadcast rooms on viewers. The explanations of the terms that are the same or corresponding to the above embodiments will not be repeated here.

[0078] See also Figure 3 The risk control method provided in this embodiment specifically includes the following steps:

[0079] Step 310: Obtain the interaction information of the audience with respect to the live broadcast content of the target anchor within a set period according to a preset frequency.

[0080] Step 320: determine the risk text features based on the barrage information in the interactive information, determine the audience's behavior consistency features based on the like information and / or gift information in the interactive information, and determine the login increase features of the target anchor's live broadcast room within the current set period based on the login information in the interactive information.

[0081] Step 330: input the risk text feature, the consistency feature, and the login increase feature into a preset risk identification model, and output the probability that the live broadcast content of the target anchor contains risky content through the preset risk identification model.

[0082] Step 340: If the probability is greater than the probability threshold, a risk check is performed on the live broadcast content within the current set period through manual review.

[0083] Step 350: If it is determined through manual review that the live broadcast content within the current set period contains risky content, the risk text features, the consistency features and the login increase features within the current set period are used as manually labeled samples to train the preset risk identification model to update the preset risk identification model.

[0084] Step 360: If the live broadcast content of the target anchor contains risky content, the live broadcast room of the target anchor is subjected to flow control based on a preset mechanism to prevent new viewers from logging into the live broadcast room, or to prevent the delivery of barrages sent by viewers.

[0085] It is understandable that if the live broadcast content of the live broadcast room is extremely risky and the impact caused is extremely bad, the current limiting control can also directly block the live broadcast room of the target anchor, that is, the target anchor no longer has the authority to broadcast live, so as to optimize the live broadcast ecology.

[0086] For further reference, Figure 4 The flowchart of another risk control method shown in the figure specifically includes: the background of the live broadcast room sets up a feature collection system for each live broadcast room, and the feature collection system sends the abnormal group behavior characteristics of the audience (such as the above-mentioned risk text characteristics, consistency characteristics and login increase characteristics) into the preset risk identification model, and the characteristics of each live broadcast room are sent to the preset risk identification model as a single sample. The preset risk identification model performs risk assessment on the current live broadcast room based on the received abnormal group behavior characteristics. If the output value of the preset risk identification model is high, that is, it is identified that the current live broadcast room has a high risk, the live broadcast content of the current live broadcast room is sent to the manual review system for manual data review. If it is determined through manual review that the current live broadcast room has a high-risk live broadcast behavior, the live broadcast line is controlled or the anchor is punished, such as preventing new viewers from logging into the current live broadcast room, preventing the delivery of the barrage sent by the audience, or closing the current live broadcast room, canceling the anchor's live broadcast account, etc. At the same time, the abnormal group behavior characteristics of the current live broadcast room are stored for automatic training and updating of the preset risk identification model, and the training results are loaded into the online risk identification model for risk control identification of subsequent live broadcast rooms.

[0087] The technical solution of this embodiment, based on the above embodiments, adds a manual review step if the probability of the target anchor's live broadcast content containing risky content is greater than the probability threshold output by the preset risk identification model, further improving the risk control accuracy of the live broadcast room. At the same time, if it is manually confirmed that the live broadcast content of the live broadcast room does contain risky content, the identified risky content is used as a new annotation sample, and the preset risk identification model is trained online to update the preset risk identification model and continuously improve the recognition performance of the preset risk identification model. At the same time, the operation of controlling the risk live broadcast room is added to reduce the negative impact of the risk live broadcast room on the audience.

[0088] The following is an embodiment of a wind control device provided in an embodiment of the present invention. The device and the wind control methods of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the wind control device, reference can be made to the embodiments of the above wind control methods.

[0089] Embodiment 4

[0090] Figure 5This is a schematic diagram of the structure of a risk control device provided in Embodiment 4 of the present invention. The device specifically includes: an acquisition module 510 and a risk control module 520.

[0091] Among them, the acquisition module 510 is used to obtain the interaction information of at least two first users on the output content of the second user within a set period at a preset frequency; the risk control module 520 is used to perform risk control on the output content of the second user according to the interaction information.

[0092] Furthermore, when the first user includes viewers watching a live broadcast room, and the second user includes a host of the live broadcast room, the interactive information includes at least one of the following: barrage information, like information, gift information, and login information, and the output content includes the host's live broadcast content.

[0093] Furthermore, the risk control module 520 includes:

[0094] A risk text feature determination submodule, used to determine risk text features according to the barrage information in the interactive information;

[0095] The risk control submodule is used to perform risk control on the live broadcast content of the target anchor according to the risk text features and other information in the interactive information.

[0096] Furthermore, the risk text feature determination submodule includes:

[0097] A first determining unit is used to determine whether the increase rate of the total amount of barrage information sent by each audience to the live broadcast content of the target anchor in the current set period exceeds a set threshold according to the barrage information in the interaction information;

[0098] A clustering unit, configured to perform text clustering on the barrage information in the current set period to obtain clustered text if the increase rate of the total amount of barrage information sent by each audience to the live broadcast content of the target anchor in the current set period exceeds a set threshold;

[0099] The second determining unit is used to determine the risk text feature according to the clustered text.

[0100] Furthermore, the second determining unit includes:

[0101] A first determination subunit is used to determine whether the barrage information sent by each audience in a current set period has convergence based on the clustered text;

[0102] The identification subunit is used to identify sensitive words in the clustered text if the bullet message sent by each audience in the current set period has convergence;

[0103] The second determination subunit is used to determine the risk text feature according to the number of barrage information containing sensitive words and the total number of barrage information in a current set period if there are sensitive words in the clustered text.

[0104] Furthermore, the first determination subunit is specifically used to: determine the ratio between the number of text categories after clustering and the total number of barrage information; if the ratio is less than a ratio threshold, determine that the barrage information sent by each audience within the current set period is similar.

[0105] Furthermore, the risk control submodule includes:

[0106] A behavior consistency feature determination unit, configured to determine the behavior consistency feature of the audience according to the like information and / or gift information in the interaction information;

[0107] A login increase feature determination unit, configured to determine a login increase feature of the target anchor's live broadcast room within a current set period according to the login information in the interaction information;

[0108] The risk control unit is used to perform risk control on the live broadcast content of the target anchor based on the risk text feature, the consistency feature and the login increase feature.

[0109] Furthermore, the behavior consistency feature determination unit includes:

[0110] A first determining subunit is used to determine a ratio between the number of like information and / or gift information in the interaction information and the total number of the interaction information;

[0111] The second determination subunit is used to determine the number of the like information and / or gift information, the total number of the interaction information and the ratio as the behavior consistency characteristics of the audience.

[0112] Furthermore, the login increase feature determination unit is specifically used for:

[0113] The login increase feature is determined based on the number of logins to the target anchor's live broadcast room in the previous set period adjacent to the current set period and the number of logins to the target anchor's live broadcast room in the current set period.

[0114] Furthermore, the wind control unit is specifically used for:

[0115] The risk text feature, the consistency feature and the login increase feature are input into a preset risk identification model, and the probability that the live broadcast content of the target anchor contains risky content is output through the preset risk identification model.

[0116] Furthermore, the device also includes:

[0117] The verification module is used to perform risk verification on the live broadcast content within the current set period through manual review if the probability is greater than the probability threshold.

[0118] Furthermore, the device also includes:

[0119] The training module is used to train the preset risk identification model using the risk text features, consistency features and login increase features within the current set period as manually annotated samples if it is determined through manual review that the live broadcast content within the current set period contains risky content, so as to update the preset risk identification model.

[0120] Furthermore, the device also includes:

[0121] The control module is used to perform flow control on the live broadcast room of the target anchor based on a preset mechanism if the live broadcast content of the target anchor contains risky content, so as to prevent new viewers from logging into the live broadcast room, or to prevent the delivery of barrages sent by viewers.

[0122] The technical solution of this embodiment proposes a method for risk control of live broadcast content based on the interactive information between the audience and the anchor. By automatically and intelligently analyzing the risky behavior of the audience in real time, the feedback information from the audience on the anchor's live broadcast content is used to reversely infer the risks of the live broadcast content. This provides a method for quickly locating risky live broadcast content, which can effectively solve the problem of difficulty in identifying risky behavior on the anchor side.

[0123] The risk control device provided in the embodiment of the present invention can execute the risk control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the risk control method.

[0124] Embodiment 5

[0125] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. Figure 6 A block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0126] like Figure 6 As shown, the electronic device 12 is in the form of a general purpose computing electronic device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting various system components (including the system memory 28 and the processing unit 16).

[0127] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0128] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0129] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set of program modules (e.g., at least one acquisition module 510 and a risk control module 520) that are configured to perform the functions of various embodiments of the present invention.

[0130] A program / utility 40 having a set of program modules 42 (at least one acquisition module 510 and a risk control module 520) may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0131] The electronic device 12 may also communicate with one or more external electronic devices 14 (e.g., keyboards, pointing electronic devices, displays 24, etc.), may also communicate with one or more electronic devices that enable a user to interact with the electronic device 12, and / or communicate with any electronic device (e.g., a network card, a modem, etc.) that enables the electronic device 12 to communicate with one or more other computing electronic devices. Such communication may be performed through an input / output (I / O) interface 22. In addition, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, electronic device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0132] The processing unit 16 executes various functional applications and risk control by running the program stored in the system memory 28, for example, implementing a risk control method provided by the embodiment of the present invention, the method comprising:

[0133] Acquiring, at a preset frequency, interaction information between at least two first users with respect to output content of a second user within a set period;

[0134] The output content of the second user is risk controlled according to the interaction information.

[0135] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the risk control method provided by any embodiment of the present invention.

[0136] Embodiment 6

[0137] This embodiment 6 provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the risk control method provided in any embodiment of the present invention are implemented. The method includes:

[0138] Acquiring, at a preset frequency, interaction information between at least two first users with respect to output content of a second user within a set period;

[0139] The output content of the second user is risk controlled according to the interaction information.

[0140] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0141] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0142] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0143] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0144] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0145] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A risk control method, characterized in that: include: Acquiring, at a preset frequency, interaction information between at least two first users with respect to output content of a second user within a set period; performing risk control on the output content of the second user according to the interaction information; The first user includes viewers watching the live broadcast room, the second user includes the anchor of the live broadcast room, and the output content includes the live broadcast content of the anchor; The performing risk control on the output content of the second user according to the interaction information includes: According to the barrage information in the interaction information, determining whether the increase rate of the total amount of barrage information sent by each audience to the live broadcast content of the target anchor in the current set period exceeds a set threshold; If yes, perform text clustering on the bullet message information within the current set period to obtain clustered text; Determining risk text features based on the clustered texts; The live broadcast content of the target anchor is risk controlled according to the risk text features.

2. The method according to claim 1, characterized in that The interactive information includes at least one of the following: barrage information, like information, gift information and login information.

3. The method according to claim 1, characterized in that: The step of determining the risk text feature according to the clustered text includes: Determine whether the bullet message sent by each audience in the current set period has convergence based on the clustered text; If the bullet-screen messages sent by the viewers in the current set period have similarity, then the clustered text is subjected to sensitive word recognition; If the clustered text contains sensitive words, the risk text feature is determined according to the number of barrage information containing the sensitive words and the total number of barrage information in the current set period.

4. The method according to claim 3, characterized in that: The determining whether the bullet screen messages sent by each audience in the current set period have similarity based on the clustered texts includes: Determine the ratio between the number of text categories after clustering and the total number of barrage information; If the ratio is less than the ratio threshold, it is determined that the barrage information sent by each audience within the current set period is similar.

5. The method according to claim 2, characterized in that: The risk control of the live broadcast content of the target anchor according to the risk text feature includes: Determining the audience's behavioral consistency characteristics based on the like information and / or gift information in the interactive information; Determine the login increase characteristics of the target anchor's live broadcast room within the current set period according to the login information in the interactive information; The live broadcast content of the target anchor is risk controlled based on the risk text feature, the consistency feature and the login increase feature.

6. The method according to claim 5, characterized in that The determining of the audience's behavior consistency characteristics according to the like information and / or gift information in the interactive information includes: Determine a ratio between the number of like information and / or gift information in the interaction information and the total number of the interaction information; The number of the like information and / or gift information, the total number of the interaction information, and the ratio are determined as the behavior consistency characteristics of the audience.

7. The method according to claim 5, characterized in that The determining, according to the login information in the interaction information, the login increase feature of the target anchor's live broadcast room within the current set period includes: The login increase feature is determined based on the number of logins to the target anchor's live broadcast room in the previous set period adjacent to the current set period and the number of logins to the target anchor's live broadcast room in the current set period.

8. The method according to claim 5, characterized in that The risk control of the live broadcast content of the target anchor based on the risk text feature, the consistency feature and the login increase feature includes: The risk text feature, the consistency feature and the login increase feature are input into a preset risk identification model, and the probability that the live broadcast content of the target anchor contains risky content is output through the preset risk identification model.

9. The method according to claim 8, characterized in that Also includes: If the probability is greater than the probability threshold, a risk check is performed on the live broadcast content within the current set period through manual review.

10. The method according to claim 9, characterized in that Also includes: If it is determined through manual review that the live broadcast content within the current set period contains risky content, the risk text features, the consistency features and the login increase features within the current set period are used as manually labeled samples to train the preset risk identification model to update the preset risk identification model.

11. The method according to any one of claims 1 to 9, characterized in that: Also includes: If the live broadcast content of the target anchor contains risky content, the live broadcast room of the target anchor will be subject to flow control based on a preset mechanism to prevent new viewers from logging into the live broadcast room, or to prevent the delivery of barrages sent by viewers.

12. A wind control device, characterized in that: include: An acquisition module, configured to acquire, at a preset frequency, interaction information of at least two first users with respect to output content of a second user within a set period; a risk control module, configured to perform risk control on the output content of the second user according to the interaction information; The first user includes viewers watching the live broadcast room, the second user includes the anchor of the live broadcast room, and the output content includes the live broadcast content of the anchor; The wind control module includes: A risk text feature determination submodule, used to determine risk text features according to the barrage information in the interactive information; A risk control submodule, used to perform risk control on the live broadcast content of the target anchor according to the risk text features; Wherein, the risk text feature determination submodule includes: A first determination unit is used to determine whether the increase rate of the total amount of barrage information sent by each audience to the live broadcast content of the target anchor in the current set period exceeds a set threshold according to the barrage information in the interaction information; A clustering unit, configured to perform text clustering on the barrage information in the current set period to obtain clustered text if the increase rate of the total amount of barrage information sent by each audience to the live broadcast content of the target anchor in the current set period exceeds a set threshold; The second determining unit is used to determine the risk text feature according to the clustered text.

13. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the risk control method steps as described in any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the risk control method as described in any one of claims 1 to 11 are implemented.

Images