Live broadcast platform information security detection method and its device, equipment, medium and product

Abstract

The present application discloses a method for detecting information security of a live broadcast platform and its apparatus, equipment, medium, and product, the method comprising: receiving a public release message submitted by an application that has pre-agreed with the live broadcast platform, the public release message comprising characteristic information of the application, a publishing user, and a public release text; calling filtering rules corresponding to the characteristic information of the application from a preset rule library, the filtering rules comprising the characteristic information of the application, the classification dictionary specified information adopted by the application, and the classification dictionary penalty type; filtering the public release text according to the filtering rules to determine one or more corresponding penalty types; calling the penalty processing interface corresponding to the highest level penalty type to process the public release message, and returning a corresponding notification message to the publishing user of the application. The present application can realize the decoupling and coordination of various configuration resources and business logics in the process of information security detection, and can improve the information security of the live broadcast platform.

Description

Technical Field

[0001] The present application relates to the field of network live broadcast technology, and in particular to a live broadcast platform information security detection method and its corresponding device, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In the online live broadcast ecosystem, there is a large amount of text business information, such as information, titles, barrage, comments, etc. With the rapid increase in the number of users and information, spam and network regulatory content have also become rampant, hindering user communication and affecting the health of the platform ecosystem.

[0003] As the main way to improve the availability of network information, filtering spam has become an important task in the Internet field. There are regular expression-based filtering and machine learning-based filtering technologies in network live spam filtering, but the most basic one is keyword-based filtering technology.

[0004] Traditional business keyword filtering uses one business to hang multiple dictionaries (such as A1, B1, C1), and different dictionary hit results execute different decision logic. If there is one more business, multiple dictionaries will be hung (such as A1, B2, C2, where B2&B1, C2&C1 are dictionaries of the same type, such as pornographic dictionaries). As the number of businesses increases, each business may have a pornographic dictionary, and these n dictionaries may have a large amount of duplicate content, or keywords entered according to different standards.

[0005] One of the traditional keyword filtering modes, for example, in the comment business, when filtering keywords, the business operation divides the penalty keyword library, the comment deletion keyword library, the comment hiding keyword library, etc. according to the needs of the scene, and the keyword filtering matches the above word libraries separately. In this way, the business will add words to each word library according to different disposal methods. The advantage of this move is that it is intuitive and easy to understand and can add new entries on demand, but its disadvantages are more obvious, namely: 1. There will be various types of words in a single word library, such as the penalty library will have words with serious pornographic attributes and malicious insults; 2. A single word may be repeatedly added to multiple word libraries; 3. When adding a new business, the word library is not universal and cannot be reused (such as the penalty keywords of comments and barrages, there are intersections, but they are not completely consistent), and the new business has to maintain a new dictionary; 4. When a new penalty method is added, the word library may be split and merged again.

[0006] Another traditional keyword filtering mode, such as comment business scenarios, directly regulates pornography, vulgarity and other types, and establishes a vocabulary library according to the supervision type, and each vocabulary library corresponds to different disposal methods. When filtering keywords, the corresponding disposal method is executed according to the type matching result. Its advantage is that it is also intuitive and easy to understand according to the supervision scope, but its disadvantages are also more obvious. For example: 1. The supervision vocabulary library established for the business scenario is only used under this business and cannot be used in all business scenarios (for example: abusive words that cannot appear in comments may be allowed in barrages). In this way, different businesses need to maintain different abusive vocabulary libraries. 2. The vocabulary and disposal methods are not universal (for example, the disposal method for abuse in comments is deletion, and the disposal method for abuse in private chats is no punishment).

[0007] In the online live broadcast ecosystem, the live broadcast platform operates multiple application products and has multiple application scenarios. There are many business scenarios where text information appears. At the same time, there are different classification criteria and punishment methods for text filtering in different scenarios. In addition, with the accumulation of entries, the vocabulary has become a complex task for content operation. It is necessary to manually classify the entries and associate them with the business. It also requires more manpower and a certain amount of work accumulation to master the configuration of various filtering rules. In addition, there is a professional division of labor between business operations and content security operations, which belong to two different teams. The business does not understand the scale and standard of text penalties, and the content security does not understand which keyword dictionaries the business needs to connect to and what kind of hit penalties to perform. Therefore, these characteristics in reality require the live broadcast platform to provide a more efficient management system based on its own characteristics, so the upgrading of related technologies has become an expectation of the industry. Summary of the invention

[0008] The primary purpose of the present application is to solve at least one of the above problems and to provide a live broadcast platform information security detection method and its corresponding device, computer equipment, computer-readable storage medium, and computer program product.

[0009] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0010] A live broadcast platform information security detection method proposed to meet one of the purposes of this application includes the following steps:

[0011] Receiving a public release message submitted by an application that has pre-agreed with the live broadcast platform, wherein the public release message includes feature information of the application, a publishing user, and a public release text;

[0012] Calling a filtering rule corresponding to the characteristic information of the application from a preset rule library, wherein the filtering rule includes the characteristic information of the application, the classification dictionary designation information adopted by the application, and the classification dictionary penalty type;

[0013] Filtering the publicly released text according to the filtering rules, and determining one or more corresponding penalty types when a keyword in the publicly released text hits an entry in one or more classification dictionaries of the specified information;

[0014] The penalty processing interface corresponding to the highest level penalty type among the determined penalty types is called to process the public release message, and a corresponding notification message is returned to the release user of the application.

[0015] In a specific embodiment, the step of receiving a public release message submitted by an application that has pre-agreed with the live broadcast platform includes the following steps:

[0016] Concurrently receiving message publishing requests from applications that have pre-agreed with the live broadcast platform, and adding the message publishing requests to the request queue in order;

[0017] Control the request queue to flow out the message publishing request according to a preset dequeue rule, and start a consumption thread corresponding to the message publishing request;

[0018] The consumer thread parses the message publishing request to obtain the publicly published message therein.

[0019] In a specific embodiment, filtering the publicly released text according to the filtering rule includes the following steps:

[0020] Calling a named entity recognition model that has been pre-trained to a convergence state to extract a plurality of named entities from the publicly released text;

[0021] Matching the named entity with each preset classification dictionary, determining the named entity that hits any entry in any of the classification dictionaries as a hit keyword obtained by word segmentation of the publicly released text, and obtaining mapping relationship data between the hit keyword and the classification dictionary;

[0022] According to the filtering rules corresponding to the classification dictionary in the mapping relationship data between the hit keyword and the classification dictionary, the corresponding penalty type is determined.

[0023] In a further embodiment, the named entity recognition model performs the following steps:

[0024] Vectorizing the publicly released text to obtain corresponding encoding information;

[0025] Extracting features from the encoded information to obtain a text feature vector representing its deep semantic information;

[0026] The publicly released text is sequence labeled according to the text feature vector to extract each named entity therein.

[0027] In a specific embodiment, calling the penalty processing interface corresponding to the highest level penalty type among the determined penalty types to process the public release message and returning a corresponding notification message to the release user of the application program includes the following steps:

[0028] Compare multiple penalty types and determine the highest level of penalty type;

[0029] Call and run the penalty processing interface corresponding to the highest level penalty type;

[0030] The penalty processing interface processes the publicly released text according to the preset business logic, including shielding the keywords of the hit entry or deleting the full text of the publicly released text, and returns the processing result;

[0031] A corresponding notification message is generated according to the processing result, and the notification message is sent to the instant messaging interface of the publishing user.

[0032] In the extended embodiment, the following steps are also included:

[0033] In response to the remote call to the first service interface, a rule base editing page is pushed to the corresponding terminal device, where the page includes classification dictionary list information and penalty type list information;

[0034] Obtaining one or more filtering rules submitted based on the rule base editing page, and storing them in the rule base;

[0035] In response to the remote call to the second service interface, a classification dictionary editing page is pushed to the corresponding terminal device, wherein the classification dictionary list information is included in the page;

[0036] One or more entries submitted based on the classification dictionary edit page and the classification dictionary specified from the classification dictionary list information are obtained, and the entries are stored in the classification dictionary.

[0037] A live broadcast platform information security detection device provided to meet one of the purposes of the present application includes: a message acquisition module, a rule matching module, a security detection module, and a penalty processing module, wherein the message acquisition module is used to receive a public release message submitted by an application that has pre-agreed with the live broadcast platform, and the public release message includes the characteristic information of the application, the publishing user, and the public release text; the rule matching module is used to call a filtering rule corresponding to the characteristic information of the application from a preset rule library, and the filtering rule includes the characteristic information of the application, the classification dictionary specified information adopted by the application, and the classification dictionary penalty type; the security detection module is used to filter the public release text according to the filtering rule, and when the keywords in the public release text hit one or more classification dictionary entries of the specified information, determine the corresponding one or more penalty types; the penalty processing module is used to call the penalty processing interface corresponding to the highest level penalty type among the determined penalty types to process the public release message and return a corresponding notification message to the publishing user of the application.

[0038] In a specific embodiment, the message acquisition module includes: a concurrent receiving sub-module, which is used to concurrently receive message publishing requests from applications that have pre-agreed with the live broadcast platform, and add the message publishing requests to the request queue in order; a request dequeueing sub-module, which is used to control the request queue to flow out the message publishing request according to a preset dequeueing rule, and start the corresponding consumption thread of the message publishing request; a message consumption sub-module, which is used to parse the message publishing request by the consumption thread to obtain the publicly released message therein.

[0039] In a specific embodiment, the security detection module includes: an entity recognition submodule, which is used to call a named entity recognition model that has been pre-trained to a convergent state to extract multiple named entities from the publicly released text; a dictionary matching submodule, which is used to match the named entities with various preset classification dictionaries, and determine the named entities that hit any entry in any of the classification dictionaries as hit keywords obtained by word segmentation of the publicly released text, and obtain mapping relationship data between the hit keywords and the classification dictionaries; a penalty determination submodule, which is used to determine the corresponding penalty type according to the filtering rules corresponding to the classification dictionary in the mapping relationship data between the hit keywords and the classification dictionary.

[0040] In a further embodiment, a control device is constructed when the named entity recognition model is running, and the control device includes: an encoding processing unit, which is used to vectorize the publicly released text to obtain its corresponding encoding information; a feature extraction unit, which is used to extract features from the encoding information to obtain a text feature vector that represents its deep semantic information; an entity extraction unit, which is used to sequence the publicly released text according to the text feature vector to extract each named entity therein.

[0041] In a specific embodiment, the penalty processing module includes: a penalty selection submodule, which is used to compare multiple penalty types and determine the highest level penalty type; an interface calling submodule, which is used to call and run the penalty processing interface corresponding to the highest level penalty type; an interface response submodule, which is used to process the publicly released text according to the preset business logic by the penalty processing interface, including shielding the keywords of the hit entries or deleting the full text of the publicly released text, and returning the processing result; a notification delivery submodule, which is used to generate a corresponding notification message according to the processing result, and send the notification message to the instant messaging interface of the publishing user.

[0042] In an extended embodiment, the live broadcast platform information security detection device of the present application also includes: a first service module, which is used to respond to a remote call to a first service interface, and push a rule library editing page to a corresponding terminal device, wherein the page includes classification dictionary list information and penalty type list information; a first update module, which is used to obtain one or more filtering rules submitted based on the rule library editing page, and store them in the rule library; a second service module, which is used to respond to a remote call to a second service interface, and push a classification dictionary editing page to a corresponding terminal device, wherein the page includes classification dictionary list information; a second update module, which is used to obtain one or more entries submitted based on the classification dictionary editing page and a classification dictionary specified from the classification dictionary list information, and store the entries in the classification dictionary.

[0043] A computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the live broadcast platform information security detection method described in the present application.

[0044] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the live broadcast platform information security detection method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the method are executed.

[0045] A computer program product provided to meet another purpose of the present application includes a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any embodiment of the present application.

[0046] Compared with the prior art, the advantages of the present application are as follows: the present application processes user messages posted in the application of the live broadcast platform through a division of labor and cooperation in multiple steps, including uniformly receiving user messages posted by various applications pre-agreed by the live broadcast platform, calling the corresponding filtering rules of the application through a unified rule library, calling a unified classification dictionary for matching according to the corresponding filtering rules to determine the corresponding penalty type, and centrally processing the penalty types according to a unified mechanism. Among them, each processing step as well as the filtering rules, classification dictionary, penalty processing mechanism, etc. are both independent of each other and cooperate with each other to achieve logical decoupling and coordination, allowing the live broadcast platform to implement flexible and centralized information security detection technical services, thereby improving the information security detection capabilities of the live broadcast platform and ensuring information security in the live broadcast platform ecosystem. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0048] Figure 1 A flowchart of a typical embodiment of the live broadcast platform information security detection method of the present application;

[0049] Figure 2 A flowchart of a process of processing concurrent message publishing requests through a request queue in an embodiment of the present application;

[0050] Figure 3 A flowchart of a process for determining a penalty type using a named entity recognition model in an embodiment of the present application;

[0051] Figure 4 This is a schematic diagram of the network architecture principle of the named entity recognition model exemplified in this application;

[0052] Figure 5 A flowchart of the recognition process of an exemplary named entity recognition model of the present application;

[0053] Figure 6 A flowchart of a process of executing penalty processing according to penalty type in an embodiment of the present application;

[0054] Figure 7 A flowchart of a process of opening filtering rules and classification dictionary maintenance functions through two service interfaces in an embodiment of the present application;

[0055] Figure 8This is a principle block diagram of the live broadcast platform information security detection device of this application;

[0056] Fig. 9 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0057] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0058] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0059] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0060] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0061] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0062] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0063] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0064] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0065] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0066] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0067] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0068] The present application provides a method for detecting information security of a live broadcast platform, which can be programmed as a computer program product and deployed in a server for execution. The method can be executed by accessing an interface opened after the computer program product is executed and performing human-computer interaction with the process of the computer program product through a graphical user interface.

[0069] See also Figure 1 In a typical embodiment, the live broadcast platform information security detection method of the present application includes the following steps:

[0070] Step S1100: Receive a public release message submitted by an application that has pre-agreed with the live broadcast platform, wherein the public release message includes feature information of the application, the publishing user, and the public release text:

[0071] In an exemplary application scenario, the computer program product implemented according to the technical solution of this application is deployed in the server of the service cluster of the network live broadcast platform, and provides open services for a large number of terminal applications implemented by the network live broadcast platform according to a pre-agreement. These applications serve different businesses respectively, but can all transparently transmit the public release messages submitted from their terminal devices to the server of this application according to the agreement. In short, as long as an application can submit the public release message to the server of this application in accordance with the specification of this application and receive corresponding feedback, it can be regarded as an application that has pre-agreed with the live broadcast platform of this application.

[0072] For example, in one of the live broadcast applications, the text information entered by the user of the client device in the live broadcast chat interface of a live broadcast room presented by the application is transmitted to the server as a publicly released text after being processed by its own business logic, and is subsequently processed by the business logic implemented by the technical solution of the present application. For the application running on the client device and pre-agreed with the live broadcast platform, it encapsulates the user's input into a publicly released message, adds the characteristic information of the current application (AppID), the publishing user (which can be represented as the user ID of the current user), and the publicly released text containing the user input. It can then be submitted directly to the server of the present application, or forwarded to the server of the present application through other background servers used to support the implementation of the business logic of the current application.

[0073] For another example, in an application that provides short video services pre-agreed on the live broadcast platform, a user of a client device enters comment text in the comment page of a short video played by the application, which can be used as a publicly released text to encapsulate the publicly released message, and can also be directly or indirectly transmitted to the server of this application for processing.

[0074] As for the server of the present application, it can concurrently respond to public release messages submitted by multiple pre-protocol applications, so as to achieve concurrent response to massive public release messages generated by multiple applications.

[0075] Step S1200: calling a filtering rule corresponding to the characteristic information of the application from a preset rule library, wherein the filtering rule includes the characteristic information of the application, the classification dictionary specified information adopted by the application, and the classification dictionary penalty type:

[0076] In order to serve the information security detection of massive public release messages generated by a large number of applications, the present application pre-constructs a rule library for storing filtering rules that can be edited by the background. The filtering rules can be used to perform keyword filtering of the public release text in the public release messages to ensure information security.

[0077] To cooperate with the filtering rules, the present application also pre-constructs multiple classification dictionaries, each of which is used to store keywords corresponding to a security attribute. The security attributes include, for example, "pornography", "violence", "slander", "vulgarity", etc., which are preset to ensure the healthy development of the information ecology of the live broadcast platform.

[0078] The filtering rules and classification dictionaries are all uniformly set to centrally and uniformly serve the information security detection of all applications pre-agreement on the entire live broadcast platform. Accordingly, for each application, the classification dictionary and the corresponding penalty adopted by the application can be pre-configured to construct one or more filtering rules associated with the application. Accordingly, each filtering rule can be implemented to include the characteristic information of the application, the classification dictionary designation information corresponding to the classification dictionary adopted by the application, and the corresponding penalty type after the publicly released text submitted by the application hits the classification dictionary. The penalty type is used to limit the processing actions required to be performed on the corresponding publicly released message after hitting the classification dictionary. These elements are organized into mapping relationship data and stored in the rule library for easy calling.

[0079] Those skilled in the art can construct the rule base according to the construction principle disclosed above, and provide a background management page for accessing the filtering rules in the rule base, so as to edit and update the filtering rules in the rule base, delete one or more filtering rules, or add new filtering rules to the rule base through the background management page. For example, when a pre-protocol application is added to the live broadcast platform, the corresponding filtering rules for the application can be added to the rule base.

[0080] When it is necessary to perform information security detection on the publicly released text in any publicly released message, first search for filtering rules containing the characteristic information from the rule library based on the characteristic information of the application carried in the publicly released message. There may be multiple filtering rules corresponding to one application, which are used to correspond to multiple designated classification dictionaries, and they can be called out one by one for centralized processing, so as to perform keyword-based information filtering on the publicly released text according to these filtering rules.

[0081] Step S1300: filtering the publicly released text according to the filtering rules, and determining one or more corresponding penalty types when a keyword in the publicly released text hits one or more entries of the classification dictionary of the specified information:

[0082] As mentioned above, each filtering rule contains classification dictionary designation information, which is used to determine the classification dictionary designated by the filtering rule. Therefore, after determining the corresponding classification dictionary according to the designated information, the classification dictionary is used to detect whether the publicly released text contains any one or more entries in the classification dictionary, that is, whether there are one or more keywords in the publicly released text that hit the classification dictionary designated by the designated information. Once the judgment is established, it indicates that the publicly released text contains violation information of the security attribute category corresponding to the classification dictionary, and therefore, it is necessary to apply a penalty to it. Since the penalty is represented as a penalty type in the filtering rule, the corresponding interface call needs to be made according to the penalty type in the subsequent process to execute the penalty action for the publicly released message.

[0083] When detecting for each filtering rule whether it contains entries that hit a certain classification dictionary, the principles of exact matching, fuzzy matching, semantic matching, etc. can be used for implementation. Technically, regular rule matching or keyword extraction technology based on a neural network model in natural language processing technology can be used for implementation. Those skilled in the art can be flexible in this regard.

[0084] For multiple filtering rules corresponding to the same application, after filtering according to each filtering rule one by one, if at least some of the filtering rules match the penalty type, these penalty types can be summarized and passed to the next step for centralized processing to achieve decoupling and coordination of filtering logic and processing logic.

[0085] Step S1400: calling the penalty processing interface corresponding to the highest level penalty type among the determined penalty types to process the public release message, and returning a corresponding notification message to the release user of the application:

[0086] There are multiple preset penalty types, corresponding to different penalty levels. In this application, the same business logic is uniformly implemented, so that among the multiple penalty types output from the previous step, only the penalty type with the highest processing level is applicable. That is, no matter whether a publicly released text triggers multiple penalties, the one with the highest penalty level is always executed. These processing levels can be, for example, "shielding", "interception", "deduction", "penalty", etc. from low to high. For the "shielding" item, other mask characters can be directly used to replace the corresponding keywords that trigger the penalty; for the "interception" item, the publicly released message is directly filtered out and not pushed to its corresponding release address and page; for the "deduction" item, a preset value of points can be deducted from the personal points account of the publishing user; for the "penalty" item, the authority of the publishing user to publicly release information can be modified. Such as this, can be flexibly implemented by those skilled in the art.

[0087] This application implements the penalty business logic corresponding to each penalty type as the corresponding penalty processing interface in advance. When a specific penalty type needs to be applied, the penalty processing interface corresponding to the penalty type is directly called, and the penalty processing interface executes the corresponding penalty business logic on the initiator of the public release message, that is, the publishing user. When the called penalty processing interface completes the processing, a corresponding notification message can be constructed and returned to the publishing user as a warning.

[0088] In the present application, the filtering step S1300 can be implemented as a unified service filtering engine, and the penalty processing interfaces for executing the processing can also be implemented as corresponding standardized service interfaces. In this way, the two processing logics are decoupled and allowed to collaborate with each other through interface calls, thereby meeting the purpose of this application.

[0089] Through this typical embodiment, it can be known that the present application has abundant positive advantages. Specifically, the present application processes user-posted messages in the application of the live broadcast platform through a division of labor and cooperation in multiple steps, including uniformly receiving user-posted messages of various applications pre-agreed by the live broadcast platform, calling the corresponding filtering rules of the application through a unified rule library, calling a unified classification dictionary for matching according to the corresponding filtering rules to determine the corresponding penalty type, and centrally processing the penalty types according to a unified mechanism. Among them, each processing step and filtering rules, classification dictionary, penalty processing mechanism, etc. are both independent of each other and cooperate with each other to achieve logical decoupling and coordination, allowing the live broadcast platform to implement flexible and centralized information security detection technical services, improving the information security detection capabilities of the live broadcast platform, and ensuring information security in the live broadcast platform ecosystem.

[0090] See also Figure 2In a specific embodiment, the step S1100, the step of receiving a public release message submitted by an application that has pre-agreed with the live broadcast platform, includes the following steps:

[0091] Step S1110: concurrently receive message publishing requests from applications that have pre-agreed with the live broadcast platform, and add the message publishing requests to the request queue in order:

[0092] The server of the present application can construct a standardized receiving response module for uniformly receiving message publishing requests submitted by various applications pre-protocoled by the live broadcast platform, and the message publishing request is used to encapsulate the publicly released message. In order to resolve the conflict caused by concurrent requests, a request queue is also constructed to orderly process a large number of concurrent message publishing requests from multiple sources.

[0093] Step S1120: Control the request queue to flow out the message publishing request according to the preset dequeue rule, and start the consumption thread corresponding to the message publishing request:

[0094] The request queue may apply a first-in-first-out rule, giving priority to the first queued message publishing request, so that each message publishing request in the queue flows out of the queue continuously, thereby starting a consumption thread corresponding to the message publishing request to consume the message publishing request.

[0095] Step S1130: The consumer thread parses the message publishing request to obtain the publicly published message:

[0096] The consumer thread is first responsible for parsing the corresponding message publishing request to obtain the public publishing message therein, and further can obtain the public publishing text in the public publishing message.

[0097] This embodiment implements a standardized concurrent response mechanism to serve the massive message publishing requests submitted by users of a large number of pre-protocol applications in the live broadcast platform, so that multi-source information can be processed in parallel and in an orderly manner, and assist in realizing the unified and centralized detection of information on the entire live broadcast platform. Each application does not need to independently establish its own information security detection logic, nor does it need to establish a proprietary dictionary corresponding to its own business. Various penalties do not need to establish their own dedicated dictionaries. It only needs to respond to requests in a unified manner and perform information security detection according to filtering rules to achieve centralized and unified information security detection services for the entire platform, thereby facilitating the access of applications for new businesses in the platform ecosystem.

[0098] See also Figure 3 In a specific embodiment, the step S1300, filtering the publicly released text according to the filtering rule, includes the following steps:

[0099] Step S1310: Calling a named entity recognition model that has been pre-trained to a convergence state to extract multiple named entities from the publicly released text:

[0100] In this embodiment, considering the existence of massive publicly released messages on the entire platform, in order to improve detection accuracy and efficiency, a named entity recognition model based on a neural network architecture is used to implement filtering of publicly released texts according to filtering rules.

[0101] Specifically, the named entity recognition model can be used as follows Figure 4 The network architecture shown is composed of a file feature extraction network and a conditional random field network. The text feature extraction network can be based on LSTM (Long Short-Term Memory Model), Lattice LSTM, Transformer, Bert and other basic models to extract features from the embedded vector of the publicly released text input therein to obtain its corresponding deep semantic information; the conditional random field network, i.e., CRF network, performs part-of-speech tagging on the publicly released text based on the deep semantic information to obtain multiple named entities, i.e., candidate matching keywords, according to the segmentation of the tagging results.

[0102] The named entity recognition model should be pre-trained to a convergent state so that it has the ability to extract multiple named entities from the input publicly released text. The multiple named entities extracted are used as candidate matching keywords and are matched with the classification dictionary determined according to the filtering rules to obtain matching results, and penalties are applied based on the matching results.

[0103] Step S1320: Match the named entity with each preset classification dictionary, determine the named entity that hits any entry in any classification dictionary as the hit keyword obtained by publicly released text segmentation, and obtain mapping relationship data between the hit keyword and the classification dictionary:

[0104] Please refer to Figure 4 The multiple named entities identified by the conditional random field model are matched one by one with each classification dictionary corresponding to the filtering rules corresponding to the application specified in the public release message. Here, an exact matching method is generally adopted. When a named entity is a candidate for matching keyword and hits an entry in one of the classification dictionaries, the keyword is the hit keyword, and then the mapping relationship data between the hit keyword and the classification dictionary is constructed.

[0105] Step S1330: Determine the corresponding penalty type according to the filtering rule corresponding to the classification dictionary in the mapping relationship data between the hit keyword and the classification dictionary:

[0106] Since the application program that submits the publicly released text may be configured with multiple filtering rules, each filtering rule may specify different classification dictionaries and different penalty types, and different keywords in the previous step may hit different classification dictionaries corresponding to different filtering rules, it is possible to obtain multiple mapping relationship data, which correspond to multiple different classification dictionaries, and similarly correspond to different penalty types. The collection of these penalty types can be used for subsequent selection processing.

[0107] This embodiment is based on the principle of semantic matching and applies a named entity recognition model based on a neural network architecture to segment publicly released texts to obtain named entities therein, and then matches the named entities as candidate matching keywords in conjunction with a preset classification dictionary, extracts the penalty type corresponding to the classification dictionary where there are entries matching the candidate matching keywords, and realizes the identification of whether the keywords are in violation of regulations based on deep semantic information, which can improve the intelligence level of information security detection and improve the efficiency of information security detection.

[0108] See also Figure 5 In a further embodiment, the named entity recognition model performs the following steps:

[0109] Step S2100: vectorize the publicly released text to obtain its corresponding encoding information:

[0110] This step aims to encode the publicly released text, realize vectorization, and obtain the corresponding embedding vector. There are many technologies for encoding text in the prior art, which can be flexibly implemented by those skilled in the art. After encoding, the embedding vector contains the encoding information corresponding to each character of the publicly released text, and the encoding information of each character contains the word vector of the character and the word vectors of all possible word segments of the character.

[0111] Step S2200: extract features from the coded information to obtain a text feature vector representing its deep semantic information:

[0112] The text feature extraction module is preferably implemented using Lattice LSTM, which performs representation learning on the embedded vector of the publicly released text obtained based on pre-encoding, with reference to the context, to obtain a corresponding text feature vector.

[0113] Step S2300: perform sequence annotation on the publicly released text according to the text feature vector, and extract each named entity therein:

[0114] The text feature vector is input into the conditional random field module (CRF) for part-of-speech tagging. Under the action of the conditional random field module, the probability matrix output by Lattice LSTM and the state transition matrix of CRF are combined for prediction to complete the part-of-speech tagging. According to the part-of-speech tagging results, multiple named entities in the publicly released text can be extracted.

[0115] In this embodiment, LSTM can also be replaced by a model based on the Transformer kernel such as Bert. In addition, although these models can also independently perform the task of part-of-speech tagging, combining them with conditional random fields can significantly improve the accuracy of named entity extraction, so they are recommended.

[0116] In this embodiment, with the help of a specific neural network architecture, on the basis of vectorizing the publicly released text to obtain its effective representation embedding vector, the accuracy of deep semantic information extraction and named entity recognition is improved. Therefore, the total amount of data samples required for the training process of the corresponding named entity recognition model can be reduced, making the model easier to be trained to a convergence state, thereby improving the model training efficiency and saving the model training cost.

[0117] See also Figure 6 In a specific embodiment, the step S1400, calling the penalty processing interface corresponding to the highest level penalty type among the determined penalty types to process the public release message, and returning a corresponding notification message to the release user of the application, includes the following steps:

[0118] Step S1410: compare multiple penalty types and determine the highest level penalty type:

[0119] In each embodiment of step S1300, a set of penalty types will eventually be output, and the penalty types are pre-matched with priorities, that is, penalty levels. According to the default rules, when multiple penalty types compete with each other, the penalty type with the highest level is determined as the penalty type that needs to be executed in the end to avoid multiple penalties for the same public release message.

[0120] Step S1420: Call and run the penalty processing interface corresponding to the highest level penalty type:

[0121] As mentioned above, each penalty type is preset with a corresponding penalty processing interface, which implements the processing business logic corresponding to the penalty type. Based on this, it is only necessary to call the penalty processing interface for the penalty type that is finally determined to be executed and perform the corresponding processing.

[0122] Step S1430: The penalty processing interface processes the publicly released text according to the preset business logic, including shielding the keywords that hit the entry or deleting the entire publicly released text, and returns the processing result:

[0123] The penalty processing interface processes the publicly released text according to its preset business logic, such as shielding the keywords of the hit entry or deleting the full text of the publicly released text, and then returns the corresponding processing result. In the example here, unlike other embodiments, the information release authority of the user who publishes the publicly released text is not controlled, which can simplify the business logic of penalty processing and improve the efficiency of information security detection.

[0124] Step S1440: Generate a corresponding notification message according to the processing result, and send the notification message to the instant messaging interface of the publishing user:

[0125] According to the notification message returned by the penalty processing interface, the notification message can be sent to the instant messaging interface of the publishing user who submitted the publicly released text. For example, in a live broadcast room, it can be the message notification interface corresponding to the user, so that the corresponding notification message can be displayed and read in the message notification section of the publishing user's application map.

[0126] This embodiment simplifies the penalty processing business logic, and selectively handles the competition between multiple keyword naming and multiple classification dictionaries, thereby simplifying the implementation logic of the entire system, facilitating development and implementation, and improving the development efficiency of software engineering.

[0127] See also Figure 7 In the extended embodiment, the need for professional division of labor is met and the decoupling relationship of each business logic implemented by the present application is strengthened. Therefore, the live broadcast platform information security detection method of the present application further includes the following steps:

[0128] Step S3100: In response to the remote call to the first service interface, a rule base editing page is pushed to the corresponding terminal device, and the page includes classification dictionary list information and penalty type list information:

[0129] In order to facilitate the first management user who is responsible for maintaining the rule base to centrally maintain the filtering rules therein, a first service interface is constructed. After the first management user remotely calls the first service interface through page access, he can obtain the rule base editing page pushed by the server to his terminal device. In this editing page, each data item of each stored filtering rule in the rule base can be displayed in the form of a list, that is, the application feature information, classification dictionary specified information, classification dictionary penalty type, etc. corresponding to each filtering rule are displayed; and at the same time, the first management user is allowed to perform operations such as adding, modifying, and deleting data tables therein. For a corresponding classification dictionary specified information item, a drop-down list can be provided, and the drop-down list lists the classification dictionary list information of each preset classification dictionary, thereby facilitating the management user to quickly select a classification dictionary. Similarly, the data item corresponding to the penalty type can also provide a corresponding drop-down list for displaying various penalty types, thereby providing penalty type list information.

[0130] Step S3200: Obtain one or more filtering rules submitted based on the rule base editing page, and store them in the rule base:

[0131] After the first management user performs the aforementioned editing processing on one or more filtering rules in the rule library editing page, including adding, deleting or modifying, a submission instruction can be triggered to submit it to the server of this application, and the server of this application will store it in the rule library.

[0132] Step S3300: In response to the remote call to the second service interface, a classification dictionary editing page is pushed to the corresponding terminal device, and the page includes classification dictionary list information:

[0133] In order to facilitate the second management user who is responsible for maintaining illegal entries to centrally maintain the entries in various classification dictionaries, a second service interface is constructed. After the second management user remotely calls the second service interface through page access, he can obtain the classification dictionary editing page pushed by the server to his terminal device. Since there are multiple classification dictionaries, a drop-down list can be displayed in the editing page to provide classification dictionary list information. After the user selects one of the classification dictionaries, the various entries in the selected classification dictionary are listed accordingly, allowing the second management user to perform editing operations such as adding, modifying, and deleting each entry to implement entry maintenance.

[0134] Step S3400: Obtain one or more entries submitted based on the classification dictionary editing page and the classification dictionary specified from the classification dictionary list information, and store the entries in the classification dictionary:

[0135] After the second management user performs the aforementioned editing processing on one or more entries in a classification dictionary in the classification dictionary editing page, including adding, deleting or modifying, a submission instruction can be triggered to submit it to the server of this application, and the server of this application will store it in the corresponding classification dictionary to complete the update of the classification dictionary.

[0136] It should be noted that steps S3100, S3200 and steps S3300, S3400 can be executed concurrently and are not constrained by the exemplary order of this embodiment, so that the two service interfaces can serve management users with different divisions of labor in parallel.

[0137] This embodiment provides different entrances for different data through two service interfaces with different divisions of labor, so as to professionally maintain the filtering rules and classification dictionaries respectively. For a large platform, this kind of division of labor and business logic is necessary. Through this technical means, the content quality of the classification dictionary and filtering rules can be improved with the help of professional division of labor, thereby ensuring the accuracy of information security identification and improving the efficiency of information security detection.

[0138] See also Figure 8 , a live broadcast platform information security detection device provided to meet one of the purposes of the present application, comprising: a message acquisition module 1100, a rule matching module 1200, a security detection module 1300, and a penalty processing module 1400, wherein the message acquisition module 1100 is used to receive a public release message submitted by an application that has pre-agreed with the live broadcast platform, and the public release message includes the characteristic information of the application, the publishing user, and the public release text; the rule matching module 1200 is used to call a filtering rule corresponding to the characteristic information of the application from a preset rule library, and the filtering rule includes the characteristic information of the application, the classification dictionary specified information adopted by the application, and the classification dictionary penalty type; the security detection module 1300 is used to filter the public release text according to the filtering rule, and when the keywords in the public release text hit one or more classification dictionary entries of the specified information, determine the corresponding one or more penalty types; the penalty processing module 1400 is used to call the penalty processing interface corresponding to the highest level penalty type among the determined penalty types to process the public release message, and return a corresponding notification message to the publishing user of the application.

[0139] In a specific embodiment, the message acquisition module 1100 includes: a concurrent receiving submodule, which is used to concurrently receive message publishing requests from applications that have pre-agreed with the live broadcast platform, and add the message publishing requests to the request queue in order; a request dequeueing submodule, which is used to control the request queue to flow out the message publishing request according to a preset dequeueing rule, and start the corresponding consumption thread of the message publishing request; a message consumption submodule, which is used to parse the message publishing request by the consumption thread to obtain the publicly released message therein.

[0140] In a specific embodiment, the security detection module 1300 includes: an entity recognition submodule, which is used to call a named entity recognition model that has been pre-trained to a convergent state to extract multiple named entities from the publicly released text; a dictionary matching submodule, which is used to match the named entities with various preset classification dictionaries, and determine the named entities that hit any entry in any of the classification dictionaries as hit keywords obtained by segmenting the publicly released text, and obtain mapping relationship data between the hit keywords and the classification dictionaries; a penalty determination submodule, which is used to determine the corresponding penalty type according to the filtering rules corresponding to the classification dictionary in the mapping relationship data between the hit keywords and the classification dictionary.

[0141] In a further embodiment, a control device is constructed when the named entity recognition model is running, and the control device includes: an encoding processing unit, which is used to vectorize the publicly released text to obtain its corresponding encoding information; a feature extraction unit, which is used to extract features from the encoding information to obtain a text feature vector that represents its deep semantic information; an entity extraction unit, which is used to sequence the publicly released text according to the text feature vector to extract each named entity therein.

[0142] In a specific embodiment, the penalty processing module 1400 includes: a penalty selection submodule, which is used to compare multiple penalty types and determine the highest level penalty type; an interface calling submodule, which is used to call and run the penalty processing interface corresponding to the highest level penalty type; an interface response submodule, which is used to process the publicly released text according to the preset business logic by the penalty processing interface, including shielding the keywords of the hit entries or deleting the full text of the publicly released text, and returning the processing result; a notification delivery submodule, which is used to generate a corresponding notification message according to the processing result, and send the notification message to the instant messaging interface of the publishing user.

[0143] In an extended embodiment, the live broadcast platform information security detection device of the present application also includes: a first service module, which is used to respond to a remote call to a first service interface, and push a rule library editing page to a corresponding terminal device, wherein the page includes classification dictionary list information and penalty type list information; a first update module, which is used to obtain one or more filtering rules submitted based on the rule library editing page, and store them in the rule library; a second service module, which is used to respond to a remote call to a second service interface, and push a classification dictionary editing page to a corresponding terminal device, wherein the page includes classification dictionary list information; a second update module, which is used to obtain one or more entries submitted based on the classification dictionary editing page and a classification dictionary specified from the classification dictionary list information, and store the entries in the classification dictionary.

[0144] In order to solve the above technical problems, the present application also provides a computer device. Fig. 9 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a live broadcast platform information security detection method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor may execute the live broadcast platform information security detection method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0145] In this embodiment, the processor is used to execute Figure 8 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the live platform information security detection device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0146] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the live broadcast platform information security detection method of any embodiment of the present application.

[0147] The present application also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any embodiment of the present application when executed by one or more processors.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0149] In summary, the present application can be used to construct an information security detection mechanism for multiple application products that serve the live broadcast platform, perform information security detection on user messages submitted by each application, and allow flexible configuration and management of various resource data and processing rules involved in the information security detection process. In particular, the application decouples the classification dictionary and filtering rules required for information security detection, allows professional division of labor, improves the flexibility of system maintenance, and is conducive to maintaining the information ecology of the live broadcast platform.

[0150] Those skilled in the art will appreciate that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, altered, combined, or deleted. Further, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted.

[0151] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting information security of a live broadcast platform, characterized in that: The steps include: Receiving a public release message submitted by an application that has pre-agreed with the live broadcast platform, wherein the public release message includes feature information of the application, a publishing user, and a public release text; Calling a filtering rule corresponding to the characteristic information of the application from a preset rule library, wherein the filtering rule includes the characteristic information of the application, the classification dictionary designation information adopted by the application, and the classification dictionary penalty type; Filtering the publicly released text according to the filtering rules, and determining one or more corresponding penalty types when a keyword in the publicly released text hits one or more entries of the classification dictionary of the specified information, including: calling a named entity recognition model that has been pre-trained to a convergence state to extract multiple named entities from the publicly released text; matching the named entities with each preset classification dictionary, determining a named entity that hits any entry of any one of the classification dictionaries as a hit keyword obtained by word segmentation of the publicly released text, and obtaining mapping relationship data between the hit keyword and the classification dictionary; determining the corresponding penalty type according to the filtering rule corresponding to the classification dictionary in the mapping relationship data between the hit keyword and the classification dictionary; The penalty processing interface corresponding to the highest level penalty type among the determined penalty types is called to process the public release message, and a corresponding notification message is returned to the release user of the application.

2. The live broadcast platform information security detection method according to claim 1, characterized in that: The step of receiving a public release message submitted by an application that has pre-agreed with the live broadcast platform includes the following steps: Concurrently receiving message publishing requests from applications that have pre-agreed with the live broadcast platform, and adding the message publishing requests to the request queue in order; Control the request queue to flow out the message publishing request according to a preset dequeue rule, and start a consumption thread corresponding to the message publishing request; The consumer thread parses the message publishing request to obtain the publicly published message therein.

3. The live broadcast platform information security detection method according to claim 1, characterized in that: The named entity recognition model performs the following steps: Vectorizing the publicly released text to obtain corresponding encoding information; Extracting features from the encoded information to obtain a text feature vector representing its deep semantic information; The publicly released text is sequence labeled according to the text feature vector to extract each named entity therein.

4. The live broadcast platform information security detection method according to claim 1, characterized in that: Calling the penalty processing interface corresponding to the highest level penalty type among the determined penalty types to process the public release message, and returning a corresponding notification message to the release user of the application, comprises the following steps: Compare multiple penalty types and determine the highest level of penalty type; Call and run the penalty processing interface corresponding to the highest level penalty type; The penalty processing interface processes the publicly released text according to the preset business logic, including shielding the keywords of the hit entry or deleting the full text of the publicly released text, and returns the processing result; A corresponding notification message is generated according to the processing result, and the notification message is sent to the instant messaging interface of the publishing user.

5. The live broadcast platform information security detection method according to any one of claims 1 to 4, characterized in that: The following steps are also included: In response to the remote call to the first service interface, a rule base editing page is pushed to the corresponding terminal device, where the page includes classification dictionary list information and penalty type list information; Obtaining one or more filtering rules submitted based on the rule base editing page, and storing them in the rule base; In response to the remote call to the second service interface, a classification dictionary editing page is pushed to a corresponding terminal device, wherein the page includes classification dictionary list information; One or more entries submitted based on the classification dictionary edit page and the classification dictionary specified from the classification dictionary list information are obtained, and the entries are stored in the classification dictionary.

6. A live broadcast platform information security detection device, characterized in that: include: A message acquisition module, used to receive a public release message submitted by an application that has pre-agreed with the live broadcast platform, wherein the public release message includes feature information of the application, a publishing user, and a public release text; A rule matching module, used to call a filtering rule corresponding to the characteristic information of the application from a preset rule library, wherein the filtering rule includes the characteristic information of the application, the classification dictionary specified information adopted by the application, and the classification dictionary penalty type; A security detection module is used to filter the publicly released text according to the filtering rules, and when a keyword in the publicly released text hits one or more entries of the classification dictionary of the specified information, determine one or more corresponding penalty types, including: calling a named entity recognition model that has been pre-trained to a convergence state to extract multiple named entities from the publicly released text; matching the named entities with each preset classification dictionary, determining a named entity that hits any entry of any one of the classification dictionaries as a hit keyword obtained by word segmentation of the publicly released text, and obtaining mapping relationship data between the hit keyword and the classification dictionary; and determining the corresponding penalty type according to the filtering rule corresponding to the classification dictionary in the mapping relationship data between the hit keyword and the classification dictionary; The penalty processing module is used to call the penalty processing interface corresponding to the highest level penalty type among the determined penalty types to process the public release message and return a corresponding notification message to the release user of the application.

7. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 5 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

Images