Content detection method and device, equipment cluster, program product and storage medium

Abstract

The invention relates to a content detection method and device, an equipment cluster, a program product and a medium. According to the scheme of the invention, the method comprises the steps: obtaining the to-be-detected content, so as to obtain M keywords from the to-be-detected content, and M is a positive integer; determining whether a first keyword and a second keyword associated with the first keyword exist in the M keywords at the same time or not; under the condition that it is confirmed that the first keyword independently appears in the to-be-detected content, it is confirmed that the to-be-detected content meets preset regulations; under the condition that it is confirmed that the first keyword and the second keyword associated with the first keyword appear in the to-be-detected content at the same time, it is confirmed that the to-be-detected content does not conform to preset regulations, and the first keyword and the second keyword associated with the first keyword which appear at the same time are used for indicating scenes which do not conform to the preset regulations. According to the technical scheme disclosed by the invention, the processing efficiency and accuracy of content detection are improved, and the auditing quality and the security management capability are effectively improved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of cloud computing technology, and more particularly to a content detection method, a content detection device, a cluster of computing devices, a computer program product, and a computer-readable storage medium. Background Art

[0002] In recent years, the Internet has become the main channel for information dissemination, and the diversity and immediacy of its content have greatly enriched people's lives. However, the explosive growth and diversification trend of Internet information have brought a series of challenges. In particular, the security monitoring of Internet content has become one of the important topics. In order to maintain the health and order of the network environment, content operators or content platform parties need content review measures to review whether there are sensitive issues in terms of security in Internet content.

[0003] As a commonly used content review method, currently mainly adopt the sensitive word filtering method, the manual review method, or a combination of the two. In the sensitive word filtering method, the corresponding Internet content that hits the sensitive word is identified through a preset sensitive word library, and the Internet content is blocked to avoid the leakage of sensitive information. In the manual review method, the Internet content is observed manually to determine whether the Internet content directly or indirectly involves illegal content or sensitive information. In the method combining sensitive word filtering and manual review, the Internet content can be filtered through the sensitive word library in advance, and then manual judgment and decision-making are carried out on the remaining Internet content or mis-screened Internet content. Summary of the Invention

[0004] According to some embodiments of the present disclosure, there are provided a content detection method, a content detection device, a cluster of computing devices, a computer program product, and a computer-readable storage medium.

[0005] In a first aspect of the present disclosure, a content detection method is provided. The content detection method includes: obtaining the content to be detected; obtaining M keywords from the content to be detected, where M is a positive integer; determining whether a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords; in the case where it is confirmed that the first keyword appears alone in the content to be detected, confirming that the content to be detected complies with a preset rule; in the case where it is confirmed that the first keyword and the second keyword associated with the first keyword appear simultaneously in the content to be detected, confirming that the content to be detected does not comply with the preset rule, where the simultaneously appearing first keyword and the second keyword associated with the first keyword are used to indicate a scenario that does not comply with the preset rule. According to an embodiment of the present disclosure, based on the non-fully independent correlation between keywords, the content to be detected is detected from a global perspective, reducing the risk of incorrect deletion or addition of keywords caused by manual intervention while improving the processing efficiency and accuracy for huge amounts of information and complex variability. Thereby, the requirements for timeliness and soundness can be greatly met, labor costs and operation pressure are saved, and thus the review quality and safety management ability are effectively improved.

[0006] In some embodiments, the determining whether a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords includes: based on a trained neural network model, determining whether a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords, where the neural network model is used to determine whether any combination of two or more of the M keywords appearing simultaneously complies with a preset rule or does not comply with a preset rule. According to an embodiment of the present disclosure, the overall content can be detected through the neural network model according to the correlation between keywords.

[0007] In some embodiments, before determining whether the first keyword and the second keyword associated with the first keyword exist simultaneously among the M keywords based on the trained neural network model, it includes: obtaining training data; obtaining H keywords from the training data, where H is a positive integer greater than M; training the neural network model to determine whether any combination of two or more of the H keywords appears simultaneously in accordance with a preset rule or not, where the H keywords include the first keyword and the second keyword. In some embodiments, training the neural network model to determine whether any combination of two or more of the H keywords appears simultaneously in accordance with a preset rule or not includes: determining whether the first keyword and the second keyword associated with the first keyword exist simultaneously among the H keywords; when it is confirmed that the first keyword appears alone in the training data, confirming that the label of the training data is a label that conforms to the preset rule; when it is confirmed that the first keyword and the second keyword associated with the first keyword appear simultaneously in the training data, confirming that the label of the training data is a label that does not conform to the preset rule; based on the H keywords included in the training data and the label of the training data, obtaining the trained neural network model. According to the embodiments of the present disclosure, the neural network model can be trained based on the relevance between keywords, adaptively learn and update keywords, thereby forming a daily-level dynamic update based on the confidence evaluation model, so as to further improve the processing ability and accuracy for a huge amount of information.

[0008] In some embodiments, obtaining the trained neural network model based on the H keywords included in the training data and the label of the training data includes: generating a keyword index of the H keywords; generating a word vector of the training data based on the keyword index of the H keywords. In some embodiments, obtaining the trained neural network model based on the H keywords included in the training data and the label of the training data further includes: generating an evaluation score of the label of the training data; generating a score vector of the label of the training data based on the evaluation score of the label of the training data. In some embodiments, obtaining the trained neural network model based on the H keywords included in the training data and the label of the training data further includes: training the neural network model based on the word vector of the training data and the score vector of the label of the training data; in response to the performance index of the neural network model meeting a predetermined condition, obtaining the trained neural network model. According to the embodiments of the present disclosure, through continuous iteration and update of the neural network model, the evaluation result of the content to be detected can be determined more accurately, and the risk of incorrect addition or deletion of keywords caused by manual intervention can be further reduced.

[0009] In some embodiments, the method further includes: based on the trained neural network model, updating the keyword attributes of the H keywords, where the keyword attributes include at least one of the following: keyword category, keyword tag, keyword update time, keyword historical performance, and keyword source. According to the embodiments of the present disclosure, various applications and scenarios are widely implemented in various dimensions, improving the security monitoring ability. For example, it can be applied to the design of content security service products, the improvement of content security detection technology capabilities, the operation of content security data quality, etc., and can effectively provide the adaptive operation ability of the content review keyword model. For example, it can be applied to threat intelligence services. By collecting IP and domain name intelligence data from intelligence vendors or user feedback, it can effectively analyze and improve the quality operation ability of threat intelligence. For another example, it can be applied to the services of black and white lists. By collecting a large amount of feedback data on black and white lists, dynamic operation is carried out to improve the quality of black and white lists.

[0010] In a second aspect of the present disclosure, a content detection device is provided. The content detection device includes: an acquisition module for acquiring the content to be detected to obtain M keywords from the content to be detected, where M is a positive integer; a determination module for determining whether a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords; a confirmation module for, in the case of confirming that the first keyword appears alone in the content to be detected, confirming that the content to be detected meets the preset regulations; and in the case of confirming that the first keyword and the second keyword associated with the first keyword appear simultaneously in the content to be detected, confirming that the content to be detected does not meet the preset regulations, where the simultaneously appearing first keyword and the second keyword associated with the first keyword are used to indicate a scenario that does not meet the preset regulations.

[0011] In a third aspect of the present disclosure, a computing device cluster is provided. The computing device cluster includes at least one computing device, and each computing device includes a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method as described in the first aspect of the present disclosure. In some embodiments, the computing device cluster includes one computing device. In some other embodiments, the computing device cluster includes multiple computing devices. In some embodiments, the computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0012] In a fourth aspect of the present disclosure, there is provided a computer program product comprising instructions. When the instructions are run by a cluster of computing devices, the cluster of computing devices is caused to execute the method as described in the first aspect of the present disclosure.

[0013] In a fifth aspect of the present disclosure, there is provided a computer-readable storage medium. The computer-readable storage medium includes computer program instructions which, when executed by a cluster of computing devices, cause the cluster of computing devices to execute the method as described in the first aspect of the present disclosure. In some embodiments, the computer-readable storage medium may be non-transitory. The computer-readable storage medium includes, but is not limited to, volatile memory (such as random access memory), non-volatile memory (such as flash memory, hard disk drive (HDD), solid state drive (SSD), etc.).

[0014] It should be understood that the content described in the summary of the invention section is not intended to limit the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In conjunction with the accompanying drawings and with reference to the following detailed description, the features, advantages, and other aspects of the various implementations of the present disclosure will become more apparent. Several implementations of the present disclosure are shown herein by way of example and not limitation, in the drawings:

[0016] Figure 1 There is shown an exemplary diagram of the architecture of a content detection system according to an embodiment of the present disclosure;

[0017] Figure 2 There is shown an exemplary flowchart of a content detection method according to an embodiment of the present disclosure;

[0018] Figure 3 There is shown a further exemplary flowchart of a content detection method according to an embodiment of the present disclosure;

[0019] Figure 4 There is shown yet a further exemplary flowchart of a content detection method according to an embodiment of the present disclosure;

[0020] Figure 5 There is shown a schematic diagram of keyword attributes according to an embodiment of the present disclosure;

[0021] Figure 6 There is shown a schematic diagram of the historical performance of keywords according to an embodiment of the present disclosure;

[0022] Figure 7 There is shown an exemplary schematic diagram of the neural model training process according to an embodiment of the present disclosure;

[0023] Figure 8 shows a schematic block diagram of a content detection device according to an embodiment of the present disclosure;

[0024] Figure 9 shows a schematic block diagram of an example device that can be used to implement an exemplary implementation of the present disclosure;

[0025] Figure 10 shows a schematic block diagram of a cluster of example devices that can be used to implement an exemplary implementation of the present disclosure; and

[0026] Figure 11 shows a schematic block diagram of another example device that can be used to implement an exemplary implementation of the present disclosure. Detailed implementation manners

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0028] In the description of the embodiments of the present disclosure, the term "including" and its like should be understood as an open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". Terms such as "first", "second", etc. may refer to different or the same objects. The term "and / or" means at least one of the two items associated therewith. For example, "A and / or B" means A, B, or A and B. There may also be other explicit and implicit definitions hereinafter.

[0029] It should be understood that for the technical solutions provided in the embodiments of the present application, in the introduction of the following specific embodiments, some repetitions may not be elaborated again, but it should be regarded that there are mutual references between these specific embodiments and they can be combined with each other.

[0030] As described above, in the conventional solution, as a commonly used content review method, mainly the sensitive word filtering method, the manual review method, or a combination of the two is adopted. However, these commonly used content review methods have problems such as poor recognition and low recognition efficiency. For example, in the sensitive word filtering method, in addition to the preset sensitive words in the sensitive word library, it is often necessary for the content operator or the content platform operator to configure customized sensitive words related to content operation in the sensitive word library. However, with the increasing redundancy and complexity of Internet information and the rapid change of hot sensitive information, the update of sensitive words often fails to meet the requirements of timeliness and integrity. On the other hand, both the sensitive word filtering method and the manual review method involve labor costs. For the sensitive words that may be encountered, it is necessary to verify them one by one manually. The volume of the sensitive word library is as high as more than ten thousand levels. Relying solely on manual review leads to high review costs and maintenance costs, resulting in a decline in review quality and security management. Sometimes, the inevitable intervention of humans may lead to, for example, the accidental deletion or addition of sensitive words, or the problem that the sensitive words are too generalized or too specific due to the inconsistency of manual standards. Further, the inventors of the present disclosure also noticed that usually, sensitive words are not completely independent, but there are cases where they express other meanings when they appear together. For example, if each sensitive word appears alone, the content of each sensitive word appears normal, but if multiple specific sensitive words appear in a piece of content together, it may cause the whole piece of content to appear sensitive and violate the regulations. This requires the operators of the content platform or the content operator to have a high overall ability to extract sensitive words and the ability to distinguish content sensitivity for a piece of content. However, due to the huge amount of information, complex diversity and variability of Internet content, the current conventional solution cannot effectively consider the relevance between a large number of sensitive words.

[0031] To this end, the present disclosure provides a content detection solution. In the content detection solution of the present disclosure, the content to be detected is obtained to obtain M keywords as positive integers, and it is determined whether a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords. In the case where it is confirmed that the first keyword appears alone in the content to be detected, it is confirmed that the content to be detected complies with the preset regulations. On the other hand, in the case where it is confirmed that the first keyword and the second keyword associated with the first keyword appear simultaneously in the content to be detected, it is confirmed that the content to be detected does not comply with the preset regulations, where the simultaneously appearing first keyword and the second keyword associated with the first keyword are used to indicate a scenario that does not comply with the preset regulations. Thus, according to the not completely independent relevance of the keywords in the content to be detected, the content to be detected is detected from a global perspective, reducing the risk of misdeletion or misaddition of keywords caused by manual intervention while improving the processing efficiency and accuracy for huge amounts of information and complex variability. According to the technical solution of the present disclosure, compared with the past, it can greatly meet the requirements of timeliness and soundness, saving labor costs and operation pressure, thereby effectively improving the review quality and safety management ability.

[0032] Figure 1 FIG. shows an exemplary architecture diagram of a content detection system according to an embodiment of the present disclosure, which specifically shows the system architecture of a content detection system 10 for evaluating the content to be detected. As Figure 1As shown, in an embodiment of the present disclosure, the content detection system 10 includes a keyword generation module 110, a keyword operation module 120, and a content detection module 130, and is deployed on a computing node cluster 101 and a communication network 102. In an embodiment of the present disclosure, the keyword generation module 110 is used to generate or determine sensitive keywords, such as sensitive words, as input data for the keyword operation module 120. In some embodiments, the keyword generation module 110 includes, for example, a preset keyword set 1101, and the input data may be a preset keyword set, and the preset keyword set includes one or more preset keywords. For example, the preset keyword set may be the initial offline keywords of the content detection system 10, for example, it may be a set of sensitive keywords inherently configured before the operation of the content platform party or the content operation party goes live. In some embodiments, the keyword generation module 110 includes, for example, an artificial intelligence (AI) generated keyword set 1102, and the input data may be one or more keywords generated based on an artificial intelligence model, for example, it is a set of sensitive keywords extracted by means of word segmentation from texts or text combinations with a higher frequency according to the full AI function. In some embodiments, the keyword generation module 110 includes, for example, a user input keyword set 1103, and the input data may be one or more keywords based on user input, for example, it may be manually entered keywords, and these new keywords are a set of keywords added by the content operation party during operation based on its own experience or needs. In an embodiment of the present disclosure, the keyword operation module 120 is used to store, evaluate, maintain, and update the keywords of the content detection system 10, for example, it includes a keyword library 1201, a model matching module 1202, a keyword evaluation module 1203, a model update module 1204, and a keyword library update module 1205. In some embodiments, the keyword library 1201 may store a keyword database of the content detection system 10 for the keyword operation of the content operation party, and for each keyword, a corresponding attribute may be associated, for example, it includes at least one of a keyword category, a keyword tag, a keyword update time, a keyword historical performance, and a keyword source. In some embodiments, the model matching module 1202 may be used to obtain keywords that meet the detection strategy from the keyword library 1201, and identify keywords whose sensitivity meets a predetermined condition to input to the keyword evaluation module 1203. In some specific embodiments, the model matching module 1202 may obtain keywords that meet the detection strategy from the keyword library 1201 based on keyword confidence to add to the automaton detection model, and then perform keyword sensitivity identification through the automaton detection model, and output an alarm result for keywords whose sensitivity meets a predetermined condition to prompt for special review.In some embodiments, the keyword evaluation module 1203 is configured to review keywords that meet a predetermined condition, determine keyword performance, and store the results by classification and statistics for use in the training of the model update module 1204. In some embodiments, the model update module 1204 can perform model training based on the stored keyword performance. For example, it can learn and train the on-network performance of keywords through a keyword confidence evaluation model based on a neural network. In some embodiments, the keyword library update module 1205 updates the confidence levels of the existing and incremental keywords in the keyword library 1201 through the trained neural network model in the model update module 1204. In the embodiments of the present disclosure, the content detection module 130 may include a content acquisition module 1301, a content detection module 1302, and a content feedback module 1303. In some embodiments, the content acquisition module 1301 is configured to obtain the content to be detected and information related to the content to be detected for use in detecting the content to be detected. In some embodiments, the content detection module 1302 is configured to perform detection processing on the content to be detected through the keyword operation module 120 to determine the overall sensitivity of the content to be detected. In some embodiments, the content feedback module 1303 is configured to, for example, determine the content to be detected as sensitive content based on the evaluation of the content to be detected meeting a predetermined condition, and perform shielding processing on the sensitive data to prevent the sensitive data from being accessed externally.

[0033] In some embodiments, the computing node cluster 101 may include one or more computing nodes 101-1, 101-2, 101-3, 101-4, ……, 101-N, where the computing nodes can be physical nodes or virtual nodes. In some embodiments, all or part of the computing node cluster 110 may be deployed on one or more physical nodes, or on one or more virtual nodes, or on a combination of physical nodes and virtual nodes. As a specific embodiment, the computing nodes can be server devices, client devices, workstations, hubs, switches, routers, etc. In some embodiments, when the computing node is a server device, for example, it may include any type of server device such as a physical server, a cloud server, a bare metal server, etc. In some embodiments, when the computing node is a client device, for example, it may include computer devices or terminals such as personal notebooks, desktops, mobile phones, etc. In the embodiments of the present disclosure, the communication network 102 is used to connect all or part of the computing node cluster 101, and may include, for example, a local area network (LAN), a wide area network (WAN), the Internet, a virtual LAN (VLAN), an enterprise LAN, a layer 3 virtual private network (VPN), an intranet, or any combination. It should be understood that each unit in the content detection system 10 can communicate directly with each other via the communication network 102, or indirectly communicate with each other through any unit connected by the communication network 102. Of course, they may also not communicate with each other. It should be understood that in the embodiments of the present disclosure, any one or more computing nodes in the computing node cluster 101 can be one or more of a content platform party, a content operation party, and a content publishing party. As a specific embodiment, the content platform party provides storage resources, computing resources, management functions, and service operations for content operation to the content operation party. The content operation party provides interfaces and platforms for content publishing and access to the content publishing party and manages the published content. It should be understood that the actions and functions of the content platform party, the content operation party, and the content publishing party are not fixed and can be changed according to the actual needs of those skilled in the art in some embodiments.

[0034] Figure 2 FIG. shows an example flowchart of a content detection method according to an embodiment of the present disclosure. As Figure 2As shown, at block 211, the content to be detected is obtained. At block 213, M keywords are obtained from the content to be detected, where M is a positive integer. At block 215, it is determined whether both a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords. At block 217, in the case where it is confirmed that the first keyword appears alone in the content to be detected, it is confirmed that the content to be detected complies with a preset rule. At block 219, in the case where it is confirmed that both the first keyword and the second keyword associated with the first keyword appear simultaneously in the content to be detected, it is confirmed that the content to be detected does not comply with the preset rule, where the simultaneously appearing first keyword and the second keyword associated with the first keyword are used to indicate a scenario that does not comply with the preset rule. In some embodiments, the preset rule is, for example, a rule or condition for determining whether the content to be detected is considered sensitive data. In some embodiments, for example, in the case where the keyword "game license" or "live game" appears alone in the content to be detected, it is confirmed that the content to be detected containing the keyword "game license" or containing "live game" complies with the preset rule and is not sensitive data. On the other hand, in the case where the keywords "game license" and "live game" are associated and appear simultaneously in the content to be detected, since the two keywords indicate a scenario with a possibility of not complying with the preset rule, it is confirmed that the content to be detected containing both the keywords "game license" and "live game" does not comply with the preset rule and belongs to sensitive data.

[0035] In some embodiments, determining whether both a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords at block 215 may include: based on a trained neural network model, determining whether both a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords, where the neural network model is used to determine whether any combination of two or more of the M keywords appearing simultaneously complies with the preset rule or does not comply with the preset rule.

[0036] In some embodiments, in the case where it is confirmed that the first keyword appears alone in the content to be detected at block 217, confirming that the content to be detected complies with the preset rule may further include: determining the content to be detected as non-sensitive data without imposing access restrictions.

[0037] In some embodiments, when it is confirmed at block 217 that the first keyword and the second keyword associated with the first keyword both appear in the content to be detected, confirming that the content to be detected does not meet the preset regulations may include: determining the content to be detected as sensitive content, and performing a shielding process on the content to be detected to prevent the sensitive data from being externally accessed. In some embodiments, a confidence level of a keyword associated with the content to be detected being higher than a specific threshold means that, for example, the proportion of non-"good" tags in the content to be detected is relatively high, and it belongs to sensitive data.

[0038] Figure 3 Fig. further shows an example flowchart of a content detection method according to an embodiment of the present disclosure. Among them, Figure 3 The method may be performed before determining whether the first keyword and the second keyword associated with the first keyword both exist among the M keywords based on a trained neural network model. At block 301, training data is obtained. At block 303, H keywords are obtained from the training data, where H is a positive integer greater than M. At block 305, the neural network model is trained to determine whether any combination of two or more of the H keywords appears in accordance with the preset regulations or does not appear in accordance with the preset regulations, where the H keywords include the first keyword and the second keyword.

[0039] In some embodiments, training the neural network model at block 305 to determine whether any combination of two or more of the H keywords appears in accordance with the preset regulations or does not appear in accordance with the preset regulations may include: at block 3051, determining whether the first keyword and the second keyword associated with the first keyword both exist among the H keywords. At block 3053, when it is confirmed that the first keyword appears alone in the training data, it is confirmed that the label of the training data is a label that meets the preset regulations. At block 3055, when it is confirmed that the first keyword and the second keyword associated with the first keyword both appear in the training data, it is confirmed that the label of the training data is a label that does not meet the preset regulations. At block 3057, based on the H keywords included in the training data and the label of the training data, the trained neural network model is obtained.

[0040] Figure 4 Fig. further shows another example flowchart of a content detection method according to an embodiment of the present disclosure. Among them, Figure 4 The method of Figure 3The operations of the box 3057 in [reference] correspond. At block 401, a keyword index for the H keywords is generated. At block 403, based on the keyword index of the H keywords, word vectors of the training data are generated. At block 402, an evaluation score for the label of the training data is generated. At block 404, based on the evaluation score of the label of the training data, a score vector for the label of the training data is generated. At block 405, based on the word vectors of the training data and the score vector of the label of the training data, the neural network model is trained. At block 407, in response to the performance metric of the neural network model meeting a predetermined condition, the trained neural network model is obtained. In some embodiments, based on the training data, the neural network model is iteratively trained according to the associated word vectors and score vectors corresponding to each training data.

[0041] In some embodiments, the label for evaluating the training data is determined by one or both of the following: determining the label of the training data based on a user's evaluation of the training data; and determining the label of the training data from a preset set of labels based on a second neural network model.

[0042] In some embodiments, the training data or the content to be detected is derived from at least one of text information, image information, video information, voice information, communication information, user information, domain name information, Internet protocol address information, and application program information. In some embodiments, the keyword includes one or more keywords parsed from the training data or the content to be detected. In the embodiments of the present disclosure, the neural network model is a model for keyword confidence evaluation. For example, a feedforward neural network (FNN) model such as a fully connected network (FCN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), a generative adversarial network (GAN), a Hopfield network, etc., a feedback neural network model, a deep learning model, etc. can be applied. As a specific embodiment, the neural network model of the present disclosure can adopt the BiLSTM (bidirectional long short-term memory network) model in the recurrent neural network (RNN). For example, it processes the forward (from left to right) and reverse (from right to left) sequence data through two independent LSTM (long short-term memory) layers respectively, thereby effectively improving the accuracy of the keyword confidence evaluation model.

[0043] In some embodiments, the content detection method of the present disclosure may further include: updating the keyword attributes of the H keywords based on the trained neural network model, where the keyword attributes include at least one of the following: keyword category, keyword label, keyword update time, keyword historical performance, and keyword source. In some embodiments, the confidence level corresponding to each keyword is determined through a preset neural network model; and based on the confidence level of the keyword, the input data including the keyword with the confidence level greater than a predetermined threshold is determined as the training data of the neural network model. In some embodiments, the input data includes at least one of the following: a preset keyword set, the preset keyword set including one or more preset keywords; one or more keywords generated based on an artificial intelligence model; and one or more keywords input by a user. In some embodiments, the keyword set determined based on the input data is input into the keyword library, and the associated attribute fields of the keyword library can be updated regularly for each keyword. Figure 5Shows a schematic diagram of keyword attributes according to an embodiment of the present disclosure. As Figure 5 shown, the attributes include at least one of a keyword category, a keyword tag, a keyword update time, a keyword historical performance, and a keyword source, where the keyword category represents a separate indication or a combined indication of keywords, the keyword tag represents the tag with the highest frequency associated with the target keyword, the keyword update time represents the time when the confidence or attribute of the keyword is updated, the keyword historical performance represents each tag corresponding to the target keyword and the frequencies respectively corresponding to each tag, and the keyword source represents the input source of the target keyword.

[0044] In some embodiments, a filtering strategy for keywords can be specified based on a confidence field in an attribute field associated with the keyword, so as to generate an automaton detection model based on the filtered keywords, thereby detecting the text content and recording the keyword hit situation and the detection result to determine one or more keywords for composing each training data.

[0045] In some embodiments, obtaining the trained neural network model based on the H keywords included in the training data and the label of the training data at block 3057 may further include: determining the keyword and the label and storing them as the keyword historical performance in the attribute field of the keyword. Figure 6 Shows a schematic diagram of keyword historical performance according to an embodiment of the present disclosure. As Figure 6 shown, for the keyword "game license", it is determined that there are tags "play" and "good". The "play" tag is derived from keyword training data such as "amusement game license on-site game" with a cumulative count of 3 times, "game license on-site game" with a count of 4 times, and keyword data "amusement game license double on-site game" with a count of 1 time, etc. The "good" tag is derived from keyword training data of "game license" with a count of 2 times. In further some embodiments, for the one or more keywords, the attributes associated with each keyword are updated through the trained neural network model. In some embodiments, the confidence field associated with each keyword is updated through the trained neural network model and iteration is performed.

[0046] Figure 7 Shows an exemplary schematic diagram of the neural model training process according to an embodiment of the present disclosure, where Figure 7 the shown neural model training process is applicable to the various operations described in the above embodiments. As Figure 7As shown, as shown in process 710, a training data set, which is the training data of the neural network model, is obtained. Based on the keyword historical performance field associated with the keyword-related attributes, it can be determined that the training data set includes one or more keywords X and their associated labels Y. Then, as shown in process 720, for one or more keywords X, through the Word2vec transformation operation, a keyword index corresponding to each keyword X is generated to identify the keyword, where 720’ shows a specific example of process 720. Next, at process 725, word vectors for representing each training data are formed according to the keyword index, where 725’ shows a specific example of process 725. Thus, the training data can be represented as a keyword combination for training the neural network model. On the other hand, at process 730, an evaluation score corresponding to each label is determined for label Y. For example, the evaluation score of “good” is determined to be 0, and the evaluation scores of others such as “play”, “game”, “bad”, etc. are determined to be 1. Then, at process 735, for example, through the one-hot encoding operation on the evaluation scores, the evaluation score 0 of “good” is formed into the supervision label [0, 1], and the evaluation scores 1 of others such as “play”, “game”, “bad”, etc. are formed into the supervision label [1, 0]. Thus, for the training data set, the word vectors formed at process 725 are associated with the supervision labels formed at process 735 and are input into the confidence evaluation neural network model P at process 740. At process 750, the confidence evaluation model P is iteratively trained based on the training data set until the performance metrics of the neural network model meet the predetermined conditions, such as the model effect no longer improves, and the optimal confidence evaluation model P is saved for updating the keyword library and evaluating the content to be detected. In some embodiments, the confidence evaluation neural network model P includes, for example, a word embedding unit (Word Embedding), a convolution / max pooling (Convolution / MaxPooling) layer, a BiLSTM (bidirectional long short-term memory network) layer, a Dropout layer, and a fully connected (FullyConnected) layer for iterative training. It should be understood that the neural network model of the present disclosure is not limited thereto, and those skilled in the art can make matches and adaptations based on specific situations.

[0047] According to an embodiment of the present disclosure, whether the content to be detected meets the preset regulations is confirmed based on whether the first keyword and the second keyword associated with the first keyword exist simultaneously among the M keywords. Thus, based on the not completely independent relevance of the keywords in each training data, the content to be detected is detected from a global perspective, reducing the risk of misdeleting or adding keywords caused by manual intervention while improving the processing efficiency and accuracy for huge amounts of information and complex variability. According to an embodiment of the present disclosure, compared with the past, it can greatly meet the requirements of timeliness and soundness, saving labor costs and operation pressure, thereby effectively improving the review quality and safety management ability.

[0048] In the embodiment of the present disclosure, the example of improving the adaptive operation ability in the review scenario of Internet content is used for illustration, but it should be understood that the application scenarios of the content detection solution of the present disclosure are not limited thereto. For example, the content detection solution of the present disclosure can be used for threat intelligence services. For example, by collecting IP and domain name intelligence data fed back by intelligence vendors or users as content, and using the information in the IP and domain names as keywords for learning, training, and evaluation to determine the evaluation of the target IP or domain name information, thereby being able to analyze and improve the threat intelligence quality operation ability. For another example, the content detection solution of the present disclosure can be used for black and white list control services. For example, by collecting feedback data on black and white lists including email addresses, usernames, mobile phone numbers, etc. as content, and using the information in the email addresses, usernames, mobile phone numbers, etc. as keywords for learning, training, and evaluation to determine the detection of the target objects in the black and white lists, thereby being able to dynamically operate to improve the quality of the black and white lists.

[0049] The present application also provides a content detection device. Figure 8 The schematic block diagram of the content detection device according to an embodiment of the present disclosure is shown. As Figure 8 shown, the content detection device 800 includes: an acquisition module 810, configured to acquire the content to be detected to obtain M keywords from the content to be detected, where M is a positive integer; a determination module 820, configured to determine whether the first keyword and the second keyword associated with the first keyword exist simultaneously among the M keywords; a confirmation module 830, configured to confirm that the content to be detected meets the preset regulations when it is confirmed that the first keyword appears alone in the content to be detected; and to confirm that the content to be detected does not meet the preset regulations when it is confirmed that the first keyword and the second keyword associated with the first keyword appear simultaneously in the content to be detected, where the simultaneously appearing first keyword and the second keyword associated with the first keyword are used to indicate a scenario that does not meet the preset regulations.

[0050] In some embodiments, the determining module 830 is further configured to: based on the trained neural network model, determine whether a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords, where the neural network model is used to determine whether any combination of two or more of the M keywords appears simultaneously in accordance with a preset rule or does not conform to the preset rule.

[0051] In some embodiments, the content detection device further includes a model training module, and the model training module is configured to: before determining whether a first keyword and a second keyword associated with the first keyword exist simultaneously among the M keywords based on the trained neural network model, obtain training data; obtain H keywords from the training data, where H is a positive integer greater than M; and train the neural network model to determine whether any combination of two or more of the H keywords appears simultaneously in accordance with a preset rule or does not conform to the preset rule, where the H keywords include the first keyword and the second keyword.

[0052] In some embodiments, the model training module is further configured to: determine whether a first keyword and a second keyword associated with the first keyword exist simultaneously among the H keywords; when it is confirmed that the first keyword appears alone in the training data, confirm that the label of the training data is a label that conforms to the preset rule; when it is confirmed that the first keyword and the second keyword associated with the first keyword appear simultaneously in the training data, confirm that the label of the training data is a label that does not conform to the preset rule; based on the H keywords included in the training data and the label of the training data, obtain the trained neural network model.

[0053] In some embodiments, the model training module is further configured to: generate a keyword index of the H keywords; based on the keyword index of the H keywords, generate a word vector of the training data.

[0054] In some embodiments, the model training module is further configured to: generate an evaluation score of the label of the training data; based on the evaluation score of the label of the training data, generate a score vector of the label of the training data.

[0055] In some embodiments, the model training module is further configured to: train the neural network model based on the word vector of the training data and the score vector of the label of the training data; in response to the performance index of the neural network model meeting a predetermined condition, obtain the trained neural network model.

[0056] In some embodiments, based on the trained neural network model, the keyword attributes of the H keywords are updated, and the keyword attributes include at least one of the following: keyword category, keyword tag, keyword update time, keyword historical performance, and keyword source.

[0057] Among them, the acquisition module, the determination module, and the confirmation module can all be implemented by software or by hardware. Exemplarily, next, taking the acquisition module as an example, the implementation manner of the acquisition module is introduced. Similarly, the implementation manners of the determination module and the confirmation module can refer to the implementation manner of the acquisition module.

[0058] As an example of a software functional unit, the acquisition module may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above computing instance may be one or more. For example, the acquisition module may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers for running this code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers for running this code may be distributed in the same availability zone (AZ) or in different AZs, and each AZ includes one data center or multiple geographically proximate data centers. Among them, generally one region may include multiple AZs.

[0059] Similarly, the multiple hosts / virtual machines / containers for running this code may be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Among them, generally one VPC is set within one region. For cross-region communication between two VPCs within the same region and between VPCs in different regions, a communication gateway needs to be set in each VPC, and the interconnection between VPCs is realized through the communication gateway.

[0060] As an example of a hardware functional unit, the acquisition module may include at least one computing device, such as a server. Alternatively, the acquisition module may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). Among them, the above PLD may be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0061] The multiple computing devices included in the acquisition module may be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module may be distributed in the same availability zone (AZ) or in different AZs. Similarly, the multiple computing devices included in the acquisition module may be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Among them, the multiple computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0062] It should be noted that in other embodiments, the acquisition module may be used to execute any step in the content detection method, the determination module may be used to execute any step in the content detection method, the confirmation module may be used to execute any step in the content detection method, and the steps implemented by the acquisition module, the determination module, and the confirmation module can be specified as needed. The entire function of the content detection device is realized by implementing different steps in the content detection method through the acquisition module, the determination module, and the confirmation module respectively.

[0063] This application also provides a computing device 900, Figure 9 which shows a schematic block diagram of an example device that can be used to implement an exemplary implementation of the present disclosure. As Figure 9 shown, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate with each other through the bus 902. The computing device 900 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 900.

[0064] The bus 902 can be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 only one line is used in Figure 9 , but it does not mean that there is only one bus or one type of bus. The bus 904 can include a path for transmitting information between various components of the computing device 900 (e.g., the memory 906, the processor 904, the communication interface 908).

[0065] The processor 904 can include any one or more of processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0066] The memory 906 can include volatile memory, such as random access memory (RAM). The processor 904 can also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0067] The memory 906 stores executable program code, and the processor 904 executes the executable program code to respectively implement the functions of the foregoing acquisition module, determination module, and confirmation module, thereby implementing the content detection method. That is, the memory 906 stores instructions for executing the content detection method.

[0068] Alternatively, the memory 906 stores executable code, and the processor 904 executes the executable code to respectively implement the functions of the foregoing content detection device, AA device, and BB device, thereby implementing the content detection method. That is, the memory 906 stores instructions for executing the content detection method.

[0069] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or a communication network.

[0070] Embodiments of the present application also provide a computing device cluster. Figure 10 FIG. shows a schematic block diagram of an exemplary device cluster that can be used to implement the exemplary implementation of the present disclosure. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0071] As Figure 10 shown, the computing device cluster includes at least one computing device 900. Instructions for executing the content detection method can be stored in the same manner in the memories 906 of one or more of the computing devices 900 in the computing device cluster.

[0072] In some possible implementation manners, partial instructions for executing the content detection method can also be stored separately in the memories 906 of one or more of the computing devices 900 in the computing device cluster. In other words, a combination of one or more computing devices 900 can jointly execute the instructions for executing the content detection method.

[0073] It should be noted that the memories 906 in different computing devices 900 in the computing device cluster can store different instructions, respectively for executing partial functions of the content detection device. That is, the instructions stored in the memories 906 of different computing devices 900 can implement the functions of one or more of the acquisition module, the determination module, and the confirmation module.

[0074] In some possible implementation manners, one or more computing devices in the computing device cluster can be connected through a network. Among them, the network can be a wide area network or a local area network, etc. Figure 11 FIG. shows a possible implementation manner, that is Figure 11 FIG. shows a schematic block diagram of another exemplary device that can be used to implement the exemplary implementation of the present disclosure. As Figure 11 shown, two computing devices 900A and 900B are connected through a network. Specifically, they are connected to the network through the communication interfaces in each computing device. In this type of possible implementation manner, instructions for executing the function of the acquisition module are stored in the memory 906 of the computing device 900A. At the same time, instructions for executing the functions of the determination module and the confirmation module are stored in the memory 906 of the computing device 900B.

[0075] Figure 11 The connection manner between the computing device clusters shown can be considered because the content detection method provided in the present application requires, for example, a large amount of data storage. Therefore, it is considered to hand over the functions implemented by the determination module and the confirmation module to the computing device 900B for execution.

[0076] It should be understood that Figure 11 the functions of the computing device 900A shown in [reference] can also be completed by multiple computing devices 900. Similarly, the functions of the computing device 900B can also be completed by multiple computing devices 900.

[0077] The embodiments of the present application also provide another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similarly referred to Figure 10 and Figure 11 the connection method of the computing device cluster. The difference is that the same instructions for executing the content detection method can be stored in the memory 906 of one or more computing devices 900 in the computing device cluster.

[0078] In some possible implementation manners, partial instructions for executing the content detection method can also be separately stored in the memory 906 of one or more computing devices 900 in the computing device cluster. In other words, a combination of one or more computing devices 900 can jointly execute the instructions for executing the content detection method.

[0079] The embodiments of the present application also provide a computer program product containing instructions. The computer program product can be software or a program product containing instructions that can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, at least one computing device is caused to execute the content detection method.

[0080] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive), etc. The computer-readable storage medium includes instructions that instruct the computing device to execute the content detection method.

[0081] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A content detection method, characterized in that: The content detection method comprises: Get the content to be detected; Obtain M keywords from the content to be detected, where M is a positive integer; Determine whether a first keyword and a second keyword associated with the first keyword exist simultaneously in the M keywords; When it is confirmed that the first keyword appears alone in the content to be detected, confirming that the content to be detected meets the preset regulations; When it is confirmed that the first keyword and the second keyword associated with the first keyword appear in the content to be detected at the same time, it is confirmed that the content to be detected does not comply with the preset regulations, wherein the first keyword and the second keyword associated with the first keyword that appear at the same time are used to indicate a scenario that does not comply with the preset regulations.

2. The content detection method according to claim 1, characterized in that: The determining whether the first keyword and the second keyword associated with the first keyword exist simultaneously in the M keywords includes: Based on the trained neural network model, determine whether a first keyword and a second keyword associated with the first keyword exist simultaneously in the M keywords, wherein the neural network model is used to determine whether the simultaneous occurrence of any two or more combinations of the M keywords complies with preset regulations or does not comply with preset regulations.

3. The content detection method according to claim 2, characterized in that: Before determining whether the M keywords simultaneously contain a first keyword and a second keyword associated with the first keyword based on the trained neural network model, the method further includes: Get training data; Obtain H keywords from the training data, where H is a positive integer greater than M; The neural network model is trained to determine whether a combination of any two or more of the H keywords simultaneously occurring satisfies a preset requirement or does not comply with a preset requirement, wherein the H keywords include the first keyword and the second keyword.

4. The content detection method according to claim 3, characterized in that: The training of the neural network model to determine whether the combination of any two or more of the H keywords simultaneously appears in accordance with a preset requirement or does not meet the preset requirement includes: Determine whether the first keyword and a second keyword associated with the first keyword exist simultaneously in the H keywords; After confirming that the first keyword appears alone in the training data, confirming that a label of the training data is a label that complies with a preset requirement; After confirming that the first keyword and a second keyword associated with the first keyword appear in the training data at the same time, confirming that the label of the training data is a label that does not comply with a preset requirement; Based on the H keywords included in the training data and the labels of the training data, the trained neural network model is obtained.

5. The content detection method according to claim 4, characterized in that: The step of obtaining the trained neural network model based on the H keywords included in the training data and the labels of the training data comprises: Generate a keyword index of the H keywords; Based on the keyword indexes of the H keywords, word vectors of the training data are generated.

6. The content detection method according to claim 4, characterized in that: The step of obtaining the trained neural network model based on the H keywords included in the training data and the labels of the training data further includes: Generating evaluation scores for the labels of the training data; Based on the evaluation scores of the labels of the training data, score vectors of the labels of the training data are generated.

7. The content detection method according to any one of claims 4 to 6, characterized in that: The step of obtaining the trained neural network model based on the H keywords included in the training data and the labels of the training data further includes: Training the neural network model based on the word vectors of the training data and the score vectors of the labels of the training data; In response to the performance indicator of the neural network model satisfying a predetermined condition, a trained neural network model is obtained.

8. The content detection method according to any one of claims 3 to 7, characterized in that: The content detection method further includes: Based on the trained neural network model, keyword attributes of the H keywords are updated, and the keyword attributes include at least one of the following: keyword category, keyword tag, keyword update time, keyword historical performance, and keyword source.

9. A content detection device, characterized in that: The content detection device comprises: An acquisition module, used for acquiring the content to be detected, so as to obtain M keywords from the content to be detected, where M is a positive integer; a determination module, configured to determine whether a first keyword and a second keyword associated with the first keyword exist simultaneously in the M keywords; A confirmation module is used to confirm that the content to be detected complies with preset regulations when it is confirmed that the first keyword appears alone in the content to be detected; and to confirm that the content to be detected does not comply with preset regulations when it is confirmed that the first keyword and the second keyword associated with the first keyword appear in the content to be detected at the same time, wherein the first keyword and the second keyword associated with the first keyword that appear at the same time are used to indicate a scenario that does not comply with the preset regulations.

10. The content detection device according to claim 9, characterized in that: The determination module is also used to: Based on the trained neural network model, determine whether a first keyword and a second keyword associated with the first keyword exist simultaneously in the M keywords, wherein the neural network model is used to determine whether the simultaneous occurrence of any two or more combinations of the M keywords complies with preset regulations or does not comply with preset regulations.

11. The content detection device according to claim 10, characterized in that: It also includes a model training module, which is used to: obtain training data before determining whether a first keyword and a second keyword associated with the first keyword exist simultaneously in the M keywords based on the trained neural network model; obtain H keywords from the training data, where H is a positive integer greater than M; and train the neural network model to determine whether the simultaneous occurrence of any two or more combinations of the H keywords complies with or does not comply with preset regulations, wherein the H keywords include the first keyword and the second keyword.

12. The content detection device according to claim 11, characterized in that: The model training module is also used to: Determine whether the first keyword and a second keyword associated with the first keyword exist simultaneously in the H keywords; After confirming that the first keyword appears alone in the training data, confirming that a label of the training data is a label that complies with a preset requirement; After confirming that the first keyword and a second keyword associated with the first keyword appear in the training data at the same time, confirming that the label of the training data is a label that does not comply with a preset requirement; Based on the H keywords included in the training data and the labels of the training data, the trained neural network model is obtained.

13. The content detection device according to claim 12, characterized in that: The model training module is also used to: Generate a keyword index of the H keywords; Based on the keyword indexes of the H keywords, word vectors of the training data are generated.

14. The content detection device according to claim 12, characterized in that: The model training module is also used to: Generating evaluation scores for the labels of the training data; Based on the evaluation scores of the labels of the training data, score vectors of the labels of the training data are generated.

15. The content detection device according to any one of claims 12 to 14, characterized in that: The model training module is also used to: Training the neural network model based on the word vectors of the training data and the score vectors of the labels of the training data; In response to the performance indicator of the neural network model satisfying a predetermined condition, a trained neural network model is obtained.

16. The content detection device according to any one of claims 11 to 15, characterized in that: Based on the trained neural network model, keyword attributes of the H keywords are updated, and the keyword attributes include at least one of the following: keyword category, keyword tag, keyword update time, keyword historical performance, and keyword source.

17. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 8.

18. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 8.

Images