Abnormal broadcast content analysis method based on semantics

Abstract

The invention discloses broadcast content anomaly detection and electronic equipment, and the method comprises the steps: converting to-be-detected broadcast audio data into text data, and extracting a plurality of keywords included in the text data; determining at least one theme associated with the plurality of keywords; and for any theme, if the similarity between the keyword associated with the theme and any keyword group in the keyword library is greater than a preset threshold value, determining that the broadcast content is abnormal. The keyword library comprises at least one keyword group, and the distribution probability of the keyword group corresponding to each theme is greater than a preset threshold value. According to the method, the keywords associated with semantics (themes) implied in the broadcast content serve as the recognition rules for recognizing whether the broadcast content is abnormal or not, and compared with manual keyword screening, the accuracy is higher. And the abnormal publisher corresponding to the theme can be determined by using the mapping relationship between the theme and the abnormal publisher, so that the refined analysis of tracing the abnormal broadcast content is realized.

Description

Technical Field

[0001] The present invention relates to the field of content detection technology, and more particularly to a method and device for detecting abnormalities in broadcast content. Background Art

[0002] Some bad information will be broadcast on a fixed frequency along with authorized information. Compared with the broadcast of authorized information, the broadcast of bad information is also called abnormal broadcast. Abnormal broadcasts include broadcasts of false advertisements, fraudulent information, vulgar programs, etc. Once abnormal broadcasts are discovered, relevant departments will crack down on them. In order to avoid crackdowns, the content of abnormal broadcasts is more scattered and varied. For example, irrelevant information such as music is usually inserted into the content of the broadcast, which increases the difficulty of identifying whether the broadcast content is abnormal. How to efficiently and accurately identify whether the broadcast content is abnormal is an urgent problem to be solved. Summary of the invention

[0003] The present application provides a broadcast content anomaly detection method and electronic device, which can efficiently and accurately identify whether the broadcast content includes abnormal information.

[0004] In a first aspect, a method for detecting anomalies in broadcast content is provided. The method is performed by an electronic device with computing capabilities, such as a computer or a personal computer.

[0005] The method for detecting abnormal broadcast content includes: converting broadcast audio data to be detected into text data, extracting multiple keywords included in the text data; determining at least one topic associated with multiple keywords, wherein one topic is associated with one or more keywords; for any topic, if the similarity between the keywords associated with the topic and any keyword group in a keyword library is greater than a preset threshold, determining that the broadcast content is abnormal, wherein the keyword library includes at least one keyword group, and the keyword group corresponding to each topic consists of a group of keywords whose distribution probability is greater than a preset threshold.

[0006] In this scheme, keywords associated with the semantics (topics) implied in the broadcast content are used as identification rules to determine whether the broadcast content is abnormal. Compared with manual keyword screening, it does not rely on subjectivity and has a higher accuracy rate.

[0007] In a possible implementation, the method further includes: The publisher associated with the topic representing the abnormal content is determined based on a mapping relationship between the abnormal topic and the abnormal publisher, wherein the mapping relationship is determined based on the publisher associated with the historical abnormal broadcast audio data.

[0008] The solution associates the subject of the abnormal broadcast with the publisher of the abnormal broadcast, and uses the determined subject of the abnormal broadcast to determine the abnormal publisher corresponding to the abnormal broadcast through the mapping relationship between the subject and the abnormal publisher, thereby tracing the abnormal broadcast.

[0009] In a possible implementation, the method further includes: A plurality of historical abnormal broadcast audio data are collected, each of which is converted into text data, and N keywords and K topics associated with each text data are extracted, where N and K are both positive integers; a semantic analysis model is established based on the Latent Dirichlet Allocation (LDA) algorithm and N keywords and K topics, and the semantic analysis model is used to indicate the mapping relationship between keywords and topics; the semantic analysis model is solved to obtain the topic associated with each keyword, and the probability distribution of the keywords associated with each topic is obtained; a keyword library is established based on the probability distribution of the keywords associated with each topic.

[0010] In this solution, based on historical abnormal broadcast audio data, keywords are used as feature values ​​of text data, and the topics associated with keywords represent the main content expressed by the text data. By extracting topics and the probability distribution of keywords under each topic, the keywords associated with each topic can be determined, so that a keyword library for determining text semantics can be established.

[0011] In a possible implementation, the method further includes: For any historical abnormal broadcast audio data, the abnormal topic associated with the historical abnormal broadcast audio data is determined based on the semantic analysis model, and the abnormal topic is the topic with the largest distribution probability among multiple abnormal topics associated with the historical abnormal broadcast audio data; a corresponding relationship between the publishers of multiple historical abnormal broadcast audio data and the abnormal topics of multiple historical abnormal broadcast audio data is established; based on the corresponding relationship, a mapping relationship between multiple abnormal topics and abnormal publishers is obtained.

[0012] It is understandable that the publisher of the historical abnormal broadcast audio data is known. In this solution, the subject of the historical abnormal broadcast audio data can be associated with the publisher, so that the abnormal broadcast can be traced back based on the association.

[0013] In a possible implementation, based on the corresponding relationship, a mapping relationship between multiple abnormal topics and abnormal publishers is obtained, including: For any topic, record the occurrence frequency of the associated publisher; determine at least one candidate publisher associated with the topic based on the occurrence frequency; determine the candidate publishers with greater than the minimum support and minimum confidence among the candidate publishers as abnormal publishers corresponding to the topic; and establish a mapping relationship between the topic and the abnormal publisher.

[0014] In this scheme, an association relationship between the subject and publisher of each abnormal broadcast audio data can be established in advance, and the subject and publisher of each abnormal broadcast audio data can be represented by Boolean variables, so that the target publisher of the subject can be determined based on the frequency of occurrence of each associated publisher for the same subject.

[0015] In a possible implementation, after determining that the broadcast content is abnormal and the corresponding publisher, the method further includes: Output prompt information, which includes subject information associated with the broadcast content, information about the publisher corresponding to the broadcast content, etc.

[0016] In this solution, when it is determined that the broadcast content is abnormal, prompt information can be output to prompt that the broadcast content is abnormal. For example, the prompt information may include subject information associated with the broadcast content, information about the publisher corresponding to the broadcast content, and the like.

[0017] In a second aspect, an electronic device is provided, which includes a processing module and a communication interface, the communication interface is used to send and receive data, and the processing module is used to: convert the broadcast audio data to be detected into text data, and extract multiple keywords included in the text data; determine at least one topic associated with multiple keywords, wherein one topic is associated with one or more keywords; for any topic, if the similarity between the keyword associated with the topic and any keyword group in the keyword library is greater than a preset threshold, the broadcast content is determined to be abnormal, wherein the keyword library includes at least one keyword group, and the keyword group corresponding to each topic consists of a group of keywords whose distribution probability is greater than a preset threshold.

[0018] In a possible implementation, the processing module is further configured to: The publisher associated with the topic representing the abnormal content is determined based on a mapping relationship between the abnormal topic and the abnormal publisher, wherein the mapping relationship is determined based on the publisher associated with the historical abnormal broadcast audio data.

[0019] In a possible implementation, the processing module is further configured to: Collect multiple pieces of historical abnormal broadcast audio data, convert each piece of historical abnormal broadcast audio data into text data, and extract N keywords and K topics associated with each piece of text data, where N and K are both positive integers; A semantic analysis model is established based on the latent Dirichlet allocation (LDA) algorithm, N keywords and K topics. The semantic analysis model is used to indicate the mapping relationship between keywords and topics. Solve the semantic analysis model to obtain the topics associated with each keyword and the probability distribution of the keywords associated with each topic; A keyword library is established based on the probability distribution of the keywords associated with each topic.

[0020] In a possible implementation, the processing module is further configured to: For any historical abnormal broadcast audio data, determine the abnormal topic associated with the historical abnormal broadcast audio data based on the semantic analysis model, and the abnormal topic is the topic with the largest distribution probability among multiple abnormal topics associated with the historical abnormal broadcast audio data; Establishing a correspondence between publishers of multiple historical abnormal broadcast audio data and abnormal topics of multiple historical abnormal broadcast audio data; Based on the corresponding relationship, a mapping relationship between multiple abnormal topics and abnormal publishers is obtained.

[0021] In a possible implementation, the processing module is specifically used for: For any topic, record the frequency of occurrence of the associated publishers; Determine at least one candidate publisher associated with the topic according to the occurrence frequency; Determine the candidate publishers whose support and confidence are greater than the minimum among the candidate publishers as abnormal publishers corresponding to the topic; A mapping relationship between the topic and the abnormal publisher is established.

[0022] In a possible implementation, after determining that the broadcast content is abnormal and the corresponding publisher, the processing module is further used to: Output prompt information, which includes subject information associated with the broadcast content, information about the publisher corresponding to the broadcast content, etc.

[0023] In a third aspect, an electronic device is provided, which has the function of implementing the behavior in the method embodiment of the first aspect above. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The electronic device includes a communication interface and a processor, and optionally, also includes a memory. The memory is used to store computer programs or instructions, and the processor is coupled to the memory and the communication interface. When the processor executes the computer program or instruction, the electronic device executes the method executed by the above method embodiment.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which, when executed, implements the method performed in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flowchart of a method for detecting abnormal broadcast content provided by an embodiment of the present application; Figure 2 A schematic diagram of the structure of the LDA model provided in the embodiment of the present application; Figure 3 A schematic diagram of a publisher corresponding to an abnormal broadcast provided in an embodiment of the present application; Figure 4 A flowchart for obtaining a mapping relationship between abnormal topics and abnormal publishers provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0027] Abnormal broadcasting refers to broadcasting unauthorized information on a frequency used to broadcast normal content. Unauthorized information includes, for example, false advertisements, fraudulent information, and vulgar programs. Abnormal broadcasting can also be seen as a way to spread illegal content. In addition, abnormal broadcasting has a higher transmission power than normal broadcasting, which interferes with normal broadcasting signals.

[0028] For this reason, once abnormal broadcasts are discovered, relevant departments will crack down on them. Identifying abnormal broadcasts means identifying whether the content of the broadcasts includes illegal or prohibited content, or in other words, identifying whether the content of the broadcasts includes abnormal content. There are two commonly used methods for identifying abnormal broadcasts.

[0029] The first method is to build an abnormal word library and collect various types of abnormal words. The abnormal word library is used to match the abnormal word library to detect whether there are abnormal words in the broadcast content, so as to determine whether the broadcast is an abnormal broadcast. For example, if the abnormal words included in the broadcast content exceed a certain threshold, the broadcast is considered to be an abnormal broadcast. Common matching methods include the finite state machine (deterministic finite automaton, DFA) algorithm that builds the abnormal word library into a dictionary tree, and the automaton (Aho-Corasick, AC) algorithm of the multi-mode matching algorithm that converts character comparison into state transition to match the string through a finite state machine.

[0030] However, usually abnormal words are manually screened out from the keywords of abnormal content, which requires technical personnel in professional fields and has high maintenance costs. In addition, manual screening of keywords has a certain degree of subjectivity, and the accuracy of the obtained abnormal vocabulary is relatively low, and the accuracy of identifying abnormal broadcasts based on the abnormal vocabulary is also low. In addition, the publisher of abnormal broadcasts can deform the abnormal content by replacing pinyin, replacing the first letter of pinyin, replacing similar-sounding characters, inserting meaningless symbols between the characters of banned words, etc., thereby increasing the difficulty of recognition. The accuracy of identifying abnormal broadcasts based on the first recognition method is low, and even abnormal broadcasts cannot be identified.

[0031] The second method is to obtain identification rules based on machine learning methods, and identify abnormal broadcasts based on the identification rules. The identification rules are obtained by training multiple content samples based on machine learning methods, and classifying multiple content samples from at least one dimension (for example, into two categories: normal and abnormal). Then, based on the classification rules, the broadcast content is classified to determine whether the broadcast is an abnormal broadcast. This application does not limit the machine learning method, for example, it can be a decision tree algorithm, an algorithm using naive Bayes, a support vector machine (SVM) algorithm, a neural network algorithm, and a genetic algorithm.

[0032] Although the second identification method can improve the recognition efficiency, the analysis process of the content sample is invisible, and the understanding of the actual meaning of the content may be wrong. In addition, in order to avoid missed detection, the scope of abnormal content is usually set relatively broad, that is, the broadcast content will not be analyzed at a finer granularity. In this way, even if the broadcast is determined to be abnormal, it is difficult to trace the publisher related to the abnormal broadcast, and the purpose of combating abnormal broadcasts cannot be achieved.

[0033] To this end, an embodiment of the present application provides a method for detecting content anomalies. The method constructs recognition rules based on the semantics implied by the broadcast content, and identifies whether the broadcast is an abnormal broadcast based on the rules. Since the recognition rules are constructed based on the semantics implied by the broadcast content and do not rely on subjectivity, the recognition accuracy is higher. In addition, the method can also associate the abnormal broadcast with the publisher of the abnormal broadcast, thereby tracing the abnormal broadcast and realizing the refined analysis of the abnormal broadcast content.

[0034] The technical solution provided by the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0035] See also Figure 1, which is a flow chart of the broadcast content anomaly detection method provided in an embodiment of the present application. The method can be executed by an electronic device, and the method includes the following steps. The broadcast audio data to be detected can be converted into text data through a speech recognition tool, and the semantics expressed by the broadcast audio data to be detected can be analyzed based on the text data. For example, the semantics expressed by the broadcast audio data to be detected can be determined by extracting multiple keywords included in the text data. When extracting multiple keywords included in the text data, the text data can be segmented, and stop words can be removed to form one or more keyword sets. For example, the text data is data related to a hospital, so after the word segmentation process, the keyword set that may be obtained is ,For example, If the text data is investment-related, then after word segmentation, the possible keyword set is ,For example, .

[0036] After extracting multiple keywords included in the text data, the semantics expressed by the text data are determined based on these multiple keywords. In an embodiment of the present application, the semantics expressed by the broadcast audio data to be detected can be characterized by the topic associated with the broadcast audio data to be detected. A topic may be associated with multiple keywords, and a keyword may be associated with multiple topics. The probability of the distribution of keywords associated with each topic is different. Therefore, the probability distribution of keywords associated with each topic can be used as a criterion for judging whether the broadcast content is abnormal. This judgment criterion is also called an abnormal content identification criterion. The following describes how to obtain the abnormal content identification criterion. With the aforementioned and For example, The keyword is composed of "hospital, cure, heart treatment". Length =3, that is The length is The number of keywords included. It is composed of the key words "investment, get rich overnight, 100%, high return", Length =4. and ,The keyword set consists of {hospital, cure the disease with medicine, heart treatment, investment, get rich overnight, 100%, high return}, and There are V keywords included, V=7. and The values ​​of the associated topics Z can be determined by the type of actual abnormal broadcast content and adjusted by the perplexity. By training the keyword set and the topic as samples, the probability distribution of keywords associated with the topic is obtained, thereby inferring the mapping relationship between the topic and the keyword of the abnormal broadcast content, that is, inferring the recognition rules of abnormal content.

[0037] Specific to the present application, firstly, a plurality of historical abnormal broadcast audio data can be collected, each piece of historical abnormal broadcast audio data can be converted into text data, and N keywords included in each text data and K topics associated therewith can be extracted, where N and K are both positive integers. A semantic analysis model can be established through text data, keywords and topics. For example, based on the LDA algorithm, the text data converted from a plurality of historical abnormal broadcast audio data, the extracted N keywords and K topics can be used as samples to generate an LDA model. That is, a semantic analysis model. That is, the semantic analysis module is divided into three layers of elements: text data, keywords and topics, and these three layers of elements conform to the Dirichlet Distribution.

[0038] For example, taking D pieces of historical abnormal broadcast audio data as an example, each piece of historical abnormal broadcast audio data is converted into text data to obtain D text data. For any text data, after word segmentation, extract keywords (in , ). represents the nth keyword in the text data d. In the following introduction, represents the i-th word among all keywords. N is the number of keywords included in a text data, d represents the d-th text data, To represent the length of the dth text data, that is, the dth text data is represented by The number of keywords included in D text data is V. D text data is associated with K topics, recorded as The topic number corresponding to the nth keyword in the text data d is recorded as In the following introduction, It is represented as the topic corresponding to the i-th word in V keywords, where It is a two-dimensional subscript, corresponding to the nth word in the dth text data.

[0039] by As Figure 2 The input of the LDA model structure shown is based on the LDA model Perform training to obtain a semantic analysis model. Figure 2 middle, is the model parameter of LDA. For any text data, its associated topic distribution ,in, is the hyperparameter of the K-dimensional Dirichlet distribution, let ; is the Dirichlet variable, Represents the multinomial distribution of the mth text data on the topic, with a length of K. Assume ,but For any topic K, we have is the hyperparameter of the Dirichlet distribution of the V-dimensional vector, is the probability parameter of the keyword distribution associated with each topic to generate the corresponding vocabulary, let , It is the probability distribution of the keywords associated with each topic. , with a length of V.

[0040] Furthermore, the semantic analysis model (ie, LDA model) is solved to obtain the topics associated with each keyword, and the probability distribution of the keywords associated with each topic is obtained.

[0041] It can be seen that the D text data includes V keywords and K related topics. The embodiment of the present application can solve the LDA model based on the Gibbs sampling algorithm and use the Dirichlet hyperparameter is a known priori input, obtained by Gibbs sampling , The corresponding topic distribution and keyword distribution associated with all text data.

[0042] Specifically, the keywords of abnormal broadcasts can be obtained through the Gibbs sampling algorithm. , the joint distribution of the topic Z of the abnormal broadcast , further, we can find out the keywords of a certain abnormal broadcast The topic corresponding to the abnormal broadcast The conditional probability distribution of .in, represents the distribution of topics corresponding to the keywords of abnormal broadcasts after removing the subscript i. In the following introduction, Indicates to remove the keyword with subscript i. According to the conditional probability distribution Gibbs sampling is performed to obtain the topic of the keyword of the ith abnormal broadcast. In this way, the topics of the keywords of all abnormal broadcasts can be obtained through Gibbs sampling. By counting the topic counts of all keywords, the keyword distribution of the topics of each abnormal broadcast can be obtained. By counting the topic counts of the keywords of each abnormal broadcast, the topic distribution of each abnormal broadcast can be obtained.

[0043] For example, the text data-topic distribution of abnormal broadcasts and the topic-keyword distribution of abnormal broadcasts in the LDA model conform to the Dirichlet distribution. The expression of the Dirichlet distribution is as follows, where is the normalization parameter: (1) Keywords of abnormal broadcasts obtained by Bayes' rule , the subject of abnormal broadcast The joint distribution of for: (2) In formula (2) is the conditional distribution of keywords of all abnormal broadcast topics, is the conditional distribution of all topics broadcasted by anomalies, namely: (3) In this model, there are Dirichlet distributions of D abnormal broadcast topics, corresponding to multinomial distributions of D topic numbers, Using this Dirichlet-multi conjugate distribution, calculate the conditional distribution of the topic of the d-th abnormal broadcast The formula is as follows:

[0044]

[0045]

[0046]

[0047] (4) Among them, in the text data of the dth abnormal broadcast, the number of keywords of the topic of the kth abnormal broadcast is expressed as: The corresponding multinomial distribution count can be expressed as: (5) Based on the topic distribution of any abnormal broadcast in formula (5), the topic conditional distribution of all abnormal broadcasts can be obtained: (6) For the distribution of topics and keywords of abnormal broadcasts, the LDA model has Dirichlet distributions of K abnormal broadcast topics and keywords, and corresponds to the multinomial distribution of K abnormal broadcast topics. This forms a Dirichlet-multi conjugate, where is the keyword corresponding to the topic k. Similar to calculating the conditional distribution of the topic of abnormal broadcasts, the conditional distribution of keywords corresponding to all topics can be obtained. : (7) Among them, the number of the vth keyword in the kth topic is expressed as , the corresponding multinomial distribution count can be expressed as: (8) Finally, we get the joint distribution of topics and keywords as follows: (9) when When , only the text data of the d-th abnormal broadcast and the two Dirichlet-multi conjugates of the topic of the k-th abnormal broadcast are involved, that is: (10) (11) The remaining D+K−2 Dirichlet-multi conjugates are independent of these two conjugates. Remove all the topics in the abnormal broadcast , remove all the associated keywords It does not change the previous D+K Dirichlet-multi conjugate structure, but the counts of some positions of the vector will be reduced, so for , The corresponding posterior distribution calculation formula is as follows: (12) (13) in, , It is after removing the i , Statistics, the conditional probability calculation required for Gibbs sampling is as follows:

[0048]

[0049]

[0050]

[0051] (14) In formula (14), the expectation calculation of the Dirichlet distribution obtains formula (15) and formula (16): (15) (16) From formula (15) and formula (16), the conditional probability of Gibbs sampling of the topic corresponding to the keyword of each abnormal broadcast can be obtained as formula (17): (17) According to formula (17), Gibbs sampling is used to sample the topics associated with the keywords of all abnormal broadcasts. After convergence, the sampled topics of the keywords of all abnormal broadcasts are obtained. Then, the corresponding relationship between the keywords of abnormal broadcasts and the topics of abnormal broadcasts is obtained, and the distribution of text data-topics of each abnormal broadcast is obtained. and the distribution of all keywords in the topic of each abnormal broadcast .

[0052] For example, solving The process includes: 1) Input , set parameters according to actual data conditions , for example, setting , ; 2) For each keyword in each text data, a topic number z is randomly assigned; 3) Re-traverse all the keywords associated with all historical abnormal broadcasts, use formulas 6 and 7 to calculate the conditional distribution of the topics of all abnormal broadcasts and the conditional distribution of the keywords corresponding to the topics of all abnormal broadcasts, and then calculate the joint distribution of the topics and keywords of abnormal broadcasts according to formula 9. For each keyword, use Gibbs sampling formula 17 to update the topic number of the keyword and update the keyword's own number.

[0053] 4) Repeat the Gibbs sampling based on axis rotation in step 3 until the Gibbs sampling converges.

[0054] 5) Count the topics of each keyword in each historical abnormal broadcast to obtain the abnormal broadcast-topic distribution θd, and count the distribution of keywords associated with the topic of the historical abnormal broadcast to obtain the distribution of topics and keywords βk.

[0055] Furthermore, an abnormal content identification criterion may be constructed based on the probability distribution of keywords associated with each topic, and the criterion may be a keyword library. If the keywords of a topic associated with a broadcast are in the keyword library, then the broadcast is an abnormal broadcast.

[0056] Specifically, based on the LDA model (i.e., the semantic analysis model in the embodiment of the present application), according to the known D abnormal broadcast text data, an appropriate number of topics K and an appropriate hyperparameter vector can be selected. Determine the theme of each keyword as described above, count the themes associated with the keywords included in each text data in the D text data, and obtain the theme distribution of the D text data ; Then count the distribution of keywords associated with each topic to get the distribution of topics and keywords . , that is, the probability distribution of keywords under each topic. In an embodiment of the present application, for each topic, a group of keywords with a higher distribution probability among the associated keywords can be selected as the discrimination criterion of the topic. In other words, each topic may correspond to a group of keywords, which is composed of a group of keywords whose distribution probability is greater than a preset threshold. The keyword groups corresponding to multiple topics can form a keyword library, which can be used as a discrimination criterion for abnormal content. It can be understood that based on historical abnormal broadcast audio data, the discrimination criteria for abnormal topics can be obtained, and then the abnormal content identification criterion library (i.e., keyword library) R={ }, used to detect whether the broadcast content is abnormal.

[0057] It should be noted that the embodiments of the present application do not limit the specific implementation method of constructing the abnormal content identification criterion library (i.e., keyword library). For example, the LDA semantic analysis model parameters can also be solved by the variational inference maximization algorithm (Expectation-Maximum, EM) algorithm. In this case, it can be assumed that all They are all formed through their own independent distribution, and the joint distribution of the topic and keywords of the text data is obtained. Then the E step of the EM algorithm is used to obtain the optimal variational parameters, and the M step is used to update the LDA model parameters. .

[0058] S102: Determine at least one topic associated with multiple keywords, wherein one topic is associated with one or more keywords.

[0059] When determining whether the content of broadcast audio data is abnormal, after extracting multiple keywords included in the text data converted from the broadcast audio data, at least one topic associated with the multiple keywords can be determined, thereby determining whether the content of the broadcast spectrum data is abnormal based on the keyword library.

[0060] S103: For any topic, if the similarity between the keywords associated with the topic and any keyword group in the keyword library is greater than a preset threshold, it is determined that the broadcast content is abnormal.

[0061] The keyword library includes at least one keyword group, and the keyword group corresponding to each topic is composed of a group of keywords whose distribution probability is greater than a preset threshold. For any topic, the keywords associated with the topic can be matched with the keyword library. If the similarity between the keywords associated with the topic and any keyword group in the keyword library is greater than a preset threshold, it can be determined that the broadcast content corresponding to the topic is abnormal.

[0062] Taking the broadcast audio data d as an example, d can be preprocessed, such as converting d into text data, performing word segmentation, removing stop words, etc.; then, semantic analysis is performed on d based on the established semantic analysis model. It can be understood that when Gibbs sampling is performed on d, Already fixed, for the front half Perform sampling calculation to obtain the topic-keyword distribution of d . and abnormal broadcast content identification criteria library (keyword library) The similarity is calculated for each abnormal criterion in ; if the similarity with all abnormal broadcast criteria is lower than the set threshold, it is a normal broadcast, otherwise it is regarded as an abnormal broadcast.

[0063] In the embodiment of the present application, the keywords associated with the semantics (topics) implied in the broadcast content are used as the identification rules for identifying whether the broadcast content is abnormal. Compared with manual keyword screening, it does not rely on subjectivity and has a higher accuracy rate. It is also more efficient than manual keyword screening.

[0064] S104: Determine the abnormal publisher corresponding to the broadcast-related topic according to the mapping relationship between the topic and the abnormal publisher.

[0065] In order to reduce abnormal broadcasts, they can be blocked from the source. To this end, in an embodiment of the present application, an association mapping relationship can also be established between the subject of the abnormal broadcast and the publisher of the abnormal broadcast, and the abnormal publisher corresponding to the subject can be determined by the mapping relationship between the subject and the abnormal publisher using the determined abnormal subject, so as to prevent the publisher from publishing abnormal broadcasts. The publisher of the abnormal broadcast is also called the abnormal publisher.

[0066] First, by analyzing the historical abnormal broadcast audio data, a mapping relationship between abnormal topics and abnormal publishers can be established. For an abnormal broadcast audio data, one or more topics may be associated, including normal topics and abnormal topics. The topic bound to the publisher of the abnormal broadcast can be one or more abnormal topics. For example, see Figure 3Taking M abnormal broadcast audio data as an example, a part of the M abnormal broadcast audio data is related to the abnormal publisher m; the topic of N abnormal broadcast audio data is topic n; and among the abnormal broadcast audio data of topic n, X abnormal broadcast audio data are related to the abnormal publisher m.

[0067] In this case, the subject and publisher of each piece of historical abnormal broadcast data can be represented by Boolean variables to indicate whether they appear, to form a corresponding relationship between the subject of abnormal broadcast and the publisher of abnormal broadcast, to build a model for associating the subject and publisher of abnormal broadcast, and then analyze the Boolean variables to obtain the mapping relationship (or association criteria) between the subject of abnormal broadcast and the publisher. Taking any historical abnormal broadcast audio data as an example, the abnormal subject associated with the historical abnormal broadcast audio data can be determined based on the semantic analysis model. The abnormal subject is the subject with the highest distribution probability among the multiple subjects associated with the historical abnormal broadcast audio data; then, a corresponding relationship between the publishers of multiple pieces of historical abnormal broadcast audio data and the subjects of multiple pieces of historical abnormal broadcast audio data is established; and then, based on the corresponding relationship, a mapping relationship between multiple abnormal subjects and abnormal publishers is obtained.

[0068] Among them, for any historical abnormal broadcast audio data, the topic with the highest probability is taken as the topic of the historical abnormal broadcast audio data. The abnormal publisher corresponding to the topic can be marked as True. The correspondence between the publishers of multiple historical abnormal broadcast audio data and the topics of multiple historical abnormal broadcast audio data can be used as data set B, as shown in Table 1. The item set in Table 1 is topic-publisher.

[0069]

[0070] In a possible implementation, for any topic, the frequency of occurrence of the associated publisher is recorded; at least one candidate publisher associated with the topic is determined based on the frequency of occurrence; the candidate publishers with a value greater than the minimum support and the minimum confidence among the candidate publishers are determined as abnormal publishers corresponding to the topic; and a mapping relationship between abnormal topics and abnormal publishers is established. For example, the corresponding relationship between the publishers of multiple historical abnormal broadcast audio data and the topics of multiple historical abnormal broadcast audio data can be analyzed through an iterative process of layer-by-layer search based on the Apriori algorithm, so as to infer the mapping relationship between abnormal topics and abnormal publishers. Among them, the minimum support and minimum confidence values ​​are set according to the actual data situation. For example, it can be assumed that the minimum support is 40% and the minimum confidence value is 80%.

[0071] For details, see Figure 4, a flowchart for analyzing the correspondence between publishers of multiple historical abnormal broadcast audio data and topics of multiple historical abnormal broadcast audio data based on the Apriori algorithm, and obtaining the mapping relationship between abnormal topics and abnormal publishers, includes the following steps.

[0072] S401. Obtain database B, which includes correspondences between publishers of multiple historical abnormal broadcast audio data and topics of multiple historical abnormal broadcast audio data.

[0073] Using the example in Table 1, database B includes a set of all items, such as {{topic 1}, {topic 2}, {publisher 1}, {publisher 2}, {publisher 3}}. k=1, the frequent 0-item set is an empty set.

[0074] S402: Count the number of times the publisher associated with each topic appears, and generate a candidate item set C1.

[0075] The candidate item set C1 may be a set consisting of all items included in the database B.

[0076] S403 . Generate a frequent 1-item set L1 according to the candidate item set C1 , wherein the frequent 1-item set L1 is a set of items remaining after deleting single item sets whose times are lower than a minimum support threshold.

[0077] A connection operation is performed on the frequent 1-item set L1 to generate a candidate item set C2. Specifically, the support of the candidate frequent k-item set is calculated using Formula 19. For example, the support of each item set in C1 is 2 / 3, 1 / 3, 2 / 3, 1 / 3, and 1 / 3. For example, C2 is {topic 1, publisher 1}.

[0078] After deleting the single item sets whose occurrence times in the candidate item set C2 are lower than the minimum support threshold, the frequent 2-item set L2 is obtained until all topics are traversed. That is, iterate S402-S403 until no higher-order frequent-item sets are generated. At this time, the frequent k-item set L K For all the frequent itemsets mined, the association rules (or mapping relationships) between the topics of abnormal broadcasts and publishers are generated according to the set minimum support and minimum confidence, as shown in steps S304-S309.

[0079] S404. Set K=1.

[0080] S405: Generate frequent k-item set L based on database B and the support of each candidate item K .

[0081] S406, determine L K Whether it is empty; S407, if LK Not empty, according to the frequent k-item set L K Generate candidate item set C K+1 .

[0082] S408. Let K=K+1.

[0083] S409, if L K Empty, according to the frequent k-item set L K A mapping between topics and publishers that generate exception broadcasts.

[0084] Among them, the mapping relationship between the topic of abnormal broadcast and the publisher satisfies formula (18): The subject of the abnormal broadcast Exception Publisher (18) In the formula: "The subject of abnormal broadcast Exception Publisher " represents the parameters associated with "abnormal broadcast topic n" and "abnormal publisher m" . in, represents the support of the association, Indicates the confidence of the association, the value range of a and b is .

[0085] Support Indicates all abnormal broadcasts The topic of the abnormal broadcast is topic n and is related to the abnormal publisher m, such as Figure 3 The gray part in occupies the percentage of abnormal broadcast set; the confidence b% means that b% of abnormal broadcasts with topic n are related to abnormal publisher m, such as Figure 4 The gray part in accounts for the percentage of abnormal broadcasts with the topic n. The support and confidence associated with “abnormal broadcast topic n” and “abnormal publisher m” are calculated by the following formula: (19) (20) The minimum support threshold and the minimum confidence threshold are set based on experience. If the relevant parameters of "abnormal broadcast topic n" and "abnormal publisher m" are met at the same time, the abnormal broadcast topic is formed. Association rules of abnormal publisher m. It can be understood that all association rules are collected to construct an abnormal publishing association rule library G, which can be used to trace the publisher of abnormal broadcasts.

[0086] It should be noted that, in a possible implementation, the correspondence between the publishers of multiple historical abnormal broadcast audio data and the topics of multiple historical abnormal broadcast audio data can also be analyzed by the Frequent-Pattern Growth (FP-Growth) algorithm, so as to infer the mapping relationship between abnormal topics and abnormal publishers. In this case, it is necessary to calculate the support of each candidate item, and readjust the order of items in each historical abnormal broadcast audio data in database B, establish an FP-Tree according to the frequent-item sets arranged in descending order, and mine the frequent item sets through the FP-Tree to obtain the mapping relationship between the topic and the publisher.

[0087] If it is determined that the content of the broadcast is abnormal, a prompt message may be output to indicate that the broadcast content is abnormal. For example, the prompt message may include subject information associated with the broadcast content or information related to the subject information. Thus, the receiving end receiving the broadcast may determine that the broadcast content is abnormal based on the subject information or information related to the subject information.

[0088] The following describes the device used to implement the above method in the embodiment of the present application in conjunction with the accompanying drawings. Therefore, the above content can be used in subsequent embodiments, and the repeated content will not be repeated.

[0089] The embodiment of the present application also provides an electronic device that implements the functions of each of the above-mentioned method embodiments. The electronic device includes at least a processing module and a communication interface, etc. The communication interface is used to send and receive data, and the processing module is used to: convert the broadcast audio data to be detected into text data, extract multiple keywords included in the text data; determine at least one topic associated with multiple keywords, wherein one topic is associated with one or more keywords; for any topic, if the similarity between the keywords associated with the topic and any keyword group in the keyword library is greater than a preset threshold, the broadcast content is determined to be abnormal, wherein the keyword library includes at least one keyword group, and the keyword group corresponding to each topic is composed of a group of keywords with a distribution probability greater than a preset threshold.

[0090] In a possible implementation, the processing module is further configured to: The publisher associated with the topic representing the abnormal content is determined based on a mapping relationship between the abnormal topic and the abnormal publisher, wherein the mapping relationship is determined based on the publisher associated with the historical abnormal broadcast audio data.

[0091] In a possible implementation, the processing module is further configured to: Collect multiple pieces of historical abnormal broadcast audio data, convert each piece of historical abnormal broadcast audio data into text data, and extract N keywords and K topics associated with each piece of text data, where N and K are both positive integers; A semantic analysis model is established based on the latent Dirichlet allocation (LDA) algorithm, N keywords and K topics. The semantic analysis model is used to indicate the mapping relationship between keywords and topics. Solve the semantic analysis model to obtain the topics associated with each keyword and the probability distribution of the keywords associated with each topic; A keyword library is established based on the probability distribution of the keywords associated with each topic.

[0092] In a possible implementation, the processing module is further configured to: For any historical abnormal broadcast audio data, determine the abnormal topic associated with the historical abnormal broadcast audio data based on the semantic analysis model, and the abnormal topic is the topic with the largest distribution probability among multiple abnormal topics associated with the historical abnormal broadcast audio data; Establishing a correspondence between publishers of multiple historical abnormal broadcast audio data and abnormal topics of multiple historical abnormal broadcast audio data; Based on the corresponding relationship, a mapping relationship between multiple abnormal topics and abnormal publishers is obtained.

[0093] In a possible implementation, the processing module is specifically used for: For any topic, record the frequency of occurrence of the associated publishers; Determine at least one candidate publisher associated with the topic according to the occurrence frequency; Determine the candidate publishers with a support greater than the minimum and a minimum confidence among the candidate publishers as abnormal publishers corresponding to the topic; A mapping relationship between the topic and the abnormal publisher is established.

[0094] In a possible implementation, after determining that the broadcast content is abnormal and the corresponding publisher, the processing module is further used to: Output prompt information, which includes subject information associated with the broadcast content, information about the publisher corresponding to the broadcast content, etc.

[0095] A computer-readable storage medium is also provided in an embodiment of the present application, including instructions, which, when executed on a computer, enable the computer to execute the method in the above method example. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0096] The term "plurality" in the embodiments of the present application refers to two or more. The term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the associated objects before and after are in an "or" relationship.

[0097] "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0098] Furthermore, unless otherwise specified, ordinal numbers such as "first" and "second" mentioned in the embodiments of the present application are used to distinguish multiple objects and are not used to limit the size, content, order, timing, priority or importance of multiple objects.

[0099] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0100] The various illustrative logic units and circuits described in the embodiments of the present application can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0101] The steps of the method or algorithm described in the embodiments of the present application can be directly embedded in the software unit executed by the hardware, the processor, or a combination of the two. The software unit can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0104] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0107] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting abnormal broadcast content, characterized in that: include: Convert the broadcast audio data to be detected into text data, and extract multiple keywords included in the text data; Determining at least one topic associated with the plurality of keywords, wherein a topic is associated with one or more keywords; For any topic, if the similarity between the keywords associated with the topic and any keyword group in the keyword library is greater than a preset threshold, the broadcast content is determined to be abnormal, wherein the keyword library includes at least one keyword group, and the keyword group corresponding to each topic consists of a group of keywords whose distribution probability is greater than a preset threshold.

2. The method according to claim 1, characterized in that The method further comprises: The publisher associated with the topic representing the abnormal content is determined based on a mapping relationship between the abnormal topic and the abnormal publisher, wherein the mapping relationship is determined based on the publisher associated with the historical abnormal broadcast audio data.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Collect multiple pieces of historical abnormal broadcast audio data, convert each piece of the historical abnormal broadcast audio data into text data, and extract N keywords and K topics associated with each piece of the text data, where N and K are both positive integers; Establishing a semantic analysis model based on a latent Dirichlet allocation (LDA) algorithm and the N keywords and the K topics, wherein the semantic analysis model is used to indicate a mapping relationship between keywords and topics; Solving the semantic analysis model to obtain topics associated with each keyword and obtaining a probability distribution of keywords associated with each topic; The keyword library is established based on the probability distribution of the keywords associated with each topic.

4. The method according to claim 3, characterized in that The method further comprises: For any historical abnormal broadcast audio data, determine the abnormal topic associated with the historical abnormal broadcast audio data based on the semantic analysis model, wherein the abnormal topic is the topic with the largest distribution probability among multiple abnormal topics associated with the historical abnormal broadcast audio data; Establishing a correspondence between publishers of the plurality of historical abnormal broadcast audio data and abnormal topics of the plurality of historical abnormal broadcast audio data; Based on the corresponding relationship, a mapping relationship between multiple abnormal topics and abnormal publishers is obtained.

5. The method according to claim 4, characterized in that Based on the corresponding relationship, a mapping relationship between multiple abnormal topics and abnormal publishers is obtained, including: For any topic, record the frequency of occurrence of the associated publishers; Determine at least one candidate publisher associated with the topic according to the occurrence frequency; Determine a candidate publisher whose support and confidence are greater than the minimum among the at least one candidate publisher as an abnormal publisher corresponding to the topic; A mapping relationship between the topic and the abnormal publisher is established.

6. The method according to any one of claims 2 to 5, characterized in that: After determining that the broadcast content is abnormal and the corresponding publisher, the method further includes: Output prompt information, wherein the prompt information includes subject information associated with the broadcast content and information of a publisher corresponding to the broadcast content.

7. An electronic device, characterized in that: It includes a processing module and a communication interface, wherein the communication interface is used to send and receive data, and the processing module is used to: Convert the broadcast audio data to be detected into text data, and extract multiple keywords included in the text data; Determining at least one topic associated with the plurality of keywords, wherein a topic is associated with one or more keywords; For any topic, if the similarity between the keywords associated with the topic and any keyword group in the keyword library is greater than a preset threshold, the broadcast content is determined to be abnormal, wherein the keyword library includes at least one keyword group, and the keyword group corresponding to each topic consists of a group of keywords whose distribution probability is greater than a preset threshold.

8. The electronic device according to claim 7, characterized in that: The processing module is also used for: The publisher associated with the topic representing the abnormal content is determined based on a mapping relationship between the abnormal topic and the abnormal publisher, wherein the mapping relationship is determined based on the publisher associated with the historical abnormal broadcast audio data.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store computer programs or instructions. When the processor executes the computer program or instructions, the electronic device executes the method as claimed in any one of claims 1 to 6.

Images