A news event classification method
By using a multi-level keyword lexicon and similarity calculation method, combined with a keyword extraction and fusion model and a multilayer perceptual network model, the problem of insufficient information capture in news event classification is solved, and more accurate and flexible classification results are achieved.
Patent Information
- Application Number
- CN202411234056.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2044-09-04
AI Technical Summary
Existing technologies ignore relevant information other than the original text in news event classification, and simple rules or thesaurus cannot capture the semantics and contextual relationships of keywords, resulting in single and inaccurate classification results.
By employing a multi-level keyword lexicon and similarity calculation method, combined with a keyword extraction fusion model and a multi-layer perceptual network model, and by acquiring direct and indirect data, multi-stage similarity calculation and normalization processing are performed to determine the probability distribution and classification results of news events.
It achieves more accurate, comprehensive and interpretable news event classification, can capture the semantics and contextual relationships of keywords, and provides flexible classification results.
Smart Images

Figure CN119128155B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of natural language processing, and more particularly to a news event classification method. BACKGROUND
[0002] By classifying news events, readers can more easily find content of interest, saving reading time. It can help news organizations or platforms better organize and manage large amounts of news content, making it easier to search and archive. Classification can help monitor and analyze various events, providing decision support and public opinion feedback for governments, businesses and institutions.
[0003] Currently, traditional methods may only use the original text of news articles for classification, ignoring other sources of relevant information. And based on simple rules or word lists, important information in the text cannot be fully captured.
[0004] At the same time, using simple similarity calculation methods cannot accurately assess the semantic and contextual relationships between keywords. And lack of processing and transformation methods for similarity calculation results, resulting in single classification results.
[0005] Therefore, how to design a news event classification method to provide diversified technical solutions for data acquisition and specific processing methods, and to provide more accurate, comprehensive and interpretable news event classification results is a problem that needs to be solved by those skilled in the art. SUMMARY
[0006] Therefore, the present application provides a news event classification method that can capture important information in the text, use a multi-level keyword vocabulary and similarity calculation method to better capture the semantic and contextual relationships between keywords, and more flexibly determine the classification results.
[0007] To achieve the above purpose, the present application adopts the following technical solutions:
[0008] In a first aspect, the present application provides a news event classification method, comprising:
[0009] S1, obtaining text data of a news event; the text data includes direct data and indirect data;
[0010] S2, extracting a plurality of keywords from the text data using a keyword extraction fusion model to obtain a keyword group;
[0011] S3, based on a multi-level keyword vocabulary, performing multi-stage similarity calculation on the keyword group to obtain a similarity calculation result; the multi-level keyword vocabulary includes a first-level keyword library, a second-level keyword library and a third-level keyword library;
[0012] S4, normalize the similarity calculation result, and obtain a probability distribution result of the news event based on a multi-layer perception network model;
[0013] S5, compare a difference value between probability values of two news events with the highest and second highest probability values based on the probability distribution result with a preset probability threshold value, and obtain a classification result of the news event.
[0014] In the S1, direct data is obtained by using an API interface provided by a news website and a data provider; indirect data is obtained by using a crawler model to access the news website or the social media platform.
[0015] Further, keywords in the text data are extracted by using a keyword extraction fusion model, including:
[0016] S21, the text data is preprocessed, including word segmentation and stop word removal;
[0017] S22, based on the preprocessed text data, a term frequency (TF) and an inverse document frequency (IDF) of each word item are calculated to obtain a TF-IDF value of each word item;
[0018] S23, each word item is taken as a node of a graph, a co-occurrence matrix is used to calculate a relationship between the word items, and a text graph is constructed;
[0019] S24, the text graph is sorted by using a TextRank algorithm to determine a TextRank score of each word item;
[0020] S25, the TF-IDF value and the TextRank score are weighted and fused to obtain a comprehensive score of each word item;
[0021] S26, based on the comprehensive score of each word item, a plurality of keywords are extracted by a preset score threshold value, and a keyword group is obtained.
[0022] Further, in the S3, the multi-level keyword library is updated based on an intelligent recommendation mechanism.
[0023] Further, in the S3, a plurality of stages of similarity calculation are performed on the keyword group, including:
[0024] S31, similarity between the keyword group and each word in a three-level keyword library is calculated;
[0025]
[0026] S32, similarity between the keyword group and each word in a two-level keyword library is calculated;
[0027]
[0028] S33, calculate the similarity of the keyword phrase and each vocabulary in the first-level vocabulary library:
[0029]
[0030] wherein c(m, n) represents the keyword phrase vector A m and the word vector B in the second-level vocabulary library n distance.
[0031] Further, in the S4, the news event classification model comprises: an input layer, a hidden layer and an output layer; the hidden layer is provided with a ReLU activation function; and the output layer is provided with a Softmax activation function.
[0032] Further, in the S5, based on the probability distribution result, the difference between the probability values of the two news events with the highest and second highest probability values is compared with a preset probability threshold value to obtain the classification result of the news event, comprising:
[0033] if the difference between the probability values of the two news events with the highest and second highest probability values is greater than the preset probability threshold value, the news event with the highest probability value is output;
[0034] if the difference between the probability values of the two news events with the highest and second highest probability values is less than the preset probability threshold value, the two news events with the highest and second highest probability values are output.
[0035] In a second aspect, the present application provides a news event classification system, comprising:
[0036] a data acquisition module for acquiring text data of a news event; the text data comprises direct data and indirect data;
[0037] a keyword extraction module for extracting a plurality of keywords in the text data by using a keyword extraction fusion model to obtain a keyword phrase;
[0038] a similarity calculation module for performing multi-stage similarity calculation on the keyword phrase based on a multi-level keyword vocabulary library to obtain a similarity calculation result; the multi-level keyword vocabulary library comprises a first-level vocabulary library, a second-level vocabulary library and a third-level vocabulary library;
[0039] a probability distribution calculation module for normalizing the similarity calculation result and obtaining a probability distribution result of the news event based on a multi-layer perception network model;
[0040] a classification result output module for comparing the difference between the probability values of the two news events with the highest and second highest probability values with a preset probability threshold value based on the probability distribution result to obtain the classification result of the news event.
[0041] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the news event classification method when executing the computer program.
[0042] In a fourth aspect, the present application provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is executable on a processor to implement the news event classification method.
[0043] According to the technical solutions described above, compared with the prior art, the news event classification method provided by the present application has the following beneficial effects:
[0044] 1. The keyword extraction fusion model can automatically extract several keywords from the text data. It can capture important information in the text and convert it into keyword phrases, providing more effective feature representation. Compared with traditional rule-based or dictionary-based keyword extraction methods, the fusion model can better adapt to keyword extraction needs in different fields and contexts.
[0045] 2. The use of multi-level keyword lexicon and similarity calculation method can better capture the semantic and contextual relationship between keywords. Through multi-stage similarity calculation, the similarity between keywords can be more accurately evaluated, improving the accuracy and robustness of classification.
[0046] 3. The similarity calculation results are normalized and processed using a multi-layer perception network model, which can map the original data to a probability distribution result. It can effectively handle differences between different data scales and ranges, and convert them into probability distributions, providing more interpretable and comparable classification results.
[0047] 4. By comparing the probability value difference between the two news events with the highest and second highest probability values with the preset probability threshold, the classification result can be flexibly determined according to the size of the difference. Both the accuracy and comprehensiveness of the classification result are taken into account, providing a definite classification result when the probability difference is large, and providing more comprehensive information when the probability difference is small. BRIEF DESCRIPTION OF DRAWINGS
[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.
[0049] Figure 1 The news event classification method flowchart provided by the embodiments of the present application;
[0050] Figure 2 A method flow chart for extracting keywords in text data by using a keyword extraction fusion model is provided for the embodiments of the present application.
[0051] Figure 3 A method flow chart for multi-stage similarity calculation on keyword phrases is provided for the embodiments of the present application.
[0052] Figure 4 A schematic diagram of a news event classification system is provided for the embodiments of the present application.
[0053] Figure 5 An electronic device structure schematic diagram is provided for the embodiments of the present application. DETAILED DESCRIPTION
[0054] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.
[0055] The detection method provided by the embodiments of the present application can be applied to a detection server. The detection server can be hardware or software. When the detection server is hardware, it can be implemented as a distributed server cluster providing detection services, or as a single server. When the detection server is software, it can be installed in the above-mentioned servers. It can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here.
[0056] Embodiment 1:
[0057] As shown in Figure 1 The present embodiment provides a news event classification method, which includes:
[0058] S1, obtaining text data of a news event; the text data includes direct data and indirect data;
[0059] S2, extracting a plurality of keywords in the text data by using a keyword extraction fusion model to obtain keyword phrases;
[0060] S3, performing multi-stage similarity calculation on the keyword phrases based on a multi-level keyword library to obtain a similarity calculation result; the multi-level keyword library includes a first-level keyword library, a second-level keyword library, and a third-level keyword library;
[0061] S4. Normalize the similarity calculation results and obtain the probability distribution results of news events based on the multilayer perceptron model.
[0062] S5. Based on the probability distribution results, compare the difference between the probability values of the two news events with the highest and second highest probability values with a preset probability threshold to obtain the classification results of the news events.
[0063] This method, by comprehensively utilizing different types of data, fusion models, multi-level keyword lexicons and similarity calculations, normalization processing, and flexible classification result processing, can provide more accurate, comprehensive, and interpretable news event classification results.
[0064] The following provides a more detailed explanation of each of the above steps:
[0065] In S1, direct data is obtained by utilizing APIs provided by news websites and data providers; indirect data is obtained by using crawler models to access news websites or social media platforms.
[0066] The data acquisition method in this embodiment is not limited to a single specific text. It can combine the use of API interfaces to obtain direct data and the use of web crawler models to obtain indirect data, thereby acquiring news event data from multiple data sources. Direct data sources provide high-quality, processed data through API interfaces, which is suitable for scenarios with high data quality requirements; while indirect data sources, through web crawler models, can obtain more diverse and comprehensive data, which is suitable for obtaining detailed information from specific websites or social media platforms.
[0067] like Figure 2 As shown, in S2, the keyword extraction fusion model is used to extract keywords from the text data, including:
[0068] S21. Preprocess the text data, including word segmentation and stop word removal. Word segmentation is the process of breaking down the text into terms, while stop word removal refers to removing frequently occurring words in the text that are irrelevant to distinguishing text features. This step helps to transform the text into a discrete sequence of terms, reducing redundant information and noise.
[0069] S22. Based on the preprocessed text data, calculate the term frequency (TF) and inverse document frequency (IDF) for each term to obtain the TF-IDF value for each term; TF represents the frequency of a term in the text, and IDF represents the importance of a term in the entire text set.
[0070] S23, each word item is taken as a node of a graph, the relationship between word items is calculated using a co-occurrence matrix, and a text graph is constructed; the co-occurrence matrix records the co-occurrence relationship of word items in the text, and the semantic correlation between them can be revealed by counting the co-occurrence times between word items.
[0071] S24, the TextRank algorithm is used to sort the text graph to determine the TextRank score of each word item; the weight transfer between word items is calculated by iteration to assign a higher score to important word items. It considers the connection relationship and importance of word items in the text graph, and can accurately reflect the keyness of word items.
[0072] S25, the TF-IDF value and the TextRank score are weighted and fused to obtain the comprehensive score of each word item; by weighting the TF-IDF value and the TextRank score, the frequency and importance of word items in the text can be considered comprehensively to obtain more comprehensive and accurate word item scores. Such a fusion method can balance the advantages of the two methods in keyword extraction, improve the accuracy and diversity of keywords.
[0073] S26, based on the comprehensive score of each word item, the keywords are sorted and extracted by a preset score threshold to obtain keyword phrases. According to the comprehensive score, the word items are sorted according to importance, and then the word items with scores higher than the threshold are selected as keywords according to the preset score threshold.
[0074] In this embodiment, the keyword extraction fusion model combines different technical methods at different steps, fully considers the frequency, importance, correlation and other factors of word items, so as to extract the most representative and key word items in the text.
[0075] In S3, the multi-level keyword vocabulary is updated based on an intelligent recommendation mechanism.
[0076] In the intelligent recommendation mechanism, the TF-IDF algorithm is used to extract keywords from new data, the semantic similarity between new words and existing vocabulary words is calculated through word embedding technology, and a multi-level classifier is constructed to combine word frequency, context information, similarity and other information to determine the vocabulary level to which the new word belongs.
[0077] At the same time, there is also a user feedback channel, a user feedback interface is set to allow users to provide opinions on the accuracy and completeness of the vocabulary. An online community is built to promote communication between users and experts, share vocabulary use experiences, and collect update suggestions.
[0078] The intelligent recommendation update mechanism of the multi-level keyword vocabulary can efficiently and accurately capture emerging words and timely reflect the latest developments in the fields of science and technology, society and others, providing strong support for intelligent analysis of news events.
[0079] As Figure 3 shown, in S3, multi-stage similarity calculation is performed on the keyword phrase, including:
[0080] S31, similarity between the keyword phrase and each vocabulary in the third-level vocabulary library is calculated:
[0081]
[0082] S32, similarity between the keyword phrase and each vocabulary in the second-level vocabulary library is calculated:
[0083]
[0084] S33, similarity between the keyword phrase and each vocabulary in the first-level vocabulary library is calculated:
[0085]
[0086] wherein c(m, n) represents the similarity between the keyword phrase vector A m and the word vector B n in the second-level vocabulary library.
[0087] For example, the keyword phrase obtained based on a news article is: (vector machine, speech to text, private cloud, HDFS, remote monitoring)
[0088] The similarity between the keyword phrase and each vocabulary in the third-level vocabulary library is calculated as follows:
[0089] Supervised learning 0.95, unsupervised learning 0.75; speech recognition 0.95, sentiment analysis 0.55; public cloud service 0.65, hybrid cloud architecture 0.95; data mining 0.65, Hadoop ecosystem 0.85; millimeter wave communication 0.25, network slicing 0.35; smart home 0.35, industrial Internet of Things 0.75;
[0090] The similarity between the keyword phrase and each vocabulary in the second-level vocabulary library is calculated as follows:
[0091] The similarity between the keyword phrase and each vocabulary in the second-level vocabulary library is calculated as follows:
[0092] Based on supervised learning and unsupervised learning, the machine learning is 1.7, based on speech recognition and sentiment analysis, the natural language processing is 1.5; based on public cloud service and hybrid cloud architecture, the cloud computing is 1.6, based on data mining and Hadoop ecosystem, the big data is 1.5; based on millimeter wave communication and network slicing, the 5G network is 0.6, based on smart home and industrial Internet of Things, the Internet of Things is 1.1;
[0093] The similarity between the keyword phrase and each vocabulary in the first-level vocabulary library is calculated as follows:
[0094] The artificial intelligence based on machine learning and natural language processing is 3.2, the information technology based on cloud computing and big data is 3.1, and the communication technology based on 5G network and Internet of Things is 1.6.
[0095] In S4, the original similarity calculation result obtained in S3 is normalized, the purpose is to eliminate the dimension influence, so that the comparison between different characteristics is more fair and reasonable. The minimum-maximum normalization formula is as follows:
[0096]
[0097] For the given data: artificial intelligence is 3.2, information technology is 3.1, and communication technology is 1.6, the minimum value is 1.6, and the maximum value is 3.2. Therefore, after normalization:
[0098] After normalization, artificial intelligence: 3.2-1.6 3.2-1.6 = 1.0 3.2-1.6 3.2-1.6 = 1.0
[0099] After normalization, information technology: 3.1-1.6 3.2-1.6 = 0.96 8753.2-1.6 3.1-1.6 = 0.96 875
[0100] After normalization, communication technology: 1.6-1.6 3.2-1.6 = 0.0 3.2-1.6 1.6-1.6 = 0.0
[0101] Further, the normalized data is input into the multi-layer perception network model as an input feature. The multi-layer perception network model includes an input layer, one or more hidden layers, and an output layer.
[0102] The input layer has three nodes, respectively corresponding to the similarity scores of "artificial intelligence", "information technology" and "communication technology" after normalization. Specifically, there are 3 nodes, a hidden layer with 5 nodes, and an output layer with C nodes according to the predefined number of categories of news events, assuming C categories, i.e. the output layer has C nodes.
[0103] During the training of the multi-layer perception network model, a large number of pre-labeled news samples are used to train the MLP model, which contains news text and its category label.
[0104] The trained MLP model receives the normalized feature vector [1.0, 0.96875, 0.0] as input, and after nonlinear transformation of the hidden layer, a C-dimensional probability vector is finally generated in the output layer, representing the probability of the news event belonging to each category. The output is [0.5, 0.45, 0.05]
[0105] In S5, based on the probability distribution result, the difference between the probability values of the two news events with the highest and second highest probability values is compared with a preset probability threshold to obtain the classification result of the news event, including:
[0106] If the difference between the probability values of the two news events with the highest and second highest probability values is greater than the preset probability threshold, the news event with the highest probability value is output.
[0107] If the difference between the probability values of the two news events with the highest and second highest probability values is less than the preset probability threshold, the two news events with the highest and second highest probability values are output.
[0108] By comparing the difference between the probability values and the preset probability threshold, the certainty of the classification result can be determined according to the size of the difference. When the difference is large, the news event with the highest probability value is output, emphasizing certainty and importance; when the difference is small, the two news events with the highest and second highest probability values are output, providing more choices and information. This method can provide news event classification results with reliability and flexibility according to the credibility and difference of the probability distribution result.
[0109] The preset probability threshold is set to 0.1, and the final classification result of the news is artificial intelligence and information technology.
[0110] Embodiment 2:
[0111] As shown in Figure 4 , a news event classification system of the present embodiment includes:
[0112] Data acquisition module: acquire text data of news events; the text data includes direct data and indirect data;
[0113] Keyword extraction module: extract a plurality of keywords in the text data using a keyword extraction fusion model to obtain keyword groups;
[0114] Similarity calculation module: based on a multi-level keyword library, perform multi-stage similarity calculation on the keyword groups to obtain similarity calculation results; the multi-level keyword library includes a first-level keyword library, a second-level keyword library, and a third-level keyword library;
[0115] Probability distribution calculation module: normalize the similarity calculation results and process them using a multi-layer perception network model to obtain the probability distribution results of the news events;
[0116] Classification result output module: based on the probability distribution results, compare the difference between the probability values of the two news events with the highest and second highest probability values with a preset probability threshold to obtain the classification result of the news events.
[0117] Embodiment 3:
[0118] AsFigure 5 The embodiment shown provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the news event classification method of the above embodiment when executing the computer program.
[0119] Embodiment 4:
[0120] The embodiment provides a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the news event classification method of the above embodiment.
[0121] The technical solutions in the above embodiments make full use of direct data and indirect data, keyword extraction fusion models, multi-level keyword libraries, normalization processing and multi-layer perception network models, and the classification strategy based on the probability value difference and the preset probability threshold. These technical features are mutually fused, so that the classification of news events is more accurate, comprehensive and flexible, and the effect of news event processing is improved.
[0122] In the above embodiments provided in the present application, it should be understood that the disclosed methods, systems, devices and media can be implemented in other ways. The above described method, system, device and medium embodiments are only illustrative, for example, the division of modules or units is only a logical function division, and actual implementation can have another division way. Each functional unit can be integrated in one processing unit, or each unit can be physically present, or two or more units can be integrated in one unit.
[0123] The units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solutions. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
[0124] The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc.
[0125] The above examples are only used to illustrate the technical solutions of the present application, but not limit the same; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. A method of classifying a news event, characterized by, The application comprises the following steps: S1, obtaining text data of a news event; the text data comprises direct data and indirect data; S2, extracting several keywords in the text data by using a keyword extraction fusion model to obtain a keyword group; S3, performing multi-stage similarity calculation on the keyword group based on a multi-level keyword library to obtain a similarity calculation result; the multi-level keyword library comprises a first-level keyword library, a second-level keyword library and a third-level keyword library; wherein the multi-stage similarity calculation on the keyword group comprises: S31, calculate the similarity k of the keyword phrase and each vocabulary in the third-level vocabulary library ij : S32, calculate the similarity K of the keyword phrase and each vocabulary in the secondary vocabulary library i : S33, calculating the similarity M between the keyword group and each vocabulary in the first-level keyword library; Wherein, c(m, n) represents the keyword phrase vector A m and the word vector B in the three-level lexicon n distance, m, n respectively represent the keyword phrase vector A m , the word vector B n index, N represents the number of words in the three-level lexicon, j represents the word index in the two-level lexicon, J represents the number of words in the two-level lexicon, i represents the word index in the one-level lexicon, and I represents the number of words in the one-level lexicon. S4, performing normalization processing on the similarity calculation result and obtaining a probability distribution result of the news event based on a multi-layer perception network model; S5, comparing the probability value difference between the two news events with the highest and second-highest probability values with a preset probability threshold based on the probability distribution result to obtain a classification result of the news event.
2. The method of claim 1, wherein, In S1, direct data is obtained by using an API interface provided by a news website and a data provider; indirect data is obtained by accessing a news website or a social media platform by using a crawler model.
3. The method of claim 1, wherein, In S2, the keyword extraction fusion model is used to extract keywords in the text data, which comprises the following steps: S21, preprocessing the text data, including word segmentation and stop word removal; S22, calculating the term frequency (TF) and inverse document frequency (IDF) of each term based on the preprocessed text data to obtain the TF-IDF value of each term; S23, taking each term as a node of a graph, calculating the relationship between terms by using a co-occurrence matrix, and constructing a text graph; S24, sorting the text graph by using a TextRank algorithm to determine the TextRank score of each term; S25, weighting and fusing the TF-IDF value and the TextRank score to obtain the comprehensive score of each term; S26, sorting based on the comprehensive score of each term and extracting several keywords by using a preset score threshold to obtain a keyword group.
4. The method of claim 3, wherein, In S3, the multi-level keyword library is updated based on an intelligent recommendation mechanism.
5. The method of claim 1, wherein, In S4, the news event classification model comprises an input layer, a hidden layer and an output layer; the hidden layer is provided with a ReLU activation function; and the output layer is provided with a Softmax activation function.
6. The method of claim 1, wherein, In S5, the probability value difference between the two news events with the highest and second-highest probability values is compared with the preset probability threshold based on the probability distribution result to obtain the classification result of the news event, which comprises the following steps: If the probability value difference between the two news events with the highest and second-highest probability values is greater than the preset probability threshold, the news event with the highest probability value is outputted; If the probability value difference between the two news events with the highest and second-highest probability values is less than the preset probability threshold, the two news events with the highest and second-highest probability values are outputted.
7. A news event classification system characterized by, The application comprises the following steps: A data acquisition module is configured to obtain text data of a news event; the text data comprises direct data and indirect data; A keyword extraction module is configured to extract several keywords in the text data by using a keyword extraction fusion model to obtain a keyword group; The similarity calculation module is configured to perform multi-stage similarity calculation on the keyword phrase based on a multi-level keyword library to obtain a similarity calculation result. The multi-level keyword library includes a first-level keyword library, a second-level keyword library, and a third-level keyword library. The multi-stage similarity calculation on the keyword phrase includes: S31, calculate the similarity k of the keyword phrase and each vocabulary in the third-level vocabulary library ij : S32, calculate the similarity K of the keyword phrase and each vocabulary in the secondary vocabulary library i : S33, calculating the similarity M between the keyword phrase and each word in the first-level keyword library; wherein c(m, n) represents the keyword phrase vector A m and the word vector B in the three-level lexicon n distance, m and n represent the keyword phrase vector A m and the word vector B n respectively, N represents the number of words in the three-level lexicon, j represents the word index in the two-level lexicon, J represents the number of words in the two-level lexicon, i represents the word index in the one-level lexicon, and I represents the number of words in the one-level lexicon. The probability distribution calculation module is configured to normalize the similarity calculation result and obtain a probability distribution result of the news event based on a multi-layer perception network model. The classification result output module is configured to compare a difference between the two highest probability values of the news event and a preset probability threshold based on the probability distribution result to obtain a classification result of the news event.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the news event classification method in any one of claims 1 to 6.
9. A computer readable storage medium, the storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the news event classification method in any one of claims 1 to 6.
Citation Information
Patent Citations
Classification method based on big data
CN108052556A
Complaint text processing method and device, computer equipment and storage medium
CN110377731A