A method for determining a text category, related apparatus, and device

Abstract

Embodiments of the present application disclose a method, related device, and equipment for determining text categories in natural language processing, which are used to improve the accuracy of determining text categories, thereby facilitating text classification or query. The method of the embodiments of the present application includes: obtaining the text to be classified, determining the topic words from the text to be classified, where the topic words are used to represent the topic type of the text to be classified, determining K domain words in the text to be classified according to P domain dictionaries, the K domain words are derived from the P domain dictionaries, and the domain words are used to represent the domain type to which the text to be classified belongs. Each domain dictionary corresponds to a text category. Determining the probabilities that the text to be classified belongs to P text categories respectively according to the K domain words and the topic words, and determining the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of artificial intelligence, and in particular, to a method for determining text categories, related devices, and equipment. Background Art

[0002] With the rapid development of information, the daily office work of government departments at all levels has been inseparable from the application of computer systems. Moreover, with the proposal of new governance and service concepts such as one-stop handling of people's livelihood services and one-network interconnection of government services, it is conducive to accelerating the digital development of the government.

[0003] However, in the process of people's livelihood services and social governance, a large amount of government data will be generated, such as data on the handling of people's livelihood matters, official document texts, digital services, etc. If these data cannot be better mined and analyzed, it will be difficult to realize and accelerate the intelligence of the government affairs industry, thereby reducing the convenience of people and government staff in handling service matters.

[0004] Among them, automatic classification of official documents is an inevitable way to realize the digital transformation of government affairs and the online handling of people's livelihood services. Therefore, in order to strengthen and accelerate official document work and ensure the smooth progress of official document work, generally, an electronic official document classification method based on templates is used for official document classification.

[0005] However, this method relies on manually given rules and templates. When building an expert knowledge base and official document field rules, it requires a large amount of time and labor costs. At the same time, due to the limitations of the rules and the free format of official document texts, the built rules often have a reduced generalization ability after a certain period of time, and the lack of generality leads to many official documents being unable to be accurately classified, and the iteration cycle is long and not flexible enough. Summary of the Invention

[0006] Embodiments of the present application provide a method for determining text categories, related devices, and equipment. By mining the topic words and domain words in the text to be classified, and since the topic words can reflect the topic type of the text to be classified, and the domain words reflect the domain type of the text to be classified, the dimension of the reference information for determining the text category can be increased. Furthermore, by using the topic words and domain words to determine the probability of the text category to which the text to be classified can belong, the text topic trend of the text to be classified can be accurately and intuitively reflected by the probability. Thus, the target text category of the text to be classified can be determined by using the probability of the text category, improving the accuracy of determining the target text category, and without the need to manually build rules, nor being restricted by rules and text formats, which can save costs and improve the efficiency of determining the target text category.

[0007] On the one hand, embodiments of the present application provide a method for determining text categories, including:

[0008] Obtain the text to be classified, where the text to be classified includes multiple words;

[0009] Determine a topic word from the text to be classified, where the topic word is used to represent the topic type of the text to be classified;

[0010] Determine K domain words in the text to be classified according to P domain dictionaries, where the K domain words are from the P domain dictionaries, and the domain words are used to represent the domain type to which the text to be classified belongs. Each domain dictionary corresponds to a text category, and both P and K are integers greater than or equal to 1;

[0011] Determine the probabilities that the text to be classified belongs to P text categories respectively according to the K domain words and the topic word;

[0012] Determine the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to P text categories respectively.

[0013] On the other hand, the present application provides a device for determining text categories, including:

[0014] An obtaining unit, configured to obtain the text to be classified, where the text to be classified includes multiple words;

[0015] A determining unit, configured to determine a topic word from the text to be classified, where the topic word is used to represent the topic type of the text to be classified;

[0016] The determining unit is further configured to determine K domain words in the text to be classified according to P domain dictionaries, where the K domain words are from the P domain dictionaries, and the domain words are used to represent the domain type to which the text to be classified belongs. Each domain dictionary corresponds to a text category, and both P and K are integers greater than or equal to 1;

[0017] The determining unit is further configured to determine the probabilities that the text to be classified belongs to P text categories respectively according to the K domain words and the topic word;

[0018] The determining unit is further configured to determine the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to P text categories respectively.

[0019] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the determining unit may specifically be configured to:

[0020] Obtain the first feature occurrence probability of the topic word in the text set according to the first probability mapping relationship, where the first probability mapping relationship includes the correspondence between the topic word feature and the first feature occurrence probability, and the topic word has a correspondence with the topic word feature;

[0021] Determine P groups of candidate domain terms among the K domain terms according to P domain dictionaries, where each group of candidate domain terms includes at least 1 domain term;

[0022] Obtain the P second feature occurrence probabilities of the P groups of candidate domain terms in the text set according to the second probability mapping relationship, where the second probability mapping relationship includes the correspondence between the local text features and the second feature occurrence probabilities, and the local text features have a correspondence with the candidate domain terms;

[0023] Obtain the probabilities of P text categories according to the first feature occurrence probability, the P second feature occurrence probabilities, the classification probabilities, and the Bayesian model, where the classification probabilities are from N preset text categories, each classification probability corresponds to a text category, and N is an integer greater than 1.

[0024] In a possible design, in an implementation manner on the other hand of the embodiment of the present application, the determining unit may specifically be used for:

[0025] Determine the first mutual information between the classification probability and the first feature occurrence probability, and the P second mutual informations between the classification probability and the P second feature occurrence probabilities;

[0026] Sum the first mutual information with each of the P second mutual informations to obtain P candidate mutual informations, where the P candidate mutual informations have a correspondence with the probabilities of the P text categories;

[0027] Determine the target mutual information that meets the preset mutual information threshold from the P candidate mutual informations, and determine the probability of the text category corresponding to the target mutual information as the target probability;

[0028] Determine the text category corresponding to the target probability as the target category.

[0029] In a possible design, in an implementation manner on the other hand of the embodiment of the present application, the determining unit may specifically be used for:

[0030] If P is equal to 1, determine the text category corresponding to the probability of the P text categories as the target text category;

[0031] If P is greater than 1, determine the probability that meets the preset probability condition from the probabilities of the P text categories to obtain the target text category.

[0032] In a possible design, in an implementation manner on the other hand of the embodiment of the present application, the determining unit may specifically be used for:

[0033] According to the preset category threshold, determine M candidate text category probabilities from the probabilities of the P text categories, where M is an integer greater than or equal to 1;

[0034] When M is equal to 1, the text category corresponding to the probability of the text category to be selected is determined as the target text category;

[0035] When M is greater than 1, the maximum text category probability is determined from the probabilities of the text categories to be selected, and the text category corresponding to the maximum text category probability is determined as the target text category.

[0036] In a possible design, in an implementation manner on the other hand of the embodiment of the present application, the determining unit may specifically be used for:

[0037] Obtain the feature models respectively corresponding to P domain dictionaries to obtain P category feature models;

[0038] Use the P category feature models to perform category feature extraction on the text to be classified respectively to obtain P text category features;

[0039] Based on the entity recognition model, perform entity recognition on the text to be classified to obtain Z entity words, where Z is an integer greater than or equal to 1;

[0040] Concatenate the P text category features with K domain words and Z entity words respectively to obtain P text local features.

[0041] In a possible design, in an implementation manner on the other hand of the embodiment of the present application,

[0042] The obtaining unit is further used to obtain a sample set, where the sample set includes N sample subsets, each sample subset includes at least two category samples, and each sample subset corresponds to a text category;

[0043] The processing unit is used to, for each sample subset, preprocess each category sample in at least two category samples respectively to obtain at least two sample part-of-speech sequences, where each sample part-of-speech sequence includes a sample word and the part of speech of the sample word;

[0044] The processing unit is further used to calculate the weight of each sample part-of-speech sequence to obtain the importance weight value of each sample part-of-speech sequence;

[0045] The determining unit is further used to determine a target weight value from the importance weight values according to a preset importance threshold, and determine the sample word corresponding to the target weight value as the target domain word;

[0046] The processing unit is further used to add the target domain word to the domain dictionary corresponding to the text category.

[0047] In a possible design, in an implementation manner on the other hand of the embodiment of the present application, the processing unit may specifically be used for:

[0048] Stop word filtering is performed on each category of samples to obtain the text to be processed, and the text to be processed includes at least two sentences to be processed;

[0049] Word segmentation is performed on each of at least two sentences to be processed to obtain at least two words to be processed;

[0050] Part-of-speech tagging is performed on each of at least two words to be processed to obtain the sample part-of-speech sequence.

[0051] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the determining unit may specifically be used for:

[0052] Vector processing is performed on the text to be classified to obtain at least two word vectors, where one word vector corresponds to one word;

[0053] The at least two word vectors are input into a topic model for topic distribution processing to obtain the topic distribution vector of the text to be classified;

[0054] The distances between each of the at least two word vectors and the topic distribution vector are calculated respectively to obtain X distances, where X is an integer greater than 1;

[0055] Candidate distances that meet a preset distance threshold are determined from the X distances, and the word vectors corresponding to the candidate distances are determined as topic-related word vectors;

[0056] The words corresponding to the topic-related word vectors are determined as topic words.

[0057] In a possible design, in an implementation manner of another aspect of the embodiments of the present application,

[0058] The determining unit is further configured to, if the number of topic words is one, determine the topic-related word vector corresponding to the topic word as the topic word feature;

[0059] The processing unit is further configured to, if the number of topic words is greater than one, perform vector integration processing on the topic-related word vectors corresponding to the topic words to obtain the topic word feature.

[0060] In a possible design, in an implementation manner of another aspect of the embodiments of the present application,

[0061] The determining unit is further configured to determine the target domain dictionary and the target sample subset corresponding to the target text category;

[0062] The processing unit is further configured to add the text to be classified to the target sample subset to update the target domain dictionary.

[0063] In a possible design, in an implementation manner of another aspect of the embodiments of the present application,

[0064] The determining unit is further configured to determine a target associated department corresponding to the target text category;

[0065] The processing unit is further configured to push the text to be classified to the terminal device corresponding to the target associated department, so that the terminal device displays the text to be classified.

[0066] Another aspect of the present application provides a computer device, including: a memory, a processor, and a bus system;

[0067] The memory is used to store program codes;

[0068] The processor is configured to execute the method for determining the text category described in any of the above aspects according to the instructions in the program code;

[0069] The bus system is used to connect the memory and the processor, so that the memory and the processor can communicate with each other.

[0070] Another aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions run on a computer, the computer is enabled to execute the methods in the above aspects.

[0071] One aspect of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for determining the text category provided in any of the above aspects.

[0072] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0073] First, determine the topic words that can be used to represent the topic type of the text to be classified from the text containing multiple words to be classified, and mine the domain words existing in the text to be classified. Then, determine the probabilities that the text to be classified belongs to P text categories according to the domain words and the topic words, and determine the target text category corresponding to the text to be classified from these P text categories based on the probabilities that the text to be classified belongs to P text categories. Through the above method, it is realized that by mining the topic words and domain words in the text to be classified, and since the topic words can reflect the topic type of the text to be classified, and the domain words reflect the domain type of the text to be classified, the dimension of the reference information for determining the text category can be increased. Furthermore, by using the topic words and domain words to determine the probabilities of the text categories that the text to be classified can belong to, the text topic trend of the text to be classified can be accurately and intuitively reflected by the probabilities, so that the target text category of the text to be classified can be determined by using the probabilities of the text categories, improving the accuracy of determining the target text category, and without the need to manually construct rules, nor being restricted by the rules and text formats, which can save costs and improve the efficiency of determining the target text category. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 is a schematic structural diagram of text processing in an embodiment of the present application;

[0075] Figure 2 is a schematic diagram of an embodiment of a method for determining a text category in an embodiment of the present application;

[0076] Figure 3 is another schematic diagram of an embodiment of a method for determining a text category in an embodiment of the present application;

[0077] Figure 4 is another schematic diagram of an embodiment of a method for determining a text category in an embodiment of the present application;

[0078] Figure 5 is another schematic diagram of an embodiment of a method for determining a text category in an embodiment of the present application;

[0079] Figure 6 is another schematic diagram of an embodiment of a method for determining a text category in an embodiment of the present application;

[0080] Figure 7 is another schematic diagram of an embodiment of a method for determining a text category in an embodiment of the present application;

[0081] Figure 8 is another schematic diagram of an embodiment of a method for determining a text category in an embodiment of the present application;

[0082] Figure 9It is a schematic diagram of another embodiment of the method for determining the text category in the embodiment of the present application;

[0083] Figure 10 It is a schematic diagram of another embodiment of the method for determining the text category in the embodiment of the present application;

[0084] Figure 11 It is a schematic diagram of another embodiment of the method for determining the text category in the embodiment of the present application;

[0085] Figure 12 It is a schematic diagram of another embodiment of the method for determining the text category in the embodiment of the present application;

[0086] Figure 13 It is a schematic diagram of another embodiment of the method for determining the text category in the embodiment of the present application;

[0087] Figure 14 It is a schematic diagram of an interface of the method for determining the text category in the embodiment of the present application;

[0088] Figure 15 It is a schematic diagram of an embodiment of the device for determining the text category in the embodiment of the present application;

[0089] Figure 16 It is a schematic diagram of an embodiment of the computer device in the embodiment of the present application. Detailed implementation manners

[0090] The embodiments of the present application provide a method for determining a text category, a related device, and a device, which are used to mine the topic words and domain words in the text to be classified. Since the topic words can reflect the topic type of the text to be classified, and the domain words reflect the domain type of the text to be classified, the dimension of the reference information for determining the text category can be increased. Furthermore, by using the topic words and domain words to determine the probability that the text to be classified belongs to a certain text category, the text topic trend of the text to be classified can be accurately and intuitively reflected by the probability. Thus, the target text category of the text to be classified can be determined by using the probability of the text category, improving the accuracy of determining the target text category, and without the need to manually construct rules, nor being restricted by rules and text formats, which can save costs and improve the efficiency of determining the target text category.

[0091] In the description and claims of the present invention and the above-mentioned drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0092] With the rapid development of information, the daily work of government departments at all levels, various enterprises, and ordinary people cannot do without the application of computer systems. Therefore, with the development of computers, artificial intelligence (AI) and databases have gradually penetrated into all aspects of people's lives. Artificial intelligence has extensive practical significance in aspects such as text translation, intelligent question answering, and sentiment analysis. The emergence of artificial intelligence has also greatly facilitated people's work and life. Among them, artificial intelligence is the theory, method, technology, and application system that uses machines controlled by mathematical computers or digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning, and decision-making.

[0093] Artificial intelligence is an interdisciplinary subject, involving a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. In natural language processing technology, artificial intelligence can be used to process text and reasonably interpret the words in the text. And in machine learning / deep learning technology, artificial intelligence can be used to process text and count or interpret the word frequency or importance of words in the text. The method for determining text categories provided in the embodiments of the present application can be implemented through natural language processing technology and machine learning technology.

[0094] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can enable effective communication between humans and computers in natural language. Natural language processing is a science that combines linguistics, computer science, and mathematics. Therefore, the research in this field will involve natural language, that is, the language people use in daily life, so it has some close connections with linguistics. Natural language processing technologies usually include text processing, semantic understanding, machine translation, robot question answering, pointing maps and other technologies.

[0095] Machine learning (ML) is an interdisciplinary subject involving multiple fields, such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0096] Databases have extensive practical significance in aspects such as text storage, updating, and data query. The use of databases has also greatly facilitated people's work and life. Among them, a database can be simply regarded as an electronic filing cabinet - a place to store electronic files. Users can perform operations such as adding, querying, updating, and deleting data in the files. The so-called "database" is a data set stored together in a certain way, shared by multiple users, with as little redundancy as possible, and independent of application programs.

[0097] A database management system (English: Database Management System, abbreviated as DBMS) is a computer software system designed to manage databases. It generally has basic functions such as storage, interception, security guarantee, and backup. Database management systems can be classified according to the database models they support, such as relational, XML (Extensible Markup Language); or according to the types of computers they support, such as server clusters, mobile phones; or according to the query languages they use, such as SQL (Structured Query Language), XQuery; or according to the key points of performance impulse, such as maximum scale, highest running speed; or other classification methods. No matter which classification method is used, some DBMS can cross categories. For example, they can support multiple query languages at the same time.

[0098] It should be understood that the method for determining the text category provided in this application can be applied to the field of artificial intelligence and is used in scenarios for text classification by determining the text category. As an example, for instance, by determining the category of official documents for official document texts, the classification of official document texts can be completed according to the official document category; as another example, for instance, by determining and classifying the news category for news texts, the classification of news texts can be completed according to the news category; as another example, for instance, by determining the story category for story texts, the classification of story texts can be completed according to the story category; as another example, for instance, by determining the email category for email texts, the classification and sorting of emails in the mailbox can be completed according to the email category. In the above various scenarios, in order to achieve text classification, the traditional method for determining the text category mainly performs text classification through manually given rules and templates, which not only consumes a large amount of human and time costs, but is also easily interfered by the limitations of the rules and the free format of official document texts, resulting in low accuracy of text classification. Moreover, due to the long iteration cycle and lack of flexibility, the efficiency of text classification is low.

[0099] To solve the above problems, this application proposes a method for determining the text category, which is applied to Figure 1 the text processing system shown. Please refer to Figure 1 , Figure 1 which is a schematic architecture diagram of the text processing system in an embodiment of this application. As shown in Figure 1 , the server obtains the text to be classified provided by the terminal device, and determines the topic words that can be used to represent the topic type of the text to be classified from the text to be classified containing multiple words. Then, by mining the domain words existing in the text to be classified, and then, according to the domain words and the topic words, determining the probabilities that the text to be classified belongs to P text categories respectively, and based on the probabilities that the text to be classified belongs to P text categories respectively, determining the target text category corresponding to the text to be classified from these P text categories. Through the above method, it realizes the mining of the topic words and domain words in the text to be classified. And because the topic words can reflect the topic type of the text to be classified, and the domain words reflect the domain type of the text to be classified, it can increase the dimension of the reference information for determining the text category. Furthermore, by using the topic words and domain words to determine the probabilities of the text categories that the text to be classified can belong to, it can accurately and intuitively reflect the text topic trend of the text to be classified through the probabilities, so that the target text category of the text to be classified can be determined by using the probabilities of the text categories, improving the accuracy of determining the target text category, and without the need to manually construct rules, nor being restricted by the rules and text format, it can save costs and improve the efficiency of determining the target text category.

[0100] To solve the above problems, the present application proposes a method for determining text categories. This method is generally executed by a server or a terminal device. Correspondingly, the device for determining text categories is generally set in the server or the terminal device.

[0101] It can be understood that, as the method for determining text categories, related devices, and apparatuses disclosed in the present application, multiple servers / terminal devices can form a blockchain, and the servers / terminal devices are nodes on the blockchain. In practical applications, data sharing between nodes may be required in the blockchain. Each node may store a text collection and polysemous words to be recognized.

[0102] The following will introduce the method for determining text categories in the present application. Please refer to Figure 2 , an embodiment of the method for determining text categories in the embodiments of the present application includes:

[0103] In step S101, obtain the text to be classified, where the text to be classified includes multiple words;

[0104] In this embodiment, in order to accurately classify the obtained text data and then accurately push these text data to professional departments for processing according to the classification results, so as to strengthen and speed up the text data processing work and ensure the smooth progress of the text data processing work, therefore, these text data can be used as the text to be classified. Among them, the text to be classified can specifically be official document texts, news texts, historical textbook texts, magazine texts, etc., or other forms of texts, which are not specifically limited here.

[0105] Furthermore, since the text content can specifically be one or more sentences, or one or more paragraphs, or an article, etc. composed of multiple words or phrases, which are not specifically limited here, therefore, the text to be classified contains multiple words.

[0106] For example, assume that a text to be classified is "Xiaoli wants to consult question A. When can it be implemented". This text to be classified contains words such as "Xiaoli / wants / to consult / A / question / what / time / can / be implemented".

[0107] In step S102, determine the topic words from the text to be classified, where the topic words are used to represent the topic type of the text to be classified;

[0108] In this embodiment, for a text to be classified, such as an article, if one wants to determine the type of this article by the words in the article, and if there are many sports-related words in the article, such as basketball, football, etc., the topic type of this article can be sports. Then, the computer device may classify this article as a sports article. Therefore, the text category of the text to be classified can be reflected by the topic type of the text, and a text can contain multiple topics, and each word in the text is generated by one of the topics. Therefore, this embodiment can obtain topic words that can be used to represent the topic type of the text to be classified as reference information for determining the text to be classified, so as to accurately determine the text category of the text to be classified through the topic words.

[0109] Specifically, after obtaining the text to be classified, since the text to be classified contains multiple words, and these multiple words may come from different topics. Therefore, in order to accurately and quickly obtain topic words that can be used to represent the topic type of the text to be classified, so that the text category of the text to be classified can be accurately determined through the topic words in the subsequent process, this embodiment can use a Probability Latent Semantic Analysis (PLSA) topic model, a Latent Dirichlet Allocation (LDA) topic generation model, a label latent dirichlet allocation (L-LDA) topic model, or other topic models to obtain topic words, and no specific limitation is made here.

[0110] In step S103, K domain words in the text to be classified are determined according to P domain dictionaries, where the K domain words are from the P domain dictionaries, and the domain words are used to represent the domain type to which the text to be classified belongs. Each domain dictionary corresponds to a text category, and both P and K are integers greater than or equal to 1.

[0111] In step 104, the probabilities that the text to be classified belongs to P text categories are determined according to the K domain words and the topic words.

[0112] In this embodiment, the domain dictionary expresses the key information of a professional domain through refined and concise words. In this embodiment, it is used to express the key information of different text categories. The essence of the content of the domain dictionary is the "information extraction" of text, that is, domain-related words are extracted from a large number of unordered texts and classified according to different domains. Therefore, one text category corresponds to one domain dictionary, and each domain dictionary contains multiple domain words. Domain words can be used to represent the domain information to which the text belongs and are words that can be used to reflect the domain type to which the text belongs. Since the text category of the text to be classified can be reflected by the domain type of the text, in this embodiment, the domain words that can be used to represent the domain type of the text to be classified can be obtained as the reference information for determining the text to be classified, so as to accurately determine the text category of the text to be classified through the domain words.

[0113] For example, a text category is the municipal education bureau category. The corresponding domain dictionary for this category is the education domain. Domain words that can be used to represent the key information of the professional domain of this education domain include education, teacher, student, enrollment, college entrance examination, scholarship, etc. Suppose a text to be classified is "Xiaoli wants to consult about the enrollment issue. When can it be implemented". According to the education domain dictionary, the domain word "enrollment" can be determined from this text to be classified, and then the domain type to which this text to be classified may belong, that is, the education domain, can be reflected through the domain word "enrollment".

[0114] Furthermore, the text category is set according to actual application requirements and can be used to indicate operations such as text classification or text push. For example, suppose the text to be classified is an official document text. Then the text category of the official document text can be set according to the type of official document processing department, such as the municipal education bureau category, the municipal health bureau category, the municipal administration bureau category, etc. Or, suppose the text to be classified is a news text. Then the text category of the news text can be set according to the type of news processing department, such as the entertainment information category, the financial information category, the agricultural information category, etc. It can also be other categories, which are not specifically limited here.

[0115] Further, since probability can be used to represent the likelihood of something happening, the probability of the text category to which the text to be classified belongs in this embodiment is used to represent the likelihood of the text to be classified belonging to this text category. And since the topic words can reflect the topic type of the text to be classified, and the domain words can reflect the domain type of the text to be classified, therefore, by using the topic words and domain words to determine the probability of the text category to which the text to be classified can belong in this embodiment, the dimension of the reference information for determining the text category can be increased. Furthermore, the text topic trend of the text to be classified can be accurately and intuitively reflected by the probability of the text category to which the text to be classified belongs, so that the target text category of the text to be classified can be determined by using the probability of the text category subsequently, thereby improving the accuracy of determining the target text category.

[0116] Specifically, after obtaining the topic words of the text to be classified, the computer device can respectively use the domain dictionary corresponding to each text category to hit the words in the text to be classified for domain words. When a word is hit in a text category, such as the domain dictionary corresponding to the Municipal Education Bureau category hits a word in the text to be classified, the hit word is recorded as the domain word corresponding to the Municipal Education Bureau category. Until the domain dictionary hits are completed, S domain words hit by the domain dictionary can be obtained, where S is an integer greater than or equal to 1 and less than or equal to K. Then, the S domain words hit are used to calculate the probability with the obtained topic words, and the probability of belonging to the Municipal Education Bureau category can be obtained as 0.56. Similarly, the probabilities of the text to be classified belonging to other text categories can be obtained, such as the probability of belonging to the Municipal Health Bureau category being 0.54, or the probability of belonging to the Municipal Administration Bureau category being 0.59, etc., so that subsequent processing is carried out according to the probability of 0.56 of the text to be classified belonging to the Municipal Education Bureau category, the probability of 0.54 of belonging to the Municipal Health Bureau category, or the probability of 0.59 of belonging to the Municipal Administration Bureau category, etc., to accurately determine the target text category of the text to be classified.

[0117] In step S105, according to the probabilities of the text to be classified belonging to P text categories respectively, the target text category corresponding to the text to be classified is determined from the P text categories.

[0118] In this embodiment, the target text category is the text category of the text to be classified. After obtaining the probabilities of the text to be classified belonging to P text categories respectively, the maximum probability can be obtained by comparing the obtained probabilities or performing a weighted operation, or the target probability can be obtained through other probability processing methods, which is not specifically limited here. Then, the text category corresponding to the target probability is determined as the target text category of the text to be classified.

[0119] Specifically, after obtaining the probabilities that the text to be classified belongs to P text categories respectively, the obtained probabilities can be compared pairwise to obtain the maximum probability, and this maximum probability is used as the target probability. Then, the text category corresponding to this target probability is determined as the target text category of the text to be classified. There is no need to manually construct rules for text classification, nor is it restricted by rules and text formats, which can save labor costs and time costs, thereby improving the efficiency of determining the target text category.

[0120] In an embodiment of the present application, a method for determining a text category is provided. Through the above method, it is realized that by mining the theme words and domain words in the text to be classified, and since the theme words can reflect the theme type of the text to be classified, and the domain words reflect the domain type of the text to be classified, it can increase the dimension of the reference information for determining the text category. Furthermore, by using the theme words and domain words to determine the probabilities that the text to be classified can belong to the text categories, the text theme trend of the text to be classified can be accurately and intuitively reflected by the probabilities. Thus, the target text category of the text to be classified can be determined by using the probabilities of the text categories, improving the accuracy of determining the target text category, and there is no need to manually construct rules, nor is it restricted by rules and text formats, which can save costs and improve the efficiency of determining the target text category.

[0121] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the method for determining a text category provided by the embodiment of the present application, as Figure 3 shown, determining the probabilities that the text to be classified belongs to P text categories according to K domain words and theme words includes:

[0122] In step S301, obtain the first feature occurrence probability of the theme word in the text set according to the first probability mapping relationship, where the first probability mapping relationship includes the correspondence between the theme word feature and the first feature occurrence probability, and there is a correspondence between the theme word and the theme word feature;

[0123] In step S302, according to P domain dictionaries, determine P groups of candidate domain words among the K domain words, and each group of candidate domain words includes at least 1 domain word;

[0124] In step S303, obtain P second feature occurrence probabilities of the P groups of candidate domain words in the text set according to the second probability mapping relationship, where the second probability mapping relationship includes the correspondence between the text local feature and the second feature occurrence probability, and there is a correspondence between the text local feature and the candidate domain word;

[0125] In step S304, according to the first feature occurrence probability, the occurrence probabilities of P second features, the classification probability, and the Bayesian model, the probabilities of P text categories are obtained, where the classification probability is derived from N preset text categories, each classification probability corresponds to a text category, and N is an integer greater than 1.

[0126] In this embodiment, the first feature occurrence probability refers to the probability that a topic word appears in the sample subset. After obtaining the topic word of the text to be classified, the first feature occurrence probability of the topic word of the text to be classified can be determined according to the first probability mapping relationship between the topic word feature corresponding to the topic word and the feature occurrence probability.

[0127] Specifically, the computer device can accurately count the probability that the topic word feature appears in the sample subset by identifying the topic word feature of the topic word, that is, the probability that the topic word appears in the sample subset. Therefore, the probability that the topic word feature appears in the sample subset can be determined as the first feature occurrence probability of the topic word.

[0128] For example, assume that the topic word is "Chinese". There are 100 texts in the sample subset corresponding to the Municipal Education Bureau category, among which 18 texts have the topic word "Chinese". Then the first feature occurrence probability of the topic word "Chinese" can be 18÷100 = 0.18.

[0129] Furthermore, since the K domain words are from P domain dictionaries, in order to accurately obtain the probabilities corresponding to each of the P text categories, the obtained K domain words can be divided according to the P domain dictionaries. The K domain words can be divided into P groups of candidate domain words, such that each group of candidate domain words contains at least 1 domain word.

[0130] Furthermore, the second feature occurrence probability refers to the probability that a group of candidate domain words appears in the sample subset. After obtaining the domain words of the text to be classified, the second feature occurrence probability of the candidate domain words of the text to be classified can be determined according to the second probability mapping relationship between the text local feature corresponding to the candidate domain words and the feature occurrence probability.

[0131] Among them, the text local feature is used to represent the word features of the candidate domain words and the words with strong relevance to the candidate domain words.

[0132] Specifically, the computer device can accurately count the probability that the text local feature appears in the sample subset by identifying the text local feature of the candidate domain words, which can be used to approximately represent the probability that the candidate domain words appear in the sample subset. Therefore, the probability that the text local feature appears in the sample subset can be determined as the second feature occurrence probability of the candidate domain words.

[0133] For example, assume that the candidate domain term is "fine". There are 80 text samples corresponding to the city urban management committee category. Among them, 20 texts contain words such as "fine" or "with fine". Then, the probability of the second feature of the candidate domain term "fine" appearing can be approximately expressed as 20÷80 = 0.25.

[0134] Furthermore, since the topic word feature and the text local feature are independent of each other and satisfy the feature independence assumption of Bayes, therefore, in this embodiment, a Bayes model can be used to predict the probabilities of the P text categories to which the text to be classified belongs.

[0135] Among them, the computer device can also obtain N classification probabilities corresponding to the text category, and each classification probability corresponds to a text category. Therefore, when there are N text categories, the probabilities that each text to be classified may belong to each text category are equal, that is, 1 / N. That is to say, the values of the N classification probabilities corresponding to the text to be classified are the same.

[0136] For example, assume that there are 3 text categories in an official document text library, namely the city education bureau category, the city administration bureau category, and the city urban management committee category. If the city education bureau category is denoted as event B1, the city administration bureau category is denoted as event B2, and the city urban management committee category is denoted as event B3, then, P(B1) = P(B2) = P(B3) = 1 / 3.

[0137] Furthermore, according to the definition of Bayes, the probability of the topic word feature appearing can be denoted as P(q1), and the probability of the text local feature appearing can be denoted as P(q2). Then, according to the formula the probability P(Q) of a text category can be obtained. Then, the probability P(Q) of the text category and the classification probability 1 / N can be input into the Bayes model to obtain the probabilities of the P text categories.

[0138] Furthermore, after obtaining the probabilities of the P text categories, to determine the target text category corresponding to the text to be classified from the P text categories, Bayes can be used , first, according to the binary classification characteristic of Bayes, determine the category of the text to be classified.

[0139] For example, when P is 1, if P(B1|Q) tends to 1, then it can be determined that the target text category of the text to be classified is the city education bureau category. If P(B1|Q) tends to 0, then it can be determined that the target text category of the text to be classified is not the city education bureau category.

[0140] For example, when P is greater than 1, if both P(B1|Q) and P(B2|Q) tend to 1, then from P(B1|Q) and P(B2|Q), determine the event with the maximum conditional probability, and determine the text category corresponding to this event as the target text category of the text to be classified. For example, if P(B1|Q) > P(B2|Q), then the target text category can be determined as the Municipal Education Bureau category.

[0141] Among them, P(A|Q) represents the probability that the text to be classified belongs to a certain text category A under the condition that the input feature is event Q. P(Q|A) represents the probability that the input feature is event Q under the condition that the text to be classified is a certain text category A. P(A) represents the probability that the text to be classified belongs to a certain text category. In the above example, P(A) = P(B1) = P(B2) = P(B3).

[0142] Among them, according to the Bayesian definition, there is , so, it can be obtained that .

[0143] Among them, P(q i , A) represents the probability that q i and A appear simultaneously. q i includes the topic word feature q1 and the text local feature q2, and A represents a certain text category to which the text to be classified belongs.

[0144] Specifically, after obtaining the topic word and K domain words, according to the first probability mapping relationship and the second probability mapping relationship, the topic word feature corresponding to the topic word and the first feature occurrence probability, as well as the P text local features and the P second feature occurrence probabilities corresponding to the K domain words can be obtained respectively. Then, by constructing a Bayesian model, the topic word feature of the text to be classified and the P text local features of the text to be classified are used as input features to obtain P input features, and the classification probability and the first feature occurrence probability are respectively input into the Bayesian model together with the P second feature occurrence probabilities to obtain the probabilities of P text categories. Since the topic word feature and the text local feature are independent of each other, the input features satisfy the independence assumption of the Bayesian formula and meet the operation conditions of the Bayesian formula. Therefore, the advantages of the Bayesian formula can be fully utilized to accurately obtain the probabilities of the P text categories corresponding to the P input features, thereby improving the accuracy of determining the target text category to a certain extent.

[0145] Optionally, on the basis of the above Figure 3 corresponding embodiment, in another optional embodiment of the method for determining the text category provided by the embodiment of the present application, such as Figure 4As shown, determining the target text category corresponding to the text to be classified from P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively includes:

[0146] In step S401, determine the first mutual information between the classification probability and the occurrence probability of the first feature, and the P second mutual informations between the classification probability and the occurrence probabilities of the P second features;

[0147] In step S402, add the first mutual information to each of the P second mutual informations respectively to obtain P candidate mutual informations, where the P candidate mutual informations have a corresponding relationship with the probabilities of the P text categories;

[0148] In step S403, determine the target mutual information that meets the preset mutual information threshold from the P candidate mutual informations, and determine the probability of the text category corresponding to the target mutual information as the target probability;

[0149] In step S404, determine the text category corresponding to the target probability as the target category.

[0150] In this embodiment, after obtaining the probabilities of the P text categories based on Bayes, this embodiment can determine the target probability from the P text probabilities according to the binary classification characteristic of Bayes, so as to determine the target text category through the target probability. However, in order to be able to further quickly and accurately obtain the target text category, this embodiment can also determine the text category of the text to be classified by means of mutual information.

[0151] Among them, mutual information is a useful information measure in information theory, which can be used for the correlation between two event sets and for measuring the statistical independence relationship between a certain word and a category. It can be understood that if a certain feature word belongs to this category, then their mutual information quantity is the largest. Therefore, this embodiment can use the first mutual information to represent the correlation between the classification probability and the occurrence probability of the first feature, and use the second mutual information to represent the correlation between the classification probability and the occurrence probability of the second feature. Then, obtain the target mutual information through the first mutual information and the second mutual information, so as to determine the target text category according to the target mutual information. Since mutual information does not need to make any assumptions about the nature of the relationship between words, features, and categories and is not restricted by hypothesis conditions, the target text category can be accurately and quickly obtained through mutual information, thereby improving the efficiency of obtaining the target text category to a certain extent.

[0152] Specifically, according to the obtained first feature occurrence probability and the occurrence probabilities of P second features, the first mutual information between the classification probability and the first feature occurrence probability can be calculated, and the P second mutual informations between the classification probability and the occurrence probabilities of the P second features can be calculated respectively. Then, the first mutual information is summed with the P second mutual informations respectively to obtain the P candidate mutual informations that can be obtained. Then, the P candidate mutual informations are compared with a preset mutual information threshold respectively. Then, from the mutual informations greater than or equal to the preset mutual information threshold, the maximum mutual information is selected and determined as the target mutual information. Furthermore, the probability of the text category corresponding to the target mutual information is determined as the target probability, and the text category corresponding to the target probability is determined as the target category.

[0153] For example, assume that there are 3 text categories in an official document text library, namely the Municipal Education Bureau category, the Municipal Administration Bureau category, and the Municipal Urban Management Commission category. If the Municipal Education Bureau category is denoted as event B1, the Municipal Administration Bureau category is denoted as event B2, the Municipal Urban Management Commission category is denoted as event B3, the first feature occurrence probability is denoted as P(q1), and the second feature occurrence probability is denoted as P(q2). A candidate mutual information can be determined based on the following formula:

[0154]

[0155] where, represents the first mutual information, represents the second mutual information, represents the candidate mutual information. For example, according to the above three text categories, three candidate mutual informations can be obtained, 、 and .

[0156] It should be noted that since the target text category for determining the text to be classified is a certain text category to which the text to be classified belongs, generally, 、 and there is only one target mutual information greater than or equal to the preset threshold. However, if two target mutual informations are both greater than or equal to the preset threshold, then, it can be determined that the text category corresponding to the larger target mutual information among the two target mutual informations is the text category of the text to be classified.

[0157] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the method for determining a text category provided by the embodiments of the present application, as Figure 5 shown, according to the probabilities that the text to be classified belongs to P text categories respectively, the target text category corresponding to the text to be classified is determined from the P text categories, including:

[0158] In step S501, if P is equal to 1, then the text category corresponding to the probability of P text categories is determined as the target text category;

[0159] In step S502, if P is greater than 1, then the probability that meets the preset probability condition is determined from the probabilities of P text categories to obtain the target text category.

[0160] In this embodiment, the preset probability condition is set according to actual application requirements, and is used to screen the target probability from the probabilities of P text categories. Specifically, it can be expressed as being greater than or equal to a preset threshold, or obtaining the probability with the largest value, or a combination of the above two conditions, or other conditional forms, which are not specifically limited here.

[0161] Specifically, after obtaining the probabilities that the text to be classified belongs to P text categories respectively, when P is equal to 1, it can be understood that there is only one probability of the text category to which the text to be classified belongs. Then, the text category corresponding to this one probability of the text category can be determined as the target text category of the text to be classified. Or, when P is greater than 1, it can be understood that there is more than one probability of the text category to which the text to be classified belongs. Then, by comparing the probabilities of these P text categories pairwise, the probability with the largest value can be obtained, and this probability with the largest value can be used as the target probability. Then, the text category corresponding to the target probability can be determined as the target text category of the text to be classified. By selecting the corresponding method for determining the target text category according to the value of P, not only can the target text category be obtained quickly and accurately, improving the accuracy and efficiency of determining the target text category, but also the method for determining the target text category can be flexibly selected according to the actual application needs, which also improves the flexibility of the technical solution.

[0162] For example, assume that P is equal to 1. Then the probability of P text categories is expressed as the probability of one text category. Assume that the text category corresponding to the probability of this text category is the Municipal Urban Management Committee category. Then, the text category corresponding to the probability of the text category, which is the Municipal Urban Management Committee category, can be determined as the target text category of the text to be classified.

[0163] For example, assume that P is greater than 1, and the probability of the Municipal Education Bureau category is denoted as P(B1)=0.56, the probability of the Municipal Administration Bureau category is denoted as P(B2)=0.69, and the probability of the Municipal Urban Management Committee category is denoted as P(B3)=0.52. Then, by comparison, it can be obtained that P(B2)>P(B1)>P(B3). Furthermore, the probability with the largest value can be obtained as P(B2). Therefore, P(B2) can be used as the target probability. Then, the text category corresponding to the target probability P(B2), that is, the Municipal Administration Bureau category, can be determined as the target text category of the text to be classified.

[0164] Optionally, in the above Figure 5Based on the corresponding embodiments, in another optional embodiment of the method for determining the text category provided by the embodiments of the present application, as Figure 6 shown, determining the probabilities that meet the preset probability conditions from the probabilities of P text categories to obtain the target text category, including:

[0165] In step S601, according to the preset category threshold, determine the probabilities of M candidate text categories from the probabilities of P text categories, where M is an integer greater than or equal to 1;

[0166] In step S602, when M is equal to 1, determine the text category corresponding to the probability of the candidate text category as the target text category;

[0167] In step S603, when M is greater than 1, determine the maximum text category probability from the probabilities of the candidate text categories, and determine the text category corresponding to the maximum text category probability as the target text category.

[0168] In this embodiment, the preset category threshold is set according to the actual application requirements and is used to screen the probabilities that meet the requirements of the preset category threshold from the probabilities of P text categories. The probabilities of the candidate text categories are the probabilities that meet the requirements of the preset category threshold among the probabilities of P text categories and are used to further screen the target probability that can accurately reflect the theme trend of the text to be classified according to the probabilities of the candidate text categories, so that the text category corresponding to the target probability can be determined as the target text category.

[0169] Specifically, when P is greater than 1, probabilities that meet a preset probability condition are determined from the probabilities of P text categories. Specifically, it can be done by comparing the probabilities of the P text categories with a preset category threshold respectively, that is, obtaining the probabilities greater than or equal to the preset category threshold to get M candidate text category probabilities, where M is an integer greater than or equal to 1 and less than or equal to P. Then, when M equals 1, it can be understood that there is only one candidate text category probability obtained, indicating that this candidate text category probability can best reflect the theme trend of the text to be classified. Then, the text category corresponding to the candidate text category probability can be determined as the target text category. Or, when M is greater than 1 and less than or equal to P, by comparing the M candidate text category probabilities pairwise, the text category probability with the largest value is obtained, and this text category probability with the largest value is used as the target probability. Then, the text category corresponding to the target probability can be determined as the target text category of the text to be classified. It can quickly screen out from the probabilities of P text categories the candidate text category probabilities that can better reflect the theme trend of the text to be classified relative to the probabilities of the P text categories according to the preset category threshold under the condition that there are multiple probabilities of text categories, and determine the target category by obtaining the target probability with the largest value among the candidate text category probabilities, so as to accurately obtain the target probability through simple constraint conditions, thereby accurately obtaining the target text category and improving the accuracy of determining the target text category.

[0170] For example, assume that P is greater than 1, the preset category threshold is set to 0.54, the probability of the Municipal Education Bureau category is denoted as P(B1)=0.56, the probability of the Municipal Administration Bureau category is denoted as P(B2)=0.69, and the probability of the Municipal Urban Management Commission category is denoted as P(B3)=0.52. Then, P(B1), P(B2), and P(B3) can be compared with the preset category threshold 0.54 respectively, and the M candidate text category probabilities greater than or equal to the preset category threshold are P(B1) and P(B2). Then, since M is greater than 1, by comparing P(B1) and P(B2), it can be obtained that P(B2)>P(B1), that is, the text category probability with the largest value is P(B2). Therefore, P(B2) can be used as the target probability. Then, the text category corresponding to the target probability P(B2), that is, the Municipal Administration Bureau category, can be determined as the target text category of the text to be classified.

[0171] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the method for determining a text category provided by an embodiment of the present application, as Figure 7 shown, the method further includes:

[0172] In step S701, obtain the feature models respectively corresponding to P domain dictionaries to get P category feature models;

[0173] In step S702, P category feature models are used to extract category features from the text to be classified respectively, obtaining P text category features;

[0174] In step S703, entity recognition is performed on the text to be classified based on the entity recognition model, obtaining Z entity words, where Z is an integer greater than or equal to 1;

[0175] In step S704, the P text category features are respectively concatenated with K domain words and Z entity words to obtain P text local features.

[0176] In this embodiment, since there are many words in the text to be classified, and each word can be regarded as a feature of the text, then the text to be classified can be regarded as a feature set composed of many features. Also, since each feature can be represented by a vector, the feature set can be regarded as a high-dimensional feature vector. In order to more accurately predict the target text category of the text to be classified, this embodiment can mine text local feature vectors from the text to be classified, that is, some of the most representative features can be selected from the high-dimensional feature vector, so as to achieve the reduction of the dimension while also being able to well predict the target text category of the text to be classified.

[0177] Therefore, in this embodiment, P domain dictionaries corresponding to P category feature models can be obtained first, and the P category feature models are used to extract category features from the text to be classified respectively, obtaining P text category features. Among them, the category feature model is an optimized model obtained by training the feature extraction model based on the sample set. The feature extraction model can be a word2vec model, or a feature numerical calculation class (CountVectorizer) model. In addition, it can also be other feature extraction models, such as the glove model, or the term frequency-inverse document frequency (TF-IDF) model. No specific limitation is made here.

[0178] It should be noted that there is no necessary sequence between step S702 and step S703. Step S702 can be executed first, step S703 can be executed first, or step S702 and step S703 can be executed simultaneously, as long as it is executed after step S101. No specific limitation is made here.

[0179] Further, after obtaining the text to be classified, in order to more quickly and accurately obtain the local text features in the text to be classified, in this embodiment, the understanding of the text to be classified can be transformed into word processing to improve the efficiency of text processing. Therefore, the text to be classified can be segmented to obtain a segmentation result. Since the segmentation result obtained after segmenting the text to be classified will include place names, organization names, etc., and usually most of the named entities can express the text theme, therefore, in this embodiment, the named entities in the segmentation result can be recognized to obtain the entity words in the text to be classified that can be used to express the text theme. Specifically, the named entity recognition (Named Entities Recognition, NER) can be used to recognize the named entities in the segmentation result, and other recognition methods can also be used, such as the hidden Markov model, the maximum entropy model, the support vector machine or the conditional random field, etc., which are not specifically limited here. Among them, named entity recognition refers to recognizing the entities with specific meanings in the text to be classified, mainly including person names, place names, organization names, proper nouns, etc.

[0180] Specifically, after obtaining the text to be classified, in order to more accurately determine the target text category of the text to be classified, in this embodiment, by mining the local text features in the text to be classified, the dimension of the reference information for determining the target text category is increased. In this embodiment, P category feature models corresponding to the text to be classified can be obtained according to P domain dictionaries, and the text to be classified can be input into the P category feature models respectively for feature extraction, and the text category features that can be used to express the most text category in the text to be classified can be obtained, that is, P text category features. At the same time, the text to be classified can also be subjected to entity recognition to obtain the named entities in the text to be classified that can express the text theme, that is, Z entity words. Then, the P text category features can be respectively concatenated with K domain words and Z entity words, such as performing dimension concatenation processing, to obtain P local text features, so that the target text category of the text to be classified can be predicted more accurately through the mined local text features.

[0181] Optionally, on the basis of the above Figure 2 corresponding embodiment, in another optional embodiment of the method for determining the text category provided by the embodiment of the present application, as Figure 8 shown, the method further includes:

[0182] In step S801, a sample set is obtained, where the sample set includes N sample subsets, each sample subset includes at least two category samples, and each sample subset corresponds to a text category;

[0183] In step S802, for each sample subset, preprocess each category sample in at least two category samples respectively to obtain at least two sample part-of-speech sequences, where each sample part-of-speech sequence includes a sample word and the part-of-speech of the sample word;

[0184] In step S803, calculate the weight for each sample part-of-speech sequence to obtain the importance weight value of each sample part-of-speech sequence;

[0185] In step S804, determine the target weight value from the importance weight values according to a preset importance threshold, and determine the sample word corresponding to the target weight value as the target domain word;

[0186] In step S805, add the target domain word to the domain dictionary corresponding to the text category.

[0187] In this embodiment, before obtaining the sample set, a sample set can be constructed first according to a preset classification criterion. The sample set is a text classification standard database constructed by classifying the pre-collected text data. Each sample set contains at least one sample subset, each sample subset corresponds to a text category, each sample subset contains at least one category sample, and each category sample contains at least one word.

[0188] For example, construct a document classification standard database <classification department, document text>, where the classification department is the text category, and each document text corresponds to a classification department, that is, a document text corresponds to a text category. Among them, first collect a batch of case samples through the document classification handling department, which can be used as the training set samples for subsequent model training. Then, construct sample subsets according to each classification department, that is, the text category, and use multiple relevant case information collected by each classification department as category samples and add them to the corresponding sample subsets. Integrate multiple sample subsets to obtain the sample set, that is, obtain the <classification department, document text> document classification standard database. The classification department can specifically be represented by the category id marks shown in Table 1, such as: 0, 1, or 2, etc., or the category name labels shown in Table 2, such as the Municipal Education Bureau, the Municipal Administration Bureau, or the Municipal Urban Management Committee, etc. It can also adopt other forms of expression, and no specific limitation is made here. Further, the category samples also include: positive samples and negative samples, where the positive sample corresponding to each text category is the case information of this department, and the negative sample is the case information of other departments.

[0189] Among them, Table 1 constructs a <classification department, document text> document classification standard database with the category id mark as the text category:

[0190] Text category (classification department ID) Sample (document case) 0 Category sample 1 1 Category sample 2 1 Category sample 3 1 Category sample 4 2 Category sample 5 2 Category sample 6 ... ... Q Category sample n

[0191] Table 1

[0192] Among them, Table 2 constructs a document classification standard database of <classification department, official document text> with the category name label as the text category:

[0193]

[0194] Table 2

[0195] Furthermore, after obtaining the text to be classified, in order to quickly and accurately index the domain words in the text to be classified and better mine the local features of the text from the text to be classified, this embodiment can be implemented by obtaining a sample set, extracting domain words for each sample subset of the sample set, and constructing a domain dictionary corresponding to each sample subset through the extracted domain words.

[0196] Furthermore, since the sample set, such as the official document classification standard database, contains category samples of different classification departments, therefore, this embodiment can respectively perform category feature statistics and new word discovery on each sample subset to obtain the domain dictionary of each sample subset, so that the domain dictionary can be used to mine the domain words and local features of the text to be classified subsequently, thereby improving the accuracy of determining the target text category to a certain extent.

[0197] Specifically, after obtaining the sample set, each category sample in each sample subset can be preprocessed first, such as punctuation removal processing, date removal processing or part-of-speech tagging, etc. Other processing methods can also be used, which are not specifically limited here, and at least two sample part-of-speech sequences containing sample words and the part-of-speech of sample words can be obtained.

[0198] Furthermore, since there is an importance relationship between parts of speech. For example, the importance of proper nouns is greater than that of common nouns, and the importance of nouns is greater than that of verbs. Therefore, in order to better sort the importance of the sample sequences and thus more accurately extract domain words, this embodiment uses the method of lexical classification to achieve the importance ranking of words, that is, by calculating the weights of each sample part-of-speech sequence, the importance weight value of each sample part-of-speech sequence is obtained. Among them, the importance weight value of each sample part-of-speech sequence can specifically be to divide the sample words into three types of words, such as candidate domain words, common words and irrelevant words.

[0199] Among them, since the acquisition of the domain dictionary mainly compares the frequency differences between different categories, the word type can be defined by the frequency difference. That is, in a sample set, if a word appears with a relatively high frequency in a certain sample subset and a relatively low frequency in other sample subsets, then this word can be defined as a candidate domain word for this sample subset; if the frequency difference of a word changes relatively little among each sample subset, then this word is defined as an irrelevant word in the sample set; if the frequency difference of a word changes relatively greatly among each sample subset, then this word is defined as a common word in the sample set.

[0200] Furthermore, obtain the importance weights of each sample part-of-speech sequence. Specifically, it can be obtained according to a preset weight calculation formula, or by using the Term Frequency-Inverse Document Frequency (TF-IDF) algorithm, or by using other weight calculation formulas, and no specific limitation is made here.

[0201] Among them, in this embodiment, the importance weights of each sample part-of-speech sequence are obtained by using a preset weight calculation formula. Specifically, it can be by setting the importance of irrelevant words to 0 or a value close to 0, and according to the preset weight formula calculate the importance weights of common words, where represents a common word, is the common word the frequency of appearance in all category samples of the sample set, is the maximum text frequency among the text frequencies of each common word in all category samples of the sample set, that is, the number of category texts in which each common word appears in each sample subset, divided by the total number of category samples in this sample set, to obtain k text frequencies, and obtain the maximum probability from the k text frequencies, is the common word the document frequency of appearance in all category samples of the sample set, that is, the common word the number of category texts in which it appears in each category sample, divided by the total number of category samples in this sample set. And according to the preset weight formula calculate the importance weights of candidate domain words, where is the maximum importance weight among the importance weights corresponding to all common words, is a candidate domain word, is the candidate domain word the frequency of appearance in all category samples of the sample set, is the maximum category text frequency among the category text frequencies of each candidate domain word in all sample categories of the sample set, that is, the number of category texts in which each candidate domain word appears in each sample subset, divided by the total number of category samples in the sample set, to obtain l text frequencies, and the maximum probability is obtained from the l text frequencies. is the candidate domain word in the category text frequencies that appear in all category samples of the sample set, that is, the candidate domain word the number of category texts that appear in each category sample, divided by the total number of category samples in the sample set.

[0202] It should be noted that if a word is a proper noun in terms of its part of speech, the importance weight of this word will be multiplied by a coefficient to obtain the proper noun weighting ; if a word is a noun in terms of its part of speech, the importance weight of this word will be multiplied by a coefficient to obtain the noun part of speech weighting , therefore, if the part of speech of a candidate domain word is a noun, its corresponding comprehensive weight formula is: , where includes , .

[0203] Furthermore, after obtaining the importance weights of each sample part of speech sequence, the importance weights of the candidate domain words corresponding to each sample subset can be first selected, and then the importance weights of the candidate domain words corresponding to each sample subset are respectively compared with the preset importance threshold corresponding to the sample subset, and the weights greater than or equal to the preset importance threshold are selected and determined as the target weights, and the candidate domain words corresponding to the target weights are determined as the target domain words corresponding to the sample subset. Then, the extracted target domain words are added to the domain dictionary corresponding to the sample subset to complete the construction of the domain dictionary corresponding to each sample subset.

[0204] Optionally, on the basis of the above Figure 8 corresponding embodiment, in another optional embodiment of the text category determination method provided by the embodiments of the present application, as Figure 9 shown, preprocessing is respectively performed on each category sample in at least two category samples to obtain at least two sample part of speech sequences, including:

[0205] In step S901, stop word filtering is performed on each category sample to obtain the text to be processed, and the text to be processed includes at least two sentences to be processed;

[0206] In step S902, word segmentation is performed on each of the at least two sentences to be processed to obtain at least two words to be processed;

[0207] In step S903, part-of-speech tagging is performed on each of at least two words to be processed, obtaining a sample part-of-speech sequence.

[0208] In this embodiment, after obtaining the sample set, preprocessing is performed on each category sample in each sample subset, obtaining at least two sample part-of-speech sequences, which can bring great convenience to information extraction, text classification, information retrieval or machine translation of the text. Specifically, this embodiment can perform stop-word filtering on each category sample, that is, first perform sentence splitting on the text to be classified according to the sentence splitting delimiter, obtaining at least two sentences, and then filter out useless information such as date and time, name, email, mobile phone number, etc. in each sentence, obtaining at least two sentences to be processed. Among them, stop-word filtering of the sentences can be based on a stop-word library, and filtering of date and time, name, email, mobile phone number, etc. in the sentences can be based on regular expressions.

[0209] Furthermore, since the purpose of word segmentation is to divide a continuous sentence into individual word units, the understanding of the text is thus transformed into the processing of sentences, thereby improving the efficiency of text processing. Further, word segmentation is performed on each of at least two sentences to be processed. Specifically, it can be performed based on a dictionary method, a statistical method or a rule-based method, or other word segmentation algorithms can also be used, and no specific limitation is made here. This embodiment can use a general bigram model to segment the sentences, obtaining at least two words to be processed.

[0210] Further, part-of-speech tagging is a process of determining the grammatical category of each word to be processed in a sentence to be processed, determining its part of speech, and tagging it. Part-of-speech tagging can mark the part of speech of the word to be processed as a noun, verb, adjective, etc. For the obtained words to be processed, part-of-speech tagging can be specifically performed through a Hidden Markov model (HMM), or a rule-based method. Other part-of-speech tagging methods can also be used, such as a statistics-based method, a deep learning-based method, etc. There is no specific limitation here. In this embodiment, a general Hidden Markov model is used to perform part-of-speech tagging on the words to be processed. Specifically, the possible parts of speech of each word to be processed are regarded as the hidden states of the HMM. Then, the transition probability P(t2|t1) defined on each hidden state is used between part of speech t1 and part of speech t2. Among them, the transition probability matrix describes the probability value of the part of speech transferring from one state to another state, and by defining the emission probability P(w|t), which represents the probability that the word to be processed w belongs to the part of speech t. Furthermore, an observation sequence of the words to be processed is obtained through the Hidden Markov model. Then, the Viterbi algorithm can be used to calculate the observation sequence to obtain the optimal word segmentation sequence, that is, the sample part-of-speech sequence.

[0211] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the method for determining the text category provided by the embodiments of the present application, as Figure 10 shown, determining the topic words from the text to be classified includes:

[0212] In step S1001, vector processing is performed on the text to be classified to obtain at least two word vectors, where one word vector corresponds to one word;

[0213] In step S1002, the at least two word vectors are input into a topic model for topic distribution processing to obtain a topic distribution vector of the text to be classified;

[0214] In step S1003, the distance between each of the at least two word vectors and the topic distribution vector is calculated respectively to obtain X distances, where X is an integer greater than 1;

[0215] In step S1004, candidate distances that meet a preset distance threshold are determined from the X distances, and the word vectors corresponding to the candidate distances are determined as topic-related word vectors;

[0216] In step S1005, the words corresponding to the topic-related word vectors are determined as topic words.

[0217] In this embodiment, after obtaining the text to be classified, the text category of the text to be classified can be reflected by the theme type of the text, and the theme words can be used to represent the theme type of the text to be classified. Therefore, this embodiment can increase the dimension of the reference information of the target text category by mining the theme words in the text to be classified, so that the text category of the text to be classified can be determined by the theme words subsequently, and the accuracy of determining the target category can be improved.

[0218] Specifically, after obtaining the text to be classified, the text to be classified can be segmented to obtain at least two words. Furthermore, a word vector extraction model can be used to convert the words in natural language into the form of word vectors, which is convenient for machines to identify and process, and can determine the relationship between words according to the relationship between word vectors. Among them, the word vector extraction model can specifically be a word2vec model, or a glove model. In addition, it can also be other word vector training models, such as the Bidirectional Encoder Representations from Transformers (BERT) model, or the ELMo model, etc., which are not specifically limited here.

[0219] Furthermore, after obtaining at least two word vectors, each word vector can be input into a topic model for topic distribution prediction to obtain the topic distribution vector of the text to be classified. Among them, the topic model can specifically be a latent dirichlet allocation (LDA) topic model, or other topic models, such as a probabilistic latent semantic analysis (PLSA) topic model, or a label latent dirichlet allocation (L-LDA) topic model, which are not specifically limited here.

[0220] Among them, this embodiment uses the LDA model to perform topic distribution prediction on the word vectors to obtain the topic distribution vector of the text to be classified. Among them, LDA is a statistical model used to find a set of potential topics with specific probabilities from a document collection, thereby forming a three-layer structure of words - topics - texts. That is to say, the topic can be characterized by the distribution of words, thereby reflecting the topic distribution of the text. The core formula of LDA is as follows:

[0221]

[0222] Among them, topic represents the topic, p represents the probability, represents the text in the words, is the implicit theme in the text. In the embodiments of the present application, the Expectation Maximization algorithm can be used to train the LDA model, which can converge after fewer iterations to obtain the text-topic distribution matrix, that is, the topic distribution vector.

[0223] Further, after obtaining the topic distribution vector of the text to be classified, the distance between each of at least two word vectors and the topic distribution vector can be calculated respectively, and the topic-related word vectors can be determined according to the principle of the distance from near to far. Among them, calculating the distance between each word vector and the topic distribution vector can specifically be using the cosine distance calculation formula, or the sine distance calculation formula. In addition, it can also be other forms of distance, such as the Euclidean distance calculation formula. In practical applications, which form to use to calculate the distance between each word vector and the topic distribution vector can be selected according to actual application requirements, and no specific limitation is made here.

[0224] It can be understood that since the closer the distance between the word vector and the topic distribution vector, the closer the word vector is to the theme of the text to be classified. Therefore, the principle of from near to far is used to select the topic-related word vectors. The number of topic-related word vectors is set according to the actual application requirements, and it can be related to the number of words contained in the text to be classified. Optionally, if the number of words contained in the text to be classified is more, the number of determined topic-related word vectors is also more. For example, for an official document text of about a thousand words, 10 topic-related word vectors can be determined. Then, after determining the topic-related word vectors, it can be understood that the words corresponding to the topic-related word vectors can be expressed as words closer to the theme of the text to be classified. Therefore, in this embodiment, the words corresponding to the topic-related word vectors can be used as the topic words, so that the topic words can be used as key reference information in the subsequent process to determine the target text category of the text to be classified, thereby improving the accuracy of determining the target text category.

[0225] Optionally, on the basis of the above Figure 10 corresponding embodiments, in another optional embodiment of the method for determining the text category provided by the embodiments of the present application, as Figure 11 shown, the method further includes:

[0226] In step S1101, if the number of topic words is one, the topic-related word vector corresponding to the topic word is determined as the topic word feature;

[0227] In step S1102, if the number of topic words is greater than one, vector integration processing is performed on the topic-related word vectors corresponding to the topic words to obtain the topic word feature.

[0228] In this embodiment, since the number of theme-related word vectors obtained can be one or multiple, the theme words obtained can also be one or multiple. Also, since each theme word can be regarded as a feature of the text, this feature can also be one or multiple. Therefore, there can be multiple ways to determine the theme word features according to the theme words in this embodiment, enabling subsequent selection of different ways to determine the theme word features based on the different numbers of theme words, which can improve the flexibility of the technical solution, and thus enable the extraction of theme word features from the text to be classified to more accurately predict the target text category of the text to be classified.

[0229] Specifically, when there is only one theme word, it can be understood that there is also only one theme-related word vector, so there is only one feature that can be used to represent the theme of the text to be classified. And since each feature can be represented in the form of a vector, the theme-related word vector can be determined as the theme word feature. Or, when the number of theme-related word vectors is greater than one, it can be understood that the number of theme-related word vectors is also greater than one, that is, there are more than one features that can be used to represent the theme of the text to be classified. Then, in this embodiment, multiple theme-related word vectors can be integrated to obtain the theme word feature.

[0230] Furthermore, the integration process of multiple theme-related word vectors in this embodiment can specifically be to calculate the average value of multiple theme-related word vectors and use the obtained result as the theme word feature; or perform weighted summation on multiple theme-related word vectors and use the summation result as the theme word feature; or, it can be to splice the dimensions of each theme-related word vector to obtain a high-dimensional theme word feature, etc. There can also be other processing methods. For example, use the maximum value among multiple theme-related word vectors as the theme word feature. Among them, the method of integrating multiple theme-related word vectors can be selected according to the actual application needs and is not specifically limited here.

[0231] It can be understood that when performing weighted summation on multiple theme-related word vectors, the weights of each theme-related word vector can be obtained. Among them, the weight can be determined according to the distance between the theme-related word vector and the theme distribution vector. Specifically, it can be shown that the weight size of the theme-related word is negatively correlated with the distance. It can be understood that the closer the theme-related word is to the theme distribution vector, the greater the weight of the theme-related word vector.

[0232] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the method for determining the text category provided by the embodiment of the present application, as Figure 12 shown, the method further includes:

[0233] In step S1201, determine the target domain dictionary and the target sample subset corresponding to the target text category;

[0234] In step S1202, add the text to be classified to the target sample subset to update the target domain dictionary.

[0235] In this embodiment, the target domain dictionary refers to the dictionary of the same category as the target text category of the text to be classified, and the target sample subset refers to the sample subset of the same category as the target text category of the text to be classified.

[0236] Specifically, after obtaining the target text category of the text to be classified, the target domain dictionary and the target sample subset corresponding to the text to be classified can be determined through the target text category. Then, add the text to be classified to the target sample subset, and further use the text to be classified as a category sample for text processing to obtain the target domain words in the text to be classified and add them to the target domain dictionary to update the target domain dictionary. Among them, using the text to be classified as a category sample for text processing is similar to the method of obtaining domain words in steps S802 to S804, which will not be elaborated here. At the same time, the text to be classified can also be used as a training sample for training the target feature extraction model corresponding to the target domain dictionary to optimize the target feature extraction model. Similarly, in this embodiment, by iteratively updating the target domain dictionary with the obtained text to be classified, expanding the sample set, and optimizing the target feature extraction model, the accuracy and stability of determining the target text category can be improved.

[0237] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the text category determination method provided by the embodiment of the present application, as Figure 13 shown, the method further includes:

[0238] In step S1301, determine the target associated department corresponding to the target text category;

[0239] In step S1302, push the text to be classified to the terminal device corresponding to the target associated department so that the terminal device displays the text to be classified.

[0240] In this embodiment, the target associated department refers to the department whose processed text category is the same as the target text category of the text to be classified.

[0241] Specifically, after obtaining the target text category of the text to be classified, the target associated department with the same text category as the processed text category can be quickly indexed according to the target text category. Then, the text to be classified is pushed to the terminal device corresponding to the target associated department. Specifically, the text to be classified can be pushed to the terminal device in the form of an email, a document, or a data packet. It can also be pushed in other forms, which are not specifically limited here. After the terminal device receives the text to be classified, as Figure 14 shown, it can be displayed on the display interface of the terminal device, so as to facilitate the target associated department to process the obtained text to be classified in a timely manner, thereby improving the efficiency of processing the text to be classified.

[0242] The text category determination device in the present application will be described in detail below. Please refer to Figure 15 , Figure 15 which is a schematic diagram of an embodiment of the text category determination device in the embodiment of the present application. The text category determination device 20 includes:

[0243] An acquisition unit 201, configured to acquire the text to be classified, where the text to be classified includes a plurality of words;

[0244] A determination unit 202, configured to determine a topic word from the text to be classified, where the topic word is used to represent the topic type of the text to be classified;

[0245] The determination unit 202 is further configured to determine K domain words in the text to be classified according to P domain dictionaries, where the K domain words are from the P domain dictionaries, the domain words are used to represent the domain type to which the text to be classified belongs, each domain dictionary corresponds to a text category, and both P and K are integers greater than or equal to 1;

[0246] The determination unit 202 is further configured to determine the probabilities that the text to be classified belongs to P text categories respectively according to the K domain words and the topic word;

[0247] The determination unit 202 is further configured to determine the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively.

[0248] Optionally, on the basis of the above Figure 15 corresponding embodiment, in another embodiment of the text category determination device provided in the embodiment of the present application, the determination unit 202 may specifically be configured to:

[0249] Obtain the first feature occurrence probability of the topic word in the text set according to the first probability mapping relationship, where the first probability mapping relationship includes the correspondence between the topic word feature and the first feature occurrence probability, and the topic word has a correspondence with the topic word feature;

[0250] Determine P groups of candidate domain terms among the K domain terms according to P domain dictionaries, where each group of candidate domain terms includes at least 1 domain term;

[0251] Obtain the P second feature occurrence probabilities of the P groups of candidate domain terms in the text set according to the second probability mapping relationship, where the second probability mapping relationship includes the correspondence between the text local features and the second feature occurrence probabilities, and the text local features have a correspondence with the candidate domain terms;

[0252] Obtain the probabilities of P text categories according to the first feature occurrence probability, the P second feature occurrence probabilities, the classification probabilities, and the Bayesian model, where the classification probabilities are from N preset text categories, each classification probability corresponds to a text category, and N is an integer greater than 1.

[0253] Optionally, based on the above Figure 15 corresponding embodiment, in another embodiment of the text category determination device provided by the embodiments of the present application, the determination unit 202 may specifically be used for:

[0254] Determine the first mutual information between the classification probability and the first feature occurrence probability, and the P second mutual informations between the classification probability and the P second feature occurrence probabilities;

[0255] Sum the first mutual information with the P second mutual informations respectively to obtain P candidate mutual informations, where the P candidate mutual informations have a correspondence with the probabilities of the P text categories;

[0256] Determine the target mutual information that meets the preset mutual information threshold from the P candidate mutual informations, and determine the probability of the text category corresponding to the target mutual information as the target probability;

[0257] Determine the text category corresponding to the target probability as the target category.

[0258] Optionally, based on the above Figure 15 corresponding embodiment, in another embodiment of the text category determination device provided by the embodiments of the present application, the determination unit 202 may specifically be used for:

[0259] If P is equal to 1, determine the text category corresponding to the probabilities of the P text categories as the target text category;

[0260] If P is greater than 1, determine the probability that meets the preset probability condition from the probabilities of the P text categories to obtain the target text category.

[0261] Optionally, based on the above Figure 15 corresponding embodiment, in another embodiment of the text category determination device provided by the embodiments of the present application, the determination unit 202 may specifically be used for:

[0262] Determine the probabilities of M candidate text categories from the probabilities of P text categories according to a preset category threshold, where M is an integer greater than or equal to 1;

[0263] When M is equal to 1, determine the text category corresponding to the probability of the candidate text category as the target text category;

[0264] When M is greater than 1, determine the maximum text category probability from the probabilities of the candidate text categories, and determine the text category corresponding to the maximum text category probability as the target text category.

[0265] Optionally, based on the above Figure 15 On the basis of the corresponding embodiment, in another embodiment of the text category determination device provided by the embodiment of the present application, the determination unit 202 may specifically be used for:

[0266] Obtain the feature models corresponding to P domain dictionaries respectively to obtain P category feature models;

[0267] Use the P category feature models to extract category features from the text to be classified respectively to obtain P text category features;

[0268] Perform entity recognition on the text to be classified based on the entity recognition model to obtain Z entity words, where Z is an integer greater than or equal to 1;

[0269] Concatenate the P text category features with K domain words and Z entity words respectively to obtain P text local features.

[0270] Optionally, based on the above Figure 15 On the basis of the corresponding embodiment, in another embodiment of the text category determination device provided by the embodiment of the present application,

[0271] The acquisition unit 201 is further configured to acquire a sample set, where the sample set includes N sample subsets, each sample subset includes at least two category samples, and each sample subset corresponds to a text category;

[0272] The processing unit 203 is configured to, for each sample subset, preprocess each category sample in at least two category samples respectively to obtain at least two sample part-of-speech sequences, where each sample part-of-speech sequence includes a sample word and the part-of-speech of the sample word;

[0273] The processing unit 203 is further configured to calculate the weight of each sample part-of-speech sequence to obtain the importance weight value of each sample part-of-speech sequence;

[0274] The determining unit 202 is further configured to determine a target weight from the importance weights according to a preset importance threshold, and determine the sample words corresponding to the target weight as the target domain words;

[0275] The processing unit 203 is further configured to add the target domain words to the domain dictionary corresponding to the text category.

[0276] Optionally, based on the corresponding embodiment above Figure 15 In another embodiment of the text category determining device provided by the embodiment of the present application, the processing unit 203 may specifically be configured to:

[0277] Filter stop words from each category sample to obtain a text to be processed, where the text to be processed includes at least two sentences to be processed;

[0278] Perform word segmentation on each of the at least two sentences to be processed to obtain at least two words to be processed;

[0279] Perform part-of-speech tagging on each of the at least two words to be processed to obtain a sample part-of-speech sequence.

[0280] Optionally, based on the corresponding embodiment above Figure 15 In another embodiment of the text category determining device provided by the embodiment of the present application, the determining unit 202 may specifically be configured to:

[0281] Perform vector processing on the text to be classified to obtain at least two word vectors, where one word vector corresponds to one word;

[0282] Input the at least two word vectors into a topic model for topic distribution processing to obtain a topic distribution vector of the text to be classified;

[0283] Calculate the distance between each of the at least two word vectors and the topic distribution vector respectively to obtain X distances, where X is an integer greater than 1;

[0284] Determine candidate distances that meet a preset distance threshold from the X distances, and determine the word vectors corresponding to the candidate distances as topic-related word vectors;

[0285] Determine the words corresponding to the topic-related word vectors as topic words.

[0286] Optionally, based on the corresponding embodiment above Figure 15 In another embodiment of the text category determining device provided by the embodiment of the present application,

[0287] The determining unit 202 is further configured to, if the number of topic words is one, determine the topic-related word vector corresponding to the topic word as the topic word feature;

[0288] The processing unit 203 is further configured to, if the number of topic words is greater than one, perform vector integration processing on the topic-related word vectors corresponding to the topic words to obtain topic word features.

[0289] Optionally, based on the above Figure 15 corresponding embodiment, in another embodiment of the text category determination device provided by the embodiments of the present application,

[0290] The determination unit 202 is further configured to determine a target domain dictionary and a target sample subset corresponding to the target text category;

[0291] The processing unit 203 is further configured to add the text to be classified to the target sample subset to update the target domain dictionary.

[0292] Optionally, based on the above Figure 15 corresponding embodiment, in another embodiment of the text category determination device provided by the embodiments of the present application,

[0293] The determination unit is further configured to determine a target associated department corresponding to the target text category;

[0294] The processing unit is further configured to push the text to be classified to the terminal device corresponding to the target associated department, so that the terminal device displays the text to be classified.

[0295] On the other hand, the present application provides another schematic diagram of a computer device, as Figure 16 shown, Figure 16 is a schematic structural diagram of a computer device provided by an embodiment of the present application. The computer device 300 may vary greatly due to configuration or performance differences, and may include one or more central processing units (CPUs) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) for storing application programs 331 or data 332. Among them, the memory 320 and the storage media 330 may be transient storage or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the computer device 300. Further, the central processor 310 may be configured to communicate with the storage media 330 and execute a series of instruction operations in the storage media 330 on the computer device 300.

[0296] The computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 333, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM and so on.

[0297] The above computer device 300 is also used to execute the steps in the corresponding embodiments as Figures 2 to 13 shown.

[0298] Another aspect of the present application provides a computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are run on a computer, the computer is caused to execute the steps in the method described in the embodiments as Figures 2 to 13 shown.

[0299] Another aspect of the present application provides a computer program product containing instructions. When the computer program product is run on a computer or a processor, the computer or the processor is caused to execute the steps in the method described in the embodiments as Figures 2 to 13 shown.

[0300] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0301] In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0302] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0303] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0304] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

Claims

1. A method for determining a text category, characterized in that, including: obtaining the text to be classified, where the text to be classified includes a plurality of words; determining a topic word from the text to be classified, where the topic word is used to represent the topic type of the text to be classified; determining K domain words in the text to be classified according to P domain dictionaries, where the K domain words are from the P domain dictionaries, and the domain words are used to represent the domain type to which the text to be classified belongs. Each domain dictionary corresponds to a text category, and both P and K are integers greater than or equal to 1; obtaining the first feature occurrence probability of the topic word in the text set according to the first probability mapping relationship, where the first probability mapping relationship includes the correspondence between the topic word feature and the first feature occurrence probability, and the topic word has a correspondence with the topic word feature; determining P groups of candidate domain words from the K domain words according to the P domain dictionaries, where each group of candidate domain words includes at least 1 domain word; obtaining P second feature occurrence probabilities of the P groups of candidate domain words in the text set according to the second probability mapping relationship, where the second probability mapping relationship includes the correspondence between the text local feature and the second feature occurrence probability, and the text local feature has a correspondence with the candidate domain word; obtaining the probabilities of P text categories according to the first feature occurrence probability, the P second feature occurrence probabilities, the classification probability, and the Bayesian model, where the classification probability comes from N preset text categories, each classification probability corresponds to a text category, and N is an integer greater than 1; determining the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively.

2. The determination method according to claim 1, characterized in that The determining the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively includes: determining the first mutual information between the classification probability and the first feature occurrence probability, and P second mutual informations between the classification probability and the P second feature occurrence probabilities; adding the first mutual information to each of the P second mutual informations respectively to obtain P candidate mutual informations, where the P candidate mutual informations have a correspondence with the probabilities of the P text categories; determining a target mutual information that meets the preset mutual information threshold from the P candidate mutual informations, and determining the probability of the text category corresponding to the target mutual information as the target probability; determining the text category corresponding to the target probability as the target category.

3. The determination method according to claim 1, characterized in that The determining the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively includes: if P is equal to 1, determining the text category corresponding to the probability of the P text categories as the target text category; if P is greater than 1, determining a probability that meets the preset probability condition from the probabilities of the P text categories to obtain the target text category.

4. The determination method according to claim 3, characterized in that Determining the probability that meets a preset probability condition from the probabilities of the P text categories to obtain the target text category includes: Determining M candidate text category probabilities from the probabilities of the P text categories according to a preset category threshold, where M is an integer greater than or equal to 1; When M is equal to 1, determining the text category corresponding to the candidate text category probability as the target text category; When M is greater than 1, determining the maximum text category probability from the candidate text category probabilities, and determining the text category corresponding to the maximum text category probability as the target text category.

5. The determination method according to claim 1, characterized in that, Before obtaining the P second feature occurrence probabilities of the P groups of candidate domain words in the text set according to the second probability mapping relationship, the method further includes: Obtaining the feature models corresponding to the P domain dictionaries respectively to obtain P category feature models; Using the P category feature models to extract category features from the text to be classified respectively to obtain P text category features; Performing entity recognition on the text to be classified based on an entity recognition model to obtain Z entity words, where Z is an integer greater than or equal to 1; Performing feature splicing on the P text category features with the K domain words and the Z entity words respectively to obtain P text local features.

6. The determination method according to claim 1, characterized in that, Before determining the K domain words in the text to be classified according to the P domain dictionaries, the method further includes: Obtaining a sample set, where the sample set includes N sample subsets, each sample subset includes at least two category samples, and each sample subset corresponds to a text category; For each sample subset, preprocessing each of the at least two category samples in the sample subset to obtain at least two sample part-of-speech sequences, where each sample part-of-speech sequence includes a sample word and the part of speech of the sample word; Calculating the weights of each sample part-of-speech sequence to obtain the importance weights of each sample part-of-speech sequence; Determining a target weight from the importance weights according to a preset importance threshold, and determining the sample word corresponding to the target weight as the target domain word; Adding the target domain word to the domain dictionary corresponding to the text category.

7. The determination method according to claim 6, wherein The preprocessing each of the at least two category samples in the sample subset to obtain at least two sample part-of-speech sequences includes: Filtering stop words from each category sample to obtain a text to be processed, where the text to be processed includes at least two sentences to be processed; Performing word segmentation on each of the at least two sentences to be processed to obtain at least two words to be processed; Performing part-of-speech tagging on each of the at least two words to be processed to obtain a sample part-of-speech sequence.

8. The determination method according to claim 1, wherein Determining the topic word from the text to be classified includes: Performing vector processing on the text to be classified to obtain at least two word vectors, where one word vector corresponds to one word; Inputting the at least two word vectors into a topic model for topic distribution processing to obtain the topic distribution vector of the text to be classified; Calculate the distances between each of the at least two word vectors and the topic distribution vector respectively, obtaining X distances, where X is an integer greater than 1; Determine the candidate distances that meet a preset distance threshold from the X distances, and determine the word vectors corresponding to the candidate distances as the topic-related word vectors; Determine the words corresponding to the topic-related word vectors as the topic words.

9. The determination method according to claim 8, wherein After determining the topic words from the text to be classified, the method further includes: If the number of the topic words is one, determine the topic-related word vector corresponding to the topic word as the topic word feature; If the number of the topic words is greater than one, perform vector integration processing on the topic-related word vectors corresponding to the topic words to obtain the topic word feature.

10. The determination method according to claim 1, characterized in that After determining the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively, the method further includes: Determine the target domain dictionary and the target sample subset corresponding to the target text category; Add the text to be classified to the target sample subset to update the target domain dictionary.

11. The determination method according to claim 1, wherein After determining the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively, the method further includes: Determine the target associated department corresponding to the target text category; Push the text to be classified to the terminal device corresponding to the target associated department, so that the terminal device displays the text to be classified.

12. An apparatus for determining a text category, characterized in that, including: An acquisition unit, configured to acquire a text to be classified, where the text to be classified includes multiple words; A determination unit, configured to determine topic words from the text to be classified, where the topic words are used to represent the topic type of the text to be classified; The determination unit is further configured to, if there are K domain words in the text to be classified, determine the probabilities that the text to be classified belongs to P text categories according to the K domain words and the topic words, where the K domain words are from P domain dictionaries, each domain dictionary corresponds to a text category, and both P and K are integers greater than or equal to 1; The determination unit is further configured to determine the target text category corresponding to the text to be classified from the P text categories according to the probabilities that the text to be classified belongs to the P text categories respectively; The determining unit is specifically configured to: obtain the first feature occurrence probability of the topic word in the text set according to the first probability mapping relationship, where the first probability mapping relationship includes the correspondence between the topic word feature and the first feature occurrence probability, and the topic word has a correspondence with the topic word feature; determine P groups of candidate domain words among the K domain words according to the P domain dictionaries, where each group of candidate domain words includes at least one domain word; obtain the P second feature occurrence probabilities of the P groups of candidate domain words in the text set according to the second probability mapping relationship, where the second probability mapping relationship includes the correspondence between the text local feature and the second feature occurrence probability, and the text local feature has a correspondence with the candidate domain word; obtain the probabilities of P text categories according to the first feature occurrence probability, the P second feature occurrence probabilities, the classification probability, and the Bayesian model, where the classification probability is derived from N preset text categories, each classification probability corresponds to a text category, and N is an integer greater than 1.

13. The device according to claim 12, characterized in that The determining unit is specifically configured to: determine the first mutual information between the classification probability and the first feature occurrence probability, and P second mutual informations between the classification probability and the P second feature occurrence probabilities; add the first mutual information to each of the P second mutual informations respectively to obtain P candidate mutual informations, where the P candidate mutual informations have a correspondence with the probabilities of the P text categories; determine a target mutual information that meets a preset mutual information threshold from the P candidate mutual informations, and determine the probability of the text category corresponding to the target mutual information as the target probability; determine the text category corresponding to the target probability as the target category.

14. The device according to claim 12, wherein, The determining unit is specifically configured to: if P is equal to 1, determine the text category corresponding to the probability of the P text categories as the target text category; if P is greater than 1, determine a probability that meets a preset probability condition from the probabilities of the P text categories to obtain the target text category.

15. The device according to claim 14, characterized in that, The determining unit is specifically configured to: determine M candidate text category probabilities from the probabilities of the P text categories according to a preset category threshold, where M is an integer greater than or equal to 1; when M is equal to 1, determine the text category corresponding to the candidate text category probability as the target text category; when M is greater than 1, determine the maximum text category probability from the candidate text category probabilities, and determine the text category corresponding to the maximum text category probability as the target text category.

16. The device according to claim 12, characterized in that, The determining unit is specifically configured to: obtain the feature models respectively corresponding to the P domain dictionaries to obtain P category feature models; use the P category feature models to extract category features from the text to be classified respectively to obtain P text category features; perform entity recognition on the text to be classified based on the entity recognition model to obtain Z entity words, where Z is an integer greater than or equal to 1; The P text category features are respectively concatenated with the K domain words and the Z entity words to obtain P text local features.

17. The device according to claim 12, characterized in that, The apparatus further includes a processing unit; The obtaining unit is further configured to obtain a sample set, where the sample set includes N sample subsets, each sample subset includes at least two category samples, and each sample subset corresponds to a text category; The processing unit is configured to, for each sample subset, preprocess each category sample in the at least two category samples to obtain at least two sample part-of-speech sequences, where each sample part-of-speech sequence includes a sample word and the part-of-speech of the sample word; The processing unit is further configured to calculate a weight for each sample part-of-speech sequence to obtain an importance weight value for each sample part-of-speech sequence; The determining unit is further configured to determine a target weight value from the importance weight values according to a preset importance threshold, and determine the sample word corresponding to the target weight value as a target domain word; The processing unit is further configured to add the target domain word to the domain dictionary corresponding to the text category.

18. The device according to claim 17, characterized in that, Specifically, the processing unit is configured to: Filter out stop words from each category sample to obtain a text to be processed, where the text to be processed includes at least two sentences to be processed; Perform word segmentation on each of the at least two sentences to be processed to obtain at least two words to be processed; Perform part-of-speech tagging on each of the at least two words to be processed to obtain a sample part-of-speech sequence.

19. The device according to claim 12, wherein Specifically, the determining unit is configured to: Perform vector processing on the text to be classified to obtain at least two word vectors, where one word vector corresponds to one word; Input the at least two word vectors into a topic model for topic distribution processing to obtain a topic distribution vector of the text to be classified; Calculate the distance between each of the at least two word vectors and the topic distribution vector respectively to obtain X distances, where X is an integer greater than 1; Determine a candidate distance that meets a preset distance threshold from the X distances, and determine the word vector corresponding to the candidate distance as a topic-related word vector; Determine the word corresponding to the topic-related word vector as the topic word.

20. The device according to claim 19, characterized in that, The apparatus further includes a processing unit; The determining unit is further configured to, if the number of topic words is one, determine the topic-related word vector corresponding to the topic word as a topic word feature; The processing unit is further configured to, if the number of topic words is greater than one, perform vector integration processing on the topic-related word vectors corresponding to the topic words to obtain the topic word feature.

21. The device according to claim 12, characterized in that, The apparatus further includes a processing unit; The determining unit is further configured to determine a target domain dictionary and a target sample subset corresponding to the target text category; The processing unit is configured to add the text to be classified to the target sample subset to update the target domain dictionary.

22. The device according to claim 12, characterized in that, The apparatus further includes a processing unit; The determining unit is further configured to determine a target associated department corresponding to the target text category; The processing unit is configured to push the text to be classified to the terminal device corresponding to the target associated department, so that the terminal device displays the text to be classified.

23. A computer device, characterized in that, The computer device includes a processor and a memory; The memory is used to store program code; The processor is configured to execute the determination method according to any one of claims 1 to 11 based on the instructions in the program code; The bus system is used to connect the memory and the processor, so that the memory and the processor can communicate.

24. A computer-readable storage medium stores instructions that, when run on a computer, cause the computer to execute the determination method according to any one of claims 1 to 11 above.

25. A computer program product, characterized in that, The computer program product includes computer instructions, and the processor of the computer device executes the computer instructions, so that the computer device executes the determination method according to any one of claims 1 to 11 above.

Images