Text processing method and apparatus, and computer storage medium

Abstract

Embodiments of the present application provide a text processing method, device and computer storage medium. The method comprises: obtaining a text to be processed; calling a first classification network to process a secondary category code representation, category matching parameters and semantic code representation of the text to be processed, to obtain classification information of a secondary category of the text to be processed; and determining a classification result of the text to be processed based on the classification information of the secondary category and classification information of a primary category of the text to be processed, the classification information of the primary category of the text to be processed being obtained by processing the text to be processed by a second classification network, so that the accuracy of hierarchical classification of the text can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a text processing method and device and computer storage medium. BACKGROUND

[0002] Hierarchical Multi-Label Classification (HMC) is an important multi-classification task in the field of natural language processing / computer vision, which is characterized in that the class labels have a hierarchical relationship, the upper class is the parent of the lower class, and the granularity becomes finer as it goes down, such as the upper class being "game" and the lower class being "small game", "mobile game" and "PC game".

[0003] At present, hierarchical classification is usually taken as a basic multi-classification task, that is, the lower class is directly predicted, and the upper class is obtained by inversely deducing from the lower class, such as predicting the lower class "small game", and then deducing the upper class "game", but directly predicting the lower class often faces the problem of a small training sample set, and due to the excessive number of lower classes, accurate classification cannot be performed. SUMMARY

[0004] The embodiments of the present application provide a text processing method, device and computer storage medium, which can improve the accuracy of hierarchical classification.

[0005] In one aspect, the embodiments of the present application provide a text processing method, which comprises:

[0006] obtaining a text to be processed;

[0007] calling a first classification network to process the secondary class code representation, the class matching parameter and the semantic code representation of the text to be processed, and obtaining the classification information of the secondary class of the text to be processed;

[0008] determining the classification result of the text to be processed based on the classification information of the secondary class and the classification information of the primary class of the text to be processed, wherein the classification information of the primary class of the text to be processed is obtained by processing the text to be processed by a second classification network.

[0009] In one aspect, the embodiments of the present application provide a text processing device, which comprises:

[0010] an acquisition module configured to acquire a text to be processed;

[0011] a processing module configured to call a first classification network to process the secondary class code representation, the class matching parameter and the semantic code representation of the text to be processed, and obtain the classification information of the secondary class of the text to be processed;

[0012] The processing module is further configured to determine the classification result of the text to be processed based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, wherein the classification information of the primary category of the text to be processed is obtained by processing the text to be processed by using a second classification network.

[0013] In one aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory and a communication interface, the processor, the memory and the communication interface are connected to each other, the processor is adapted to execute a computer program, and the memory stores the computer program.

[0014] In one aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program, and the computer program is adapted to be loaded and executed by a processor to implement the text processing method.

[0015] In one aspect, an embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer storage medium. A processor of a computer device reads the computer instructions from the computer storage medium, and the processor executes the computer instructions to enable the computer device to execute the text processing method.

[0016] In the embodiment of the present application, first, the second classification network is called to process the text to be processed to obtain the semantic encoding representation of the text to be processed and the classification information of the primary category of the text to be processed, then the first classification network is called to process the secondary category encoding representation, the category matching parameter and the semantic encoding representation of the text to be processed to obtain the classification information of the secondary category of the text to be processed, and finally the classification result of the text to be processed can be determined based on the classification information of the secondary category and the classification information of the primary category of the text to be processed; the text processing method can obtain the matching result between the text to be processed and the secondary category by using the secondary category encoding representation, the category matching parameter and the semantic encoding representation of the text to be processed, and the secondary category of the text to be processed can be accurately obtained according to the matching result, so that the secondary category of the text can be obtained based on the matching manner, and the accuracy of the hierarchical classification of the text can be improved by combining the classification information of the primary category of the text to be processed. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a schematic diagram of an architecture of a computer device provided by an exemplary embodiment of the present application;

[0019] Figure 2 is a schematic diagram of a flow of a text processing method provided by an exemplary embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a flow of a text processing method provided by another exemplary embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a flow of a text processing method provided by another exemplary embodiment of the present application;

[0022] Figure 5 is a schematic diagram of a flow of a text processing method provided by another exemplary embodiment of the present application;

[0023] Figure 6 is a schematic diagram of a structure of a text processing apparatus provided by an exemplary embodiment of the present application;

[0024] Figure 7 is a schematic diagram of a structure of a computer device provided by another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0026] It should be noted that the descriptions of “first”, “second” and the like in the embodiments of the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the technical features or implicitly indicating the number of the technical features. Therefore, the technical features limited by “first”, “second” can explicitly or implicitly include at least one of the features.

[0027] In order to achieve the purpose of improving the accuracy of hierarchical classification of text, the embodiments of the present application propose a text processing method based on cloud technology.

[0028] Artificial Intelligence (AI) technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes, such as sensors, special artificial intelligence chip cloud computing, cloud storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and several other major directions.

[0029] Nature Language processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies the various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, that is, the language used in daily life, so it has a close relationship with the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies.

[0030] Machine Learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a branch of computer science that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0031] With the research and progress of artificial intelligence technology, artificial intelligence technology is researched and applied in many fields. The process of determining the classification result of the text in the embodiments of the present application involves natural language processing and machine learning of artificial intelligence, which is specifically described as follows.

[0032] In order to better understand the text processing method, device and computer storage medium provided by the embodiments of the present application, the architecture of the text processing system to which the embodiments of the present application are applicable will be described first. Please refer to Figure 1 , Figure 1 is a schematic diagram of the architecture of a text processing system provided by an exemplary embodiment of the present application. As Figure 1As shown, the text processing system can specifically include a terminal device 101 and a server 102, and the terminal device 101 and the server 102 are connected through a network, for example, connected through a wireless network.

[0033] The terminal device 101 is also referred to as a terminal, a user equipment (UE), an access terminal, a subscriber unit, a mobile device, a user terminal, a wireless communication device, a user agent, or a user apparatus. The terminal device can be a smart television, a handheld device (for example, a smartphone, a tablet computer) with wireless communication function, a computing device (for example, a personal computer (PC), a vehicle-mounted device, a wearable device, or other smart devices, etc.), but is not limited thereto.

[0034] The server 102 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

[0035] As shown, Figure 2 Figure 2 is a flow diagram of a text processing method provided by an exemplary embodiment of the present application. The server 102 can call a semantic encoder included in the second classification network to process the input text to be processed to obtain a semantic encoding representation of the text to be processed, and process the semantic encoding representation of the text to be processed by using a classification layer included in the second classification network to obtain classification information of a first category of the text to be processed. Further, the first classification network is called to process the second category encoding representation, the category matching parameter, and the semantic encoding representation of the text to be processed to obtain classification information of a second category of the text to be processed. Based on the classification information of the second category and the classification information of the first category of the text to be processed, a classification result of the text to be processed is obtained. This method can obtain the matching result between the text to be processed and the second category by using the second category encoding representation, the category matching parameter, and the semantic encoding representation of the text to be processed. According to the matching result, the second category of the text to be processed can be accurately obtained. The second category of the text can be obtained based on the matching manner, and the classification accuracy of the text can be improved by combining the classification information of the first category of the text to be processed.

[0036] ​In one embodiment, server 102 can use video titles as text to be processed, and users can input video titles on terminal devices 101 with video playback clients installed to perform hierarchical classification of video titles, thereby completing the video search task; server 102 can also use product titles as text to be processed, perform hierarchical classification of product titles, thereby completing the product search task, and so on.

[0037] It is understood that the system architecture diagrams described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0038] The following section details one text processing method proposed in this application.

[0039] like Figure 3 As shown, Figure 3 This is a flowchart illustrating a text processing method provided in another exemplary embodiment of this application, in which the method is applied... Figure 1 Taking server 102 as an example, the method may include the following steps:

[0040] S301. Obtain the text to be processed.

[0041] The text to be processed is the text that needs to be categorized, which can be video titles, product titles, etc.

[0042] In one embodiment, the text to be processed can be a short text with a length less than a preset number of characters (e.g., 15 characters or 15 Chinese characters), or it can be a long text. This application does not limit the length of the text to be processed.

[0043] S302. The first classification network is invoked to process the secondary category encoding representation, category matching parameters, and semantic encoding representation of the text to be processed, so as to obtain the classification information of the secondary category of the text to be processed.

[0044] Since categories themselves also have semantic information, this application uses word encoding representations (such as word vectors) of secondary categories to construct secondary category encoding representations, and uses secondary category encoding representations to enhance the ability to identify secondary categories.

[0045] Specifically, the secondary category code representation is a matrix structure representation composed of word code representations (e.g., word vectors) of multiple secondary categories in a predefined category hierarchy, which includes two levels of categories: primary categories and secondary categories, wherein the secondary categories are categories at a lower level of the primary categories, such as the primary categories being "games" and "technology", and the secondary categories thereof can be "small games", "mobile games", and "cell phones", wherein assuming that the word code representations of "small games", "mobile games", and "cell phones" are , , , the secondary category code representation is or .

[0046] The semantic code representation of the text to be processed is obtained by encoding the text to be processed using a semantic encoder (e.g., CNN (Convolutional Neural Networks), LSTM (Long Short-Term Memory), and LSTM+Attention) in the second classification network.

[0047] The category matching parameters are obtained by adjusting the initialized matching parameters by adjusting the initialized matching parameters in the process of obtaining the first classification network and the second classification network, which are obtained by initializing the parameters in the predefined matrix structure.

[0048] The server can process the category matching parameters and the semantic code representation of the text to be processed using the secondary category code representation to obtain the matching results between the text to be processed and each secondary category in the predefined category hierarchy, which can reflect the probability of the text to be processed belonging to each secondary category. The server can process the matching results between the text to be processed and each secondary category in the predefined category hierarchy by calling the second classification network, thereby obtaining the classification information of the secondary category of the text to be processed, such as the matching results between the text to be processed and each secondary category being [60, 20, 10, 10], and the processing method being softmax (matching results), then the predicted classification information of the secondary category is [0.6, 0.2, 0.1, 0.1], assuming that the secondary categories are "small games", "mobile games", "cell phones", and "Latin dance", then the predicted classification information of the secondary category indicates that the probabilities of the text being predicted as "small games", "mobile games", "cell phones", and "Latin dance" are 0.6, 0.2, 0.1, and 0.1, respectively, at this time the second classification network includes a classification layer with a softmax function as the classification function.

[0049] In this embodiment, the server processes the semantic encoding representation and secondary category encoding representation of the text to be processed through category matching parameters, so that the matching results (similar to matching scores) between the text to be processed and each secondary category can be obtained based on the matching method, and the secondary category of the text to be processed can be obtained based on the matching results. This can solve the problem of insufficient training sample set and improve the accuracy of secondary category classification.

[0050] S303. Based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, determine the classification result of the text to be processed. The classification information of the primary category of the text to be processed is obtained by processing the text to be processed through a second classification network.

[0051] The primary category classification information of the text to be processed is obtained by inputting the text to be processed into a second classification network, which includes a semantic encoder and a classifier. The semantic encoder is used to obtain the semantic encoding features of the text to be processed, and the classifier uses the semantic encoding features of the text to obtain the primary category classification information of the text to be processed.

[0052] The server can obtain the secondary category of the text to be processed based on the classification information of the secondary category. For example, if the secondary category includes "mini-games", "mobile games", and "mobile phones", and the classification information of the secondary category is [0.6, 0.3, 0.1], then the secondary category of the text to be processed is "mini-games". At the same time, it can obtain the primary category of the text to be processed based on the classification information of the primary category. For example, if the primary category includes "games" and "technology", and the classification information of the primary category is [0.8, 0.2], then the primary category of the text to be processed is "games". The primary and secondary categories of the text to be processed are combined as the classification result of the text to be processed. For example, the classification result of the text to be processed is "games / mini-games".

[0053] In this embodiment, the second classification network is first invoked to process the text to be processed to obtain the semantic encoding representation of the text to be processed and the classification information of the first-level category of the text to be processed. Then, the first classification network is invoked to process the second-level category encoding representation, category matching parameters, and the semantic encoding representation of the text to be processed to obtain the classification information of the second-level category of the text to be processed. Finally, based on the classification information of the second-level category and the classification information of the first-level category of the text to be processed, the classification result of the text to be processed can be determined. This text processing method can obtain the matching result between the text to be processed and the second-level category through the second-level category encoding representation, category matching parameters, and the semantic encoding representation of the text to be processed. Based on the matching result, the second-level category of the text to be processed can be accurately obtained. At the same time, combined with the classification information of the first-level category of the text to be processed, the accuracy of the hierarchical classification of the text can be improved.

[0054] Figure 4This is a flowchart illustrating a text processing method provided in another exemplary embodiment of this application, in which the method is applied... Figure 1 Taking server 102 as an example, the method may include the following steps:

[0055] S401. Obtain the training sample set, the secondary category encoding representation, and the initial matching parameters.

[0056] The training sample set includes multiple texts and first-level and second-level target tags for each text. The second-level target tags are the next level down from the first-level target tags, as shown in Table 1 below. This is a specific example of the training sample set. For instance, the first-level target tag for the text "Jump Jump, a guide to getting 600 points" is "games," and the second-level target tag is "mini-games." Table 1

[0057]

[0058] The initial matching parameters are obtained by initializing the parameters in a predefined matrix structure.

[0059] In one embodiment, obtaining the secondary category code representation includes the following steps:

[0060] (1) Determine multiple first-level categories and multiple second-level categories based on the predefined category hierarchy structure.

[0061] To enable upper-level categories to constrain lower-level categories, this application predefines a category hierarchy structure. This predefined hierarchy includes two levels of categories: primary categories and secondary categories, where secondary categories are the next level down from the primary categories. For example, the predefined category hierarchy structure could be defined as follows: primary categories include "Sports," "Games," and "Entertainment"; secondary categories under "Sports" include "Football" and "Running"; secondary categories under "Games" include "Mini-Games" and "Mobile Games"; and secondary categories under "Entertainment" include "Music" and "Movies."

[0062] (2) Obtain the word encoding representation of each of the multiple secondary categories.

[0063] Perform word encoding processing (such as word2vec, one-hot encoding, etc.) on each of the multiple secondary categories to obtain the word encoding representation of each secondary category.

[0064] (3) Determine the secondary category code representation based on the word code representation of each secondary category in multiple secondary categories.

[0065] The secondary category code representation is obtained based on the word code representation of each secondary category. For example, the word code representations corresponding to the secondary categories are as follows: , , The second-level category code is represented as Or expressed as .

[0066] S402. The first neural network initialized is used to process each text included in the training sample set to obtain the semantic encoding representation and the classification information of the first-level category of each text.

[0067] In one embodiment, the initialized first neural network includes an initialized semantic encoder (e.g., CNN, LSTM+, LSTM+Attention) and an initialized classifier, which includes a classification layer that uses a softmax function as the classification function.

[0068] The server encodes each text input in the training sample set into a semantic encoder to obtain a semantic encoding representation of each text. This semantic encoding representation is then input into an initialized classifier for classification to obtain the predicted first-level category information. For example, if the predicted first-level category information is [0.8, 0.1, 0.1], and the first-level category is "games", "technology", or "dance", then the probabilities of the predicted first-level category information text being "games", "technology", or "dance" are 0.8, 0.1, and 0.1, respectively.

[0069] S403. The initialized second neural network is used to process the secondary category encoding representation, the initialized matching parameters, and the semantic encoding representation of each text to obtain the predicted secondary category classification information of each text.

[0070] In one embodiment, the initialized second neural network includes a classification layer that uses the softmax function as the classification function. When the classification layer included in the initialized second neural network processes the secondary category encoding representation, the initialized matching parameters, and the semantic encoding representation of each text to obtain the classification information of the predicted secondary category of each text, as shown in the following formula (1), the server can use the semantic encoding representation of each text multiplied by the initialized matching parameters multiplied by the secondary category encoding representation to obtain the matching result between each text and each secondary category in the above predefined category hierarchy structure.

[0071] Results= (1)

[0072] Among them, Results For the matching results, The semantic encoding representation of each text, For the initial matching parameters, This is represented by a second-level category code. The number of second-level categories, The dimension corresponding to the word encoding representation of the second-level category (the word encoding representation is a word vector) is 1× The semantic encoding of each text represents the corresponding dimension.

[0073] It is understandable that the matrix structure of the initial matching parameters is as follows: Therefore, it is necessary to represent the corresponding dimension (1×) according to the semantic encoding of each text. ), and the corresponding dimensions of word encoding representation for secondary categories ( The size of the matrix corresponding to the initial matching parameters is determined by initializing the parameters in the predefined matrix structure. Since the initial matching parameters are obtained by initializing the parameters in the predefined matrix structure, the size of the predefined matrix structure is determined according to the dimension corresponding to the semantic encoding representation of each text and the dimension corresponding to the word encoding representation of the second-level category.

[0074] In one embodiment, as shown in formula (2) below, the server can also use the second-level category encoding representation multiplied by the initialized matching parameter multiplied by the semantic encoding representation of each text to obtain the matching result between each text and each second-level category in the predefined category hierarchy structure above.

[0075] Results (2)

[0076] Among them, Results For the matching results, The semantic encoding representation of each text, For the initial matching parameters, This is represented by a second-level category code. 1 represents the number of second-level categories, and 1×d represents the dimension corresponding to the word encoding representation of the second-level category (this word encoding representation is a word vector). The semantic encoding of each text represents the corresponding dimension.

[0077] Furthermore, the server utilizes the classification layer included in the initialized second neural network to process the matching results between each text and each second-level category in the predefined category hierarchy structure to obtain the predicted classification information of the second-level category. For example, if the matching result is [60,20,10,10] and the processing method is softmax (matching result), then the predicted classification information of the second-level category is [0.6,0.2,0.1,0.1]. Assuming that the second-level categories are "mini-games", "mobile games", "mobile phones", and "Latin dance", then the predicted classification information of the second-level category indicates that the probability of the text being predicted as "mini-games", "mobile games", "mobile phones", and "Latin dance" is 0.6, 0.2, 0.1, and 0.1, respectively.

[0078] S404. Based on the predicted primary category classification information, the predicted secondary category classification information, the primary category target label, and the secondary category target label of each text, train the first neural network, the second neural network, and the initialized matching parameters to obtain category matching parameters, the first classification network, and the second classification network.

[0079] The server can use the predicted primary category classification information, predicted secondary category classification information, primary category tags, and secondary category tags of each text to train the first neural network, the second neural network, and the initialized matching parameters to obtain category matching parameters, the first classification network, and the second classification network.

[0080] In one embodiment, the server trains the first neural network, the second neural network, and the initialized matching parameters to obtain category matching parameters, the first classification network, and the second classification network, which may include the following steps:

[0081] (1) Based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level target label and the second-level target label of each text, determine the first loss value of the first neural network and the second loss value of the second neural network.

[0082] The server determines the first loss value of the first neural network based on the predicted first-level category classification information, the first-level category tag, and the first loss function of the first neural network for each text. As shown in the following formula (3), it is the first loss function of the first neural network. When the first-level category is "game", "technology", or "dance", the predicted first-level category classification information is [0.8, 0.1, 0.1], and the first-level category tag is [1, 0, 0] (indicating that the text belongs to "game"), the first loss value of the first neural network is -log(0.8).

[0083] (3)

[0084] in, Let the first loss function of the first neural network be _____. For first-level target labels, For the classification information of the predicted primary category, This refers to the number of first-level categories.

[0085] Furthermore, the server determines the second loss value of the second neural network based on the predicted classification information of the second-level category of each text, the second-level category tag, and the second loss function of the second neural network. As shown in the following formula (4), it is the second loss function of the second neural network. When the second-level category is "mini-game", "mobile game", "mobile phone", "Latin dance", the predicted classification information of the second-level category is [0.6,0.2,0.1,0.1], and the second-level category tag is [1,0,0,0] (indicating that the text belongs to "mini-game"), the second loss value of the second neural network is -log(0.6).

[0086] (4)

[0087] in, This is the second loss function of the second neural network. For secondary category targets, For the predicted classification information of the secondary categories, This refers to the number of second-level categories.

[0088] (2) Determine the total loss value based on the first loss value and the second loss value.

[0089] To ensure the consistency of classification results, that is, to ensure the consistency between the second-level categories and the first-level categories of the text (at this time, the second-level categories are the next level of the first-level categories), this application introduces a third loss value when determining the total loss value. This third loss value is obtained through the constraint loss function between the first-level categories and the second-level categories.

[0090] In one embodiment, the server determines a third loss value based on a first loss value, a second loss value, and a constraint loss function between the first-level category and the second-level category.

[0091] Specifically, to ensure the consistency of classification results, and since first-level categories are always easier to classify into second-level categories, this application adds a constraint loss function between first-level and second-level categories to ensure that the probability of a text belonging to a first-level category is always greater than the probability of it belonging to a second-level category. For example, if the probability of a text belonging to the first-level category "games" is 0.8, then the probability of it belonging to the second-level category "mini-games" is less than 0.8 and greater than or equal to 0. The constraint loss function is shown in the following formula (5):

[0092] (5)

[0093] in, These are custom parameters used to limit the error range of the first and second loss values. This is the constraint loss function.

[0094] Furthermore, the server can determine the total loss function based on the weight coefficients of the first loss function, the second loss function, and the constraint loss function, and input the first loss value, the second loss value, and the third loss value into the total loss function to obtain the total loss value. The total loss function is shown in the following formula (6):

[0095] (6)

[0096] Where q is the number of texts included in the training sample set. For the total loss function, , , The weight coefficients for the first loss function, the second loss function, and the constraint loss function are user-defined parameters.

[0097] It is understandable that when the second loss value is greater than the first loss value + At that time, the probability of a text belonging to a first-level category is less than the probability of it belonging to a second-level category. For example, When the first-level category is "Games", "Technology", or "Dance", and the predicted classification information for the first-level category is [0.6, 0.2, 0.2], and the first-level category tag is [1, 0, 0] (indicating the text belongs to "Games"), the first loss value is -log(0.6). When the second-level category is "Mini-Games", "Mobile Games", "Mobile Phones", or "Latin Dance", and the predicted classification information for the second-level category is [0.8, 0.1, 0.1, 0], and the second-level category tag is [1, 0, 0, 0] (indicating the text belongs to "Mini-Games"), the second loss value is -log(0.8). In this case, the probability of the text being "Games" is 0.6, and the probability of it being "Mini-Games" is 0.8. At this point, the second loss value (-log(0.8)) is greater than the first loss value (-log(0.6)). The probability that the text belongs to the first-level category ("game") is less than the probability that it belongs to the second-level category ("mini-game"). Since the probability that the text belongs to the first-level category is always greater than the probability that it belongs to the second-level category, the classification is inaccurate and the consistency of the classification results is not guaranteed. This application can increase the total loss value by calculating the constraint loss function so that the third loss value is positive at this point, thereby slowing down the convergence speed of the total loss function and constraining the consistency of the classification results. This also enables the category matching parameters, the first classification network and the second classification network obtained by subsequent training to achieve accurate hierarchical classification of the text.

[0098] In this embodiment, a constraint loss function is introduced into the total loss function, so that the total loss value can constrain the consistency of the classification results, effectively utilizing the upper and lower constraint relationships in the category hierarchy structure.

[0099] (3) The total loss value is used to adjust the network parameters of the first neural network, the network parameters of the second neural network, and the initial matching parameters. When the total loss value meets the convergence condition, the category matching parameters, the first classification network, and the second classification network are trained.

[0100] The server uses the total loss value to adjust the network parameters of the first neural network, the network parameters of the second neural network, and the initial matching parameters. When the total loss value meets the convergence condition, or when the number of training iterations reaches a preset value (set manually), the adjustment of the network parameters of the first neural network, the network parameters of the second neural network, and the initial matching parameters is stopped. The category matching parameters are obtained based on the adjusted initial matching parameters, the second classification network is obtained based on the adjusted first neural network, and the first classification network is obtained based on the adjusted second neural network.

[0101] In this embodiment of the application, by training the initialized first neural network, the initialized second neural network, and the initialized matching parameters, category matching parameters are obtained. The first classification network and the second classification network can be used to achieve accurate hierarchical classification of text.

[0102] Figure 5 This is a flowchart illustrating a text processing method provided in another exemplary embodiment of this application, in which the method is applied... Figure 1 Taking server 102 as an example, the method may include the following steps:

[0103] S501. Obtain the text to be processed.

[0104] S502. Call the semantic encoder of the second classification network to process the text to be processed, and obtain the semantic encoded representation of the text to be processed.

[0105] The semantic encoder of the second classification network is used to obtain the semantic information of the text to be processed. This semantic encoder can be CNN, LSTM, LSTM+Attention, etc.

[0106] By inputting the text to be processed into the semantic encoder of the second classification network, a semantic encoded representation of the text to be processed can be obtained.

[0107] S503. Call the classifier of the second classification network to process the semantic encoding representation of the text to be processed, and obtain the classification information of the first-level category of the text to be processed.

[0108] The classifier of the second classification network includes a classification layer that uses the softmax function as the classification function. By inputting the semantic encoding representation of the text to be processed into the classification layer of the second classification network, the classification information of the first-level category of the text to be processed can be obtained. For example, if the first-level category is "games", "technology", and "dance", and the classification information of the first-level category is [0.8, 0.1, 0.1], then the classification information of the first-level category means that the probability of the text being predicted as "games", "technology", and "dance" is 0.8, 0.1, and 0.1, respectively.

[0109] S504. Process the secondary category encoding representation and the semantic encoding representation of the text to be processed using category matching parameters to determine the matching result between each secondary category and the text to be processed.

[0110] In one embodiment, when the server processes the secondary category encoding representation and the semantic encoding representation of the text to be processed using the category matching parameter, as shown in the following formula (7), the server can multiply the semantic encoding representation of the text to be processed by the category matching parameter multiplied by the secondary category encoding representation to obtain the matching result between the text to be processed and each secondary category in the above predefined category hierarchy structure.

[0111] Results= (7)

[0112] Among them, Results For the matching results, The semantic encoding representation of the text to be processed. Matching parameters for the directory, This is represented by a second-level category code. d×1 represents the number of second-level categories, and d×1 represents the dimension of the word encoding representation (which is a word vector) of the second-level category. The semantic encoding of the text to be processed represents the corresponding dimension.

[0113] In one embodiment, when the server processes the secondary category encoding representation and the semantic encoding representation of the text to be processed using the category matching parameter, as shown in the following formula (8), the server can multiply the secondary category encoding representation by the category matching parameter by the semantic encoding representation of the text to be processed to obtain the matching result between the text to be processed and each secondary category in the above-defined category hierarchy structure.

[0114] Results (8)

[0115] Among them, Results For the matching results, The semantic encoding representation of the text to be processed. For category matching parameters, This is represented by a second-level category code. 1 represents the number of second-level categories, and 1×d represents the dimension corresponding to the word encoding representation of the second-level category (this word encoding representation is a word vector). The semantic encoding of the text to be processed represents the corresponding dimension.

[0116] In this embodiment, the server processes the semantic encoding representation and the secondary category encoding representation of the text to be processed through category matching parameters, so that the matching result (similar to a matching score) between the text to be processed and each secondary category can be obtained based on the matching method.

[0117] S505. The first classification network is invoked to process the matching results between each secondary category and the text to be processed, so as to obtain the classification information of the secondary category of the text to be processed.

[0118] The first classification network includes a classification layer that uses the softmax function as the classification function. The server obtains the classification information of the second-level categories of the text to be processed by inputting the matching results between each second-level category and the text to be processed into the classification layer of the first classification network. For example, if the second-level categories are "mini-games", "mobile games", "mobile phones", and "Latin dance", and the classification information of the second-level categories is [0.6, 0.2, 0.1, 0.1], then the classification information of the second-level categories indicates that the probability of the text being predicted as "mini-games", "mobile games", "mobile phones", and "Latin dance" is 0.6, 0.2, 0.1, and 0.1, respectively. In this embodiment, the server uses the first classification network to process the matching results (the first classification network is mainly used for normalization processing) to obtain the classification information of the second-level categories of the text to be processed. This can solve the problem of insufficient training sample set and improve the accuracy of second-level category classification.

[0119] S506. Determine the classification result of the text to be processed based on the classification information of the secondary category of the text to be processed and the classification information of the primary category of the text to be processed.

[0120] The server obtains the secondary category of the text to be processed based on the classification information of the secondary category, and obtains the primary category of the text to be processed based on the classification information of the primary category. The primary and secondary categories of the text to be processed are combined as the classification result of the text to be processed.

[0121] In this embodiment, the second classification network is first invoked to process the text to be processed to obtain the semantic encoding representation of the text to be processed and the classification information of the first-level category of the text to be processed. Then, the category matching parameters process the second-level category encoding representation and the semantic encoding representation of the text to be processed to obtain the matching result between each second-level category and the text to be processed. The first classification network is then invoked to process the matching result to obtain the classification information of the second-level category of the text to be processed. Finally, based on the classification information of the second-level category and the classification information of the first-level category of the text to be processed, the classification result of the text to be processed can be determined. This text processing method can obtain the matching result between the text to be processed and the second-level category by processing the second-level category encoding representation and the semantic encoding representation of the text to be processed through the category matching parameters. According to the matching result, the second-level category of the text to be processed can be accurately obtained, so that the second-level category of the text to be processed can be obtained based on the matching method. At the same time, combined with the classification information of the first-level category of the text to be processed, the accuracy of the hierarchical classification of the text can be improved.

[0122] The methods of the embodiments of this application have been described in detail above. To facilitate better implementation of the above solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below. Please refer to... Figure 6 , Figure 6 This is a schematic diagram of the structure of a text processing apparatus provided in an exemplary embodiment of this application. The apparatus 60 may include:

[0123] Module 601 is used to acquire the text to be processed;

[0124] The processing module 602 is used to call the first classification network to process the secondary category encoding representation, category matching parameters and the semantic encoding representation of the text to be processed, so as to obtain the classification information of the secondary category of the text to be processed;

[0125] The processing module 602 is further configured to determine the classification result of the text to be processed based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, wherein the classification information of the primary category of the text to be processed is obtained by processing the text to be processed through a second classification network.

[0126] In one embodiment, the acquisition module 601 is further configured to:

[0127] Based on a predefined category hierarchy, multiple first-level categories and multiple second-level categories are determined, wherein the second-level categories are the next level categories of the first-level categories;

[0128] Obtain the word encoding representation of each of the multiple secondary categories;

[0129] The secondary category code representation is determined based on the word code representation of each of the multiple secondary categories.

[0130] In one embodiment, the processing module 602 is further configured to:

[0131] The category matching parameters are used to process the secondary category encoding representation and the semantic encoding representation of the text to be processed to determine the matching result between each secondary category and the text to be processed.

[0132] The first classification network is invoked to process the matching results between each secondary category and the text to be processed, thereby obtaining the classification information of the secondary category of the text to be processed.

[0133] In one embodiment, the processing module 602 is further configured to:

[0134] The semantic encoder of the second classification network is invoked to process the text to be processed, and the semantic encoded representation of the text to be processed is obtained.

[0135] The classifier of the second classification network is invoked to process the semantic encoding representation of the text to be processed, thereby obtaining the classification information of the first-level category of the text to be processed.

[0136] In one embodiment, the acquisition module 601 is further configured to:

[0137] Obtain the training sample set, the secondary category encoding representation, and the initial matching parameters. The training sample set includes multiple texts and the primary category target tag and the secondary category target tag of each of the multiple texts.

[0138] The aforementioned processing module 602 is also used for:

[0139] The first neural network initialized is used to process each text included in the training sample set to obtain the semantic encoding representation and the classification information of the first-level category of each text;

[0140] The second neural network is used to process the secondary category encoding representation, the initial matching parameters, and the semantic encoding representation of each text to obtain the predicted secondary category classification information of each text.

[0141] Based on the predicted primary category classification information, the predicted secondary category classification information, the primary category tags, and the secondary category tags of each text, the first neural network, the second neural network, and the initialized matching parameters are trained to obtain category matching parameters, the first classification network, and the second classification network.

[0142] In one embodiment, the processing module 602 is further configured to:

[0143] Based on the predicted primary category classification information, the predicted secondary category classification information, the primary category target label, and the secondary category target label of each text, the first loss value of the first neural network and the second loss value of the second neural network are determined.

[0144] The total loss value is determined based on the first loss value and the second loss value;

[0145] The total loss value is used to adjust the network parameters of the first neural network, the network parameters of the second neural network, and the initialized matching parameters;

[0146] When the total loss value meets the convergence condition, the category matching parameters, the first classification network, and the second classification network are trained.

[0147] In one embodiment, the processing module 602 is further configured to:

[0148] Based on the predicted primary category classification information of each text, the primary category tag, and the first loss function of the first neural network, the first loss value of the first neural network is determined.

[0149] The second loss value of the second neural network is determined based on the predicted classification information of the secondary category of each text, the secondary category tag, and the second loss function of the second neural network.

[0150] In one embodiment, the processing module 602 is further configured to:

[0151] A third loss value is determined based on the first loss value, the second loss value, and the constraint loss function between the first-level category and the second-level category;

[0152] The total loss value is determined based on the first loss value, the second loss value, the third loss value, and the weight coefficients of the first loss function, the second loss function, and the constraint loss function.

[0153] In this embodiment, the second classification network is first invoked to process the text to be processed to obtain the semantic encoding representation of the text to be processed and the classification information of the first-level category of the text to be processed. Then, the first classification network is invoked to process the second-level category encoding representation, category matching parameters, and the semantic encoding representation of the text to be processed to obtain the classification information of the second-level category of the text to be processed. Finally, based on the classification information of the second-level category and the classification information of the first-level category of the text to be processed, the classification result of the text to be processed can be determined. This text processing method can obtain the matching result between the text to be processed and the second-level category through the second-level category encoding representation, category matching parameters, and the semantic encoding representation of the text to be processed. Based on the matching result, the second-level category of the text to be processed can be accurately obtained. At the same time, combined with the classification information of the first-level category of the text to be processed, the accuracy of the hierarchical classification of the text can be improved.

[0154] Figure 7 This is a schematic diagram of the structure of a computer device 70 provided in an exemplary embodiment of this application. The computer device 70 includes at least a processor 701, a memory 702, and a communication interface 703. The processor 701, memory 702, and communication interface 703 can be connected via a bus or other means. The communication interface 703 can be used to receive or send data. The memory 702 stores a computer program, which includes computer instructions. The processor 701 is used to execute the computer instructions included in the computer program. The processor 701 (or CPU (Central Processing Unit)) is the computing and control core of the computer device 70, and is suitable for implementing one or more computer instructions, specifically suitable for loading and executing one or more computer instructions to achieve a corresponding method flow or corresponding function. In specific implementation, the computer instructions in the memory 702 are loaded by the processor 701 and executed in the following steps:

[0155] Get the text to be processed;

[0156] The first classification network is invoked to process the secondary category encoding representation, category matching parameters, and semantic encoding representation of the text to be processed, so as to obtain the classification information of the secondary category of the text to be processed;

[0157] The classification result of the text to be processed is determined based on the classification information of the secondary category and the classification information of the primary category of the text to be processed. The classification information of the primary category of the text to be processed is obtained by processing the text to be processed through a second classification network.

[0158] In one embodiment, the processor 701 is further configured to:

[0159] Based on a predefined category hierarchy, multiple first-level categories and multiple second-level categories are determined, wherein the second-level categories are the next level categories of the first-level categories;

[0160] Obtain the word encoding representation of each of the multiple secondary categories;

[0161] The secondary category code representation is determined based on the word code representation of each of the multiple secondary categories.

[0162] In one embodiment, the processor 701 is further configured to:

[0163] The category matching parameters are used to process the secondary category encoding representation and the semantic encoding representation of the text to be processed to determine the matching result between each secondary category and the text to be processed.

[0164] The first classification network is invoked to process the matching results between each secondary category and the text to be processed, thereby obtaining the classification information of the secondary category of the text to be processed.

[0165] In one embodiment, the processor 701 is further configured to:

[0166] The semantic encoder of the second classification network is invoked to process the text to be processed, and the semantic encoded representation of the text to be processed is obtained.

[0167] The classifier of the second classification network is invoked to process the semantic encoding representation of the text to be processed, thereby obtaining the classification information of the first-level category of the text to be processed.

[0168] In one embodiment, the processor 701 is further configured to:

[0169] Obtain the training sample set, the secondary category encoding representation, and the initial matching parameters. The training sample set includes multiple texts and the primary category target tag and the secondary category target tag of each of the multiple texts.

[0170] The first neural network initialized is used to process each text included in the training sample set to obtain the semantic encoding representation and the classification information of the first-level category of each text;

[0171] The second neural network is used to process the secondary category encoding representation, the initial matching parameters, and the semantic encoding representation of each text to obtain the predicted secondary category classification information of each text.

[0172] Based on the predicted primary category classification information, the predicted secondary category classification information, the primary category tags, and the secondary category tags of each text, the first neural network, the second neural network, and the initialized matching parameters are trained to obtain category matching parameters, the first classification network, and the second classification network.

[0173] In one embodiment, the processor 701 is further configured to:

[0174] Based on the predicted primary category classification information, the predicted secondary category classification information, the primary category target label, and the secondary category target label of each text, the first loss value of the first neural network and the second loss value of the second neural network are determined.

[0175] The total loss value is determined based on the first loss value and the second loss value;

[0176] The total loss value is used to adjust the network parameters of the first neural network, the network parameters of the second neural network, and the initialized matching parameters;

[0177] When the total loss value meets the convergence condition, the category matching parameters, the first classification network, and the second classification network are trained.

[0178] In one embodiment, the processor 701 is further configured to:

[0179] A third loss value is determined based on the first loss value, the second loss value, and the constraint loss function between the first-level category and the second-level category;

[0180] The total loss value is determined based on the first loss value, the second loss value, the third loss value, and the weight coefficients of the first loss function, the second loss function, and the constraint loss function.

[0181] In this embodiment, the second classification network is first invoked to process the text to be processed to obtain the semantic encoding representation of the text to be processed and the classification information of the first-level category of the text to be processed. Then, the first classification network is invoked to process the second-level category encoding representation, category matching parameters, and the semantic encoding representation of the text to be processed to obtain the classification information of the second-level category of the text to be processed. Finally, based on the classification information of the second-level category and the classification information of the first-level category of the text to be processed, the classification result of the text to be processed can be determined. This text processing method can obtain the matching result between the text to be processed and the second-level category through the second-level category encoding representation, category matching parameters, and the semantic encoding representation of the text to be processed. Based on the matching result, the second-level category of the text to be processed can be accurately obtained. At the same time, combined with the classification information of the first-level category of the text to be processed, the accuracy of the hierarchical classification of the text can be improved.

[0182] In this embodiment of the application, a computer storage medium is also provided. This computer storage medium is a memory device within the computer device 70, used to store programs and data. It is understood that the computer storage medium here may include the built-in storage medium of the computer device 70, or it may include extended storage media supported by the computer device 70. The computer storage medium provides storage space, which stores the operating system of the computer device 70. Furthermore, this storage space also stores one or more computer instructions suitable for loading and execution by the processor 701. These computer instructions may be one or more computer programs (including program code). It should be noted that the computer storage medium here may be high-speed RAM, or non-volatile memory, such as at least one disk storage device; optionally, it may also be at least one computer storage medium located remotely from the aforementioned processor 701.

[0183] One or more embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer storage medium. A processor of a computer device reads the computer instructions from the computer storage medium and executes the computer instructions, causing the computer device to perform the steps performed in the embodiments of the above methods.

[0184] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A text processing method, characterized in that, The method includes: Get the text to be processed; The first classification network is invoked to process the secondary category encoding representation, category matching parameters, and semantic encoding representation of the text to be processed, to obtain the classification information of the secondary category of the text to be processed; the secondary category encoding representation contains word encoding representations of multiple secondary categories; the category matching parameters are used to match the semantic encoding representation of the text to be processed with the word encoding representation of each secondary category in the secondary category encoding representation; The classification result of the text to be processed is determined based on the classification information of the secondary category and the classification information of the primary category of the text to be processed. The classification information of the primary category of the text to be processed is obtained by processing the text to be processed through a second classification network. The first classification network, the second classification network, and the category matching parameters are obtained by training the second neural network, the first neural network, and the initialized matching parameters using the total loss value; the total loss value is determined based on the first loss value of the first neural network and the second loss value of the second neural network. The first loss value is determined based on the difference between the classification information of the first-level category of the text predicted by the first neural network and the first-level category tag of the text; the second loss value is determined based on the difference between the classification information of the second-level category of the text predicted by the second neural network and the second-level category tag of the text.

2. The method according to claim 1, characterized in that, The method further includes: Based on a predefined category hierarchy, multiple first-level categories and multiple second-level categories are determined, wherein the second-level categories are the next level categories of the first-level categories; Obtain the word encoding representation of each of the multiple secondary categories; The secondary category code representation is determined based on the word code representation of each of the multiple secondary categories.

3. The method according to claim 2, characterized in that, The first classification network is invoked to process the secondary category encoding representation, category matching parameters, and semantic encoding representation of the text to be processed, to obtain the classification information of the secondary category of the text to be processed, including: The category matching parameters are used to process the secondary category encoding representation and the semantic encoding representation of the text to be processed to determine the matching result between each secondary category and the text to be processed. The first classification network is invoked to process the matching results between each secondary category and the text to be processed, thereby obtaining the classification information of the secondary category of the text to be processed.

4. The method according to any one of claims 1 to 3, characterized in that, Before determining the classification result of the text to be processed based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, the method further includes: The semantic encoder of the second classification network is invoked to process the text to be processed, and the semantic encoded representation of the text to be processed is obtained. The classifier of the second classification network is invoked to process the semantic encoding representation of the text to be processed, thereby obtaining the classification information of the first-level category of the text to be processed.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the training sample set, the secondary category encoding representation, and the initial matching parameters. The training sample set includes multiple texts and the primary category target tag and the secondary category target tag of each of the multiple texts. The first neural network initialized is used to process each text included in the training sample set to obtain the semantic encoding representation and the classification information of the first-level category of each text; The second neural network is used to process the secondary category encoding representation, the initial matching parameters, and the semantic encoding representation of each text to obtain the predicted secondary category classification information of each text. Based on the predicted primary category classification information, the predicted secondary category classification information, the primary category tags, and the secondary category tags of each text, the first neural network, the second neural network, and the initialized matching parameters are trained to obtain category matching parameters, the first classification network, and the second classification network.

6. The method according to claim 5, characterized in that, The method involves training the first neural network, the second neural network, and the initialized matching parameters based on the predicted primary category classification information, the predicted secondary category classification information, the primary category tag, and the secondary category tag for each text, to obtain category matching parameters, the first classification network, and the second classification network, including: Based on the predicted primary category classification information, the predicted secondary category classification information, the primary category target label, and the secondary category target label of each text, the first loss value of the first neural network and the second loss value of the second neural network are determined. The total loss value is determined based on the first loss value and the second loss value; The total loss value is used to adjust the network parameters of the first neural network, the network parameters of the second neural network, and the initialized matching parameters; When the total loss value meets the convergence condition, the category matching parameters, the first classification network, and the second classification network are trained.

7. The method according to claim 6, characterized in that, The determination of the first loss value of the first neural network and the second loss value of the second neural network based on the predicted primary category classification information, the predicted secondary category classification information, the primary category tag, and the secondary category tag for each text includes: Based on the predicted primary category classification information of each text, the primary category tag, and the first loss function of the first neural network, the first loss value of the first neural network is determined. The second loss value of the second neural network is determined based on the predicted classification information of the secondary category of each text, the secondary category tag, and the second loss function of the second neural network.

8. The method according to claim 6 or 7, characterized in that, The step of determining the total loss value based on the first loss value and the second loss value includes: A third loss value is determined based on the first loss value, the second loss value, and the constraint loss function between the first-level category and the second-level category; The total loss value is determined based on the first loss value, the second loss value, the third loss value, and the weight coefficients of the first loss function, the second loss function, and the constraint loss function.

9. A text processing device, characterized in that, The device includes: The acquisition module is used to acquire the text to be processed; The processing module is used to call the first classification network to process the secondary category encoding representation, category matching parameters, and semantic encoding representation of the text to be processed, so as to obtain the classification information of the secondary category of the text to be processed; the secondary category encoding representation contains word encoding representations of multiple secondary categories; the category matching parameters are used to match the semantic encoding representation of the text to be processed with the word encoding representation of each secondary category in the secondary category encoding representation; The processing module is further configured to determine the classification result of the text to be processed based on the classification information of the secondary category and the classification information of the primary category of the text to be processed. The classification information of the primary category of the text to be processed is obtained by processing the text to be processed through a second classification network. The first classification network, the second classification network, and the category matching parameters are obtained by training the second neural network, the first neural network, and the initialized matching parameters using the total loss value; the total loss value is determined based on the first loss value of the first neural network and the second loss value of the second neural network. The first loss value is determined based on the difference between the classification information of the first-level category of the text predicted by the first neural network and the first-level category tag of the text; the second loss value is determined based on the difference between the classification information of the second-level category of the text predicted by the second neural network and the second-level category tag of the text.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program adapted to be loaded by a processor and executed by the text processing method according to any one of claims 1 to 8.

11. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a processor, implement the text processing method as described in any one of claims 1 to 8.

Images