Text level classification method and device based on center loss

Abstract

The present invention discloses a text hierarchical classification method and device based on center loss, the method comprising: determining a target processing text to be classified; inputting the target processing text into a trained hierarchical classification neural network model to output a plurality of categories corresponding to the target processing text and having a hierarchical relationship with each other; the hierarchical classification neural network model is optimized using a center loss function during training until the center loss function is minimized; the value of the center loss function is used to characterize the training vector distance between the text vector of the training text and the center characterization vector of any corresponding training output category. It can be seen that the present invention can use the hierarchical classification algorithm model trained based on the center loss function to classify the text hierarchically, so as to achieve multi-level classification of the text, thereby improving the accuracy and efficiency of text classification, and facilitating the subsequent use of the classified text for further intelligent algorithm processing.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text level classification method and device based on center loss. Background Art

[0002] With the development of natural language processing algorithm technology, more and more companies have begun to use algorithms to classify text, and use the classified text to achieve various algorithm effects, such as automatically generating text or automatically recognizing text. However, when performing text classification, the existing technology does not take into account the connection between texts at multiple levels of classification, nor does it consider how to train an algorithm model that can achieve this effect more efficiently at the algorithm level. It can be seen that the existing technology has defects that need to be solved urgently. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and device for determining text hierarchical classification based on center loss, which can improve the accuracy and efficiency of text classification and facilitate the subsequent use of the classified text for further intelligent algorithm processing.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a text level classification method based on center loss, the method comprising:

[0005] Determine the target text to be classified;

[0006] The target processing text is input into a trained hierarchical classification neural network model to output a plurality of categories corresponding to the target processing text and having a hierarchical relationship with each other; the hierarchical classification neural network model is optimized using a central loss function during training until the central loss function is minimized; the value of the central loss function is used to represent the training vector distance between the text vector of the training text and the central representation vector of any corresponding training output category.

[0007] As an optional implementation, in the first aspect of the present invention, the value of the central loss function is used to represent the sum of the training vector distances between the text vector of the training text and the central representation vector of each corresponding training output category.

[0008] As an optional embodiment, in the first aspect of the present invention, among the multiple categories that have a hierarchical relationship with each other, the category vector distance between the central representation vector of each upper category and the central representation vector of its directly subordinate lower category is less than a preset category distance threshold.

[0009] As an optional implementation, in the first aspect of the present invention, the category distance threshold is inversely proportional to the hierarchical order of the upper category and / or the lower category in all categories.

[0010] As an optional embodiment, in the first aspect of the present invention, the value of the central loss function is also used to characterize the maximum value between the distance difference corresponding to any upper-level category in all the training output categories corresponding to the training text and zero; the distance difference is the difference between the category vector distance corresponding to the upper-level category and the category distance threshold.

[0011] As an optional implementation, in the first aspect of the present invention, the value of the center loss function is also used to characterize the sum of the maximum values ​​between zero and the distance differences corresponding to all upper-level categories in all the training output categories corresponding to the training text.

[0012] As an optional embodiment, in the first aspect of the present invention, the value of the central loss function includes the sum of a first function value and a second function value; the first function value includes the sum of the training vector distances between the text vector of the training text and the central representation vector of each corresponding training output category; the second function value includes the sum of the maximum value between the distance difference corresponding to all upper categories in all the training output categories corresponding to the training text and zero; the distance difference is the difference between the category vector distance between the central representation vector of the upper category and the central representation vector of the subordinate category directly under it, and the category distance threshold.

[0013] As an optional implementation, in the first aspect of the present invention, the hierarchical classification neural network model is trained by the following steps:

[0014] Determine a training data set including a plurality of training texts;

[0015] The training data set is input into the hierarchical classification neural network model for repeated training until the value of the central loss function is minimized to obtain a trained hierarchical classification neural network model.

[0016] The second aspect of the present invention discloses a text level classification device based on center loss, the device comprising:

[0017] A text determination module, used to determine the target processing text to be classified;

[0018] The text prediction module is used to input the target processing text into a trained hierarchical classification neural network model to output multiple categories corresponding to the target processing text that have a hierarchical relationship with each other; the hierarchical classification neural network model adopts a central loss function for optimization during training until the central loss function is minimized; the value of the central loss function is used to represent the training vector distance between the text vector of the training text and the central representation vector of any corresponding training output category.

[0019] As an optional implementation, in the second aspect of the present invention, the value of the central loss function is used to represent the sum of the training vector distances between the text vector of the training text and the central representation vector of each corresponding training output category.

[0020] As an optional embodiment, in the second aspect of the present invention, among the multiple categories that have a hierarchical relationship with each other, the category vector distance between the central representation vector of each upper category and the central representation vector of its directly subordinate lower category is less than a preset category distance threshold.

[0021] As an optional implementation, in the second aspect of the present invention, the category distance threshold is inversely proportional to the hierarchical order of the upper category and / or the lower category in all categories.

[0022] As an optional embodiment, in the second aspect of the present invention, the value of the central loss function is also used to characterize the maximum value between the distance difference corresponding to any upper-level category in all the training output categories corresponding to the training text and zero; the distance difference is the difference between the category vector distance corresponding to the upper-level category and the category distance threshold.

[0023] As an optional implementation, in the second aspect of the present invention, the value of the center loss function is also used to characterize the sum of the maximum values ​​between zero and the distance differences corresponding to all upper-level categories in all the training output categories corresponding to the training text.

[0024] As an optional embodiment, in the second aspect of the present invention, the value of the central loss function includes the sum of a first function value and a second function value; the first function value includes the sum of the training vector distances between the text vector of the training text and the central representation vector of each corresponding training output category; the second function value includes the sum of the maximum value between the distance difference corresponding to all upper categories in all the training output categories corresponding to the training text and zero; the distance difference is the difference between the category vector distance between the central representation vector of the upper category and the central representation vector of the subordinate category directly under it, and the category distance threshold.

[0025] As an optional implementation, in the second aspect of the present invention, the device further includes a training module, which is used to perform the following steps to train and obtain the hierarchical classification neural network model:

[0026] Determine a training data set including a plurality of training texts;

[0027] The training data set is input into the hierarchical classification neural network model for repeated training until the value of the central loss function is minimized to obtain a trained hierarchical classification neural network model.

[0028] The third aspect of the present invention discloses another text level classification device based on center loss, the device comprising:

[0029] A memory storing executable program code;

[0030] a processor coupled to the memory;

[0031] The processor calls the executable program code stored in the memory to execute part or all of the steps in the text level classification method based on center loss disclosed in the first aspect of the embodiment of the present invention.

[0032] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the text level classification method based on center loss disclosed in the first aspect of the embodiment of the present invention.

[0033] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0034] In an embodiment of the present invention, a target processing text to be classified is determined; the target processing text is input into a trained hierarchical classification neural network model to output a plurality of categories corresponding to the target processing text that have a hierarchical relationship with each other; the hierarchical classification neural network model is optimized using a center loss function during training until the center loss function is minimized; the value of the center loss function is used to characterize the training vector distance between the text vector of the training text and the center characterization vector of any corresponding training output category. It can be seen that the present invention can use the hierarchical classification algorithm model trained based on the center loss function to perform hierarchical classification on the text to achieve multi-level classification of the text, thereby improving the accuracy and efficiency of text classification, and facilitating the subsequent use of the classified text for further intelligent algorithm processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 It is a flowchart of a text level classification method based on center loss disclosed in an embodiment of the present invention;

[0037] Figure 2 It is a flowchart of another text level classification method based on center loss disclosed in an embodiment of the present invention;

[0038] Figure 3 It is a structural schematic diagram of a text level classification device based on center loss disclosed in an embodiment of the present invention;

[0039] Figure 4 It is a structural schematic diagram of another text level classification device based on center loss disclosed in an embodiment of the present invention;

[0040] Figure 5 It is a structural diagram of another text level classification device based on center loss disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0043] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0044] The present invention discloses a text hierarchical classification method and device based on center loss, which can hierarchically classify text using a hierarchical classification algorithm model trained based on a center loss function to achieve multi-level classification of text, thereby improving the accuracy and efficiency of text classification and facilitating the subsequent use of the classified text for further intelligent algorithm processing. The following are detailed descriptions.

[0045] Embodiment 1

[0046] See also Figure 1 , Figure 1 is a flow chart of a text level classification method based on center loss disclosed in an embodiment of the present invention. Figure 1 The described method is applied to a text processing device, which may be a corresponding processing terminal, processing equipment or processing server, and the server may be a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 1 As shown, the text level classification method based on center loss may include the following operations:

[0047] 101. Determine the target text to be processed.

[0048] Optionally, the type of the target text to be processed can be any text type, such as an advertisement text or a literary work text, and the character length of the text is not limited, and can be a word text with only one or two characters, or a large sentence text. Optionally, the language type of the target text to be processed can be Chinese, English, or other language types with mature semantic analysis possibilities, which is not limited by the present invention.

[0049] Optionally, the target processing text may be determined by a preset algorithm, or may be determined by receiving text input by a user, which is not limited in the present invention.

[0050] 102. Input the target processing text into the trained hierarchical classification neural network model to output a plurality of categories corresponding to the target processing text and having a hierarchical relationship with each other.

[0051] Specifically, the hierarchical classification neural network model uses a central loss function for optimization during training until the central loss function is minimized, wherein the value of the central loss function is used to represent the training vector distance between a text vector of the training text and a central representation vector of any corresponding training output category.

[0052] Optionally, the vector distance described in the present invention may be an L1 distance or an L2 distance, or other distance formulas that can be used to measure vector similarity, which is not limited in the present invention.

[0053] Optionally, multiple category hierarchies with hierarchical relationships between each other can be preset, each category hierarchy includes multiple categories, and each category corresponds to a central representation vector. In subsequent training or prediction, the output prediction category belongs to these preset category hierarchies.

[0054] Optionally, the central representation vector can be determined by a text set corresponding to each category or training output category, and the text set can include one or more texts, and the text can be used to represent the characteristics of the category. For example, an adjective category can correspond to a text set including multiple adjective texts. Optionally, the texts in the text set can be converted into text vectors, and the central representation vector can be calculated based on the text vectors. For example, the average value or other statistical value of multiple text vectors can be calculated to obtain the central representation vector.

[0055] Optionally, the hierarchical classification neural network model may include a feature extraction network for extracting text features and a classification network for predicting text categories, wherein preferably, the feature extraction network may adopt a Transformer network architecture, which has excellent effects on the extraction of text features. Optionally, the hierarchical classification neural network model will output multiple training output categories with hierarchical relationships between each other for any training text during training, and calculate the loss function value through the above-mentioned central loss function. In repeated training, the gradient descent method is used to continuously optimize the model parameters until the loss function value is minimized, thereby obtaining a trained hierarchical classification neural network model.

[0056] Optionally, in the specific prediction process of the hierarchical classification neural network model, it can first process the text features of the target processing text, and then, in turn, calculate the vector distance between the text features of the target processing text and the central representation vector of each category of each level of the preset multiple levels from high to low, and determine the category with the smallest corresponding vector distance among all categories in each level as the category corresponding to the target processing text in the level, so as to output multiple categories corresponding to the target processing text that have a hierarchical relationship with each other.

[0057] It can be seen that the method described in the embodiment of the present invention can use the hierarchical classification algorithm model trained based on the central loss function to perform hierarchical classification of text, so as to achieve multi-level classification of text, thereby improving the accuracy and efficiency of text classification, and facilitating the subsequent use of the classified text for further intelligent algorithm processing.

[0058] As an optional implementation, the value of the central loss function is the sum of the training vector distances between the text vector representing the training text and the central representation vector of each corresponding training output category.

[0059] Specifically, the hierarchical classification neural network model will output multiple training output categories with hierarchical relationships among each other for any training text during training.

[0060] It can be seen that this optional implementation measures the accuracy of the prediction results in training by calculating the sum of the training vector distances between the text vector of the training text and the central representation vector of each corresponding training output category, so that a hierarchical classification neural network model with better results can be trained more accurately.

[0061] As an optional implementation, among the multiple categories corresponding to the above-mentioned target processing text that have a hierarchical relationship with each other, the category vector distance between the central representation vector of each upper-level category and the central representation vector of its directly subordinate lower-level category is less than a preset category distance threshold. By setting it in this way, it is possible to design upper and lower-level categories that are more closely related to each other, which helps to make the hierarchical classification of the text more accurate.

[0062] Optionally, the category distance threshold may be inversely proportional to the hierarchical order of the upper category and / or the lower category in all categories. For example, the category distance threshold may be an inversely proportional value of the hierarchical order, such as the category distance threshold may be equal to the ratio of 1 or other preset values ​​to the hierarchical order. For another example, the category distance threshold may also be a negative value of the hierarchical order, such as the category distance threshold may be equal to 1 or other preset values ​​minus the result of the hierarchical order.

[0063] It can be seen that this optional implementation further limits the connection between upper and lower categories by setting the category distance threshold to be inversely proportional to the hierarchical order, thereby determining upper and lower categories that are more closely related to each other and improving the accuracy of text classification.

[0064] As an optional implementation, the value of the center loss function is also used to represent the maximum value between the distance difference corresponding to any upper-level category in all training output categories corresponding to the training text and zero.

[0065] Specifically, the distance difference is the difference between the category vector distance corresponding to the upper category and the category distance threshold, wherein the distance difference has a positive and negative relationship, and is negative when the category vector distance is less than the category distance threshold.

[0066] It can be seen that this optional implementation method is also used to characterize the maximum value between the distance difference corresponding to any upper-level category in all training output categories corresponding to the training text and zero by setting the value of the central loss function, so that when using the central loss function to train the classification network model, the hierarchical connection between multiple prediction categories output by the model can be further limited, thereby improving the accuracy of text classification.

[0067] As an optional implementation, the value of the central loss function is also used to represent the sum of the maximum values ​​between the distance differences corresponding to all upper-level categories in all training output categories corresponding to the training text and zero.

[0068] It can be seen that this optional implementation measures the accuracy of the hierarchical relationship between multiple prediction categories in training by calculating the sum of the distance differences corresponding to all upper categories in all training output categories corresponding to the training text and the maximum value between zero, so as to more accurately train a hierarchical classification neural network model with better results.

[0069] As an optional implementation, the value of the center loss function includes the sum of a first function value and a second function value. The first function value includes the sum of the training vector distances between the text vector of the training text and the center representation vector of each corresponding training output category, and the second function value includes the sum of the maximum value between the distance difference corresponding to all upper-level categories in all training output categories corresponding to the training text and zero.

[0070] Preferably, the value of the central loss function may be the sum of the first function value and the first product value, wherein the first product value is the product of the second function value and the weight parameter. Optionally, the weight parameter may be set by an operator based on empirical values ​​or experimental values, and subsequently adjusted based on actual results.

[0071] It can be seen that by implementing this optional implementation method, the specific architecture of the central loss function can be determined so that the central loss function can fully characterize the two training effects of the target, that is, on the one hand, the output training prediction categories need to be sufficiently accurate to characterize the characteristics of the training text, and on the other hand, the output multiple training prediction categories need to have an accurate hierarchical relationship, so that a hierarchical classification neural network model with better effects can be trained more accurately.

[0072] Embodiment 2

[0073] See also Figure 2 , Figure 2 is a flow chart of another text level classification method based on center loss disclosed in an embodiment of the present invention. Figure 2 The described method is applied to a text processing device, which may be a corresponding processing terminal, processing equipment or processing server, and the server may be a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 2 As shown, the text level classification method based on center loss may include the following operations:

[0074] 201. Determine a training data set including a plurality of training texts.

[0075] 202. Input the training data set into the hierarchical classification neural network model for repeated training until the value of the central loss function is minimized to obtain a trained hierarchical classification neural network model.

[0076] Specifically, the value of the center loss function can refer to the description in Example 1, and this embodiment will not be described in detail here.

[0077] 203. Determine the target processing text to be classified.

[0078] 204. Input the target processing text into the trained hierarchical classification neural network model to output a plurality of categories corresponding to the target processing text and having a hierarchical relationship with each other.

[0079] The specific technical details and technical terminology of the above steps 203-204 can be referred to the description of steps 101-102 in the first implementation, and will not be repeated here.

[0080] It can be seen that the method described in the embodiment of the present invention can use the hierarchical classification algorithm model trained based on the central loss function to perform hierarchical classification of text, so as to achieve multi-level classification of text, thereby improving the accuracy and efficiency of text classification, and facilitating the subsequent use of the classified text for further intelligent algorithm processing.

[0081] Embodiment 3

[0082] See also Figure 3 , Figure 3 is a structural diagram of a text level classification device based on center loss disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to a corresponding text processing device, which can be a corresponding processing terminal, processing equipment or processing server, and the server can be a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 3 As shown, the device may include:

[0083] The text determination module 301 is used to determine the target processing text to be classified.

[0084] Optionally, the type of the target text to be processed can be any text type, such as an advertisement text or a literary work text, and the character length of the text is not limited, and can be a word text with only one or two characters, or a large sentence text. Optionally, the language type of the target text to be processed can be Chinese, English, or other language types with mature semantic analysis possibilities, which is not limited by the present invention.

[0085] Optionally, the target processing text may be determined by a preset algorithm, or may be determined by receiving text input by a user, which is not limited in the present invention.

[0086] The text prediction module 302 is used to input the target processing text into the trained hierarchical classification neural network model to output a plurality of categories corresponding to the target processing text and having a hierarchical relationship with each other.

[0087] Specifically, the hierarchical classification neural network model uses a central loss function for optimization during training until the central loss function is minimized, wherein the value of the central loss function is used to represent the training vector distance between a text vector of the training text and a central representation vector of any corresponding training output category.

[0088] Optionally, multiple category hierarchies with hierarchical relationships between each other can be preset, each category hierarchy includes multiple categories, and each category corresponds to a central representation vector. In subsequent training or prediction, the output prediction category belongs to these preset category hierarchies.

[0089] Optionally, the central representation vector can be determined by a text set corresponding to each category or training output category, and the text set can include one or more texts, and the text can be used to represent the characteristics of the category. For example, an adjective category can correspond to a text set including multiple adjective texts. Optionally, the texts in the text set can be converted into text vectors, and the central representation vector can be calculated based on the text vectors. For example, the average value or other statistical value of multiple text vectors can be calculated to obtain the central representation vector.

[0090] Optionally, the hierarchical classification neural network model may include a feature extraction network for extracting text features and a classification network for predicting text categories, wherein preferably, the feature extraction network may adopt a Transformer network architecture, which has excellent effects on the extraction of text features. Optionally, the hierarchical classification neural network model will output multiple training output categories with hierarchical relationships between each other for any training text during training, and calculate the loss function value through the above-mentioned central loss function. In repeated training, the gradient descent method is used to continuously optimize the model parameters until the loss function value is minimized, thereby obtaining a trained hierarchical classification neural network model.

[0091] Optionally, in the specific prediction process of the hierarchical classification neural network model, it can first process the text features of the target processing text, and then, in turn, calculate the vector distance between the text features of the target processing text and the central representation vector of each category of each level of the preset multiple levels from high to low, and determine the category with the smallest corresponding vector distance among all categories in each level as the category corresponding to the target processing text in the level, so as to output multiple categories corresponding to the target processing text that have a hierarchical relationship with each other.

[0092] It can be seen that the method described in the embodiment of the present invention can use the hierarchical classification algorithm model trained based on the central loss function to perform hierarchical classification of text, so as to achieve multi-level classification of text, thereby improving the accuracy and efficiency of text classification, and facilitating the subsequent use of the classified text for further intelligent algorithm processing.

[0093] As an optional implementation, the value of the central loss function is the sum of the training vector distances between the text vector representing the training text and the central representation vector of each corresponding training output category.

[0094] Specifically, the hierarchical classification neural network model will output multiple training output categories with hierarchical relationships among each other for any training text during training.

[0095] It can be seen that this optional implementation measures the accuracy of the prediction results in training by calculating the sum of the training vector distances between the text vector of the training text and the central representation vector of each corresponding training output category, so that a hierarchical classification neural network model with better results can be trained more accurately.

[0096] As an optional implementation, among the multiple categories corresponding to the above-mentioned target processing text that have a hierarchical relationship with each other, the category vector distance between the central representation vector of each upper-level category and the central representation vector of its directly subordinate lower-level category is less than a preset category distance threshold. By setting it in this way, it is possible to design upper and lower-level categories that are more closely related to each other, which helps to make the hierarchical classification of the text more accurate.

[0097] Optionally, the category distance threshold may be inversely proportional to the hierarchical order of the upper category and / or the lower category in all categories. For example, the category distance threshold may be an inversely proportional value of the hierarchical order, such as the category distance threshold may be equal to the ratio of 1 or other preset values ​​to the hierarchical order. For another example, the category distance threshold may also be a negative value of the hierarchical order, such as the category distance threshold may be equal to 1 or other preset values ​​minus the result of the hierarchical order.

[0098] It can be seen that this optional implementation further limits the connection between upper and lower categories by setting the category distance threshold to be inversely proportional to the hierarchical order, thereby determining upper and lower categories that are more closely related to each other and improving the accuracy of text classification.

[0099] As an optional implementation, the value of the center loss function is also used to represent the maximum value between the distance difference corresponding to any upper-level category in all training output categories corresponding to the training text and zero.

[0100] Specifically, the distance difference is the difference between the category vector distance corresponding to the upper category and the category distance threshold, wherein the distance difference has a positive and negative relationship, and is negative when the category vector distance is less than the category distance threshold.

[0101] It can be seen that this optional implementation method is also used to characterize the maximum value between the distance difference corresponding to any upper-level category in all training output categories corresponding to the training text and zero by setting the value of the central loss function, so that when using the central loss function to train the classification network model, the hierarchical connection between multiple prediction categories output by the model can be further limited, thereby improving the accuracy of text classification.

[0102] As an optional implementation, the value of the central loss function is also used to represent the sum of the maximum values ​​between the distance differences corresponding to all upper-level categories in all training output categories corresponding to the training text and zero.

[0103] It can be seen that this optional implementation measures the accuracy of the hierarchical relationship between multiple prediction categories in training by calculating the sum of the distance differences corresponding to all upper categories in all training output categories corresponding to the training text and the maximum value between zero, so as to more accurately train a hierarchical classification neural network model with better results.

[0104] As an optional implementation, the value of the center loss function includes the sum of a first function value and a second function value. The first function value includes the sum of the training vector distances between the text vector of the training text and the center representation vector of each corresponding training output category, and the second function value includes the sum of the maximum value between the distance difference corresponding to all upper-level categories in all training output categories corresponding to the training text and zero.

[0105] Preferably, the value of the central loss function may be the sum of the first function value and the first product value, wherein the first product value is the product of the second function value and the weight parameter. Optionally, the weight parameter may be set by an operator based on empirical values ​​or experimental values, and subsequently adjusted based on actual results.

[0106] It can be seen that by implementing this optional implementation method, the specific architecture of the central loss function can be determined so that the central loss function can fully characterize the two training effects of the target, that is, on the one hand, the output training prediction categories need to be sufficiently accurate to characterize the characteristics of the training text, and on the other hand, the output multiple training prediction categories need to have an accurate hierarchical relationship, so that a hierarchical classification neural network model with better effects can be trained more accurately.

[0107] As an optional implementation, Figure 4 As shown, the device also includes a training module 303, which is used to perform the following steps to train and obtain a hierarchical classification neural network model:

[0108] Determine a training data set including a plurality of training texts;

[0109] The training data set is input into the hierarchical classification neural network model for repeated training until the value of the central loss function is minimized to obtain a trained hierarchical classification neural network model.

[0110] It can be seen that by implementing this optional implementation, a hierarchical classification algorithm model can be obtained through training module 303 based on the central loss function to achieve multi-level classification of text, thereby improving the accuracy and efficiency of text classification, and facilitating the subsequent use of the classified text for further intelligent algorithm processing.

[0111] Embodiment 4

[0112] See also Figure 5 , Figure 5 is a structural diagram of another text level classification device based on center loss disclosed in an embodiment of the present invention. Figure 5 As shown, the device may include:

[0113] A memory 401 storing executable program codes;

[0114] a processor 402 coupled to the memory 401;

[0115] The processor 402 calls the executable program code stored in the memory 401 to execute part or all of the steps in the text level classification method based on center loss disclosed in the first or second embodiment of the present invention.

[0116] Embodiment 5

[0117] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute part or all of the steps in the text level classification method based on center loss disclosed in Embodiment 1 or Embodiment 2 of the present invention.

[0118] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0119] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0120] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, device, and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus, device, and non-volatile computer storage medium will not be repeated here.

[0121] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many improvements to method flows today can be regarded as direct improvements to hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0122] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0123] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0124] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0125] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0126] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0130] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0131] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0132] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0133] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0134] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0135] Finally, it should be noted that the text-level classification method and device based on center loss disclosed in the embodiment of the present invention discloses only the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A text level classification method based on center loss, characterized in that: The method comprises: Determine the target text to be classified; The target processing text is input into a trained hierarchical classification neural network model to output a plurality of categories corresponding to the target processing text and having a hierarchical relationship with each other; the hierarchical classification neural network model is optimized using a central loss function during training until the central loss function is minimized; the value of the central loss function is used to characterize the training vector distance between the text vector of the training text and the central representation vector of any corresponding training output category, wherein the value of the central loss function includes the sum of a first function value and a second function value; the first function value includes the sum of the training vector distances between the text vector of the training text and the central representation vector of each corresponding training output category; the second function value includes the sum of the maximum value between the distance difference corresponding to all upper categories in all the training output categories corresponding to the training text and zero; the distance difference is the difference between the category vector distance between the central representation vector of the upper category and the central representation vector of the subordinate category directly under it and the category distance threshold; There are a plurality of category levels that are in a hierarchical relationship with each other, each of the category levels includes a plurality of categories, and each of the categories corresponds to a central representation vector; The multiple categories corresponding to the target processing text and having a hierarchical relationship with each other belong to a preset multiple category levels that have a hierarchical relationship with each other.

2. The text level classification method based on center loss according to claim 1 is characterized in that: The value of the central loss function is used to represent the sum of the training vector distances between the text vector representing the training text and the central representation vector of each corresponding training output category.

3. The text level classification method based on center loss according to claim 1 is characterized in that: Among the multiple categories that are in a hierarchical relationship with each other, a category vector distance between a central representation vector of each upper category and a central representation vector of a lower category directly under it is less than a preset category distance threshold.

4. The text level classification method based on center loss according to claim 3 is characterized in that: The category distance threshold is inversely proportional to the hierarchical order of the upper category and / or the lower category in all categories.

5. The text level classification method based on center loss according to claim 4 is characterized in that: The value of the central loss function is also used to characterize the maximum value between the distance difference corresponding to any upper-level category in all the training output categories corresponding to the training text and zero; the distance difference is the difference between the category vector distance corresponding to the upper-level category and the category distance threshold.

6. The text level classification method based on center loss according to claim 5 is characterized in that: The value of the central loss function is also used to characterize the sum of the maximum values ​​between the distance differences corresponding to all upper-level categories in all the training output categories corresponding to the training text and zero.

7. The text level classification method based on center loss according to any one of claims 1 to 6, characterized in that: The hierarchical classification neural network model is trained by the following steps: Determine a training data set including a plurality of training texts; The training data set is input into the hierarchical classification neural network model for repeated training until the value of the central loss function is minimized to obtain a trained hierarchical classification neural network model.

8. A text level classification device based on center loss, characterized in that: The device comprises: A text determination module, used to determine the target processing text to be classified; A text prediction module, used for inputting the target processing text into a trained hierarchical classification neural network model to output a plurality of categories corresponding to the target processing text and having a hierarchical relationship with each other; the hierarchical classification neural network model adopts a central loss function for optimization during training until the central loss function is minimized; the value of the central loss function is used to characterize the training vector distance between the text vector of the training text and the central representation vector of any corresponding training output category, wherein the value of the central loss function includes the sum of a first function value and a second function value; the first function value includes the sum of the training vector distances between the text vector of the training text and the central representation vector of each corresponding training output category; the second function value includes the sum of the maximum value between the distance difference corresponding to all upper categories in all the training output categories corresponding to the training text and zero; the distance difference is the difference between the category vector distance between the central representation vector of the upper category and the central representation vector of the subordinate category directly under it and the category distance threshold; There are a plurality of category levels that are in a hierarchical relationship with each other, each of the category levels includes a plurality of categories, and each of the categories corresponds to a central representation vector; The multiple categories corresponding to the target processing text and having a hierarchical relationship with each other belong to a preset multiple category levels that have a hierarchical relationship with each other.

9. A text level classification device based on center loss, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the text level classification method based on center loss as described in any one of claims 1-7.

Images