Text intention classification model training method and device, text intention classification model recognition method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a text intention classification model training method, a recognition method, a device, an electronic device and a storage medium, belonging to the field of data processing. The training method comprises: obtaining the word group of each training text and the labeled label of each training text, obtaining an initial model comprising a graph neural network and a classifier, taking all labeled labels as the iterative input of the graph neural network to enable the graph neural network to learn the correlation between all intention labels, taking the word group of the training text and the correlation as the iterative input of the classifier, training the initial model, and training a classification model for obtaining the labeled label of the training text, so that the trained classification model can adapt to the application scenario of data distribution change, greatly improving the classification accuracy of the classification model.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically, to a text intent classification model training method, recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] Text classification algorithms are one of the most fundamental and essential tasks in natural language processing, widely used in various fields such as sentiment analysis, news classification, and question-answering systems. Intent recognition is an important aspect of text classification, primarily applied in intelligent question-answering systems. It identifies the intent behind a user's question and matches corresponding answers for the user to view.

[0003] With the development of deep learning, intent recognition often employs deep learning-based classification models to predict intent. Currently, there is a hierarchical multi-label text classification method that applies pre-determined associations between different intent labels to the training of a classification model, and then uses the trained model for intent recognition. However, in practical applications, the classification model suffers from significant bias due to the changing data distribution as data accumulates in the production environment. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a text intent classification model training method, recognition method, device, electronic device and storage medium, which can improve the problem of large classification bias in current classification models used for intent recognition.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0006] In a first aspect, embodiments of the present invention provide a method for training a text intent classification model, the method comprising:

[0007] The training corpus is processed to obtain corpus data; wherein, the corpus data includes word groups of each training text and annotation labels of each training text; the annotation labels include multiple label levels with hierarchical relationships and intent labels under each label level, and the annotation labels of each training text include the intent label to which the training text belongs under each label level;

[0008] Obtain an initial model, which includes a graph neural network and a classifier;

[0009] All the labeled tags are used as iterative inputs to the graph neural network, so that the graph neural network learns the association between all the intent tags. The word groups of the training text and the association are used as iterative inputs to the classifier to train the initial model and train a classification model for obtaining the labeled tags of the training text.

[0010] Further, the step of using all the labeled tags as iterative input to the graph neural network to enable the graph neural network to learn the association relationships between all the intent tags, and using the word groups of the training text and the association relationships as iterative input to the classifier to train the initial model includes:

[0011] All the labeled tags are input into the graph neural network so that the graph neural network learns from all the labeled tags and outputs a relation vector, which represents the association between all the intent tags;

[0012] A word group of a training text is multiplied by the relation vector, and the product is input into the classifier so that the classifier classifies the text according to the product and obtains the predicted label of the training text output by the classifier.

[0013] Calculate the loss value between the predicted label and the labeled label of the training text;

[0014] If the loss value does not reach the termination condition, the process returns to the step of inputting all the labeled labels into the graph neural network so that the graph neural network can learn from all the labeled labels and output a relation vector, which represents the association between all the intent labels, so as to continue iterative training of the initial model.

[0015] If the loss value reaches the termination condition, the iteration stops, and the classification model is obtained.

[0016] Furthermore, the initial model further includes a first encoding model and a second encoding model, and the method further includes:

[0017] Each word group of the training text is input into the first encoding model to obtain the word vector of the word group;

[0018] The standard label of each training text is input into the second encoding model to obtain the label vector of the labeled label, wherein the label vector includes the vector of each intent label in the labeled label;

[0019] The step of inputting all the labeled tags into the graph neural network includes:

[0020] Input the label vectors of all the labeled labels into the graph neural network;

[0021] The step of multiplying a word group of a training text with the relation vector includes:

[0022] Multiply the word vectors of a training text's word groups by the relation vectors.

[0023] Secondly, embodiments of the present invention provide a text intent recognition method, the method comprising:

[0024] The target text to be identified is segmented into words to obtain the target word groups of the target text;

[0025] The target word group is input into a pre-trained classification model; wherein the classification model is trained using the text intent classification model training method described in the first aspect;

[0026] The target label to which the target text belongs is obtained through the classification model; wherein, the target label includes the intent label to which the target text belongs at each label level.

[0027] Further, the step of obtaining the target label to which the target text belongs through the classification model includes:

[0028] The target word group is processed by the classification model to obtain the predicted data output by the classification model; wherein, the predicted data includes the probability value of each intent label under each label level;

[0029] Based on the hierarchical relationship between the tags, the target tag to which the target text belongs is obtained from the predicted data.

[0030] Furthermore, the step of obtaining the target label to which the target text belongs from the prediction data based on the hierarchical relationship between label levels includes:

[0031] Among the multiple intent tags in the topmost tag hierarchy, the intent tag with the highest probability value is selected as the first tag;

[0032] From multiple layers of labels in the next layer of the first label, select the intent label with the highest probability value as the second label, and so on, until the intent label with the highest probability value is selected from the bottom layer of the label hierarchy as the final label, to obtain the target label.

[0033] Thirdly, embodiments of the present invention provide a text intent classification model training device, the device comprising a sample acquisition module, a model acquisition module, and a model training module;

[0034] The sample acquisition module is used to process the training corpus to obtain corpus data; wherein, the corpus data includes word groups of each training text and annotation labels of each training text; the annotation labels include multiple label levels with hierarchical relationships and intent labels under each label level, and the annotation labels of each training text include the intent label to which the training text belongs under each label level;

[0035] The model acquisition module is used to acquire an initial model, which includes a graph neural network and a classifier;

[0036] The model training module is used to take all the labeled tags as iterative inputs to the graph neural network, so that the graph neural network learns the association between all the intent tags, and takes the word groups of the training text and the association as iterative inputs to the classifier to train the initial model, thereby training a classification model for obtaining the labeled tags of the training text.

[0037] Fourthly, embodiments of the present invention provide a text intent recognition device, the device comprising a word segmentation module and a recognition module;

[0038] The word segmentation module is used to segment the target text to be identified into words, thereby obtaining the target word groups of the target text;

[0039] The recognition module is used to input the target word group into a pre-trained classification model; wherein the classification model is trained using the text intent classification model training method described in the first aspect;

[0040] The recognition module is further configured to obtain the target label to which the target text belongs through the classification model; wherein the target label includes the intent label to which the target text belongs at each label level.

[0041] Fifthly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the text intent classification model training method as described in the first aspect, or the text intent recognition method as described in the second aspect.

[0042] In a sixth aspect, embodiments of the present invention provide a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the text intent classification model training method as described in the first aspect, or the text intent recognition method as described in the second aspect.

[0043] The text intent classification model training method, recognition method, device, electronic device, and storage medium provided in this invention process the training corpus to obtain word groups for each training text and annotation labels for each training text at each label level, i.e., hierarchical labels. All annotation labels are then used as iterative inputs to the graph neural network in the initial model, enabling the graph neural network to learn the relationships between all intent labels. The learned relationships and word groups of the training text are then used as iterative inputs to the classifier in the initial model for training. The initial model is trained by continuously optimizing the relationships between all intent labels in the graph neural network to improve the classifier's ability to predict the annotation labels of the training text. This allows the trained classification model to adapt to application scenarios with changing data distributions, significantly improving the classification accuracy of the model.

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A block diagram of the text intent classification model training system provided in an embodiment of the present invention is shown.

[0047] Figure 2 A schematic diagram of the label structure provided in an embodiment of the present invention is shown.

[0048] Figure 3 This is one of the flowcharts of the text intent classification model training system provided in an embodiment of the present invention.

[0049] Figure 4 It shows Figure 3 One of the flowcharts for some sub-steps in step S15.

[0050] Figure 5 A schematic diagram of the framework of the initial model provided in an embodiment of the present invention is shown.

[0051] Figure 6 It shows Figure 3 The second flowchart of some sub-steps in step S15.

[0052] Figure 7 A flowchart illustrating the text intent recognition method provided in an embodiment of the present invention is shown.

[0053] Figure 8 It shows Figure 7 A flowchart illustrating some sub-steps of step S22.

[0054] Figure 9 A block diagram of a text intent classification model training device provided in an embodiment of the present invention is shown.

[0055] Figure 10 A block diagram of a text intent recognition device provided in an embodiment of the present invention is shown.

[0056] Figure 11 A block diagram of an electronic device provided in an embodiment of the present invention is shown.

[0057] Figure reference numerals: 100-Text intent classification model training system; 110-Server; 120-Terminal device; 130-Text intent classification model training device; 140-Sample acquisition module; 150-Model acquisition module; 160-Model training module; 170-Text intent recognition device; 180-Word segmentation module; 190-Recognition module; 200-Electronic device. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0059] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0060] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] In knowledge graph-based intelligent question-answering scenarios, user intent recognition directly determines whether the information returned by the dialogue system meets the user's needs. Incorrect intent recognition results in information that fails to meet the user's requirements, leading to a very poor user experience. With the development of deep learning, intent recognition increasingly employs deep learning-based classification models to predict intent.

[0062] In intent classification scenarios, different intent category labels exist in a hierarchical relationship. For example, among the intents "open an account," "new shares," "account opening materials," "account opening progress," "new share listing time," and "new share payment time," the intents "account opening materials" and "account opening progress" belong to the intent "open an account," while the intents "new share listing time" and "new share payment time" belong to the intent "new shares." Different intent labels exhibit implicit and mutually exclusive relationships, similar to a tree structure. Applying these relationships to model training to obtain a classification model is called a hierarchical multi-label text classification method.

[0063] However, current hierarchical multi-label classification methods represent the relationships between hierarchical labels using data distribution during model training, specifically the proportion of each label in the training data. This data distribution is then used as the transition probability between upper and lower level labels, applied with a fixed probability during model training. Classification models trained in this way are only effective for applications where the data distribution remains constant. They are unsuitable for production environments where data distribution changes with continuous data accumulation, leading to significant classification bias. Furthermore, using only data distribution as the basis for label relationships provides overly simplistic information.

[0064] Based on the above considerations, embodiments of the present invention provide a method for training a text intent classification model and a method for recognizing text intent. The trained classification model is applicable to both scenarios where data distribution changes and scenarios where data distribution remains constant, and it can reduce the classification bias of the classification model, thereby improving the accuracy of text intent recognition. The above method is described below.

[0065] The text intent classification model training method provided by the present invention can be applied to the text intent classification model training system 100 shown in the figure. The text intent classification model training system 100 includes a server 110 and a terminal device 120. The server 110 and the terminal device 120 can communicate with each other through wired or wireless connection or through a network.

[0066] Terminal device 120 is used to capture training corpora and to create an initial model, and inputs the training corpora and the created initial model into server 110.

[0067] The initial model includes a graph neural network and a classifier.

[0068] Server 110 is used to process the training corpus to obtain corpus data.

[0069] The corpus data includes word groups for each training text and annotations for each training text. The annotations include multiple hierarchical label levels and intent labels under each label level. The annotations for each training text include the intent label to which the training text belongs at each label level.

[0070] Server 110 is also used to take all labeled tags as iterative input to the graph neural network so that the graph neural network learns the association between all intention tags, and takes the word groups and associations of the training text as iterative input to the classifier to train the initial model and train a classification model to obtain the labeled tags of the training text.

[0071] It should be understood that the training corpus can also be processed through the terminal device 120 to obtain corpus data, and the corpus data can be transmitted to the server 110.

[0072] In this embodiment, the label can be in the form of... Figure 2 In the tree structure shown, "Account Opening" and "New Stocks" are intent tags at the first level of the tag hierarchy. "Track and Field," "Account Opening Materials," "Account Opening Progress," "New Stock Listing Time," and "New Stock Payment Time" are intent tags at the second and third levels of the tag hierarchy. Among them, "Account Opening Materials" and "Account Opening Progress" are subordinate to "Account Opening." "Account Opening - Account Opening Materials" and "New Stocks - New Stock Listing Time" each constitute a separate label.

[0073] Based on the above, in one implementation method, refer to Figure 3 This paper presents a method for training a text intent classification model, which can be applied to... Figure 1 Server 110 in the middle includes the following steps.

[0074] S11, the training corpus is processed to obtain the corpus data.

[0075] The training corpus consists of multiple training texts. The corpus data includes word groups and annotations for each training text. The annotations consist of multiple hierarchical levels and intent labels for each level. Each training text's annotation includes the intent label it belongs to at each level.

[0076] Labels can be formatted as "Account Opening - Account Opening Materials".

[0077] S13, Obtain the initial model.

[0078] The initial model includes a graph neural network and a classifier.

[0079] S15, all labeled tags are used as iterative input to the graph neural network so that the graph neural network learns the relationship between all intention tags, and the word groups and relationships of the training text are used as iterative input to the classifier to train the initial model and train a classification model to obtain the labeled tags of the training text.

[0080] In this embodiment, the association relationships include the hierarchical relationships between intent tags and the transition probability matrix between tag levels. Based on the predicted data obtained in S11, the initial model is iteratively trained to obtain a classification model.

[0081] For example, after processing the training data, the resulting corpus data can be in the form of "(S,l k ,l ki ),[[l1,…,l n ],[l 11 ,…,l 1i ],[l 21 ,…,l 2i ]...]".

[0082] Among them, [[l1,…,l n ],[l 11 ,…,l 1i ],[l 21 ,…,l 2i [...] is a tree-structured tag hierarchy composed of all intent tags, [l1,...,l...]n [l] represents the first level of label hierarchy and the intent labels within it. 11 ,...,l 1i [] represents the second layer of label hierarchy and the intent labels within it. k∈n, S represents the training text, l k This represents the intent label to which the training text S belongs in the first layer of the label hierarchy, l ki This represents the intent label to which the training text S belongs at the k-th label level.

[0083] Each iteration of the initial model involves inputting all labeled data into a Graph Neural Network (GNN) to enable the GNN to learn the hierarchical relationships between all intent labels and the transition probability matrices between label levels, thus constructing a hierarchical label representation model. Furthermore, each iteration of the initial model also inputs a word group from a training text, along with the relationships learned by the GNN in that iteration, into a classifier. The classifier then outputs the predicted labels for the training text.

[0084] Through continuous iteration, the graph neural network continuously optimizes the hierarchical relationship between intent labels and the transition probability matrix (i.e., the association relationship) between label levels, and learns the optimal transition probability matrix as much as possible. In this way, it can train the classifier to predict the labeled labels of the training text based on the continuously optimized association relationship between intent labels, that is, the classification accuracy of the classifier.

[0085] Therefore, compared with the existing training methods of hierarchical multi-label classification models, the above-mentioned text intent classification model training method can train the classifier to predict the labeled labels of training text based on the continuously optimized correlation between intent labels (hierarchical relationship between intent labels and transition probability matrix between label levels). This enables the trained classification model to adapt to application scenarios with changing data distribution, greatly improving the classification accuracy of the classification model.

[0086] Furthermore, the text intent classification model training method provided in this embodiment of the invention includes the hierarchical relationship between intent labels and the transition probability matrix between label levels, which provides richer association information and helps improve the prediction accuracy of the classification model.

[0087] Furthermore, in one possible implementation, to ensure the trained classification model achieves the required accuracy, a loss value can be introduced during model training to determine the model's maturity. Specifically, refer to... Figure 4 Step S15 above includes the following steps.

[0088] S152, input all labeled data into the graph neural network so that the graph neural network can learn from all the label vectors and output a relation vector.

[0089] Among them, the relationship vector represents the association between all intent tags.

[0090] S154 multiplies the word groups and relation vectors of a training text, and inputs the product into the classifier so that the classifier can classify the text according to the product and obtain the predicted label of the training text output by the classifier.

[0091] The word groups in the training text are randomly selected from all word groups in the training text. It should be noted that the multiplication here refers to matrix multiplication.

[0092] S156, calculate the loss value between the predicted label and the labeled label of the training text.

[0093] S158, determine if the loss value has reached the termination condition. If yes, stop the iteration and obtain the classification model. If not, return to step S152 to continue iterative training of the initial model.

[0094] It should be understood that ending training when the loss value reaches the termination condition is only one implementation method for ending training. Other implementation methods may include stopping training when the number of iterations reaches a preset number. In this implementation method, the condition for ending training is not limited to a single one.

[0095] To enable the graph neural network to learn and establish semantic hierarchical relationships among all intent labels during iteration, thereby further expanding the applicability and improving the accuracy of the classification model, in one embodiment, the initial model provided by this invention further includes a first encoding model and a second encoding model. Both the first and second encoding models can be pre-trained Albert Tiny models. In this case, the structure of the initial model can be as follows: Figure 5 As shown.

[0096] To further improve the classification accuracy of the model, semantic information (language vectors) from the training text and labeled tags is incorporated into the model training process, building upon the initial model described above. For more details, refer to... Figure 6 Step S15 also includes the following sub-steps.

[0097] S150: Input the word groups of each training text into the first encoding model to obtain the word vectors of the word groups.

[0098] S151, input the standard label of each training text into the second encoding model to obtain the label vector of the labeled text.

[0099] The label vector includes the vector of each intent label in the annotation label.

[0100] The word vector is the vector representation of the word group in the training text, and the label vector is the vector representation of the labeled label in the training text.

[0101] In other implementations, the first encoding model and the second encoding model may also be other models capable of obtaining semantic vectors.

[0102] For example, the training text S = (w1, w2, ... w i After inputting the first encoding model AlbertTiny1, the output word vector is: (e s1 e s2 , ...e si )=AlbertTiny1(w1,w2,...w i ), where w i Let w1, w2, ... w be a word obtained after segmenting the training text S. i ) represents a word group in the training text S, e s1 The vector representation of word w1, (e s1 ,es2,...e si ) is a word group (w1, w2, ... w i The vector representation of ), i.e., word vector.

[0103] All labels (l1, l2, ... l) i After inputting the second encoding model AlbertTiny2, the output label vector is: (e l1 e l2 , ...e li )=AlbertTiny2(l1, l2,...l i (Where, l1 is a label, e) l1 This is the vector representation of the label l1, i.e., the label vector.

[0104] Building upon this, step S152, "inputting all labeled vectors into the graph neural network," is further implemented as "inputting the label vectors of all labeled vectors into the graph neural network." At this point, in step S132, all label vectors are input into the graph neural network so that the graph neural network learns from all label vectors and outputs a relation vector.

[0105] Similarly, step S154 above, "multiplying the word group and relation vector of a training text", can be further implemented as "multiplying the word vector and relation vector of the word group of a training text". In this case, S154 multiplies the word vector and relation vector of the word group of a training text, and inputs the product result into the classifier so that the classifier can classify according to the product result and obtain the predicted label of the training text output by the classifier.

[0106] For example, all label vectors are input into a graph neural network (GNN). After learning the hierarchical relationships between intention labels and the transition probability matrices between label levels, the GNN outputs an n-dimensional relationship vector E. ln E ln =GNN(e l1 e l2 , ...e li Then, the word vectors (e) of the training text S are... s1 e s2 , ...e si ) and relation vector E ln Multiplication (i.e., matrix multiplication) transforms the relation vector E ln As weights for intent classification, the classifier constructs the relationship between the training text and the labeled text, and outputs the predicted label of the training text S predicted by the classifier.

[0107] Among them, the predicted label of the training text S predicted by the output classifier Clssifier. It can be represented as:

[0108] In one implementation, the binary cross-entropy loss function (BCE function) is used to calculate the loss value between the predicted label and the labeled label of the training text S.

[0109] The loss value can be expressed as:

[0110] Where C represents the number of intent tags, y (i) The annotation labels represent the training text. This indicates the predicted label.

[0111] After training the classification model, a deep learning model optimization framework can be used to train the model to obtain an optimized classification model.

[0112] The final classification model obtained by training the text intent classification model using the above method can include a first encoding model (Albert Tiny model), a graph neural network, and a classifier.

[0113] The text intent classification model training method provided in this invention introduces the vectorized label vectors (intent label semantic information) of all labeled tags into a graph neural network to establish the association between intent tags. Through the propagation of intent label semantic information from the label hierarchy, the establishment of associations between intent tags becomes more accurate. Simultaneously, the association between intent tags is obtained through a graph neural network (GNN) using a self-learning method. This allows the GNN to learn the transition probability matrix between label levels and the hierarchical relationship between intent tags, thereby obtaining the optimal transition probability matrix parameters and ultimately learning the optimal association between intent tags.

[0114] The text intent classification model training method provided in this invention continuously optimizes the association relationship and trains the classifier's ability to predict the labeled labels of the training text based on the continuously optimized association relationship of all intent labels in the graph neural network. This enables the trained classification model to adapt to application scenarios with changing data distribution, greatly improving the classification accuracy of the classification model.

[0115] In one implementation, reference Figure 7 This invention also provides a text intent recognition method, which can be applied to electronic devices and includes the following steps.

[0116] S21, segment the target text to be identified into words to obtain the target word groups of the target text.

[0117] S22, input the target word group into the pre-trained classification model.

[0118] The classification model was trained using the text intent classification model training method described above.

[0119] After the server trains the initial model using the aforementioned text intent classification model training method and reaches the training termination condition, it can retain the model parameters from the last iteration and send the model with retained parameters to the electronic device. The electronic device receives this model and uses it as the classification model.

[0120] S23, using a classification model, obtain the target label to which the target text belongs.

[0121] Among them, target labels include the intent labels to which the target text belongs at each label level.

[0122] For example, after the target word group is input into a pre-trained classification model, the Albert Tiny model encodes the target word group to obtain its vector representation, which includes the vector representation of each word. This vector representation is then multiplied by the association between intent labels constructed by the graph neural network (which can be a hierarchical label representation model). The classifier then performs classification based on the product of the target word group's vector representation and the association between the intent labels, outputting a predicted label. This predicted label is then processed to obtain the target label to which the target text belongs.

[0123] The final target label can be in the form of "Account Opening - Account Opening Materials" or similar formats.

[0124] It should be noted that when the classification model includes the Albert Tiny model, the word segmentation function of the Albert Tiny model can be directly used to segment the target text. In this case, the target text can be directly input into the classification model.

[0125] Through steps S21-S23 above, a classification model with low classification bias and applicability to various scenarios is used to identify the intent of the target text, resulting in more accurate target labels with low bias. This also improves the user experience. Furthermore, the target labels include the intent labels belonging to the target text at each label level, making the obtained target labels more precise.

[0126] Furthermore, in order to select the optimal group of intent labels as target labels from the predicted data, a hierarchical relationship of labels can be introduced during the acquisition of target labels. (See reference...) Figure 8 Step S22 above may include the following sub-steps.

[0127] S221, The target word group is processed by a classification model to obtain the predicted data output by the classification model.

[0128] The prediction data includes the probability value of each intent label at each label level.

[0129] S222, based on the hierarchical relationship between the label levels, obtains the target label to which the target text belongs from the prediction data.

[0130] After obtaining the predicted data, the target label can be obtained by processing the topmost label hierarchy. For example, among the multiple intent labels in the topmost label hierarchy, the intent label with the highest probability value is selected as the first label. Then, from the multiple labels in the next lower label hierarchy, the intent label with the highest probability value is selected as the second label. The remaining predicted data is processed in the same way until the intent label with the highest probability value is selected from the bottommost label hierarchy as the final label. The selected first label, second label, ..., and final label are then used as the target label.

[0131] For example, if the predicted data is [[Account opening -0.9, new shares -0.1, [Account opening materials -0.9, account opening progress -0.1], then the final target label will be "Account opening - account opening progress".

[0132] Based on the above-mentioned text intent classification model training method concept, in one implementation, referring to Figure 9 A text intent classification model training device 130 is provided, including a sample acquisition module 140, a model acquisition module 150 and a model training module 160.

[0133] The sample acquisition module 140 is used to process the training corpus to obtain corpus data.

[0134] The corpus data includes word groups for each training text and annotations for each training text. The annotations include multiple hierarchical label levels and intent labels under each label level. The annotations for each training text include the intent label to which the training text belongs at each label level.

[0135] The model acquisition module 150 is used to acquire the initial model.

[0136] The initial model includes a graph neural network and a classifier.

[0137] The model training module 160 is used to take all the labeled tags as iterative inputs to the graph neural network so that the graph neural network learns the association between all the intent tags, and takes the word groups and associations of the training text as iterative inputs to the classifier to train the initial model and train a classification model for obtaining the labeled tags of the training text.

[0138] Through the text intent classification model training device 130 described above, and through the synergistic effect of the sample acquisition module 140 and the model training module 160, the association relationship of all intent labels continuously optimized by the graph neural network, as well as the word groups of the training text, is used to train the classifier's ability to predict the labeled labels of the training text. This enables the trained classification model to adapt to application scenarios with changing data distribution, greatly improving the classification accuracy of the classification model.

[0139] Specific limitations regarding the text intent classification model training device 130 can be found in the limitations of the text intent classification model training method described above, and will not be repeated here. Each module in the aforementioned text intent classification model training device 130 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the electronic device 200 in hardware form, or stored in the memory of the electronic device 200 in software form, so that the processor can call and execute the operations corresponding to each module.

[0140] Based on the above-mentioned text intent recognition method concept, in one implementation, referring to Figure 10 A text intent recognition device 170 is provided, including a word segmentation module 180 and a recognition module 190.

[0141] The word segmentation module 180 is used to segment the target text to be identified into words, thereby obtaining the target word groups of the target text.

[0142] The recognition module 190 is used to input the target word group into a pre-trained classification model.

[0143] The classification model was trained using the same method described above for training text intent classification models.

[0144] The recognition module 190 is also used to obtain the target label to which the target text belongs through a classification model.

[0145] Target tags include the intent tags to which the target text belongs at each tag level.

[0146] In the aforementioned text intent recognition device 170, through the combined action of the classification module and the recognition module 190, a classification model with small classification bias and applicability to various scenarios is used to recognize the intent of the target text, resulting in target labels with small bias and greater accuracy. This also improves the user experience. Furthermore, the target labels include the intent labels belonging to the target text at each label level, making the obtained target labels more precise.

[0147] For specific limitations regarding the text intent recognition device 170, please refer to the limitations of the text intent recognition method above, which will not be repeated here. Each module in the aforementioned text intent recognition device 170 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the electronic device, or stored in software in the memory of the electronic device, so that the processor can call and execute the operations corresponding to each module.

[0148] In one embodiment, an electronic device 200 is provided, which may be a server 110, and its internal structure diagram may be as follows. Figure 11 As shown, the electronic device 200 includes a processor, memory, and network interface connected via a system bus. The processor of the electronic device 200 provides computational and control capabilities. The memory of the electronic device 200 includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a database, and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the electronic device 200 stores training data for a text intent classification model or text intent recognition data. The network interface of the electronic device 200 is used for network connection communication with external terminals. When the computer program is executed by the processor, it implements a text intent classification model training method or a text intent recognition method.

[0149] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device 200 to which the present invention is applied. The specific electronic device 200 may include, but is not limited to, the following: Figure 11 The diagram shows more or fewer components, or combinations of certain components, or different component arrangements.

[0150] In one embodiment, the text intent classification model training device 130 provided by the present invention can be implemented as a computer program, which can be implemented in the form of, for example... Figure 11 The electronic device 200 shown operates on this device. The memory of the electronic device 200 can store the various program modules that make up the text intent classification model training device 130, for example, Figure 9 The sample acquisition module 140, model acquisition module 150, and model training module 160 are shown. The computer program comprised of these modules causes the processor to execute the steps described in this specification for training a text intent classification model.

[0151] For example, Figure 11 The electronic device 200 shown can be used as follows Figure 9 The sample acquisition module 140 in the text intent classification model training device 130 shown executes step S11. The electronic device 200 can execute step S13 through the model acquisition module 150. The electronic device 200 can execute step S15 through the model training module 160.

[0152] In one embodiment, an electronic device 200 is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: processing training corpus to obtain corpus data; wherein the corpus data includes word groups of each training text and annotation labels of each training text; the annotation labels include multiple label hierarchies with hierarchical relationships and intent labels under each label hierarchy, and the annotation labels of each training text include the intent label to which the training text belongs under each label hierarchy; obtaining an initial model, the initial model including a graph neural network and a classifier; using all annotation labels as iterative inputs to the graph neural network to enable the graph neural network to learn the association relationships between all intent labels, and using the word groups and association relationships of the training text as iterative inputs to the classifier to train the initial model, thereby training a classification model for obtaining the annotation labels of the training text.

[0153] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program performs the following steps: processing training corpus to obtain corpus data; wherein the corpus data includes word groups of each training text and annotation labels of each training text; the annotation labels include multiple label hierarchies with hierarchical relationships and intent labels under each label hierarchy, and the annotation labels of each training text include the intent label to which the training text belongs under each label hierarchy; obtaining an initial model, the initial model including a graph neural network and a classifier; using all annotation labels as iterative inputs to the graph neural network to enable the graph neural network to learn the association relationships between all intent labels, and using the word groups and association relationships of the training text as iterative inputs to the classifier to train the initial model, thereby training a classification model for obtaining the annotation labels of the training text.

[0154] In one embodiment, the text intent recognition device 170 provided by the present invention can be implemented as a computer program, which can be implemented in the form of, for example... Figure 11 The electronic device 200 shown operates on this device. The memory of the electronic device 200 can store the various program modules that make up the text intent classification model training device 130, for example, Figure 10 The word segmentation module 180 and the recognition module 190 are shown. The computer program, consisting of these modules, causes the processor to execute the steps described in this specification for the text intent recognition method.

[0155] For example, Figure 11 The electronic device 200 shown can be used as follows Figure 10 The word segmentation module 180 in the text intent recognition device 170 shown executes step S21. The electronic device 200 can execute steps S22 and S23 through the recognition module 190.

[0156] In one embodiment, an electronic device 200 is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: segmenting the target text to be identified to obtain target word groups of the target text; inputting the target word groups into a pre-trained classification model; wherein the classification model is trained using the text intent classification model training method described above; and obtaining the target label to which the target text belongs through the classification model; wherein the target label includes the intent label to which the target text belongs at each label level.

[0157] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: segmenting the target text to be identified to obtain target word groups of the target text; inputting the target word groups into a pre-trained classification model; wherein the classification model is trained using the text intent classification model training method described above; and obtaining the target label to which the target text belongs through the classification model; wherein the target label includes the intent label to which the target text belongs at each label level.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0159] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0160] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0161] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training a text intent classification model, characterized in that, The method includes: The training corpus is processed to obtain corpus data; wherein, the corpus data includes word groups of each training text and annotation labels of each training text; the annotation labels include multiple label levels with hierarchical relationships and intent labels under each label level, and the annotation labels of each training text include the intent label to which the training text belongs under each label level; Obtain an initial model, which includes a graph neural network and a classifier; All the labeled tags are used as iterative inputs to the graph neural network, so that the graph neural network learns the association between all the intent tags. The word groups of the training text and the association are used as iterative inputs to the classifier to train the initial model and train a classification model for obtaining the labeled tags of the training text.

2. The text intent classification model training method according to claim 1, characterized in that, The step of training the initial model by using all the labeled tags as iterative input to the graph neural network to enable the graph neural network to learn the association relationships between all the intent tags, and using the word groups of the training text and the association relationships as iterative input to the classifier, includes: All the labeled tags are input into the graph neural network so that the graph neural network learns from all the labeled tags and outputs a relation vector, which represents the association between all the intent tags; A word group of a training text is multiplied by the relation vector, and the product is input into the classifier so that the classifier classifies the text according to the product and obtains the predicted label of the training text output by the classifier. Calculate the loss value between the predicted label and the labeled label of the training text; If the loss value does not reach the termination condition, the process returns to the step of inputting all the labeled labels into the graph neural network so that the graph neural network can learn from all the labeled labels and output a relation vector, which represents the association between all the intent labels, so as to continue iterative training of the initial model. If the loss value reaches the termination condition, the iteration stops, and the classification model is obtained.

3. The text intent classification model training method according to claim 2, characterized in that, The initial model further includes a first encoding model and a second encoding model, and the method further includes: Each word group of the training text is input into the first encoding model to obtain the word vector of the word group; The standard label of each training text is input into the second encoding model to obtain the label vector of the labeled label, wherein the label vector includes the vector of each intent label in the labeled label; The step of inputting all the labeled tags into the graph neural network includes: Input the label vectors of all the labeled labels into the graph neural network; The step of multiplying a word group of a training text with the relation vector includes: Multiply the word vectors of a training text's word groups by the relation vectors.

4. A text intent recognition method, characterized in that, The method includes: The target text to be identified is segmented into words to obtain the target word groups of the target text; The target word group is input into a pre-trained classification model; wherein the classification model is trained using the text intent classification model training method as described in any one of claims 1 to 3; The target label to which the target text belongs is obtained through the classification model; wherein, the target label includes the intent label to which the target text belongs at each label level.

5. The text intent recognition method according to claim 4, characterized in that, The step of obtaining the target label to which the target text belongs through the classification model includes: The target word group is processed by the classification model to obtain the predicted data output by the classification model; wherein, the predicted data includes the probability value of each intent label under each label level; Based on the hierarchical relationship between the tags, the target tag to which the target text belongs is obtained from the predicted data.

6. The text intent recognition method according to claim 5, characterized in that, The step of obtaining the target label to which the target text belongs from the predicted data based on the hierarchical relationship between label levels includes: Among the multiple intent tags in the topmost tag hierarchy, the intent tag with the highest probability value is selected as the first tag; From multiple layers of labels in the next layer of the first label, select the intent label with the highest probability value as the second label, and so on, until the intent label with the highest probability value is selected from the bottom layer of the label hierarchy as the final label, to obtain the target label.

7. A text intent classification model training device, characterized in that, The device includes a sample acquisition module, a model acquisition module, and a model training module; The sample acquisition module is used to process the training corpus to obtain corpus data; wherein, the corpus data includes word groups of each training text and annotation labels of each training text; the annotation labels include multiple label levels with hierarchical relationships and intent labels under each label level, and the annotation labels of each training text include the intent label to which the training text belongs under each label level; The model acquisition module is used to acquire an initial model, which includes a graph neural network and a classifier; The model training module is used to take all the labeled tags as iterative inputs to the graph neural network, so that the graph neural network learns the association between all the intent tags, and takes the word groups of the training text and the association as iterative inputs to the classifier to train the initial model, thereby training a classification model for obtaining the labeled tags of the training text.

8. A text intent recognition device, characterized in that, The device includes a word segmentation module and a recognition module; The word segmentation module is used to segment the target text to be identified into words, thereby obtaining the target word groups of the target text; The recognition module is used to input the target word group into a pre-trained classification model; wherein the classification model is trained using the text intent classification model training method as described in any one of claims 1 to 3; The recognition module is further configured to obtain the target label to which the target text belongs through the classification model; wherein the target label includes the intent label to which the target text belongs at each label level.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that can be executed by the processor, the processor being able to execute the computer program to implement the text intent classification model training method as described in any one of claims 1 to 3, or the text intent recognition method as described in any one of claims 4 to 6.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the text intent classification model training method as described in any one of claims 1 to 7, or the text intent recognition method as described in any one of claims 4 to 6.

Images