Methods, apparatus, devices, and storage media for Prompt structure intent recognition based on clustering

Abstract

This invention provides a method, apparatus, device, and storage medium for intent recognition based on a clustering-based Prompt structure. By constructing a zero-sample Prompt structure, it eliminates the need for extensive text intent labeling, thus saving on the cost of dialogue intent recognition. Furthermore, the Prompt structure construction process utilizes clustering to discover the intent center text, automatically building the necessary prompt structure for intent recognition. This avoids the fluctuations in intent prediction results that occur when manually constructing prompt structures.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a method, apparatus, device, and storage medium for identifying Prompt structure intents by combining clustering. Background Technology

[0002] In human-computer dialogue, numerous methods have been proposed for representing the dialogue text in each round. Currently, commonly used methods typically involve unsupervised training of a language model on a large-scale corpus, followed by parameter fine-tuning of the trained model to achieve text representation for intent classification tasks. Using a pre-trained language model with parameter fine-tuning requires acquiring a large amount of labeled data from downstream tasks, resulting in significant time and manpower costs. While using a pre-trained language model in conjunction with a prompt structure can capture user intent with zero samples, creating the prompt structure requires considerable human experience and may lead to fluctuations in intent prediction results. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, device, and storage medium for identifying Prompt structures by combining clustering.

[0004] This invention provides a Prompt structure intent recognition method combining clustering, which includes the following steps:

[0005] Obtain the dialogue text, perform unsupervised model training on the dialogue text, and obtain a pre-trained language model;

[0006] The dialogue text is clustered, and the cluster center is obtained based on the clustering results. The text closest to the cluster center is then selected as the intent center text.

[0007] A Prompt template is constructed based on the intent center text. The dialogue text is input into the Prompt template slot, and the Prompt construction text is output.

[0008] The Prompt construct text is used by the pre-trained language model to determine whether the intent of the input dialogue text and the intent-centered text are consistent.

[0009] As a further improvement of the present invention, the step of obtaining dialogue text and performing unsupervised model training on the dialogue text to obtain a pre-trained language model further includes:

[0010] Based on the dialogue text, sentence templates are constructed to automatically generate training corpus.

[0011] A pre-trained language model is obtained by training the dialogue text and the training corpus.

[0012] As a further improvement of the present invention, the step of performing unsupervised model training on the training dialogue text to obtain a pre-trained language model specifically includes:

[0013] The pre-trained language model is obtained by training the dialogue text using BERT, ELECTRA, and GPT models.

[0014] As a further improvement of the present invention, the clustering process of the dialogue text specifically includes:

[0015] The dialogue text was clustered using the K-means clustering algorithm, the cluster center K value was selected, and the text clusters were divided.

[0016] The text closest to the cluster center K value in each text cluster is selected as the representative text in that text cluster, and the representative texts are combined to generate the intent center text.

[0017] As a further improvement of the present invention, the selection of the cluster center K value specifically includes:

[0018] Calculate the silhouette coefficient of the cluster centers, and select the optimal cluster center K value based on the size of the silhouette coefficient.

[0019] As a further improvement of the present invention, the clustering process of the dialogue text further includes:

[0020] For representative texts in the aforementioned text clusters, the intent-centered text is formed by manually identifying the text category.

[0021] The present invention also provides a Prompt structure intent recognition device that combines clustering, comprising:

[0022] The model training module is configured to acquire dialogue text, perform unsupervised model training on the dialogue text, and obtain a pre-trained language model.

[0023] A clustering module is configured to perform clustering processing on the dialogue text, obtain cluster centers based on the clustering results, and obtain the text closest to the cluster center as the intent center text.

[0024] The Prompt template construction module is configured to construct a Prompt template based on the intent center text, input the dialogue text into the Prompt template slot, and output the Prompt construction text.

[0025] The judgment module is configured to use the pre-trained language model to determine whether the intent of the input dialogue text and the intent-centered text are consistent.

[0026] The present invention also provides an electrical device comprising:

[0027] Memory, used to store executable instructions;

[0028] When a processor runs executable instructions stored in the memory, it implements the above-described Prompt structure intent recognition method that combines clustering.

[0029] The present invention also provides a refrigerator, comprising:

[0030] Memory, used to store executable instructions;

[0031] When a processor runs executable instructions stored in the memory, it implements the above-described Prompt structure intent recognition method that combines clustering.

[0032] The present invention also provides a computer-readable storage medium storing executable instructions, wherein the executable instructions, when executed by a processor, implement the above-described Prompt structure intent recognition method combining clustering.

[0033] The beneficial effects of this invention are: by constructing a zero-sample Prompt structure, it eliminates the need for extensive text intent labeling, thus saving on the cost of dialogue intent recognition. Furthermore, the Prompt structure construction process utilizes clustering methods to discover the intent center text, automatically building the prompt structure required for intent recognition. This avoids the fluctuations in intent prediction results that occur when manually constructing prompt structures. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the steps of the Prompt structure intent recognition method combining clustering in one embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram illustrating the steps of clustering dialogue text in one embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0037] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0038] This embodiment provides a clustering-based Prompt structure intent recognition method. Prompt learning, or cue learning, can add extra text to the language model input segment by constructing a Prompt template without significantly altering the structure and parameters of the pre-trained language model, thereby increasing cue information and enabling intent recognition for downstream tasks. In this embodiment, by clustering the dialogue text, intent center text is automatically generated, and a semantically consistent Prompt template is automatically constructed based on the intent center text. This avoids the fluctuation problem in intent prediction results that occurs when manually constructing the prompt structure. This embodiment uses a smart refrigerator as an example to illustrate the method; its application in other devices can be referenced in this embodiment.

[0039] In the intelligent application of electrical appliances such as refrigerators, text classification is a common natural language processing task. It requires the ability to correctly determine the intent of the input text data. Intent determination involves identifying the user's specific usage intent based on the input corpus data. For example, the input corpus data can be divided into several major usage intent categories, each with corresponding training data to train the intent classification model. In this embodiment, for a smart refrigerator, usage intents may include: recipe lookup, refrigerator food reminders, music playback, news broadcasts, etc.

[0040] like Figure 1 As shown, the Prompt structure intent recognition method combining clustering includes the following steps:

[0041] S1: Obtain the dialogue text and perform unsupervised model training on the dialogue text to obtain a pre-trained language model.

[0042] S2: Perform clustering processing on the dialogue text, obtain cluster centers based on the clustering results, and obtain the text closest to the cluster center as the intent center text.

[0043] S3: Construct a Prompt template based on the intent center text, input the dialogue text into the Prompt template slot, and output the Prompt construction text.

[0044] S4: The Prompt construction text is used to determine whether the intent of the input dialogue text and the intent center text are consistent through the pre-trained language model.

[0045] Step S1 specifically includes:

[0046] The pre-trained language model is obtained by training the dialogue text using BERT, ELECTRA, and GPT models.

[0047] The dialogue text mentioned here refers to the transcribed text of questions or instructions spoken by the user to a smart electronic device or a client terminal device connected to the smart electronic device. For example, in this embodiment, the user may ask questions such as "What vegetables are in the refrigerator today?" or "What recipes are recommended today?", or issue commands such as "Remind me of the yogurt that's about to expire in the refrigerator" or "Give me seasonal fruits." Based on the above information, the processor of the smart refrigerator performs voice recognition using the method provided by this invention to determine the user's intention.

[0048] Unsupervised model training involves learning the ability to extract, represent, and predict data features from unlabeled data, thereby indirectly achieving data augmentation. Unsupervised model training, based on pre-training on large-scale text databases, can effectively learn general language representations, which is beneficial for downstream application tasks. Furthermore, unsupervised language models can provide better model initialization methods, resulting in models with better generalization capabilities and accelerating the convergence speed of the target task.

[0049] Depending on the language model used, different text preprocessing steps are included before model training. For example, when using the BERT model, the dialogue text is first processed into text data in the BERT model input format, and then the text data is padded to make each text data piece have a consistent length.

[0050] In other embodiments of the present invention, common unsupervised training models may also be used, and the text preprocessing steps may be adjusted according to the model type.

[0051] Furthermore, in some embodiments of the present invention, step S1 further includes:

[0052] Based on the dialogue text, sentence templates are constructed to automatically generate training corpus.

[0053] A pre-trained language model is obtained by training the dialogue text and the training corpus.

[0054] In practical applications, due to the limited amount of real user dialogue text or the uneven distribution of text across different intent categories, it is necessary to automatically generate corpus data as supplementary text data for model training. Existing technologies can be referenced for corpus generation methods, which will not be elaborated upon here.

[0055] By training a language model using real-world user dialogue text and a large-scale corpus generated from it, it is possible to...

[0056] like Figure 2 As shown, step S2 specifically includes:

[0057] S21: Cluster the dialogue text using the K-means clustering algorithm, select the optimal cluster center K value using the silhouette coefficient, and divide the text into clusters.

[0058] S22: Select the text in each text cluster class that is closest to the optimal cluster center K value as the representative text in that text cluster class, and combine the representative texts to generate the intent center text.

[0059] The intent-centered text mentioned here is the constructed text that best reflects the intent of a type of dialogue text, and it can be regarded as representing the intent of this type of dialogue text.

[0060] By clustering the dialogue text and taking the text with the closest K value to the cluster center as the most representative text in that text cluster, it is possible to automatically generate intent center text that reflects the intent of the dialogue text for a large amount of text data. The generation method is simple, efficient and accurate. In the subsequent process, the intent center text is constructed as a Prompt template, which can effectively avoid the fluctuation problem of intent prediction results caused by manually constructing Prompt templates.

[0061] Specifically, step S21 includes: preprocessing the dialogue text by segmenting words and removing stop words, calculating text features and constructing a vector space model, and using the K-means algorithm for clustering.

[0062] Furthermore, in step S21, the optimal cluster center K value is selected by calculating the contour coefficient of the cluster center and, based on the magnitude of the contour coefficient, calculating the maximum value of the contour coefficient.

[0063] Furthermore, in step S2, the clustering process of the dialogue text further includes:

[0064] For representative texts in the aforementioned text clusters, the intent-centered text is formed by manually identifying the text category.

[0065] For the clustered text, the text category is determined manually to further improve the text accuracy of the intent-centered text.

[0066] In other embodiments of the present invention, the dialogue text can also be clustered using other commonly used clustering algorithms.

[0067] In step S3, a Prompt template is constructed based on the intent-centered text. Here, we will illustrate this with a specific example. The intent-centered text obtained in step S2 is "Put the eggplant in the refrigerator." The constructed Prompt template is: [X] Is it consistent with "Put the eggplant in the refrigerator?" [MASK]. Here, [X] is the input dialogue text, and [MASK] represents the prediction result, i.e., whether it is consistent or inconsistent.

[0068] For the dialogue text "Add potatoes to the refrigerator", inputting it into the above Prompt template will output the Prompt construction text "Is adding potatoes to the refrigerator the same as putting eggplants in the refrigerator? [MASK]". Inputting the Prompt construction text into a pre-trained language model for sentence intent matching and outputting whether it is consistent or inconsistent allows for the determination of the intent of the dialogue text based on the intent-centered text.

[0069] Based on the same inventive concept, the present invention also provides a Prompt structure intent recognition device that combines clustering, comprising:

[0070] The model training module is configured to acquire dialogue text, perform unsupervised model training on the dialogue text, and obtain a pre-trained language model.

[0071] A clustering module is configured to perform clustering processing on the dialogue text, obtain cluster centers based on the clustering results, and obtain the text closest to the cluster center as the intent center text.

[0072] The Prompt template construction module is configured to construct a Prompt template based on the intent center text, input the dialogue text into the Prompt template slot, and output the Prompt construction text.

[0073] The judgment module is configured to use the pre-trained language model to determine whether the intent of the input dialogue text and the intent-centered text are consistent.

[0074] Based on the same inventive concept, the present invention also provides an electrical device, comprising:

[0075] Memory, used to store executable instructions;

[0076] When a processor runs executable instructions stored in the memory, it implements the above-described Prompt structure intent recognition method that combines clustering.

[0077] Based on the same inventive concept, the present invention also provides a refrigerator, which includes:

[0078] Memory, used to store executable instructions;

[0079] When a processor runs executable instructions stored in the memory, it implements the above-described Prompt structure intent recognition method that combines clustering.

[0080] Based on the same inventive concept, the present invention also provides a computer-readable storage medium storing executable instructions, characterized in that the executable instructions, when executed by a processor, implement the above-described Prompt structure intent recognition method combining clustering.

[0081] In summary, this implementation method, by constructing a zero-sample Prompt structure, can identify the intent of text without requiring extensive text intent labeling, thus saving on the cost of dialogue intent recognition. Furthermore, the Prompt structure construction process uses clustering methods to discover the intent center text, automatically building the prompt structure required for intent recognition. This avoids the fluctuations in intent prediction results that occur when manually constructing prompt structures.

[0082] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0083] The detailed descriptions listed above are merely specific descriptions of feasible implementations of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent implementations or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for prompt structure intent recognition combined with clustering, characterized in that, The method comprises the steps of: obtaining dialogue text, performing unsupervised model training on the dialogue text to obtain a pre-trained language model; performing clustering processing on the dialogue text, calculating a clustering center contour coefficient, selecting an optimal clustering center K value based on the contour coefficient size, and dividing text clusters; selecting the text closest to the clustering center K value in each text cluster as the representative text in the text cluster, combining the representative texts to generate the intent center text; or, for the representative texts in the text cluster, artificially distinguishing the text categories to form the intent center text; constructing a Prompt template based on the intent center text, the Prompt template being {dialogue text} and {intent center text} consistent? [MASK], wherein [MASK] is used to represent the prediction result, i.e., consistent or inconsistent, inputting the dialogue text into the Prompt template slot, and outputting a Prompt construction text; judging whether the intent of the input dialogue text and the intent center text is consistent through the pre-trained language model.

2. The method of claim 1, wherein the Prompt structure is a combination of a plurality of structures. The method further comprises the steps of: constructing a sentence template based on the dialogue text to automatically generate training corpus; training the dialogue text and the training corpus to obtain the pre-trained language model.

3. The method of claim 1, wherein the prompt structure intent recognition method is characterized in that, The method further comprises the steps of: training the dialogue text through BERT, ELECTRA, and GPT models to obtain the pre-trained language model.

4. A prompt structure intent recognition device combined with clustering, characterized by, The method comprises: a model training module configured to obtain dialogue text, perform unsupervised model training on the dialogue text, and obtain a pre-trained language model; a clustering module configured to perform clustering processing on the dialogue text, calculate a clustering center contour coefficient, select an optimal clustering center K value based on the contour coefficient size, and divide text clusters; the clustering module is further configured to select the text closest to the clustering center K value in each text cluster as the representative text in the text cluster, combine the representative texts to generate the intent center text; or, for the representative texts in the text cluster, artificially distinguish the text categories to form the intent center text; a Prompt template construction module configured to construct a Prompt template based on the intent center text, the Prompt template being {dialogue text} and {intent center text} consistent? [MASK], wherein [MASK] is used to represent the prediction result, i.e., consistent or inconsistent, inputting the dialogue text into the Prompt template slot, and outputting a Prompt construction text; A judgment module configured to judge whether the intent of the dialogue text and the intent of the intent center text input by the pre-trained language model are consistent through the Prompt construction text.

5. An electrical appliance characterized by Comprise: a memory for storing executable instructions; a processor for implementing the prompt structure intent recognition method combined with clustering according to any one of claims 1 to 3 when running the executable instructions stored in the memory.

6. A refrigerator characterized by comprising: Comprise: a memory for storing executable instructions; a processor for implementing the prompt structure intent recognition method combined with clustering according to any one of claims 1 to 3 when running the executable instructions stored in the memory.

7. A computer-readable storage medium storing executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform operations comprising: The executable instructions are executed by the processor to implement the prompt structure intent recognition method combined with clustering according to any one of claims 1 to 3.

Images