Method, device, equipment and medium for intention recognition

Abstract

The present application discloses a method, apparatus, device and medium for intent recognition, which includes: obtaining a training sample set, the training sample set including multiple training samples, each training sample being a character combination of an abstract intent, and the training sample including at least ordered arranged characters of interest; when the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold, compressing each training sample according to the number of character appearances to obtain a compressed sample set; inputting the compressed sample set into a pre-constructed masked language model for training, and outputting an intent recognition result, the masked language model being used to fill the input compressed sample into a preset position in a language prompt template to be trained and then train. The embodiment provided in the present application effectively improves the convergence speed of the masked language model by uniformly pre-processing the training samples.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a method, apparatus, device and medium for intent recognition. Background Art

[0002] Intent recognition involves classifying and identifying user search queries. Applications of intent recognition include search engines, conversational systems, the intelligent Internet of Things, and robotics. In these applications, user input can be non-standard, involve diverse input methods, and even employ non-standard natural language. Therefore, training intent recognition models typically requires a large number of sample learning examples.

[0003] In dialogue systems, intent recognition can be considered a typical text classification task. Given user input, the task is to determine whether it falls within a predefined intent category. Classifying user input text requires a large amount of training data to achieve good results. However, when dialogue systems create new dialogue tasks, there is often a lack of standard data; each intent often only has a few or a dozen examples. In this situation, building intent classification models using a small number of examples has become a new development in natural language processing technology. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a method, apparatus, device and medium for intent recognition to solve the problems of slow convergence speed and poor prediction effect in the pre-training process of the existing language model.

[0005] In a first aspect, an embodiment of the present invention provides a method for intent recognition, the method comprising:

[0006] Obtaining a training sample set, the training sample set including a plurality of training samples, each training sample being a character combination of an abstract intention, and the training sample at least including orderly arranged characters of interest;

[0007] When the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold, compressing each training sample according to the number of occurrences of the characters to obtain a compressed sample set;

[0008] The compressed sample set is input into a pre-built masked language model for training, and the intent recognition result is output. The masked language model is used to fill the input compressed sample into the preset position in the language prompt template to be trained and then train it.

[0009] Optionally, each training sample is compressed according to the number of times the character appears to obtain a compressed sample set, including: obtaining a word frequency list corresponding to the training sample set, the word frequency list including the focus characters included in the training sample set and the number of times each focus character appears in the training sample set; performing character deduplication processing on each training sample in the training sample set to obtain a deduplication sample set, the deduplication sample set including multiple deduplication samples, and the deduplication samples correspond one-to-one to the training samples; performing compression processing on each deduplication sample in the deduplication sample set according to the word frequency list to obtain a compressed sample set, wherein the compressed sample set includes multiple compressed samples, and the compressed samples correspond one-to-one to the deduplication samples.

[0010] Optionally, character deduplication processing is performed on each training sample in the training sample set to obtain a deduplication sample set, including: for each training sample, when it is determined that the training sample contains non-focus characters, replacing the non-focus characters with preset characters to obtain a replacement sample corresponding to the training sample; performing character deduplication processing on the replacement sample corresponding to the training sample to obtain a deduplication sample corresponding to the replacement sample; adding the deduplication sample corresponding to the replacement sample to the deduplication sample set; when it is determined that the training sample does not contain non-focus characters, performing character deduplication processing on the training sample to obtain a deduplication sample corresponding to the training sample; and adding the deduplication sample corresponding to the training sample to the deduplication sample set.

[0011] Optionally, each deduplicated sample in the deduplicated sample set is compressed according to the word frequency list to obtain a compressed sample set, including: determining the character length of the deduplicated sample for each deduplicated sample; when the character length is greater than or equal to a second threshold, filtering the deduplicated samples according to the word frequency list to obtain filtered samples corresponding to the deduplicated samples; compressing the filtered samples corresponding to the deduplicated samples to obtain compressed samples corresponding to the filtered samples; and adding the compressed samples corresponding to the filtered samples to the compressed sample set.

[0012] Optionally, the method further includes: for each deduplicated sample, when the character length is less than a second threshold, determining the deduplicated sample as a compressed sample.

[0013] Optionally, the deduplicated samples are screened according to the word frequency list to obtain screened samples corresponding to the deduplicated samples, including: determining a candidate character set in accordance with the arrangement order of the word frequency list, starting from the first focus character in the word frequency list, the candidate character set including a first threshold number of consecutive focus characters, wherein the word frequency list is sorted in descending order according to the number of occurrences of the focus characters in the training sample set; and screening the focus characters contained in the deduplicated samples according to the candidate character set to obtain screened samples corresponding to the deduplicated samples.

[0014] Optionally, compression processing is performed on the screening samples corresponding to the deduplicated samples to obtain compressed samples corresponding to the screening samples, including: performing sliding window processing on the screening samples using a clipping window to obtain clipped samples corresponding to the screening samples, wherein the size of the clipping window is a second threshold, the step size of each sliding of the clipping window is a preset number of characters, and each clipped sample includes the second threshold number of characters; when it is determined that the clipped sample belongs to a compressed sample already included in the compressed sample set, sliding the clipping window according to the step size, and returning to the step of performing sliding window processing on the screening sample using the clipping window to obtain the clipped sample corresponding to the screening sample, until it is determined that the clipped sample does not belong to a compressed sample already included in the compressed sample set; when it is determined that the clipped sample does not belong to a compressed sample already included in the compressed sample set, determining that the clipped sample is a compressed sample corresponding to the screening sample.

[0015] Optionally, a sliding window process is performed on the screening sample using a clipping window to obtain a clipping sample corresponding to the screening sample, including: when it is determined that the number of characters contained in the screening sample corresponding to the clipping window is less than the size of the clipping window, then according to the character arrangement order of the screening sample, a completion character of the number of differences is set at the end of the characters contained in the screening sample corresponding to the clipping window, and the characters contained in the screening sample corresponding to the clipping window and the set completion characters are extracted as the clipping sample corresponding to the screening sample, and the number of differences is determined by the difference between the size of the clipping window and the number of characters to be clipped in the screening sample corresponding to the clipping window; when it is determined that the number of characters contained in the screening sample corresponding to the clipping window is greater than or equal to the size of the clipping window, then according to the character arrangement order of the screening sample, the characters corresponding to the clipping window are extracted from the screening sample as the clipping sample corresponding to the screening sample.

[0016] Optionally, if the deduplicated sample contains a preset character, the preset character is set at the first position of the deduplicated sample.

[0017] Optionally, obtaining a word frequency list corresponding to the training sample set includes: counting the number of occurrences of each focus character in the training sample set in the training sample set; sorting the focus characters in the training sample set according to the number of occurrences of each focus character, and obtaining a word frequency list corresponding to the training sample set.

[0018] In a second aspect, an embodiment of the present invention further provides a device for intent recognition, the device comprising:

[0019] A training sample acquisition module is used to acquire a training sample set, where the training sample set includes a plurality of training samples, each training sample is a character combination of an abstract intention, and the training sample includes at least orderly arranged characters of interest;

[0020] A sample compression processing module is used to compress each training sample according to the number of occurrences of the characters when the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold value to obtain a compressed sample set;

[0021] The model training module is used to input the compressed sample set into a pre-built masked language model for training and output the intent recognition result. The masked language model is used to fill the input compressed sample into the preset position in the language prompt template to be trained and then train it.

[0022] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present invention when executing the program.

[0023] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the embodiment of the present invention when the computer program is executed by a processor.

[0024] The beneficial effects brought about by the technical solution provided by the present invention are:

[0025] The present invention provides a method, apparatus, device, and medium for intent recognition. The method comprises obtaining a training sample set; then, when the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold, compressing each training sample based on the number of character occurrences to obtain a compressed sample set; finally, inputting the compressed sample set into a pre-built masked language model for training, and outputting the intent recognition result. By obtaining training samples and performing a unified preprocessing modification on the training samples, the method effectively improves the convergence speed of the masked language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A schematic diagram showing a flow chart of a method for intention recognition proposed in an embodiment of the present invention;

[0028] Figure 2 A schematic flow chart of a method for intention recognition according to an embodiment of the present invention is shown;

[0029] Figure 3 A schematic flow chart showing a method for intention recognition according to another embodiment of the present invention is shown;

[0030] Figure 4 A schematic structural diagram of an apparatus for intention recognition provided by an embodiment of the present invention is shown;

[0031] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the present invention more apparent, embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0033] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] With the continuous development of artificial intelligence technology, user intent expressions in chat systems often suffer from colloquialism and broad content, and the amount of chat text is limited. To address these issues, the industry has proposed using natural language prompts to insert user input samples into pre-set templates, thereby converting the original query task into a language mask model. For example, PET (Pattern-Exploiting Training) is essentially a semi-supervised training process that concatenates the input into a task description by adding a [mask] label at a certain position in the task description, thereby converting the input into a cloze-style task. While applying PET to intent recognition can reduce the reliance on labeled data, the variable length of user-entered query information can lead to slow or even non-convergence problems during PET model training. Consequently, directly applying the PET model to various intent recognition scenarios does not yield very good prediction results.

[0035] The present invention proposes a method for intent recognition, which can accelerate the convergence speed of PET training during the intent recognition training process and effectively improve the efficiency of template construction.

[0036] In order to more clearly understand the inventive concept provided by the present invention, Figure 1-5 The test case management method proposed in the present invention is described.

[0037] Please refer to Figure 1 , Figure 1The following is a flow chart of a method for intention recognition according to an embodiment of the present invention. The method can be implemented by an apparatus for intention recognition configured in an electronic device. The method includes:

[0038] Step 101: Obtain a training sample set, where the training sample set includes a plurality of training samples. Each training sample is a character combination of an abstract intention, and the training sample at least includes orderly arranged characters of interest.

[0039] Step 102 : When the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold, compress each training sample according to the number of occurrences of the characters to obtain a compressed sample set.

[0040] In step 103, the compressed sample set is input into a pre-built masked language model for training, and the intention recognition result is output. The masked language model is used to fill the input compressed sample into a preset position in the language prompt template to be trained and then perform training.

[0041] In the above steps, training samples refer to character combinations obtained by abstracting the input data into intent. For example, the input data is: What's your name?, What's your name?, What's your name?.

[0042] The character combination obtained by performing intent abstraction processing on the above input data is the training sample, and the training sample is: What is your name?

[0043] Training samples can be understood as character combinations, including but not limited to Chinese characters, punctuation marks, and English characters. Training samples consist of an ordered sequence of characters of interest and non-characters of interest. For example, training samples might include "What is your name?" or "Ask for the location of a certain attraction / place / item." The ordered sequence of Chinese characters and punctuation marks conveys the original intent of the input data.

[0044] Similarly, multiple input data are processed for intent abstraction to obtain multiple training samples. Multiple training samples are combined into a training sample set. For example, the training sample set can be:

[0045] {(a certain scenic spot / place / item) asks about the location; (a certain item / service) asks about the price; (a certain place) seeks recommendations for scenic spots / tourist destinations; (a certain place) seeks recommendations for food; (a certain item / service) asks about how to buy; (a certain scenic spot / place / item) asks about the characteristics; (a certain scenic spot / place) asks about activities; (a certain scenic spot / place) asks about performances; (a certain scenic spot / place) asks about the history; (a certain scenic spot / place / project) asks about the person in charge; (a certain company / place) asks about the culture; (a person / thing) asks about the nationality, (a person / thing) asks about the age; (a certain place) asks about local specialties;}

[0046] Among them, the query location (of a certain scenic spot / place / item) can be called a training sample.

[0047] The total number of characters in the training sample set after deduplication refers to the number of Chinese characters contained in the training sample set after deduplication. Perform deduplication processing on the training sample set; then, obtain the total number of characters corresponding to the training sample set after deduplication. That is, perform deduplication word count on the training sample set. Assume that the training sample set is {(a certain scenic spot / place / item) asks for location, (a certain item / service) asks for price, (a certain place) seeks recommended scenic spots / tourist destinations, (a certain place) seeks recommended food, (a certain item / service) asks how to buy, (a certain scenic spot / a certain place / a certain item) asks for features, (a certain scenic spot / a certain place) asks for activities, (a certain scenic spot / a certain place) asks for performances, (a certain scenic spot / a certain place) asks for history, (a certain scenic spot / a certain place / a certain project) asks for negative The following sentences are from the training sample set: {person in charge, (a certain company / place) inquires about culture, (a person / thing) inquires about nationality, (a person / thing) inquires about age, (a certain place) inquires about specialties}. After merging and removing duplicates, the resulting deduplicated list corresponding to the training sample set is: {a certain, point, inquires, place, scenery, thing, product, person, service, seeks, recommends, special, location, price, travel, entertainment, food, shopping, style, color, activity, performance, history, item, responsible, company, culture, country, nationality, age, product}. The deduplicated list after merging and removing duplicates here contains more than 20 characters.

[0048] The number of character occurrences refers to the number of occurrences of each character of interest in the training sample set.

[0049] The first threshold refers to the ideal threshold condition determined after multiple intent recognition trainings. The first threshold can be 2-30 words. Optionally, the first threshold is 20 words.

[0050] A compressed sample is a character combination obtained by compressing a training sample. The character length of the character combination is fixed, and the sum of the occurrences of the focus characters in the character combination is maximized. The compression process includes but is not limited to truncating the training sample according to the preset character length and the semantics of the training sample.

[0051] Focus characters are characters that appear in training samples and are used to represent intent. For example, if the training sample is {(a certain scenic spot / location / item) ask for location}, the focus characters included in this training sample are "(", ")", and " / ". The non-focus characters included in this training sample are "(", ")", and " / ". Optionally, the focus characters include, but are not limited to, Chinese characters and English words.

[0052] A masked language model is a model that trains by inserting input samples into preset positions in a language prompt template. Language prompt templates can include, for example, "The following sentence is asking about [mask]" or "The following sentence intends to [mask]."

[0053] Masked language models can be pre-trained for the BERT model (Bidirectional Encoder Representations from Transformers). This pre-training task includes, but is not limited to, the MLM model (Masked Language Model). The BERT model is a new language model that can pre-train deep bidirectional representations by jointly regulating the bidirectional Transformer encoders in all layers. The input representation of the BERT model can represent a single text sentence in a word sequence, or a pair of texts, such as [question, answer]. The purpose of pre-training is to build a language model. The BERT model is a bidirectional encoder. When pre-training a language model to handle downstream tasks, linguistic information to the left and right of a word is required. The MLM randomly masks some words from the input, allowing the integration of context from both the left and right sides, thereby achieving bidirectional Transformer representations. Specifically, the MLM model randomly masks some words in the sentence and then uses the context to predict the masked words. This approach integrates bidirectional text information, making it possible to pre-train a deep bidirectional Transformer model.

[0054] The method for intent recognition provided by an embodiment of the present invention performs a unified preprocessing transformation on the training sample set based on word frequency during language model pre-training, which can reduce the size of the candidate word list and shorten the length required for prediction during multiplication, thereby effectively improving the convergence speed of the PET model.

[0055] Please refer to Figure 2 , Figure 2 The following is a flow chart of a method for intention recognition according to an embodiment of the present invention. The method can be implemented by an intention recognition device configured in an electronic device. The method includes:

[0056] Step 201 : obtaining a training sample set, where the training sample set includes a plurality of training samples, each training sample is a character combination of an abstract intention, and the training sample at least includes orderly arranged characters of interest.

[0057] Step 202: When the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold, obtain a character frequency list corresponding to the training sample set, the character frequency list including the focus characters included in the training sample set and the number of occurrences of each focus character in the training sample set.

[0058] Step 203 , performing character deduplication processing on each training sample in the training sample set to obtain a deduplication sample set, wherein the deduplication sample set includes a plurality of deduplication samples, and the deduplication samples correspond to the training samples one-to-one.

[0059] Step 204 : compress each deduplicated sample in the deduplicated sample set according to the word frequency list to obtain a compressed sample set, wherein the compressed sample set includes a plurality of compressed samples, and the compressed samples correspond one-to-one to the deduplicated samples.

[0060] In step 205 , the compressed sample set is input into a pre-built masked language model for training, and the intention recognition result is output. The masked language model is used to fill the input compressed sample into a preset position in the language prompt template to be trained and then perform training.

[0061] Step 206: When the total number of characters in the training sample set after deduplication is less than the first threshold, no special processing is required.

[0062] Optionally, obtaining a word frequency list corresponding to the training sample set may include the following steps:

[0063] Count the number of occurrences of each focus character in the training sample set;

[0064] According to the number of occurrences of each focus character, the focus characters in the training sample set are sorted to obtain a word frequency list corresponding to the training sample set.

[0065] Optionally, counting the number of occurrences of each focus character in the training sample set may include:

[0066] After word segmentation processing for each training sample, a single focus character is obtained;

[0067] Traverse the training sample set, accumulate and count each focus character, and obtain the number of occurrences of each focus character.

[0068] Suppose the training sample set is {(attraction / place / item) inquires about location; (item / service) inquires about price; (item / place) seeks recommended attractions / places; (item / service) seeks recommended food; (item / service) inquires about purchase methods; (attraction / place / item) inquires about features; (attraction / place) inquires about events; (attraction / place) inquires about performances; (attraction / place) inquires about history; (attraction / place / project) inquires about the person in charge; (company / place) inquires about culture; (person / item) inquires about nationality, (person / item) inquires about age; (item / place) inquires about local specialties;}. The focus characters in this training sample set are Chinese characters.

[0069] In the above training sample set, only the frequency of Chinese characters is sorted, and the frequency list is as follows:

[0070] {'Some':22,'Point':17,'Inquiry':12,'Ask':12,'Place':11,'Scenic Spots':7,'Things':6,'Products':4,'People':3,'Service':2,'Service':2,'Search':2,'Seek':2,'Recommend':2,'Recommend':2,'Special':2,'Location':1,'Place':1,'Price':1,'Price':1,'Travel':1,'Play:1,'Delicious':1,'Food' :1,'purchase':1,'buy':1,'method':1,'style':1,'color':1,'activity':1,'action':1,'performance':1,'exhibition':1,'history':1,'history':1,'project':1,'responsibility':1,'responsibility':1,'enterprise':1,'industry':1,'culture':1,'culture':1,'country':1,'nationality':1,'year':1,'age':1,'production':1}

[0071] The above sorting can optionally be performed in order of word frequency from high to low, or from low to high.

[0072] Optionally, performing character deduplication processing on each training sample in the training sample set includes:

[0073] For each training sample, when it is determined that the training sample contains non-focus characters, the non-focus characters are replaced with preset characters to obtain a replacement sample corresponding to the training sample;

[0074] Performing character deduplication processing on the replacement sample corresponding to the training sample to obtain a deduplication sample corresponding to the replacement sample;

[0075] Add the deduplication sample corresponding to the replacement sample to the deduplication sample set;

[0076] When it is determined that the training sample does not contain non-focus characters, performing character deduplication processing on the training sample to obtain a deduplication sample corresponding to the training sample;

[0077] Add the deduplicated samples corresponding to the training samples to the deduplicated sample set.

[0078] The non-attention characters mentioned above refer to the characters other than the attention characters in each training sample. Non-attention characters include but are not limited to English characters, special symbols, etc. English characters can be single letters.

[0079] The non-Chinese characters in the training sample set are replaced with preset characters. For example, the non-Chinese characters in the training sample set are replaced with the [UNK] symbol.

[0080] The preset characters are characters or character combinations that are predefined to replace non-focus characters. The preset characters include but are not limited to the form of [UNK].

[0081] For example, after replacing the training sample set with the [UNK] symbol, the result after replacement is obtained:

[0082] {[UNK]A certain scenic spot[UNK]place[UNK]item[UNK]inquire about the location,[UNK]A certain item[UNK]service[UNK]inquire about the price,[UNK]A certain place[UNK]seek recommended scenic spots[UNK]travel places,[UNK]A certain place[UNK]seek recommended food,[UNK]A certain item[UNK]service[UNK]inquire about how to buy,[UNK]A certain scenic spot[UNK]place[UNK]A certain item[UNK]inquire about the features,[UNK]A certain scenic spot[UNK]place[UNK] NK] to inquire about activities, [UNK] to inquire about performances at a certain scenic spot [UNK] or a certain place [UNK], [UNK] to inquire about the history of a certain scenic spot [UNK] or a certain place [UNK], [UNK] to inquire about the person in charge of a certain project at a certain scenic spot [UNK] or a certain place [UNK], [UNK] to inquire about the culture of a certain company [UNK] or a certain place, [UNK] to inquire about the nationality of a certain person [UNK] or a thing, [UNK] to inquire about the age of a certain person [UNK] or a thing, [UNK] to inquire about the specialty of a certain place}.

[0083] After the replacement process, duplicate characters are removed to obtain duplicate samples corresponding to each replacement sample. For example, if the replacement sample is {[UNK] a certain scenic spot [UNK] location [UNK] item [UNK] inquiry location}, the duplicated sample corresponding to this replacement sample is {[UNK] a certain scenic spot location item inquiry location}.

[0084] The above replacement results are added to the deduplication sample set which is initially empty, and finally the deduplication sample set is obtained as shown below:

[0085] {[UNK]Ask about the location of an item at a certain tourist attraction, [UNK]Ask about the price of an item or service, [UNK]Ask for recommended attractions at a certain place, [UNK]Ask for recommended food at a certain place, [UNK]Ask about how to buy an item or service, [UNK]Ask about the features of an item at a certain tourist attraction, [UNK]Ask about activities at a certain tourist attraction, [UNK]Ask about performances at a certain tourist attraction, [UNK]Ask about the history of a certain tourist attraction, [UNK]Ask about the person in charge of a project at a certain tourist attraction, [UNK]Ask about the culture of a certain company, [UNK]Ask about the nationality of a certain person, [UNK]Ask about the age of a certain person, [UNK]Ask about the local specialties}.

[0086] Optionally, if the deduplicated sample contains a preset character, the preset character is set at the first position of the deduplicated sample.

[0087] Since the sampling intent does not process the special symbols of the input data itself, different special symbols have no substantial impact on intent recognition. However, their existence will occupy the processing character length in the form of characters, resulting in the PET prediction length cannot be effectively shortened, thereby affecting the training effect and slowing convergence.

[0088] The present invention sets the preset character at the first position of the deduplicated sample on the basis of retaining the character arrangement of the training sample, which can retain the characteristics of the input data with special symbols, while effectively solving the problem of PET prediction length. It effectively improves the convergence speed of the masked language model by performing unified preprocessing transformation on the training samples.

[0089] Optionally, each deduplicated sample in the deduplicated sample set is compressed according to the word frequency list to obtain a compressed sample set, wherein the compressed sample set includes multiple compressed samples, and the compressed samples correspond one-to-one with the deduplicated samples. The compressed sample set is then input into a pre-built masked language model for training, and the intent recognition result is output.

[0090] The method for intent recognition provided by the embodiment of the present invention uses a word frequency list to deduplicate a training sample set, thereby reducing the size of the candidate word list and effectively improving the model convergence speed.

[0091] Please refer to Figure 3 , Figure 3 The following is a flow chart of a method for intention recognition according to another embodiment of the present invention, which can be implemented by an apparatus for intention recognition configured in an electronic device. The method includes:

[0092] Step 301: Obtain a training sample set, where the training sample set includes multiple training samples. Each training sample is a character combination of abstract intent, and the training sample at least includes ordered arranged characters of interest.

[0093] Step 302: When the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold, a character frequency list corresponding to the training sample set is obtained. The character frequency list includes the focus characters included in the training sample set and the number of occurrences of each focus character in the training sample set.

[0094] Step 303: perform character deduplication processing on each training sample in the training sample set to obtain a deduplication sample set.

[0095] Step 304: For each deduplicated sample, determine the character length of each deduplicated sample.

[0096] Step 305 : When the character length is greater than or equal to the second threshold, the deduplicated samples are screened according to the character frequency list to obtain screened samples corresponding to the deduplicated samples.

[0097] Step 306: compress the screened samples corresponding to the deduplicated samples to obtain compressed samples corresponding to the screened samples.

[0098] Step 307: Add the compressed samples corresponding to the screened samples to the compressed sample set.

[0099] Step 308: For each deduplicated sample, when the character length is less than a second threshold, the deduplicated sample is determined as a compressed sample, and the compressed sample is added to the compressed sample set.

[0100] In step 309 , the compressed sample set is input into a pre-built masked language model for training, and the intention recognition result is output. The masked language model is used to fill the input compressed sample into a preset position in the language prompt template to be trained and then perform training.

[0101] Step 310: When the total number of characters in the training sample set after deduplication is less than the first threshold, no special processing is required.

[0102] In the above steps, the character length of each deduplicated sample refers to the number of characters contained in the character combination obtained after the deduplication process. For example, the deduplicated sample is {[UNK] a certain scenic spot item location inquiry}, and its corresponding character length is 11.

[0103] The second threshold is a maximum length constraint value preset for deduplication samples, which may be a value related to the characteristics of intent recognition. The second threshold may be represented by max_length, for example, and its value is preferably 5.

[0104] Optionally, the deduplicated samples are screened according to the word frequency list to obtain screened samples corresponding to the deduplicated samples, including:

[0105] According to the sorting order of the character frequency list, starting from the first concerned character in the character frequency list, a candidate character set is determined. The candidate character set includes the first threshold number of consecutive concerned characters. Here, the character frequency list is sorted in descending order of the number of occurrences of concerned characters in the training sample set;

[0106] The concerned characters included in the deduplicated sample are screened according to the candidate character set to obtain a screened sample corresponding to the deduplicated sample.

[0107] Based on the above character frequency list, assuming that the value of the first threshold is 20, starting from the first concerned character in the character frequency list, 20 consecutive concerned characters are extracted. The obtained character set is the candidate character set. The candidate character set, for example, is {'a certain': 22, 'point': 17, 'inquire': 12, 'ask': 12, 'place': 11,'scene': 7, 'object': 6, 'product': 4, 'person': 3,'service': 2,'search': 2, 'demand': 2, 'promote': 2,'recommend': 2,'special': 2, 'position': 1, 'location': 1, 'price': 1, 'cost': 1}. The candidate character set contains the high-frequency concerned characters that appear in the training samples. Using these high-frequency concerned characters to screen each deduplicated sample can further reduce the time for processing data.

[0108] Screening the concerned characters included in the deduplicated sample according to the candidate character set may include:

[0109] Compare each concerned character included in the deduplicated sample with the concerned characters included in the candidate character set in the character arrangement order of the deduplicated sample;

[0110] When it is determined that the concerned character included in the deduplicated sample belongs to the concerned character included in the candidate character set, retain the concerned character included in the deduplicated sample;

[0111] When it is determined that the concerned character included in the deduplicated sample does not belong to the concerned character included in the candidate character set, delete the concerned character included in the deduplicated sample.

[0112] For example, the deduplicated sample is {[UNK]a certain scenic spot, place, object, inquire, special feature}. Compare each Chinese character in the deduplicated sample with the above candidate character set to obtain the screened sample {[UNK]a certain scenic spot, place, object, inquire, special} corresponding to the deduplicated sample.

[0113] Optionally, perform compression processing on the screened sample corresponding to the deduplicated sample to obtain a compressed sample corresponding to the screened sample, including:

[0114] Performing sliding window processing on the screening sample using a clipping window to obtain a clipping sample corresponding to the screening sample, wherein the size of the clipping window is a second threshold, the step length of each sliding of the clipping window is a preset number of characters, and each clipping sample includes the second threshold number of characters;

[0115] When it is determined that the intercepted sample belongs to a compressed sample already included in the compressed sample set, sliding the interception window according to the step size, returning to the step of performing sliding window processing on the screening sample using the interception window to obtain a intercepted sample corresponding to the screening sample, until it is determined that the intercepted sample does not belong to a compressed sample already included in the compressed sample set;

[0116] When it is determined that the clipped sample does not belong to the compressed samples already included in the compressed sample set, the clipped sample is determined to be the compressed sample corresponding to the screened sample.

[0117] In the above steps, the interception window is used to extract multiple consecutive characters from the screening sample. The size of the interception window is used to determine the number of characters extracted from the screening sample. For example, when the interception window size is 5, it means that 5 consecutive characters are extracted from the screening sample each time. The position of the characters is determined by the character arrangement order of the screening sample. For example, if the screening sample is {[UNK]Something about a tourist attraction, item, special}, the first character of its character arrangement order is [UNK], the second character is "Something", and so on, and the last character is "Special".

[0118] A clipped sample is the result of sliding a window over a selected sample using a clipping window. For example, if the clipping window size is 5, the first clipped sample corresponding to the selected sample obtained from the first sliding window operation is {[UNK]Some Scenic Spot}. The step size is the number of characters that the sliding window moves each time. The step size can be 1 character.

[0119] Optionally, the initial state of the compressed sample set is empty.

[0120] When the current screening sample is the first sample to be processed, the compressed sample set is empty, indicating that the compressed samples already included in the compressed sample set are empty. When it is determined that the intercepted sample does not belong to the compressed samples already included in the compressed sample set, the intercepted sample is determined to be a compressed sample and is added to the compressed sample set.

[0121] When the current screening sample is not the first sample to be processed, the state of the compressed sample set is non-empty, then compare whether the clipped sample obtained from the current screening sample belongs to the compressed sample already included in the compressed sample set. If so, determine that the clipped sample belongs to the compressed sample already included in the compressed sample set; if not, determine that the clipped sample does not belong to the compressed sample already included in the compressed sample set.

[0122] If it is determined that the intercepted sample belongs to the compressed samples already included in the compressed sample set, the current screened sample is processed by sliding window again to obtain an intercepted sample corresponding to the current screened sample, and then this intercepted sample is compared with the compressed samples already included in the compressed sample set until the intercepted sample corresponding to the current screened sample does not belong to the compressed samples already included in the compressed sample set. Then, the next sample of the current screened sample is processed until the last screened sample in the screened sample set.

[0123] In the present invention, each screened sample is separately compressed to obtain a compressed sample corresponding to each screened sample, which ensures that the character lengths of each compressed sample are the same and the compressed samples themselves are unique, thus ensuring the richness of the samples for model training.

[0124] Optionally, using an intercept window to perform a sliding window process on the screened sample to obtain an intercepted sample corresponding to the screened sample, including:

[0125] When it is determined that the number of characters included in the screened sample corresponding to the intercept window is less than the size of the intercept window, then according to the character arrangement order of the screened sample, a difference number of padding characters are set at the end of the characters included in the screened sample corresponding to the intercept window, and the characters included in the screened sample corresponding to the intercept window and the set padding characters are extracted as the intercepted sample corresponding to the screened sample, and the difference number is determined by the difference between the size of the intercept window and the number of characters to be intercepted in the screened sample corresponding to the intercept window;

[0126] When it is determined that the number of characters included in the screened sample corresponding to the intercept window is greater than or equal to the size of the intercept window, then according to the character arrangement order of the screened sample, the characters corresponding to the intercept window are extracted from the screened sample as the intercepted sample corresponding to the screened sample.

[0127] For example, the screened sample is {[UNK]A certain scenic spot item inquiry special}, and the size of the intercept window is 5. After multiple sliding window processes, it may occur that the characters included in the screened sample corresponding to the intercept window are {product, inquiry, question, special}, then according to the character arrangement order of the screened sample, padding symbols are set after the characters included in the screened sample corresponding to the intercept window are {product, inquiry, question, special}. Among them, the padding character is a symbol or character combination used to fill the intercepted sample when the number of characters included in the intercepted sample is less than the second threshold number. The padding character includes, but is not limited to, the [PAD] character form.

[0128] Through the above processing, a character combination with the optimal semantics and the optimal processing length can be obtained, effectively improving the convergence efficiency of the PET model.

[0129] Furthermore, the compressed sample set is input into a pre-built masked language model for training, and the output is the intent recognition result. Assume that the pre-built masked language model may include one or more language prompt templates, such as: The following sentence is asking [mask]; The intention of the following sentence is [mask]; The intention of the following sentence is to ask [mask].

[0130] After obtaining the compressed sample set, each compressed sample is spliced ​​with the selected language prompt template and sent to the PET training framework to complete the PET model training.

[0131] The intent recognition method provided by the embodiment of the present invention performs unified preprocessing and transformation on training samples, shortens the length of prediction during multiplication, and effectively improves the convergence speed of the masked language model.

[0132] The PET template determined based on the intent recognition method provided by the embodiments of the present invention can be applied to many classification-related tasks, such as emotion recognition and relevance determination. However, different tasks generally require different templates. Therefore, different training samples can be obtained and pre-trained according to different application scenarios to obtain different PET templates.

[0133] Taking text recognition in a dialogue system as an example, let's assume that user-entered intent data is obtained in the dialogue system and that the intent names (i.e., training samples) are subjected to standardized preprocessing. An intent name is a character combination used to represent an intent. An intent name is an abstract representation of an intent. Intents can be expressed in various forms, such as text and speech.

[0134] a. Preprocessing decision module: After obtaining the full amount of intent name data, the characters of all intent names are deduplicated and the total number of characters in the current intent set is calculated.

[0135] For example, there is a list of intent names as follows: (a certain attraction / place / item) asks about the location, (a certain item / service) asks about the price, (a certain place) seeks recommended attractions / tourist destinations, (a certain place) seeks recommended food, (a certain item / service) asks about how to buy, (a certain attraction / place / item) asks about the characteristics, (a certain attraction / place) asks about activities, (a certain attraction / place) asks about performances, (a certain attraction / place) asks about the history, (a certain attraction / place / project) asks about the person in charge, (a certain company / place) asks about the culture, (a person / thing) asks about the nationality, (a person / thing) asks about the age, (a certain place) asks about the specialty.

[0136] The Chinese characters in the above intent name list are deduplicated, and the resulting deduplicated word list is as follows: a certain, point, inquiry, place, scenery, object, product, person, service, seek, seek, recommend, special, location, place, price, travel, play, beauty, food, purchase, way, style, color, activity, performance, history, project, responsible, enterprise, culture, country, nationality, year, age, production.

[0137] If the total number of characters in the deduplicated intent name list is greater than or equal to 20, the following steps are required (i.e., triggering intent name pre-training). Otherwise, no special processing is required.

[0138] b. If the total number of characters in the intent name list after deduplication is greater than or equal to 20, the intent name will be processed as follows:

[0139] i. First, count the Chinese characters that appear in the intent name list separately, and sort the Chinese characters that appear in the intent name list from high to low according to the number of times they appear, to obtain a word frequency sorted list corresponding to the intent name list.

[0140] The word frequency sorted list corresponding to the list of intent names is:

[0141] {'Some':22,'Point':17,'Inquiry':12,'Ask':12,'Place':11,'Scenic Spots':7,'Things':6,'Products':4,'People':3,'Service':2,'Service':2,'Search':2,'Seek':2,'Recommend':2,'Recommend':2,'Special':2,'Location':1,'Place':1,'Price':1,'Price':1,'Travel':1,'Play:1,'Delicious':1,'Food' :1,'purchase':1,'buy':1,'method':1,'style':1,'color':1,'activity':1,'action':1,'performance':1,'exhibition':1,'history':1,'history':1,'project':1,'responsibility':1,'responsibility':1,'enterprise':1,'industry':1,'culture':1,'country':1,'nationality':1,'year':1,'age':1,'production':1}

[0142] ii. Replace special symbols and non-Chinese characters in the intent name with the [UNK] symbol. Since intent names are often freely entered, they may contain special symbols or English letters. These special symbols can be uniformly represented using the special symbol [UNK]. For example, replace each intent name in the intent name list mentioned in the previous step.

[0143] (A certain scenic spot / place / item) ask for the location -> [UNK] a certain scenic spot [UNK] place [UNK] item [UNK] ask for the location,

[0144] (A certain item / service) asks for price->[UNK]A certain item[UNK]service[UNK] asks for price,

[0145] (A certain place) seeking recommended attractions / places to visit -> [UNK] A certain place [UNK] seeking recommended attractions [UNK] places to visit,

[0146] (A certain place) looking for recommended food->[UNK]A certain place[UNK] looking for recommended food,

[0147] (A certain item / service) asks how to buy->[UNK]A certain item[UNK]Service[UNK] asks how to buy,

[0148] (A certain scenic spot / a certain place / a certain item) ask for features->[UNK]A certain scenic spot[UNK]A certain place[UNK]A certain item[UNK] ask for features,

[0149] (A certain scenic spot / a certain place) ask about activities->[UNK]A certain scenic spot[UNK]A certain place[UNK] ask about activities,

[0150] (A certain scenic spot / a certain place) ask about performances->[UNK]A certain scenic spot[UNK]A certain place[UNK] ask about performances,

[0151] (A certain scenic spot / place) ask about history->[UNK]A certain scenic spot[UNK]A certain place[UNK] ask about history,

[0152] (A certain scenic spot / a certain place / a certain project) ask the person in charge->[UNK]A certain scenic spot[UNK]A certain place[UNK]A certain project[UNK] ask the person in charge,

[0153] (A company / a place) ask about culture->[UNK]A company[UNK]A place[UNK] ask about culture,

[0154] (someone / thing) asks about nationality->[UNK]someone[UNK]thing[UNK] asks about nationality,

[0155] (someone / thing) asks about age->[UNK]someone[UNK]thing[UNK] asks about age,

[0156] (A certain place) Ask about specialties -> [UNK] A certain place [UNK] Ask about specialties

[0157] iii. Perform internal character deduplication on the replaced intent name. For example, perform character deduplication on each replaced intent name.

[0158] [UNK] a certain scenic spot [UNK] location [UNK] item [UNK] ask for location -> [UNK] a certain scenic spot item ask for location,

[0159] [UNK] a certain item [UNK] service [UNK] ask for price -> [UNK] a certain item service ask for price,

[0160] [UNK] a certain place [UNK] seeking recommended attractions [UNK] places to visit -> [UNK] a certain place seeking recommended attractions to visit,

[0161] [UNK] A certain place [UNK] Seeking recommended food -> [UNK] A certain place seeking recommended food,

[0162] [UNK] a certain item [UNK] service [UNK] ask how to buy -> [UNK] a certain item service ask how to buy,

[0163] [UNK] a certain scenic spot [UNK] a certain place [UNK] a certain item [UNK] ask for characteristics -> [UNK] ask for characteristics of an item at a certain scenic spot,

[0164] [UNK] a certain scenic spot [UNK] a certain place [UNK] inquire about activities -> [UNK] inquire about activities at a certain scenic spot,

[0165] [UNK] a certain scenic spot [UNK] a certain place [UNK] inquire about performances -> [UNK] inquire about performances at a certain scenic spot,

[0166] [UNK] a certain scenic spot [UNK] a certain place [UNK] ask about history -> [UNK] ask about history at a certain scenic spot,

[0167] [UNK] a certain scenic spot [UNK] a certain location [UNK] a certain project [UNK] ask the person in charge -> [UNK] ask the person in charge of a certain scenic spot project,

[0168] [UNK] a certain company [UNK] a certain place [UNK] ask about culture -> [UNK] a certain company place ask about culture,

[0169] [UNK] Someone [UNK] asked about his nationality -> [UNK] Someone asked about his nationality,

[0170] [UNK] someone [UNK] thing [UNK] asked about age -> [UNK] someone asked about age,

[0171] [UNK] Ask for specialties at a certain place -> [UNK] Ask for specialties at a certain place

[0172] iv. Perform constraint length extraction on each intent name after character deduplication to obtain the truncated intent name.

[0173] Since the character length of each deduplicated intent name is still too long, which is not conducive to downstream training, each deduplicated intent name is processed by obtaining preferred characters from the character list, and then each deduplicated intent name is truncated according to the truncation window of max_length=5 to shorten the predicted length, thereby ensuring the convergence speed of the PET model.

[0174] For example, obtain the first 20 characters of the word frequency list, and then filter each intent name after character deduplication and the first 20 characters of the word frequency arrangement list, so that each intent name after character deduplication only contains characters within the range of 20 characters.

[0175] Then, a maximum length constraint called max_length is set for each intent name. If the character length of the current intent name is less than max_length after deduplication and filtering, the deduplication-filtered intent name is directly added to the intent name training set (i.e., the compressed sample set mentioned above). If the character length of the current intent name is greater than or equal to max_length, the current intent name is intercepted using the interception window.

[0176] For example, the first 20 characters in the word frequency list are as follows:

[0177] {'Some':22,'Point':17,'Inquiry':12,'Question':12,'Place':11,'Scenic Spot':7,'Things':6,'Product':4,'People':3,'Service':2,'Service':2,'Search':2,'Request':2,'Recommend':2,'Recommend':2,'Feature':2,'Location':1,'Place':1,'Price':1,'Price':1}

[0178] After filtering each intent name after deduplication, determine whether it is necessary to use the interception window to perform sliding window processing on the filtered results. For example,

[0179] For "[UNK] a certain place seeking food recommendations", compare it with the first 20 characters in the word frequency list to get the filtered result "[UNK] a certain place seeking recommendations" (i.e., the filtered sample). Then, the first sliding window processing is performed on "[UNK] a certain place seeking recommendations" using the intercepted window to obtain

[0180] [UNK] Seeking food recommendations at a certain place -> [UNK] Seeking recommendations at a certain place -> [UNK] Seeking food recommendations at a certain place.

[0181] Similarly, process each of the deduplicated intent names one by one.

[0182] [UNK]Seek recommendations for food at a certain location -> [UNK]Seek recommendations at a certain location -> [UNK]Seek at a certain location;

[0183] [UNK]Seek recommendations for scenic spots to visit at a certain location -> [UNK]Seek recommendations at a certain location -> Seek at a certain location; # Since "[UNK]Seek at a certain location" has already appeared, slide the window one position to the right

[0184] [UNK]Ask about local specialties at a certain location -> [UNK]Ask at a certain location -> [UNK]Ask at a certain location;

[0185] [UNK]Ask about the culture at a certain enterprise location -> [UNK]Ask at a certain location -> Ask at a certain location; # Similarly, since the characters "enterprise", "culture" are not among the top 20 characters in the character frequency list and are defaulted, and "[UNK]Ask at a certain location" has already appeared, slide the window one position to the right

[0186] [UNK]Ask about the purchase method of a certain item / service -> [UNK]Ask about a certain item / service -> [UNK]Ask about a certain item / service;

[0187] [UNK]Ask about the price of a certain item / service -> [UNK]Ask about the price of a certain item / service -> Ask about a certain item / service; # Slide the window one position to the right

[0188] [UNK]Ask about the features of an item at a certain scenic spot -> [UNK]Ask about an item at a certain scenic spot -> [UNK]Ask at a certain scenic spot;

[0189] [UNK]Ask about the location of an item at a certain scenic spot -> [UNK]Ask about an item at a certain scenic spot -> Ask about an item at a certain scenic spot, # Slide the window one position to the right

[0190] [UNK]Ask about activities at a certain scenic spot -> [UNK]Ask about a certain scenic spot -> Ask about a certain scenic spot; # Slide the window one position to the right

[0191] [UNK]Ask about performances at a certain scenic spot -> [UNK]Ask about a certain scenic spot -> Ask about a scenic spot; # Slide the window two positions to the right

[0192] [UNK]Ask about the history of a certain scenic spot -> [UNK]Ask about a certain scenic spot -> Ask about a scenic spot [PAD]; # After sliding the window two positions to the right, it is found that there are less than 5 characters, and fill in with the [PAD] symbol

[0193] [UNK]Ask about the person in charge of a project at a certain scenic spot -> [UNK]Ask about a person at a certain scenic spot -> Ask about a person at a scenic spot; <00[UNK] A certain person asks about age -> [UNK] A certain person asks -> [UNK] A certain person asks;

[0195] [UNK] Someone asks about nationality -> [UNK] Someone asks -> Someone asks; # Slide window, move right one position

[0196] After the above processing, the intent name is obtained, and the intent names with different character lengths and different meanings are compressed into the same character length, which shortens the length required for prediction during multiplication and improves the speed of model convergence.

[0197] The abbreviated representations of intent names obtained through the above steps (the compressed samples described above) are used as training targets for the language model. Several templates suitable for PET learning are designed and experimented with, and the optimal template is determined through training. In experiments, we found that a fixed template, such as "The following sentence is a question," can achieve relatively good results in intent recognition.

[0198] Fixed language prompt template Intent recognition accuracy The following sentence is asking 95.02％ The intention of the following sentence is 94.13％ The next sentence is intended to ask 92.446％

[0199] Table (1)

[0200] The prompt template processing method provided by the embodiment of the present invention performs unified preprocessing and transformation on training samples, shortens the length of prediction during continuous multiplication, and effectively improves the convergence speed of the mask language model.

[0201] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0202] Alternatively, refer to Figure 4 , Figure 4 The following is a schematic diagram showing the structure of an apparatus for intention recognition provided by an embodiment of the present invention. The apparatus can be configured in an electronic device and includes:

[0203] A training sample acquisition module 401 is configured to acquire a training sample set, wherein the training sample set includes a plurality of training samples, each training sample being a character combination of an abstract intention, and the training sample at least including orderly arranged characters of interest;

[0204] The sample compression processing module 402 is used to compress each training sample according to the number of occurrences of the characters when the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold value to obtain a compressed sample set;

[0205] The model training module 403 is used to input the compressed sample set into a pre-built masked language model for training and output the intent recognition result. The masked language model is used to fill the input compressed sample into a preset position in the language prompt template to be trained and then perform training.

[0206] Optionally, the sample compression processing module 402 further includes:

[0207] A word frequency list acquisition submodule is used to acquire a word frequency list corresponding to the training sample set, wherein the word frequency list includes the focus characters included in the training sample set and the number of occurrences of each focus character in the training sample set;

[0208] A character deduplication processing submodule is used to perform character deduplication processing on each training sample in the training sample set to obtain a deduplication sample set, wherein the deduplication sample set includes multiple deduplication samples, and the deduplication samples correspond one to one to the training samples;

[0209] A compression processing submodule, configured to compress each deduplication sample in the deduplication sample set according to the word frequency list to obtain a compressed sample set, wherein the compressed sample set includes a plurality of compressed samples, and the compressed samples correspond one-to-one to the deduplication samples;

[0210] Optionally, the word frequency list acquisition submodule is further used to:

[0211] Count the number of occurrences of each focus character in the training sample set;

[0212] According to the number of occurrences of each focus character, the focus characters in the training sample set are sorted to obtain a word frequency list corresponding to the training sample set.

[0213] Optionally, the deduplication processing submodule is further used to:

[0214] For each training sample, when it is determined that the training sample contains non-focus characters, the non-focus characters are replaced with preset characters to obtain a replacement sample corresponding to the training sample;

[0215] Performing character deduplication processing on the replacement sample corresponding to the training sample to obtain a deduplication sample corresponding to the replacement sample;

[0216] Add the deduplication sample corresponding to the replacement sample to the deduplication sample set;

[0217] When it is determined that the training sample does not contain non-focus characters, performing character deduplication processing on the training sample to obtain a deduplication sample corresponding to the training sample;

[0218] Add the deduplicated samples corresponding to the training samples to the deduplicated sample set.

[0219] Optionally, the compression processing submodule is further configured to:

[0220] For each deduplicated sample, determine the character length of the deduplicated sample;

[0221] When the character length is greater than or equal to the second threshold, filtering the deduplicated samples according to the word frequency list to obtain a filtered sample corresponding to the deduplicated samples;

[0222] Compressing the filtered samples corresponding to the deduplicated samples to obtain compressed samples corresponding to the filtered samples;

[0223] Add the compressed samples corresponding to the filtered samples to the compressed sample set.

[0224] Optionally, the compression processing submodule is further configured to: for each deduplicated sample, when the character length is less than a second threshold, determine the deduplicated sample as a compressed sample.

[0225] Optionally, the compression processing submodule is further configured to:

[0226] Determining a candidate character set starting from a first focus character in the word frequency list according to the order of arrangement of the word frequency list, the candidate character set including a first threshold number of consecutive focus characters, wherein the word frequency list is sorted in descending order of the number of occurrences of the focus characters in the training sample set;

[0227] The characters of interest included in the deduplication sample are screened according to the candidate character set to obtain a screened sample corresponding to the deduplication sample.

[0228] Optionally, the compression processing submodule is further configured to:

[0229] Performing sliding window processing on the screening sample using a clipping window to obtain a clipping sample corresponding to the screening sample, wherein the size of the clipping window is the second threshold, the step size of each sliding of the clipping window is a preset number of characters, and each clipping sample includes the second threshold number of characters;

[0230] When it is determined that the intercepted sample belongs to a compressed sample already included in the compressed sample set, sliding the interception window according to the step size, returning to the step of performing sliding window processing on the screening sample using the interception window to obtain a intercepted sample corresponding to the screening sample, until it is determined that the intercepted sample does not belong to a compressed sample already included in the compressed sample set;

[0231] When it is determined that the clipped sample does not belong to the compressed samples already included in the compressed sample set, the clipped sample is determined to be the compressed sample corresponding to the screened sample.

[0232] Optionally, the compression processing submodule is further configured to:

[0233] When it is determined that the number of characters included in the screening sample corresponding to the clipping window is less than the size of the clipping window, a supplementary character corresponding to the difference number is set at the end of the characters included in the screening sample corresponding to the clipping window according to the character arrangement order of the screening sample, and the characters included in the screening sample corresponding to the clipping window and the set supplementary character are extracted as the clipping sample corresponding to the screening sample, wherein the difference number is determined by the difference between the size of the clipping window and the number of characters to be clipped in the screening sample corresponding to the clipping window;

[0234] When it is determined that the number of characters contained in the screening sample corresponding to the clipping window is greater than or equal to the size of the clipping window, the characters corresponding to the clipping window are extracted from the screening sample according to the character arrangement order of the screening sample as the clipping sample corresponding to the screening sample.

[0235] Optionally, the position setting module is configured to set the preset character at the first position of the deduplicated sample if the deduplicated sample contains a preset character.

[0236] The prompt template processing device provided by the embodiment of the present invention obtains a training sample set during model pre-training, and then performs unified pre-processing and transformation on the training samples, thereby shortening the length of prediction during multiplication and effectively improving the convergence speed of the mask language model.

[0237] Reference below Figure 5 , Figure 5 The following is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device can be a smartphone (such as an Android phone, iOS phone, etc.), a tablet computer, a PDA, other mobile internet devices (MID), a PAD, a desktop computer, or other terminals. Figure 5 It does not limit the structure of electronic equipment. Figure 5 As shown, the electronic device at least includes a memory 501 and a processor 502. For example, the electronic device may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) may be shown.

[0238] In particular, according to the embodiment provided by the present invention, the above reference flow chart Figure 1-3 The described process can be implemented as a computer software program. For example, embodiments provided by the present invention include a computer program product comprising a computer program carried on a machine-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication portion and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-described functions defined in the system of the present invention are performed.

[0239] It should be noted that the computer-readable medium shown in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0240] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and computer program products described in accordance with the various embodiments provided by the present invention. Each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the aforementioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0241] The units or modules involved in the embodiments described in the present invention may be implemented in software or hardware. The units or modules described may also be provided in a processor. For example, they may be described as follows: a processor includes a training sample acquisition module, a sample compression processing module, and a model training module. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves. For example, the training sample acquisition module may also be described as a "module for acquiring a training sample set."

[0242] As another aspect, embodiments of the present invention further provide a computer-readable storage medium. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently and not incorporated into the electronic device. The computer-readable storage medium stores one or more programs, which, when used by one or more processors, execute the method for intent recognition described in the present invention.

[0243] The above description is merely a preferred embodiment of the present invention and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present invention.

Claims

1. A method for intention recognition, characterized in that: The method includes: Acquire a training sample set, the training sample set including a plurality of training samples, each of the training samples being a character combination of an abstract intention, and the training sample including at least orderly arranged characters of interest; When the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold, compressing each of the training samples according to the number of occurrences of the characters to obtain a compressed sample set; The compressed sample set is input into a pre-built masked language model for training, and the intention recognition result is output. The masked language model is used to fill the input compressed sample into a preset position in the language prompt template to be trained and then train; wherein, The compression processing is performed on each of the training samples according to the number of occurrences of the characters to obtain a compressed sample set, which includes: Obtain a word frequency list corresponding to the training sample set, Performing character deduplication processing on each of the training samples in the training sample set to obtain a deduplication sample set, wherein the deduplication sample set includes a plurality of deduplication samples; Compress each of the deduplicated samples in the deduplicated sample set according to the word frequency list to obtain the compressed sample set; The step of compressing each of the deduplicated samples in the deduplicated sample set according to the word frequency list to obtain the compressed sample set includes: For each of the deduplicated samples, determining the character length of the deduplicated sample; When the character length is greater than or equal to a second threshold, screening the deduplicated samples according to the word frequency list to obtain a screening sample corresponding to the deduplicated samples; Compressing the screened samples corresponding to the deduplicated samples to obtain compressed samples corresponding to the screened samples; The compressing the screening samples corresponding to the deduplicated samples to obtain compressed samples corresponding to the screening samples includes: Performing sliding window processing on the screening sample using a clipping window to obtain a clipping sample corresponding to the screening sample, wherein the size of the clipping window is the second threshold, the step size of each sliding of the clipping window is a preset number of characters, and each clipping sample includes the second threshold number of characters; When it is determined that the clipped sample belongs to a compressed sample already included in the compressed sample set, sliding the clipping window according to the step size, and returning to the step of performing sliding window processing on the screening sample using the clipping window to obtain a clipped sample corresponding to the screening sample, until it is determined that the clipped sample does not belong to a compressed sample already included in the compressed sample set; When it is determined that the clipped sample does not belong to the compressed samples already included in the compressed sample set, the clipped sample is determined to be the compressed sample corresponding to the screened sample.

2. The method according to claim 1, characterized in that The word frequency list includes the focus characters included in the training sample set and the number of occurrences of each focus character in the training sample set; The deduplication samples correspond one-to-one to the training samples; The compressed sample set includes multiple compressed samples, and the compressed samples correspond to the deduplicated samples one by one.

3. The method according to claim 2, characterized in that The performing character deduplication processing on each of the training samples in the training sample set to obtain a deduplication sample set includes: For each training sample, when it is determined that the training sample contains a non-focus character, the non-focus character is replaced with a preset character to obtain a replacement sample corresponding to the training sample; Performing character deduplication processing on the replacement sample corresponding to the training sample to obtain a deduplication sample corresponding to the replacement sample; Adding the deduplicated sample corresponding to the replacement sample to the deduplicated sample set; When it is determined that the training sample does not contain non-focus characters, performing character deduplication processing on the training sample to obtain a deduplication sample corresponding to the training sample; Adding the deduplicated samples corresponding to the training samples to the deduplicated sample set.

4. The method according to claim 3, characterized in that The method further comprises: For each of the deduplicated samples, when the character length is less than a second threshold, the deduplicated sample is determined to be a compressed sample.

5. The method according to claim 1, wherein The step of screening the deduplicated samples according to the word frequency list to obtain screened samples corresponding to the deduplicated samples includes: Determining a set of candidate characters starting from a first focus character in the word frequency list according to the order of arrangement of the word frequency list, the set of candidate characters including a first threshold number of consecutive focus characters, wherein the word frequency list is sorted in descending order of the number of occurrences of the focus characters in the training sample set; The characters of interest included in the deduplicated sample are screened according to the candidate character set to obtain a screened sample corresponding to the deduplicated sample.

6. The method according to claim 1, characterized in that The step of performing sliding window processing on the screening sample using a clipping window to obtain a clipping sample corresponding to the screening sample includes: When it is determined that the number of characters included in the screening sample corresponding to the clipping window is less than the size of the clipping window, a supplementary character corresponding to the difference number is set at the end of the characters included in the screening sample corresponding to the clipping window according to the character arrangement order of the screening sample, and the characters included in the screening sample corresponding to the clipping window and the set supplementary character are extracted as the clipping sample corresponding to the screening sample, wherein the difference number is determined by the difference between the size of the clipping window and the number of characters to be clipped in the screening sample corresponding to the clipping window; When it is determined that the number of characters contained in the screening sample corresponding to the clipping window is greater than or equal to the size of the clipping window, the characters corresponding to the clipping window are extracted from the screening sample according to the character arrangement order of the screening sample as the clipping sample corresponding to the screening sample.

7. The method according to any one of claims 2 to 6, characterized in that: If the deduplicated sample includes a preset character, the preset character is set at the first position of the deduplicated sample.

8. The method according to claim 2, characterized in that The obtaining of a word frequency list corresponding to the training sample set includes: Counting the number of occurrences of each focus character in the training sample set; The focus characters in the training sample set are sorted according to the number of occurrences of each focus character to obtain a word frequency list corresponding to the training sample set.

9. A device for intention recognition, characterized in that: The device includes: A training sample acquisition module is used to acquire a training sample set, wherein the training sample set includes a plurality of training samples, each of which is a character combination of abstract intent, and the training sample at least includes orderly arranged characters of interest; a sample compression processing module, configured to compress each of the training samples according to the number of occurrences of the characters when the total number of characters in the training sample set after deduplication is greater than or equal to a first threshold, to obtain a compressed sample set, wherein the compressed sample set includes a plurality of compressed samples; The model training module is used to input the compressed sample set into a pre-built masked language model for training and output the intent recognition result. The masked language model is used to fill the input compressed sample into a preset position in the language prompt template to be trained and then train it; wherein, The compression processing is performed on each of the training samples according to the number of occurrences of the characters to obtain a compressed sample set, which includes: Obtain a word frequency list corresponding to the training sample set, Performing character deduplication processing on each of the training samples in the training sample set to obtain a deduplication sample set, wherein the deduplication sample set includes a plurality of deduplication samples; Compress each of the deduplicated samples in the deduplicated sample set according to the word frequency list to obtain the compressed sample set; The step of compressing each of the deduplicated samples in the deduplicated sample set according to the word frequency list to obtain the compressed sample set includes: For each of the deduplicated samples, determining the character length of the deduplicated sample; When the character length is greater than or equal to a second threshold, screening the deduplicated samples according to the word frequency list to obtain a screening sample corresponding to the deduplicated samples; Compressing the screened samples corresponding to the deduplicated samples to obtain compressed samples corresponding to the screened samples; The compressing the screening samples corresponding to the deduplicated samples to obtain compressed samples corresponding to the screening samples includes: Performing sliding window processing on the screening sample using a clipping window to obtain a clipping sample corresponding to the screening sample, wherein the size of the clipping window is the second threshold, the step size of each sliding of the clipping window is a preset number of characters, and each clipping sample includes the second threshold number of characters; When it is determined that the clipped sample belongs to a compressed sample already included in the compressed sample set, sliding the clipping window according to the step size, and returning to the step of performing sliding window processing on the screening sample using the clipping window to obtain a clipped sample corresponding to the screening sample, until it is determined that the clipped sample does not belong to a compressed sample already included in the compressed sample set; When it is determined that the clipped sample does not belong to the compressed samples already included in the compressed sample set, the clipped sample is determined to be the compressed sample corresponding to the screened sample.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images