Corpus screening method, intent recognition model optimization method, device and storage medium

Abstract

The embodiments of the present application provide a corpus screening method, an intent recognition model optimization method, a device and a storage medium, and relate to the field of artificial intelligence technology. It includes: obtaining a dictionary file and an online user corpus; performing sliding window segmentation on the online user corpus to obtain character segments; performing matching calculations on the character segments according to the dictionary file to obtain the incoherence scores of each of the character segments; performing weighted calculations on a number of incoherence scores corresponding to the field lengths of the online user corpus to obtain a weighted score; clustering the online user corpus whose weighted scores exceed a preset threshold to obtain a high-frequency corpus; annotating the high-frequency corpus to obtain a new training corpus for optimizing the intent recognition model of the intelligent customer service system. It is possible to screen out new expressions with a higher frequency by regularly processing the customer expression texts in the intelligent customer service system and the customer conversation log, and to enable the intelligent customer service system to make reasonable feedback to customer needs in a timely manner.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a corpus screening method, an intent recognition model optimization method, a device, and a storage medium. Background Art

[0002] The intelligent customer service system is a computer software system based on artificial intelligence that provides customers with information related to the products or services sold by the enterprise. Its main function is to identify the customer's intentions, then give corresponding replies and solve the customer's problems. As the core module of the intelligent customer service system, the underlying implementation of the intention recognition function is generally a retrieval or classification model directly built based on the corpus or trained by machine learning methods. This means that the intention recognition effect of the intelligent customer service system depends to a large extent on the quality of the corpus it uses. In particular, for expressions that have not appeared in the corpus, it is difficult for the current intelligent customer service system to effectively recognize them. In order to achieve the purpose of effective recognition, the traditional approach is for the business side of the enterprise to propose the need to update the intelligent customer service system and provide the corresponding corpus related to the new business. After the developer updates the corpus, the intention recognition model is retrained to realize the recognition of the new corpus. However, in reality, this approach has many disadvantages. On the one hand, the company's products are often in rapid iteration, and new products and new activities emerge in an endless stream. The business side has limited manpower and material resources, and it is difficult to ensure that every time a new business is launched, it can promptly and proactively propose the need to update the intelligent customer service system. On the other hand, the scale of customers' customer service needs often depends on many factors such as product popularity and functional completeness. For businesses with less market popularity and less customer response, the urgency of updating the intelligent customer service system is lower. Indiscriminately updating the intelligent customer service system in advance for each business change is a waste of the company's manpower and material resources. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present application provide a corpus screening method and an intent recognition model optimization method, which can screen out new expressions with higher frequency by regularly processing customer expression texts in the intelligent customer service system and customer conversation logs. These new expressions often reflect the customer's latest customer service needs for the services provided by the company. These new expressions are added as corpus to the current training corpus to retrain the intent recognition model, which can enable the intelligent customer service system to make reasonable feedback to customer needs in a timely manner, while effectively reducing the waste of manpower and material resources caused by frequent updates of the intelligent customer service system.

[0005] In a first aspect, an embodiment of the present application provides a corpus screening method, comprising:

[0006] Obtain dictionary files and online user corpus;

[0007] Performing sliding window segmentation on the online user corpus to obtain character segments;

[0008] Perform matching calculation on the character segments according to the dictionary file to obtain an incoherence score of each character segment;

[0009] Performing weighted calculation on a number of incoherence scores corresponding to the length of the online user corpus field to obtain a weighted score;

[0010] Clustering the online user corpora whose weighted scores exceed a preset threshold to obtain high-frequency corpora;

[0011] The high-frequency corpus is annotated to obtain new training corpus for optimizing the intent recognition model of the intelligent customer service system.

[0012] In some optional implementations, before performing sliding window segmentation on the online user corpus to obtain character segments, the method further includes:

[0013] Each of the online user corpus is subjected to character duplication removal and stop word filtering processing.

[0014] In some optional implementations, the online user corpus includes user corpus that is incorrectly recognized or cannot be recognized by the intelligent customer service system in the online conversation log.

[0015] In some optional implementations, performing sliding window segmentation on the online user corpus to obtain character segments includes:

[0016] Taking the number of characters as the segment length, the online user corpus is segmented by single-step sliding window according to the segment lengths of 1 to N to obtain a plurality of the character segments; N is a positive integer greater than or equal to 2.

[0017] In some optional implementations, performing matching calculations on the character segments according to the dictionary file to obtain incoherence scores of the character segments includes:

[0018] Searching the character segment in the dictionary file of corresponding length;

[0019] According to the query result, the probability of occurrence of the character fragment is obtained;

[0020] The incoherence score is obtained by subtracting the probability of occurrence of the character segment from 1.

[0021] In some optional embodiments, the weight of the corresponding incoherence score is configured according to the field length of the online user corpus and a first preset rule; the first preset rule includes: configuring a first weight for the incoherence score corresponding to the character segment whose field length is less than a first threshold or greater than a second threshold, otherwise configuring a second weight, the second threshold is greater than the first threshold, and the second weight is greater than the first weight.

[0022] In some optional implementations, clustering the online user corpora whose weighted scores exceed a preset threshold to obtain high-frequency corpora includes:

[0023] The online user corpus whose weighted score exceeds a preset threshold is converted into multiple vectors through a language model;

[0024] Selecting a core point from a plurality of said vectors;

[0025] According to the clustering radius, the vector in the vicinity of the core point is calculated as a similar vector;

[0026] The high-frequency corpus is obtained according to the number of the similar vectors.

[0027] In some optional implementations, obtaining a dictionary file includes:

[0028] Taking the number of characters as the segment length, the current training prediction is segmented into character segments one by one according to the segment length of 1 to N to generate the dictionary file; N is a positive integer greater than or equal to 2.

[0029] In a second aspect, an embodiment of the present application provides a method for optimizing an intent recognition model, characterized in that a new training corpus is obtained by applying the corpus screening method described in the first aspect, and the intent recognition model of the intelligent customer service system is optimized based on the new training corpus.

[0030] In a third aspect, the present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the corpus screening method described in the first aspect or the intent recognition model optimization method described in the second aspect is implemented.

[0031] In a fourth aspect, the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to implement, when executed by a processor:

[0032] Such as the corpus screening method described in the first aspect or the intention recognition model optimization method described in the second aspect.

[0033] The first aspect of the embodiment of the present application provides a corpus screening method, firstly obtain a dictionary file and an online user corpus, then perform a single-step sliding window segmentation on the online user corpus to obtain character segments, use the number of characters as the segment length, perform a single-step sliding window segmentation one by one according to the segment length of 1 to 4, obtain multiple character segments, and record the number of occurrences of each character segment during the segmentation process. For example, if the field length of the user corpus on an online line is 10, a window with a length of 1 is used for sliding segmentation once to obtain 10 character segments, a window with a length of 2 is used for sliding segmentation once to obtain 9 character segments, a window with a length of 3 is used for sliding segmentation once to obtain 8 character segments, and a window with a length of 4 is used for sliding segmentation once to obtain 7 character segments. After all the corpus are processed, for each character segment, its number of occurrences is divided by the total number of occurrences of all character segments with the same length to obtain its probability of occurrence at that length. All character segments with the same length and their corresponding probability of occurrence are stored in the dictionary file respectively. Then, the character fragments are matched and calculated according to the dictionary file to obtain the incoherence score of each character fragment, and then the several incoherence scores corresponding to the field length of the online user corpus are weighted to obtain the weighted score; the online user corpus whose weighted score exceeds the preset threshold is clustered to obtain the high-frequency corpus; the high-frequency corpus is annotated to obtain the new training corpus, and then the new training corpus is added to the current training corpus to train and optimize the intention recognition model of the intelligent customer service system, and the new training corpus is segmented and added to the dictionary file.

[0034] The corpus screening method provided in the embodiment of the present application can screen out new expressions with higher frequency by regularly processing customer expression texts in the intelligent customer service system and customer conversation logs. These new expressions often reflect the customer's latest customer service needs for the services provided by the company. These new expressions are added as corpus to the current training corpus to retrain the intent recognition model, which can enable the intelligent customer service system to make reasonable feedback to customer needs in a timely manner, while effectively reducing the waste of manpower and material resources caused by frequent updates of the intelligent customer service system.

[0035] It can be understood that the beneficial effects of the second to third aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 is a flow chart of a corpus screening method provided by an embodiment of the present invention;

[0038] Figure 2 is a flow chart of a corpus screening method provided by another embodiment of the present invention;

[0039] Figure 3 is a flow chart of a corpus screening method provided by another embodiment of the present invention;

[0040] Figure 4 is a flow chart of a corpus screening method provided by another embodiment of the present invention;

[0041] Figure 5 It is a flow chart of a corpus screening method provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0042] In the following description, specific details such as specific system structures, technologies, etc. are proposed for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the embodiments of the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from hindering the description of the embodiments of the present application.

[0043] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0044] It should also be understood that the references to "one embodiment" or "some embodiments" described in the specification of the embodiments of the present application mean that one or more embodiments of the embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0045] Taking into account the difficulties in continuous optimization of the intent recognition model of the existing intelligent customer service system, the present application proposes a corpus screening method, which is based on a user data return mechanism of text fluency and clustering methods. By regularly processing the customer expression texts in the intelligent customer service system and customer conversation logs, new expressions with higher frequency are screened. These new expressions often reflect the customer's latest customer service needs for the services provided by the company. After manual review, these new expressions are added as corpus to the current training corpus to retrain the intent recognition model, so that the intelligent customer service system can make reasonable feedback to customer needs in a timely manner.

[0046] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0047] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0048] The intelligent customer service system is a computer software system based on artificial intelligence that provides customers with information related to the products or services sold by the enterprise. Its main function is to identify the customer's intentions, then give corresponding replies and solve the customer's problems. As the core module of the intelligent customer service system, the underlying implementation of the intention recognition function is generally a retrieval or classification model directly built based on the corpus or trained by machine learning methods. This means that the intention recognition effect of the intelligent customer service system depends to a large extent on the quality of the corpus it uses. In particular, for expressions that have not appeared in the corpus, it is difficult for the current intelligent customer service system to effectively recognize them. In order to achieve the purpose of effective recognition, the traditional approach is for the business side of the enterprise to propose the need to update the intelligent customer service system and provide the corresponding corpus related to the new business. After the developer updates the corpus, the intention recognition model is retrained to realize the recognition of the new corpus. However, in reality, this approach has many disadvantages. On the one hand, the company's products are often in rapid iteration, and new products and new activities emerge in an endless stream. The business side has limited manpower and material resources, and it is difficult to ensure that every time a new business is launched, it can promptly and proactively propose the need to update the intelligent customer service system. On the other hand, the scale of customers' customer service needs often depends on many factors such as product popularity and functional completeness. For businesses with less market popularity and less customer response, the urgency of updating the intelligent customer service system is lower. Indiscriminately updating the intelligent customer service system in advance for each business change is a waste of the company's manpower and material resources.

[0049] Based on this, the corpus screening method, intent recognition model optimization method, device and storage medium provided in the embodiments of the present application can screen out new expressions with higher frequency by regularly processing customer expression texts in the intelligent customer service system and customer conversation logs. These new expressions often reflect the customer's latest customer service needs for the services provided by the company. These new expressions are added as corpus to the current training corpus to retrain the intent recognition model, which can enable the intelligent customer service system to make reasonable feedback to customer needs in a timely manner, while effectively reducing the waste of manpower and material resources caused by frequent updates of the intelligent customer service system.

[0050] The corpus screening method provided by the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0051] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0052] Reference Figure 1 and Figure 2 , Figure 1 is a flow chart of a corpus screening method provided by an embodiment of the present invention. Figure 2 It is a flowchart of a corpus screening method provided by another embodiment of the present invention. The corpus screening method includes but is not limited to steps S110 to S160.

[0053] Step S110, obtaining a dictionary file and online user corpus;

[0054] Step S120, performing sliding window segmentation on the online user corpus to obtain character segments;

[0055] Step S130, performing matching calculations on the character segments according to the dictionary file to obtain an incoherence score for each character segment;

[0056] Step S140, performing weighted calculation on a number of incoherence scores corresponding to the length of the field of the online user corpus to obtain a weighted score;

[0057] Step S150, clustering online user corpora whose weighted scores exceed a preset threshold to obtain high-frequency corpora;

[0058] Step S160: annotate the high-frequency corpus to obtain new training corpus for optimizing the intent recognition model of the intelligent customer service system.

[0059] In one embodiment, first, a dictionary file and online user corpus are obtained, and then the online user corpus is segmented by a single-step sliding window to obtain character segments, with the number of characters as the segment length, and the segment length is N. For example, single-step sliding window segmentation is performed one by one according to the segment lengths of 1 to 4, to obtain multiple character segments, and the number of occurrences of each character segment during the segmentation process is recorded. For example, if the field length of the user corpus on an online line is 10, N=1, a window with a length of 1 is used for sliding segmentation once, and 10 character segments are obtained; N=2, a window with a length of 2 is used for sliding segmentation once, and 9 character segments are obtained; N=3, a window with a length of 3 is used for sliding segmentation once, and 8 character segments are obtained; N=4, a window with a length of 4 is used for sliding segmentation once, and 7 character segments are obtained. After all corpora are processed, for each character segment, its number of occurrences is divided by the total number of occurrences of all character segments with the same length, to obtain its probability of occurrence at that length. All character segments of the same length and their corresponding probability of occurrence are stored in the dictionary file respectively. Then, the character fragments are matched and calculated according to the dictionary file to obtain the incoherence score of each character fragment, and then the several incoherence scores corresponding to the field length of the online user corpus are weighted to obtain the weighted score; the online user corpus with weighted scores exceeding the preset threshold is clustered to obtain the high-frequency corpus; the high-frequency corpus is annotated to obtain the new training corpus; the new training corpus is used as the current training corpus and input into the intent recognition model for training, and the current training corpus is segmented and added to the dictionary file.

[0060] It can be understood that for each character segment, the probability of the character segment appearing can be obtained by querying the dictionary file, and then the incoherence score of the character segment is calculated, and its probability is subtracted from 1, which is defined as the incoherence score of the character segment. For all segments of each length, the arithmetic mean of their incoherence scores is used as the incoherence score of the current corpus of that length, so that each corpus can get at most 4 incoherence scores, and then these scores are weighted averaged to obtain the final incoherence score of the corpus. For sentences of a certain length, the impact of a single character on coherence is small. On the other hand, the longer the character segment, the lower its probability of occurrence itself should be. Therefore, when calculating the above weighted average result, the weights of the incoherence scores of single-character segments and longer character segments should be low. A reference weight ratio (length from 1 to 4) is 1:4:4:1. If the corpus length is less than 4, the corresponding ratio can be truncated. For example, if the corpus length is 2, it has two incoherence scores, and the weighted average weights are 1 / 5 and 4 / 5 respectively; if the corpus length is 3, it has 3 incoherence scores, and the weighted average weights are 1 / 9, 4 / 9, and 4 / 9 respectively; if the corpus length is 4, it has 4 incoherence scores, and the weighted average weights are 1 / 10, 4 / 10, 4 / 10, and 1 / 10 respectively. Generally speaking, the corpus length is usually less than 4. The four weighting methods mentioned in this application also cover most of the possibilities and have good reliability.

[0061] It should be noted that after weighted calculation of several incoherence scores corresponding to the length of the field of the online user corpus to obtain a weighted score, the online user corpus with a weighted score exceeding a preset threshold is clustered to obtain a high-frequency corpus, that is, an online user corpus with a high degree of incoherence but a high frequency of occurrence. These high-frequency corpora are usually corpora with a high frequency of occurrence but cannot be well recognized by the existing intelligent customer service system, but actually reflect the high-frequency needs of users. Because the number of corpora that can be processed by clustering is limited, the threshold can be adjusted according to the number of corpora screened during screening, generally not exceeding 20,000. The corpus screening method provided in this embodiment provides for adjusting the preset threshold so that the high-frequency corpus is usually not more than 20,000, and then the high-frequency corpus is processed by the language model to obtain a vector, and then clustered using the DBSCAN (Density-Based Spatial Clustering of Applications with Noise, density-based clustering algorithm) algorithm.

[0062] Finally, after clustering is completed, user corpora with semantic similarity and high frequency of occurrence will be screened out and marked with corresponding category information. These corpora can be annotated according to the category, and it can be judged whether the incoherent features of these corpora are caused by problems with user-side input or by imperfect corpora of the intent recognition model itself. The clustering method can not only exclude most of the long-tail abnormal texts caused by noise or improper user input, but also screen and annotate the clustered texts according to category information, which can greatly improve the efficiency of discovering meaningful new corpora, and effectively reduce the waste of manpower and material resources caused by frequent updates of the intelligent customer service system. It can be understood that the method of annotating user corpora with semantic similarity and high frequency of occurrence can be through machine annotation, or it can be manual annotation of some user corpora with high frequency of occurrence, and this embodiment does not limit it.

[0063] refer to Figure 3 , Figure 3 2 is a flowchart of a corpus screening method provided by another embodiment of the present invention. The corpus screening method includes but is not limited to steps S210 to S260.

[0064] Step S210, obtaining a dictionary file and online user corpus, and performing character deduplication and stop word filtering processing on each online user corpus;

[0065] Step S220, performing sliding window segmentation on the online user corpus to obtain character segments;

[0066] Step S230, performing matching calculations on the character segments according to the dictionary file to obtain an incoherence score for each character segment;

[0067] Step S240, performing weighted calculation on a number of incoherence scores corresponding to the length of the online user corpus field to obtain a weighted score;

[0068] Step S250, clustering the online user corpora whose weighted scores exceed a preset threshold to obtain high-frequency corpora;

[0069] Step S260: annotate the high-frequency corpus to obtain new training corpus for optimizing the intent recognition model of the intelligent customer service system.

[0070] In one embodiment, first, a dictionary file and online user corpus are obtained, and each piece of online user corpus is subjected to character deduplication and stop word filtering. Then, the online user corpus is segmented by single-step sliding window to obtain character segments, with the number of characters as the segment length. The single-step sliding window segmentation is performed one by one according to the segment lengths from 1 to 4, obtaining multiple character segments. The occurrence times of each character segment during the segmentation process are recorded. After all the corpora are processed, for each character segment, its occurrence probability at this length is obtained by dividing its occurrence times by the total occurrence times of all character segments of the same length. All character segments of the same length and their corresponding occurrence probabilities are stored in the dictionary file respectively. Then, based on the dictionary file, matching calculations are performed on the character segments to obtain the unsmooth scores of each character segment, and then several unsmooth scores corresponding to the field length of the online user corpus are weighted to obtain a weighted score; the online user corpus with the weighted score exceeding the preset threshold is clustered to obtain high-frequency corpus; the high-frequency corpus is labeled to obtain new training corpus, and then the new training corpus is trained in the model and added to the dictionary file after segmentation.

[0071] It can be understood that each piece of online user corpus is subjected to character deduplication and stop word filtering. The obtained online user corpus may contain some repeated characters in some cases, such as corpora like "the the the", "good good", "I I I", etc. However, the semantics of this type of corpus need to be deduplicated to be accurately obtained; moreover, there are also some stop words in the intelligent customer service system. Before segmenting the online user corpus by single-step sliding window to obtain character segments, stop word filtering can improve the efficiency of data processing, save computing resources, and has good practical value.

[0072] In one embodiment, the online user corpus includes the user corpus that the intelligent customer service system fails to recognize or recognizes incorrectly in the online conversation log. The corpus that the intelligent customer service system fails to recognize is usually the user corpus not included in the dictionary file, but it appears frequently, indicating that these user corpora are usually some common user problems brought about by version updates, new product launches, etc. These user corpora often also contain some useful information. After segmenting the unrecognized user corpus and the corpus with incorrect system recognition by sliding window to obtain multiple character segments, then, based on the dictionary file, matching calculations are performed on the character segments to obtain the unsmooth scores of each character segment, and then several unsmooth scores corresponding to the field length of the online user corpus are weighted to obtain a weighted score; the online user corpus with the weighted score exceeding the preset threshold is clustered to obtain high-frequency corpus; the high-frequency corpus is labeled to obtain new training corpus, and then the new training corpus is trained in the model and added to the dictionary file after segmentation, which can improve the efficiency of data processing, save computing resources, and extract useful information from the user corpus that the intelligent customer service system fails to recognize or recognizes incorrectly better.

[0073] refer to Figure 4 , Figure 4 It is a flowchart of a corpus screening method provided by another embodiment of the present invention. The corpus screening method includes but is not limited to steps S310 to S360.

[0074] Step S310, obtaining a dictionary file and online user corpus;

[0075] Step S320, using the number of characters as the segment length, and performing single-step sliding window segmentation on the online user corpus one by one according to the segment lengths from 1 to N, to obtain multiple character segments, where N is a positive integer greater than or equal to 2;

[0076] Step S330, performing matching calculations on the character segments according to the dictionary file to obtain an incoherence score for each character segment;

[0077] Step S340, performing weighted calculation on a number of incoherence scores corresponding to the length of the field of the online user corpus to obtain a weighted score;

[0078] Step S350, clustering online user corpora whose weighted scores exceed a preset threshold to obtain high-frequency corpora;

[0079] Step S360: annotate the high-frequency corpus to obtain new training corpus for optimizing the intent recognition model of the intelligent customer service system.

[0080] In one embodiment, a sliding window segmentation is performed on the current training prediction to obtain character segments and add them to a dictionary file, the dictionary file and the online user corpus are obtained, and then a single-step sliding window segmentation is performed on the online user corpus to obtain character segments. The number of characters is used as the segment length, and single-step sliding window segmentation is performed one by one according to the segment length of 1 to 4 to obtain multiple character segments. The number of occurrences of each character segment in the segmentation process is recorded. For example, if the field length of the user corpus on an line is 10, a window with a length of 1 is used for sliding segmentation once to obtain 10 character segments, a window with a length of 2 is used for sliding segmentation once to obtain 9 character segments, a window with a length of 3 is used for sliding segmentation once to obtain 8 character segments, and a window with a length of 4 is used for sliding segmentation once to obtain 7 character segments. After all the corpora are processed, for each character segment, its number of occurrences is divided by the total number of occurrences of all character segments with the same length to obtain its probability of occurrence at that length, and all character segments with the same length and their corresponding probability of occurrence are stored in the dictionary file respectively.

[0081] In one embodiment, the current training prediction is segmented by sliding window to obtain character segments and added to the dictionary file, the dictionary file and online user corpus are obtained, and each online user corpus is subjected to character deduplication and stop word filtering. The online user corpus is then segmented by single-step sliding window to obtain character segments, and the number of characters is used as the segment length. Single-step sliding window segmentation is performed one by one according to the segment length of 1 to 4 to obtain multiple character segments, and the number of occurrences of each character segment during the segmentation process is recorded. After all corpora have been processed, for each character segment, its number of occurrences is divided by the total number of occurrences of all character segments of the same length to obtain its probability of occurrence at that length. All character segments of the same length and their corresponding probability of occurrence are stored in the dictionary file respectively. Then, the character segment is queried in the dictionary file of the corresponding length to obtain the probability of the character segment appearing, and the probability of the character segment appearing is subtracted from 1 to obtain the incoherence score. Then, several incoherence scores corresponding to the field lengths of the online user corpus are weighted and calculated to obtain a weighted score; the online user corpus whose weighted scores exceed the preset threshold is clustered to obtain a high-frequency corpus; the high-frequency corpus is annotated to obtain a new training corpus, and then the new training corpus is trained in the model, segmented, and added to the dictionary file.

[0082] In one embodiment, the current training prediction is segmented by sliding window to obtain character segments and added to the dictionary file, the dictionary file and the online user corpus are obtained, and then the online user corpus is segmented by single-step sliding window to obtain character segments. The number of characters is used as the segment length, and the single-step sliding window segmentation is performed one by one according to the segment length of 1 to 4 to obtain multiple character segments. The number of occurrences of each character segment in the segmentation process is recorded. For example, if the field length of an online user corpus is 10, a window with a length of 4 is used for sliding segmentation once to obtain 7 character segments. When the field length of the online user corpus is 4, the weight ratio of the character segment is 1:4:4:1, and a weighted score is obtained by weighted calculation based on several incoherent scores of the field length of the online user corpus. After all the corpus are processed, for each character segment, its number of occurrences is divided by the total number of occurrences of all character segments with the same length to obtain its probability of occurrence at that length, and all character segments with the same length and their corresponding probability of occurrence are stored in the dictionary file respectively.

[0083] refer to Figure 5 , Figure 5 is a flow chart of a corpus screening method provided by another embodiment of the present invention, the corpus screening method includes but is not limited to steps S410 to S460;

[0084] Step S410, obtaining a dictionary file and online user corpus;

[0085] Step S420, performing sliding window segmentation on the online user corpus to obtain character segments;

[0086] Step S430, performing matching calculations on the character segments according to the dictionary file to obtain an incoherence score for each character segment;

[0087] Step S440, performing weighted calculation on a number of incoherence scores corresponding to the length of the field of the online user corpus to obtain a weighted score;

[0088] Step S450, converting the online user corpus whose weighted scores exceed a preset threshold into multiple vectors through a language model, selecting a core point from the multiple vectors, calculating the vectors in the vicinity of the core point according to the clustering radius as similar vectors, and obtaining high-frequency corpus according to the number of similar vectors;

[0089] Step S460: annotate the high-frequency corpus to obtain new training corpus for optimizing the intent recognition model of the intelligent customer service system.

[0090] In one embodiment, the current training prediction is segmented by sliding window to obtain character segments and added to the dictionary file, the dictionary file and online user corpus are obtained, and each online user corpus is subjected to character deduplication and stop word filtering. The online user corpus is then segmented by single-step sliding window to obtain character segments, and the number of characters is used as the segment length. Single-step sliding window segmentation is performed one by one according to the segment length of 1 to 4 to obtain multiple character segments, and the number of occurrences of each character segment during the segmentation process is recorded. After all corpuses have been processed, for each character segment, its number of occurrences is divided by the total number of occurrences of all character segments of the same length to obtain its probability of occurrence at that length. All character segments of the same length and their corresponding probability of occurrence are stored in the dictionary file respectively. Then, the character fragments are matched and calculated according to the dictionary file to obtain the incoherence score of each character fragment, and then the several incoherence scores corresponding to the field length of the online user corpus are weighted to obtain the weighted score; the online user corpus with weighted scores exceeding the preset threshold is clustered to obtain high-frequency corpus; the high-frequency corpus is annotated to obtain new training corpus, and then the new training corpus is trained in the model, segmented and added to the dictionary file.

[0091] It is understandable that the online user corpus with a weighted score exceeding the preset threshold is clustered to obtain a high-frequency corpus. Because the number of corpora that can be processed by clustering is limited, the threshold can be adjusted according to the number of screened corpora during screening, generally not exceeding 20,000. The online user corpus with a weighted score exceeding the preset threshold is converted into multiple vectors through a language model. The language model can be a model that can convert the corpus into a vector, including but not limited to TF-IDF (term frequency-inverse document frequency), BERT (Bidirectional Encoder Representations from Transformer, changing bidirectional encoder representation) model, etc., which is not limited in this application. If an object contains more than a preset threshold number of points within its clustering radius, the object is set as a core point, and then the vector within the core point clustering radius is calculated according to the clustering radius of the cluster, as a similar vector, and then the high-frequency corpus is obtained according to the number of similar vectors. The objects on the circumference of the clustering radius are boundary points, and the objects outside the clustering radius are noise points. The set of samples connected by the maximum density derived from the density-reachable relationship is a cluster (a set of multiple similar vectors) of the final clustering. Of course, multiple core points can be selected, and there can be one or more core points in a cluster. If there is only one core point, the other non-core point samples in the cluster are in the neighborhood of this core point. If there are multiple core points, there must be another core point in the neighborhood of any core point in the cluster. The size of the clustering radius can also be adjusted according to demand, thereby adjusting the number of clusters formed. In some embodiments, a cluster with a large number of similar vectors can be selected as a high-frequency corpus. For example, a cluster with a number of similar vectors greater than a preset threshold can be selected as a high-frequency corpus for subsequent processing steps.

[0092] The corpus screening method provided in the embodiment of the present application can screen out new expressions with higher frequency by regularly processing customer expression texts in the intelligent customer service system and customer conversation logs. These new expressions often reflect the customer's latest customer service needs for the services provided by the company. These new expressions are added as corpus to the current training corpus to retrain the intent recognition model, which can enable the intelligent customer service system to make reasonable feedback to customer needs in a timely manner, while effectively reducing the waste of manpower and material resources caused by frequent updates of the intelligent customer service system.

[0093] The embodiment of the present application also provides an intent recognition model optimization method, which applies the above-mentioned corpus screening method to obtain new training corpus, and optimizes the intent recognition model of the intelligent customer service system based on the new training corpus.

[0094] The intent recognition model optimization method provided in the embodiment of the present application can screen out new expressions with high frequency by regularly processing the text of customer expressions in the intelligent customer service system and customer dialogue logs. These new expressions often reflect the latest customer service needs of customers for the services provided by the company, and add these new expressions as corpus to the current training corpus to retrain the intent recognition model and continuously optimize the intent recognition model. It can enable the intelligent customer service system to make reasonable feedback to customer needs in a timely manner, and effectively reduce the waste of manpower and material resources caused by frequent updates of the intelligent customer service system.

[0095] In addition, the present application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the aforementioned corpus screening method when executing the computer program.

[0096] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0097] It should be noted that the electronic device in this embodiment can be applied as follows Figure 1 The electronic device in the system architecture of the embodiment shown in the figure; in addition, the electronic device in this embodiment can execute the following Figure 2 , Figure 3 , Figure 4 or Figure 5 The corpus screening method in the embodiment shown in FIG. Figure 1 The electronic device in the system architecture of the embodiment shown, and Figure 2 , Figure 3 , Figure 4 or Figure 5 The corpus screening methods in the illustrated embodiments all belong to the same inventive concept, and therefore these embodiments have the same implementation principles and technical effects, which will not be described in detail here.

[0098] The non-transient software program and instructions required to implement the corpus screening method of the above embodiment are stored in the memory. When executed by the processor, the corpus screening method of the above embodiment is executed, for example, the above described Figure 1 Steps S110 to S160 of the method, Figure 3Steps S210 to S260 of the method, Figure 4 Steps S310 to S360 of the method, Figure 5 The method comprises steps S410 to S460.

[0099] In addition, the present application also provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the following when executed by a processor:

[0100] As mentioned above corpus screening method.

[0101] In some embodiments, the computer-readable storage medium stores computer-executable instructions, which are executed by a processor or controller, for example, by a processor in the above embodiment, so that the processor can execute the corpus screening method in the above embodiment, for example, execute the above described Figure 1 Steps S110 to S160 of the method, Figure 3 Steps S210 to S260 of the method, Figure 4 Steps S310 to S360 of the method, Figure 5 Method steps S410 to S460.

[0102] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0104] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0105] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0106] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A corpus screening method, characterized in that: include: Obtain dictionary files and online user corpus; Taking the number of characters as the segment length, the online user corpus is segmented by single-step sliding window according to the segment lengths from 1 to N, to obtain multiple character segments; N is a positive integer greater than or equal to 2; Searching the character segment in the dictionary file of corresponding length; According to the query result, the probability of occurrence of the character fragment is obtained; Subtract the probability of occurrence of the character segment from 1 to obtain the incoherence score of the character segment; Performing weighted calculation on a number of incoherence scores corresponding to the length of the online user corpus field to obtain a weighted score; Converting the online user corpus whose weighted score exceeds a preset threshold into multiple vectors through a language model; Selecting a core point from a plurality of said vectors; According to the clustering radius, the vector in the vicinity of the core point is calculated as a similar vector; According to the number of the similar vectors, high-frequency corpus is obtained; The high-frequency corpus is annotated to obtain new training corpus for optimizing the intent recognition model of the intelligent customer service system.

2. The corpus screening method according to claim 1, characterized in that: Before performing sliding window segmentation on the online user corpus to obtain character segments, the method further includes: Each of the online user corpus is subjected to character duplication removal and stop word filtering processing.

3. The corpus screening method according to claim 1, characterized in that: The online user corpus includes user corpus that is incorrectly recognized or cannot be recognized by the intelligent customer service system in the online conversation log.

4. The corpus screening method according to claim 1, characterized in that: The weighted calculation of a plurality of incoherence scores corresponding to the length of the field of the online user corpus to obtain a weighted score includes: The weight of the corresponding incoherence score is configured according to the field length of the online user corpus and the first preset rule; the first preset rule includes: configuring a first weight for the incoherence score corresponding to the character segment whose field length is less than a first threshold or greater than a second threshold, otherwise configuring a second weight, the second threshold is greater than the first threshold, and the second weight is greater than the first weight.

5. The corpus screening method according to claim 1, characterized in that: The step of obtaining a dictionary file includes: Taking the number of characters as the segment length, the current training prediction is segmented into character segments one by one according to the segment length of 1 to N to generate the dictionary file; N is a positive integer greater than or equal to 2.

6. A method for optimizing an intent recognition model, characterized in that: The corpus screening method as described in any one of claims 1 to 5 is applied to obtain new training corpus, and the intelligent customer service system intention recognition model is optimized based on the new training corpus.

7. Electronic equipment, including: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the corpus screening method as described in any one of claims 1 to 5 or the intent recognition model optimization method as described in claim 6 is implemented.

8. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used to implement, when executed by a processor: The corpus screening method according to any one of claims 1 to 5 or the intent recognition model optimization method according to claim 6.

Images