Classification model training method, text mining method, device, and storage medium

Abstract

The application discloses a kind of classification model training method, text mining method, equipment and medium, it is related to computer technical field.Training method will obtain multiple dialogue texts as training data, according to the training data, training is executed to classification model, and the classification model of stage training is completed is obtained.Wherein, the training data is filtered using clustering analysis mode, to scene category, training data is marked, so as to greatly reduce the annotation amount of training data.Moreover, according to the difference information determined by training data, it is judged whether the classification model of stage training is completed needs to continue training.Thus, in the case where difference information meets scene mining condition, more scene categories can be mined by continuing training, so that in the process of text mining, more refined scene categories and less information related texts can be mined, so as to facilitate the statistical analysis of related dialogue texts.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a classification model training method, a text mining method, a device and a storage medium. BACKGROUND

[0002] In some online business scenarios, when a business staff provides business consultation for a customer, a large amount of consultation data will be retained. For the business staff with less experience, the professional words in the consultation data are worth learning. In the prior art, a classification model is generally used to statistically analyze the consultation data, so that learning can be performed based on different scene categories. However, the classification model needs a large amount of labeled data in the training process, and lacks the ability to discover new scenes after fixed scene classification. SUMMARY

[0003] In view of the above problems, the present application is proposed in order to provide a classification model training method, a text mining method, a device and a storage medium which overcome the above problems or at least partially solve the above problems.

[0004] According to a first aspect of the present application, a classification model training method is provided, the method comprising:

[0005] obtaining a plurality of dialogue texts as training data;

[0006] performing training on the classification model according to the training data to obtain a classification model trained in a stage;

[0007] determining difference information based on the training data, and judging whether the classification model trained in the stage meets a scene mining condition according to the difference information;

[0008] if the difference information meets the scene mining condition, updating the training data and continuing training according to the training data and the classification model trained in the stage;

[0009] if the difference information does not meet the scene mining condition, taking the classification model trained in the stage as a classification model trained.

[0010] wherein performing training on the classification model according to the training data to obtain a classification model trained in a stage comprises:

[0011] performing cluster analysis on the training data to determine a plurality of training class clusters;

[0012] determining a scene category corresponding to the training class cluster, and labeling the text of the training class cluster as the scene category;

[0013] training the classification model using the labeled text data to obtain the classification model trained in the stage.

[0014] According to a second aspect of the present application, a text mining method is also provided, the method comprising:

[0015] receiving dialogue information, and obtaining dialogue text of a first user from the dialogue information;

[0016] inputting the dialogue text into a classification model for classification recognition, determining a corresponding target scene category, training the classification model by using training data to obtain a classification model trained in a stage, determining difference information based on the training data, judging whether the classification model trained in the stage meets a scene mining condition, and determining whether to update the training data to continue training the classification model trained in the stage according to a judgment result;

[0017] querying target reply text corresponding to the target scene category;

[0018] using the target reply text as dialogue text of a second user, and feeding back the dialogue text of the first user.

[0019] According to a third aspect of the present application, an electronic device is also provided, comprising:

[0020] one or more processors;

[0021] a memory;

[0022] one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute any of the above-mentioned methods.

[0023] According to a fourth aspect of the present application, a computer readable storage medium is also provided, which stores a computer program used in combination with an electronic device, and the computer program can be executed by a processor to complete any of the above-mentioned methods.

[0024] In the present application, a plurality of obtained dialogue texts are used as training data, and a classification model is trained according to the training data to obtain a classification model trained in a stage. In this way, the training data is screened by using a clustering analysis method, and the training data is labeled by using a scene category, so that the labeling amount of the training data can be greatly reduced. In addition, whether the classification model trained in the stage needs to be continuously trained is determined according to difference information of the training data. In the case that the difference information meets a scene mining condition, more scene categories can be mined by continuous training. In the text mining process, more refined scene categories and related texts with less information can be mined, so that statistical analysis of related dialogue texts can be facilitated.

[0025] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be construed as limiting the present invention. The same reference numerals are used throughout the accompanying drawings to denote the same components.

[0027] In the attached figure:

[0028] Figure 1 This is a flowchart of the steps of a classification model training method provided by an embodiment of the present invention;

[0029] Figure 2 is a flowchart of another classification model training method provided by an embodiment of the present invention;

[0030] Figure 3 This is a flowchart of the steps of a text mining method provided by an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of display content of a display page of a first client provided by an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of display content of a display page of a first client provided by an embodiment of the present invention;

[0033] Figure 6 This is another schematic diagram of display content of a display page of a first client provided by an embodiment of the present invention;

[0034] Figure 7 This is a schematic diagram of display content of a display page of a client provided by an embodiment of the present invention;

[0035] Figure 8 This is another schematic diagram of display content of a display page of a client provided by an embodiment of the present invention;

[0036] Figure 9 is a block diagram of a classification model training device provided by an embodiment of the present invention;

[0037] Figure 10 This is a block diagram of a text data mining device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0039] Referring to Figure 1 A classification model training method provided by an embodiment of the present application can include:

[0040] S101, obtaining a plurality of dialogue texts as training data.

[0041] S102, performing training on a classification model according to the training data to obtain a classification model trained at a stage.

[0042] In some voice dialogue scenarios or text dialogue scenarios in an embodiment of the present application, dialogue records in a dialogue process can be stored under the authorization of a customer. Thus, a plurality of dialogue texts can be determined based on a plurality of collected dialogue records. The customer can be regarded as a first user and a business staff can be regarded as a second user. If the dialogue records are stored in the form of voice data, the dialogue records can be converted into voice to obtain corresponding dialogue texts. After obtaining a plurality of dialogue texts, the dialogue texts are used as training data to perform training on a preset classification model until the classification model meets a first training condition, the training on the classification model is stopped, and the classification model after the training is stopped is determined as a classification model trained at a stage. In an example, the first training condition can be that a loss function of the classification model no longer decreases or decreases by a small amount.

[0043] Performing training on a classification model according to the training data to obtain a classification model trained at a stage includes the following steps:

[0044] Performing clustering analysis on the training data to determine a plurality of training clusters.

[0045] Determining a scene category corresponding to the training cluster, and labeling the training cluster with a text in the scene category.

[0046] Training the classification model using the labeled text data to obtain a classification model trained at a stage.

[0047] In the embodiment of the present application, the training data can be subjected to cluster analysis to determine a plurality of training clusters, and a corresponding scene category can be determined from the plurality of training clusters. The scene category can be determined according to the text semantic information in each training cluster. After determining the scene category corresponding to the training cluster, the text in each training cluster can be labeled with the corresponding scene category. By labeling the scene category of each training cluster, the labeled text data is formed. Thus, the labeling of each dialogue text is avoided, and the labeling amount of the training data is greatly reduced. Then, the labeled text data is used to train the preset classification model to obtain the classification model trained in the stage.

[0048] In S103, difference information is determined based on the training data, and whether the classification model trained in the stage meets the scene mining condition is determined according to the difference information.

[0049] In S104, if the difference information meets the scene mining condition, the training data is updated and the training is continued according to the training data and the classification model trained in the stage.

[0050] In S105, if the difference information does not meet the scene mining condition, the classification model trained in the stage is taken as the trained classification model.

[0051] In the embodiment of the present application, after determining the scene category corresponding to the training cluster, the difference information of the text data in the same training cluster can be determined. The difference information can be understood as the semantic similarity of the semantic information of different texts. Thus, whether the classification model trained in the stage meets the scene mining condition can be determined according to the difference information.

[0052] In an example, the scene mining condition is used to evaluate whether the scene category of the classification model trained in the stage needs to be refined. For example, when the semantic similarity of the semantic information of different texts in the same training cluster is high, it indicates that the boundary between the corresponding two texts is not clear, and the classification effect is not obvious. Thus, a semantic threshold can be preset. When the semantic similarity is less than the semantic threshold, it is determined that the difference information does not meet the scene mining condition, that is, the classification effect is obvious, and thus the classification model trained in the stage is directly taken as the trained classification model. When the semantic similarity is equal to or greater than the semantic threshold, it is determined that the difference information meets the scene mining condition, and thus the training data needs to be updated, and the updated training data is used to continue training the classification model trained in the stage until the classification model trained in the stage meets the first training condition, the training of the classification model is stopped, and the classification model trained in the stage after the training is stopped is determined as the trained classification model.

[0053] In summary, the classification model training method provided by the embodiment of the present application uses the obtained multiple dialogue texts as training data, performs training on the classification model according to the training data, and obtains a classification model that has completed stage training. The training data is filtered by using a clustering analysis method, and the training data is labeled according to the scene categories, so that the labeling amount of the training data can be greatly reduced. In addition, whether the classification model that has completed stage training needs to be continuously trained is determined according to the difference information determined from the training data. Therefore, in the case that the difference information meets the scene mining condition, more scene categories can be mined by continuous training, so that more refined scene categories and related texts with small information amount can be mined in the text mining process, thereby facilitating statistical analysis of related dialogue texts.

[0054] With reference to Figure 2 Another classification model training method provided by the embodiment of the present application is shown, which can include:

[0055] S201, obtaining multiple dialogue texts as training data.

[0056] S202, performing training on a classification model according to the training data, and obtaining a classification model that has completed stage training.

[0057] In the embodiment of the present application, after obtaining several dialogue texts, the dialogue texts are used as training data to perform training on a preset classification model, until the classification model meets a first training condition, the training of the classification model is stopped, and the classification model after stopping training is determined as a classification model that has completed stage training.

[0058] The training data is subjected to clustering analysis to determine multiple training clusters, and corresponding scene categories are determined from the multiple training clusters. The first clustering model can be preset to perform first clustering analysis on the training data. For example, the first clustering model can use a K-means (k-means clustering algorithm, K-means clustering) algorithm. According to the number of clusters, two dialogue texts with close distances can be determined as a cluster. Therefore, when performing statistical analysis of scene categories on multiple dialogue texts, the number of clusters m1 output by the first clustering model for the first time can be determined in advance, wherein the number of clusters m1 can be determined according to the actual number of scene categories n1, which is not limited herein. However, the number of clusters m1 needs to be greater than the number of scene categories n1. The number of clusters m can be the number of scene categories n plus a preset threshold. The range of the preset threshold can be set to 3-10. For example, when it is clear that the initial business scenes include a sales scene and an after-sales scene, the number of corresponding scene categories n1 is determined to be 2. The number of clusters m1 can be 3, 4, 5, or 12, etc., so that the clustering result can be prevented from being too divergent.

[0059] After the first clustering analysis by the first clustering model, m1 training clusters are determined. In an example, after the plurality of training clusters are determined, a display page can be provided to display the plurality of training clusters, so that based on triggering of a filtering control, a scenario category-specific cluster can be filtered from the plurality of training clusters, and the training cluster can be labeled according to a preset business scenario. For example, 2 business scenario-specific training clusters are filtered from 3 training clusters and are labeled respectively. When labeling, an editing control can be provided in the display page for a user to customize the corresponding scenario category. Based on triggering of the editing control, the labeling of the training cluster is completed. For example, after the labeling is completed, text data corresponding to a sales scenario and text data corresponding to an after-sales scenario are obtained.

[0060] By labeling the scenario category of each training cluster, the text of the training cluster labeled is formed into labeled text data. Thus, the step of labeling each dialogue text during model training is avoided. The labeling amount of training data is greatly reduced. Then, the labeled text data is trained on the preset classification model to obtain a classification model M1 trained in this stage. The classification model can be a BERT-base model including N hidden layers. When training the classification model for the first time, the text features output by the Nth hidden layer can be taken for classification, and the number of scenario categories corresponding to the classification is n1. At this time, the obtained classification model is the classification model M1 trained in this stage.

[0061] In an optional embodiment, during the process of inputting the training data into the first clustering model for first clustering analysis and outputting a plurality of training clusters, each dialogue text in the training data can be divided into text segments to determine a plurality of corresponding text segments. The clustering analysis is performed according to the semantic information of the text segments of each dialogue text. In some dialogue texts, the semantic information corresponding to some text segments is classified, which can improve the effect of clustering analysis. For example, the intention of the user asking “how long does it take to deliver to Beijing” and “how long does it take to deliver to Shanghai” is consistent. Shanghai and Beijing can be classified as addresses. Thus, before semantic analysis, named entity recognition is performed on the plurality of text segments to determine target named entities that need to be classified. The target named entities at least include one of the following: a person name, an organization name, and a place name. After the target named entities are determined, the target named entities can be replaced by corresponding target keywords. The target keywords can be words related to the target named entities.

[0062] After completing the replacement of all target named entities, the replaced text segments are converted into text word vectors or text TFIDF (term frequency-inverse document frequency) values, respectively. The text word vectors or text TFIDF values are input into the first clustering model to perform the clustering analysis operation described above.

[0063] S203, determining difference information based on the training data.

[0064] S204, providing a display page to display the difference information, and obtaining mining operation information based on the display page.

[0065] S205, determining whether the classification model trained in the stage meets the scene mining condition according to the mining operation information.

[0066] In the embodiment of the application, after determining the scene category corresponding to the training cluster, the difference information of the text data in the same training cluster can be determined. The difference information can be understood as the semantic similarity of the semantic information of different texts. Therefore, according to the difference information, it can be determined whether the classification model trained in the stage meets the scene mining condition.

[0067] In an example, a display page can be provided, in which the difference information is displayed, and a first selection control is arranged in the display page. Based on the triggering of the first selection control, the corresponding mining operation information in the display page is obtained. For example, the mining operation information can include yes or no. When the mining operation information is yes, it is determined that the classification model trained in the stage meets the scene mining condition, and step S206 is performed. When the mining operation information is no, it is determined that the classification model trained in the stage does not meet the scene mining condition, and step S208 is performed.

[0068] S206, inputting the labeled text data into the classification model trained in the stage to perform feature extraction, and obtaining corresponding feature text.

[0069] S207, using the feature text as training data.

[0070] In the embodiment of the present application, the labeled text data is input into the classification model trained in the stage to perform feature extraction. The feature extraction can be understood as directly outputting the corresponding feature text through the hidden layer in the classification model trained in the stage. In an example, when performing feature extraction for the first time, the output feature of the first hidden layer is determined as the corresponding feature text, and the feature text is taken as new training data. It is considered that the closer to the output layer, the more accurate the text feature output by the hidden layer. When performing feature extraction for the first time, the text feature output by the N-1th hidden layer can be determined as the feature text, and the training data is updated according to the feature text. Then, step S202 is performed to continue training the classification model trained in the stage by using the updated training data.

[0071] After taking the feature text as new training data, the classification model is trained according to the training data. Since the scene categories need to be further mined, when the training data is analyzed by the first clustering model for the first time, the number of clusters output by the first clustering model is adjusted to m2, where the number of clusters m2 is greater than the number of scene categories n1. After the first clustering analysis by the first clustering model, m2 training clusters are determined. The m2 training clusters are displayed on the display page, so that the scene category-specific cluster can be selected from the m2 training clusters based on the triggering of the filtering control, and the scene category of the training cluster can be marked based on the training cluster. At this time, the number of scene categories n2 is obtained, and the number of scene categories n2 is greater than the number of scene categories n1. For example, based on the sales scene and the after-sales scene, the scene categories are further divided into product consultation scene, agreement to purchase scene, agreement to after-sales scene, and refusal to exchange scene.

[0072] By marking the scene category of each training cluster, the text of the training cluster marked with the category forms the text data with the label. Then, the labeled text data is used to continue training the classification model M1 trained in the stage to obtain the classification model M2 trained in the stage, which can classify n2 scene categories, so that the classification model M2 trained in the stage increases the number of scene categories compared with the classification model M1 trained in the stage.

[0073] Then, the difference information is determined based on the training data of the classification model M2 trained in the training stage, and whether the classification model M2 trained in the stage meets the scene mining condition is judged according to the difference information, and when it is determined that the classification model M2 trained in the stage meets the scene mining condition, step S206 is executed; when it is determined that the classification model M2 trained in the stage does not meet the scene mining condition, step S208 is executed. For example, when it is determined that the classification model M2 trained in the stage meets the scene mining condition, the scene category needs to be further refined. For example, the product consultation scene is further divided into: price inquiry scene, size inquiry scene, style inquiry scene, and the like. In this way, the trained classification model is finally obtained. The trained classification model is applied to text mining, which can accurately classify the dialogue text of the customer, and can determine the target reply text with high matching degree based on the scene category obtained by accurate classification, so as to improve the service level of the business personnel and improve the user experience.

[0074] S208, the classification model trained in the stage is taken as the trained classification model.

[0075] In the embodiment of the application, it is determined that the difference information does not meet the scene mining condition, that is, the classification effect is obvious, and therefore the classification model trained in the stage is directly taken as the trained classification model.

[0076] Referring to Figure 3 , a step flowchart of a text mining method provided by an embodiment of the application is shown, and the method can include:

[0077] S301, dialogue information is received, and dialogue text of a first user is obtained from the dialogue information.

[0078] S302, the dialogue text is input into a classification model for classification and recognition to determine a corresponding target scene category, the classification model is trained by training data to obtain a classification model trained in a stage, difference information is determined based on the training data, whether the classification model trained in the stage meets a scene mining condition is judged, and whether the training data is updated to continue training the classification model trained in the stage is determined according to the judgment result.

[0079] S303, a target reply text corresponding to the target scene category is queried.

[0080] S304, the target reply text is taken as dialogue text of a second user, and the dialogue text of the first user is fed back.

[0081] In the embodiment of the present application, the classification model is obtained by the above classification model training method. After obtaining the dialogue information, the dialogue text of the first user is obtained from the dialogue information. The dialogue text of the first user is input into the classification model for classification and recognition to obtain the target scene category corresponding to the dialogue text of the first user. Then, the historical dialogue text of the second user is obtained from the dialogue record, wherein the scene category corresponding to the historical dialogue text of the second user has been labeled, and the number and type of the scene category corresponding to the historical dialogue text of the second user are consistent with the number and type of the scene category classified and recognized by the classification model.

[0082] Therefore, after determining the target scene category corresponding to the dialogue text of the first user, the historical dialogue text of the second user under the target scene category is queried from the database, and the target reply text is determined and used as the dialogue text of the second user for feedback of the dialogue text of the first user. The target reply text can be multiple, which can be displayed in the form of at least one cluster. In one example, when the target reply text is multiple, one of the target reply texts can be determined as the dialogue text of the second user by clicking the selection control on the display page, and fed back to the first user. Therefore, even if the business personnel with little experience, the most suitable dialogue text for responding to the dialogue text of the first user can be mined from the historical dialogue text of the second user. Thus, the business personnel can be facilitated to learn and improve the service level, and the user experience of the first user is also improved.

[0083] In another example, if the number of target reply texts under the same target scene category is large, in order to facilitate the mining of different reply methods, a preset second clustering model can be used for second clustering analysis of the target reply texts. The number of clusters output by the second clustering model dynamically changes with the semantic information of the target reply text. The DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm can be used. Thus, in the second clustering analysis process, the specific number of clusters does not need to be set, and the target reply texts of several different reply methods are obtained. Thus, one of the target reply texts can be determined as the dialogue text of the second user by clicking the selection control on the display page, and fed back to the first user.

[0084] In one example, referring to Figure 4 and Figure 5In the first client of the first user and the second client of the second user, a display page is provided to display the conversation message. In the display page corresponding to the second user, a classification control (such as a circle in the middle of the page) corresponding to each conversation text of the first user is arranged, and a classification instruction is generated and sent to the server according to the triggering of the classification control. The conversation text corresponding to the classification control is input into the classification model for classification recognition to determine the target scene category and determine the target reply text. After the target reply text is determined, the target reply text is sent to the second client. The corresponding target reply text is displayed in the display page corresponding to the second user. Referring to Figure 6 Each target reply text is matched with a reply selection control (such as an arrow in the middle of the page), and the target reply text corresponding to the reply selection control is sent to the first client as the conversation text of the second user based on the triggering of the reply selection control.

[0085] In another example, the first client and the second client can be the same client. Referring to Figure 7 The user can input the conversation text in the display page of the client, and the conversation text is the conversation text of the first user by default. The conversation text of the first user is input into the classification model for classification recognition based on the triggering of the classification control in the display page to obtain the target scene category corresponding to the conversation text of the first user. Then, the historical conversation text of the second user is obtained from the conversation record, wherein the scene category corresponding to the historical conversation text of the second user has been annotated, and the number and types of the scene categories corresponding to the historical conversation text of the second user are consistent with the number and types of the scene categories classified and recognized by the classification model.

[0086] Therefore, after the target scene category corresponding to the conversation text of the first user is determined, the historical conversation text of the second user under the target scene category is queried from the database and determined as the target reply text, and the target reply text is used as the conversation text of the second user for feedback of the conversation text of the first user. The conversation text of the second user is displayed in the client of the user. Referring to Figure 8 Therefore, in the business training and learning scene of the new employee, the new employee can customize the conversation text of the first user, so as to learn different dialogues based on the target reply text obtained from different conversation texts of the first user. These target reply texts can provide materials for the training of new employees or the mutual learning of old employees.

[0087] It should be noted that, for the method embodiments, the series of acts complement each other to achieve the purpose of this application, therefore, the sequence of the acts should not be construed as a limitation on the implementation of the embodiments of this application. In addition, those skilled in the art should understand that the acts described in the specification are only preferred acts, and thus, not all of the acts are necessary for the implementation of the embodiments of this application.

[0088] With reference to Figure 9 , a classification model training device provided by an embodiment of the application is shown, which can include:

[0089] The data acquisition module 901 is configured to acquire a plurality of dialogue texts as training data.

[0090] The model stage training module 902 is configured to perform training on a classification model according to the training data, to obtain a classification model after stage training.

[0091] The condition determination module 903 is configured to determine difference information based on the training data, and determine whether the classification model after stage training meets a scenario mining condition according to the difference information.

[0092] The first training module 904 is configured to update the training data and continue training according to the training data and the classification model after stage training, if the difference information meets the scenario mining condition.

[0093] The second training module 905 is configured to use the classification model after stage training as a classification model after training, if the difference information does not meet the scenario mining condition.

[0094] The model stage training module is further configured to:

[0095] perform cluster analysis on the training data, to determine a plurality of training clusters.

[0096] determine a scenario category corresponding to the training cluster, and label the text of the training cluster as the scenario category.

[0097] train the classification model using the labeled text data, to obtain the classification model after stage training.

[0098] An optional embodiment of the application is based on the determination of the difference information from the training data, which includes:

[0099] perform analysis on the text of the training cluster corresponding to different scenario categories, to determine the difference information.

[0100] An optional embodiment of the application is that the first training module can include:

[0101] a feature extraction submodule configured to input the labeled text data into the classification model trained in the stage to perform feature extraction and obtain corresponding feature text.

[0102] a data updating submodule configured to use the feature text as training data.

[0103] In an optional embodiment, the model stage training module is further configured to:

[0104] input the training data into a first clustering model to perform first clustering analysis and output a plurality of training clusters, wherein the number of the training clusters is preset by the first clustering model.

[0105] In an optional embodiment, the model stage training module can include:

[0106] a word segmentation submodule configured to divide the training data into a plurality of text segments.

[0107] a text segmentation conversion submodule configured to convert the plurality of text segments into text word vectors or text TFIDF values.

[0108] a cluster output submodule configured to input the text word vectors or text TFIDF values into the first clustering model to perform first clustering analysis and output a plurality of training clusters.

[0109] In an optional embodiment, the cluster output submodule can further include:

[0110] an identification unit configured to perform named entity recognition on the plurality of text segments to determine target named entities in the plurality of text segments, wherein the target named entities include at least one of the following: a person name, an organization name, and a place name.

[0111] a replacement unit configured to replace the target named entities with target keywords and convert the plurality of text segments after replacement into text TFIDF values.

[0112] In an optional embodiment, the method can further include:

[0113] an information display module configured to display the difference information on a display page and obtain mining operation information based on the display page.

[0114] a condition determination module configured to determine, according to the mining operation information, whether the classification model trained in the stage meets a scenario mining condition.

[0115] Reference Figure 10As shown in FIG. 1, a text mining device is provided, which can include:

[0116] A dialogue receiving module 1001 is configured to receive dialogue information and obtain dialogue text of a first user from the dialogue information.

[0117] A scene identifying module 1002 is configured to input the dialogue text into a classification model for classification and recognition to determine a corresponding target scene category, wherein the classification model is trained by training data to obtain a stage-trained classification model, and based on the training data, difference information is determined to judge whether the stage-trained classification model meets scene mining conditions, and according to the judgment result, it is determined whether to update the training data to continue training the stage-trained classification model.

[0118] A text querying module 1003 is configured to query target reply text corresponding to the target scene category.

[0119] A text feedback module 1004 is configured to use the target reply text as dialogue text of a second user to feed back the dialogue text of the first user.

[0120] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0121] It is easy for those skilled in the art to think that any combination of the above embodiments is feasible, so any combination of the above embodiments is an embodiment of the present application, but due to the limitation of the length, the above is not described in detail.

[0122] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known methods, structures and techniques are not described in detail in order not to obscure the understanding of the specification.

[0123] Similarly, it is to be understood that the embodiments of the present application can be alternately grouped together in a single embodiment, figure, or description of embodiments thereof, for the purpose of brevity and understanding. However, the disclosure is not to be interpreted that the claimed application requires more features than those explicitly recited in each claim. Rather, the inventive aspects lie in fewer than all features of a previously disclosed single embodiment. Accordingly, the claims appended hereto follow, wherein each claim is hereby expressly incorporated into this Detailed Description, in which each claim is a separate embodiment of the application.

[0124] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination of all the features disclosed in the present specification (including the accompanying claims, abstract and drawings), and all processes or units of any methods or apparatuses disclosed so far can be adopted. Unless explicitly stated, each feature disclosed in the present specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature providing the same, equivalent or similar function.

[0125] An electronic device, comprising:

[0126] one or more processors;

[0127] a memory;

[0128] one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the methods described in the above embodiments.

[0129] A computer readable storage medium storing a computer program for use in conjunction with an electronic device, the computer program executable by a processor to perform the methods described in the above embodiments.

[0130] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, embodiments of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer program instructions.

[0131] Embodiments of the present application are described herein with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0132] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing terminal apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal apparatus to cause a series of operational steps to be performed on the computer or other programmable terminal apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0134] While preferred embodiments of the present application have been described, modifications and alterations thereto will occur to those skilled in the art upon reading the preceding description. In particular, it will be apparent to those skilled in the art that parts can be added to, or substituted for, parts of the described embodiments of the present application. Accordingly, the application is intended to be

[0135] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose additional identical elements of the process, method, article, or apparatus including the specific element.

[0136] The classification model training method, the text mining method, the classification model training device and the text mining device provided by the present application are described in detail above, the principle and implementation mode of the present application are described by applying specific examples in the present article, the above example is only used to help understand the method of the present application and its core idea; meanwhile, for the general technical personnel in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description should not be understood as the limitation of the present application.

Claims

1. A classification model training method, characterized in that: The method comprises: Obtain multiple conversation texts as training data; Training the classification model based on the training data to obtain a classification model that has completed stage training; Determining difference information based on the training data, and judging whether the classification model trained in the stage meets the scene mining conditions according to the difference information; wherein the difference information is the semantic similarity of semantic information of different texts; If the difference information meets the scenario mining conditions, the training data is updated based on the training data and the classification model completed in the stage training and the training is continued; If the difference information does not meet the scenario mining conditions, the classification model trained in the stage is used as the trained classification model; The classification model is trained based on the training data to obtain a classification model that has completed stage training, including: Performing cluster analysis on the training data to determine a plurality of training clusters; Determine the scene category corresponding to the training cluster, and mark the text in the training cluster as the scene category; The labeled text data is used to train the classification model to obtain a classification model that has completed stage training.

2. The classification model training method according to claim 1, characterized in that Determining difference information based on the training data includes: Analyze the text of the training clusters corresponding to different scene categories to determine the difference information.

3. The classification model training method according to claim 1, characterized in that The updating of the training data based on the training data and the classification model completed in the stage training includes: Input the labeled text data into the classification model trained in the stage, perform feature extraction, and obtain the corresponding feature text; The feature text is used as training data.

4. The classification model training method according to claim 1, characterized in that The performing cluster analysis on the training data to determine a plurality of training clusters includes: The training data is input into a first clustering model to perform a first clustering analysis, and a plurality of training clusters are output, wherein the number of the training clusters is pre-set by the first clustering model.

5. The classification model training method according to claim 4, characterized in that: The step of inputting the training data into a first clustering model to perform a first clustering analysis and outputting a plurality of training clusters comprises: Performing word segmentation on the training data to determine a number of corresponding text segmentations; Convert several text segmentations into text word vectors or text TFIDF values; Several text word vectors or text TFIDF values ​​are input into the first clustering model to perform a first clustering analysis, and several training clusters are output.

6. The classification model training method according to claim 5, characterized in that: The step of converting several text segmentations into text TFIDF values ​​includes: Performing named entity recognition on a plurality of text segmentations to determine target named entities in the plurality of text segmentations, wherein the target named entities include at least one of the following: a person's name, an organization's name, and a place's name; The target named entity is replaced with the target keyword, and the replaced text segmentations are converted into text TFIDF values.

7. The classification model training method according to claim 2, characterized in that: The method further comprises: Providing a display page to display the difference information, and obtaining mining operation information based on the display page; Based on the mining operation information, it is determined whether the classification model trained in the stage meets the scene mining conditions.

8. A text mining method, characterized in that: The method comprises: receiving conversation information, and obtaining a conversation text of the first user from the conversation information; Inputting the conversation text into a classification model for classification and recognition to determine the corresponding target scene category, the classification model is trained in the following manner: training the classification model based on training data to obtain a classification model trained in a stage, determining difference information based on the training data, judging whether the classification model trained in the stage meets the scene mining conditions based on the difference information, if the difference information meets the scene mining conditions, updating the training data based on the training data and the classification model trained in the stage and continuing training; if the difference information does not meet the scene mining conditions, using the classification model trained in the stage as the trained classification model; wherein the difference information is the semantic similarity of semantic information of different texts; Query the target response text corresponding to the target scenario category; The target reply text is used as the conversation text of the second user, and the conversation text of the first user is fed back.

9. An electronic device comprising: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program for use in conjunction with an electronic device, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 8.

Images