Work order category recognition method, model training method, device and equipment

Abstract

The work order category recognition method, model training method, device and equipment provided by the present disclosure relate to computer text category recognition technology, and include: obtaining the work order text data to be recognized, inputting the work order text data to be recognized into a preset work order matching model based on ERNIE2, and obtaining and displaying similar historical work order text data; obtaining the modified work order text data; inputting the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category. This solution can obtain similar historical work order text data according to the preset work order matching model based on ERNIE2; customer service personnel can modify the work order text data to be recognized according to the similar historical work order text data; the work order category can be obtained according to the preset work order classification model based on ERNIE2. This solution can automatically perform category recognition on the work order text data to be recognized, improving the category recognition efficiency and accuracy.

Description

Technical Field

[0001] The present disclosure relates to computer text category recognition technology, and in particular, to a work order category recognition method, a model training method, a device, and a device. Background Art

[0002] With the rapid development of the telecommunications industry, the services provided by communication operators to users are becoming increasingly rich, the usage volume of users is also increasing, and the business scenarios are becoming more and more complex. When users encounter problems such as Internet access, calls, and business handling when using services, they will go to the business hall or call the customer service hotline to make complaints, and then the customer service staff will assign the problems to the corresponding technical personnel in the background for processing.

[0003] In the prior art, after manually reading and identifying the text content of the work order, the work order is manually classified and then transferred to the corresponding technical personnel for processing.

[0004] However, this method cannot meet the requirements of the increasingly growing data volume, and both the efficiency and the accuracy rate are relatively low, thereby resulting in a poor user experience for users. Summary of the Invention

[0005] The present disclosure provides a work order category recognition method, a model training method, a device, and a device, so as to solve the problem in the prior art that after manually reading and identifying the text content of the work order, the work order is manually classified and then transferred to the corresponding technical personnel for processing, which cannot meet the requirements of the increasingly growing data volume, and both the efficiency and the accuracy rate are relatively low, thereby resulting in a poor user experience for users.

[0006] According to a first aspect of the present disclosure, there is provided a work order category recognition method, including:

[0007] Obtain the work order text data to be recognized, input the work order text data to be recognized into a preset work order matching model based on ERNIE2, and obtain and display the historical work order text data similar to the work order text data to be recognized;

[0008] Obtain the modified work order text data, where the modified work order text data is obtained by modifying the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized;

[0009] Input the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the work order text data to be recognized.

[0010] According to a second aspect of the present disclosure, there is provided a model training method applied to work order category recognition, including:

[0011] Obtain a set to be trained, where the set to be trained includes multiple historical work order text data, and each historical work order text data in the set to be trained has an initial work order category;

[0012] Repeat the following steps until a preset condition is reached. After reaching the preset condition, a work order classification model based on ERNIE2 is obtained: Input the historical work order text data into the initial model to obtain the predicted work order category of the historical work order text data; Adjust the parameters of the initial model according to the initial work order category and the predicted work order category of the historical work order text data;

[0013] Wherein, the work order classification model based on ERNIE2 is used to identify the modified work order text data to obtain the work order category; The modified work order text data is obtained by modifying the work order text data to be identified based on the historical work order text data similar to the work order text data to be identified; The historical work order text data similar to the work order text data to be identified is obtained by inputting the work order text data to be identified into a preset work order matching model based on ERNIE2.

[0014] According to the third aspect of the present disclosure, there is provided a work order category recognition device, including:

[0015] A work order matching unit, configured to obtain work order text data to be identified, input the work order text data to be identified into a preset work order matching model based on ERNIE2, and obtain and display historical work order text data similar to the work order text data to be identified;

[0016] An obtaining unit, configured to obtain modified work order text data, where the modified work order text data is obtained by modifying the work order text data to be identified based on the historical work order text data similar to the work order text data to be identified;

[0017] A category determination unit, configured to input the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the work order text data to be identified.

[0018] According to the fourth aspect of the present disclosure, there is provided a model training device applied to work order category recognition, including:

[0019] An obtaining unit, configured to obtain a set to be trained, where the set to be trained includes multiple historical work order text data, and each historical work order text data in the set to be trained has an initial work order category;

[0020] A model training module is used to repeat the following steps until a preset condition is reached. After reaching the preset condition, a work order classification model based on ERNIE2 is obtained: Input the historical work order text data into the initial model to obtain the predicted work order category of the historical work order text data; Adjust the parameters of the initial model according to the initial work order category and the predicted work order category of the historical work order text data.

[0021] Among them, the work order classification model based on ERNIE2 is used to identify the modified work order text data to obtain the work order category; The modified work order text data is obtained by modifying the to-be-identified work order text data based on the historical work order text data similar to the to-be-identified work order text data; The historical work order text data similar to the to-be-identified work order text data is obtained by inputting the to-be-identified work order text data into a preset work order matching model based on ERNIE2.

[0022] According to the fifth aspect of the present disclosure, an electronic device is provided, including a memory and a processor; Among them,

[0023] The memory is used to store a computer program;

[0024] The processor is used to read the computer program stored in the memory and execute the work order category recognition method described in the first aspect or the model training method applied to work order category recognition described in the second aspect according to the computer program in the memory.

[0025] According to the sixth aspect of the present disclosure, a computer-readable storage medium is provided. Computer-executable instructions are stored in the computer-readable storage medium. When the processor executes the computer-executable instructions, the work order category recognition method described in the first aspect or the model training method applied to work order category recognition described in the second aspect is implemented.

[0026] According to the seventh aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the work order category recognition method described in the first aspect or the model training method applied to work order category recognition described in the second aspect is implemented.

[0027] The work order category recognition method, model training method, device and equipment provided by the present disclosure include: obtaining the work order text data to be recognized, inputting the work order text data to be recognized into a preset work order matching model based on ERNIE2, and obtaining and displaying the historical work order text data similar to the work order text data to be recognized; obtaining the modified work order text data, where the modified work order text data is obtained by modifying the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized; inputting the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the work order text data to be recognized. The work order category recognition method provided by the present disclosure inputs the work order text data to be recognized into a preset work order matching model based on ERNIE2, and can match similar historical work order text data; the customer service staff can optimize the work order text data to be recognized according to the similar historical work order text data to obtain the modified work order text data, where the modified work order text data is standardized work order text data; inputting the modified work order text into a preset work order classification model based on ERNIE2 can obtain the work order category corresponding to the work order text data to be recognized. The method provided by the present disclosure can automatically perform category recognition on the work order text data to be recognized, improve the category recognition efficiency and accuracy, and enhance the user experience. Description of the Drawings

[0028] Figure 1 It is a schematic flow chart of the work order category recognition method shown in an exemplary embodiment of the present application;

[0029] Figure 2 It is a schematic flow chart of the work order category recognition method shown in another exemplary embodiment of the present application;

[0030] Figure 3 It is a schematic flow chart of the model training method applied to work order category recognition shown in an exemplary embodiment of the present application;

[0031] Figure 4 It is a schematic flow chart of the model training method applied to work order category recognition shown in another exemplary embodiment of the present application;

[0032] Figure 5 It is a structural diagram of the work order category recognition device shown in an exemplary embodiment of the present application;

[0033] Figure 6 It is a structural diagram of the work order category recognition device shown in another exemplary embodiment of the present application;

[0034] Figure 7 It is a structural diagram of the model training device applied to work order category recognition shown in an exemplary embodiment of the present application;

[0035] Figure 8 The structural diagram of a model training device for work order category recognition shown in another exemplary embodiment of the present application;

[0036] Figure 9 The structural diagram of an electronic device shown in an exemplary embodiment of the present application. Detailed implementation manners

[0037] With the rapid development of the telecommunications industry, the services provided by communication operators to users are becoming increasingly rich, the usage volume of users is also increasing, and the business scenarios are becoming more and more complex. When users encounter problems such as Internet access, calls, and business handling while using services, they will go to the business hall or call the customer service hotline to make complaints, and then the customer service staff will assign the problems to the corresponding technical personnel in the background for processing. In the prior art, after manually reading and identifying the text content of the work order, the work order is manually classified and then transferred to the corresponding technical personnel for processing.

[0038] However, this method cannot meet the requirements of the increasing data volume, and both the efficiency and accuracy are relatively low, which in turn leads to a poor user experience for users.

[0039] To solve the above technical problems, the solution provided by the present disclosure includes a work order category recognition method. First, according to a preset work order matching model based on ERNIE2, historical work order text data similar to the work order text data to be recognized can be matched; the customer service staff can optimize the work order text data to be recognized according to the similar historical work order text data to obtain the modified work order text data, where the modified work order text data is normalized work order text data; then, according to a preset work order classification model based on ERNIE2, the work order category of the work order text data to be recognized can be identified. The method provided by the present disclosure can automatically perform category recognition on the work order text data to be recognized, and improve the category recognition efficiency and accuracy, and enhance the user experience.

[0040] Figure 1 The flow schematic diagram of a work order category recognition method shown in an exemplary embodiment of the present application.

[0041] As Figure 1 shown, the work order category recognition method provided in this embodiment includes:

[0042] Step 101, obtain the work order text data to be recognized, input the work order text data to be recognized into a preset work order matching model based on ERNIE2, and obtain and display historical work order text data similar to the work order text data to be recognized.

[0043] Among them, the method provided by this application can be executed by an electronic device with computing capabilities, such as a computer or other devices. The electronic device can obtain the work order text data to be recognized, and according to the preset work order matching model based on ERNIE2, obtain and display the historical work order text data similar to the work order text data to be recognized; the electronic device can also obtain the modified work order text data, and according to the preset work order classification model based on ERNIE2, obtain the work order category corresponding to the work order text data to be recognized.

[0044] Among them, the work order text data can be, for example, the text data edited by customer service staff according to the user's complaint problems to describe the user's complaint problems. Specifically, in the operator business scenario, the work order text data can include, for example: (1) The package of Li Si's number cannot be changed. Please check the reason. Thank you! (2) The package of Zhang San's number has been changed. Please check the reason. Thank you! (3) The user has a question about the monthly rent deduction for the package used. Please assist in checking whether the deduction is abnormal.

[0045] Among them, the work order text data to be recognized can be the work order text data of the work order category to be recognized.

[0046] Among them, the work order category can be a category set according to the actual business situation. Specifically, in the operator business scenario, the work order category can include: network problem group, deduction problem group, package problem group, etc. For example, if the work order text data (for example, the work order text data is: The package of Li Si's number cannot be changed. Please check the reason. Thank you!) characterizes the user's complaint problem as a package problem, then the work order category of this work order text data can be classified into the package problem group, and then it can be processed by the technical personnel corresponding to the package problem.

[0047] Among them, the historical work order text data can be, for example, the work order text data that has occurred in the past and is stored in the historical database.

[0048] Among them, ERNIE2 is the second generation of ERNIE. ERNIE2 is a pre-trained language model that unifies the modeling of lexical structure, syntactic structure, and semantic information in the training data, greatly enhancing the general semantic representation ability and achieving good results in multiple Chinese tasks.

[0049] Among them, the preset work order matching model based on ERNIE2 can be a pre-trained ERNIE2 model using the Transformer structure. Among them, Transformer is a kind of neural network structure. Inputting the work order text data to be recognized into this model can obtain the historical work order text data similar to the work order text data to be recognized.

[0050] Specifically, if the complaint problem described in the work order text data to be recognized is similar to or the same as the complaint problem described in the historical work order text data, it can be considered that the historical work order text data is similar to the work order text data to be recognized.

[0051] Step 102, obtain the modified work order text data, where the modified work order text data is obtained by modifying the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized.

[0052] Specifically, the customer service staff can optimize the work order text data to be recognized according to the displayed historical work order text data similar to the work order text data to be recognized to obtain the modified work order text data. This modified work order text data can be a standardized work order text data, and the standardized work order text data describes the user's complaint problem more normatively, which is more conducive to the subsequent work order category recognition.

[0053] Specifically, the electronic device can obtain the modified work order text data.

[0054] Step 103, input the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the work order text data to be recognized.

[0055] Among them, the preset work order classification model based on ERNIE2 can be a preset work order matching model based on ERNIE2, which can be a pre-trained ERNIE2 model using the Transformer structure. Inputting the modified work order text data into this model can obtain the work order category of the work order text data to be recognized.

[0056] Optionally, the preset work order matching model based on ERNIE2 and the preset work order classification model based on ERNIE2 can be based on the same pre-trained pre-training model based on ERNIE2.

[0057] Furthermore, after the electronic device recognizes the work order category of the work order text data to be recognized, it can automatically assign this work order to the technical personnel corresponding to the corresponding work order category for processing.

[0058] For example, Example 1: "User 186xxxxxx111, unable to use". Example 2: "User 186xxxxxx222 wants to use the XX package, please assist in processing, thank you!". Example 3: "User 186xxxxxx333 doesn't want to use the XX package, please assist in processing, thank you!".

[0059] First, Examples 1-3 go through a pre-set work order matching model based on ERNIE2 to obtain recommended historical work order text data that are similar to each of them. Among them, the requirements of the users are described in Examples 2-3, and there are also many highly similar ones in the historical work order text data regarding the users' requirements of not wanting or wanting to switch to a certain package. Therefore, the descriptions of Examples 2-3 by the work order creators are relatively standard, and the role of the pre-set work order matching model based on ERNIE2 cannot be reflected in Examples 2-3. However, the description in Example 1 is very vague and does not describe what the user cannot use. If such a work order is successfully created, the back-end specialized technical personnel will definitely not be able to understand which aspect of the user number cannot be used. After Example 1 goes through the pre-set work order matching model based on ERNIE2, the recommended historical work order text data that are similar can be: (1) "The XXX number cannot use the Internet access function. Please assist in verification. Thank you!" (2) "The XXX number cannot make normal calls. Please assist in verification. Thank you!" (3) "The XXX number is in normal use but cannot send text messages. Please assist in verification. Thank you!". Therefore, the customer service staff can determine the final problem of the user in Example 1 as being unable to make calls based on the descriptions of the similar historical work order text data. Therefore, finally, Example 1 can be modified to: "User 186xxxxxx111 cannot make normal calls. Please assist in verification. Thank you!"

[0060] The customer service staff can modify the work order text data to be recognized according to the similar historical work order text data to obtain the modified work order text data, and the descriptions in the modified work order text data are all highly standardized. The modified work order text data are respectively sent into the pre-set work order classification model based on ERNIE2, and the obtained categories are: Example 1 is in the voice service problem group; Example 2 is in the package switching problem group; Example 3 is in the package cancellation problem group.

[0061] Finally, the work order text data to be recognized can be directly transferred to the corresponding back-end technical personnel's workstations according to the work order categories.

[0062] The work order category recognition method provided by the present disclosure includes: obtaining the work order text data to be recognized, inputting the work order text data to be recognized into a preset work order matching model based on ERNIE2, and obtaining and displaying the historical work order text data similar to the work order text data to be recognized; obtaining the modified work order text data, where the modified work order text data is obtained by modifying the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized; inputting the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the work order text data to be recognized. The work order category recognition method provided by the present disclosure inputs the work order text data to be recognized into a preset work order matching model based on ERNIE2, and can match similar historical work order text data. The customer service staff can optimize the work order text data to be recognized according to the similar historical work order text data to obtain the modified work order text data, where the modified work order text data can be standardized work order text data; inputting the modified work order text into a preset work order classification model based on ERNIE2 can obtain the work order category of the work order text data to be recognized. The method provided by the present disclosure can automatically perform category recognition on the work order text data to be recognized, and improves the category recognition efficiency and accuracy, and enhances the user experience.

[0063] Figure 2 It is a schematic flowchart of the work order category recognition method shown in another exemplary embodiment of this application.

[0064] As Figure 2 shown, the work order category recognition method provided in this embodiment includes:

[0065] Step 201, obtaining the work order text data to be recognized.

[0066] Among them, the work order text data can be, for example, the text data edited by the customer service staff according to the user's complaint problem to describe the user's complaint problem. Specifically, in the operator service scenario, the work order text data can include, for example: (1) The package of Li Si's number cannot be changed. Please check the reason. Thank you! (2) The package of Zhang San's number is changed. Please check the reason. Thank you! (3) The user has a question about the monthly rent deduction for the used package. Please assist in checking whether the deduction is abnormal.

[0067] Among them, the work order text data to be recognized can be the work order text data of the work order category to be recognized.

[0068] Among them, the work order category can be a category preset according to the actual business situation. Specifically, in the operator business scenario, the work order category can include: network problem group, charging problem group, package problem group, etc. For example, if the work order text data represents that the user's complaint problem is a network problem, then the work order category of this work order text data can be classified into the network problem group, and the technical personnel corresponding to handling network problems can handle this work order.

[0069] Step 202: Input the work order text data to be recognized into a preset work order matching model based on ERNIE2 to obtain the feature vector of the work order text data to be recognized, and input the historical work order text data in the historical database into the preset work order matching model based on ERNIE2 to obtain the feature vector of the historical work order text data.

[0070] Among them, the preset work order matching model based on ERNIE2 can be a pre-trained ERNIE2 model using the Transformer structure. Inputting the work order text data to be recognized into this model can obtain the historical work order text data similar to the work order text data to be recognized.

[0071] Among them, the historical work order text data can be, for example, the work order text data that occurred in the past and is stored in the historical database.

[0072] Specifically, for example, inputting the work order text data to be recognized into the preset work order matching model based on ERNIE2 can perform tokenization processing on the work order text data to be recognized in the model, that is, convert each smallest calculation unit (such as a character or punctuation mark) in the work order text data into a word vector (Token Embeddings). The first token is output at the last layer of the Transformer as the feature vector of the work order text data to be recognized, which can be, for example, vector a(x1, x2, x3, x4, x5); similarly, inputting the historical work order text data into the preset work order matching model based on ERNIE2 can obtain the feature vector of the historical work order text data, which can be, for example, vector b(y1, y2, y3, y4, y5).

[0073] Step 203: Determine the similarity between the work order text data to be recognized and the historical work order text data according to the feature vector of the work order text data to be recognized and the feature vector of the historical work order text data.

[0074] Optionally, determine the cosine similarity between the feature vector of the work order text data to be recognized and the feature vector of the historical work order text data to obtain the similarity between the work order text data to be recognized and the historical work order text data.

[0075] Specifically, if the feature vector of the work order text data to be recognized is vector a(x1, x2, x3, …, x n ), and the feature vector of the historical work order text data is vector b(y1, y2, y3, …, y n ), then the similarity between the two can be calculated through the cosine formula. The larger the value, the more similar they are. The similarity formula between the work order text data to be recognized and the historical work order text data can be expressed as follows:

[0076]

[0077] where cosθ represents the cosine value between vector a(x1, x2, x3, …, x n ) and vector b(y1, y2, y3, …, y n ); where n represents the dimension of vector a and vector b, and n is a positive integer greater than or equal to 1.

[0078] Step 204, determine the historical work order text data with the top N similarity rankings as the historical work order text data similar to the work order text data to be recognized; where N is a positive integer greater than or equal to 1.

[0079] Specifically, the value of N can be preset according to actual needs. For example, N can be set to 5.

[0080] Specifically, the five values with the largest numerical values can be calculated according to the above similarity formula. The 5 historical work order text data corresponding to these five values can be the historical work order text data similar to the work order text data to be recognized.

[0081] Step 205, display the historical work order text data similar to the work order text data to be recognized.

[0082] Specifically, after the electronic device obtains the historical work order text data similar to the work order text data to be recognized, it can be displayed to the customer service staff through a display device. The display device can be, for example, a computer display screen or a mobile phone display screen, etc.

[0083] Furthermore, the customer service staff can modify and optimize the work order text data to be recognized according to the historical work order text data similar to the work order text data to be recognized. If the historical work order text data is standardized work order text data, the customer service staff can modify the work order text data to be recognized into standardized work order text data according to the historical work order text data similar to the work order text data to be recognized. The standardized work order text data describes the user's complaint problem more normatively, which is more conducive to the subsequent work order category recognition.

[0084] Step 206: Obtain the modified work order text data, where the modified work order text data is obtained by modifying the work order text data to be recognized based on historical work order text data similar to the work order text data to be recognized.

[0085] Specifically, the principle and implementation method of Step 206 are similar to those of Step 102, and will not be elaborated here.

[0086] Step 207: According to a preset stop word list, where the stop word list includes multiple stop words, match the modified work order text data to remove the words in the modified work order text that are the same as the stop words in the stop word list.

[0087] Among them, the multiple stop words included in the stop word list can be preset according to the actual situation. Specifically, the stop words can be words that have nothing to do with business description words.

[0088] Optionally, the modified work order text data can be preprocessed first, and after the preprocessing is completed, it is then input into a preset work order classification model based on ERNIE2. The preprocessing includes four steps from front to back: word segmentation using the Jieba library, removing stop words, sentence splicing, and making the sentence lengths uniform by padding with 0. Specifically, the modified work order text data can be segmented using the Jieba library, then the preset stop word list can be used to clean the useless words, and then the cleaned words can be spliced into a new sentence to remove the useless information.

[0089] Among them, the Jieba library is a Chinese word segmentation library. Among them, a word is the smallest meaningful language component that can move independently, and converting a sentence into a word representation is Chinese word segmentation.

[0090] For example, the stop words included in the preset stop word list can include: thank you, please, please assist. Then, if the modified work order text data is: The number of Li Si cannot be changed to a package, please check the reason, thank you! After removing the stop words, the modified work order text data after removing the stop words can be: The number of Li Si cannot be changed to a package, check the reason!

[0091] Step 208: Input the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the work order text data to be recognized.

[0092] Specifically, the modified work order text data can first remove the words that are the same as the stop words in the preset stop word list, and then be input into the preset work order classification model based on ERNIE2.

[0093] Among them, the preset work order classification model based on ERNIE2 can be the pre-trained ERNIE2 model adopting the Transformers structure. Inputting the modified work order text data into this model can obtain the work order category of the work order text data to be recognized.

[0094] Optionally, the work order matching model based on ERNIE2 is a model obtained by removing the SOFTMAX layer in the work order classification model based on ERNIE2, and the work order matching model based on ERNIE2 has a similarity processing module.

[0095] Among them, the role of the SOFTMAX layer in the work order classification model based on ERNIE2 is to convert the probabilities of various categories and obtain the work order category result.

[0096] Among them, the role of the similarity processing module is to calculate the cosine value of the feature vectors of two work order text data. Among them, the larger the value, the more similar the two work order text data are.

[0097] Figure 3 It is a schematic flowchart of a model training method applied to work order category recognition shown in an exemplary embodiment of the present application.

[0098] As Figure 3 shown, the model training method applied to work order category recognition provided in this embodiment includes:

[0099] Step 301, obtain a training set to be trained. Among them, the training set to be trained includes multiple historical work order text data, and each historical work order text data in the training set to be trained has an initial work order category.

[0100] Among them, the work order text data can be, for example, the text data edited by customer service staff according to the user's complaint problem to describe the user's complaint problem. Specifically, in the operator business scenario, the work order text data can include, for example: (1) The package of Li Si's number cannot be changed. Please check the reason. Thank you! (2) The package of Zhang San's number is changed. Please check the reason. Thank you! (3) The user has a question about the monthly rent deduction for the used package. Please assist in checking whether the deduction is abnormal.

[0101] Among them, the work order category can be a category preset according to the actual business situation. Specifically, in the operator business scenario, the work order category can include: network problem group, deduction problem group, package problem group, etc. For example, if the work order text data represents that the user's complaint problem is a network problem, the work order category of this work order text data can be classified into the network problem group, and then the technical personnel dealing with network problems can handle this work order.

[0102] Among them, the historical work order text data can be, for example, the work order text data that has occurred in the past and is stored in the historical database.

[0103] Among them, the set to be trained can include multiple historical work order text data stored in the historical database.

[0104] Specifically, the actual work order category to which each historical work order text data in the set to be trained belongs can be called the initial work order category.

[0105] Specifically, the electronic device can obtain the set to be trained.

[0106] Step 302: Repeat the following steps until a preset condition is reached. After reaching the preset condition, a work order classification model based on ERNIE2 is obtained: Input the historical work order text data into the initial model to obtain the predicted work order category of the historical work order text data; Adjust the parameters of the initial model according to the initial work order category and the predicted work order category of the historical work order text data.

[0107] Among them, the work order classification model based on ERNIE2 is used to identify the modified work order text data to obtain the work order category; The modified work order text data is obtained by modifying the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized; The historical work order text data similar to the work order text data to be recognized is obtained by inputting the work order text data to be recognized into a preset work order matching model based on ERNIE2.

[0108] Among them, ERNIE2 is the second generation of ERNIE. ERNIE2 is a pre-trained language model that unifies the modeling of lexical structure, syntactic structure, and semantic information in the training data, greatly enhancing the general semantic representation ability and achieving good results in multiple Chinese tasks.

[0109] Among them, the initial model can be a pre-built model, such as a pre-trained ERNIE2 model adopting a Transformers structure. Among them, Transformer is a kind of neural network structure.

[0110] Specifically, the electronic device can train this ERNIE2 model adopting a Transformers structure according to each historical work order text data and the corresponding initial work order category to obtain the target model, that is, the work order classification model based on ERNIE2.

[0111] Furthermore, each historical work order text data can be used as training data, and the initial work order category corresponding to each historical work order text data can be used as a data label, so that the electronic device can use the historical work order text data with the initial work order category to train the initial model.

[0112] Specifically, the initial model can process the input historical work order text data and identify the predicted work order category corresponding to the historical work order text data. It is also possible to compare the predicted work order category of the historical work order text data with the initial work order category, and then, based on the comparison result, adjust the parameters in the initial model. Through multiple iterations, the difference between the predicted work order category determined by the initial model and the initial work order category can meet the requirements, and then a work order classification model based on ERNIE2 that meets the requirements can be obtained.

[0113] Specifically, the initial model can perform tokenization processing on the historical text data, that is, convert each smallest computing unit (such as a character or punctuation mark) of the work order text data into a word vector (Token Embeddings). The CLS flag and SEP flag are added to the head and tail of the sentence in the historical text data respectively. Then, fine-tune the initial model. For the text classification task, take the output representation of the last layer Transformer of the first token, that is, the output representation of the CLS flag, and then connect a SOFTMAX layer to convert the probabilities of each classification and obtain the classification result of the historical text data.

[0114] Specifically, the preset condition can be that the number of repeated executions is greater than the preset number threshold; or the recognition probability obtained after verifying the initial model based on the test set is greater than the preset probability value; or the initial work order category and the predicted work order category of the historical work order text data are the same.

[0115] Specifically, the parameters of the work order classification model based on ERNIE2 obtained after meeting the preset conditions can be used to obtain a preset work order matching model based on ERNIE2. Specifically, the work order text data to be recognized can be input into the preset work order matching model based on ERNIE2 to obtain the historical work order text data similar to the work order text data to be recognized, and display it for the customer service staff to view. The customer service staff can modify the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized to obtain the modified work order text data. Among them, the historical work order text data can be a standard work order text data, and the modified work order text data obtained based on the historical work order text data can also be a standard work order text data. The modified work order text data can be input into the work order classification model based on ERNIE2 to obtain the work order category corresponding to the work order text data to be recognized.

[0116] Figure 4 A flowchart of a model training method for work order category recognition shown in another exemplary embodiment of the present application.

[0117] As Figure 4 shown, the model training method for work order category recognition provided in this embodiment includes:

[0118] Step 401, obtain a training set, where the training set includes multiple historical work order text data, and each historical work order text data in the training set has an initial work order category.

[0119] Specifically, the principle and implementation method of step 401 are similar to those of step 301, and will not be elaborated here.

[0120] Step 402, according to a preset stop word list, where the stop word list includes multiple stop words, match the historical work order text data to remove the words in the historical work order text data that are the same as the stop words in the stop word list.

[0121] Among them, the multiple stop words included in the stop word list can be preset according to the actual situation. Specifically, the stop words can be words that are irrelevant to business description words.

[0122] Optionally, the historical work order text data can be preprocessed first, and after the preprocessing, it is input into the initial model. The preprocessing can include four steps from front to back: word segmentation using the Jieba library, removing stop words, sentence splicing, and padding zeros for sentences with insufficient unified length. Specifically, after word segmentation of the historical work order text data using the Jieba library, the preset stop word list is used to clean the useless words, and then the cleaned words are spliced into a new sentence to remove the useless information.

[0123] Among them, the Jieba library is a Chinese word segmentation library. Among them, a word is the smallest meaningful language component that can move independently, and converting a sentence into a word representation is Chinese word segmentation.

[0124] For example, the stop words included in the preset stop word list can include: thank you, please, please assist. Then, if the modified work order text data is: The number of Li Si cannot be changed to a package, please check the reason, thank you! After removing the stop words, the modified work order text data after removing the stop words is: The number of Li Si cannot be changed to a package, check the reason!

[0125] Step 403: Repeat the following steps until a preset condition is met. After meeting the preset condition, a work order classification model based on ERNIE2 is obtained: Input the historical work order text data into the initial model to obtain the predicted work order category of the historical work order text data; Adjust the parameters of the initial model according to the initial work order category and the predicted work order category of the historical work order text data.

[0126] Among them, the work order classification model based on ERNIE2 is used to identify the work order category of the modified work order text data. The modified work order text data is obtained by modifying the work order text data to be identified based on the historical work order text data similar to the work order text data to be identified. The historical work order text data similar to the work order text data to be identified is obtained by inputting the work order text data to be identified into a preset work order matching model based on ERNIE2.

[0127] Among them, the initial model can be a pre-built model. For example, it can be a model based on the ERNIE2 model that adopts the Transformers structure.

[0128] Specifically, the electronic device can train the model based on the ERNIE2 model that adopts the Transformers structure according to each historical work order text data and the corresponding initial work order category to obtain the target model, that is, the work order classification model based on ERNIE2.

[0129] Furthermore, each historical work order text data can be used as training data, and the initial work order category corresponding to each historical work order text data can be used as a data label, so that the electronic device can use the historical work order text data with the initial work order category to train the initial model.

[0130] Specifically, the initial model can process the input historical work order text data and identify the predicted work order category corresponding to the historical work order text data. It can also compare the predicted work order category and the initial work order category of the historical work order text data, and then adjust the parameters in the initial model based on the comparison result. Through multiple iterations, the difference between the predicted work order category determined by the initial model and the initial work order category can meet the requirements, and then a work order classification model based on ERNIE2 that meets the requirements can be obtained.

[0131] Specifically, the initial model can perform tokenization on historical text data, that is, convert each smallest computational unit (such as characters or punctuation marks) of the work order text data into word vectors, Token Embeddings. The CLS flag and SEP flag are added to the head and tail of the sentence respectively. Then, fine-tune the initial model. For the text classification task, take the output representation of the last layer of Transformer of the first token, that is, the output representation of the CLS flag, and then connect a SOFTMAX layer to convert the probabilities of each classification and obtain the sentence classification result.

[0132] Specifically, based on the parameters of the work order classification model based on ERNIE2 obtained after meeting the preset conditions, a preset work order matching model based on ERNIE2 can be obtained. Specifically, the text data of the work order to be recognized can be input into the preset work order matching model based on ERNIE2 to obtain historical work order text data similar to the text data of the work order to be recognized, and display it for the customer service staff to view. The customer service staff can modify the text data of the work order to be recognized based on the historical work order text data similar to the text data of the work order to be recognized to obtain the modified work order text data. Among them, the historical work order text data can be standardized work order text data, and the modified work order text data obtained based on the historical work order text data can also be standardized work order text data. The modified work order text data can be input into the work order classification model based on ERNIE2 to obtain the work order category corresponding to the text data of the work order to be recognized.

[0133] Optionally, the preset condition is that the number of repeated executions is greater than the preset number threshold.

[0134] Among them, the preset number threshold can be a threshold set in advance according to the actual situation. For example, it can be set to 100,000 times. Specifically, it can be set that when the number of repeated executions is greater than 100,000 times, it is determined that the preset conditions are met, and a work order classification model based on ERNIE2 is obtained.

[0135] Optionally, the preset condition is that the initial work order category and the predicted work order category of the historical work order text data are the same.

[0136] Specifically, input the historical work order text data into the initial model, and the predicted work order category of the historical work order text data can be obtained. If the predicted work order category is the same as the initial work order category of the historical work order text data, it is determined that the preset conditions are met, and a work order classification model based on ERNIE2 is obtained.

[0137] Optionally, the preset condition is that the recognition probability obtained after verifying the initial model based on the test set is greater than the preset probability value, where the test set includes multiple work order text data to be tested, and each work order text data in the test set has a work order category.

[0138] Among them, the work order text data to be tested can be historical work order text data.

[0139] Among them, each work order text data in the test set has a work order category, and this work order category can be the actual work order category of the work order text data to be tested.

[0140] Among them, the recognition probability represents the probability of the consistency between the predicted work order category obtained from the initial model for the work order text data to be tested and the work order category it has.

[0141] Among them, the preset probability value is a probability value preset according to the actual situation, for example, it can be 95%.

[0142] For example, the number of historical work order text data included in the training set to be trained and the work order text data to be tested included in the test set can be fixed values respectively. For example, the number of historical work order text data included in the training set to be trained and the work order text data to be tested included in the test set can be 40,000 and 10,000 respectively. After the number of repeated executions of the initial model training reaches 400 times, the recognition probability obtained by verifying the initial model based on the test set. If the recognition probability value is less than the preset probability value, the initial model can continue to be trained; if the recognition probability value is greater than the preset probability value, it is determined that the preset condition is satisfied, and a work order classification model based on ERNIE2 is obtained.

[0143] Optionally, the work order matching model based on ERNIE2 is a model obtained by removing the SOFTMAX layer in the work order classification model based on ERNIE2, and the work order matching model based on ERNIE2 has a similarity processing module.

[0144] Among them, the role of the SOFTMAX layer in the work order classification model based on ERNIE2 is to convert the probabilities of each category to obtain the work order category result.

[0145] Among them, the role of the similarity processing module is to calculate the cosine value of the feature vectors of two work order text data. Among them, the larger the value, the more similar the two work order text data are.

[0146] Figure 5 This is the structural diagram of the work order category recognition device shown in an exemplary embodiment of the present application.

[0147] As Figure 5 shown, the work order category recognition device 500 provided in this embodiment includes:

[0148] The work order matching unit 510 is configured to obtain the work order text data to be recognized, input the work order text data to be recognized into a preset work order matching model based on ERNIE2, and obtain and display the historical work order text data similar to the work order text data to be recognized.

[0149] The obtaining unit 520 is configured to obtain the modified work order text data, where the modified work order text data is obtained by modifying the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized.

[0150] The category determination unit 530 is configured to input the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the work order text data to be recognized.

[0151] Figure 6 It is a structural diagram of a work order category recognition device shown in another exemplary embodiment of the present application.

[0152] As Figure 6 shown, based on the above embodiment, in the work order category recognition device 600 provided in this embodiment, the work order matching unit 510 includes:

[0153] The feature vector determination module 511 is configured to input the work order text data to be recognized into a preset work order matching model based on ERNIE2 to obtain the feature vector of the work order text data to be recognized, and input the historical work order text data in the historical database into the preset work order matching model based on ERNIE2 to obtain the feature vector of the historical work order text data.

[0154] The similarity processing module 512 is configured to determine the similarity between the work order text data to be recognized and the historical work order text data according to the feature vector of the work order text data to be recognized and the feature vector of the historical work order text data.

[0155] The similar historical work order determination module 513 is configured to determine the historical work order text data with the top N similarity rankings as the historical work order text data similar to the work order text data to be recognized; where N is a positive integer greater than or equal to 1.

[0156] The display module 514 is configured to display the historical work order text data similar to the work order text data to be recognized.

[0157] The similarity processing module 512 is further configured to determine the cosine similarity between the feature vector of the work order text data to be recognized and the feature vector of the historical work order text data to obtain the similarity between the work order text data to be recognized and the historical work order text data.

[0158] In the work order category recognition device 600 provided in this embodiment, a stop word removal unit 540 is further included, which is configured to, after obtaining the modified work order text data, match the modified work order text data according to a preset stop word list, where the stop word list includes multiple stop words, so as to remove the words in the modified work order text that are the same as the stop words in the stop word list.

[0159] Optionally, the work order matching model based on ERNIE2 is a model obtained by removing the SOFTMAX layer in the work order classification model based on ERNIE2, and the work order matching model based on ERNIE2 has a similarity processing module 512.

[0160] Figure 7 It is a structural diagram of a model training device applied to work order category recognition shown in an exemplary embodiment of the present application.

[0161] As Figure 7 shown, the model training device 700 applied to work order category recognition provided in this embodiment includes:

[0162] An acquisition unit 710, configured to acquire a training set, where the training set includes multiple historical work order text data, and each historical work order text data in the training set has an initial work order category;

[0163] A model training module 720, configured to repeat the following steps until a preset condition is reached, where a work order classification model based on ERNIE2 is obtained after reaching the preset condition: input the historical work order text data into the initial model to obtain the predicted work order category of the historical work order text data; adjust the parameters of the initial model according to the initial work order category and the predicted work order category of the historical work order text data;

[0164] Among them, the work order classification model based on ERNIE2 is used to recognize the modified work order text data to obtain the work order category; the modified work order text data is obtained by modifying the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized; the historical work order text data similar to the work order text data to be recognized is obtained by inputting the work order text data to be recognized into a preset work order matching model based on ERNIE2.

[0165] Figure 8 It is a structural diagram of a model training device applied to work order category recognition shown in another exemplary embodiment of the present application.

[0166] As Figure 8 shown, on the basis of the above embodiment, the model training device 800 applied to work order category recognition provided in this embodiment further includes:

[0167] A stop word removal unit 730 is configured to match historical work order text data with a preset stop word list before inputting the historical work order text data into an initial model to obtain a predicted work order category of the historical work order text data. The stop word list includes multiple stop words, so as to remove words in the historical work order text data that are the same as the stop words in the stop word list.

[0168] The model training module 720 is further configured that the preset condition is any one of the following:

[0169] The number of repeated executions is greater than a preset number threshold;

[0170] The initial work order category and the predicted work order category of the historical work order text data are the same;

[0171] The recognition probability obtained after validating the initial model based on a test set is greater than a preset probability value. The test set includes multiple work order text data to be tested, and each work order text data in the test set has a work order category.

[0172] Optionally, the work order matching model based on ERNIE2 is a model obtained by removing the SOFTMAX layer in the work order classification model based on ERNIE2, and the work order matching model based on ERNIE2 has a similarity processing module.

[0173] Figure 9 It is a structural diagram of an electronic device shown in an exemplary embodiment of the present application.

[0174] As Figure 9 shown, the electronic device provided in this embodiment includes:

[0175] A memory 901;

[0176] A processor 902; and

[0177] A computer program;

[0178] Wherein, the computer program is stored in the memory 901 and is configured to be executed by the processor 902 to implement any one of the above work order category recognition methods.

[0179] This embodiment further provides a computer-readable storage medium, on which a computer program is stored,

[0180] The computer program is executed by a processor to implement any one of the above work order category recognition methods.

[0181] This embodiment further provides a computer program product, including a computer program, which when executed by a processor, implements any one of the above work order category recognition methods.

[0182] Those of ordinary skill in the art will understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the foregoing storage medium includes various media such as ROM, RAM, magnetic disk, or optical disc that can store program codes.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A work order category recognition method, characterized in that, Including: Obtain the work order text data to be recognized, input the work order text data to be recognized into a preset work order matching model based on ERNIE2, and obtain and display historical work order text data similar to the work order text data to be recognized; Obtain the modified work order text data, where the modified work order text data is the work order text data obtained by performing normalization modification on the work order text data to be recognized based on the historical work order text data similar to the work order text data to be recognized; Input the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the work order text data to be recognized. The work order matching model based on ERNIE2 is a model obtained by removing the SOFTMAX layer in the work order classification model based on ERNIE2, and the work order matching model based on ERNIE2 has a similarity processing module.

2. The method according to claim 1, wherein Inputting the work order text data to be recognized into a preset work order matching model based on ERNIE2 to obtain and display historical work order text data similar to the work order text data to be recognized includes: Input the work order text data to be recognized into a preset work order matching model based on ERNIE2 to obtain the feature vector of the work order text data to be recognized, and input the historical work order text data in the historical database into a preset work order matching model based on ERNIE2 to obtain the feature vector of the historical work order text data; Determine the similarity between the work order text data to be recognized and the historical work order text data according to the feature vector of the work order text data to be recognized and the feature vector of the historical work order text data; Determine the historical work order text data with the top N similarity rankings as the historical work order text data similar to the work order text data to be recognized; where N is a positive integer greater than or equal to 1; Display the historical work order text data similar to the work order text data to be recognized.

3. The method according to claim 2, wherein Determining the similarity between the work order text data to be recognized and the historical work order text data according to the feature vector of the work order text data to be recognized and the feature vector of the historical work order text data includes; Determine the cosine similarity between the feature vector of the work order text data to be recognized and the feature vector of the historical work order text data to obtain the similarity between the work order text data to be recognized and the historical work order text data.

4. The method according to claim 1, characterized in that, After obtaining the modified work order text data, it further includes: Match the modified work order text data according to a preset stop word list, where the stop word list includes multiple stop words, so as to remove the words in the modified work order text that are the same as the stop words in the stop word list.

5. A model training method applied to work order category recognition, characterized in that, Including: Obtain a training set, where the training set includes multiple historical work order text data, and each historical work order text data in the training set has an initial work order category; Repeat the following steps until a preset condition is met, where a work order classification model based on ERNIE2 is obtained after meeting the preset condition: Input the historical work order text data into the initial model to obtain the predicted work order category of the historical work order text data; Adjust the parameters of the initial model according to the initial work order category and the predicted work order category of the historical work order text data. Among them, the work order classification model based on ERNIE2 is used to identify the modified work order text data to obtain the work order category; The modified work order text data is the work order text data obtained by normalizing the to-be-identified work order text data based on the historical work order text data similar to the to-be-identified work order text data. The historical work order text data similar to the to-be-identified work order text data is obtained by inputting the to-be-identified work order text data into a preset work order matching model based on ERNIE2. The work order matching model based on ERNIE2 is a model obtained by removing the SOFTMAX layer in the work order classification model based on ERNIE2, and the work order matching model based on ERNIE2 has a similarity processing module.

6. The method according to claim 5, wherein Before inputting the historical work order text data into the initial model to obtain the predicted work order category of the historical work order text data, it further includes: According to a preset stop word list, where the stop word list includes multiple stop words, match the historical work order text data to remove the words in the historical work order text data that are the same as the stop words in the stop word list.

7. The method according to claim 5, characterized in that The preset condition is any one of the following: The number of repeated executions is greater than a preset number threshold; The initial work order category and the predicted work order category of the historical work order text data are the same; The recognition probability obtained after validating the initial model based on a test set is greater than a preset probability value, where the test set includes multiple to-be-tested work order text data, and each to-be-tested work order text data in the test set has a work order category.

8. A work order category recognition device, characterized in that It includes: A work order matching unit, configured to obtain the to-be-identified work order text data, input the to-be-identified work order text data into a preset work order matching model based on ERNIE2, and obtain and display the historical work order text data similar to the to-be-identified work order text data; An acquisition unit, configured to acquire the modified work order text data, where the modified work order text data is the work order text data obtained by normalizing the to-be-identified work order text data based on the historical work order text data similar to the to-be-identified work order text data; A category determination unit, configured to input the modified work order text data into a preset work order classification model based on ERNIE2 to obtain the work order category of the to-be-identified work order text data. The work order matching model based on ERNIE2 is a model obtained by removing the SOFTMAX layer in the work order classification model based on ERNIE2, and the work order matching model based on ERNIE2 has a similarity processing module.

9. A model training device applied to work order category recognition, characterized in that It includes: An acquisition unit for acquiring a training set, where the training set includes a plurality of historical work order text data, and each piece of historical work order text data in the training set has an initial work order category; A model training module for repeating the following steps until a preset condition is reached, where after reaching the preset condition, a work order classification model based on ERNIE2 is obtained: inputting the historical work order text data into an initial model to obtain a predicted work order category of the historical work order text data; adjusting the parameters of the initial model according to the initial work order category and the predicted work order category of the historical work order text data; wherein, the work order classification model based on ERNIE2 is used to identify the modified work order text data to obtain a work order category; the modified work order text data is work order text data obtained by performing a normalization modification on the to-be-identified work order text data based on historical work order text data similar to the to-be-identified work order text data; the historical work order text data similar to the to-be-identified work order text data is obtained by inputting the to-be-identified work order text data into a preset work order matching model based on ERNIE2, the work order matching model based on ERNIE2 is a model obtained by removing the SOFTMAX layer in the work order classification model based on ERNIE2, and the work order matching model based on ERNIE2 has a similarity processing module.

10. An electronic device, characterized in that, It includes a memory and a processor; wherein, the memory is used for storing a computer program; the processor is used for reading the computer program stored in the memory and executing the method according to any one of claims 1-4 or 5-7 above based on the computer program in the memory.

11. A computer-readable storage medium, characterized in that, A computer-executable instruction is stored in the computer-readable storage medium, and when the processor executes the computer-executable instruction, the method according to any one of claims 1-4 or 5-7 above is implemented.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1-4 or 5-7 above is implemented.

Images