Work order data classification method and device, equipment, storage medium and product
By roughly classifying, sub-categorizing and keyword classification of complaint work order data, the problem of handling massive work order data is solved, efficiency and accuracy are improved, and massive data needs in the telecommunications field are met.
Patent Information
- Application Number
- CN202411083412.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-22
AI Technical Summary
In the field of telecommunications, it is difficult to classify massive complaint work order data efficiently and accurately. The existing technology is applicable when the data volume is small but cannot meet the needs of massive data processing.
By roughly classifying the complaint work ticket data, subclassing it using a preset language model, and hierarchically classifying the data according to keywords, a deeper subclassing is achieved, and a hierarchical classification method is used to process massive data.
It improves the processing efficiency and classification accuracy of massive work order data, reduces the workload of customer service, quickly finds similar complaint solutions, and prevents resource waste.
Smart Images

Figure CN119004293A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment, storage medium and product for classifying work order data. Background Art
[0002] Currently, there are relatively few intelligent means of handling customer complaint tickets, but the demand for intelligent analysis and processing of complaint tickets is increasing day by day. Through intelligent analysis and processing of complaint tickets, the workload of customer service can be greatly reduced and work efficiency can be improved; solutions to similar complaints can be quickly found to solve customer problems; problems with high frequency of complaints can be found, so as to prevent or solve them in advance to avoid waste of resources. The current technical solution is more suitable for deployment when the amount of work order data is relatively small, but in the telecommunications field, the number of complaint tickets may not be limited to less than one million, because the number of users in the telecommunications field is huge, with hundreds of millions of users. Even if the number of complaint tickets per day is small, after a period of accumulation, the text data of complaint tickets is likely to reach a considerable amount. At this time, simple clustering cannot meet the needs of massive work order data. Therefore, how to efficiently and accurately classify massive work order data has become an urgent problem to be solved. Summary of the invention
[0003] The main purpose of this application is to provide a method, device, equipment, storage medium and product for classifying work order data, aiming to solve the technical problem of how to efficiently and accurately classify massive work order data.
[0004] To achieve the above purpose, the present application provides a method for classifying work order data, and the method for classifying work order data comprises the following steps:
[0005] Roughly classify the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data;
[0006] The various types of first work order data are subdivided by using a preset language model to obtain multiple types of second work order data;
[0007] The various types of second work order data are classified according to keywords corresponding to the various types of second work order data to obtain multiple types of third work order data.
[0008] Optionally, before the step of sub-classifying each type of first work order data by using a preset language model to obtain multiple types of second work order data, the step further includes:
[0009] Classify the initial work order data to obtain multiple types of training data;
[0010] Selecting training samples from the multiple types of training data;
[0011] The feature representation layer and the classification layer in the initial language model are trained through the training samples to obtain a preset language model, and the initial language model is a model for performing knowledge distillation on the BERT model.
[0012] Optionally, the step of selecting training samples from the multiple types of training data specifically includes:
[0013] Selecting initial positive samples from training data of the same category, and performing data enhancement on the initial positive samples to obtain enhanced positive samples;
[0014] Determine the target category corresponding to the initial positive sample, and select remaining training data corresponding to remaining categories except the target category from the multiple categories of training data;
[0015] Calculating the similarity between the positive sample and the remaining training data by a text similarity algorithm, and selecting a negative sample from the remaining training data according to the similarity;
[0016] A training sample is determined according to the enhanced positive sample and the negative sample.
[0017] Optionally, the step of training the feature representation layer and the classification layer in the initial language model by using the training samples to obtain a preset language model specifically includes:
[0018] Selecting target positive samples and target negative samples corresponding to the original samples from the training samples;
[0019] Determine an original vector corresponding to the original sample, a positive vector corresponding to the target positive sample, and a negative vector corresponding to the target negative sample;
[0020] Calculating a first cosine similarity between the original vector and the positive vector, and calculating a second cosine similarity between the original vector and a random Gaussian noise vector;
[0021] An objective function is constructed according to the first cosine similarity, the temperature parameter and the second cosine similarity, and a feature representation layer and a classification layer in an initial language model are trained based on the objective function to obtain a preset language model.
[0022] Optionally, the step of classifying each type of second work order data according to keywords corresponding to each type of second work order data to obtain multiple types of third work order data specifically includes:
[0023] Extract keywords corresponding to work order titles and work order contents in various types of second work order data;
[0024] Combining the keywords to obtain combined keywords;
[0025] The various types of second work order data are classified respectively according to the combined keywords to obtain multiple types of third work order data.
[0026] Optionally, after the step of classifying the various types of second work order data according to the combined keywords to obtain multiple types of third work order data, the method further includes:
[0027] For each type of third work order data, a feature vector output by the preset language model is obtained, where the feature vector represents a feature vector obtained by concatenating the work order title and the work order content;
[0028] The cosine similarity between the feature vectors is calculated, and the third work order data of each category is partitioned and stored according to the cosine similarity.
[0029] In addition, to achieve the above-mentioned purpose, the present application also provides a work order data classification device, the work order data classification device comprising:
[0030] A data rough classification module, used to roughly classify the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data;
[0031] A data sub-classification module is used to sub-classify various types of first work order data respectively through a preset language model to obtain multiple types of second work order data;
[0032] The data classification module is used to classify each type of second work order data according to the keywords corresponding to each type of second work order data to obtain multiple types of third work order data.
[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a work order data classification device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the work order data classification method as described above.
[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the work order data classification method described above are implemented.
[0035] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the work order data classification method as described above are implemented.
[0036] This application roughly classifies the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data, and then finely classifies each type of first work order data through a preset language model to obtain multiple categories of second work order data, and then classifies each type of second work order data according to the keywords corresponding to each type of second work order data to obtain multiple categories of third work order data. This application first roughly classifies the complaint work order data, and then finely classifies each type of first work order data. For each roughly classified category, each type of first work order data can be further finely classified to achieve a more accurate classification effect. Then, each type of second work order data can be classified according to keywords. By adopting a hierarchical classification method, it is possible to efficiently process massive amounts of work order data, improve processing efficiency and maintain the accuracy of work order data classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0039] Figure 1 This is a flow chart of the first embodiment of the classification method for work order data of this application;
[0040] Figure 2 This is a flow chart of a second embodiment of the method for classifying work order data of this application;
[0041] Figure 3 A flowchart of a preset language model for training in an embodiment of a method for classifying work order data of the present application;
[0042] Figure 4 This is a flow chart of a third embodiment of the method for classifying work order data of this application;
[0043] Figure 5 This is an overall schematic diagram of an embodiment of a device for classifying work order data of the present application;
[0044] Figure 6 This is a structural block diagram of the first embodiment of the work order data classification device of the present application;
[0045] Figure 7 It is a structural diagram of a work order data classification device in a hardware operating environment involved in an embodiment of the present application.
[0046] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0049] The main solution of the embodiment of the present application is: roughly classify the complaint work order data according to the work order title in the complaint work order data to obtain multiple categories of first work order data; finely classify each type of first work order data through a preset language model to obtain multiple categories of second work order data; classify each type of second work order data according to the keywords corresponding to each type of second work order data to obtain multiple categories of third work order data.
[0050] Currently, there are relatively few intelligent means of handling customer complaint tickets, but the demand for intelligent analysis and processing of complaint tickets is increasing day by day. Through intelligent analysis and processing of complaint tickets, the workload of customer service can be greatly reduced and work efficiency can be improved; solutions to similar complaints can be quickly found to solve customer problems; problems with high frequency of complaints can be found, so as to prevent or solve them in advance and avoid waste of resources. The current technical solution is more suitable for deployment when the amount of work order data is relatively small, but the number of complaint work orders in the telecommunications field may not be limited to less than one million, because the number of users in the telecommunications field is huge, with hundreds of millions of users. Even if the number of complaint work orders is small every day, after a period of accumulation, the text data of complaint work orders is likely to reach a considerable amount. At this time, simple clustering cannot meet the needs of massive work order data.
[0051] This application roughly classifies the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data, and then finely classifies each type of first work order data through a preset language model to obtain multiple categories of second work order data, and then classifies each type of second work order data according to the keywords corresponding to each type of second work order data to obtain multiple categories of third work order data. This application first roughly classifies the complaint work order data, and then finely classifies each type of first work order data. For each roughly classified category, each type of first work order data can be further finely classified to achieve a more accurate classification effect. Then, each type of second work order data can be classified according to keywords. By adopting a hierarchical classification method, it is possible to efficiently process massive amounts of work order data, improve processing efficiency and maintain the accuracy of work order data classification.
[0052] It should be noted that the execution subject of the present application may be a computing service device with data processing, network communication and program running functions, such as a computer, etc., or an electronic device capable of realizing the above functions, a work order data classification device, etc. The following takes the work order data classification device as an example to illustrate this embodiment and the following embodiments.
[0053] Based on this, the present application embodiment provides a method for classifying work order data, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for classifying work order data of the present application.
[0054] In this embodiment, the work order data classification method includes the following steps:
[0055] Step S10: roughly classify the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data.
[0056] It is understandable that the complaint ticket data may include the ticket title, ticket content, and solution. These three fields can be concatenated, and then the jieba word segmentation tool can be used to segment the concatenated data, and the segmented text can be searched and analyzed to find the proper nouns that were incorrectly segmented, and then included in the dictionary library to generate the final dictionary library.
[0057] It should be understood that this embodiment can remove special characters in the complaint work order data, including punctuation marks, space characters, etc., and can also replace numbers, dates, or longer non-Chinese character strings in the complaint work order data with special words, such as number, date, ordernumber, jobnumber, etc.
[0058] In the specific implementation, the semantics of the work order title in the preprocessed work order data are relatively simple and the text length is relatively short. The FastText model can consider the contextual information of the vocabulary. In addition, the daily data volume of the work order is large, and the FastText model can quickly complete the diversion of the work order title. This embodiment can select this model as a lightweight model for coarse classification of titles, thereby realizing coarse classification through the FastText model and obtaining multiple first work order data.
[0059] Step S20: Sub-classify each type of first work order data using a preset language model to obtain multiple types of second work order data.
[0060] It is understandable that the preset language model in this embodiment refers to a pre-set language model, which may specifically be a DistilBERT model, and each type of first work order data is subdivided by the preset language model to obtain multiple types of second work order data.
[0061] Step S30: Classify each type of second work order data according to the keywords corresponding to each type of second work order data to obtain multiple types of third work order data.
[0062] It should be understood that for each type of second work order data, the keywords corresponding to the type of second work order data can be determined. Specifically, the keywords can be selected from the work order title and work order content of the type of second work order data, and the type of second work order data can be classified according to the keywords to obtain multiple types of third work order data.
[0063] This embodiment roughly classifies the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data, and then finely classifies each category of first work order data through a preset language model to obtain multiple categories of second work order data, and then classifies each category of second work order data according to the keywords corresponding to each category of second work order data to obtain multiple categories of third work order data. This embodiment first roughly classifies the complaint work order data, and then finely classifies each category of first work order data. For each roughly classified category, each category of first work order data can be further finely classified to achieve a more accurate classification effect. Then, each category of second work order data is classified according to keywords. By adopting a hierarchical classification method, a large amount of work order data can be efficiently processed, processing efficiency can be improved, and the accuracy of work order data classification can be maintained.
[0064] refer to Figure 2 , Figure 2 This is a flow chart of the second embodiment of the method for classifying work order data of the present application.
[0065] Based on the first embodiment above, in this embodiment, before step S20, the following steps are further included:
[0066] Step S01: Classify the initial work order data to obtain multiple categories of training data.
[0067] It is understandable that the initial work order data can be used for model training, specifically, the complaint work order data collected in the historical time period. The initial work order data can be classified to obtain multiple types of training data. Specifically, each type of training data can be identified according to the category, and the training data of each category is kept balanced.
[0068] Step S02: Select training samples from the multiple types of training data.
[0069] Furthermore, in order to effectively select training samples, in this embodiment, the step S02 includes: selecting an initial positive sample from the training data of the same category, and performing data enhancement on the initial positive sample to obtain an enhanced positive sample; determining a target category corresponding to the initial positive sample, and selecting remaining training data corresponding to remaining categories other than the target category from the multiple categories of training data; calculating the similarity between the positive sample and the remaining training data through a text similarity algorithm, and selecting a negative sample from the remaining training data based on the similarity; and determining a training sample based on the enhanced positive sample and the negative sample.
[0070] It should be understood that reference Figure 3 , Figure 3 This is a flow chart of training a preset language model in an embodiment of the classification method of work order data of this application. In the process of designing training samples, initial positive samples can be selected from the training data of the same category, and data enhancement can be performed on the initial positive samples to obtain enhanced positive samples. Specifically, data enhancement can be performed through a knowledge-enhanced large language model. The specific prompt word prompt is described as follows:
[0071] PROMPT_TEMPLATE="""I am generalizing the training sentences. Please generalize 5 more sentences. Please note that the generalized sentences should have the same meaning as the original sentences. Do not change the keywords in the original sentences, and do not change the English words and numbers contained in the original sentences.
[0072] For example:
[0073] "wms 13564440501 ordering channel question" can be generalized
[0074] "wms 13564440501 Questions about ordering channels"
[0075] "wms 13564440501 Questions about ordering channels"
[0076] "Questions about wms 13564440501 ordering channel"
[0077] "Questions about the ordering channel of wms 13564440501"
[0078] \nOriginal corpus "{question}" \nPlease include the number in the output format"""
[0079] Replace "{question}" in the "PROMPT_TEMPLATE" description with the text to be enhanced, and directly call the API interface of the knowledge-enhanced large language model to achieve data enhancement.
[0080] In a specific implementation, the target category corresponding to the initial positive sample can be determined, and the remaining training data corresponding to the remaining categories other than the target category can be selected from the multi-category training data. The remaining training data is different from the category to which the initial positive sample belongs. Then the similarity between the positive sample and the remaining training data is calculated by the text similarity algorithm, and the negative sample is selected from the remaining training data based on the similarity, that is, the negative sample is selected from the remaining training data with the maximum similarity. The purpose of contrastive learning is to reduce the distance between the same category and increase the distance between different categories, so as to obtain a better representation vector for the text. The negative sample is designed in this way with the intention of enhancing the model to have a better vector representation of texts of similar but different categories. The enhanced positive and negative samples are then used as training samples.
[0081] Step S03: Train the feature representation layer and the classification layer in the initial language model through the training samples to obtain a preset language model, where the initial language model is a model for performing knowledge distillation on the BERT model.
[0082] It is understandable that the initial language model can be trained through training samples. Specifically, the initial layers can be frozen and only the last two network layers can be trained, that is, the feature representation layer and the representation layer can be trained to obtain a preset language model. This can not only retain the Chinese semantic understanding ability of the pre-trained model, but also improve the Chinese semantic understanding ability of the work order data. On this basis, it is easy to quickly understand the work order data, so as to achieve rapid convergence and improve the training speed.
[0083] In the specific implementation, the initial credit model can be a model that performs knowledge distillation on the BERT model. Specifically, the knowledge distillation technology can be used to select a large-scale Chinese STS data set to train the teacher model BERT and the student model DistilBERT, obtain the pre-trained DistilBERT as the preset language model model, and let the DistilBERT model learn the Chinese semantic expression ability. For example, for the work order data "wms13564440501 order channel question", the frozen layers can already understand the semantic information of "13564440501", "order channel", and "question". It is only necessary to let the model understand the complete semantic information of "wms 13564440501 order channel question" at the feature representation layer and classification layer.
[0084] Furthermore, in order to effectively train the initial language model, in this embodiment, the step S03 includes: selecting a target positive sample and a target negative sample corresponding to the original sample from the training sample; determining an original vector corresponding to the original sample, a positive vector corresponding to the target positive sample, and a negative vector corresponding to the target negative sample; calculating a first cosine similarity between the original vector and the positive vector, and calculating a second cosine similarity between the original vector and a random Gaussian noise vector; constructing an objective function according to the first cosine similarity, a temperature parameter, and the second cosine similarity, and training the feature representation layer and the classification layer in the initial language model based on the objective function to obtain a preset language model.
[0085] It should be understood that target positive samples and target negative samples corresponding to the original samples can be selected from the training samples, that is, target positive samples corresponding to the original samples are selected from the enhanced positive samples, and target negative samples corresponding to the original samples are selected from the negative samples.
[0086] It can be understood that the objective function is expressed as:
[0087]
[0088] Among them, h i is the original vector corresponding to the original sample, is the positive vector corresponding to the target positive sample, is the negative vector corresponding to the target negative sample, sim() is the cosine similarity of the two vectors, τ is the temperature parameter, and g m is a random Gaussian noise vector. Gaussian noise can simulate the uncertainty of the real environment and play a regularization effect. In order to maximize the effect of the added Gaussian noise, this embodiment will Adding it to the objective function can improve the robustness of the DistilBERT model training in this scenario, so that the model training can stably reach a convergence state, while improving the generalization performance of the model and suppressing overfitting.
[0089] This embodiment obtains multiple types of training data by classifying the initial work order data, then selects training samples from the multiple types of training data, and then trains the feature representation layer and classification layer in the initial language model through the training samples to obtain a preset language model, which is a model for knowledge distillation of the BERT model. This embodiment trains the feature representation layer and classification layer in the initial language model through training samples to obtain a preset language model, which not only retains the model's understanding of Chinese semantics, but also enables rapid convergence of training and accelerates the training process.
[0090] refer to Figure 4 , Figure 4This is a flow chart of the third embodiment of the method for classifying work order data of the present application.
[0091] Based on the above embodiments, in this embodiment, step S30 includes:
[0092] Step S301: extract keywords corresponding to the work order titles and work order contents in each type of second work order data.
[0093] It is understandable that since the work order title has a shorter text length but contains more key information, while the work order content has a longer text length but contains less key information than the work order title. TF-IDF is used to measure the importance of keywords in work order data, so TF-IDF is used to extract the top two keywords in the work order title and the top three keywords in the work order content that are not repeated with the keywords in the work order title, and a total of five most important and non-repeating keywords are extracted.
[0094] Step S302: Combine the keywords to obtain combined keywords.
[0095] It should be understood that the extracted keywords are arranged and combined to form combined keywords, and the number of possible combinations is: Note that the work order title and work order content are not concatenated here, and the top 5 keywords are directly calculated. The reason is that the work order title contains a lot of information but is short in length. If it is concatenated with the work order content, it may be diluted by the long text of the work order content. The extracted keywords all come from the work order content.
[0096] Step S303: Classify the various types of second work order data respectively according to the combined keywords to obtain multiple types of third work order data.
[0097] Understandably, referring to Figure 5 , Figure 5 This is an overall schematic diagram of an embodiment of a classification device for work order data of this application. Figure 5As shown in the figure, based on the first-layer trained FastText coarse flow, the work order data is further diverted to the second-layer fine-tuned DistilBERT segmented flow according to the first-level classification categories (such as ['complaint', 'business channel', 'other']). These second-level classification categories (such as ["Internet service", "secondary number service", "detailed order management", "other"]) are subdivided categories of the first-level classification categories. Compared with the coarse classification of the first layer, the second-layer fine-tuning will classify the relatively small amount of work order data more finely, thereby improving the accuracy of classification. In the second-layer fine-tuned DistilBERT sub-classification, for each sub-category (such as "Internet service"), the keywords of the work order data are extracted (such as ["upgrade complaint", "4G upgrade", "traffic package", "directional traffic", "koi card set"]). Then, these keywords are combined to construct the third-layer keyword combination diversion. In this way, for each sub-category, the work order data can be further processed in a deeper segmentation flow, so as to achieve a more accurate classification effect. The same is true for other categories. The specific implementation details are as follows:
[0098] 1) First-level classification: Use FastText to roughly classify the work order data based on the 'work order title'. For example, if the classification categories are ['complaints', 'business channels', 'others'], data other than 'complaints' and 'business channels' are all placed in the 'others' category. After a period of time, randomly sample the data in the 'others' category and manually screen whether there is a need to build new categories. If there is data in a new category, it is necessary to add a new category to the first-level classification, such as 'cooperative investigation'. Collect and organize the training data set, retrain the FastText model, replace the original model, re-predict the data in the 'others' category, and realize the addition of the 'cooperative investigation' category, thus avoiding the migration of all archived data. 2) Secondary classification: Use the special symbol " / " to connect 'work order title' and 'work order content'. Use the fine-tuned improved DistilBERT model to directly perform secondary classification on the data after the first-level diversion, so as to achieve a finer classification category for the work order data. As the second-level classification. Here, the second-level categories under each first-level branch include the 'others' category, and the reason is the same as the creation of the 'first-level classification'. 3) Three-level classification: Since the text length of the "work order title" is shorter but the key information is larger, and the text length of the "work order content" is longer but the key information is smaller than that of the "work order title", TF-IDF is used to measure the importance of keywords in the work order data. Therefore, TF-IDF is used to extract the top two keywords in the "work order title" and the top three keywords in the "work order content" that are not repeated with the keywords of the "work order title". A total of five most important and non-repeating keywords are extracted, and the extracted keywords are arranged and combined to form a three-level classification.
[0099] Furthermore, in order to effectively store various types of third work order data, in this embodiment, after step S303, it also includes: for each type of third work order data, obtaining the feature vector output by the preset language model, the feature vector represents the feature vector after the work order title and the work order content are concatenated; calculating the cosine similarity between the feature vectors, and partitioning and storing the various types of third work order data according to the cosine similarity.
[0100] It should be understood that the detailed implementation steps of this embodiment are: 1. Obtain complaint work order data (assuming there are 2000 items), the work order data is roughly as follows: {Work order title: [User complains about poor signal], work order content: [User A calls and complains that the mobile phone cannot connect to the phone and cannot receive messages, which has seriously affected his work and life. We need to help him solve it quickly, address ********], solution: [After verification, the equipment in the ****** area is currently under maintenance, and the customer has been informed to wait for the maintenance to be completed], priority: [Medium],...}. Analyze the work order data, the "work order title" and "work order content" contain a lot of information, "solution" can provide a reference for the operator, "priority" can be used as a grading standard, and the work orders with high "priority" will be pushed to the operator for processing first, forming a complete system. Segment the data in the data list, and then view and analyze the data to find segmentation errors or words that do not conform to common sense, and include them in the domain vocabulary.
[0101] 2. Preprocess the work order data, remove special characters in the work order data, and replace characters that affect the model effect, such as numbers, with special marks; use Jieba word segmentation to load the domain vocabulary and segment the work order data.
[0102] 3. Use the 'work order title' data to train the FastText model to build the first-level classification.
[0103] 4. Use the special symbol " / " to concatenate 'work order title' and 'work order content' to fine-tune the base model DistilBERT for each secondary category under the first-level category. The base model DistilBERT is a model pre-trained with a large-scale Chinese STS dataset. The fine-tuning process is to freeze the initial layers of the base model DistilBERT and only train the last two network layers. This not only retains the model's understanding of Chinese semantics, but also enables rapid convergence of training and accelerates the training process. In the process of diversion, the fine-tuned DistilBERT model is used to build the secondary classification.
[0104] 5. Extract 5 keywords from the work order data, and construct a three-level classification through permutations and combinations. The specific storage location under the three-level classification refers to the following process: The results of extracting the last two layers of hidden states of the base model DistilBERT model are used to represent the feature vector after the concatenation of 'work order title' and 'work order content', and use the cosine similarity measurement to store data with similar feature vectors in adjacent positions on the same partition of the disk. That is, similar data is put together. In this way, when new data needs to be diverted, it is only necessary to find the most similar data that has been stored in the database on a partition similar to it (a small range), without searching on the entire disk, and new work order data can be quickly diverted.
[0105] 6. Count the categories with the most data in each partition and return them to the front end as popular complaints. For the newly generated work order, similar partition addresses and feature vectors are obtained according to steps 3, 4, and 5 above. The work order content that is most similar to the newly generated work order is obtained based on the cosine similarity, and is greater than the given threshold (empirical value 0.85). Get the "solution" and "priority" from similar work orders to ensure that urgent work orders are pushed to operators and provide reference for their operations.
[0106] In this embodiment, keywords corresponding to the work order titles and work order contents in various types of second work order data are extracted, and then the keywords are combined to obtain combined keywords, and then various types of second work order data are classified according to the combined keywords to obtain multiple types of third work order data. In this embodiment, keywords corresponding to the work order titles and work order contents in various types of second work order data are extracted, and then the keywords are combined, and then various types of second work order data are classified according to the combined keywords, so that various types of second work order data can be further effectively and accurately classified on the basis of coarse classification and fine classification.
[0107] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the work order data classification device of the present application.
[0108] like Figure 6 As shown, the work order data classification device proposed in the embodiment of the present application includes:
[0109] A data rough classification module 10 is used to roughly classify the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data;
[0110] A data sub-classification module 20 is used to sub-classify each type of first work order data by using a preset language model to obtain multiple types of second work order data;
[0111] The data classification module 30 is used to classify each type of second work order data according to the keywords corresponding to the second work order data to obtain multiple types of third work order data.
[0112] This embodiment roughly classifies the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data, and then finely classifies each category of first work order data through a preset language model to obtain multiple categories of second work order data, and then classifies each category of second work order data according to the keywords corresponding to each category of second work order data to obtain multiple categories of third work order data. This embodiment first roughly classifies the complaint work order data, and then finely classifies each category of first work order data. For each roughly classified category, each category of first work order data can be further finely classified to achieve a more accurate classification effect. Then, each category of second work order data is classified according to keywords. By adopting a hierarchical classification method, a large amount of work order data can be efficiently processed, processing efficiency can be improved, and the accuracy of work order data classification can be maintained.
[0113] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.
[0114] In addition, for technical details that are not described in detail in this embodiment, please refer to the classification method of work order data provided in any embodiment of the present application, and will not be repeated here.
[0115] Based on the first embodiment of the device for classifying work order data of the present application, a second embodiment of the device for classifying work order data of the present application is proposed.
[0116] In this embodiment, the data subclassification module 20 is also used to classify the initial work order data to obtain multiple categories of training data; select training samples from the multiple categories of training data; train the feature representation layer and classification layer in the initial language model through the training samples to obtain a preset language model, and the initial language model is a model for performing knowledge distillation on the BERT model.
[0117] Furthermore, the data subclassification module 20 is also used to select initial positive samples from the training data of the same category, and perform data enhancement on the initial positive samples to obtain enhanced positive samples; determine the target category corresponding to the initial positive samples, and select remaining training data corresponding to remaining categories other than the target category from the multiple categories of training data; calculate the similarity between the positive samples and the remaining training data through a text similarity algorithm, and select negative samples from the remaining training data based on the similarity; determine training samples based on the enhanced positive samples and the negative samples.
[0118] Furthermore, the data subclassification module 20 is also used to select a target positive sample and a target negative sample corresponding to the original sample from the training sample; determine the original vector corresponding to the original sample, the positive vector corresponding to the target positive sample, and the negative vector corresponding to the target negative sample; calculate the first cosine similarity between the original vector and the positive vector, and calculate the second cosine similarity between the original vector and the random Gaussian noise vector; construct an objective function according to the first cosine similarity, the temperature parameter and the second cosine similarity, and train the feature representation layer and the classification layer in the initial language model based on the objective function to obtain a preset language model.
[0119] Furthermore, the data classification module 30 is also used to extract keywords corresponding to the work order titles and work order contents in each type of second work order data; combine the keywords to obtain combined keywords; and classify the various types of second work order data according to the combined keywords to obtain multiple types of third work order data.
[0120] Furthermore, the data classification module 30 is also used to obtain the feature vector output by the preset language model for each type of third work order data, the feature vector representing the feature vector after the work order title and the work order content are concatenated; calculate the cosine similarity between the feature vectors, and partition and store the each type of third work order data according to the cosine similarity.
[0121] Other embodiments or specific implementation methods of the work order data classification device of the present application can refer to the above-mentioned method embodiments and will not be repeated here.
[0122] The present application provides a work order data classification device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the work order data classification method in the above-mentioned embodiment one.
[0123] Reference below Figure 7, which shows a schematic diagram of a structure of a work order data classification device suitable for implementing the embodiment of the present application. The work order data classification device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The work order data classification device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0124] like Figure 7 As shown, the classification device for work order data may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the classification device for work order data are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the work order data classification device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a work order data classification device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.
[0125] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0126] The work order data classification device provided by the present application adopts the work order data classification method in the above embodiment, which can solve the technical problem of how to efficiently and accurately classify massive amounts of work order data. Compared with the prior art, the beneficial effects of the work order data classification device provided by the present application are the same as the beneficial effects of the work order data classification method provided by the above embodiment, and other technical features in the work order data classification device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.
[0127] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0128] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0129] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the method for classifying work order data in the above-mentioned embodiment.
[0130] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0131] The computer-readable storage medium may be included in the work order data classification device; or may exist independently without being assembled into the work order data classification device.
[0132] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the work order data classification device, the work order data classification device: roughly classifies the complaint work order data according to the work order title in the complaint work order data to obtain multiple categories of first work order data; finely classifies each type of first work order data through a preset language model to obtain multiple categories of second work order data; and classifies each type of second work order data according to the keywords corresponding to each type of second work order data to obtain multiple categories of third work order data.
[0133] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0134] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0135] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0136] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned classification method for work order data, and can solve the technical problem of how to efficiently and accurately classify massive amounts of work order data. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the classification method for work order data provided in the above-mentioned embodiment, and will not be elaborated here.
[0137] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned work order data classification method when executed by a processor.
[0138] The computer program product provided by this application can solve the technical problem of how to efficiently and accurately classify massive amounts of work order data. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the work order data classification method provided by the above embodiment, which will not be repeated here.
[0139] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A method for classifying work order data, characterized in that: The classification method of work order data comprises the following steps: Roughly classify the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data; The various types of first work order data are subdivided by using a preset language model to obtain multiple types of second work order data; The various types of second work order data are classified according to keywords corresponding to the various types of second work order data to obtain multiple types of third work order data.
2. The method for classifying work order data according to claim 1, characterized in that: Before the step of sub-classifying each type of first work order data by using a preset language model to obtain multiple types of second work order data, the step further includes: Classify the initial work order data to obtain multiple types of training data; Selecting training samples from the multiple types of training data; The feature representation layer and the classification layer in the initial language model are trained through the training samples to obtain a preset language model, and the initial language model is a model for performing knowledge distillation on the BERT model.
3. The method for classifying work order data according to claim 2, characterized in that: The step of selecting training samples from the multiple types of training data specifically includes: Selecting initial positive samples from training data of the same category, and performing data enhancement on the initial positive samples to obtain enhanced positive samples; Determine the target category corresponding to the initial positive sample, and select remaining training data corresponding to remaining categories except the target category from the multiple categories of training data; Calculating the similarity between the positive sample and the remaining training data by a text similarity algorithm, and selecting a negative sample from the remaining training data according to the similarity; A training sample is determined according to the enhanced positive sample and the negative sample.
4. The method for classifying work order data according to claim 2, characterized in that: The step of training the feature representation layer and the classification layer in the initial language model by using the training samples to obtain the preset language model specifically includes: Selecting target positive samples and target negative samples corresponding to the original samples from the training samples; Determine an original vector corresponding to the original sample, a positive vector corresponding to the target positive sample, and a negative vector corresponding to the target negative sample; Calculating a first cosine similarity between the original vector and the positive vector, and calculating a second cosine similarity between the original vector and a random Gaussian noise vector; An objective function is constructed according to the first cosine similarity, the temperature parameter, and the second cosine similarity, and a feature representation layer and a classification layer in an initial language model are trained based on the objective function to obtain a preset language model.
5. The method for classifying work order data according to claim 1, characterized in that: The step of classifying the various types of second work order data according to the keywords corresponding to the various types of second work order data to obtain multiple types of third work order data specifically includes: Extract keywords corresponding to work order titles and work order contents in various types of second work order data; Combining the keywords to obtain combined keywords; The various types of second work order data are classified respectively according to the combined keywords to obtain multiple types of third work order data.
6. The method for classifying work order data according to claim 5, characterized in that: After the step of classifying the various types of second work order data according to the combined keywords to obtain multiple types of third work order data, the method further includes: For each type of third work order data, a feature vector output by the preset language model is obtained, where the feature vector represents a feature vector obtained by concatenating the work order title and the work order content; The cosine similarity between the feature vectors is calculated, and the third work order data of each category is partitioned and stored according to the cosine similarity.
7. A device for classifying work order data, characterized in that: The work order data classification device comprises: A data rough classification module, used to roughly classify the complaint work order data according to the work order titles in the complaint work order data to obtain multiple categories of first work order data; A data sub-classification module is used to sub-classify various types of first work order data respectively through a preset language model to obtain multiple types of second work order data; The data classification module is used to classify each type of second work order data according to the keywords corresponding to each type of second work order data to obtain multiple types of third work order data.
8. A work order data classification device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for classifying work order data according to any one of claims 1 to 6.
9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the work order data classification method according to any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method for classifying work order data according to any one of claims 1 to 6 are implemented.
