Text processing method, device, readable storage medium and electronic device

Automatically classify work orders through the neural network model, the problems of low efficiency and insufficient accuracy of work order classification are solved, and efficient and accurate work order classification is achieved.

CN112800215BActive Publication Date: 2024-11-19BEIJING DA MI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110129551.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-29
Publication Date
2024-11-19
Estimated Expiration
2041-01-29

AI Technical Summary

Technical Problem

In the prior art, work order classification is inefficient and prone to errors, and manual classification is not accurate enough, resulting in the work order classification results being time-consuming and possible errors.

Method used

Using a neural network-based work ticket prediction model, train the topic prediction model and work ticket fine-tuning model by obtaining historical work ticket data, automatically determine the work ticket category of the pending text, and match or select based on the pre-classification results entered by the user to improve Classification accuracy.

Benefits of technology

It improves the accuracy and efficiency of work order classification, reduces manual errors, and can adapt to changes in the work order classification hierarchical structure.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112800215B_ABST
    Figure CN112800215B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention provide a text processing method, device, readable storage medium and electronic device. The technical solution of the embodiments of the present invention obtains a text to be processed, inputs the text to be processed into a trained work order prediction model to determine at least one first work order category whose relevance to the text to be processed meets a preset condition, determines a designated category in the at least one first work order category, and distributes the text to be processed according to the designated category, thereby improving the accuracy of work order classification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a text processing method, device, readable storage medium and electronic device. Background Art

[0002] Entities that provide services based on the Internet need to provide customers with a large amount of customer service. Customer service needs are usually circulated and managed through work orders. In the business system, a large number of work orders are generated every day. As the business adjusts, the hierarchical structure of work orders is constantly changing. Existing technical solutions usually classify work orders manually. After the work order hierarchy structure is updated, the staff needs to remember the updated work order hierarchy structure and find the work order classification result based on the work order hierarchy structure. When the staff needs to submit a work order, it often takes a long time to find the correct work order classification result. There is also the problem of inaccurate classification through manual classification. Summary of the invention

[0003] In view of this, in order to solve the problem of low efficiency and easy errors in manual classification, the purpose of the embodiments of the present invention is to provide a text processing method, device, readable storage medium and electronic device.

[0004] In a first aspect, an embodiment of the present invention provides a text processing method, comprising:

[0005] Get the text to be processed;

[0006] Determining at least one first work order category according to the text to be processed and the trained work order prediction model;

[0007] determining a designated category among the at least one first work order category; and

[0008] The text to be processed is distributed according to the specified category.

[0009] Preferably, the work order prediction model includes a topic prediction model and a work order fine-tuning model;

[0010] Determining at least one first work order category according to the to-be-processed text and the trained work order prediction model includes:

[0011] Extracting description information from the text to be processed;

[0012] Inputting the description information into a topic prediction model to predict a first topic;

[0013] The first topic and the description information are combined and input into the work order fine-tuning model to obtain the at least one first work order category.

[0014] Preferably, determining a designated category in the at least one first work order category comprises:

[0015] Obtain the second work order category pre-entered by the user;

[0016] In response to the second work order category matching at least one of the first work order categories, determining the second work order category as the designated category; and

[0017] In response to the second work order category not matching any of the first work order categories, each of the first work order categories is sent to the user end, and a specified category specified by the user end in each of the first work order categories is obtained.

[0018] Preferably, the work order category is represented by a digital label, and the digital label is determined by the following steps:

[0019] Obtaining a hierarchical relationship of historical work order classifications, and obtaining a plurality of text labels according to the hierarchical relationship of historical work order classifications, wherein the text labels can represent classification results of the historical work orders;

[0020] Map text labels to numeric labels.

[0021] Preferably, before obtaining the text to be processed, the method includes:

[0022] Train the work order prediction model based on historical work order data.

[0023] Preferably, the training of the work order prediction model according to the historical work order data includes:

[0024] Acquire a pre-training sample data set and a fine-tuning sample data set according to the historical work order data, wherein the fine-tuning sample data set is obtained by modifying the pre-training sample data set according to the historical work order data;

[0025] Pre-training the neural network model using the pre-training sample data set;

[0026] The pre-trained neural network model is fine-tuned using the fine-tuning sample data set to obtain the work order prediction model.

[0027] Preferably, the step of acquiring a pre-training sample data set and a fine-tuning sample data set according to historical work order data includes:

[0028] Acquire the historical work order data, wherein the historical work order data includes description samples, manually annotated subject samples, and label samples;

[0029] Obtain pre-training sample data based on description samples and topic samples;

[0030] Obtaining the pre-training sample data set according to a plurality of pre-training sample data;

[0031] The description information and topic information in the pre-training sample data set are combined and then stop words are removed to obtain the first description information;

[0032] Obtaining initial fine-tuning sample data according to the first description information;

[0033] Obtaining an initial fine-tuning sample data set according to multiple pieces of initial fine-tuning sample data;

[0034] A fine-tuning sample dataset is obtained based on the initial fine-tuning sample dataset.

[0035] Preferably, obtaining a fine-tuning sample data set according to the initial fine-tuning sample data set comprises:

[0036] Dividing the initial fine-tuning sample data set into an initial fine-tuning training set, an initial fine-tuning validation set, and an initial fine-tuning test set according to a predetermined ratio;

[0037] Counting the number of samples of each label in the initial fine-tuning sample data set;

[0038] Marking the initial fine-tuning sample data that is less than a predetermined number as a long-tail data set;

[0039] The fine-tuning sample dataset is obtained by updating the initial fine-tuning sample dataset with the long-tail dataset.

[0040] In a second aspect, an embodiment of the present invention provides a text processing device, including:

[0041] A receiving device, used for acquiring the text to be processed;

[0042] A work order category prediction device, used to determine at least one first work order category whose relevance to the text to be processed meets a preset condition according to the text to be processed and a trained work order prediction model;

[0043] Determining means, for determining a designated category in the at least one first work order category;

[0044] The submitting device distributes the text to be processed according to the specified category.

[0045] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium for storing computer program instructions, wherein the computer program instructions are used to implement any of the above methods when executed by the processor.

[0046] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement any of the methods described above.

[0047] The technical solution of the embodiment of the present invention obtains the text to be processed, inputs the text to be processed into the trained work order prediction model to determine at least one first work order category whose relevance to the text to be processed meets the preset conditions, determines a designated category in the at least one first work order category, and submits the text to be processed to the business represented by the designated category. Thus, the accuracy of the work order classification result is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0049] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of a work order prediction model according to an embodiment of the present invention;

[0051] Figure 3 is a flow chart of training a work order prediction model according to an embodiment of the present invention;

[0052] Figure 4 is a flow chart of obtaining a pre-training sample data set and a fine-tuning sample data set according to an embodiment of the present invention;

[0053] Figure 5 is a flow chart of obtaining a fine-tuning sample data set according to an embodiment of the present invention;

[0054] Figure 6 is a flowchart of text processing according to an embodiment of the present invention;

[0055] Figure 7 is a flow chart of predicting a first work order category according to an embodiment of the present invention;

[0056] Figure 8 is a flow chart of obtaining a specified category according to an embodiment of the present invention;

[0057] Fig. 9 is a schematic diagram of mapping text labels and digital labels in an embodiment of the present invention;

[0058] Fig.10 is a flow chart of determining a digital tag according to an embodiment of the present invention;

[0059] Fig.11 It is a schematic diagram of the hierarchical relationship of historical work orders in an embodiment of the present invention;

[0060] Fig.12 is a schematic diagram of a text processing device according to an embodiment of the present invention;

[0061] Fig.13 is a schematic diagram of a text processing system according to an embodiment of the present invention;

[0062] Fig.14 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the detailed description of the present invention below, some specific details are described in detail. It is possible for a person skilled in the art to fully understand the present invention without the description of these details. In order to avoid confusing the essence of the present invention, known methods, processes, flows, components and circuits are not described in detail.

[0064] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0065] Unless the context clearly requires otherwise, words such as “include”, “including” and the like in the specification should be interpreted as including rather than being exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0066] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0067] In order to solve the problem of low efficiency and easy error in manual classification, an embodiment of the present invention provides a text processing method. The technical solution of the embodiment of the present invention first uses historical work order data to train a work order prediction model, and then uses the trained work order prediction model to assist manual classification of pending work orders.

[0068] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present invention.

[0069] The implementation environment includes a terminal 100 and a server 200, wherein the terminal 100 and the server 200 perform data communication via a communication network. Optionally, the communication network may be a wired network or a wireless network, and the communication network may be at least one of a local area network, a metropolitan area network, and a wide area network.

[0070] The terminal 100 has a text processing application installed therein, and the application may be a work order entry application, a text classification application, a work order classification application, a work order distribution application, etc., which is not limited in the embodiment of the present invention.

[0071] Optionally, the terminal 100 may be a mobile terminal such as a tablet computer, a laptop computer, or a personal computer, a workstation, or the like, which is not limited in the embodiment of the present invention.

[0072] The server 200 can be implemented as a single server or a server cluster composed of a group of servers, which can be a physical server or a cloud server. In an optional implementation, the server 200 is a background server of an application in the terminal 100 .

[0073] Specifically, the business personnel enter the problem reflected by the customer into the descriptive information through the terminal 100, wherein the descriptive information is an unclassified work order to be processed, and the business personnel enter the possible work order classification result of the problem into the second work order category, wherein the second work order category is a pre-judgment of the work order classification result by the business personnel. The server 200 predicts the descriptive information according to the trained work order prediction model and gives at least one first work order category. The designated category is determined by comparing the first work order category with the second work order category, and the descriptive information is distributed according to the designated category. Thus, the server 200 creates the work order to be classified according to the descriptive information and distributes the work order to be classified according to the designated category. The technical solution of the embodiment of the present invention assists the business personnel in classifying the descriptive information through the second work order category predicted by the server, thereby improving the accuracy of work order classification.

[0074] Reference Figure 1 In order to make the at least one first work order category predicted by the work order prediction model more accurate, the work order prediction model of an embodiment of the present invention includes a topic prediction model and a work order fine-tuning model.

[0075] Figure 2 It is a schematic diagram of a work order prediction model according to an embodiment of the present invention.

[0076] Reference Figure 2 The work order prediction model 210 of the embodiment of the present invention includes a topic prediction model 240 and a work order fine-tuning model 260 .

[0077] The topic prediction model 240 can predict the first topic 320 according to the description information 110. The description information 110 and the first topic 320 are combined and input into the work order fine-tuning model 260, and the work order fine-tuning model 260 predicts the first work order category 300 that meets the conditions.

[0078] In order to predict at least one first work order category through the work order prediction model 210, the work order prediction model 210 of the embodiment of the present invention is obtained by pre-training a neural network.

[0079] Preferably, the neural network of the embodiment of the present invention adopts BERT (Bidirectional Encoder Representation from Transformers, bidirectional encoding neural network model).

[0080] Specifically, the embodiment of the present invention trains the work order prediction model 210 based on historical work order data.

[0081] Figure 3 It is a flow chart of training a work order prediction model according to an embodiment of the present invention.

[0082] Reference Figure 3 , training the work order prediction model includes steps 1100 to 1300.

[0083] Step 1100: Acquire a pre-training sample data set and a fine-tuning sample data set based on historical work order data.

[0084] Figure 4 It is a flow chart of obtaining a pre-training sample data set and a fine-tuning sample data set according to an embodiment of the present invention.

[0085] Specifically, in an optional implementation, referring to Figure 4 ,The pre-training sample dataset and fine-tuning sample dataset are obtained through the following steps.

[0086] Step 1110: Acquire the historical work order data. The historical work order data includes description samples, manually annotated subject samples, and label samples.

[0087] Specifically, each piece of historical work order data includes a description sample, a manually annotated topic sample, and a manually annotated label sample.

[0088] The description sample is a specific description of the problem reported by the customer, recorded in text form. The manually annotated topic sample is a summary of the problem reported by the customer. The manually annotated label sample is the classification result of the problem reported by the customer. According to the classification result, the problem reported by the customer can be submitted to a specific business department or person. For example, after submitting it to a specific person or department, the person or department with corresponding authority can receive the work order and follow up and handle the problem reported by the customer in the work order.

[0089] Step 1120, obtaining pre-training sample data according to the description sample and the theme sample.

[0090] Specifically, the description sample and manually labeled topic sample of each historical work order data are extracted to form a pair of pre-training sample data.

[0091] Step 1130: Obtain the pre-training sample data set according to the plurality of pre-training sample data.

[0092] Specifically, a predetermined number of pre-training sample data is used to form a pre-training sample data set. In an embodiment of the present invention, the predetermined number can be obtained periodically. For example, historical work order data in the past year is obtained, and a pre-training sample data set is obtained based on the historical work order data in the past year. A predetermined number of historical work order data can also be obtained by numerical value. For example, first, 200,000 historical work order data are obtained, and then a pre-training sample data set is obtained based on the 200,000 historical work order data.

[0093] The embodiment of the present invention can obtain a pre-training sample data set through steps 1110 to 1130, so as to obtain a topic prediction model after training the BERT model. Specifically, the pre-training data set is composed of description samples and manually annotated topic samples. The topic prediction model is obtained after the BERT model is trained with the pre-training data set. Subsequently, a fine-tuning sample data set is obtained based on the pre-training sample data set, and a fine-tuning prediction model is obtained based on the fine-tuning sample data set.

[0094] Step 1140: merge the description information and topic information in the pre-training sample data set and remove stop words to obtain first description information.

[0095] Stop words are irrelevant or interfering information that sometimes affect the classification results of the model. In order to make the prediction results of the work order classification model more accurate, the description information and topic information in the pre-training sample dataset are combined and the stop words are removed to obtain the first description information.

[0096] Step 1150: Obtain initial fine-tuning sample data according to the first description information and the label sample.

[0097] In an optional implementation manner, initial fine-tuning sample data is obtained according to the first description information and the label sample.

[0098] Step 1160 , obtaining an initial fine-tuning sample data set according to the plurality of initial fine-tuning sample data.

[0099] Step 1170 , obtaining a fine-tuning sample data set according to the initial fine-tuning sample data set.

[0100] The embodiment of the present invention obtains a pre-training sample data set and a fine-tuning sample data set through historical work order data, so that the bidirectional encoding neural network model BERT can be pre-trained through the pre-training sample data to obtain a topic prediction model.

[0101] The embodiment of the present invention obtains a fine-tuning sample data set based on the initial fine-tuning sample data set. Thus, the trained bidirectional encoding neural network model BERT can be fine-tuned to obtain a fine-tuning prediction model. The topic prediction model and the fine-tuning prediction model constitute a work order classification model.

[0102] Specifically, in order to obtain a fine-tuning sample data set, step 1170 includes four sub-steps. Figure 5 , which are steps 1171 to 1174 respectively.

[0103] Step 1171 , dividing the initial fine-tuning sample data set into an initial fine-tuning training set, an initial fine-tuning validation set, and an initial fine-tuning test set according to a predetermined ratio.

[0104] Preferably, the initial fine-tuning sample data set may be divided into an initial fine-tuning training set, an initial fine-tuning validation set, and an initial fine-tuning test set in a ratio of 7:1.5:1.5. The embodiment of the present invention does not limit the predetermined ratio.

[0105] Step 1172: Count the number of each initial fine-tuning sample in the initial fine-tuning sample data set.

[0106] Step 1173, marking the initial fine-tuning sample data that is less than a predetermined amount as a long-tail data set.

[0107] Specifically, in the historical work order data, the number of some label samples is very small. If the initial fine-tuning sample data set is directly divided into the initial fine-tuning training set, the initial fine-tuning verification set or the initial fine-tuning test set according to a predetermined ratio, some label samples may be divided into the verification set. For these small number of label samples, these data cannot be used to fine-tune the bidirectional encoding neural network model BERT, so that the first description samples corresponding to these label samples have no chance to participate in the training process. This will result in low accuracy in the work order classification results of the pre-trained bidirectional encoding neural network model BERT. For this reason, an embodiment of the present invention marks the initial fine-tuning sample data that is less than a predetermined number as a long-tail data set.

[0108] Step 1174: Use the long-tail dataset to update the initial fine-tuning sample dataset to obtain a fine-tuning sample dataset.

[0109] Specifically, the embodiment of the present invention divides the long-tail data set into the fine-tuning training set, so that a small amount of long-tail data can be used to train the bidirectional encoding neural network model BERT, thereby improving the accuracy of prediction.

[0110] To obtain a topic prediction model, execute step 1200.

[0111] Step 1200: pre-training the neural network model using the pre-training sample data set.

[0112] Preferably, the neural network of the embodiment of the present invention adopts BERT (Bidirectional Encoder Representation from Transformers, bidirectional encoding neural network model).

[0113] Specifically, the bidirectional encoding neural network model BERT is pre-trained through a pre-training sample dataset to obtain a topic prediction model.

[0114] To obtain a work order fine-tuning model, step 1300 is performed.

[0115] Step 1300: fine-tune the pre-trained bidirectional encoding neural network model BERT using the fine-tuning sample data set to obtain a work order fine-tuning model.

[0116] The embodiment of the present invention combines the topic prediction model and the work order fine-tuning model to obtain the work order prediction model.

[0117] The embodiment of the present invention obtains a pre-trained sample data set and a fine-tuning sample data set through historical work order data, pre-trains the bidirectional encoding neural network model BERT through the pre-trained sample data set to obtain a topic prediction model, fine-tunes the pre-trained bidirectional encoding neural network model BERT through the fine-tuning sample data set to obtain a work order fine-tuning model, and combines the topic prediction model and the work order fine-tuning model to obtain a work order prediction model. Therefore, the work order prediction model of the embodiment of the present invention can predict the work order classification results, thereby improving the efficiency of work order classification.

[0118] Figure 6 4 is a flowchart of text processing according to an embodiment of the present invention.

[0119] The text processing flow of the embodiment of the present invention includes steps 1000 to 4000.

[0120] Step 1000, obtaining the text to be processed;

[0121] The text to be processed includes description information and a second work order category, wherein the description information is the problem reflected by the customer, and the second work order category is a manual prejudgment of the work order classification result.

[0122] Step 2000: Determine at least one first work order category whose relevance to the text to be processed meets a preset condition based on the text to be processed and a trained work order prediction model.

[0123] Specifically, the description information is input into the work order prediction model, and N first work order categories are obtained through the work order prediction model. In an optional implementation, N is a positive integer, and the value of N is greater than or equal to 1.

[0124] Figure 7This is a flow chart of predicting the first work order category according to an embodiment of the present invention.

[0125] Specifically, refer to Figure 7 , step 2000 includes three sub-steps, namely step 2100 to step 2300.

[0126] Step 2100, extracting description information from the text to be processed;

[0127] Step 2200, inputting the description information into a topic prediction model to predict a first topic;

[0128] Step 2300: Combine the first topic and the description information and input them into the work order fine-tuning model to obtain the at least one first work order category that meets the preset conditions.

[0129] Specifically, the first topic is combined with the description information and then stop words are removed to obtain first description information, and the first description information is input into the work order fine-tuning model. At least one first work order category is predicted by the work order fine-tuning model.

[0130] Step 3000: Determine a designated category in the at least one first work order category.

[0131] Figure 8 This is a flowchart of obtaining a specified category according to an embodiment of the present invention.

[0132] The second work order category is a manual pre-judgment of the work order classification result. Since it is unknown whether the second work order category is accurate, the second work order category is compared with at least one first work order category. Specifically, refer to Figure 8 , step 3000 includes three sub-steps, namely step 3100 to step 3300.

[0133] Step 3100, obtaining the second work order category pre-entered by the user.

[0134] Step 3200: In response to the second work order category matching the first work order category, determine the second work order category as the designated category.

[0135] That is, the second work order category submitted by the user is compared with one or more first work order categories predicted by the server. The second work order category submitted by the user is compared with a first work order category predicted by the server. If the second work order category submitted by the user is the same as the first work order category, the second work order category is determined as the designated category. Alternatively, the second work order category submitted by the user is compared with multiple first work order categories predicted by the server. If the second work order category submitted by the user is the same as any first work order category predicted by the server, the second work order category is determined as the designated category.

[0136] Step 3300: In response to the second work order category not matching the first work order categories, the first work order categories are sent to the user end, and the specified category specified by the user end in the first work order categories is obtained.

[0137] That is, if the second work order category submitted by the user does not match one or more first work order categories predicted by the server, it means that the second work order category submitted by the user is incorrect. Then, each first work order category is sent to the user for selection.

[0138] Specifically, the second work order category is compared with a first work order category. If the second work order category is different from the first work order category, the first work order category predicted by the server is determined as the designated category. Alternatively, the second work order category submitted by the user is compared with multiple first work order categories predicted by the server. If the second work order category submitted by the user is different from any of the first work order categories predicted by the server, it means that the second work order category submitted by the user is wrong, and the multiple first work order categories are sent to the terminal for the user to select. A first work order category specified by the user from the multiple first work order categories is determined as the designated category.

[0139] Step 4000, distributing the text to be processed according to the specified category.

[0140] Distribute the text to be processed according to the specified category. Specifically, extract the descriptive information in the text to be processed, and distribute the descriptive information according to the specified category. Thus, the descriptive information can be accurately classified and submitted to the corresponding personnel or departments.

[0141] The technical solution of the embodiment of the present invention obtains the text to be processed and inputs the text to be processed into a trained work order prediction model, thereby determining at least one first work order category whose relevance to the text to be processed meets the preset conditions. Since the first work order category is obtained through prediction, it is more efficient and can avoid errors caused by human factors. A designated category is determined in the at least one first work order category, and the text to be processed is distributed according to the designated category, so that the accuracy of the work order classification result is higher.

[0142] Preferably, in order to enable the text processing method of the embodiment of the present invention to adapt to the changes in the hierarchical relationship of work order classification, the first work order category implemented in the present invention is represented by a digital label, and the digital label is determined by the following steps: obtaining the hierarchical relationship of historical work order classification, and obtaining multiple text labels according to the hierarchical relationship of historical work order classification, and the text labels can represent the classification results of historical work orders; mapping the text labels to digital labels.

[0143] Fig. 9 It is a schematic diagram of mapping text labels and digital labels according to an embodiment of the present invention.

[0144] Reference Fig. 9 , extract text labels 120 according to the hierarchical relationship of the historical work order classification. The text labels 120 can represent the classification results of the historical work orders.

[0145] A digital label 130 is created, the digital label being able to characterize the first category of the work order.

[0146] The text label 120 is mapped to the digital label 130 , and a mapping relationship 140 is created.

[0147] The specified category 310 is obtained. In an optional implementation, the specified category 310 is any one of the digital tags 130. In another optional implementation, the specified category 310 is any one of the text tags 120.

[0148] Obtain the to-be-processed text 110. The to-be-processed text 110 is a work order to be classified.

[0149] The to-be-processed text 110 is distributed according to the designated category 310. According to the mapping relationship 140, the to-be-processed text 110 can be distributed to the personnel or departments corresponding to the updated work order classification. Thus, when the work order classification of the embodiment of the present invention changes, the to-be-processed text 110 can still be distributed to the personnel or departments corresponding to the updated work order classification according to the mapping relationship 140.

[0150] Fig.10 The figure is a flow chart of determining a digital tag according to an embodiment of the present invention.

[0151] The digital tag of the embodiment of the present invention is determined through steps 5000 and 6000.

[0152] Step 5000, obtaining the hierarchical relationship of historical work order classifications, and obtaining a plurality of text labels according to the hierarchical relationship of historical work order classifications, wherein the text labels can represent the classification results of the historical work orders.

[0153] Fig.11 It is a schematic diagram of the hierarchical relationship of historical work orders according to an embodiment of the present invention.

[0154] Reference Fig.11 The hierarchical relationship of historical work order classification adopts a tree-like hierarchical relationship, so that the categories of work orders can be described hierarchically. The tree-like hierarchical relationship can help business personnel quickly locate the business represented by the leaf node. Specifically, when a business person receives a call from a customer, the business person records the customer's request and forms a work order. The business person submits the work order to the corresponding person or department.

[0155] Specifically, the business personnel know the hierarchical relationship of the work order classification in advance, and record the customer's demands in text form according to the specific content of the customer's demands to form a work order. The business personnel submit the work order to the corresponding personnel or department through subjective judgment. For example, the business personnel submits the customer's question 1 to the personnel or department represented by the first-level label 1-second-level label 1-third-level label 1-fourth-level label 1. The business personnel submits the customer's question 2 to the personnel or department represented by the first-level label 1-second-level label 1-third-level label 1-fourth-level label 2.

[0156] In the work order classification hierarchy expressed by the tree structure, each trunk node and leaf node has a corresponding text label (the text label is a description of the business). The text labels of the trunk nodes are different. The text labels of the leaf nodes can be the same or different. For example, the text labels of level 4 label 1 and level 4 label 3 can be the same. Specifically, the text label of level 4 label 1 is "other", and the text label of level 4 label 3 can also be "other". Level 4 label 1 belongs to the finance department, and level 4 label 3 belongs to the technical department. That is to say, when the text labels of the leaf nodes are the same, the personnel or departments represented by different leaf nodes are essentially different. When the text labels of the leaf nodes are different, the personnel or departments represented by different leaf nodes are essentially different.

[0157] In the hierarchical relationship of work order classification, "fourth-level label 1", "fourth-level label 2", "fourth-level label 3", "fourth-level label 4", "secondary label 2" and "primary label 3" are all text labels, and the personnel or departments they represent are actually different.

[0158] When the organizational structure is adjusted, the personnel or departments are adjusted accordingly, and the hierarchical relationship of the leaf nodes also changes accordingly. For example, the leaf node secondary label 2 is adjusted to the leaf node primary label 3. Although the path of the leaf node has changed, the personnel or department represented by the node has not changed. In order to enable the work order of the embodiment of the present invention to be accurately submitted to the corresponding personnel or department after the organizational structure is adjusted, the embodiment of the present invention maps the text label with the digital label.

[0159] Step 6000, mapping text labels to digital labels.

[0160] The text label is mapped to a numeric label, wherein the numeric label can characterize the first category of the work order.

[0161] In an optional implementation, the hierarchical relationship of the historical work order classification is obtained, and the text label is obtained according to the hierarchical relationship of the historical work order classification. All text labels are traversed according to the leaf nodes, and the text labels are mapped to the digital labels. For details, refer to Table 1:

[0162]

[0163] Table 1

[0164] It should be noted that the "+" in Table 1 represents a connection relationship of character strings, and the digital labels in Table 1 are only schematic representations, and different digital labels can be set in practice in combination with specific services. The embodiment of the present invention does not limit the representation of digital labels.

[0165] By mapping text labels with digital labels, the hierarchical relationship of text labels changes, but the corresponding digital labels remain unchanged. For example, the leaf node secondary label 2 is adjusted to the leaf node primary label 3. Then the text label is adjusted from "secondary label 2 + primary label 1" to "primary label 3". The adjusted primary label 3 still corresponds to the digital label "104". According to the digital label, the business is still submitted to the person or department corresponding to the digital label "104" in Table 1.

[0166] The technical solution of the embodiment of the present invention obtains the hierarchical relationship of the historical work order classification, and obtains multiple text labels according to the hierarchical relationship of the historical work order classification, and the text labels can represent the classification results of the historical work order. The text labels are mapped to digital labels, and the digital labels can represent the first category of the work order. Therefore, when the hierarchical relationship of the work order classification changes, the work order can still be submitted to the corresponding person or department according to the mapping relationship.

[0167] Fig.12 Schematic diagram of a text processing device according to an embodiment of the present invention.

[0168] The text processing device of the embodiment of the present invention comprises a receiving device 11, a work order category prediction device 12, a determination device 13 and a submission device 14. The text processing device of the embodiment of the present invention can be deployed in Figure 2 On the server 200 shown.

[0169] The receiving device 11 is used to obtain the text to be processed.

[0170] The work order category prediction device 12 is used to determine at least one first work order category whose relevance to the text to be processed meets a preset condition based on the text to be processed and the trained work order prediction model 210.

[0171] The determining device 13 is used to determine a designated category in the at least one first work order category.

[0172] The submitting device 14 is used to distribute the text to be processed according to the specified category, that is, to submit the text to be processed to the person or department corresponding to the specified category.

[0173] Fig.13 Schematic diagram of a text processing system according to an embodiment of the present invention.

[0174] An example of a text processing system provided in an embodiment of the present invention being used to assist business personnel in classifying work orders is used for description.

[0175] The text processing system of the embodiment of the present invention includes a terminal 100 and a server 200. The terminal 100 may be multiple or one.

[0176] When a business person receives a problem reflected by a customer through the customer service hotline, the business person saves the problem reflected by the customer as description information 110 through the terminal 100, wherein the description information 110 is an unclassified work order to be processed, and the business person enters the possible work order classification result of the problem into the second work order category 120, wherein the second work order category 120 is a prejudgment of the work order classification result by the business person. Due to the lack of work experience of the business person or the adjustment of the hierarchical relationship of the work order classification, the second work order category 120 submitted by the business person may be inaccurate. For this reason, the text processing system of the embodiment of the present invention assists the business person in classifying the description information 110 through the server 200.

[0177] Specifically, the business personnel submit the description information and the second work order category to the server 200 through the terminal 100, so that the problem represented by the description information 110 can be accurately submitted to the business corresponding to the work order classification result. The server 200 predicts N first work order categories 300 that meet the predetermined conditions with the description information 110 through the trained work order prediction model 210, and compares whether the second work order category 120 submitted by the business personnel matches the predicted N first work order categories 300. N is a positive integer greater than or equal to 1, and the embodiment of the present invention does not limit the specific value of N.

[0178] Specifically, the server 200 receives the description information 110 and the second work order category 120. The server 200 uses the description information 110 and the second work order category 120 as text to be processed. The server 200 inputs the description information 110 into the work order prediction model 210, and obtains N first work order categories 300 through the work order prediction model 210. Among them, N is a positive integer. The N first work order categories are the N work order classification results predicted by the work order prediction model 210 according to predetermined conditions. The predetermined condition of an embodiment of the present invention may be a threshold value of correlation. For example, if the correlation is greater than 90%, N first work order categories with a correlation greater than 90% are output. The preset condition may also be a given numerical value. For example, if N is set to 5, it is sorted by correlation and the 5 first work order categories with the highest correlation are given.

[0179] The following description assumes that N is equal to 5. Specifically, the server 200 compares the five first work order categories 300 with the second work order category 120 submitted by the terminal 100 .

[0180] If the second work order category 120 is any one of the five first work order categories 300 , it means that the classification result determined by the business personnel subjectively is accurate. Then the second work order category 120 is determined as the designated category 310 .

[0181] If the second work order category 120 is not any one of the five first work order categories 300, it means that the second work order category 120 determined by the business personnel through subjective judgment is wrong. In this case, the server feeds back the five first work order categories 300 to the terminal 100 for the business personnel to select, and the business personnel selects one of the first work order categories. The server 200 determines the first work order category selected by the business personnel as the designated category 310.

[0182] The server 200 distributes the description information 110 according to the specified category 310. Thus, a work order is formed and submitted to the corresponding personnel or department.

[0183] In the embodiment of the present invention, the descriptive information and the second work order category are inputted through the terminal, and the second work order category is a pre-judgment of the work order classification result by the business personnel. The server obtains the descriptive information and the second work order category on the terminal, and the server inputs the descriptive information into the trained work order prediction model 210, predicts N first work order categories with the highest relevance, and compares the second work order category submitted by the terminal with the N first work order categories to determine the designated category. The server distributes the descriptive information to the corresponding personnel or departments according to the designated category. In the embodiment of the present invention, the server assists manual work order classification, thereby improving the accuracy of work order classification.

[0184] Fig.14 is a schematic diagram of an electronic device according to an embodiment of the present invention.

[0185] like Fig.14 The electronic device 14 shown includes a general hardware structure, which at least includes a processor 141 and a memory 142. The processor 141 and the memory 142 are connected via a bus 143. The memory 142 is suitable for storing instructions or programs executable by the processor 141. The processor 141 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 141 executes the instructions stored in the memory 142, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 143 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 144 and the display device and the input / output (I / O) device 145. The input / output (I / O) device 145 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 145 is connected to the system through an input / output (I / O) controller 146.

[0186] It should be understood by those skilled in the art that the embodiments of the present application may provide methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0188] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0189] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0190] Another embodiment of the present invention relates to a non-volatile readable storage medium for storing a computer-readable program, wherein the computer-readable program is used to enable a computer to execute some or all of the above method embodiments.

[0191] That is, those skilled in the art can understand that all or part of the steps in the method for implementing the above embodiments can be completed by specifying relevant hardware through a program, and the program is stored in a readable storage medium, including a number of instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0192] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A text processing method, characterized in that: The method comprises: Obtaining a text to be processed, the text to be processed including description information and a second work order category, the description information is a problem reflected by a customer, and the second work order category is a manual pre-judgment of a work order classification result; Determine, according to the text to be processed and the trained work order prediction model, at least one first work order category whose relevance to the text to be processed meets a preset condition; Determining a designated category from the at least one first work order category according to a second work order category pre-input by a user terminal; and Distributing the text to be processed according to the specified category; The first work order category that meets the preset conditions includes: The first work order category whose corresponding relevance is greater than a set relevance threshold; or The first work order category that is in the top N positions in the relevance ranking, where N is a given value; Wherein, determining the designated category in the at least one first work order category includes: In response to the second work order category matching at least one of the first work order categories, determining the second work order category as the designated category; and In response to the second work order category not matching any of the first work order categories, sending each of the first work order categories to a user terminal, and obtaining a designated category designated by the user terminal in each of the first work order categories; Wherein, before obtaining the text to be processed, the method includes: Train the work order prediction model based on historical work order data; Wherein, the training of the work order prediction model according to the historical work order data includes: Acquire a pre-training sample data set and a fine-tuning sample data set according to the historical work order data, wherein the fine-tuning sample data set is obtained by modifying the pre-training sample data set according to the historical work order data; Pre-training the neural network model using the pre-training sample data set; Fine-tune the pre-trained neural network model using the fine-tuning sample data set to obtain the work order prediction model; The step of obtaining a pre-training sample data set and a fine-tuning sample data set according to historical work order data includes: Acquire the historical work order data, wherein the historical work order data includes description samples, manually annotated subject samples, and label samples; Obtain pre-training sample data based on description samples and topic samples; Obtaining the pre-training sample data set according to a plurality of pre-training sample data; The description information and topic information in the pre-training sample data set are combined and then stop words are removed to obtain the first description information; Obtaining initial fine-tuning sample data according to the first description information and the label sample; Obtaining an initial fine-tuning sample data set according to multiple pieces of initial fine-tuning sample data; Obtain a fine-tuning sample dataset based on an initial fine-tuning sample dataset; The step of obtaining a fine-tuning sample data set according to the initial fine-tuning sample data set includes: Dividing the initial fine-tuning sample data set into an initial fine-tuning training set, an initial fine-tuning validation set, and an initial fine-tuning test set according to a predetermined ratio; Counting the number of each initial fine-tuning sample in the initial fine-tuning sample data set; Marking the initial fine-tuning sample data that is less than a predetermined number as a long-tail data set; The fine-tuning sample dataset is obtained by updating the initial fine-tuning sample dataset with the long-tail dataset.

2. The text processing method according to claim 1, characterized in that: The work order prediction model includes a topic prediction model and a work order fine-tuning model; Determining at least one first work order category according to the to-be-processed text and the trained work order prediction model includes: Extracting description information from the text to be processed; Inputting the description information into a topic prediction model to predict a first topic; The first topic and the description information are combined and input into the work order fine-tuning model to obtain the at least one first work order category.

3. The text processing method according to claim 1, characterized in that: The first work order category is represented by a digital label, and the digital label is determined by the following steps: Obtaining a hierarchical relationship of historical work order classifications, and obtaining a plurality of text labels according to the hierarchical relationship of historical work order classifications, wherein the text labels can represent classification results of the historical work orders; Map text labels to numeric labels.

4. A text processing device, characterized in that: The text processing device comprises: A receiving device, used to obtain a text to be processed, wherein the text to be processed includes description information and a second work order category, wherein the description information is a problem reflected by a customer, and the second work order category is a manual pre-judgment of a work order classification result; A work order category prediction device, used to determine at least one first work order category whose relevance to the text to be processed meets a preset condition according to the text to be processed and a trained work order prediction model; Determining means, for determining a designated category from the at least one first work order category according to a second work order category pre-input by a user terminal; A submitting device for distributing the text to be processed according to the specified category; The first work order category that meets the preset conditions includes: The first work order category whose corresponding relevance is greater than a set relevance threshold; or The first work order category that is in the top N positions in the relevance ranking, where N is a given value; Wherein, determining the designated category in the at least one first work order category includes: In response to the second work order category matching at least one of the first work order categories, determining the second work order category as the designated category; and In response to the second work order category not matching any of the first work order categories, sending each of the first work order categories to a user terminal, and obtaining a designated category designated by the user terminal in each of the first work order categories; Wherein, the text processing device further includes: A training device, used for training a work order prediction model based on historical work order data; Wherein, the training of the work order prediction model according to the historical work order data includes: Acquire a pre-training sample data set and a fine-tuning sample data set according to the historical work order data, wherein the fine-tuning sample data set is obtained by modifying the pre-training sample data set according to the historical work order data; Pre-training the neural network model using the pre-training sample data set; Fine-tune the pre-trained neural network model using the fine-tuning sample data set to obtain the work order prediction model; The step of obtaining a pre-training sample data set and a fine-tuning sample data set according to historical work order data includes: Acquire the historical work order data, wherein the historical work order data includes description samples, manually annotated subject samples, and label samples; Obtain pre-training sample data based on description samples and topic samples; Obtaining the pre-training sample data set according to a plurality of pre-training sample data; The description information and topic information in the pre-training sample data set are combined and then stop words are removed to obtain the first description information; Obtaining initial fine-tuning sample data according to the first description information and the label sample; Obtaining an initial fine-tuning sample data set according to multiple pieces of initial fine-tuning sample data; Obtain a fine-tuning sample dataset based on an initial fine-tuning sample dataset; The step of obtaining a fine-tuning sample data set according to the initial fine-tuning sample data set includes: Dividing the initial fine-tuning sample data set into an initial fine-tuning training set, an initial fine-tuning validation set, and an initial fine-tuning test set according to a predetermined ratio; Counting the number of each initial fine-tuning sample in the initial fine-tuning sample data set; Marking the initial fine-tuning sample data that is less than a predetermined number as a long-tail data set; The fine-tuning sample dataset is obtained by updating the initial fine-tuning sample dataset with the long-tail dataset.

5. A computer-readable storage medium for storing computer program instructions, characterized in that: The computer program instructions, when executed by a processor, implement the method according to any one of claims 1 to 3.

6. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 3.