Business work order classification model establishing method and device and classification method and device

Abstract

The invention relates to the technical field of data processing, and provides a business work order classification model establishing method and device and a classification method and device. The method comprises the following steps: acquiring historical business work order data and real-time business work order data; classifying the historical business work order data according to a preset business scene to obtain a plurality of levels of fixed business labels; performing combined traversal on the multiple levels of fixed service labels to obtain a combined traversal result; a corresponding first classification model is constructed according to historical business work order data by taking a fixed business label of each level in the combined traversal result as a training target; determining an activity service label corresponding to the real-time service work order data; constructing a corresponding second classification model according to the real-time business work order data by taking identification of the active business label as a training target; and splicing the first classification model and the second classification model to obtain a business work order classification model. Through the embodiment of the invention, the response efficiency of the business work order can be improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for establishing a business work order classification model, and a classification method and device. Background Art

[0002] As businesses in all walks of life become increasingly diversified and complex, customers are increasingly demanding service quality. Among the many service links, customer service personnel are in direct contact with customers, and their work quality directly affects the company's reputation and customer satisfaction. However, in actual work, due to the lack of an effective business work order classification management mechanism, business work orders are often roughly classified based on the staff's business experience, which cannot meet the diverse real-time customer needs, resulting in low business processing efficiency, incomplete problem solving and other problems, which affect the user experience. Therefore, there is an urgent need for a method to establish a business work order classification model. The established business work order classification model can realize dynamic, real-time and efficient classification of business work orders, improve the response efficiency of business work orders, and improve user experience. Summary of the invention

[0003] In view of the current lack of an effective business work order classification management mechanism, business work orders are often roughly classified based on the business experience of staff, which cannot meet the diverse real-time customer needs, resulting in frequent problems such as inefficient business processing and incomplete problem solving, affecting user experience. This solution is proposed to overcome the above problems or at least partially solve the above problems.

[0004] On the one hand, an object of some embodiments of this specification is to provide a method for establishing a business work order classification model, the method comprising:

[0005] Obtain historical business work order data and real-time business work order data;

[0006] Classify the historical business work order data according to preset business scenarios to obtain multiple levels of fixed business tags;

[0007] Performing combined traversal on the fixed service labels of the multiple levels to obtain a combined traversal result;

[0008] Taking identification of fixed business labels at each level in the combined traversal result as a training goal, a corresponding first classification model is constructed according to the historical business work order data;

[0009] Determine the active business tags corresponding to the real-time business work order data;

[0010] Taking the identification of active business labels as the training goal, the corresponding second classification model is built based on real-time business work order data;

[0011] The first classification model and the second classification model are concatenated to obtain a business work order classification model.

[0012] Furthermore, among the fixed service tags of multiple levels, the number of fixed service tags of each level decreases as the level increases, and there is a subordinate relationship between the fixed service tags of each level and the preset fixed service tags of its adjacent level.

[0013] Furthermore, the fixed service labels of the multiple levels are combined and traversed to obtain a combined traversal result, including:

[0014] Determine that every two fixed service labels between the multiple levels have a subordinate relationship;

[0015] The fixed service labels are used as nodes and the subordinate relationships are used as links. Every two fixed service labels with subordinate relationships are connected to obtain a corresponding topological structure.

[0016] Perform a combinatorial traversal on the topological structure to obtain a combinatorial traversal result.

[0017] Furthermore, the topological structure is subjected to a combinatorial traversal to obtain a combinatorial traversal result, including:

[0018] By using a preset traversal algorithm, starting from a preset starting point of the topological structure and ending at a preset end point of the topological structure, a combined traversal is performed on the topological structure to obtain a combined traversal result.

[0019] Furthermore, taking the identification of the fixed business labels of each level in the combined traversal result as a training target, a corresponding first classification model is constructed according to the historical business work order data, including:

[0020] The historical business work order data is used as input, and the fixed business labels of each level corresponding to the historical business work order data are used as target output to construct training samples;

[0021] The deep learning model is trained using the training samples to obtain a first classification model.

[0022] Furthermore, taking identification of fixed business labels at each level in the combined traversal result as a training target, building a corresponding first classification model according to the historical business work order data further includes:

[0023] Dividing the training samples into a training set, a validation set and a test set;

[0024] Using the training set and the validation set to train the deep learning model;

[0025] Inputting the historical business work order data in the test set into the trained deep learning model to obtain a fixed business prediction label at a specified level;

[0026] Get the real label of the fixed business at the specified level corresponding to the historical business work order data;

[0027] Determining whether the fixed service prediction label matches the fixed service real label;

[0028] Filter the output results of the deep learning model obtained through training according to the matching results;

[0029] Calculate the accuracy rate according to the output result and the fixed service label of each level in the test set;

[0030] If the accuracy is less than a set threshold, retraining the deep learning model;

[0031] If the accuracy is not less than the set threshold, the training of the deep learning model is completed.

[0032] Furthermore, the active business tag corresponding to the real-time business work order data is determined, including:

[0033] Use the preset user focus dictionary to filter the real-time business work order data to obtain the filtering results;

[0034] The screening results are filtered using the preset user deactivated dictionary to obtain the corresponding active business labels.

[0035] Furthermore, taking the identification of active business labels as the training goal, a corresponding second classification model is constructed according to the real-time business work order data, including:

[0036] Get a large language model;

[0037] Taking the real-time business work order data as input, taking the active business tag corresponding to the real-time business work order data as target output, and determining the corresponding recognition instruction;

[0038] The large language model is trained using the recognition instruction to obtain a second classification model.

[0039] Furthermore, the first classification model and the second classification model are concatenated to obtain a business work order classification model, including:

[0040] The first classification model and the second classification model are connected in parallel to obtain a business work order classification model.

[0041] On the other hand, some embodiments of this specification also provide a device for establishing a business work order classification model, the device comprising:

[0042] The acquisition module is used to obtain historical business work order data and real-time business work order data;

[0043] A hierarchical classification module is used to classify the historical business work order data according to preset business scenarios to obtain multiple levels of fixed business tags;

[0044] A traversal module, used for performing a combined traversal on the fixed service tags of the multiple levels to obtain a combined traversal result;

[0045] A first training module is used to identify the fixed business labels of each level in the combined traversal result as a training target, and to construct a corresponding first classification model according to the historical business work order data;

[0046] A determination module is used to determine the active business tag corresponding to the real-time business work order data;

[0047] The second training module is used to identify active business tags as a training target and build a corresponding second classification model according to real-time business work order data;

[0048] A splicing module is used to splice the first classification model and the second classification model to obtain a business work order classification model.

[0049] Based on the same inventive concept, on one hand, an object of some embodiments of this specification is to provide a business work order classification method, the method comprising:

[0050] Receive business work order data to be classified;

[0051] The business work order data to be classified is input into a business work order classification model trained by the method described in any of the aforementioned embodiments to obtain a classification result corresponding to the business work order data to be classified.

[0052] Based on the same inventive concept, on the other hand, some embodiments of this specification further provide a business work order classification device, the device comprising:

[0053] A receiving module, used for receiving business work order data to be classified;

[0054] The classification module is used to input the business work order data to be classified into the business work order classification model trained by the method described in any of the above embodiments to obtain the classification result corresponding to the business work order data to be classified.

[0055] On the other hand, some embodiments of the present specification further provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein when the computer program is executed by the processor, the instructions of the above method are executed.

[0056] On the other hand, some embodiments of the present specification further provide a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0057] On the other hand, some embodiments of the present specification further provide a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0058] One or more technical solutions provided by some embodiments of this specification have at least the following technical effects:

[0059] The embodiments of the present specification first automatically obtain historical business work order data and real-time business work order data to classify the historical business work order data according to preset business scenarios to obtain multiple levels of fixed business tags. Since one business work order corresponds to multiple levels of fixed business tags, there are associations between different levels and they are not independent of each other. Therefore, before training the first classification model, it is also necessary to perform a combined traversal of the multiple levels of fixed business tags to obtain a combined traversal result. The combined traversal result stores the fixed business tags of each level that have specific associations with each other corresponding to the business work orders. Then, the training goal is to identify the fixed business tags of each level in the combined traversal result, and the corresponding first classification model is constructed according to the historical business work order data, so as to use the first classification model to identify the business work orders. The fixed business of the work order, but only identifying the fixed business is often difficult to meet user needs. Therefore, it is also necessary to establish a second classification model for identifying the active business tags based on the real-time adjustment of the business, so as to quickly and accurately identify the active business tags, and splice the first classification model and the second classification model to obtain the business work order classification model, so that the business work order classification model can not only finely identify multiple levels of fixed business tags and real-time changing active business tags, but also decouple the identification of fixed business tags and active business tags, thereby improving the classification efficiency of business work orders. When the active business tags change frequently with business activities, only the second classification model needs to be adjusted to quickly update the business classification model, thereby reducing the subsequent maintenance and optimization costs of the model, improving the response efficiency of business work orders, and improving user experience.

[0060] The above description is only an overview of the technical solutions of some embodiments of this specification. In order to more clearly understand the technical means of some embodiments of this specification, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of some embodiments of this specification more obvious and easy to understand, the specific implementation methods of some embodiments of this specification are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate some embodiments of this specification or technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0062] Figure 1 A schematic diagram of an implementation system of a method for establishing a business work order classification model in some embodiments of this specification is shown;

[0063] Figure 2 A flowchart of a method for establishing a business work order classification model in some embodiments of this specification is shown;

[0064] Figure 3 This is a schematic diagram of steps for combining and traversing fixed service tags at multiple levels in some embodiments of this specification;

[0065] Figure 4 This is a schematic diagram of the steps of constructing a corresponding first classification model according to historical business work order data in some embodiments of this specification;

[0066] Figure 5 A schematic diagram of the steps of optimizing the first classification model in some embodiments of this specification;

[0067] Figure 6 A schematic diagram of the steps of determining an active business tag corresponding to real-time business work order data in some embodiments of this specification;

[0068] Figure 7 A schematic diagram of the steps of constructing a corresponding second classification model according to real-time business work order data in some embodiments of this specification;

[0069] Figure 8 A flowchart of a method for classifying a business work order in some embodiments of this specification is shown;

[0070] Fig. 9 This is a schematic diagram of the structure of a device for establishing a business work order classification model in some embodiments of this specification;

[0071] Fig.10 This is a structural schematic diagram of a business work order classification device in some embodiments of this specification;

[0072] Fig.11 This is a schematic diagram of the computer device structure provided in some embodiments of this specification.

[0073] [Description of Reference Numerals]

[0074] 101. Terminal;

[0075] 102. Server;

[0076] 901. Get module;

[0077] 902, hierarchical classification module;

[0078] 903. Traversal module;

[0079] 904, first training module;

[0080] 905. Determine module;

[0081] 906, second training module;

[0082] 907, splicing module;

[0083] 1001, receiving module;

[0084] 1002. Classification module;

[0085] 1102. Computer equipment;

[0086] 1104. Processor;

[0087] 1106. Memory;

[0088] 1108. Driving mechanism;

[0089] 1110, input / output interface;

[0090] 1112. Input device;

[0091] 1114. Output device;

[0092] 1116. Presentation equipment;

[0093] 1118. Graphical user interface;

[0094] 1120. Network interface;

[0095] 1122. Communication link;

[0096] 1124. Communication bus. DETAILED DESCRIPTION

[0097] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in some embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on some embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0098] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0099] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of relevant laws and regulations.

[0100] It should be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0101] like Figure 1 The diagram is a schematic diagram of an implementation system of a method for establishing a business work order classification model according to an embodiment of the present invention, which may include: a terminal 101 and a server 102. The terminal 101 and the server 102 communicate with each other through a network, and the network may include a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, and is connected to a website, a user device (such as a computing device), and a back-end system. A staff member may send a request to the server 102 through the terminal 101. After receiving the request, the server 102 calls the business work order data in the database for calculation and processing, obtains the processing result, and sends the processing result to the terminal 101, so that the staff member processes the business according to the processing result.

[0102] In the embodiments of this specification, the server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms.

[0103] In an optional embodiment, the terminal 101 may include but is not limited to electronic devices such as self-service terminal devices, desktop computers, tablet computers, laptop computers, smart wearable devices, etc. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc. Of course, the terminal 101 is not limited to the above-mentioned electronic devices with a certain entity, and it can also be software running in the above-mentioned electronic devices.

[0104] In addition, it should be noted that Figure 1 What is shown is only an application environment provided by the present disclosure. In actual application, multiple terminals 101 may be included, and this specification does not limit this.

[0105] Figure 2 It is a flowchart of a method for establishing a business work order classification model provided by an embodiment of the present invention. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or device product is executed, it can be executed in the order or in parallel according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 2 As shown, applied to the above-mentioned server side, the method may include:

[0106] S201: Acquire historical business work order data and real-time business work order data;

[0107] S202: Classifying the historical business work order data according to preset business scenarios to obtain multiple levels of fixed business tags;

[0108] S203: performing combined traversal on the fixed service labels of the multiple levels to obtain a combined traversal result;

[0109] S204: Taking identification of fixed business labels at each level in the combined traversal result as a training target, building a corresponding first classification model according to the historical business work order data;

[0110] S205: Determine the active business tag corresponding to the real-time business work order data;

[0111] S206: Taking identification of active business labels as a training goal, a corresponding second classification model is constructed according to real-time business work order data;

[0112] S207: Concatenate the first classification model and the second classification model to obtain a business work order classification model.

[0113] The embodiments of the present specification first automatically obtain historical business work order data and real-time business work order data to classify the historical business work order data according to preset business scenarios to obtain multiple levels of fixed business tags. Since one business work order corresponds to multiple levels of fixed business tags, there are associations between different levels and they are not independent of each other. Therefore, before training the first classification model, it is also necessary to perform a combined traversal of the multiple levels of fixed business tags to obtain a combined traversal result. The combined traversal result stores the fixed business tags of each level that have specific associations with each other corresponding to the business work orders. Then, the training goal is to identify the fixed business tags of each level in the combined traversal result, and the corresponding first classification model is constructed according to the historical business work order data, so as to use the first classification model to identify the business work orders. The fixed business of the work order, but only identifying the fixed business is often difficult to meet user needs. Therefore, it is also necessary to establish a second classification model for identifying the active business tags based on the real-time adjustment of the business, so as to quickly and accurately identify the active business tags, and splice the first classification model and the second classification model to obtain the business work order classification model, so that the business work order classification model can not only finely identify multiple levels of fixed business tags and real-time changing active business tags, but also decouple the identification of fixed business tags and active business tags, thereby improving the classification efficiency of business work orders. When the active business tags change frequently with business activities, only the second classification model needs to be adjusted to quickly update the business classification model, thereby reducing the subsequent maintenance and optimization costs of the model, improving the response efficiency of business work orders, and improving user experience.

[0114] It can be understood that, in some embodiments, historical business work order data and real-time business work order data respectively represent the historical data of business work orders and the business work order data during the period of active business. Specifically, the business work order may be a complaint work order, a business processing work order, etc., which is not limited in this article. The difference between historical business work order data and real-time business work order data is that the active business tags corresponding to the historical business work order data and the real-time business work order data are often different. This is because as business activities progress, the activities held in different time periods are different. Therefore, historical business work order data can be strictly divided into historical business work order data with historical active business tags and historical business work order data without historical active business tags.

[0115] In some embodiments, after obtaining historical business work order data and real-time business work order data, it is necessary to classify the historical business work order data according to preset business scenarios to obtain multiple levels of fixed business labels. Specifically, fixed business is a long-term business, and generally speaking, the frequency of updates and changes is low. Taking complaint business as an example, a fixed business label system can be obtained from 5 levels of business category, product category, business process, and complaint point. Among them, business category, product category, business process, and complaint point are first-level labels to fifth-level labels, respectively. There are 3 categories of first-level labels, 22 categories of second-level labels, 140 categories of third-level labels, 145 categories of fourth-level labels, and 942 categories of fifth-level labels. Then, for historical business work order data, 5 different levels of fixed business labels can be matched.

[0116] Furthermore, in some embodiments, among multiple levels of fixed service tags, the number of fixed service tags at each level decreases as the level increases, and there is a subordinate relationship between the fixed service tags at each level and the preset fixed service tags at the adjacent level.

[0117] It can be understood that, in some embodiments, since a business work order corresponds to fixed business tags of multiple levels, there is an association relationship between different levels, which are not independent of each other, and the higher the level, the fewer the number of fixed business tags corresponding to it. Continuing with the above-mentioned complaint business as an example, there are 3 categories of first-level tags, 22 categories of second-level tags, 140 categories of third-level tags, 145 categories of fourth-level tags, and 942 categories of fifth-level tags, among which 3>22>140>145>942, which also means that the higher the level, the simpler the division of fixed business tags, and there is a subordinate relationship between the fixed business tags of each level and the preset fixed business tags of the adjacent level, that is, the second-level fixed business tag is subordinate to the first-level fixed business tag associated with it, the third-level fixed business tag is subordinate to the second-level fixed business tag associated with it, and so on.

[0118] It should be noted that in some embodiments, as shown in Table 1 below, the secondary fixed service tag may be subordinate to one or more primary fixed service tags associated with it, the tertiary fixed service tag may be subordinate to one or more secondary fixed service tags associated with it, and the like, and this document does not limit this.

[0119] Table 1 Schematic diagram of fixed service labels at multiple levels in complaint services

[0120]

[0121] It should be noted that, in some embodiments, the division of the first level (starting level), the second level, and the third level may also be described in other ways, for example, level A, level B, level C, etc. This document does not limit this. Furthermore, fixed service labels of multiple levels may be represented as nodes of multiple levels, and the association relationship (subordinate relationship) between fixed service labels of adjacent levels may be regarded as a vector (the vector arrows all point to the master or slave in the subordinate relationship), thereby realizing the combined traversal of fixed service labels of multiple levels.

[0122] Specifically, refer to the attached Figure 3 In some embodiments, performing combined traversal on the fixed service labels of the multiple levels to obtain the combined traversal result may include:

[0123] S301: determining that every two fixed service labels in a subordinate relationship exist among fixed service labels between multiple levels;

[0124] S302: taking the fixed service label as a node and the subordinate relationship as a link, connecting every two fixed service labels having a subordinate relationship to obtain a corresponding topological structure;

[0125] S303: Perform a combined traversal on the topological structure to obtain a combined traversal result.

[0126] It can be understood that, in some embodiments, the link corresponding to the subordinate relationship is the above-mentioned vector, and the fixed business label is used as a node, and the subordinate relationship is used as a link. Every two fixed business labels with subordinate relationships are connected to quickly obtain the corresponding topological structure, and the topological structure is a pyramid structure. Combination traversal of the topological structure can quickly obtain all possible fixed business label combinations of the business work order, and the fixed business label combination includes the fixed business label combination of each level, so that the combination traversal result is fully comprehensive and reliable.

[0127] Furthermore, in some embodiments, performing a combinatorial traversal on the topological structure to obtain a combinatorial traversal result includes:

[0128] By using a preset traversal algorithm, starting from a preset starting point of the topological structure and ending at a preset end point of the topological structure, a combined traversal is performed on the topological structure to obtain a combined traversal result.

[0129] It can be understood that, in some embodiments, the combined traversal of the topological structure is equivalent to the process of graph traversal of the topological structure, and can also be regarded as a path planning process, that is, under given optional multiple paths, from a given starting point to a given end point. Therefore, through the preset traversal algorithm, it is possible to quickly start from the preset starting point of the topological structure and reach the preset end point of the topological structure, thereby realizing efficient combined traversal of the topological structure and obtaining accurate combined traversal results. This article does not limit the specific traversal algorithm.

[0130] See attached Figure 4 In some embodiments, taking identification of fixed business labels at each level in the combined traversal result as a training target and constructing a corresponding first classification model according to the historical business work order data may include:

[0131] S401: Taking the historical business work order data as input, taking the fixed business labels of each level corresponding to the historical business work order data as target output, and constructing training samples;

[0132] S402: Use the training samples to train a deep learning model to obtain a first classification model.

[0133] It can be understood that, in some embodiments, after obtaining the combined traversal results, a first classification model for identifying the fixed business labels of each level corresponding to a business work order is constructed. Specifically, the historical business work order data is first taken as input, and the fixed business labels of each level corresponding to the historical business work order data are taken as the target output to construct a training sample, and then the training sample is used to train the deep learning model to obtain the first classification model. Since there are many types of fixed business labels and the system is complex, the deep learning model at this time can be an open source text classification model, such as the Roberta-wwm text classification model, etc., which is not limited in this article.

[0134] It should be noted that in some embodiments, when constructing training samples, in addition to identifying the fixed business labels of each level, the target output also needs to output the fixed business labels of each level in sequence. The output fixed business label sequence can be regarded as an output sequence, which represents the association relationship (subordinate relationship) between the levels.

[0135] Further, see Attachment Figure 5 In some embodiments, taking the identification of the fixed business labels of each level in the combined traversal result as the training target and constructing the corresponding first classification model according to the historical business work order data may further include:

[0136] S501: Divide the training samples into a training set, a validation set and a test set;

[0137] S502: Train the deep learning model using the training set and the validation set;

[0138] S503: Inputting the historical business work order data in the test set into the trained deep learning model to obtain a fixed business prediction label of a specified level;

[0139] S504: Obtaining a real label of a fixed business of a specified level corresponding to the historical business work order data;

[0140] S505: Determine whether the fixed service prediction label matches the fixed service real label;

[0141] S506: Filter the output results of the deep learning model obtained through training according to the matching results;

[0142] S507: Calculating the accuracy rate according to the output result and the fixed service label of each level in the test set;

[0143] S508: If the accuracy is less than a set threshold, retrain the deep learning model;

[0144] S509: If the accuracy is not less than the set threshold, the training of the deep learning model is completed.

[0145] It can be understood that, in some embodiments, in order to ensure the classification accuracy of the first classification model, the first classification model can be optimized together using the training set, the validation set, and the test set. Since the target output is a sequence data with a subordinate relationship, the sequence data starts from the starting label and ends at the end label, and from the starting label to the end label, as the degree of refinement of the label classification becomes higher and higher, correspondingly, the difficulty of label recognition increases. Therefore, after training the deep learning model with the training set, the fixed business prediction label of the specified level can be first identified, and then it can be determined whether it matches the fixed business real label of the specified level corresponding to the historical business work order data. The specified level has the characteristic of being easy to identify. Generally speaking, the specified level is the starting level, or the starting level and several levels adjacent to the starting level.

[0146] Specifically, in some embodiments, when judging whether the fixed business prediction label matches the fixed business true label, the fixed business true label represents the target output result that is desired to be predicted. Therefore, it is necessary to obtain the confidence of the fixed business prediction label relative to the fixed business true label, and select the fixed business prediction label with the highest confidence and the output result of the subsequent levels corresponding to it. It can be understood that once the confidence of the fixed business prediction label of the specified level and the fixed business true label of the specified level is less than a preset threshold, it means that the fixed business labels of other levels except the specified level predicted by the deep learning model are also unreliable. Therefore, the test set can be used to quickly and accurately evaluate the trained deep learning model, thereby achieving efficient optimization of the training process and the output results of the deep learning model.

[0147] See attached Figure 6 In some embodiments, determining the active business tag corresponding to the real-time business work order data may include:

[0148] S601: Filter the real-time business work order data using a preset user focus dictionary to obtain a filtering result;

[0149] S602: Filter the screening results using a preset user disabled dictionary to obtain corresponding active service tags.

[0150] It can be understood that in some embodiments, the activity business is time-sensitive, and the preset user deactivated dictionary is used to filter out words that appear frequently (for example, the frequency of occurrence exceeds the preset frequency) and are not associated with the activity business. The preset user attention dictionary is used to filter out words associated with the activity business, wherein the user attention dictionary can be obtained by extracting keywords based on the open source large language model in combination with the prompt instruction. Specifically, firstly, the large language model is used in combination with the prompt instruction to extract words in the limited conceptual category of the activity business, such as the activity name, activity strategy, etc., and then the specific keywords transmitted by the user are obtained in real time, so as to use the specific keywords transmitted by the user to reversely adjust the user attention dictionary to ensure the accuracy of the user attention dictionary. In some embodiments, continuing to take the complaint service work order as an example, the prompt instruction based on the open source large language model can be:

[0151] “You are an excellent customer service expert with the following abilities: Based on the complaint content and user attention dictionary, you can identify keyword phrases such as user attention words, activity business, products, organizations, rules, events, etc.

[0152] Note: Exclusion rules include user sensitive information, stop words, pure Arabic numerals, words with low business relevance, order numbers, etc.

[0153] Note: Only the most core {words_num} keyword phrases are recognized without numerical numbers (such as 1.xx, 2.xx).

[0154] Note: Words in the dictionary that the user is interested in will be matched first. If a match is found, the words will be sorted first.

[0155] Returns keyword phrases in list form, which is highly readable (convenient for subsequent code to parse the list)

[0156] Complaint content: {content}

[0157] User attention dictionary: {user_dict}"

[0158] Among them, words_num is the number of keywords that the user wants to extract, for example, it can be 5; content is the complaint business work order text from which keywords need to be extracted; user_dict is the dictionary that the user is interested in.

[0159] Furthermore, in some embodiments, a user deactivated dictionary can be constructed based on an open source large language model according to historical business work order data. When filtering using the user deactivated dictionary, a document relevance algorithm (such as the BM25 algorithm) can be used to perform a relevance evaluation on the real-time business work order data to be filtered, thereby measuring the degree of match between the stop words and the real-time business work order data to be filtered based on the relevance evaluation results to achieve accurate filtering.

[0160] See attached Figure 7 In some embodiments, taking identification of active business labels as a training goal and building a corresponding second classification model according to real-time business work order data may include:

[0161] S701: Obtain a large language model;

[0162] S702: taking the real-time business work order data as input, taking the activity business tag corresponding to the real-time business work order data as target output, and determining a corresponding recognition instruction;

[0163] S703: Using the recognition instruction to train the large language model to obtain a second classification model.

[0164] It can be understood that in some embodiments, a second classification model for identifying the latest active business tags can be constructed based on real-time business work orders and their corresponding active business tags. The large language model can be an open source large language model with a large number of model parameters. The recognition instructions match the active business tags that are desired to be identified. Therefore, multiple related active business tag identifications can be achieved by adjusting the recognition instructions. Specifically, the recognition instructions can be prompt instructions, which have been illustrated above and will not be repeated here. Furthermore, in some embodiments, due to the model characteristics of the large language model itself, when training the second classification model, the second classification model can be dynamically adjusted and optimized by adjusting the recognition instructions, and there is no need to accumulate a large number of training samples, thereby achieving accurate identification of active business tags.

[0165] Furthermore, it should be noted that the update frequency of the first classification model used to identify fixed business tags is low, and the update frequency of the second classification model used to identify active business tags is high. The method of decoupling and classifying fixed business and active business can effectively reduce the maintenance cost of the business work order classification model.

[0166] Furthermore, in some embodiments, the first classification model and the second classification model are concatenated to obtain a business work order classification model, including:

[0167] The first classification model and the second classification model are connected in parallel to obtain a business work order classification model.

[0168] It can be understood that, in some embodiments, the first classification model and the second classification model are connected in parallel with each other. For a business work order to be classified, it is input into the first classification model and the second classification model respectively, the first classification model identifies the fixed business tag of the business work order, and the second classification model identifies the active business tag of the business work order, so that the final business work order classification model outputs the fixed business tag and the active business tag corresponding to the business work order classification model, thereby determining the specific type of the business work order according to the fixed business tag and the active business tag corresponding to the business work order classification model, so as to provide users with effective, accurate and timely help according to the specific type of the business work order.

[0169] Based on the same inventive concept, some embodiments of this specification also provide a business work order classification method, referring to Figure 8 As shown, in some embodiments, the method may include:

[0170] S801: receiving business work order data to be classified;

[0171] S802: Input the business work order data to be classified into a business work order classification model trained by the method described in any of the aforementioned embodiments to obtain a classification result corresponding to the business work order data to be classified.

[0172] It can be understood that, in some embodiments, the business work order classification model can not only finely identify multiple levels of fixed business tags and real-time changing active business tags, but also decouple the identification of fixed business tags and active business tags, thereby improving the classification efficiency of business work orders. When the active business tags change frequently with business activities, only the second classification model needs to be adjusted to quickly update the business classification model, thereby reducing the subsequent maintenance and optimization costs of the model, improving the response efficiency of business work orders, and improving user experience.

[0173] It should be noted that, although the operations of the method of the present invention are described in a specific order in the above embodiments and the accompanying drawings, this does not require or imply that the operations must be performed in the specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0174] Corresponding to the above-mentioned method for establishing a business work order classification model, some embodiments of this specification also provide a device for establishing a business work order classification model, referring to Fig. 9 As shown, in some embodiments, the apparatus may include:

[0175] Acquisition module 901, used to acquire historical business work order data and real-time business work order data;

[0176] A hierarchical classification module 902 is used to classify the historical business work order data according to a preset business scenario to obtain multiple levels of fixed business tags;

[0177] A traversal module 903, configured to perform a combined traversal on the fixed service labels of the multiple levels to obtain a combined traversal result;

[0178] A first training module 904 is used to identify the fixed business labels of each level in the combined traversal result as a training target, and to construct a corresponding first classification model according to the historical business work order data;

[0179] Determination module 905, used to determine the active business tag corresponding to the real-time business work order data;

[0180] The second training module 906 is used to identify the active business label as a training target and build a corresponding second classification model according to the real-time business work order data;

[0181] The splicing module 907 is used to splice the first classification model and the second classification model to obtain a business work order classification model.

[0182] Based on the same inventive concept, corresponding to the above-mentioned business work order method, some embodiments of this specification also provide a business work order classification device, referring to Fig.10 As shown, in some embodiments, the apparatus may include:

[0183] Receiving module 1001, used for receiving business work order data to be classified;

[0184] The classification module 1002 is used to input the business work order data to be classified into the business work order classification model trained by the method described in any of the above embodiments to obtain the classification result corresponding to the business work order data to be classified.

[0185] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0186] It should be noted that in the embodiments of this specification, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user and fully authorized by all parties.

[0187] It should be noted that the computer program product described in this specification is a software product that mainly implements the method described in this specification through a computer program.

[0188] The embodiments of this specification also provide a computer device. Fig.11As shown, in some embodiments of the present specification, the computer device 1102 may include one or more processors 1104, such as one or more central processing units (CPUs) or graphics processing units (GPUs), and each processing unit may implement one or more hardware threads. The computer device 1102 may also include any memory 1106, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, the computer program on the memory 1106 and can be run on the processor 1104, when the computer program is run by the processor 1104, the instructions of the method described in any of the above embodiments may be executed. Non-limitingly, for example, the memory 1106 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent a fixed or removable component of the computer device 1102. In one embodiment, when the processor 1104 executes the associated instructions stored in any memory or combination of memories, the computer device 1102 can perform any operation of the associated instructions. The computer device 1102 also includes one or more drive mechanisms 1108 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0189] The computer device 1102 may also include an input / output interface 1110 (I / O) for receiving various inputs (via input devices 1112) and for providing various outputs (via output devices 1114). A specific output mechanism may include a presentation device 1116 and an associated graphical user interface 1118 (GUI). In other embodiments, the input / output interface 1110 (I / O), input device 1112, and output device 1114 may not be included, and the computer device 1102 may be used as a computer device in a network. The computer device 1102 may also include one or more network interfaces 1120 for exchanging data with other devices via one or more communication links 1122. One or more communication buses 1124 couple the components described above together.

[0190] The communication link 1122 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 1122 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0191] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), computer-readable storage media, and computer program products of some embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processor to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processor generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0192] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processor to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0193] These computer program instructions can also be loaded onto a computer or other programmable data processor so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0194] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0195] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0196] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computer device. As defined in this specification, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0197] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0198] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0199] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0200] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0201] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0202] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for establishing a business work order classification model, characterized in that: The method comprises: Obtain historical business work order data and real-time business work order data; Classify the historical business work order data according to preset business scenarios to obtain multiple levels of fixed business tags; Performing combined traversal on the fixed service labels of the multiple levels to obtain a combined traversal result; Taking identification of fixed business labels at each level in the combined traversal result as a training goal, a corresponding first classification model is constructed according to the historical business work order data; Determine the active business tags corresponding to the real-time business work order data; Taking the identification of active business labels as the training goal, the corresponding second classification model is built based on real-time business work order data; The first classification model and the second classification model are concatenated to obtain a business work order classification model.

2. The method according to claim 1, characterized in that: Among the fixed service tags of multiple levels, the number of the fixed service tags of each level decreases as the level increases, and there is a subordinate relationship between the fixed service tags of each level and the preset fixed service tags of the adjacent level.

3. The method according to claim 2, characterized in that The fixed service labels of the multiple levels are combined and traversed to obtain a combined traversal result, including: Determine that every two fixed service labels between the multiple levels have a subordinate relationship; The fixed service labels are used as nodes and the subordinate relationships are used as links. Every two fixed service labels with subordinate relationships are connected to obtain a corresponding topological structure. Perform a combinatorial traversal on the topological structure to obtain a combinatorial traversal result.

4. The method according to claim 3, characterized in that Perform a combinatorial traversal on the topological structure to obtain a combinatorial traversal result, including: By using a preset traversal algorithm, starting from a preset starting point of the topological structure and ending at a preset end point of the topological structure, a combined traversal is performed on the topological structure to obtain a combined traversal result.

5. The method according to claim 1, characterized in that Taking identification of fixed business labels at each level in the combined traversal result as a training target, a corresponding first classification model is constructed according to the historical business work order data, including: The historical business work order data is used as input, and the fixed business labels of each level corresponding to the historical business work order data are used as target output to construct training samples; The deep learning model is trained using the training samples to obtain a first classification model.

6. The method according to claim 5, characterized in that Taking identification of fixed business labels at each level in the combined traversal result as a training target, building a corresponding first classification model according to the historical business work order data further includes: Dividing the training samples into a training set, a validation set and a test set; Using the training set and the validation set to train the deep learning model; Inputting the historical business work order data in the test set into the trained deep learning model to obtain a fixed business prediction label at a specified level; Get the real label of the fixed business at the specified level corresponding to the historical business work order data; Determining whether the fixed service prediction label matches the fixed service real label; Filter the output results of the deep learning model obtained through training according to the matching results; Calculate the accuracy rate according to the output result and the fixed service label of each level in the test set; If the accuracy is less than a set threshold, retraining the deep learning model; If the accuracy is not less than the set threshold, the training of the deep learning model is completed.

7. The method according to claim 1, characterized in that Determine the active business tags corresponding to the real-time business work order data, including: Use the preset user attention dictionary to filter the real-time business work order data and obtain the filtering results; The screening results are filtered using the preset user deactivated dictionary to obtain the corresponding active business labels.

8. The method according to claim 1, characterized in that Taking the identification of active business labels as the training goal, the corresponding second classification model is built according to the real-time business work order data, including: Get a large language model; Taking the real-time business work order data as input, taking the active business tag corresponding to the real-time business work order data as target output, and determining the corresponding recognition instruction; The large language model is trained using the recognition instruction to obtain a second classification model.

9. The method according to claim 1, characterized in that: The first classification model and the second classification model are combined to obtain a business work order classification model, including: The first classification model and the second classification model are connected in parallel to obtain a business work order classification model.

10. A method for classifying business work orders, characterized in that: The method comprises: Receive business work order data to be classified; The business work order data to be classified is input into a business work order classification model trained by the method described in any one of claims 1 to 9 to obtain a classification result corresponding to the business work order data to be classified.

11. A device for establishing a business work order classification model, characterized in that: The device comprises: The acquisition module is used to obtain historical business work order data and real-time business work order data; A hierarchical classification module is used to classify the historical business work order data according to preset business scenarios to obtain multiple levels of fixed business tags; A traversal module, used for performing a combined traversal on the fixed service tags of the multiple levels to obtain a combined traversal result; A first training module is used to identify the fixed business labels of each level in the combined traversal result as a training target, and to construct a corresponding first classification model according to the historical business work order data; A determination module is used to determine the active business tag corresponding to the real-time business work order data; The second training module is used to identify active business tags as a training target and build a corresponding second classification model according to real-time business work order data; The splicing module is used to splice the first classification model and the second classification model to obtain a business work order classification model.

12. A business work order classification device, characterized in that: The device comprises: A receiving module, used for receiving business work order data to be classified; A classification module is used to input the business work order data to be classified into a business work order classification model trained using the method described in any one of claims 1 to 9 to obtain a classification result corresponding to the business work order data to be classified.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the computer program executes the instructions of the method according to any one of claims 1 to 10.

14. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the computer program executes the instructions of the method according to any one of claims 1 to 10.

15. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program executes instructions of the method according to any one of claims 1 to 10.

Images