Work order processing method, apparatus, device, and storage medium

Abstract

The application discloses a work order processing method and device, equipment and a storage medium, and relates to the technical field of intelligent operation and maintenance. The method comprises the following steps: acquiring work order data of a current work order; performing scene type reasoning by using a historical reasoning model or an online reasoning model according to the work order data of the current work order, so as to obtain a scene type to which the current work order belongs; wherein the online reasoning model is obtained by updating and training the historical reasoning model based on the work order data of the current work order and work order data of historical work orders; performing element information extraction by using an information extraction model according to the work order data of the current work order, so as to obtain element information of the current work order; and generating a work order reply text according to the scene type and the element information, so as to obtain a work order processing result of the current work order. The application solves the problem of low processing efficiency of the work order processing method in the prior art, realizes autonomous reply of the operation and maintenance work order, reduces the labor and time costs, and improves the work order processing efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent operation and maintenance, and in particular to a work order processing method and device, equipment and a storage medium. BACKGROUND

[0002] In various industries, it is often necessary to process complaint work orders, such as the telecommunications operation industry and the logistics industry. Currently, after a complaint occurs, customer service personnel will submit a complaint work order. The existing work order processing method mainly involves a work order processing personnel verifying the content data of the submitted complaint work order, manually selecting a text reply template corresponding to the scene type to which the work order belongs based on business experience and knowledge, and then writing a reply text according to the text reply template and the verified content data of the work order to reply.

[0003] This work order processing method has the following disadvantages: the manual determination of the scene type to which the work order belongs and the manual selection of the text reply template are prone to errors, and a large amount of manpower and time is required, resulting in low work order processing efficiency. SUMMARY

[0004] The main purpose of the present application is to provide a work order processing method, device, equipment and storage medium, which aims to solve the technical problem of low processing efficiency of the work order processing method in the related art.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a work order processing method, which comprises:

[0007] obtaining work order data of a current work order;

[0008] performing scene type reasoning using a historical reasoning model or an online reasoning model according to the work order data of the current work order, to obtain a scene type to which the current work order belongs; wherein the online reasoning model is obtained by updating and training the historical reasoning model based on the work order data of the current work order and work order data of historical work orders;

[0009] performing element information extraction using an information extraction model according to the work order data of the current work order, to obtain element information of the current work order;

[0010] generating a work order reply text according to the scene type and the element information, to obtain a work order processing result of the current work order.

[0011] Optionally, in the work order processing method described above, after the step of obtaining the work order data of the current work order, the method further comprises:

[0012] The work order data of the current work order is preprocessed to obtain preprocessed work order data; wherein, the preprocessing includes dynamically adjusting the work order content string in the work order data, and the dynamic adjustment includes removing preset characters, removing preset vocabularies and / or removing preset digit string sequences;

[0013] According to the work order data of the current work order, the scene type inference is performed by using the historical inference model or the online inference model to obtain the scene type to which the current work order belongs.

[0014] According to the preprocessed work order data, the scene type inference is performed by using the historical inference model or the online inference model to obtain the scene type to which the current work order belongs.

[0015] Optionally, in the above work order processing method, after the step of obtaining the work order data of the current work order, the method further comprises:

[0016] Obtain the configuration file and set the training flag in the configuration file to 0;

[0017] According to the work order data of the current work order and the work order data of the historical work order, the training data set and the test data set are obtained;

[0018] According to the training data set, the historical inference model is updated and trained to obtain an updated inference model; wherein, the historical inference model is constructed based on BERT (Bidirectional Encoder Representation from Transformers, Bidirectional Encoder Representation from Transformers) algorithm and BiLSTM (Bidirectional Long Short Term Memory Networks, Bidirectional Long Short Term Memory Networks) algorithm, and is trained by using the work order data of the historical work order;

[0019] According to the test data set, the updated inference model is evaluated;

[0020] If the evaluation is qualified, the updated inference model is determined as the online inference model, and the training flag is set to 1;

[0021] If the evaluation is not qualified, the training flag is kept as 0.

[0022] Optionally, in the above work order processing method, the step of evaluating the updated inference model according to the test data set comprises:

[0023] According to the test data set, the accuracy of the updated inference model is determined;

[0024] Determine whether the accuracy is greater than or equal to a preset threshold;

[0025] If the accuracy is greater than or equal to the preset threshold, it is determined that the evaluation is qualified;

[0026] If the accuracy is less than the preset threshold, it is determined that the evaluation is unqualified.

[0027] Optionally, in the work order processing method, the step of performing scene type inference by using the historical inference model or the online inference model to obtain the scene type to which the current work order belongs comprises:

[0028] Determine the content of the training flag in the configuration file acquired in real time;

[0029] When the training flag is 0, perform scene type inference by using the historical inference model to obtain the scene type to which the current work order belongs;

[0030] When the training flag is 1, perform scene type inference by using the online inference model to obtain the scene type to which the current work order belongs, and replace the historical inference model with the online inference model.

[0031] Optionally, in the work order processing method, before the step of performing element information extraction by using the information extraction model according to the work order data of the current work order to obtain the element information of the current work order, the method further comprises:

[0032] The information extraction model is constructed based on a BERT algorithm, a BiLSTM algorithm and a CRF (Conditional Random Field) algorithm.

[0033] Optionally, in the work order processing method, the step of generating a work order reply text according to the scene type and the element information to obtain the work order processing result of the current work order comprises:

[0034] Obtain the confidence of performing scene type inference by using the historical inference model or the online inference model;

[0035] When the confidence is greater than or equal to a preset threshold value, automatically generate a work order reply text according to the scene type and the element information;

[0036] When the confidence is less than the preset threshold value, obtain a work order reply text manually written;

[0037] Determine the automatically generated work order reply text or the manually written work order reply text as the work order processing result of the current work order.

[0038] In a second aspect, the present application provides a work order processing device, which comprises:

[0039] A data acquisition module is configured to acquire work order data of a current work order;

[0040] The scene classification module is configured to perform scene type reasoning on the basis of the work order data of the current work order by using a historical reasoning model or an online reasoning model to obtain a scene type to which the current work order belongs; wherein the online reasoning model is obtained by updating and training the historical reasoning model on the basis of the work order data of the current work order and the work order data of historical work orders;

[0041] The information extraction module is configured to perform element information extraction by using an information extraction model on the basis of the work order data of the current work order to obtain element information of the current work order.

[0042] The result generation module is configured to generate a work order reply text on the basis of the scene type and the element information to obtain a work order processing result of the current work order.

[0043] In a third aspect, the present application provides a work order processing device, which comprises a processor and a memory, and the memory stores a work order processing program, which, when executed by the processor, implements the work order processing method as described above.

[0044] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by one or more processors, implements the work order processing method as described above.

[0045] The above one or more technical solutions provided by the present application can have the following advantages or at least achieve the following technical effects:

[0046] The work order processing method, device, equipment and storage medium provided by the present application can obtain the scene type to which the current work order belongs by using a historical reasoning model or an online reasoning model to perform scene type reasoning, and obtain element information of the current work order by using an information extraction model to perform element information extraction, and then generate a work order reply text on the basis of the scene type and the element information to obtain a work order processing result of the current work order, thereby realizing autonomous reply of the operation and maintenance work order, reducing a large amount of manpower and time cost, improving work order processing efficiency, and thus improving customer satisfaction. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0048] Figure 1 The flowchart of the first embodiment of the work order processing method of the present application;

[0049] Figure 2 The hardware structure schematic diagram of the work order processing device related to the present application;

[0050] Figure 3 The flowchart of the second embodiment of the work order processing method of the present application;

[0051] Figure 4 The detailed flowchart of steps 210-S260 in the second embodiment of the work order processing method of the present application;

[0052] Figure 5 The functional module schematic diagram of the first embodiment of the work order processing device of the present application.

[0053] The implementation, functional features and advantages of the present application will be further described with reference to the drawings in combination with the embodiments. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort belong to the protection scope of the present application.

[0055] It should be noted that in the present application, the terms "comprising", "containing" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or system that comprises a list of elements not only includes those elements, but also includes other elements not expressly listed or inherent to such process, method, article or system. Without more limitations, the elements defined by the statement "comprise" do not exclude the presence of other identical elements in the process, method, article or system comprising the element. In addition, the meaning of "and / or" appearing throughout the text includes three parallel solutions, taking "A and / or B" as an example, including A solution, or B solution, or A and B solution. In the present application, the suffix such as "module" or "unit" used to represent elements is only for the convenience of the description of the present application, and has no specific meaning. Therefore, "module" or "unit" can be used mixedly. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances. In addition, the technical solutions of various embodiments can be combined with each other, but it is based on the fact that the technical solutions can be realized by those skilled in the art. When the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0056] The analysis of related technologies found that at present, after the complaint occurs, the customer service personnel will submit the complaint work order according to the work order template. In order to improve customer satisfaction and reduce the adverse effects of complaint events, it is necessary to timely and effectively handle the complaint work order. The existing work order processing method mainly verifies the work order content data by the work order processor for the submitted complaint work order, attributes the work order to the scene type by means of business experience knowledge, then selects the text reply template of the scene type corresponding to the work order, and then writes the reply text according to the text reply template and the verified work order content data, so as to reply.

[0057] The specific process is: the customer service personnel receives the complaint information of the customer, and then writes the complaint work order according to the complaint work order writing template; the operation and maintenance personnel receives the complaint work order, checks the data of the complaint information of the complaint work order, and checks the reason leading to the complaint; in order to quickly reply, the operation and maintenance personnel classifies different problem types into different work order scene types, and establishes the corresponding relationship between the work order scene type and the text reply template, the operation and maintenance personnel classifies the complaint work order according to the business experience knowledge, judges the scene type to which the current work order belongs, and finds the text reply template corresponding to the scene type; finally, the operation and maintenance personnel writes the reply text according to the data checked and the text reply template corresponding to the work order scene type, so as to reply.

[0058] The above-mentioned work order processing method has the following disadvantages:

[0059] 1. There are hundreds of work order scene types, and it is easy to select the wrong scene type for a work order due to time constraints and other reasons when a work order handler manually determines the scene type to which the work order belongs and manually selects a text reply template, resulting in an incorrect reply not being made;

[0060] 2. Some work order scene types are similar, and it is easy to select the wrong similar scene type due to negligence when a work order handler manually determines the scene type to which the work order belongs and manually selects a text reply template, resulting in an ineffective reply not being made;

[0061] 3. Some work order data needs to be queried from different sources by a work order handler before being checked, and it is easy to lose part of the query path, resulting in complete check data not being obtained;

[0062] 4. A large number of complaint work orders are received every day, and a large amount of manpower and time is required to process these work orders, so the efficiency of processing work orders using a manual method is very low.

[0063] In view of the technical problem of low processing efficiency of work order processing in the related art, the present application provides a work order processing method, and the general idea is as follows:

[0064] Obtain work order data of a current work order; perform scene type reasoning using a historical reasoning model or an online reasoning model according to the work order data of the current work order, to obtain a scene type to which the current work order belongs; wherein the online reasoning model is obtained by updating and training the historical reasoning model based on the work order data of the current work order and work order data of historical work orders; perform element information extraction using an information extraction model according to the work order data of the current work order, to obtain element information of the current work order; generate a work order reply text according to the scene type and the element information, to obtain a work order processing result of the current work order.

[0065] Through the above technical solution, autonomous reply of an operation and maintenance work order is realized, compared with a manual work order processing method, not only a large amount of manpower and time cost is reduced, but also work order processing efficiency is improved, thereby customer satisfaction can be improved.

[0066] The work order processing method, device, equipment and storage medium provided by the present application will be described in detail below with reference to the accompanying drawings, through specific embodiments and implementation manners.

[0067] Embodiment one

[0068] Referring to the flowchart of Figure 1 , the first embodiment of the work order processing method of the present application is proposed, and the work order processing method is applied to a work order processing device.

[0069] The work order processing device refers to a terminal device or a network device capable of realizing network connection, which can be a terminal device such as a mobile phone, a computer, a tablet computer, a portable computer, and an embedded industrial computer, or a network device such as a server and a cloud platform.

[0070] As shown in FIG. 1, it is a schematic diagram of a hardware structure of the work order processing device. The work order processing device can include a processor 1001 such as a CPU (Central Processing Unit), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Figure 2

[0071] Specifically, the communication bus 1002 is used to realize the connection and communication among the components; the user interface 1003 is used to connect the client and communicate data with the client, and the user interface 1003 can include an output unit and an input unit; the network interface 1004 is used to connect the background server and communicate data with the background server, and the network interface 1004 can include an input / output interface; the memory 1005 is used to store various types of data, which can include, for example, instructions of any application program or method in the work order processing device, and application-related data, and the memory 1005 can be an internal memory; optionally, the memory 1005 can also be a storage device independent of the processor 1001, and the memory 1005 can include an operating system, a network communication module, a user interface module, and a work order processing program. Figure 2

[0072] Obtaining work order data of a current work order;

[0073] According to the work order data of the current work order, performing scene type reasoning by using a historical reasoning model or an online reasoning model to obtain a scene type to which the current work order belongs; wherein the online reasoning model is obtained by updating and training the historical reasoning model based on the work order data of the current work order and work order data of historical work orders;

[0074] According to the work order data of the current work order, performing element information extraction by using an information extraction model to obtain element information of the current work order;

[0075] Generating a work order reply text according to the scene type and the element information to obtain a work order processing result of the current work order.

[0076] Term explanation:

[0077] ​​BERT algorithm: Bidirectional Encoder Representation from Transformers, a bidirectional encoding representation algorithm based on Transformer, with strong language representation and feature extraction capabilities, can extract more rich text semantic features and generate dynamic word vector features, and can solve the problem of polysemy; fine-tune the pre-training model based on the BERT algorithm, and when the sample data is small, it can also complete classification and other tasks well.

[0078] BiLSTM algorithm: Bidirectional Long Short Term Memory Networks, a bidirectional long short-term memory network algorithm, which is composed of a forward LSTM and a backward LSTM, can well capture long-distance bidirectional semantic dependency relationships, and contains a gate structure that can well solve the gradient problem.

[0079] CRF algorithm: Conditional Random Field, a conditional random field algorithm, often used for sequence labeling tasks, can obtain the globally optimal labeling sequence in the entity recognition task, so as to more accurately extract entities.

[0080] Based on the above work order processing device, the work order processing method of the present embodiment will be described in detail in combination with the flowchart shown in Figure 1 The method can include the following steps:

[0081] Step S100: Obtain the work order data of the current work order.

[0082] Specifically, the current work order can be any operation and maintenance work order in the operation and maintenance system that needs to obtain related replies, such as a complaint work order. The work order data can include strings related to work order content. In actual application, the work order content can include business content and order opinion, and the specific components can also be configured according to actual needs, but are all in the form of strings.

[0083] In the present embodiment, the work order processing device obtains the work order data of the work order, i.e. the work order data of the current work order.

[0084] Step S300: According to the work order data of the current work order, use the historical reasoning model or the online reasoning model to perform scene type reasoning to obtain the scene type to which the current work order belongs; wherein the online reasoning model is obtained by updating and training the historical reasoning model based on the work order data of the current work order and the work order data of the historical work order.

[0085] Specifically, the historical reasoning model and the online reasoning model are both models constructed based on a deep learning algorithm, which can be constructed according to a BERT algorithm and a BiLSTM algorithm. The algorithm model can be used to perform scene type reasoning on the work order. After the work order processing device obtains the work order data of the current work order, the historical reasoning model or the online reasoning model can be selected according to a preset rule to process the work order data, specifically to perform scene type reasoning, to obtain the scene type to which the current work order belongs. The selection of the historical reasoning model or the online reasoning model according to the preset rule can be direct matching based on the work order type, direct selection based on the specific content of other parameters, or selection based on the training state of the online reasoning model. The actual needs can be set.

[0086] When the historical reasoning model is used to perform scene type reasoning, it can be directly executed after obtaining the work order data of the current work order. When the online reasoning model is used to perform scene type reasoning, the historical reasoning model needs to be updated and trained using the work order data of the current work order and the work order data of the historical work order, and then the online reasoning model is obtained, and scene type reasoning is performed on the work order data of the current work order. It can be understood that when the online reasoning model is still in the training process and has not been trained, the historical reasoning model can be selected to perform scene type reasoning to avoid waiting for too long time and affecting the work order processing efficiency. Then when the online reasoning model is trained, the online reasoning model can be selected to perform scene type reasoning to avoid the situation that the training data of the historical reasoning model is less and the model processing accuracy is low, which may affect the accuracy of the work order processing result when the historical reasoning model is applied to process the work order. Compared with the historical reasoning model, the online reasoning model has more training data, and the algorithm model obtained by training is also a more accurate algorithm model, so the output result of the online reasoning model is also more accurate.

[0087] In this embodiment, after the work order processing device selects the reasoning model according to the preset rule, the work order data of the current work order can be input into the selected historical reasoning model or online reasoning model to perform scene type reasoning and output scene type classification result, i.e., to obtain the scene type to which the current work order belongs.

[0088] Step S500: According to the work order data of the current work order, the element information extraction model is used to extract element information, and the element information of the current work order is obtained.

[0089] Specifically, the information extraction model is a model constructed based on a deep learning algorithm, which can be constructed based on a BERT algorithm, a BiLSTM algorithm, and a CRF algorithm. The algorithm model can be used for element information extraction of the work order. The information extraction model realizes the extraction of element information, and the extraction rule is generally fixed, so the information extraction model does not need to be trained online, and an offline training method can be selected. When the information extraction model is needed, the trained information extraction model is directly used.

[0090] In this embodiment, the work order processing device can input the work order data of the current work order into the constructed information extraction model to extract the element information, and output the element information content, i.e., obtain the element information of the current work order. The algorithm model used in the scene type reasoning needs to be continuously updated and trained online, while the algorithm model used in the element information extraction can not be updated and trained offline, which can reduce the resource occupation of the work order processing device to avoid excessive resource occupation affecting the work order processing efficiency.

[0091] Step S700: generating a work order reply text according to the scene type and the element information, and obtaining a work order processing result of the current work order.

[0092] Specifically, the work order reply text can be generated by the work order processing device. After obtaining the scene type to which the current work order belongs and the element information of the current work order, the work order processing device can automatically find the corresponding text reply template according to the scene type and the element information, and generate the corresponding work order reply text as the work order processing result of the current work order.

[0093] The method can be applied to a work order processing device, and can also be applied to a work order processing system composed of two devices. In the work order processing system, the work order can be received by the operation and maintenance platform of the first terminal, including the current work order, and then the current work order is sent to the algorithm program of the second terminal, and the algorithm program completes the specific work order processing process, and returns the work order processing result to the operation and maintenance platform of the first terminal after completing the work order processing. The operation and maintenance platform assembles a text for reply according to the work order processing result, and saves the work order processing result. When the operation and maintenance platform sends the current work order to the algorithm program, it can be sent in the form of an HTTP request or a JSON data packet. The algorithm program can also train related algorithm models to apply the obtained algorithm models to the work order processing process. When the algorithm program returns the work order processing result to the operation and maintenance platform of the first terminal, it can be sent in the form of feedback information of a response request or a JSON data packet. When the operation and maintenance platform saves the work order processing result, it can also filter and store the work order data of the work order. More functions can also be set according to actual needs, which will not be described here.

[0094] The work order processing method provided by the embodiment realizes autonomous reply of the operation and maintenance work order, compared with the manual work order processing mode, not only reduces a large amount of labor and time cost, but also improves work order processing efficiency, so that customer satisfaction can be improved; the historical inference model can be updated and trained according to the work order data of the current work order and the work order data of the historical work order to obtain an online inference model, a function of uninterrupted model training is provided, when the model is trained, the application of the model in the scene type inference process is not disturbed, the robustness and overall performance of the method are improved.

[0095] Embodiment two

[0096] Based on the same inventive concept, referring to Figures 3 to 4 , the second embodiment of the work order processing method of the present application is proposed, which is applied to a work order processing device.

[0097] The work order processing method of the present embodiment will be described in detail below in combination with the flowchart shown in Figure 3 . The method can include the following steps:

[0098] Step S100: obtaining work order data of a current work order.

[0099] Specifically, the work order processing device can collect or receive a work order processing request in real time, analyze the request to obtain the work order data of the current work order.

[0100] In the present embodiment, the work order processing request can be a JSON request data packet, and can also contain specific request content, such as model iterative training or work order autonomous reply.

[0101] When the work order processing request is a model iterative training request, after obtaining the work order data of the current work order, steps S200, S210-S260 can be performed. The data packet of the model iterative training request can encapsulate the training data ID, request type parameter, training data path information and the like required for model training. For example, the user end can send a model iterative training request to the work order processing device every two weeks, the work order processing device correspondingly receives the request and analyzes the data packet to obtain the training data ID, request type parameter, training data path information and the like required for model training, so that steps S200, S210-S260 can be performed for model training.

[0102] When the work order processing request is a work order autonomous reply request, after obtaining the work order data of the current work order, steps S200, S310 and subsequent steps S400-S700 can be executed. The data packet of the work order autonomous reply request can encapsulate the work order content, request type parameter, etc. For example, the request can be initiated when a new complaint work order is added. After parsing the data packet, the work order content, request type parameter, etc. are obtained, so that steps S200, S310 and S400-S700 can be executed to process the work order to obtain the work order processing result.

[0103] By classifying the work order processing request into two categories, various user needs can be met, such as the need for periodic automatic updating of the model at certain intervals, which may not require autonomous reply to the current work order, and the need for autonomous reply to the current work order only without updating and training the inference model. The specific selection can be made according to actual needs.

[0104] Step S200: Preprocessing the work order data of the current work order to obtain preprocessed work order data; wherein the preprocessing includes dynamically adjusting the work order content string in the work order data, and the dynamic adjustment includes removing preset characters, removing preset vocabularies and / or removing preset digit string sequences.

[0105] Specifically, the preset characters are, for example, some special characters, the preset vocabularies are, for example, some stop words, and the preset digit string sequences are, for example, some digit strings that meet the set requirements, such as digit strings with a length greater than a preset length.

[0106] In this embodiment, the work order content can include business content and assignment opinion, and the strings of the business content and the assignment opinion constitute the work order content string. When the length of the work order content string exceeds the set text sequence length, the work order content string can be dynamically adjusted to implement work order data preprocessing, for example, the work order content string can be processed to remove special characters, stop words and digit string sequences with a length greater than 5 to obtain the preprocessed work order content string, i.e. the preprocessed work order data.

[0107] The dynamic adjustment of the work order content string can be the dynamic adjustment of the composition ratio of the business content and the assignment opinion. Let the maximum text sequence length of the work order content string be S, the length of the business content be S1, the length of the assignment opinion be S2, and W1 and W2 be variables in the range of 0-1. When S1+S2<S, the string is not truncated, the business content and the assignment opinion are retained, i.e. the length of the work order content string remains unchanged; otherwise, the work order content string satisfies S=W1*S1+W2*S2, wherein W1 and W2 are the optimal values obtained using the randomized search algorithm (RandomizedSearchCV).

[0108] In this embodiment, when dynamically adjusting the work order content string, the proportion of business content and dispatch opinions is dynamically adjusted. While retaining key features, this improves the quality of work order data used for model training or subsequent input into the model. This embodiment can improve the text representation capability of work orders and reduce the interference of irrelevant and noisy data during feature extraction.

[0109] The work order processing method in this embodiment specifically includes two parts: model iterative training and work order autonomous response. In order to improve the work order processing performance and to avoid interfering with the application of the algorithm model based on the work order autonomous response part during algorithm model training, a non-disruptive training function can be added to this method.

[0110] In one implementation, after step S100 "obtain the work order data of the current work order" and before step S300, the method may further include a model training process.

[0111] Specifically, such as Figure 4 The detailed flowchart shown illustrates that the model training process may include:

[0112] Step S210: Obtain the configuration file and set the training flag in the configuration file to 0.

[0113] Specifically, step S210 can be executed after step S100 or after step S200. Before model training begins, the work order processing device can read the pre-stored configuration file, set the training flag `train_flag` to 0, and then update and train the historical inference model using the current work order data and historical work order data to obtain the online inference model. Then, the training flag `train_flag` is set to 1 so that subsequent step S300 can select to use either the historical inference model or the online inference model based on the value of the training flag `train_flag`. It can be understood that during model training, before the online inference model is obtained, the training flag `train_flag` is still 0, and step S300 will automatically select the historical inference model for scene type inference; while after model training is completed and the online inference model is obtained, the training flag `train_flag` becomes 1, and step S300 can then select the online inference model for scene type inference.

[0114] In this embodiment, step S210 is executed after step S200. The preprocessed work order data for model training obtained in step S200 is combined with the preprocessed work order data for model training obtained in the past to perform model training.

[0115] Step S220: Based on the current work order data and the historical work order data, obtain the training dataset and the test dataset.

[0116] Specifically, the work order processing device can divide the real-time acquired work order data and the historical work order data to obtain a training data set and a test data set, the training data set is used for model training, and the test data set is used for model evaluation. Step S220 can also obtain the training data set and the test data set according to the preprocessed work order data and the historical work order data.

[0117] Optionally, the work order processing device can also store all the work order data, and correspondingly add work order data for each newly added work order, thereby realizing the addition and saving of the training data. When the work order processing of the previous work order ends and the work order reply text is generated, the relevant work order processing personnel can also audit the work order reply text in order to correct the error scene type, that is, to modify the label of the inference model. In a specific application, the work order processing device can save all the work order data added on the previous day and the corrected model parameters to a csv file in the specified training data storage path at zero o'clock the next day.

[0118] Step S230: updating and training the historical inference model according to the training data set to obtain an updated inference model; wherein the historical inference model is constructed based on the BERT algorithm and the BiLSTM algorithm and is trained by using the work order data of the historical work order.

[0119] Specifically, before updating and training the historical inference model, the training data can be obtained to train the inference model constructed based on the BERT algorithm and the BiLSTM algorithm to obtain the trained historical inference model. The training data can be the work order data of the historical work order.

[0120] In this embodiment, the acquisition of the training data can include sampling the unbalanced data in the work order data, and the process of training the inference model is as follows:

[0121] First, let Ri be the proportion of work orders of scene category i that need to be sampled, there are K scene categories in total, the number of work orders of the i-th category is Ni, and the total number of work orders is N, α is a coefficient, then

[0122]

[0123] When xi≤0, When xi>0, α=0;

[0124] According to the above formula, when the proportion of the i-th work order is greater than , down-sampling is performed, and the final number of work orders is Ni*Ri; when the proportion of the i-th work order is less than , over-sampling is performed, and the number of work orders to be replicated and increased is Ni*Ri, thereby obtaining the training data.

[0125] Then, based on the foregoing setting, the number of the i-th type of work order is Ni, the total number of work orders is N, there are K types of work order scenarios, That is, the proportion of the number of work orders of the current scenario category i is greater than The total number of work order scenario types is C, and the number Tr of the training data set can be:

[0126]

[0127] The number of the test data set is N-Tr, according to the ratio of Tr:(N-Tr), all the training data can be divided into the training data set and the test data set.

[0128] In the embodiment, by sampling the unbalanced data to obtain the training data for model training, and training the constructed inference model, the influence of easily favoring the scenario type with a large amount of algorithm model training data in the scenario type inference for unbalanced type data can be reduced, the problem of work order inference favoring the majority class of scenarios caused by unbalanced work order scenario types is solved, and the accuracy of scenario type inference is improved. That is, the embodiment can still train a better algorithm model in the case of a small amount of data.

[0129] Optionally, for the divided training data set, the above steps can be repeated to divide the training data set into a training set and a validation set, so as to complete the division of the training set, the validation set and the test data set.

[0130] Then, the optimal hyperparameter combination for model training can be obtained by using the validation set and the grid search algorithm (GridSearchCV), and the inference model constructed based on the BERT algorithm and the BiLSTM algorithm can be trained according to the optimal hyperparameter combination using the training set, to obtain a historical inference model.

[0131] The proportion of the training set, the validation set and the test set is dynamically adjusted, the scenario categories of the training data are unbalanced, and the proportion of the number of work orders of the scenario category is less than In the case that the proportion of the number of work orders of the scenario category is less than

[0132] Based on the historical inference model obtained above, after obtaining the training data set and the test data set according to the preprocessed work order data and the work order data of the historical work order, the work order processing device can update and train the historical inference model using the training data set to obtain an updated inference model.

[0133] Step S240: evaluating the updated inference model according to the test data set;

[0134] Step S250: If the evaluation is qualified, the updated inference model is determined as the online inference model, and the training flag bit is set to 1.

[0135] Step S260: If the evaluation is unqualified, the training flag bit is kept as 0.

[0136] Specifically, during the model training process, the updated inference model can be evaluated. After the evaluation is qualified, the updated inference model is determined as the online inference model in step S300, i.e., the training flag bit is set to 1, and step S300 is executed; otherwise, the inference model is not updated, and the historical inference model with better effect than the updated inference model is continued to be used in step S300, i.e., the training flag bit is kept as 0, and step S300 is executed.

[0137] Optionally, the work order processing device can also save the related training data during the model training process, for example, store the training ID, training start and end time, training evaluation result, training data start and end time, training data label type information, etc. into a database.

[0138] Further, step S240 "evaluating the updated inference model according to the test data set" can include:

[0139] Step S241: determining the accuracy of the updated inference model according to the test data set;

[0140] Step S242: judging whether the accuracy is greater than or equal to a preset threshold;

[0141] Step S243: if the accuracy is greater than or equal to the preset threshold, determining that the evaluation is qualified;

[0142] Step S244: if the accuracy is less than the preset threshold, determining that the evaluation is unqualified.

[0143] Specifically, the preset threshold can be the product of the historical optimal accuracy and a set threshold. The historical optimal accuracy can be the accuracy of the historical optimal inference model. Assuming that the historical optimal accuracy is ACC H and the preset threshold is k, the value range of k is [0, 1], and generally defaults to 0.95. Here, the preset threshold can be k*ACC H. Assuming that the accuracy of the updated inference model is ACC N, judging whether the accuracy is greater than or equal to the preset threshold is to judge whether ACC N≥k*ACC H is true. If yes, it can be determined that the evaluation is qualified; otherwise, it can be determined that the evaluation is unqualified.

[0144] In this embodiment, the model training is performed in a training non-stop manner, which can perform service such as work order scene type inference and work order element information extraction at the same time as training, thereby improving system robustness. The inference model constructed based on the BERT algorithm and the BiLSTM algorithm can not only extract rich text features, but also learn context semantics.

[0145] Step S300: performing scene type inference by using the historical inference model or the online inference model according to work order data of the current work order, to obtain a scene type to which the current work order belongs; wherein the online inference model is obtained by updating and training the historical inference model based on work order data of the current work order and work order data of historical work orders.

[0146] Specifically, step S300 can include:

[0147] Step S310: performing scene type inference by using the historical inference model or the online inference model according to the preprocessed work order data, to obtain a scene type to which the current work order belongs.

[0148] In this embodiment, the work order processing device can call the historical inference model or the online inference model to perform inference after obtaining the preprocessed work order data, and obtain a scene type obtained by inference, i.e., a scene type to which the current work order belongs.

[0149] In an implementation, “performing scene type inference by using the historical inference model or the online inference model to obtain a scene type to which the current work order belongs” in step S310 can include:

[0150] Step S311: determining content of a training flag in a configuration file acquired in real time;

[0151] Step S312: when the training flag is 0, performing scene type inference by using the historical inference model to obtain a scene type to which the current work order belongs;

[0152] Step S313: when the training flag is 1, performing scene type inference by using the online inference model to obtain a scene type to which the current work order belongs, and replacing the historical inference model with the online inference model.

[0153] Specifically, the work order processing device can read the configuration file in real time, detect the training flag, select the historical inference model to perform scene type inference if the training flag is 0, select the online inference model to perform scene type inference if the training flag is 1, and replace the historical inference model with the online inference model, so as to use the online inference model as the historical inference model for processing a next work order.

[0154] Step S400: constructing an information extraction model based on a BERT algorithm, a BiLSTM algorithm and a CRF algorithm.

[0155] Specifically, the information extraction model can be an algorithm model trained offline, and does not need to be trained online, so the information extraction model only needs to be constructed before step S500. The model parameters of the information extraction model can be set according to actual needs, which are not limited here.

[0156] Step S500: According to the work order data of the current work order, the element information of the current work order is obtained by using the information extraction model to extract the element information.

[0157] Specifically, the element information can be time, user type, user ID, and the like, which are not limited here. The work order processing device can directly extract the element information from the original work order data by using the information extraction model to obtain the element information of the current work order.

[0158] In this embodiment, the information extraction model constructed based on the BERT algorithm, the BiLSTM algorithm and the CRF algorithm can more accurately obtain a globally optimal annotation sequence, so as to extract optimal element information such as time information, customer ID, and the like, which are not limited here.

[0159] Step S700: Generating a work order reply text according to the scene type and the element information to obtain a work order processing result of the current work order.

[0160] Specifically, step S700 can include:

[0161] Step S710: Obtaining a confidence degree of using a historical reasoning model or an online reasoning model to reason the scene type;

[0162] Step S720: When the confidence degree is greater than or equal to a preset threshold value, automatically generating a work order reply text according to the scene type and the element information;

[0163] Step S730: When the confidence degree is less than the preset threshold value, obtaining a manually written work order reply text;

[0164] Step S740: Determining the automatically generated work order reply text or the manually written work order reply text as the work order processing result of the current work order.

[0165] Specifically, the work order processing device can obtain the confidence of the scene type inference when obtaining the scene type to which the current work order belongs, and determine whether to perform autonomous reply; if the confidence is greater than or equal to a preset threshold value, autonomous reply is performed, at this time, the work order data can be automatically queried and checked according to the obtained scene type and element information, and the reply text, that is, the work order reply text, is generated according to the text template corresponding to the scene type, so as to be used as the work order processing result of the current work order for reply; if the confidence is less than the preset threshold value, the work order processing personnel can be transferred to perform manual reply, and the manually written work order reply text is obtained to be used as the work order processing result of the current work order for reply.

[0166] In the embodiment, the user can customize the reply threshold, that is, the preset threshold value of the confidence, and can transfer the work order with a confidence less than 0.4 in the scene type inference to be a special and complex scene for manual processing, thereby improving the correctness of the work order reply.

[0167] Optionally, the work order processing device can also realize storage and query of work order data, and specifically, the work order content, work order ID, request type parameter, inferred scene type, confidence of scene type inference, extracted element information, generated work order reply text and the like can be saved to a database, so as to facilitate subsequent algorithm model analysis and optimization, and the user can also query.

[0168] For brevity of the description, more implementation details of the above method steps are not repeated here, and the description of the specific implementation of the embodiment one can be referred to.

[0169] The work order processing method provided in the embodiment includes model iterative training and work order autonomous reply, in the model iterative training, the inference model can be trained using the newly added work order data and the historical work order data to continuously improve the inference ability and generalization performance of the algorithm model; in the work order autonomous reply, the scene type inference and element information extraction of the work order can be completed online to assemble the work order reply text of the current work order for reply. The method can better weaken the influence of noise data, extract more rich long-distance context semantic features, improve the algorithm model performance of scene type inference and element information extraction, construct a reliable and stable work order processing system, and can be applied to various industries that need to process work orders to improve the processing efficiency of the work order and the satisfaction of the customer.

[0170] Embodiment three

[0171] Based on the same inventive concept, referring to Figure 5 , the first embodiment of the work order processing device is proposed, which can be a virtual device and applied to a work order processing device.

[0172] The following will be described in detail with reference to the functional module diagram Figure 5 The work order processing device provided by the embodiment can include:

[0173] The data acquisition module is configured to acquire work order data of a current work order.

[0174] The scene classification module is configured to perform scene type reasoning on the work order data of the current work order by using a historical reasoning model or an online reasoning model to obtain a scene type to which the current work order belongs; the online reasoning model is obtained by updating and training the historical reasoning model based on the work order data of the current work order and work order data of historical work orders.

[0175] The information extraction module is configured to perform element information extraction on the work order data of the current work order by using an information extraction model to obtain element information of the current work order.

[0176] The result generation module is configured to generate a work order reply text according to the scene type and the element information to obtain a work order processing result of the current work order.

[0177] Further, the device can further include:

[0178] The preprocessing module is configured to pre-process the work order data of the current work order to obtain pre-processed work order data; the pre-processing includes dynamic adjustment of a work order content string in the work order data, and the dynamic adjustment includes removal of preset characters, removal of preset vocabularies, and / or removal of preset numerical string sequences.

[0179] Correspondingly, the scene classification module is further configured to perform scene type reasoning on the pre-processed work order data by using the historical reasoning model or the online reasoning model to obtain the scene type to which the current work order belongs.

[0180] Further, the device can further include a model training module, which can include:

[0181] The configuration acquisition unit is configured to acquire a configuration file and set a training flag in the configuration file to 0.

[0182] The data set acquisition unit is configured to obtain a training data set and a test data set according to the work order data of the current work order and the work order data of the historical work orders.

[0183] The update training unit is configured to update and train the historical reasoning model according to the training data set to obtain an updated reasoning model; the historical reasoning model is constructed based on a BERT algorithm and a BiLSTM algorithm and is trained by using the work order data of the historical work orders.

[0184] The model evaluation unit is configured to evaluate the updated inference model according to the test data set; if the evaluation is qualified, the updated inference model is determined as the online inference model, and the training flag bit is set to 1; if the evaluation is unqualified, the training flag bit is kept as 0.

[0185] Further, the model evaluation unit is further configured to determine the accuracy of the updated inference model according to the test data set; determine whether the accuracy is greater than or equal to a preset threshold; if the accuracy is greater than or equal to the preset threshold, it is determined that the evaluation is qualified; if the accuracy is less than the preset threshold, it is determined that the evaluation is unqualified.

[0186] Further, the scene classification module can include:

[0187] The configuration determination unit is configured to determine the content of the training flag bit in the real-time obtained configuration file.

[0188] The first inference unit is configured to, when the training flag bit is 0, perform scene type inference by using the historical inference model to obtain the scene type to which the current work order belongs.

[0189] The second inference unit is configured to, when the training flag bit is 1, perform scene type inference by using the online inference model to obtain the scene type to which the current work order belongs, and replace the historical inference model with the online inference model.

[0190] Further, the device can further include:

[0191] The model construction module is configured to construct the information extraction model based on the BERT algorithm, the BiLSTM algorithm and the CRF algorithm.

[0192] Further, the result generation module can include:

[0193] The confidence calculation unit is configured to obtain the confidence of the scene type inference by using the historical inference model or the online inference model.

[0194] The automatic generation unit is configured to, when the confidence is greater than or equal to a preset threshold value, automatically generate a work order reply text according to the scene type and the element information.

[0195] The manual intervention unit is configured to, when the confidence is less than the preset threshold value, obtain a manually written work order reply text.

[0196] The result determination unit is configured to determine the automatically generated work order reply text or the manually written work order reply text as the work order processing result of the current work order.

[0197] It should be noted that the functions and technical effects of the modules in the work order processing device provided in the embodiments can refer to the descriptions of the specific embodiments of the work order processing method of the present application. For the sake of brevity of the description, the details are not described here.

[0198] Embodiment Four

[0199] Based on the same inventive concept, referring to the hardware structure diagram of Figure 2 The work order processing device provided in the embodiments can include a processor and a memory, and the memory stores a work order processing program. When the work order processing program is executed by the processor, all or part of the steps of the work order processing method of the present application are implemented.

[0200] Specifically, the work order processing device refers to a terminal device or a network device that can realize network connection. It can be a terminal device such as a mobile phone, a computer, a tablet computer, a portable computer, an embedded industrial computer, etc. It can also be a network device such as a server, a cloud platform, etc.

[0201] It can be understood that the work order processing device can further include a communication bus, a user interface and a network interface. Among them, the communication bus is used to realize the connection communication between these components; the user interface is used to connect the client and communicate data with the client, and the user interface can include output units such as display screens, speakers, etc., input units such as keyboards, microphones, etc.; the network interface is used to connect the background server and communicate data with the background server, and the network interface can include input / output interfaces, such as standard wired interfaces, wireless interfaces such as Wi-Fi interfaces; the memory is used to store various types of data, which can include, for example, instructions of any application program or method in the work order processing device, and application-related data, and the memory can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), magnetic storage, flash memory, magnetic disk or optical disk, etc.; optionally, the memory can also be a storage device independent of the processor; the processor is used to call the work order processing program stored in the memory and execute the work order processing method as described above, and the processor can be an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor or other electronic elements, used to execute all or part of the steps of the work order processing method as described above.

[0202] It should be noted that Figure 2The hardware structure shown in the figure does not constitute a limitation on the ticket processing device of the present application, and can include more or less components than shown, or combine certain components, or arrange different components.

[0203] Embodiment Five

[0204] Based on the same inventive concept, the embodiment provides a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a magnetic memory, a magnetic disk, an optical disk, a server, etc., which stores a computer program. The computer program can be executed by one or more processors, and when the computer program is executed by the processor, it can implement all or part of the steps of each embodiment of the ticket processing method of the present application.

[0205] It should be noted that the above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above embodiments are only optional embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent flow conversion within the inventive concept of the present application, or direct or indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A work order processing method, characterized by, The method comprises: obtaining work order data of a current work order; performing scene type inference on the work order data of the current work order by using a historical inference model or an online inference model to obtain a scene type to which the current work order belongs; wherein the online inference model is obtained by updating and training the historical inference model based on the work order data of the current work order and work order data of historical work orders; performing element information extraction on the work order data of the current work order by using an information extraction model to obtain element information of the current work order; generating a work order reply text according to the scene type and the element information to obtain a work order processing result of the current work order; the step of performing scene type inference on the work order data of the current work order by using the historical inference model or the online inference model to obtain the scene type to which the current work order belongs comprises: determining the content of a training flag in a configuration file obtained in real time; when the training flag is 0, performing scene type inference by using the historical inference model to obtain the scene type to which the current work order belongs; when the training flag is 1, performing scene type inference by using the online inference model to obtain the scene type to which the current work order belongs, and replacing the historical inference model with the online inference model.

2. The work order processing method of claim 1, wherein, After the step of obtaining the work order data of the current work order, the method further comprises: preprocessing the work order data of the current work order to obtain preprocessed work order data; wherein the preprocessing comprises dynamically adjusting a work order content string in the work order data, and the dynamic adjustment comprises removing preset characters, removing preset vocabularies, and / or removing preset numerical string sequences; the step of performing scene type inference on the work order data of the current work order by using the historical inference model or the online inference model to obtain the scene type to which the current work order belongs comprises: performing scene type inference on the preprocessed work order data by using the historical inference model or the online inference model to obtain the scene type to which the current work order belongs.

3. The work order processing method of claim 1 or 2, wherein, After the step of obtaining the work order data of the current work order, the method further comprises: obtaining a configuration file and setting the training flag in the configuration file to 0; obtaining training data sets and test data sets from the work order data of the current work order and work order data of historical work orders; updating and training the historical inference model based on the training data sets to obtain an updated inference model; wherein the historical inference model is constructed based on a BERT algorithm and a BiLSTM algorithm and is trained by using the work order data of historical work orders; evaluating the updated inference model based on the test data sets; if the evaluation is qualified, determining the updated inference model as the online inference model and setting the training flag to 1; if the evaluation is unqualified, keeping the training flag as 0.

4. The work order processing method of claim 3, wherein, the step of evaluating the updated inference model based on the test data sets comprises: determining the accuracy of the updated inference model based on the test data sets; determining whether the accuracy is greater than or equal to a preset threshold; if the accuracy is greater than or equal to the preset threshold, determining that the evaluation is qualified; If the accuracy is less than a preset threshold, it is determined that the evaluation is unqualified.

5. The work order processing method of claim 1, wherein, Before the step of performing element information extraction on the current work order data by using an information extraction model to obtain element information of the current work order, the method further includes: The information extraction model is constructed based on a BERT algorithm, a BiLSTM algorithm, and a CRF algorithm.

6. The work order processing method of claim 1, wherein, The step of generating a work order reply text according to the scene type and the element information to obtain a work order processing result of the current work order includes: obtaining a confidence degree of performing scene type reasoning by using the historical reasoning model or the online reasoning model; when the confidence degree is greater than or equal to a preset threshold value, automatically generating a work order reply text according to the scene type and the element information; when the confidence degree is less than the preset threshold value, obtaining a work order reply text manually written; determining the automatically generated work order reply text or the manually written work order reply text as the work order processing result of the current work order.

7. A work order processing apparatus, characterized by comprising: The device includes: a data acquisition module configured to acquire work order data of a current work order; a scene classification module configured to perform scene type reasoning on the work order data of the current work order by using a historical reasoning model or an online reasoning model to obtain a scene type to which the current work order belongs, wherein the online reasoning model is obtained by updating and training the historical reasoning model based on the work order data of the current work order and work order data of historical work orders; an information extraction module configured to perform element information extraction on the work order data of the current work order by using an information extraction model to obtain element information of the current work order; a result generation module configured to generate a work order reply text according to the scene type and the element information to obtain a work order processing result of the current work order; The scene classification module includes: a configuration determination unit configured to determine content of a training flag in a real-time acquired configuration file; a first reasoning unit configured to, when the training flag is 0, perform scene type reasoning by using a historical reasoning model to obtain a scene type to which a current work order belongs; a second reasoning unit configured to, when the training flag is 1, perform scene type reasoning by using an online reasoning model to obtain a scene type to which a current work order belongs, and replace the historical reasoning model with the online reasoning model.

8. A work order processing apparatus, characterized by comprising: The device includes a processor and a memory, and the memory stores a work order processing program. When the work order processing program is executed by the processor, the work order processing method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program. When the computer program is executed by one or more processors, the work order processing method according to any one of claims 1 to 6 is implemented.

Images