Method, device and electronic equipment for determining work order problems

Abstract

The present application discloses a method, device and electronic device for determining work order problems. The method includes: obtaining a set of business objects corresponding to target text information in the work order; determining the process action corresponding to each business object in the business object set to form a first candidate word set; determining the information gain entropy value corresponding to the target text information based on at least different combinations of business objects in the business object set and process actions in the first candidate word set, determining the business object and process action corresponding to the maximum value of the information gain entropy value as the fusion feature of the target text information; determining the problem described in the work order based on the fusion feature. The present application solves the technical problem that the problem of identifying the work order description through keyword matching ignores the potential semantic features of the work order.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular, to a method, device and electronic device for determining work order problems. Background Art

[0002] Traditional work order analysis mainly uses manual labeling or keyword matching to identify the business problems described in the work order. Manual labeling is inefficient and relies on personal experience. Keyword matching ignores the potential semantic features of the work order and can easily lose the intention of the work order.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method, device and electronic device for determining work order problems, so as to at least solve the technical problem that the potential semantic features of the work order are ignored when identifying the work order description through keyword matching.

[0005] According to one aspect of an embodiment of the present application, a method for determining a work order problem is provided, including: obtaining a set of business objects corresponding to target text information in the work order; determining a process action corresponding to each business object in the business object set to form a first candidate word set; determining an information gain entropy value corresponding to the target text information based on at least different combinations of business objects in the business object set and process actions in the first candidate word set, and determining the business object and process action corresponding to the maximum value of the information gain entropy value as a fusion feature of the target text information; and determining the problem described in the work order based on the fusion feature.

[0006] Optionally, determining the process action corresponding to each business object in the business object set includes: obtaining multiple label sequences corresponding to the target text information, wherein the label sequence is a label type contained in the named entity recognition model; determining the transfer scores corresponding to the multiple label sequences at least based on the weight values ​​corresponding to the business objects in the business object set; determining the probability prediction values ​​corresponding to the multiple label sequences based on the transfer scores, and determining the label sequence corresponding to the maximum value in the probability prediction values ​​as the process action corresponding to the business object.

[0007] Optionally, transfer scores corresponding to multiple label sequences are determined at least based on weight values ​​corresponding to business objects in the business object set, including: obtaining the score of the first label sequence corresponding to the target text information at the target moment, and obtaining the transition probability between the first label sequence and the second label sequence at the target moment and the moment before the target moment; transfer scores corresponding to multiple label sequences are determined at least based on the score of the first label sequence, the transition probability and the weight values ​​corresponding to the business objects in the business object set, wherein the weight values ​​corresponding to the business objects in the business object set are the weight values ​​of the business objects in the target text information, and the weight values ​​are determined at least by the number of times the business objects appear in the target text information.

[0008] Optionally, after determining the process action corresponding to each business object in the business object set, the method also includes: performing word segmentation processing on the target text information to obtain a second candidate word set, wherein the second candidate word set includes at least one of the following: a normal behavior word set, an abnormal behavior word set and an unfamiliar word set; determining a first degree of association between the target candidate word in the unfamiliar word set and the normal behavior words in the normal behavior word set, and determining a second degree of association between the target candidate word and the abnormal behavior words in the abnormal behavior word set; when the first degree of association is greater than the second degree of association, determining the target candidate word to be a normal behavior word; when the first degree of association is less than the second degree of association, determining the target candidate word to be an abnormal behavior word.

[0009] Optionally, determining a first degree of association between target candidate words in the unfamiliar word set and normal behavior words in the normal behavior word set includes: obtaining a first work order number of target normal behavior words and target candidate words that co-occur in the normal behavior word set, a second work order number in which the target candidate words appear, a third work order number in which the target normal words appear, and the total number of work orders; determining a first probability of occurrence of the target candidate words and the target normal behavior words based on the first work order number, the second work order number, the third work order number, and the total number of work orders; obtaining the inverse of the average value of the distance between the target candidate words and all normal behavior words in the normal behavior word set, recorded as a first coefficient; and determining a first degree of association between the target candidate words and the target normal words based on the first coefficient and the first probability.

[0010] Optionally, determining a second degree of association between a target candidate word and an abnormal behavior word in the abnormal behavior word set includes: obtaining a fourth work order number of target abnormal behavior words and target candidate words that co-appear in the abnormal behavior word set, and a fifth work order number in which the target abnormal word appears; determining a second probability of occurrence of the target candidate word and the target abnormal behavior word based on the second work order number, the fourth work order number, the fifth work order number, and the total number of work orders; obtaining the inverse of an average value of the distance between the target candidate word and all abnormal behavior words in the abnormal behavior word set, recorded as a second coefficient; and determining a second degree of association between the target candidate word and the target abnormal word based on the second coefficient and the second probability.

[0011] Optionally, determining the information gain entropy value corresponding to the target text information includes: obtaining a third probability that the target candidate word appears in the first candidate word set, and obtaining a fourth probability that the target candidate word appears in the abnormal behavior word set; and determining the information gain entropy value corresponding to the target candidate word in the target text information based at least on the third probability and the fourth probability.

[0012] According to another aspect of an embodiment of the present application, a device for determining a work order problem is also provided, including: an acquisition module, used to acquire a set of business objects corresponding to target text information in the work order; a first determination module, used to determine the process action corresponding to each business object in the business object set to form a first candidate word set; a second determination module, used to determine the information gain entropy value corresponding to the target text information based on at least different combinations of business objects in the business object set and process actions in the first candidate word set, and determine the business object and process action corresponding to the maximum value of the information gain entropy value as a fusion feature of the target text information; a third determination module, used to determine the problem described in the work order based on the fusion feature.

[0013] According to another aspect of the embodiments of the present application, an electronic device is also provided, including: a memory for storing program instructions; a processor, connected to the memory, for executing program instructions to implement the following functions: obtaining a set of business objects corresponding to the target text information in the work order; determining the process action corresponding to each business object in the business object set to form a first candidate word set; determining the information gain entropy value corresponding to the target text information based on at least different combinations of business objects in the business object set and process actions in the first candidate word set, and determining the business object and process action corresponding to the maximum value of the information gain entropy value as the fusion feature of the target text information; determining the problem described in the work order based on the fusion feature.

[0014] According to another aspect of the embodiment of the present application, a non-volatile storage medium is provided, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned work order problem determination method.

[0015] In an embodiment of the present application, a business object set corresponding to the target text information in the work order is obtained; the process action corresponding to each business object in the business object set is determined to form a first candidate word set; the information gain entropy value corresponding to the target text information is determined based on at least different combinations of business objects in the business object set and process actions in the first candidate word set, and the business object and process action corresponding to the maximum value of the information gain entropy value are determined as the fusion feature of the target text information, thereby achieving the purpose of determining the problem described in the work order based on the fusion feature, thereby achieving the technical effect of improving the recognition efficiency of work order problems, and further solving the technical problem of ignoring the potential semantic features of the work order when identifying the work order description through keyword matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 It is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for determining a work order problem according to one embodiment of the present application;

[0018] Figure 2 is a flow chart of a method for determining a work order problem according to an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of a syntactic dependency analysis algorithm according to an embodiment of the present application;

[0020] Figure 4 is a schematic diagram of determining a process action corresponding to a business object according to an embodiment of the present application;

[0021] Figure 5 is a flow chart for determining a first degree of association between a target candidate word and a target normal word according to an embodiment of the present application;

[0022] Figure 6 is a flow chart of determining a second degree of association between a target candidate word and a target abnormal word according to an embodiment of the present application;

[0023] Figure 7 It is a structural diagram of a device for determining a work order problem according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] In order to better understand the embodiments of the present application, some nouns or terms that appear in the process of describing the embodiments of the present application are explained as follows:

[0027] Co-occurrence window: Co-occurrence means co-occurrence. It determines whether a word appears near another word. The so-called near is a concept of a moving window. After defining the radius of the window (the distance from the center word to the edge), the number of words that appear within the radius is the co-occurrence window.

[0028] Traditional work order analysis uses manual labeling, which has the following advantages: 1. The technical threshold and the amount of data to be processed are relatively low, and conclusions can be obtained quickly; 2. Traditional manually processed work orders discover work order anomalies through manual judgment, and the recognition accuracy is high.

[0029] However, the traditional work order analysis method also has the following shortcomings, including: 1. The traditional manual work order processing is to check abnormal work orders through manual screening methods, and the work order problems are relatively superficial, and can only find work order problems that are visible to the naked eye. When the scene and semantics change, it is impossible to keenly dig deep problems; 2. The traditional manual work order analysis method lacks quantitative analysis methods, and it is difficult to dig out problems from a global perspective, and it is also difficult to draw convincing conclusions about work order problems; 3. The traditional manual work order processing efficiency is low. With the increase in the number of business work orders, the labor cost consumed is large, and it is urgent to realize intelligent mining of work order problems. In order to solve the above problems, the embodiments of the present application provide corresponding solutions, which are described in detail below.

[0030] The method for determining work order problems provided in the embodiments of the present application may be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for determining a work order problem. Figure 1 As shown, the computer terminal 10 (or electronic device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0031] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or electronic device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0032] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the method for determining work order problems in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the method for determining the work order problems described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or electronic device).

[0035] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or electronic device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the above-described computer device (or electronic device).

[0036] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a method for determining a work order problem. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Figure 2 is a flow chart of a method for determining a work order problem according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0038] Step S202, obtaining a business object set corresponding to the target text information in the work order;

[0039] In the mining of product, process or business objects, the effect of using statistical analysis methods alone is very poor. The text after syntactic dependency analysis clearly divides the structural features between sentences, which has a greater effect on the precise mining of business objects. Before using the syntactic dependency analysis algorithm to analyze the target text information of the work order, the target text information is preprocessed. The preprocessing includes the following operations: data import, data cleaning, adding new words for telecommunications services, Chinese word segmentation, removing stop words, and feature processing (TF-IDF weight calculation).

[0040] The input work order corpus is preprocessed, and then the processed text is input into the syntactic dependency analysis algorithm model to build a model. Finally, the dependency relationship between words is output, and the product or process object or business object is the head of the dependency relationship (HED), which is the core component of the sentence.

[0041] like Figure 3 The schematic diagram of the syntactic dependency analysis algorithm shown in the figure inputs the work order text, such as the text content in the example, and then introduces stop words and new business words to segment the text content to obtain the segmentation result in the example. The segmentation result is input into the syntactic dependency model (statistical grammatical structure analysis) for modeling and processing, and the structural relationship between sentences is analyzed. After modeling, prediction is performed to obtain the prediction result of the relationship in the sentence. The prediction result of the relationship in the sentence is the output result of the syntactic dependency model. For example, the business object is generally the core part of the sentence, which is the head of the question (HED) in the syntactic dependency algorithm. Figure 3 In the schematic diagram shown, "number portability" is defined as HED, that is, "number portability" is a service object.

[0042] Step S204, determining the process action corresponding to each business object in the business object set to form a first candidate word set;

[0043] Step S206, determining the information gain entropy value corresponding to the target text information at least according to different combinations of the business objects in the business object set and the process actions in the first candidate word set, and determining the business object and process action corresponding to the maximum value of the information gain entropy value as the fusion feature of the target text information;

[0044] Step S208, determining the problem described in the work order based on the fusion features.

[0045] In the above steps S202 to S208, the set of business objects in the work order can be identified through dependency syntax analysis, and then the business process actions can be captured through the improved BiLSTM-CRF algorithm. The optimal combination of business objects and process actions can be calculated based on information gain, and the improved SO-PMI algorithm can be used to predict abnormal work orders. Finally, the business scenario objects and process actions are determined to obtain the fusion features corresponding to the work order.

[0046] In step S204 of the above work order problem determination method, the process action corresponding to each business object in the business object set is determined, such as Figure 4 The flowchart shown specifically includes the following steps:

[0047] Step S402, obtaining multiple tag sequences corresponding to the target text information, wherein the tag sequences are tag types included in the named entity recognition model;

[0048] The following diagram shows how to annotate the encoded text according to the BIO method. B represents the beginning of the process action, I represents the end of the process action, and O represents the process action part. For example, the process action "cannot use WeChat" is marked as an abnormal payment process.

[0049] <pad>Hemp[1] < / pad> Trouble[2] Question[3] Next [4] Flag[5] Ship[6] Shop[7] O O O O O O O Work[8] No.[9] Order

[10] Single

[11] Pay

[12] Fee

[13] No

[14] O O O O O O B-pay

[15] Micro

[16] Letter

[17] Branch

[18] pay

[19] Treasure

[20] , I-pay I-pay I-pay O O O O

[21] able Silver

[22] Union

[23] and

[24] Now

[25] gold

[26] O O O O O O O

[0050] Step S404, determining transfer scores corresponding to the plurality of tag sequences at least according to weight values ​​corresponding to the business objects in the business object set;

[0051] Step S406 , determining probability prediction values ​​corresponding to multiple label sequences according to the transfer scores, and determining the label sequence corresponding to the maximum value among the probability prediction values ​​as the process action corresponding to the business object.

[0052] In the above step S404, the transfer scores corresponding to multiple label sequences are determined at least based on the weight values ​​corresponding to the business objects in the business object set, specifically including the following steps: obtaining the score of the first label sequence corresponding to the target text information at the target moment, and obtaining the transition probability between the first label sequence and the second label sequence at the target moment and the moment before the target moment; determining the transfer scores corresponding to multiple label sequences at least based on the score of the first label sequence, the transition probability and the weight values ​​corresponding to the business objects in the business object set, wherein the weight values ​​corresponding to the business objects in the business object set are the weight values ​​of the business objects in the target text information, and the weight values ​​are determined at least by the number of times the business objects appear in the target text information.

[0053] In the above steps S402 to S406, by improving the calculation method of the transfer score in the BiLSTM-CRF algorithm, the predicted probability of the label sequence corresponding to the process action is obtained from the transfer score:

[0054] The calculation formula of the transfer score in the improved CRF is as follows:

[0055]

[0056] In the above formula, w i,j : Indicates the weight value of the business object in the target text information (the weight value of the character is the tf-idf result of the proper word composed of the character); tf t,i,j : represents the number of times the j-th feature word appears in the i-th text at time t, that is, when there are multiple work orders, each work order corresponds to a target text information; Indicates w i,j The transpose of ; T: represents the length of the target text information; Indicates the target text information input; Indicates the label sequence corresponding to the target text information; [A] i,j : represents the probability of the ijth position in the transition probability matrix of CRF, that is, the transition probability between the first label sequence and the second label sequence at the target time and the previous time of the target time; [f] i,t : represents the score of the i-th label sequence at the t-th time, that is, the score of the first label sequence corresponding to the target text information at the target time; Represents the transfer score.

[0057] According to the transfer score, the label sequence is obtained The probability of each label sequence corresponding to the input sentence has a score and a probability. Finally, through the maximum likelihood transformation, the loss function is minimized, which is then converted into the maximum probability prediction of the label sequence, which is the predicted score probability of the label sequence corresponding to the process action. The calculation formula for the probability prediction value of the label sequence corresponding to the process action is as follows:

[0058]

[0059] In the above formula, Represents all possible label sequences corresponding to the input target text information; The label sequence probability prediction value corresponding to the input target text information.

[0060] According to the results of the syntactic dependency algorithm model, the business object is replaced with start, brought into the BiLSTM model to predict the first process action, and then predict the next process action..., and finally the combination of each process action is used as the first candidate word set. For example: when predicting "180****9090 Lida Machinery Company now needs to port your number, but the system prompts an exception. Please check and handle it. Thank you", first find the business object (that is, the beginning of the sentence) "port your number", predict the process action through the trained BiLSTM model, and take the word with the highest probability as the predicted process action. For example: the first candidate word set corresponding to the process action predicted in this sentence is {system support, prompt abnormality}.

[0061] In step S204 of the above-mentioned method for determining work order problems, after determining the process action corresponding to each business object in the business object set, the method also includes the following steps: performing word segmentation processing on the target text information to obtain a second candidate word set, wherein the second candidate word set includes at least one of the following: a normal behavior word set, an abnormal behavior word set, and an unfamiliar word set; determining a first degree of association between the target candidate word in the unfamiliar word set and the normal behavior words in the normal behavior word set, and determining a second degree of association between the target candidate word and the abnormal behavior words in the abnormal behavior word set; when the first degree of association is greater than the second degree of association, determining the target candidate word to be a normal behavior word; when the first degree of association is less than the second degree of association, determining the target candidate word to be an abnormal behavior word.

[0062] In the embodiment of the present application, abnormal behavior refers to the label value with the word "abnormal" in the label; judging from the word segmentation results, the words such as "cannot be accepted", "cannot be accepted", "cannot be accepted", etc. appear; the high-frequency words with negative words, such as "cannot", "no way", "cannot", "no", etc., can also be judged as abnormal behavior. According to the abnormal behavior determined, it is marked as an abnormal behavior label, and the non-abnormal data is marked as normal for model training.

[0063] The SO-PMI algorithm can be used to determine whether there is abnormal behavior in the business, but the existing SO-PMI algorithm has two shortcomings when determining abnormal business behavior: one is the problem caused by the size of the co-linear window; the other is the problem caused by the size of the corpus. In response to the problem of the co-occurrence window size, the embodiment of the present application introduces the point-to-point distance to improve the SO-PMI algorithm. Since the size of the co-occurrence window will affect the accuracy and coverage of the prediction, if the co-occurrence window is too large, the prediction coverage will increase and the prediction accuracy will decrease. If the co-occurrence window is too small, the prediction coverage will decrease and the prediction accuracy will increase. Suppose two words appear in the same document multiple times, and the point-to-point distance of the two words in the same document is represented by calculating the minimum value of the point-to-point distance of the two words in the same document. If the two words appear in different documents, take the average value M of the point-to-point distance of the two words in different documents. i , the specific improvements are as follows:

[0064]

[0065] In the above formula, n1: the number of normal behavior words contained in the normal behavior word set; n2: the number of abnormal behavior words contained in the abnormal behavior word set; PMI (word, Zword): represents the first probability of the target candidate word in the strange word set and the normal behavior word in the normal behavior word set; PMI (word, Yword): represents the second probability of the target candidate word in the strange word set and the abnormal behavior word in the normal behavior word set; Zword: represents the normal behavior word; Yword: represents the abnormal behavior word; The inverse of the average distance between the target candidate word and the normal behavior word, i.e., the first coefficient; The second coefficient is the reciprocal of the average value of the distance between the target candidate word and the abnormal behavior word; SO-PMI: represents the word judgment standard, SO-PMI(word)≥0, the word is judged as a normal behavior word; SO-PMI(word)<0, the word is judged as an abnormal behavior word.

[0066] In the above steps, the first correlation degree between the target candidate word in the unfamiliar word set and the normal behavior word in the normal behavior word set is determined, such as Figure 5 The flowchart shown specifically includes the following steps:

[0067] Step S502, obtaining the first work order number of the target normal behavior words and the target candidate words in the common normal behavior word set, the second work order number of the target candidate words, the third work order number of the target normal words, and the total number of work orders;

[0068] Step S504, determining a first probability of occurrence of a target candidate word and a target normal behavior word according to the first work order number, the second work order number, the third work order number and the total number of work orders;

[0069] Step S506, obtaining the reciprocal of the average value of the distance between the target candidate word and all normal behavior words in the normal behavior word set, recorded as the first coefficient;

[0070] Step S508: determining a first correlation degree between the target candidate word and the target normal word according to the first coefficient and the first probability.

[0071] To solve the problem caused by the size of the corpus, we can add micro-smoothing technology based on the distance between points. We select w normal behavior words and w abnormal behavior words, where z normal behavior words i The first probability of (j=1,2,...,w) and the target candidate word h appearing can be expressed as:

[0072]

[0073] In the above formula, num(h,z i ): Normal behavior words i (j=1,2,...,w) is the number of work orders that co-occur with the target candidate word h, that is, the number of first work orders; num(h): the number of work orders that co-occur with the target candidate word h, that is, the number of second work orders; num(z i ): Normal behavior words i (j=1,2,...,w) The number of co-occurring work orders, that is, the third work order number; v: represents the total number of work orders.

[0074] Due to the size of the corpus, in the actual calculation process, there may be a zero probability problem, that is, num(h,z i ) may be 0, so we add micro-smoothing technology to solve this problem, then num(h,z i ) is transformed into the following expression:

[0075]

[0076] In the above formula, Indicates that the training corpus The number of n-tuples of times; Indicates that num(h,z occurs in the training corpus i )+1 times.

[0077] In the above steps, the second correlation degree between the target candidate word and the abnormal behavior words in the abnormal behavior word set is determined, such as Figure 6 The flowchart shown specifically includes the following steps:

[0078] Step S602, obtaining the fourth work order number of the target abnormal behavior words and the target candidate words in the common abnormal behavior word set, and the fifth work order number of the target abnormal words;

[0079] Step S604, determining a second probability of the target candidate word and the target abnormal behavior word appearing according to the second work order number, the fourth work order number, the fifth work order number and the total number of work orders;

[0080] Step S606, obtaining the reciprocal of the average value of the distances between the target candidate word and all abnormal behavior words in the abnormal behavior word set, recorded as the second coefficient;

[0081] Step S608: Determine a second correlation degree between the target candidate word and the target abnormal word according to the second coefficient and the second probability.

[0082] Similarly, the calculation method of the second probability of the target candidate word h and the abnormal behavior word appearing is similar to the above.

[0083] Therefore, the calculation formula of the improved SO-PMI is:

[0084]

[0085] In the above formula, q represents the total number of work orders.

[0086] In step S206 of the above-mentioned method for determining the work order problem, the information gain entropy value corresponding to the target text information is determined, which specifically includes the following steps: obtaining a third probability that the target candidate word appears in the first candidate word set, and obtaining a fourth probability that the target candidate word appears in the abnormal behavior word set; and determining the information gain entropy value corresponding to the target candidate word in the target text information based on at least the third probability and the fourth probability.

[0087] For example, this application uses a syntactic dependency algorithm to obtain the head (HED) as number portability; uses a BiLSTM-CRF model to obtain a process action word set {system support, prompt abnormality}; the combination of business objects and process actions is "number portability system support" and "number portability prompt abnormality", and then the above two possible results are calculated by the information gain algorithm. First, find out the proportion of the number portability system support combination in the corpus, and calculate the positive entropy and negative entropy according to the entropy algorithm, that is, obtain the third probability of the target candidate word appearing in the first candidate word set, and the fourth probability of the target candidate word appearing in the abnormal behavior word set, and determine the information gain entropy value corresponding to the target candidate word in the target text information according to the third probability and the fourth probability. It is found through empirical evidence that the information gain value of the number portability system support is larger, and the final business object and process action combination is determined as the number portability system support; the value obtained by the improved SO-PMI algorithm is less than 0, indicating that the work order input this time is an abnormal work order, and then the abnormal state is combined with the former to obtain the result of feature fusion: the number portability system support is abnormal.

[0088] In the embodiment of the present application, the combination of business objects and process actions is specifically input into the information gain algorithm for calculation, and then the combination with the highest score is combined with the abnormal state to obtain the final result of feature fusion. The information gain algorithm calculation formula for fusion features is as follows:

[0089]

[0090] In the above formula, IG_IM(ci): word c i The information gain entropy value of i : predicted words; w: process action candidate word set, i.e. the first candidate word set; Abnormal behavior word set; n: the number of text words to be predicted; CD (c i ,w): word c i The concentration in the word set w; P(c i |w):word c i The probability value of appearing in the word set w; The word ci appears in the word set The probability value in .

[0091] The method for determining work order problems provided in the embodiment of the present application has the following advantages: 1. By introducing three key factors, namely business process objects, process actions and abnormal states, the original work order content is semantically described, thereby realizing the digitization of work order problem mining; 2. The work order support process is divided into two states, abnormal and normal, and the sentiment analysis algorithm is applied to the state evaluation of the support process for the first time to realize the judgment of the abnormal state of the work order; 3. In combination with the needs of telecommunications business scenarios, the current mainstream algorithms in the market are improved, the adaptive ability of the model in specific scenarios is improved, and the accurate mining of work order problems is realized. Specifically, the BiLSTM-CRF algorithm is improved: by incorporating the weight value of telecommunications business terms into the transfer score calculation of CFR in the BiLSTM-CRF algorithm, the recognition of professional vocabulary in the telecommunications field is improved, thereby improving the accuracy of entity recognition. The SO-PMI algorithm is improved, and the calculation of SO-PMI values ​​is improved by adding point-to-point distance and micro-smoothing technology, thereby improving the adaptive ability of the model in abnormal behavior judgment scenarios and realizing accurate prediction of abnormal behavior. 4. Feature fusion algorithm, combining the feature selection method based on the abnormal behavior judgment dictionary with the feature fusion algorithm based on process action capture, proposes a method for integrating heterogeneous business process objects and process action capture feature selection, which improves the generalization ability and prediction accuracy of the model. 5. Good work order management effect: 1) Improved efficiency of work order problem classification: After the system was launched, an average of 15 common feature problems were found per month, with an average of 8,000 work orders per month, and a cumulative number of 100,000 work orders, involving 9 major scenarios; 2) Improved labor efficiency: This method deeply integrates semantic warning and syntactic structure analysis to deeply explore the potential intention problems of work orders, greatly improving the efficiency of grassroots managers in discovering and summarizing problems, and greatly shortening the time for problem discovery. From analysis to taking measures for a single problem, it is an average of 7 working days earlier.

[0092] Figure 7 is a structural diagram of a device for determining a work order problem according to an embodiment of the present application, such as Figure 7 As shown, the device comprises:

[0093] An acquisition module 702 is used to acquire a set of business objects corresponding to the target text information in the work order;

[0094] A first determination module 704 is used to determine the process action corresponding to each business object in the business object set to form a first candidate word set;

[0095] A second determination module 706 is used to determine the information gain entropy value corresponding to the target text information at least according to different combinations of business objects in the business object set and process actions in the first candidate word set, and determine the business object and process action corresponding to the maximum value of the information gain entropy value as the fusion feature of the target text information;

[0096] The third determination module 708 is used to determine the problem described in the work order according to the fusion feature.

[0097] In the first determination module in the above-mentioned work order problem determination device, the process action corresponding to each business object in the business object set is determined, which specifically includes the following process: obtaining multiple label sequences corresponding to the target text information, wherein the label sequence is a label type contained in the named entity recognition model; determining the transfer scores corresponding to the multiple label sequences at least based on the weight values ​​corresponding to the business objects in the business object set; determining the probability prediction values ​​corresponding to the multiple label sequences based on the transfer scores, and determining the label sequence corresponding to the maximum value in the probability prediction values ​​as the process action corresponding to the business object.

[0098] In the above process, the transfer scores corresponding to multiple label sequences are determined at least based on the weight values ​​corresponding to the business objects in the business object set, which specifically includes the following processes: obtaining the score of the first label sequence corresponding to the target text information at the target moment, and obtaining the transition probability between the first label sequence and the second label sequence at the target moment and the moment before the target moment; determining the transfer scores corresponding to multiple label sequences at least based on the score of the first label sequence, the transition probability and the weight values ​​corresponding to the business objects in the business object set, wherein the weight values ​​corresponding to the business objects in the business object set are the weight values ​​of the business objects in the target text information, and the weight values ​​are determined at least by the number of times the business objects appear in the target text information.

[0099] In the first determination module in the above-mentioned work order problem determination device, after determining the process action corresponding to each business object in the business object set, the module is also used to execute the following process: performing word segmentation processing on the target text information to obtain a second candidate word set, wherein the second candidate word set includes at least one of the following: a normal behavior word set, an abnormal behavior word set and an unfamiliar word set; determining a first degree of association between the target candidate word in the unfamiliar word set and the normal behavior words in the normal behavior word set, and determining a second degree of association between the target candidate word and the abnormal behavior words in the abnormal behavior word set; when the first degree of association is greater than the second degree of association, determining the target candidate word to be a normal behavior word; when the first degree of association is less than the second degree of association, determining the target candidate word to be an abnormal behavior word.

[0100] In the above process, determining the first degree of association between the target candidate words in the unfamiliar word set and the normal behavior words in the normal behavior word set specifically includes the following process: obtaining the first work order number of the target normal behavior words and the target candidate words in the common normal behavior word set, the second work order number of the target candidate words, the third work order number of the target normal words, and the total number of work orders; determining the first probability of the target candidate words and the target normal behavior words appearing based on the first work order number, the second work order number, the third work order number, and the total number of work orders; obtaining the inverse of the average value of the distance between the target candidate words and all the normal behavior words in the normal behavior word set, recorded as the first coefficient; determining the first degree of association between the target candidate words and the target normal words based on the first coefficient and the first probability.

[0101] In the above process, determining the second degree of association between the target candidate word and the abnormal behavior words in the abnormal behavior word set includes: obtaining the fourth work order number of the target abnormal behavior words and the target candidate words in the abnormal behavior word set, and the fifth work order number in which the target abnormal words appear; determining the second probability of the target candidate word and the target abnormal behavior words appearing based on the second work order number, the fourth work order number, the fifth work order number and the total number of work orders; obtaining the inverse of the average value of the distance between the target candidate word and all abnormal behavior words in the abnormal behavior word set, recorded as the second coefficient; and determining the second degree of association between the target candidate word and the target abnormal word based on the second coefficient and the second probability.

[0102] In the second determination module in the above-mentioned work order problem determination device, the information gain entropy value corresponding to the target text information is determined, which specifically includes the following processes: obtaining a third probability of the target candidate word appearing in the first candidate word set, and obtaining a fourth probability of the target candidate word appearing in the abnormal behavior word set; at least based on the third probability and the fourth probability, determining the information gain entropy value corresponding to the target candidate word in the target text information.

[0103] It should be noted that Figure 7 The work order problem determination device shown is used to perform Figure 2 The method for determining the work order problem shown in the figure, therefore the relevant explanations in the above-mentioned method for determining the work order problem are also applicable to the device for determining the work order problem, and will not be repeated here.

[0104] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the following work order problem determination method: obtain a business object set corresponding to the target text information in the work order; determine the process action corresponding to each business object in the business object set to form a first candidate word set; determine the information gain entropy value corresponding to the target text information based on at least different combinations of business objects in the business object set and process actions in the first candidate word set, and determine the business object and process action corresponding to the maximum value of the information gain entropy value as the fusion feature of the target text information; determine the problem described in the work order based on the fusion feature.

[0105] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0106] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0107] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0108] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0109] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0110] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0111] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining a work order problem, characterized in that: include: Get the business object set corresponding to the target text information in the work order; Determine a process action corresponding to each business object in the business object set to form a first candidate word set; Determine the information gain entropy value corresponding to the target text information at least according to different combinations of business objects in the business object set and process actions in the first candidate word set, and determine the business object and process action corresponding to the maximum value of the information gain entropy value as the fusion feature of the target text information; Determining the problem described in the work order according to the fusion feature; Determining the process action corresponding to each business object in the business object set, comprising: obtaining multiple label sequences corresponding to the target text information, wherein the label sequences are label types included in the named entity recognition model; determining the transfer scores corresponding to the multiple label sequences at least according to the weight values ​​corresponding to the business objects in the business object set; determining the probability prediction values ​​corresponding to the multiple label sequences according to the transfer scores, and determining the label sequence corresponding to the maximum value in the probability prediction values ​​as the process action corresponding to the business object; Determining the information gain entropy value corresponding to the target text information includes: obtaining a third probability that a target candidate word appears in the first candidate word set, and obtaining a fourth probability that the target candidate word appears in the abnormal behavior word set, wherein the target text information is segmented to obtain a second candidate word set, the second candidate word set includes at least one of the following: a normal behavior word set, an abnormal behavior word set, and an unfamiliar word set, and the target candidate word is a candidate word in the unfamiliar word set; determining the information gain entropy value corresponding to the target candidate word in the target text information at least based on the third probability and the fourth probability.

2. The method according to claim 1, characterized in that Determining the transfer scores corresponding to the multiple tag sequences at least according to the weight values ​​corresponding to the business objects in the business object set includes: Obtaining a score of a first label sequence corresponding to the target text information at a target time, and obtaining a transition probability between the first label sequence and a second label sequence at the target time and a time before the target time; The transfer scores corresponding to the multiple label sequences are determined based at least on the score of the first label sequence, the transfer probability and the weight values ​​corresponding to the business objects in the business object set, wherein the weight values ​​corresponding to the business objects in the business object set are the weight values ​​of the business objects in the target text information, and the weight values ​​are determined at least by the number of times the business objects appear in the target text information.

3. The method according to claim 1, characterized in that After determining the process action corresponding to each business object in the business object set, the method further includes: Determining a first degree of association between the target candidate word and normal behavior words in the normal behavior word set, and determining a second degree of association between the target candidate word and abnormal behavior words in the abnormal behavior word set; In a case where the first association degree is greater than the second association degree, determining the target candidate word to be the normal behavior word; When the first association degree is less than the second association degree, the target candidate word is determined to be the abnormal behavior word.

4. The method according to claim 3, characterized in that Determining a first correlation degree between a target candidate word in the unfamiliar word set and a normal behavior word in the normal behavior word set includes: Obtain the number of first work orders in which the target normal behavior words and the target candidate words co-appear in the normal behavior word set, the number of second work orders in which the target candidate words appear, the number of third work orders in which the target normal behavior words appear, and the total number of the work orders; Determining a first probability of the target candidate word and the target normal behavior word appearing according to the first work order number, the second work order number, the third work order number, and the total number of the work orders; Obtaining the reciprocal of the average value of the distances between the target candidate word and all normal behavior words in the normal behavior word set, recorded as the first coefficient; A first degree of association between the target candidate word and the target normal behavior word is determined according to the first coefficient and the first probability.

5. The method according to claim 4, characterized in that Determining a second correlation degree between the target candidate word and the abnormal behavior words in the abnormal behavior word set includes: Obtaining the fourth work order number in which the target abnormal behavior word and the target candidate word co-appear in the abnormal behavior word set, and the fifth work order number in which the target abnormal behavior word appears; Determining a second probability of the target candidate word and the target abnormal behavior word appearing according to the second work order number, the fourth work order number, the fifth work order number, and the total number of work orders; Obtaining the reciprocal of the average value of the distances between the target candidate word and all abnormal behavior words in the abnormal behavior word set, recorded as the second coefficient; A second degree of association between the target candidate word and the target abnormal behavior word is determined according to the second coefficient and the second probability.

6. A device for determining a work order problem, characterized in that: include: The acquisition module is used to obtain the business object set corresponding to the target text information in the work order; A first determination module, used to determine the process action corresponding to each business object in the business object set to form a first candidate word set; A second determination module is used to determine the information gain entropy value corresponding to the target text information at least according to different combinations of business objects in the business object set and process actions in the first candidate word set, and determine the business object and the process action corresponding to the maximum value of the information gain entropy value as the fusion feature of the target text information; A third determination module, configured to determine the problem described in the work order according to the fusion feature; The first determination module is further used to obtain multiple label sequences corresponding to the target text information, wherein the label sequences are label types included in the named entity recognition model; determine the transfer scores corresponding to the multiple label sequences at least according to the weight values ​​corresponding to the business objects in the business object set; determine the probability prediction values ​​corresponding to the multiple label sequences according to the transfer scores, and determine the label sequence corresponding to the maximum value in the probability prediction values ​​as the process action corresponding to the business object; The second determination module is also used to obtain a third probability that the target candidate word appears in the first candidate word set, and to obtain a fourth probability that the target candidate word appears in the abnormal behavior word set, wherein the target text information is segmented to obtain a second candidate word set, and the second candidate word set includes at least one of the following: a normal behavior word set, an abnormal behavior word set, and an unfamiliar word set, and the target candidate word is a candidate word in the unfamiliar word set; and the information gain entropy value corresponding to the target candidate word in the target text information is determined at least based on the third probability and the fourth probability.

7. An electronic device, characterized in that: include: A memory for storing program instructions; A processor is connected to the memory and is used to execute program instructions to implement the following functions: obtaining a set of business objects corresponding to target text information in a work order; determining a process action corresponding to each business object in the business object set to form a first candidate word set; determining an information gain entropy value corresponding to the target text information based on at least different combinations of business objects in the business object set and process actions in the first candidate word set, and determining the business object and process action corresponding to the maximum value of the information gain entropy value as a fusion feature of the target text information; Determining the problem described in the work order according to the fusion feature; Determining the process action corresponding to each business object in the business object set, comprising: obtaining multiple label sequences corresponding to the target text information, wherein the label sequences are label types included in the named entity recognition model; determining the transfer scores corresponding to the multiple label sequences at least according to the weight values ​​corresponding to the business objects in the business object set; determining the probability prediction values ​​corresponding to the multiple label sequences according to the transfer scores, and determining the label sequence corresponding to the maximum value in the probability prediction values ​​as the process action corresponding to the business object; Determining the information gain entropy value corresponding to the target text information includes: obtaining a third probability that a target candidate word appears in the first candidate word set, and obtaining a fourth probability that the target candidate word appears in the abnormal behavior word set, wherein the target text information is segmented to obtain a second candidate word set, the second candidate word set includes at least one of the following: a normal behavior word set, an abnormal behavior word set, and an unfamiliar word set, and the target candidate word is a candidate word in the unfamiliar word set; determining the information gain entropy value corresponding to the target candidate word in the target text information at least based on the third probability and the fourth probability.

8. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the method for determining the work order problem according to any one of claims 1 to 5.

Images