Work order data processing method and system based on topic model, medium and equipment

Abstract

The invention belongs to the technical field of data mining, and provides a work order data processing method and system based on a topic model, a medium and equipment, and the method comprises the steps: extracting a flow type work order data set through employing a trained representation extraction model, and obtaining a second corpus; constructing a second topic model based on the second corpus, and calculating a topic corresponding to the process type work order; counting the quantity of the flow-type work orders associated with each theme, and selecting the theme with the quantity of the associated flow-type work orders ranked as the first sixth preset rank as a second work order hotspot theme; obtaining an original process entry data set, and preprocessing the original process entry data set to obtain a third corpus and a third entry sequence; and based on the third entry sequence, utilizing the second topic model to calculate and obtain a topic corresponding to each process entry. The topic modeling is carried out on the corpus based on the topic model, the hot topic of the work order is intelligently extracted, and the topic is associated with the process entry data, so that management personnel at all levels can carry out optimization work in a targeted manner.

Description

Technical Field

[0001] The present invention belongs to the technical field of data mining, and in particular relates to a work order data processing method, system, medium and equipment based on a topic model. Background Art

[0002] As business grows, a large amount of work order data accumulates. Problem work orders are a type of work order. How to fully utilize these problem work orders to accurately identify hot issues on the front lines and promptly respond to and resolve them is a critical issue facing personnel at all levels.

[0003] In addition, how to accurately locate the business processes associated with problem work orders, so as to carry out targeted end-to-end process analysis and optimization work, trace the root causes, solve these hot issues from the source, and effectively reduce the problem work order data, is another important issue faced by personnel at all levels. Summary of the Invention

[0004] The present invention proposes a work order data processing method, system, medium and equipment based on a topic model.

[0005] The present invention provides a method for processing work order data based on a topic model, the method comprising: Obtain the original work order dataset; Extracting a process-related work order dataset from the original work order dataset using the trained representation extraction model, and preprocessing the process-related work order dataset to obtain a second corpus; Building a second topic model based on the second corpus, and calculating the topic corresponding to the process-related work order based on the second topic model; Count the number of process-related work orders associated with each topic, and select the topic with the sixth preset ranking in the number of associated process-related work orders as the second work order hot topic; Acquire an original process entry dataset, and preprocess the original process entry dataset to obtain a third corpus, wherein the third corpus includes a third word sequence; Based on the third term sequence, the second topic model is used to calculate and obtain a topic corresponding to each process item.

[0006] Further, After obtaining the original work order data set, the following steps are also included: Preprocessing the original work order dataset to obtain a first corpus; Building a first topic model based on the first corpus, and calculating the core terms of all original work orders and the topics corresponding to all original work orders based on the first topic model; The number of original work orders associated with each topic is counted, and the topic whose number of associated original work orders ranks in the top four preset positions is selected as the first work order hot topic.

[0007] Further, After calculating the topic corresponding to each process item using the second topic model, the method further includes: Count all process entries that match a certain topic and generate a list of process entries.

[0008] Further, Calculating the topic corresponding to the process-related work order based on the second topic model also includes: Using big model technology, we extract question and answer elements from all process-related work orders associated with each topic and obtain the question and answer pairs corresponding to each topic.

[0009] Further, Building a first topic model based on the first corpus, and calculating the topic core terms of all original work orders and the topics corresponding to all original work orders based on the first topic model, including: Based on the first corpus, constructing the first topic model using a sampling algorithm, the first topic model including the topic term distribution of the original work order and the topic distribution of the original work order; Based on the distribution of subject terms in the original work order, the dependence degree of each term on a certain subject and the proportion of occurrences of each term in the first corpus are calculated; Based on the degree of dependence of each term on a certain topic and the proportion of the number of occurrences of each term in the first corpus, a comprehensive score of each term ranked in the first preset position under a certain topic is calculated; According to the comprehensive score of each term ranked in the first preset position under the certain topic, the term ranked in the second preset position in the comprehensive score is selected from all the terms ranked in the first preset position under the certain topic as the core term of the certain topic; Based on the topic distribution of the original work order, a topic with an allocation probability ranking in the top third preset position is selected from all topics assigned to the original work order as the topic corresponding to the original work order; or based on the topic distribution of the original work order, a topic with an allocation probability higher than a second threshold is selected from all topics assigned to the original work order as the topic corresponding to the original work order.

[0010] Further, Based on the second topic model, an incremental work order is processed, including: Obtaining an incremental work order, preprocessing the incremental work order, and obtaining a fourth term sequence; Based on the fourth term sequence, the second topic model is used to calculate the topic corresponding to the incremental work order, and then the process item, the corresponding question-answer pair, and the associated process-related work order are obtained. By using word cloud technology, the subject corresponding to the incremental work order and the process items associated with the subject corresponding to the incremental work order, the corresponding question and answer pairs, and the associated process-type work orders are visualized.

[0011] A work order data processing system based on a topic model of the present invention is characterized by comprising: An acquisition unit, used to obtain the original work order data set; A second preprocessing unit is configured to extract a process-related work order dataset from the original work order dataset using the trained representation extraction model, and preprocess the process-related work order dataset to obtain a second corpus; A second computing unit, configured to construct a second topic model based on the second corpus, and calculate topics corresponding to the process-related work orders based on the second topic model; The second statistical unit is used to count the number of process-related work orders associated with each topic, and select the topic with the sixth preset ranking in the number of associated process-related work orders as the second work order hot topic; a third preprocessing unit, configured to obtain an original process entry dataset, and preprocess the original process entry dataset to obtain a third corpus, wherein the third corpus includes a third word sequence; The third calculation unit is configured to calculate a topic corresponding to each process item using the second topic model based on the third word sequence.

[0012] Further, Also includes: A first preprocessing unit, configured to preprocess the original work order dataset to obtain a first corpus; A first computing unit is configured to construct a first topic model based on the first corpus, and calculate the topic core terms of all original work orders and the topics corresponding to all original work orders based on the first topic model; The first statistical unit is used to count the number of original work orders associated with each topic, and select the topic whose number of associated original work orders ranks in the top four preset positions as the first work order hot topic.

[0013] Further, Also includes: The matching unit is used to count all process entries matched under a certain topic and generate a list of process entries.

[0014] Further, Also includes: The extraction unit is used to use large model technology to extract question and answer elements from all process-type work orders associated with each topic, and obtain the question and answer pairs corresponding to each topic.

[0015] Further, Also includes: a fourth preprocessing unit, configured to obtain an incremental work order, preprocess the incremental work order, and obtain a fourth term sequence; a fourth calculation unit, configured to calculate, based on the fourth term sequence, a topic corresponding to a certain incremental work order using the second topic model, and further obtain a process entry, a corresponding question-answer pair, and an associated process-type work order associated with the topic corresponding to the certain incremental work order; The second visualization unit is used to use word cloud technology to visualize the subject corresponding to the incremental work order and the process items associated with the subject corresponding to the incremental work order, the corresponding question and answer pairs, and the associated process-type work orders.

[0016] A computer-readable storage medium of the present invention stores a program or instruction. When the program or instruction is executed on a computer, the computer executes any one of the aforementioned work order data processing methods based on a topic model.

[0017] An electronic device of the present invention includes a processor coupled to a memory; The processor is configured to read and execute the computer program stored in the memory to implement any of the aforementioned methods for processing work order data based on a topic model.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention fully mines the massive problem work order data accumulated in the background of the system, performs topic modeling on the problem work order corpus based on the topic model, intelligently extracts hot topics of work orders, matches a list of relevant work orders under each topic, and uses word cloud technology to visualize the semantic information of the topic. This helps managers at all levels to accurately locate current hot spots and respond to and solve the "urgent, difficult, and anxious" problems of concern to the grassroots in a timely manner.

[0019] Automatically screen process-related problem tickets based on the representation extraction model, intelligently extract hot topics of work tickets based on the topic model, and accurately locate process entries in the process knowledge base associated with hot topics of work tickets. This helps managers at all levels to carry out targeted end-to-end process analysis and optimization work, solve hot issues of work tickets from the source, and effectively reduce the number of problem work tickets.

[0020] The incremental work order data is processed using a topic model to obtain the topics of the incremental work order data. Topics, question-and-answer pairs, and process items related to the incremental work order are recommended to users, providing problem-solving ideas for staff and improving their efficiency in problem solving. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a first flow chart of the work order data processing method based on the topic model of the present invention; Figure 2 This is a second flow chart of the work order data processing method based on the topic model of the present invention; Figure 3 This is a structural diagram of the work order data processing system based on the topic model of the present invention.

[0023] Description of reference numerals: 301-acquisition unit, 302-first preprocessing unit, 303-first calculation unit, 304-first statistical unit, 305-second preprocessing unit, 306-second calculation unit, 307-second statistical unit, 308-third preprocessing unit, 309-third calculation unit, 310-matching unit, 311-extraction unit, 312-first visualization unit, 313-fourth preprocessing unit, 314-fourth calculation unit, 315-second visualization unit. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0025] Figure 1 This is a first flow chart of a method for processing work order data based on a topic model according to the present invention. In the first embodiment of the method, the method specifically includes the following steps: S101: Obtain the original work order dataset.

[0026] Get the original problem work orders within the specified time period, define the original problem work orders as original work orders, and all original work orders within the specified time period constitute the original work order dataset.

[0027] Each original work order includes field information such as work order title and work order description.

[0028] S102: Preprocess the original work order dataset to obtain a first corpus.

[0029] Preprocessing methods include but are not limited to Chinese word segmentation, filtering stop words, and filtering low-frequency words.

[0030] Perform Chinese word segmentation on the work order title and work order description of an original work order in the original work order dataset to obtain at least one term, and all the terms form a first original term sequence. Performing Chinese word segmentation on the work order title and work order description of each original work order in the original work order dataset using this method can obtain at least one first original term sequence, and the number of first original term sequences corresponds to the number of original work orders in the original work order dataset.

[0031] Furthermore, all first original word sequences constitute a first original corpus.

[0032] The work order title and work order description of each original work order in the original work order dataset are then filtered for low-frequency words and stop words to obtain at least one first term sequence. The number of first term sequences corresponds to the number of original work orders in the original work order dataset. The process of filtering low-frequency words is as follows: a first threshold is set. When the number of occurrences of a term in the first original corpus is less than the first threshold, the term is identified as a low-frequency term and filtered. Exemplarily, the first threshold is 30 occurrences.

[0033] Furthermore, all first term sequences constitute a first corpus.

[0034] S103: Constructing a first topic model based on the first corpus, and calculating the topic core terms of all original work orders and the topics corresponding to all original work orders based on the first topic model.

[0035] S103-1: Based on the first corpus, a first topic model is constructed using a sampling algorithm.

[0036] Sampling algorithms include but are not limited to variational inference, Gibbs sampling, Laplace approximation, and expectation-propagation, etc. The present invention preferably adopts Gibbs sampling.

[0037] Topic models include but are not limited to probabilistic latent semantic analysis (PLSA), latent Dirichlet allocation (LDA), structural topic model (STM), and hierarchical latent Dirichlet allocation (hLDA).

[0038] The first topic model includes the topic term distribution of the original work order, the topic distribution of the original work order, and the number of topics in the original work order. Based on the first corpus, Gibbs sampling is used to calculate the topic term distribution and the topic distribution of the original work order. The topic term distribution of the original work order includes all terms in each topic and the probability distribution of all terms under each topic. The topic distribution of the original work order includes each topic to which each original work order is assigned and the probability distribution of each original work order being assigned to each topic. The number of topics in the original work order is determined using the empirical setting method. For example, the number of topics in the original work order is 30.

[0039] It should be noted that, other sampling algorithms besides Gibbs sampling can also be used to obtain the subject term distribution and the topic distribution of the original work order.

[0040] S103-2: Based on the distribution of subject terms in the original work order, calculate the degree of dependence of each term on a certain subject and the proportion of occurrences of each term in the first corpus.

[0041] Based on the distribution of subject terms in the original work order, the dependence of a term on a subject and the proportion of occurrences of a term in the first corpus are calculated. The calculation formula is as follows: (1), (2), Where, It is used to measure the degree of dependence of a certain term on a certain topic, referred to as z value; The number of times a term appears in a topic; is the percentage of occurrences of a term in the first corpus; is the number of occurrences of a term in the first corpus; is the total number of occurrences of all terms in the first corpus, without deduplication.

[0042] The higher the z value, the more dependent the term is on the topic, that is, the lower its independence.

[0043] Based on the distribution of subject terms in the original work order, all terms under a certain topic are calculated according to formulas (1) and (2) to obtain the degree of dependence of each term on a certain topic and the proportion of the number of occurrences of each term in the first corpus.

[0044] S103-3: Based on the degree of dependence of each term on a certain topic and the proportion of the number of occurrences of each term in the first corpus, a comprehensive score of each term ranked in the first preset position under a certain topic is calculated.

[0045] Arrange all entries under a certain topic in descending order according to the proportion of the number of occurrences of the entries in the first corpus, and select the entries ranked in the first preset rank, where the number of the first preset rank is N. For example, the first preset noun is 30, that is, N=30. Set the score of the number of occurrences of the entry ranked first to the first preset value, set the score of the number of occurrences of the entry ranked second to the second preset value, the second preset value = the first preset value - 1, and so on. Set the score of the number of occurrences of the entry ranked last to the Nth preset value, the Nth preset value = the N-1th preset value - 1.

[0046] Arrange all the terms ranked in the first preset position under a certain topic in descending order of z-score. Set the z-score score of the first-ranked term to the first preset value, the z-score score of the second-ranked term to the second preset value, the second preset value = the first preset value - 1, and so on. Set the z-score score of the last-ranked term to the Nth preset value, the Nth preset value = the N-1th preset value - 1.

[0047] Based on the z-value score of a term and the score of the number of occurrences of a term, the comprehensive score of a term ranked first in a certain topic is calculated. The calculation formula is as follows: (3), Where, The comprehensive score of a term ranked first in a preset ranking under a certain topic; Score the percentage of occurrences of a term ranked first in a certain topic; The z-score of a term ranked first in a certain topic; are the weights of the occurrence ratio score and the z-value score in the comprehensive score, respectively. .

[0048] When calculating according to formula (3), the example is Set to 0.8, Set to 0.2. This setting will make the occurrence ratio score have a greater impact on the overall score than the z-value score.

[0049] According to formula (3), all entries ranked in the first preset position under a certain topic are calculated to obtain the comprehensive score of each entry ranked in the first preset position under a certain topic.

[0050] S103-4: Based on the comprehensive score of each term ranked in the first preset rank under a certain topic, select the term ranked in the second preset rank from all terms ranked in the first preset rank under a certain topic as the core term of the certain topic. For example, the second preset rank is 10.

[0051] According to the method described in S103 - 2 to S103 - 4 , the core terms of each topic can be calculated, that is, the core terms of the topic of each original work order can be calculated.

[0052] S103-5: Based on the topic distribution of the original work orders, select the topic with the third preset ranking of allocation probability from all topics assigned to the original work order as the topic corresponding to the original work order. Exemplarily, the third preset ranking is 3. Alternatively, based on the topic distribution of the original work orders, select the topic with an allocation probability above a second threshold from all topics assigned to the original work order as the topic corresponding to the original work order. Exemplarily, the second threshold is 10%.

[0053] According to this method, an original work order corresponds to at least one topic, that is, an original work order can be assigned to one or more topics.

[0054] By following this method, the subject corresponding to each original work order can be obtained.

[0055] S104: Count the number of original work orders associated with each topic, and select the topic whose number of associated original work orders ranks in the top four preset positions as the first work order hot topic.

[0056] S105: Using the trained representation extraction model, a process-related work order dataset is extracted from the original work order dataset, and the process-related work order dataset is preprocessed to obtain a second corpus.

[0057] The representation extraction model in this embodiment is the BERT (Bidirectional Encoder Representations from Transformers) model, which is a pre-trained language model based on the Transformer architecture. The core of the BERT model lies in its bidirectional encoder structure, which can simultaneously consider the contextual information in the text, thereby better understanding the relationship between words, phrases, and sentences, and extracting more accurate representations. It should be noted that the present invention is not limited to the BERT model. Any model or algorithm that can achieve representation extraction can be applied to this embodiment.

[0058] The processing process is as follows: S105-1: Use the cleaned labeled work order dataset to train the original BERT model to obtain a trained BERT model.

[0059] Step 1: Business personnel label a batch of work order data to obtain a labeled work order data set. The labeling content is whether each work order data is related to the process.

[0060] Step 2: Business personnel import the label work order dataset into the system.

[0061] Step 3: The system cleans the label work order dataset. After cleaning, only the work order title and work order description fields are retained.

[0062] Step 4: The system uses the labeled work order dataset to train the original BERT model to obtain the trained BERT model.

[0063] S105-2: Extract the work order title and the work order description of each original work order from the original work order dataset.

[0064] S105-3: Use the trained BERT model to identify the work order title and work order description of each original work order. Through identification, the original work order dataset is divided into a process-related work order dataset and a non-process-related work order dataset. The process-related work order dataset refers to a work order dataset related to the process, and the non-process-related work order dataset refers to a work order dataset not related to the process. The process-related work order dataset includes at least one process-related work order, and each process-related work order includes information such as a work order title and a work order description. The non-process-related work order dataset includes at least one non-process-related work order, and each non-process-related work order includes information such as a work order title and a work order description.

[0065] S105-4: Preprocess the process work order dataset to obtain a second corpus.

[0066] Preprocessing methods include but are not limited to Chinese word segmentation, filtering stop words, and filtering low-frequency words.

[0067] Perform Chinese word segmentation on the work order title and work order description of a process-related work order in the process-related work order dataset to obtain at least one entry, and all the entries form a second original entry sequence. Perform Chinese word segmentation on the work order title and work order description of each process-related work order in the process-related work order dataset according to this method to obtain at least one second original entry sequence, and the number of second original entry sequences corresponds to the number of process-related work orders in the process-related work order dataset.

[0068] All second original word sequences constitute the first original corpus.

[0069] The work order title and work order description of each process-related work order in the process-related work order dataset are then filtered for low-frequency words and stop words, resulting in at least one second term sequence. The number of second term sequences corresponds to the number of process-related work orders in the process-related work order dataset. The low-frequency word filtering process is as follows: a third threshold is set. When the number of occurrences of a term in the first original corpus is less than the first threshold, the term is identified as a low-frequency term and filtered. Exemplarily, the third threshold is 30 occurrences.

[0070] All second term sequences constitute the second corpus.

[0071] S106: Construct a second topic model based on the second corpus, and calculate topics corresponding to all process-related work orders based on the second topic model.

[0072] S106-1: Based on the second corpus, a second topic model is constructed using a sampling algorithm.

[0073] Sampling algorithms include but are not limited to variational inference, Gibbs sampling, Laplace approximation, and expectation-diffusion.

[0074] The second topic model includes the subject term distribution of process-type work orders, the subject distribution of process-type work orders, and the number of topics of process-type work orders. Based on the second corpus, the subject term distribution of process-type work orders and the subject distribution of process-type work orders are calculated using Gibbs sampling. The subject term distribution of process-type work orders includes all terms under each topic and the probability distribution of all terms under each topic. The subject distribution of process-type work orders includes each topic to which each process-type work order is assigned and the probability distribution of each process-type work order being assigned to each topic. In this embodiment, the number of topics for process-type work orders can be determined using the empirical setting method. Exemplarily, the number of topics for process-type work orders is 30.

[0075] It should be noted that, in addition to Gibbs sampling, other sampling algorithms can also be used to obtain the subject term distribution and the subject distribution of process-related work orders, and the method is not limited to the Gibbs algorithm.

[0076] S106-2: Based on the topic distribution of process-related work orders, select the topic with the fifth preset ranking of allocation probability from all topics assigned to a process-related work order as the topic corresponding to the process-related work order. Exemplarily, the fifth preset ranking is 3. Alternatively, based on the topic distribution of process-related work orders, select the topic with an allocation probability above a fourth threshold from all topics assigned to a process-related work order as the topic corresponding to the process-related work order. Exemplarily, the fourth threshold is 10%.

[0077] According to this method, a process-type work order corresponds to at least one topic, that is, a process-type work order can be assigned to one or more topics.

[0078] By following this method, you can get the topic corresponding to each process work order.

[0079] S107: Count the number of process-related work orders associated with each topic, and select the topic with the sixth preset ranking in the number of associated process-related work orders as the second work order hot topic.

[0080] S108: Acquire an original process entry dataset, pre-process the original process entry dataset, and obtain a third corpus, where the third corpus includes a third term sequence.

[0081] S108-1: Obtain the original process entry dataset.

[0082] Get the complete and latest process entries. All process entries constitute the process entry dataset.

[0083] Each process entry includes field information such as process name, process introduction, brief description of process steps, flow chart, process monitoring data, rules and regulations, training manual and training video.

[0084] S108-2: Extract the full path process name of each process entry from the original process entry dataset.

[0085] Step 1: Extract the process name of each process entry from the first level to the current level.

[0086] Step 2: Based on the entry level, automatically concatenate the process names at each level in order from the first level to the current level to obtain the full path process name for each process entry. For example, the full path process name for a five-level process entry is the first-level process name + the second-level process name + the third-level process name + the fourth-level process name + the fifth-level process name.

[0087] S108-3: Extract a process profile of each process entry from the original process entry dataset.

[0088] S108-4: Preprocess the full path process name of each process entry and the process profile of each process entry to obtain a term sequence of each process entry.

[0089] Preprocessing methods include but are not limited to Chinese word segmentation, filtering stop words, and filtering low-frequency words.

[0090] Perform Chinese word segmentation on the full path process name of a process entry and the process description of a process entry to obtain at least one term. All terms form a third original term sequence. Perform Chinese word segmentation on the full path process name of each process entry and the process description of each process entry to obtain at least one third original term sequence. The number of third original term sequences corresponds to the number of process entries.

[0091] All third original term sequences constitute the third original corpus.

[0092] The full path process name of each process entry and the process description of each process entry are then filtered for low-frequency words and stop words to obtain at least one third term sequence. The number of third term sequences corresponds to the number of process entries. The low-frequency word filtering process is as follows: a fifth threshold is set. When the number of occurrences of a term in the third original corpus is less than the fifth threshold, the term is identified as a low-frequency term and filtered. Exemplarily, the fifth threshold is 5 occurrences.

[0093] All third term sequences constitute the third corpus.

[0094] It should be noted that the third term sequence is the term sequence of each process entry.

[0095] S109: Based on the third term sequence, the second topic model is used to calculate and obtain the topic corresponding to each process item.

[0096] Substitute the term sequence of each process entry into the second topic model to obtain the topic distribution of each process entry. The topic distribution of each process entry includes each topic to which each process entry is assigned and the probability distribution of each process entry being assigned to each topic.

[0097] Based on the topic distribution of each process item, the topic with the top seventh preset ranking of allocation probability is selected from all topics assigned to the process item as the topic corresponding to the process item. Exemplarily, the seventh preset ranking is 3. Alternatively, based on the topic distribution of each process item, the topic with an allocation probability above a sixth threshold is selected from all topics assigned to the process item as the topic corresponding to the process item. Exemplarily, the sixth threshold is 10%.

[0098] According to this method, a process entry corresponds to at least one topic, that is, a process entry can be assigned to one or more topics.

[0099] By following this method, we can get the subject corresponding to each process entry.

[0100] It can be seen from S109 that the topic model plays the role of a multi-topic classifier.

[0101] S110: Count all process entries matched under a certain topic and generate a process entry list.

[0102] S111: Using big model technology, extract question and answer elements from all process-related work orders associated with each topic to obtain the question and answer pairs corresponding to each topic.

[0103] For example, the Qwen model can be used to extract question and answer elements. The extraction process is as follows: Step 1: Pre-process the process work orders to obtain the marked process work orders.

[0104] Preprocessing methods include but are not limited to data cleaning, format unification, and data labeling.

[0105] Data cleaning refers to removing noise data from process-related work orders, such as garbled characters, invalid characters, and duplicate records. After data cleaning, cleaned process-related work orders are obtained.

[0106] Format unification refers to converting cleaned process-type work orders in different formats into a specified standard format to obtain formatted process-type work orders.

[0107] Data labeling refers to labeling the formatted process-type work orders, clarifying the key information in each process-type work order, such as the problem category, core content of the problem, reference answers, etc., to obtain the labeled process-type work orders.

[0108] Step 2: Divide the annotated process-related work orders into a training set, a validation set, and a test set. For example, the training set accounts for 70%, the validation set accounts for 15%, and the test set accounts for 15%.

[0109] Step 3: Set fine-tuning parameters based on the data characteristics of process-related work orders and model performance requirements. Fine-tuning parameters include but are not limited to learning rate, training rounds, and batch size.

[0110] Step 4: Based on the fine-tuned parameters, the original Qwen model is trained using the training set. During training, the original Qwen model continuously adjusts its parameters based on the reference answers to minimize the loss function. The validation set is used to monitor the training performance of the original Qwen model in real time to prevent overfitting. The trained Qwen model is obtained through training.

[0111] Step 5: Construct the annotated process work order according to the input format required by the trained Qwen model. During construction, the problem description part is used as the input text.

[0112] Step 6: Input the input text into the trained Qwen model to generate the answer and thus obtain the question-answer pair.

[0113] Step 7: Use multiple evaluation indicators to evaluate the quality of the question-answer pairs and obtain the evaluation results.

[0114] Exemplary evaluation metrics include, but are not limited to, the BLEU (bilingual evaluation understudy) metric, which is used to measure the similarity between the generated answer and the reference answer; and the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metric, which is used to evaluate the extent to which the generated answer covers key information of the reference answer.

[0115] Step 8: Optimize the question-answer pairs based on the evaluation results.

[0116] Through the first to eighth steps, the question-answer pairs corresponding to each topic can be obtained, and the number of question-answer pairs is at least one.

[0117] It is worth noting that other models besides the Qwen model can also be used to obtain the question-answer pairs corresponding to the topic of each process-type work order, such as the ChatGPT model.

[0118] Based on all the processing results in the first method embodiment, in the second method embodiment, the word cloud technology is used to visualize the core terms of each original work order, the hot topics of the first work order, and the hot topics of the second work order, as follows: When the word cloud technology is used to visualize the core terms of each original work order, the text size of the term is positively correlated with the comprehensive score, that is, the higher the comprehensive score, the larger the text.

[0119] After using word cloud technology to visualize the hot topics of the first work order, the user can click on the hot topics of the first work order on the front-end page to view the original work order associated with the hot topics of the first work order, and then click on the work order name of the original work order to view the details of each original work order and understand the specific problems mentioned in the original work order and the corresponding solutions.

[0120] After visualizing the hot topics of the second work order using word cloud technology, users can click on the hot topics of the second work order to view: 1) For the process items associated with the second work order hot topic, carry out targeted process optimization work, trace the root cause, and solve the process and business issues involved in the hot work order topic from the source.

[0121] 2) The second ticket has question and answer pairs corresponding to the hot topics, allowing for quick access to problem-solving ideas to improve response efficiency.

[0122] 3) The process-related work orders associated with the second work order hot topic. You can then click the work order name of the process-related work order to view the details of each process-related work order and understand the specific problems mentioned in the process-related work order and the corresponding solutions.

[0123] Based on all the processing results in the first method embodiment, in the third method embodiment, the following processing is performed on a certain incremental problem work order (the incremental problem work order is defined as an incremental work order), such as Figure 2 Shown, including: S201: Obtain an incremental work order, pre-process the incremental work order, and obtain a fourth term sequence.

[0124] Preprocess an incremental work order. Preprocessing methods include but are not limited to Chinese word segmentation, filtering stop words, and filtering low-frequency words.

[0125] Perform Chinese word segmentation on the work order title and the work order description of an incremental work order to obtain at least one entry, and all the entries form a fourth original entry sequence.

[0126] Continue to filter low-frequency words and stop words from the work order title of a certain incremental work order and the work order description of a certain incremental work order to obtain at least one fourth term sequence.

[0127] S202: Based on the fourth term sequence, the second topic model is used to calculate the topic corresponding to a certain incremental work order, and then the process entries, corresponding question-answer pairs and associated process-type work orders associated with the topic corresponding to the certain incremental work order are obtained.

[0128] Substitute the fourth term sequence into the second topic model to obtain the topic distribution of a certain incremental work order. The topic distribution of a certain incremental work order includes each topic to which the certain incremental work order is assigned and the probability distribution of the certain incremental work order being assigned to each topic.

[0129] From all topics assigned to an incremental work order, select the topic with an allocation probability ranking in the top eight preset positions as the topic corresponding to the incremental work order. Exemplarily, the eighth preset position is 3. Alternatively, based on the topic distribution of an incremental work order, select the topic with an allocation probability above a seventh threshold from all topics assigned to the incremental work order as the topic corresponding to the incremental work order. Exemplarily, the seventh threshold is 10%.

[0130] According to this method, an incremental work order corresponds to at least one topic, that is, an incremental work order can be assigned to one or more topics.

[0131] From this step, we can see that the topic model also plays the role of a multi-topic classifier.

[0132] Since in the first method embodiment, the process entries associated with all topics and the question-answer pairs corresponding to all topics have been obtained based on the second topic model, and all topics have all the process-type work orders associated with them, and at the same time, the topic corresponding to a certain incremental work order belongs to all topics, it is natural to obtain the process entries associated with the topic corresponding to a certain incremental work order, the question-answer pairs corresponding to the topic corresponding to a certain incremental work order, and the process-type work orders associated with the topic corresponding to a certain incremental work order.

[0133] S203: Using word cloud technology, visualize the topic corresponding to a certain incremental work order, the process items associated with the topic of the incremental work order, the corresponding question and answer pairs, and the associated process-related work orders. After visualizing the topic corresponding to an incremental work order using word cloud technology, users can click on the topic corresponding to an incremental work order to view: 1) Carry out targeted process optimization work on the process items associated with the topic corresponding to a certain incremental work order, trace the root cause, and solve the process and business issues involved in the hot work order topic from the source.

[0134] 2) The question-and-answer pairs corresponding to the topic of an incremental ticket quickly provide insights into solving the problem and improve response efficiency. The answers in the question-and-answer pairs are essentially proactively recommended solutions to the problem. Displaying these question-and-answer pairs to users can be considered a personalized recommendation algorithm.

[0135] 3) The process work orders associated with the topic corresponding to a certain incremental work order, and then click the work order name of the process work order to view the details of each process work order, and understand the specific problems mentioned in the process work order and the corresponding solutions. The embodiment of the present invention also provides a work order data processing system based on the topic model, such as Figure 3 Shown, including: The acquisition unit 301 is used to acquire the original work order data set.

[0136] The first preprocessing unit 302 is configured to preprocess the original work order dataset to obtain a first corpus.

[0137] The first calculation unit 303 is configured to construct a first topic model based on the first corpus, and calculate the topic core terms of all original work orders and the topics corresponding to all original work orders based on the first topic model.

[0138] The first statistical unit 304 is configured to count the number of original work orders associated with each topic, and select a topic with the number of associated original work orders ranked in the top four preset positions as a first work order hot topic.

[0139] The second preprocessing unit 305 is configured to extract a process-related work order dataset from the original work order dataset using the trained representation extraction model, and preprocess the process-related work order dataset to obtain a second corpus.

[0140] The second calculation unit 306 is configured to construct a second topic model based on the second corpus, and calculate topics corresponding to all process-related work orders based on the second topic model.

[0141] The second statistical unit 307 is used to count the number of process-related work orders associated with each topic, and select the topic with the number of associated process-related work orders ranked in the top six preset positions as the second work order hot topic.

[0142] The third preprocessing unit 308 is configured to obtain an original process entry dataset, and preprocess the original process entry dataset to obtain a third corpus, where the third corpus includes a third word sequence.

[0143] The third calculation unit 309 is configured to calculate a topic corresponding to each process item using the second topic model based on the third word sequence.

[0144] The matching unit 310 is used to count all the process entries matched under a certain topic and generate a process entry list.

[0145] The extraction unit 311 is used to extract question and answer elements from all process-type work orders associated with each topic using the big model technology to obtain question and answer pairs corresponding to each topic.

[0146] It should be noted that the above 301-311 correspond to steps S101 to S111 in the embodiment of the work order data processing method based on the topic model. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment.

[0147] A work order data processing system based on a topic model, further comprising: The first visualization unit 312 is used to visualize the core terms of each original work order, the hot topics of the first work order, and the hot topics of the second work order by using word cloud technology.

[0148] A work order data processing system based on a topic model, further comprising: The fourth preprocessing unit 313 is configured to obtain an incremental work order, preprocess the incremental work order, and obtain a fourth term sequence.

[0149] The fourth calculation unit 314 is used to calculate the topic corresponding to a certain incremental work order based on the fourth term sequence using the second topic model, and then obtain the process entries, corresponding question-answer pairs and associated process-type work orders associated with the topic corresponding to the certain incremental work order.

[0150] The second visualization unit 315 is used to visualize the subject corresponding to a certain incremental work order and the process items associated with the subject corresponding to the incremental work order, the corresponding question and answer pairs, and the associated process-type work orders by using word cloud technology.

[0151] It should be noted that steps 313-315 above correspond to steps S201 to S203 in an embodiment of a method for processing work order data based on a topic model. The examples and application scenarios implemented by the modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiment. Embodiments of the present invention also provide a computer-readable storage medium storing a program or instructions that, when executed on a computer, causes the computer to execute the method for processing work order data based on a topic model as described in the above method embodiment.

[0152] An embodiment of the present invention also provides an electronic device, comprising: a processor, the processor being coupled to a memory, the processor being configured to read and execute a computer program stored in the memory to implement the work order data processing method based on a topic model as described in the above method embodiment.

[0153] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A work order data processing method based on a topic model, characterized in that: include: Obtain the original work order dataset; Extracting a process-related work order dataset from the original work order dataset using the trained representation extraction model, and preprocessing the process-related work order dataset to obtain a second corpus; Building a second topic model based on the second corpus, and calculating the topic corresponding to the process-related work order based on the second topic model; Count the number of process-related work orders associated with each topic, and select the topic with the sixth preset ranking in the number of associated process-related work orders as the second work order hot topic; Acquire an original process entry dataset, and preprocess the original process entry dataset to obtain a third corpus, wherein the third corpus includes a third word sequence; Based on the third term sequence, the second topic model is used to calculate and obtain a topic corresponding to each process item.

2. The method according to claim 1, characterized in that After obtaining the original work order data set, the following steps are also included: Preprocessing the original work order dataset to obtain a first corpus; Building a first topic model based on the first corpus, and calculating the core terms of all original work orders and the topics corresponding to all original work orders based on the first topic model; The number of original work orders associated with each topic is counted, and the topic whose number of associated original work orders ranks in the top four preset positions is selected as the first work order hot topic.

3. The method according to claim 1, characterized in that After calculating the topic corresponding to each process item using the second topic model, the method further includes: Count all process entries that match a certain topic and generate a list of process entries.

4. The method according to claim 1, wherein Calculating the topic corresponding to the process-related work order based on the second topic model also includes: Using big model technology, we extract question and answer elements from all process-related work orders associated with each topic and obtain the question and answer pairs corresponding to each topic.

5. The method according to claim 2, characterized in that Building a first topic model based on the first corpus, and calculating the topic core terms of all original work orders and the topics corresponding to all original work orders based on the first topic model, including: Based on the first corpus, constructing the first topic model using a sampling algorithm, the first topic model including the topic term distribution of the original work order and the topic distribution of the original work order; Based on the distribution of subject terms in the original work order, the dependence degree of each term on a certain subject and the proportion of occurrences of each term in the first corpus are calculated; Based on the degree of dependence of each term on a certain topic and the proportion of the number of occurrences of each term in the first corpus, a comprehensive score of each term ranked in the first preset position under a certain topic is calculated; According to the comprehensive score of each term ranked in the first preset position under the certain topic, the term ranked in the second preset position in the comprehensive score is selected from all the terms ranked in the first preset position under the certain topic as the core term of the certain topic; Based on the topic distribution of the original work order, a topic with an allocation probability ranking in the top third preset position is selected from all topics assigned to the original work order as the topic corresponding to the original work order; or based on the topic distribution of the original work order, a topic with an allocation probability higher than a second threshold is selected from all topics assigned to the original work order as the topic corresponding to the original work order.

6. The method according to claim 1, characterized in that Based on the second topic model, an incremental work order is processed, including: Obtaining an incremental work order, preprocessing the incremental work order, and obtaining a fourth term sequence; Based on the fourth term sequence, the second topic model is used to calculate the topic corresponding to the incremental work order, and then the process item, the corresponding question-answer pair, and the associated process-related work order are obtained. By using word cloud technology, the subject corresponding to the incremental work order and the process items associated with the subject corresponding to the incremental work order, the corresponding question and answer pairs, and the associated process-type work orders are visualized.

7. A work order data processing system based on a topic model, characterized in that: include: An acquisition unit, used to obtain the original work order data set; A second preprocessing unit is configured to extract a process-related work order dataset from the original work order dataset using the trained representation extraction model, and preprocess the process-related work order dataset to obtain a second corpus; A second computing unit, configured to construct a second topic model based on the second corpus, and calculate topics corresponding to the process-related work orders based on the second topic model; The second statistical unit is used to count the number of process-related work orders associated with each topic, and select the topic with the sixth preset ranking in the number of associated process-related work orders as the second work order hot topic; a third preprocessing unit, configured to obtain an original process entry dataset, and preprocess the original process entry dataset to obtain a third corpus, wherein the third corpus includes a third word sequence; The third calculation unit is configured to calculate a topic corresponding to each process item using the second topic model based on the third word sequence.

8. The system according to claim 7, characterized in that Also includes: A first preprocessing unit, configured to preprocess the original work order dataset to obtain a first corpus; A first computing unit is configured to construct a first topic model based on the first corpus, and calculate the topic core terms of all original work orders and the topics corresponding to all original work orders based on the first topic model; The first statistical unit is used to count the number of original work orders associated with each topic, and select the topic whose number of associated original work orders ranks in the top four preset positions as the first work order hot topic.

9. The system according to claim 7, wherein: Also includes: The matching unit is used to count all process entries matched under a certain topic and generate a list of process entries.

10. The system according to claim 7, wherein: Also includes: The extraction unit is used to use large model technology to extract question and answer elements from all process-type work orders associated with each topic, and obtain the question and answer pairs corresponding to each topic.

11. The system according to claim 7, wherein: Also includes: a fourth preprocessing unit, configured to obtain an incremental work order, preprocess the incremental work order, and obtain a fourth term sequence; a fourth calculation unit, configured to calculate, based on the fourth term sequence, a topic corresponding to a certain incremental work order using the second topic model, and further obtain a process entry, a corresponding question-answer pair, and an associated process-type work order associated with the topic corresponding to the certain incremental work order; The second visualization unit is used to use word cloud technology to visualize the subject corresponding to the incremental work order and the process items associated with the subject corresponding to the incremental work order, the corresponding question and answer pairs, and the associated process-type work orders.

12. A computer-readable storage medium, characterized in that A program or instruction is stored, and when the program or instruction is run on a computer, the computer is caused to execute the work order data processing method based on the topic model according to any one of claims 1 to 6.

13. An electronic device, characterized in that: comprising a processor coupled to a memory; The processor is configured to read and execute the computer program stored in the memory to implement the work order data processing method based on a topic model according to any one of claims 1 to 6.

Images