A Smart Work Order Quality Inspection Method

Abstract

This invention relates to an intelligent work order quality inspection method. The method includes: constructing and optimizing a first classification model, a second classification model, a third classification model, a fourth classification model, a fifth classification model, and a sixth classification model using Naive Bayes classification and a BERT-based text classification algorithm; and performing quality inspection on the work orders to be completed by combining the constructed and optimized models with the work order text and audio-to-text transcription. This invention improves the accuracy of quality inspection for completed work orders.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to an intelligent work order quality inspection method. Background Technology

[0002] Complaint tickets for 5G services are created through channels such as the new-generation customer service system and then transferred to the EOMS platform. The EOMS platform then dispatches them to the corresponding city / prefecture branch for processing. After processing, a report with closing comments is filled out and returned to the EOMS platform, which then performs quality inspection to audit each stage of the ticket processing. Currently, the quality inspection method relies on "fixed rules" formed from manual processing experience to audit the ticket text. This method depends on analyzing and structuring the work methods of experienced quality inspectors to create detection rules and a detection dictionary. However, 5G service ticket text is typically unstructured and non-fixed in format, resulting in low coverage of "fixed rules" and low inspection accuracy. Furthermore, auditing only the ticket text itself, due to insufficient information, can lead to misjudgments, causing qualified tickets to fail the quality inspection. Summary of the Invention

[0003] In view of this, the main objective of this invention is to provide an intelligent work order quality inspection method, which uses Naive Bayes classification and BERT-based text classification algorithm, and combines work order text and audio-to-text recording to complete the quality inspection of closed work orders, thereby improving the accuracy of quality inspection of closed work orders.

[0004] To achieve the above objectives, this application provides an intelligent work order quality inspection method, comprising: constructing and optimizing a model; and performing quality inspection on work orders to be completed and reported for quality inspection; wherein,

[0005] The constructed and optimized model includes:

[0006] Filter out historical work orders with the correct business category;

[0007] A Naive Bayes algorithm is used to construct the first classification model; the work order texts of each selected historical work order are segmented and vectorized to obtain business category vectors; training and test sets are constructed using business category vectors as features and business categories as labels to optimize the first classification model; a BERT classification algorithm is used to construct the second classification model; business category text content is extracted from the audio-to-text transcripts of each selected historical work order, and training and test sets are constructed using business categories as labels to optimize the second classification model.

[0008] A third classification model is constructed using the Naive Bayes algorithm. The work order texts of each selected historical work order are segmented and vectorized to obtain resolution status vectors. Using the resolution status vectors as features and the resolution status as labels, training and testing sets are constructed to optimize the third classification model. A fourth classification model is constructed using the BERT classification algorithm. The resolution status text content is extracted from the audio-to-text transcripts of each selected historical work order, and the resolution status is used as labels to construct training and testing sets to optimize the fourth classification model.

[0009] A fifth classification model is constructed using the Naive Bayes algorithm. The work order texts of each selected historical work order are segmented and vectorized to obtain the settlement opinion vector. Using the settlement opinion vector as the feature and the settlement opinion as the label, a training set and a test set are constructed to optimize the fifth classification model. A sixth classification model is constructed using the BERT classification algorithm. The settlement opinion text content is extracted from the audio-to-text transcripts of each selected historical work order. Using the settlement opinion as the label, a training set and a test set are constructed to optimize the sixth classification model.

[0010] The quality inspection of the work orders to be completed includes:

[0011] Obtain the work order text and audio-to-text recordings from the work orders pending quality inspection and completion;

[0012] The acquired work order text is segmented and vectorized to obtain a business category vector. This business category vector is then input into the first classification model to obtain the first predicted business category. The business category text content is extracted from the acquired audio-to-text recording and input into the second classification model to obtain the second predicted business category. The first and second predicted business categories are compared with the business category of the work order to be reported for quality inspection. If at least one comparison result is consistent, the business category of the work order to be reported for quality inspection is considered to be correct.

[0013] The acquired work order text is segmented and vectorized to obtain a resolution status vector. This resolution status vector is then input into a third classification model to obtain a first predicted resolution status. The resolution status text content is extracted from the acquired audio-to-text recording and input into a fourth classification model to obtain a second predicted resolution status. The first and second predicted resolution statuses are compared with the resolution status of the work order to be reported for quality inspection. If at least one comparison result is consistent, the resolution status of the work order to be reported for quality inspection is considered correct.

[0014] The obtained work order text is segmented and vectorized to obtain the closing opinion vector. The closing opinion vector is input into the fifth classification model to obtain the first predicted closing opinion. The closing opinion text content is extracted from the obtained audio-to-text recording and input into the sixth classification model to obtain the second predicted closing opinion. The first predicted closing opinion and the second predicted closing opinion are compared with the closing opinion of the work order to be inspected and closed. If at least one comparison result is consistent, the closing opinion of the work order to be inspected and closed is considered to be correct.

[0015] If the business category, resolution status, and completion information of the work order pending quality inspection are all correct, the work order pending quality inspection is deemed to have passed the quality inspection.

[0016] In one possible implementation, the step of segmenting and vectorizing the work order text to obtain the business category vector includes:

[0017] The BERT entity extraction model is used to extract business category text content from work order text;

[0018] The extracted business category text content and work order text are merged, and the merged text is processed by word segmentation algorithm;

[0019] The word vector algorithm is used to vectorize the segmented text to obtain business category vectors.

[0020] In another possible implementation, the solution vector obtained by segmenting and vectorizing the work order text includes:

[0021] The BERT entity extraction model is used to extract the resolution status text from the work order text;

[0022] The extracted resolution status text and work order text are merged, and the merged text is processed by word segmentation algorithm;

[0023] The word vector algorithm is used to vectorize the segmented text to obtain the solution vector.

[0024] In another possible implementation, the step of segmenting and vectorizing the work order text to obtain the report and opinion vector includes:

[0025] The BERT entity extraction model is used to extract the summary opinion text from the work order text;

[0026] The extracted summary opinion text and work order text are merged, and the merged text is processed by word segmentation algorithm;

[0027] The word vector algorithm is used to vectorize the segmented text to obtain the report opinion vector. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating an intelligent work order quality inspection method according to an embodiment of the present invention. Detailed Implementation

[0029] Specifically, the process of an intelligent work order quality inspection method according to an embodiment of the present invention is as follows: Figure 1 As shown, steps 101 and 102 are included.

[0030] Step 101: Build and optimize the model.

[0031] Step 101 includes steps 1011 to 1014, specifically as follows:

[0032] Step 1011: Filter out historical work orders with the correct business category.

[0033] Step 1012: Construct the first classification model using the Naive Bayes algorithm; segment and vectorize the work order text of each selected historical work order to obtain the business category vector; construct training and test sets using the business category vector as features and the business category as labels, and optimize the first classification model; construct the second classification model using the BERT classification algorithm; extract the business category text content from the audio-to-text transcripts of each selected historical work order, construct training and test sets using the business category as labels, and optimize the second classification model.

[0034] Step 1013: Construct a third classification model using the Naive Bayes algorithm; segment and vectorize the work order text of each selected historical work order to obtain a resolution status vector; construct training and testing sets using the resolution status vector as features and the resolution status as labels, and optimize the third classification model; construct a fourth classification model using the BERT classification algorithm; extract the resolution status text content from the audio-to-text transcription of each selected historical work order, construct training and testing sets using the resolution status as labels, and optimize the fourth classification model.

[0035] Step 1014: Construct the fifth classification model using the Naive Bayes algorithm; segment and vectorize the work order texts of each selected historical work order to obtain the completion opinion vector; construct training and testing sets using the completion opinion vector as features and the completion opinions as labels, and optimize the fifth classification model; construct the sixth classification model using the BERT classification algorithm; extract the completion opinion text content from the audio-to-text transcripts of each selected historical work order, construct training and testing sets using the completion opinions as labels, and optimize the sixth classification model.

[0036] Step 102: Conduct quality inspection on the work orders to be completed.

[0037] Step 102 includes steps 1021 to 1025, specifically as follows:

[0038] Step 1021: Obtain the work order text and audio-to-text recording from the work order pending quality inspection and completion.

[0039] Step 1022: Segment and vectorize the acquired work order text to obtain a business category vector. Input the business category vector into the first classification model to obtain the first predicted business category. Extract the business category text content from the acquired audio-to-text recording and input the business category text content into the second classification model to obtain the second predicted business category. Compare the first predicted business category and the second predicted business category with the business category of the work order to be reported for quality inspection. If at least one comparison result is consistent, the business category of the work order to be reported for quality inspection is considered to be correct.

[0040] Step 1023: Segment and vectorize the acquired work order text to obtain a resolution status vector. Input the resolution status vector into the third classification model to obtain the first predicted resolution status. Extract the resolution status text content from the acquired audio-to-text recording and input the resolution status text content into the fourth classification model to obtain the second predicted resolution status. Compare the first and second predicted resolution statuses with the resolution status of the work order to be reported for quality inspection. If at least one comparison result is consistent, the resolution status of the work order to be reported for quality inspection is considered correct.

[0041] Step 1024: Segment and vectorize the acquired work order text to obtain the closing opinion vector, input the closing opinion vector into the fifth classification model to obtain the first predicted closing opinion; extract the closing opinion text content from the acquired audio-to-text recording, input the closing opinion text content into the sixth classification model to obtain the second predicted closing opinion; compare the first predicted closing opinion and the second predicted closing opinion with the closing opinion of the work order to be inspected and closed respectively, and if at least one comparison result is consistent, then the closing opinion of the work order to be inspected and closed is considered to be correct.

[0042] Step 1025: If the business category, resolution status, and closing information of the work order pending quality inspection are all correct, the work order pending quality inspection is deemed to have passed quality inspection; otherwise, the work order pending quality inspection is submitted to manual quality inspection.

[0043] The enumerated values ​​for the service categories include: weak regional signal, slow network speed or inability to open web pages, inability to roam within or between provinces, and disconnection during internet access.

[0044] The resolution status enumeration values ​​include: resolved / unresolved.

[0045] The enumerated values ​​for the reported issues include: suspected terminal or terminal device settings problems, other terminal or terminal device settings problems, terminal problems (recovered after changing the device), suspected terminal or SIM card problems (suggesting the customer change the device or SIM card and try again), etc.

[0046] Here, the work order text is compiled by the salesperson based on the specific work order and is a relatively standardized text. Therefore, the first classification model, the third classification model, and the fifth classification model adopt the Naive Bayes algorithm, which can not only achieve good classification results, but also has the advantages of low model deployment resource requirements and fast inference speed.

[0047] Recording-to-text conversion suffers from issues such as colloquialisms and ASR (Audio-to-Text) translation errors, resulting in poor text quality. Therefore, the second, fourth, and sixth classification models employ the BERT classification algorithm, which has semantic understanding capabilities, to achieve better classification results.

[0048] If either the first or second predicted business category matches the business category of the work order to be inspected and closed, it is considered correct. If either the first or second predicted resolution status matches the resolution status of the work order to be inspected and closed, it is considered correct. If either the first or second predicted closure opinion matches the closure opinion of the work order to be inspected and closed, it is considered correct. This avoids the problem of misjudging consistent situations as inconsistent ones due to insufficient information when conducting quality inspection based solely on the work order text, thus preventing qualified closure work orders from passing quality inspection. This improves the accuracy of quality inspection for closure work orders.

[0049] In one possible implementation, steps 1012 and 1022, whereby the work order text is segmented and vectorized to obtain a business category vector, includes:

[0050] The BERT entity extraction model is used to extract business category text content from work order text;

[0051] The extracted business category text content and work order text are merged, and the merged text is processed by word segmentation algorithm;

[0052] The word vector algorithm is used to vectorize the segmented text to obtain business category vectors.

[0053] The phrase "extracting business category text content from the work order text" refers to the original sentence in the complaint content of the work order text that indicates the business category. For example, if the complaint content is "Dialing some numbers from this machine will show a busy signal, but other numbers are dialed normally, and I have confirmed that the other party's number is not on a blacklist. I have tried different phones and SIM cards, but the problem persists! I request a solution!" and the business category is "mobile phone calls", then the extracted business category text content would be: "Dialing some numbers from this machine will show a busy signal".

[0054] The word segmentation algorithm can be an N-gram algorithm based on a statistical language model; the word vector algorithm can be word2vec.

[0055] Here, merging the extracted business category text content and work order text can avoid the errors caused by extracting the business category text content, while increasing the weight of the business category text content in the business category vector features, increasing the accuracy of the first predicted business category, and thus increasing the accuracy of the quality inspection of the closed work orders.

[0056] In another possible implementation, in steps 1013 and 1023, the step of segmenting and vectorizing the work order text to obtain the solution vector includes:

[0057] The BERT entity extraction model is used to extract the resolution status text from the work order text;

[0058] The extracted resolution status text and work order text are merged, and the merged text is processed by word segmentation algorithm;

[0059] The word vector algorithm is used to vectorize the segmented text to obtain the solution vector.

[0060] Among them, extracting the resolution status text content from the work order text refers to the original sentence describing the resolution status in the additional summary information of the work order text.

[0061] The word segmentation algorithm can be N-Gram; the word vector algorithm can be word2vec.

[0062] Here, merging the extracted resolution status text content and work order text can avoid the errors caused by extracting the resolution status text content, while increasing the weight of the resolution status text content in the resolution status vector features, increasing the accuracy of the first predicted resolution status, and thus increasing the accuracy of the quality inspection of the reported work orders.

[0063] In another possible implementation, steps 1014 and 1024, where the step of segmenting and vectorizing the work order text to obtain the report and opinion vector, includes:

[0064] The BERT entity extraction model is used to extract the summary opinion text from the work order text;

[0065] The extracted summary opinion text and work order text are merged, and the merged text is processed by word segmentation algorithm;

[0066] The word vector algorithm is used to vectorize the segmented text to obtain the report opinion vector.

[0067] Among them, extracting the content of the settlement opinion text from the work order text refers to the original sentence describing the settlement opinion in the supplementary settlement information of the work order text.

[0068] The word segmentation algorithm can be N-Gram; the word vector algorithm can be word2vec.

[0069] Here, merging the extracted report settlement opinion text content and work order text can avoid the errors caused by extracting the report settlement opinion text content, while increasing the weight of the report settlement opinion text content in the report settlement opinion vector features, increasing the accuracy of the first predicted report settlement opinion, and thus increasing the accuracy of the report settlement work order quality inspection.

[0070] In another possible implementation, steps 1012-1014 and 1022-1024, namely, extracting business category text content from the audio-to-text recording, extracting resolution status text content from the audio-to-text recording, and extracting conclusion opinion text content from the audio-to-text recording, are all implemented using the BERT entity extraction model.

[0071] In another possible implementation, steps 1012–1014, optimizing the first classification model, optimizing the third classification model, and optimizing the fifth classification model are all implemented using the BernoulliNB function in the scikit-leran package.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A smart work order quality inspection method, characterized in that, include: Build and optimize models; Quality inspection is conducted on work orders that have been submitted for quality inspection completion; among them... The constructed and optimized model includes: Filter out historical work orders with the correct business category; A Naive Bayes algorithm is used to construct the first classification model; the work order texts of each selected historical work order are segmented and vectorized to obtain business category vectors; training and test sets are constructed using business category vectors as features and business categories as labels to optimize the first classification model; a BERT classification algorithm is used to construct the second classification model; business category text content is extracted from the audio-to-text transcripts of each selected historical work order, and training and test sets are constructed using business categories as labels to optimize the second classification model. A third classification model is constructed using the Naive Bayes algorithm. The work order texts of each selected historical work order are segmented and vectorized to obtain resolution status vectors. Using the resolution status vectors as features and the resolution status as labels, training and testing sets are constructed to optimize the third classification model. A fourth classification model is constructed using the BERT classification algorithm. The resolution status text content is extracted from the audio-to-text transcripts of each selected historical work order, and the resolution status is used as labels to construct training and testing sets to optimize the fourth classification model. A fifth classification model is constructed using the Naive Bayes algorithm. The work order texts of each selected historical work order are segmented and vectorized to obtain the settlement opinion vector. Using the settlement opinion vector as the feature and the settlement opinion as the label, a training set and a test set are constructed to optimize the fifth classification model. A sixth classification model is constructed using the BERT classification algorithm. The settlement opinion text content is extracted from the audio-to-text transcripts of each selected historical work order. Using the settlement opinion as the label, a training set and a test set are constructed to optimize the sixth classification model. The quality inspection of the work orders to be completed includes: Obtain the work order text and audio-to-text recordings from the work orders pending quality inspection and completion; The acquired work order text is segmented and vectorized to obtain a business category vector. This business category vector is then input into the first classification model to obtain the first predicted business category. The business category text content is extracted from the acquired audio-to-text recording and input into the second classification model to obtain the second predicted business category. The first and second predicted business categories are compared with the business category of the work order to be reported for quality inspection. If at least one comparison result is consistent, the business category of the work order to be reported for quality inspection is considered to be correct. The acquired work order text is segmented and vectorized to obtain a resolution status vector. This resolution status vector is then input into a third classification model to obtain a first predicted resolution status. The resolution status text content is extracted from the acquired audio-to-text recording and input into a fourth classification model to obtain a second predicted resolution status. The first and second predicted resolution statuses are compared with the resolution status of the work order to be reported for quality inspection. If at least one comparison result is consistent, the resolution status of the work order to be reported for quality inspection is considered correct. The obtained work order text is segmented and vectorized to obtain the closing opinion vector. The closing opinion vector is input into the fifth classification model to obtain the first predicted closing opinion. The closing opinion text content is extracted from the obtained audio-to-text recording and input into the sixth classification model to obtain the second predicted closing opinion. The first predicted closing opinion and the second predicted closing opinion are compared with the closing opinion of the work order to be inspected and closed. If at least one comparison result is consistent, the closing opinion of the work order to be inspected and closed is considered to be correct. If the business category, resolution status, and completion information of the work order pending quality inspection are all correct, the work order pending quality inspection is deemed to have passed the quality inspection.

2. The method according to claim 1, characterized in that, The process of segmenting and vectorizing the work order text to obtain the business category vector includes: The BERT entity extraction model is used to extract business category text content from work order text; The extracted business category text content and work order text are merged, and the merged text is processed by word segmentation algorithm; The word vector algorithm is used to vectorize the segmented text to obtain business category vectors.

3. The method according to claim 1, characterized in that, The solution vector obtained by segmenting and vectorizing the work order text includes: The BERT entity extraction model is used to extract the resolution status text from the work order text; The extracted resolution status text and work order text are merged, and the merged text is processed by word segmentation algorithm; The word vector algorithm is used to vectorize the segmented text to obtain the solution vector.

4. The method according to claim 1, characterized in that, The step of segmenting and vectorizing the work order text to obtain the report and opinion vector includes: The BERT entity extraction model is used to extract the summary opinion text from the work order text; The extracted summary opinion text and work order text are merged, and the merged text is processed by word segmentation algorithm; The word vector algorithm is used to vectorize the segmented text to obtain the report opinion vector.

Images