Label verification method and device based on machine learning and readable storage medium

Abstract

The invention provides a label verification method based on machine learning, and the method comprises the steps: obtaining a communication audio, and obtaining a task label set by a customer service system for a potential customer; converting the communication audio into a text by using an automatic voice recognition technology; performing error correction processing on the text by adopting a text error correction model based on deep learning to obtain a to-be-processed text; performing label extraction on the to-be-processed text by using the label extraction model to obtain an extracted label; comparing the consistency between the task label and the extraction label; if the consistency exceeds a preset value, determining that the task label meets a verification standard; and if the consistency does not exceed the preset value, determining that the task label does not conform to the verification standard. Based on the mode, the processing efficiency is improved, a large number of customer service communication audios can be processed more timely, and the requirement for rapid development of services is met. The accuracy is improved, and subsequent problems caused by label errors are reduced, for example, disputes and complaints between decoration companies and owners due to wrong house label information are avoided.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a label verification method, device, and readable storage medium based on machine learning. Background Art

[0002] In today's service industry, quality control of customer communication is crucial, especially when it comes to the accuracy of task labels when communicating with potential customers. As business scale continues to expand, the volume of calls between customer service representatives and potential customers in the service industry has exploded. For example, at a certain renovation company, as its business expanded, its call volume increased significantly, with over 1,000 recordings collected daily.

[0003] However, the currently commonly used manual quality inspection method has many drawbacks. In the manual quality inspection mode, staff need to listen to the call recordings between customer service and owners one by one, and compare the recording content with the house label filled in the customer service system to determine the validity of the label. However, this method is extremely inefficient. Limited by manpower and working hours, the average length of a successfully collected recording is 5-7 minutes. A quality inspector works 8 hours a day (480 minutes) and can quality inspect a maximum of about 68 recordings. If the quality inspection department has only 4 people, a maximum of 272 recordings can be quality inspected per day. The quality inspection coverage rate is low, which makes some customer service staff have a fluke mentality and it is difficult to effectively guarantee the quality of customer service work.

[0004] Furthermore, manual quality inspections are susceptible to subjective factors, such as the inspector's mood and fatigue, which can lead to misjudgments about label accuracy. Furthermore, manual quality inspections struggle to fully and accurately identify customer service communication issues, such as forcing owners to mislead, misrepresenting their intentions, or failing to accurately understand their needs. This can lead to errors in the collected house label information. These erroneous information can easily lead to complaints from both parties when renovation companies and owners subsequently communicate, severely impacting the normal operation of their business and their reputation. Summary of the Invention

[0005] The present invention aims to provide a label verification method based on machine learning, aiming to solve the problem that existing manual label verification is inefficient and may have label verification errors. The label verification method based on machine learning provided in this application includes:

[0006] Acquire communication audio, wherein the communication audio is the audio of the communication between the human customer service and the potential customer;

[0007] Obtaining a task tag set by the customer service system for the potential customer;

[0008] Converting the communication audio into text using automatic speech recognition technology;

[0009] Performing error correction processing on the text using a text error correction model based on deep learning to obtain a text to be processed;

[0010] Using a label extraction model to extract labels from the text to be processed to obtain extracted labels;

[0011] comparing the consistency between the task label and the extracted label;

[0012] If the consistency exceeds a preset value, the task tag is deemed to meet the verification standard;

[0013] If the consistency does not exceed the preset value, it is determined that the task tag does not meet the verification standard.

[0014] Based on the machine learning-based label verification method provided in the first aspect of the embodiment of the present application, optionally,

[0015] The text error correction model based on deep learning includes: a pinyin similarity rule algorithm.

[0016] Based on the machine learning-based label verification method provided in the first aspect of the embodiment of the present application, optionally, the deep learning-based text error correction model includes: a keyword replacement algorithm.

[0017] Based on the machine learning-based label verification method provided in the first aspect of the embodiment of the present application, optionally, the deep learning-based text error correction model includes: a text similarity replacement algorithm.

[0018] Based on the machine learning-based label verification method provided in the first aspect of the embodiment of the present application, optionally, extracting labels from the to-be-processed text using a label extraction model to obtain extracted labels includes:

[0019] Processing the text to be processed using a basic language model to obtain multiple specific intent dialogue groups;

[0020] The multiple specific-purpose models are used to process the multiple specific-intent dialogue groups respectively to obtain extracted labels, and there is a corresponding relationship between the specific-purpose models and the specific-intent dialogue groups.

[0021] Based on the machine learning-based label verification method provided in the first aspect of the embodiment of the present application, optionally, extracting labels from the to-be-processed text using a label extraction model to obtain extracted labels includes:

[0022] Using a first label extraction model to extract labels from the text to be processed to obtain a first extracted label;

[0023] Using a second label extraction model to extract labels from the text to be processed to obtain second extracted labels;

[0024] Cross-validation is performed on the first extracted label and the second extracted label to obtain the extracted label.

[0025] Based on the machine learning-based label verification method provided in the first aspect of the embodiment of the present application, optionally, using a label extraction model to extract labels from the to-be-processed text to obtain extracted labels includes:

[0026] Determining whether the text to be processed exceeds a preset length;

[0027] If the text to be processed exceeds a preset length, the text to be processed is split, and the splitting results are processed using the label extraction model to obtain the extracted labels;

[0028] If the text to be processed does not exceed the preset length, a label extraction model is used to extract labels from the text to be processed to obtain extracted labels.

[0029] A second aspect of the embodiments of the present application provides a label verification device based on machine learning, including:

[0030] A first acquisition unit is configured to acquire communication audio, wherein the communication audio is audio of communication between a human customer service representative and a potential customer;

[0031] A second acquiring unit is configured to acquire a task tag set by the customer service system for the potential customer;

[0032] A conversion unit, configured to convert the communication audio into text using automatic speech recognition technology;

[0033] An error correction unit, configured to perform error correction processing on the text using a text error correction model based on deep learning to obtain a text to be processed;

[0034] An extraction unit, configured to extract labels from the text to be processed using a label extraction model to obtain extracted labels;

[0035] a comparing unit, configured to compare the consistency between the task label and the extracted label;

[0036] a compliance unit, configured to determine that the task tag meets a verification standard if the consistency exceeds a preset value;

[0037] The non-compliance unit is configured to determine that the task tag does not comply with a verification standard if the consistency does not exceed a preset value.

[0038] Based on the machine learning-based label verification device provided in the second aspect of the embodiment of the present application, optionally, the deep learning-based text error correction model includes: a pinyin similarity rule algorithm.

[0039] Based on the machine learning-based label verification device provided in the second aspect of the embodiment of the present application, optionally, the deep learning-based text error correction model includes: a keyword replacement algorithm.

[0040] Based on the machine learning-based label verification device provided in the second aspect of the embodiment of the present application, optionally, the deep learning-based text error correction model includes: a text similarity replacement algorithm.

[0041] Based on the machine learning-based label verification device provided in the second aspect of the embodiment of the present application, optionally, the extraction unit is specifically configured to:

[0042] The label extraction model is used to extract labels from the text to be processed, and obtaining the extracted labels includes:

[0043] Processing the text to be processed using a basic language model to obtain multiple specific intent dialogue groups;

[0044] Using a plurality of specific purpose models, processing the plurality of specific intent dialogue groups respectively to obtain extracted labels, wherein the specific purpose models have a corresponding relationship with the specific intent dialogue groups;

[0045] Based on the machine learning-based label verification device provided in the second aspect of the embodiment of the present application, optionally, the extraction unit is specifically configured to:

[0046] Using a first label extraction model to extract labels from the text to be processed to obtain a first extracted label;

[0047] Using a second label extraction model to extract labels from the text to be processed to obtain second extracted labels;

[0048] Cross-validation is performed on the first extracted label and the second extracted label to obtain the extracted label.

[0049] Based on the machine learning-based label verification device provided in the second aspect of the embodiment of the present application, optionally, the extraction unit is specifically used to: determine whether the text to be processed exceeds a preset length;

[0050] If the text to be processed exceeds a preset length, the text to be processed is split, and the splitting results are processed using the label extraction model to obtain the extracted labels;

[0051] If the text to be processed does not exceed the preset length, a label extraction model is used to extract labels from the text to be processed to obtain extracted labels.

[0052] A third aspect of the embodiments of the present application provides an information collection device based on machine learning, including:

[0053] CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply;

[0054] The memory is a short-term storage memory or a persistent storage memory;

[0055] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory on the device to perform the method as described in any one of the first aspects of the embodiments of the present application.

[0056] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described in any one of the first aspects of the embodiments of the present application.

[0057] A fifth aspect of the embodiments of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any one of the methods described in the first aspect of the embodiments of the present application.

[0058] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: the embodiments of the present application provide a label verification method based on machine learning, including: obtaining communication audio, wherein the communication audio is the communication audio between manual customer service and potential customers; obtaining the task label set by the customer service system for the potential customer; converting the communication audio into text using automatic speech recognition technology; using a text error correction model based on deep learning to perform error correction processing on the text to obtain a text to be processed; using a label extraction model to perform label extraction on the text to be processed to obtain an extracted label; comparing the consistency between the task label and the extracted label; if the consistency exceeds a preset value, it is determined that the task label meets the verification standard; if the consistency does not exceed the preset value, it is determined that the task label does not meet the verification standard. Based on the above method, the processing efficiency is improved, and a large amount of customer service communication audio can be processed more promptly to meet the needs of rapid business development. The accuracy rate is improved and subsequent problems caused by label errors are reduced, such as avoiding disputes and complaints between decoration companies and owners due to incorrect house label information. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] To more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. A person of ordinary skill in the art can also derive other drawings based on the provided drawings without inventive effort. It should be understood that the drawings provided in this section are only used to better understand the present solution and do not constitute a limitation of the present application.

[0060] Figure 1 This is a flow chart of an embodiment of the label verification method based on machine learning provided in this application.

[0061] Figure 2 This is another flowchart of an embodiment of the label verification method based on machine learning provided in this application.

[0062] Figure 3 A structural diagram of the machine learning-based label verification method device provided in this application.

[0063] Figure 4 This is another structural diagram of the machine learning-based label verification method device provided in this application. DETAILED DESCRIPTION

[0064] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application are clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this application. At the same time, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0065] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0066] In today's service industry, quality control of customer communication is crucial, especially when it comes to the accuracy of task labels when communicating with potential customers. As business scale continues to expand, the volume of calls between customer service representatives and potential customers in the service industry has exploded. For example, at a certain renovation company, as its business expanded, its call volume increased significantly, with over 1,000 recordings collected daily.

[0067] However, the currently commonly used manual quality inspection method has many drawbacks. In the manual quality inspection mode, staff need to listen to the call recordings between customer service and owners one by one, and compare the recording content with the house label filled in the customer service system to determine the validity of the label. However, this method is extremely inefficient. Limited by manpower and working hours, the average length of a successfully collected recording is 5-7 minutes. A quality inspector works 8 hours a day (480 minutes) and can quality inspect a maximum of about 68 recordings. If the quality inspection department has only 4 people, a maximum of 272 recordings can be quality inspected per day. The quality inspection coverage rate is low, which makes some customer service staff have a fluke mentality and it is difficult to effectively guarantee the quality of customer service work.

[0068] Furthermore, manual quality inspections are susceptible to subjective factors, such as the inspector's mood and fatigue, which can lead to misjudgments about label accuracy. Furthermore, manual quality inspections struggle to fully and accurately identify customer service communication issues, such as forcing owners to mislead, misrepresenting their intentions, or failing to accurately understand their needs. This can lead to errors in the collected house label information. These erroneous information can easily lead to complaints from both parties when renovation companies and owners subsequently communicate, severely impacting the normal operation of their business and their reputation.

[0069] To solve the above problems, the present invention provides a label verification method based on machine learning. Figure 1 An embodiment of the label verification method based on machine learning provided in this application includes: steps 101 to 105.

[0070] 101. Obtain communication audio.

[0071] Acquire audio communication between a live customer service representative and a potential customer. In real-world scenarios, this audio can be captured using methods such as phone recording systems and voice recording features in online customer service chats. This captured audio serves as essential data for subsequent processing and analysis, containing detailed information about the conversation between the customer and the representative. After acquiring the audio, some preprocessing is typically required. The presence of various noises in the call environment, such as background noise and electrical current, can affect the accuracy of subsequent speech recognition. Therefore, a noise reduction algorithm is required to remove or mitigate the effects of these noises.

[0072] 102. Obtain a task tag set by the customer service system for the potential customer.

[0073] Task tags are set by the human customer service representative during an audio conversation with the potential customer. These tags reflect the representative's assessment and summary of the potential customer's situation during the conversation, such as the type of customer needs and level of purchase intent. Extracting these task tags from the customer service system provides a reference standard for subsequent comparison with tags extracted from the audio text. Task tags are set by the human customer service representative during an audio conversation with the potential customer, summarizing and identifying the potential customer's characteristics, needs, and intentions. These tags concisely reflect the representative's understanding and assessment of the communication content. Companies can further categorize customers based on task tags to provide personalized services and product recommendations for different types of customers. However, different customer service representatives have varying levels of understanding of business knowledge, communication skills, and ability to assess customer needs. Furthermore, the volume of information involved in customer communication can lead some representatives to miss key information. For example, during a real estate renovation conversation, a customer mentioned a specific requirement for home furnishings, such as "using a specific brand of wood panels." However, due to inadequate recording or distraction, the representative failed to accurately assign this requirement to a task tag. This can lead to subsequent property recommendations failing to meet customer needs, reducing customer satisfaction. During busy periods, customer service staff face significant workload pressure and can easily become fatigued by handling numerous customer communications. Long, intensive shifts can impair customer service focus and judgment, leading to negligence when setting task labels. For example, mistakenly assigning a customer's purchase intention of "considering renovation in the near future" to "considering renovation in the long term" can cause a company to misjudge customer needs and miss out on optimal marketing opportunities. Therefore, further verification of label accuracy is necessary.

[0074] 103. Convert the communication audio into text using automatic speech recognition technology.

[0075] Specifically, automatic speech recognition (ASR) technology is based on deep learning models, such as end-to-end convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their variants (such as LSTMs and GRUs). These models are trained on large amounts of speech data, learning the mapping between speech signals and text, thereby converting input speech signals into corresponding text.

[0076] During the speech-to-text conversion process, the system can pre-define tags for customer service and customer voices, such as using specific symbols (such as "Customer Service:" and "Customer:") to distinguish between the two speakers. Before audio-to-text conversion, audio processing technology is used to extract the timbre characteristics of the speech. Different people have unique timbre, so a timbre classification model can be trained to classify the speech in the audio into two categories: customer service and customer voice. For example, a Gaussian mixture model (GMM) can be used to model timbre characteristics and train the model to distinguish between customer service and customer voices. In actual applications, audio is input into the trained model, and the audio clips are classified based on the output. Subsequent speech-to-text conversion and semantic recognition are then performed. It is understandable that different ASR technologies may vary in recognition accuracy, processing speed, and applicable scenarios. In actual applications, appropriate technologies and services can be selected based on needs and actual circumstances. Furthermore, for speech in certain domains, targeted model training may be required to improve recognition accuracy.

[0077] 104. Use a text error correction model based on deep learning to perform error correction processing on the text to obtain a text to be processed.

[0078] Specifically, the model learns the rules and patterns of text correction by learning from a large number of correct and incorrect text pairs. When given an incorrect text input, the model outputs the corrected text. Understandably, the performance of text correction models is affected by factors such as the quality and quantity of training data and the choice of model architecture. In practical applications, continuous model optimization and updated training data are necessary to improve correction accuracy. Furthermore, special vocabulary and technical terminology may require additional processing.

[0079] 105. Use a label extraction model to extract labels from the text to be processed to obtain extracted labels.

[0080] The model is trained through learning, setting up a large number of training sets with preset labels, and acquiring the ability to extract labels. The label extraction model is directly used to extract labels from the text to be processed to obtain extracted labels. For example, if expressions such as "European style" and "Mediterranean style" appear in the text, the "decoration style" label is extracted and corresponds to the specific style type. The model type can be a deep learning model. For example, recurrent neural networks (RNN) and their variants (LSTM, GRU) can process sequence data and are suitable for processing contextual information in texts; convolutional neural networks (CNN) can capture local features in texts; models based on Transformer architecture (such as BERT) are pre-trained on large-scale text data, learn rich language knowledge and semantic representations, and perform well in label extraction tasks.

[0081] To improve model performance, effective features need to be extracted from the text to be processed. Common features include lexical features (such as word frequency and part of speech), syntactic features (such as sentence structure), and semantic features (such as word vectors and topic models).

[0082] 106. Compare the consistency between the task label and the extracted label.

[0083] Specifically, the consistency between the task labels extracted by the customer service and the extracted labels extracted by the model is compared. A preset value (such as 0.8) can be set as the judgment standard. When the calculated similarity exceeds the preset value, it is considered that the consistency exceeds the preset value; otherwise, it is considered that the consistency does not exceed the preset value.

[0084] 107. Determine that the task tag meets the verification criteria.

[0085] If the consistency is determined to exceed the preset value in step 106, the system automatically marks the task tag as meeting the verification standard and records relevant information, such as the customer service representative, communication time, task tag content, extracted tag content, similarity value, etc. This information can be used for subsequent statistical analysis and quality assessment.

[0086] 108. Determine that the task tag does not meet the verification standard.

[0087] When it is determined in step 106 that the consistency does not exceed the preset value, the system automatically marks the task label as not meeting the verification standard and may generate a warning message. The warning information may include the difference between the task label and the extracted label, the similarity value, etc., so that the customer service staff can understand the problem. At the same time, the warning information is sent to the relevant management personnel and customer service staff to remind them to deal with it. In addition to the warning, the customer service staff can also be asked to correct the task label and conduct a second verification; provide training and guidance to the customer service staff to improve the accuracy of their label settings, etc. The specific operations can be determined according to the actual situation and are not limited here.

[0088] The overall processing flow can be referred to Figure 2From the above technical solutions, it can be seen that the embodiments of the present application have the following advantages: the embodiments of the present application provide a label verification method based on machine learning, including: obtaining communication audio, which is the communication audio between manual customer service and potential customers; obtaining the task label set by the customer service system for the potential customer; converting the communication audio into text using automatic speech recognition technology; using a text error correction model based on deep learning to perform error correction processing on the text to obtain a text to be processed; using a label extraction model to perform label extraction on the text to be processed to obtain an extracted label; comparing the consistency between the task label and the extracted label; if the consistency exceeds a preset value, it is determined that the task label meets the verification standard; if the consistency does not exceed the preset value, it is determined that the task label does not meet the verification standard. Based on the above method, the processing efficiency is improved, and a large amount of customer service communication audio can be processed more promptly to meet the needs of rapid business development. The accuracy rate is improved and subsequent problems caused by label errors are reduced, such as avoiding disputes and complaints between decoration companies and owners due to incorrect house label information.

[0089] The above content provides an overall description of this plan. The following describes each detailed link of the plan:

[0090] 1. Various characteristic settings of text error correction models based on deep learning.

[0091] Specifically, since the system is applied to a decoration platform system, the text error correction model based on deep learning needs to include a variety of characteristic settings, including:

[0092] (1) Pinyin similarity rule algorithm

[0093] The algorithm's core principle is Chinese pinyin. In Chinese, some characters have identical or similar pronunciations, which can easily lead to errors during speech recognition. By calculating pinyin similarity, incorrect text with similar pronunciations is corrected. Because this solution is applied to a renovation platform, certain terms related to the renovation field will be adjusted. For example, "old house renovation" and "just put renovation" are very similar in pinyin and should be recognized as "old house renovation."

[0094] (2) Keyword replacement algorithm.

[0095] In the communication between customer service and customers, there are strict regulations on certain keywords or statements, and customer service staff must convey them accurately. However, due to reasons such as speech recognition errors or slips of the tongue, these keywords may be incorrect. By analyzing a large amount of text, common error forms are determined and then forced replacement is carried out. For example, the standard expression "one-stop decoration guarantee platform" is often recognized as "one-stop decoration high platform". Through the keyword replacement algorithm, when "one-stop decoration high platform" is detected in the text, it is forced to be replaced with "one-stop decoration guarantee platform".

[0096] (3) Text similarity replacement algorithm

[0097] In Chinese, although some words have slightly different spellings, their semantics are similar. By calculating the semantic similarity between texts, incorrect or similar expressions are replaced with standard expressions. For example, for "raw house", calculate its semantic similarity with "roughcast house" in the standard vocabulary library. If the similarity exceeds the threshold, then "raw house" is replaced with "roughcast house".

[0098] 2. Different implementation methods of extracting labels from the text to be processed using a label extraction model:

[0099] (1) Combination of a basic language model and a specific-purpose model

[0100] Use a basic language model to process the text to be processed to obtain multiple specific-intent dialogue groups; use multiple specific-purpose models to process the multiple specific-intent dialogue groups respectively to obtain extraction labels, and there is a corresponding relationship between the specific-purpose model and the specific-intent dialogue group.

[0101] Specifically, first use a basic language model to process the text to be processed and select a suitable basic language model. These models have been pre-trained on a large amount of text data and have strong language understanding capabilities.

[0102] Input the dialogue text between customer service and customers into the basic language model for processing. The basic language model will perform semantic analysis on the text and divide it into multiple specific-intent dialogue groups according to the intent of the dialogue. For example, in the field of decoration, the dialogue may be divided into specific-intent dialogue groups such as "decoration style consultation", "budget discussion", "material selection", "construction period arrangement", etc.

[0103] Specific-purpose models are then used to process specific-intent conversation groups. For each specific-intent conversation group, a corresponding specific-purpose model is trained. These models are specifically designed to handle specific types of conversations, resulting in higher relevance and accuracy. Each specific-intent conversation group is input into the corresponding specific-purpose model for processing. For example, for the "decoration style consultation" conversation group, a specially trained decoration style recognition model is used to extract labels such as "European style," "Chinese style," and "modern minimalist style." For the "material selection" conversation group, a material recognition model is used to extract labels such as "marble," "wood flooring," and "latex paint."

[0104] Using a basic language model for intent classification and a specific-purpose model for label extraction can more accurately capture key information in the text and reduce label extraction errors. Different specific-purpose models can be optimized for different types of conversations, better adapting to the diverse business needs of the interior design industry.

[0105] (2) Multi-model cross-validation.

[0106] The method comprises: extracting labels from the to-be-processed text using a first label extraction model to obtain a first extracted label;

[0107] Select a suitable label extraction model as the first label extraction model, such as a rule-based model, a machine learning model, or a deep learning model.

[0108] Specifically, the conversation text between the customer service representative and the client is input into the first label extraction model, which extracts labels based on its own algorithms and rules, generating first extracted labels. For example, a rule-based model might extract labels such as "renovation budget" and "renovation area" based on pre-set keywords and rules.

[0109] A second label extraction model is used to extract labels from the text to be processed to obtain second extracted labels. A different type of label extraction model is selected as the second label extraction model to increase the diversity and reliability of the results. For example, if the first label extraction model is a rule-based model, the second label extraction model can be a machine learning model.

[0110] The same conversation text is input into the second label extraction model to obtain the second extracted label.

[0111] Cross-validate the first extracted label and the second extracted label to obtain the extracted label. Compare the first extracted label and the second extracted label, and determine the final extracted label based on specific rules. Common methods include intersection and voting. For example, if both models extract the label "European style," then this label is retained in the final extracted label. If one model extracts "wooden flooring" and the other does not, further analysis is required to determine whether to retain this label.

[0112] Multi-model cross-validation can reduce the errors and limitations of a single model and improve the accuracy and reliability of label extraction. Different models understand and process text differently, so combining multiple models can better handle a variety of complex language expressions and data situations.

[0113] (3) Text length adaptive processing.

[0114] The method comprises: determining whether the text to be processed exceeds a preset length;

[0115] Set a preset length based on the input limitations of the label extraction model and actual business needs.

[0116] Determine the length of the conversation text between customer service and customers to determine whether it exceeds the preset length.

[0117] If the text to be processed exceeds a preset length, the text to be processed is split, and the splitting results are processed using the label extraction model to obtain the extracted labels;

[0118] Specifically, if the conversation text exceeds the preset length, it needs to be split. The splitting method can be based on sentence boundaries, paragraphs, etc., to ensure that the split text segments have complete semantics. The split text segments are input into the label extraction model for processing, and then the extracted labels of each segment are merged and sorted to obtain the final extracted labels. For example, for a long conversation text about renovation needs, it can be split according to different topics, and the tags related to each topic can be extracted separately.

[0119] If the text to be processed does not exceed the preset length, the tag extraction model is used to extract the tags from the text to be processed to obtain the extracted tags. If the length does not exceed the set length, the text is processed according to the normal process.

[0120] The above content describes the machine learning-based label verification method provided by this application. To support the implementation of the above embodiment, this embodiment of the application provides a machine learning-based label verification device, including:

[0121] The first acquisition unit 301 is used to acquire communication audio, where the communication audio is the audio of the communication between the human customer service and the potential customer;

[0122] The second acquisition unit 302 is used to acquire the task label set by the customer service system for the potential customer;

[0123] A conversion unit 303 is configured to convert the communication audio into text using automatic speech recognition technology;

[0124] An error correction unit 304 is configured to perform error correction processing on the text using a text error correction model based on deep learning to obtain a text to be processed;

[0125] An extraction unit 305 is configured to extract labels from the text to be processed using a label extraction model to obtain extracted labels;

[0126] A comparison unit 306, configured to compare the consistency between the task label and the extracted label;

[0127] A compliance unit 307 is configured to determine that the task tag meets a verification standard if the consistency exceeds a preset value;

[0128] The non-compliance unit 308 is configured to determine that the task tag does not comply with a verification standard if the consistency does not exceed a preset value.

[0129] Optionally, the deep learning-based text error correction model includes: a pinyin similarity rule algorithm.

[0130] Optionally, the deep learning-based text error correction model includes: a keyword replacement algorithm.

[0131] Optionally, the deep learning-based text error correction model includes: a text similarity replacement algorithm.

[0132] Optionally, the extraction unit is specifically configured to:

[0133] The label extraction model is used to extract labels from the text to be processed, and obtaining the extracted labels includes:

[0134] Processing the text to be processed using a basic language model to obtain multiple specific intent dialogue groups;

[0135] Using a plurality of specific purpose models, processing the plurality of specific intent dialogue groups respectively to obtain extracted labels, wherein the specific purpose models have a corresponding relationship with the specific intent dialogue groups;

[0136] Optionally, the extraction unit is specifically configured to:

[0137] Using a first label extraction model to extract labels from the text to be processed to obtain a first extracted label;

[0138] Using a second label extraction model to extract labels from the text to be processed to obtain second extracted labels;

[0139] Cross-validation is performed on the first extracted label and the second extracted label to obtain the extracted label.

[0140] Optionally, the extraction unit is specifically configured to: determine whether the text to be processed exceeds a preset length;

[0141] If the text to be processed exceeds a preset length, the text to be processed is split, and the splitting results are processed using the label extraction model to obtain the extracted labels;

[0142] If the text to be processed does not exceed the preset length, a label extraction model is used to extract labels from the text to be processed to obtain extracted labels.

[0143] In this embodiment, the processes performed by each unit in the device are the same as those described above. Figure 1 The method flow described in the corresponding embodiment is similar and will not be repeated here.

[0144] Figure 4 It is a structural diagram of a label verification device based on machine learning provided in an embodiment of the present application. The label verification device based on machine learning 400 may include one or more central processing units (CPU) 401 and a memory 405, and the memory 405 stores one or more applications or data.

[0145] In this embodiment, the specific functional module division in the central processing unit 401 can be the same as the above Figure 3 The functional module division method of each unit described in is similar and will not be repeated here.

[0146] Memory 405 may be volatile or persistent storage. The program stored in memory 405 may include one or more modules, each of which may include a series of instructions for operating on the server. Furthermore, CPU 401 may be configured to communicate with memory 405 and execute the series of instructions in memory 405 on server 400.

[0147] The machine learning-based label verification device 400 may also include one or more power supplies 402, one or more wired or wireless network interfaces 403, one or more input and output interfaces 404, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0148] The CPU 401 can execute the aforementioned Figure 1 The operations performed in the illustrated embodiment will not be described in detail here.

[0149] An embodiment of the present application also provides a computer storage medium for storing computer software instructions used for the above-mentioned machine learning-based label verification method, which includes a program designed for execution.

[0150] The label verification method based on machine learning can be as described above Figure 1 、 Figure 2 The machine learning-based label verification method described in .

[0151] The present application also provides a computer program product, which includes computer software instructions that can be loaded by a processor to implement the above Figure 1 Figure 2 The process of any one of the machine learning-based label verification methods.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the equivalent transformation of circuits and the division of units are only a kind of logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0154] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A label verification method based on machine learning, characterized in that: include: Acquire communication audio, wherein the communication audio is the audio of the communication between the human customer service and the potential customer; Obtaining a task tag set by the customer service system for the potential customer; Converting the communication audio into text using automatic speech recognition technology; Performing error correction processing on the text using a text error correction model based on deep learning to obtain a text to be processed; Using a label extraction model to extract labels from the text to be processed to obtain extracted labels; comparing the consistency between the task label and the extracted label; If the consistency exceeds a preset value, the task tag is deemed to meet the verification standard; If the consistency does not exceed the preset value, it is determined that the task tag does not meet the verification standard.

2. The label verification method based on machine learning according to claim 1, characterized in that: The text error correction model based on deep learning includes: a pinyin similarity rule algorithm.

3. The label verification method based on machine learning according to claim 1, characterized in that: The text error correction model based on deep learning includes: a keyword replacement algorithm.

4. The label verification method based on machine learning according to claim 1, characterized in that: The text error correction model based on deep learning includes: a text similarity replacement algorithm.

5. The label verification method based on machine learning according to claim 1, characterized in that: The label extraction model is used to extract labels from the text to be processed, and obtaining the extracted labels includes: Processing the text to be processed using a basic language model to obtain multiple specific intent dialogue groups; The multiple specific purpose models are used to process the multiple specific intent dialogue groups respectively to obtain extracted labels, and there is a corresponding relationship between the specific purpose models and the specific intent dialogue groups.

6. The label verification method based on machine learning according to claim 1, characterized in that: The label extraction model is used to extract labels from the text to be processed, and obtaining the extracted labels includes: Using a first label extraction model to extract labels from the text to be processed to obtain a first extracted label; Using a second label extraction model to extract labels from the text to be processed to obtain second extracted labels; Cross-validation is performed on the first extracted label and the second extracted label to obtain the extracted label.

7. The label verification method based on machine learning according to claim 1, characterized in that: The step of using a label extraction model to extract labels from the text to be processed to obtain extracted labels includes: Determining whether the text to be processed exceeds a preset length; If the text to be processed exceeds a preset length, the text to be processed is split, and the splitting results are processed using the label extraction model to obtain the extracted labels; If the text to be processed does not exceed the preset length, a label extraction model is used to extract labels from the text to be processed to obtain extracted labels.

8. A label verification device based on machine learning, characterized in that: include: A first acquisition unit is configured to acquire communication audio, wherein the communication audio is audio of communication between a human customer service representative and a potential customer; A second acquiring unit is configured to acquire a task tag set by the customer service system for the potential customer; A conversion unit, configured to convert the communication audio into text using automatic speech recognition technology; An error correction unit, configured to perform error correction processing on the text using a text error correction model based on deep learning to obtain a text to be processed; An extraction unit, configured to extract labels from the text to be processed using a label extraction model to obtain extracted labels; a comparing unit, configured to compare the consistency between the task label and the extracted label; a compliance unit, configured to determine that the task tag meets a verification standard if the consistency exceeds a preset value; The non-compliance unit is configured to determine that the task tag does not comply with a verification standard if the consistency does not exceed a preset value.

9. A label verification device based on machine learning, characterized in that: include: CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply; The memory is a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory on the device to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The method comprises instructions, which, when executed on a computer, cause the computer to perform the method according to any one of claims 1 to 7.

Images