Voice quality inspection method and device, computer device and storage medium

Abstract

The application provides a voice quality inspection method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining audio data to be inspected of a target user voice call; converting the audio data to be inspected into text information; performing text matching quality inspection on the text information to obtain a first quality inspection result of the audio data to be inspected; if the first quality inspection result is that the text information contains preset text information, performing model quality inspection on the text information to obtain a second quality inspection result of the audio data to be inspected. By using the method, the audio data to be inspected is inspected twice by using text matching quality inspection and model quality inspection, so that the probability of misjudgment is reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a voice quality inspection method, device, computer equipment, and storage medium. Background Technology

[0002] With the development of the internet and e-commerce, the consultation services provided are becoming increasingly convenient. In order to ensure the quality of customer service, it is necessary to conduct quality inspections on customer service call records and dialogue information. Currently, the voice quality inspection server of the server hotline stores the audio files or text information of the communication between customer service and users, and then conducts quality inspections through manual inspection or simple voice quality inspection models.

[0003] However, manual quality inspection is not only inefficient, but the accuracy of the results is also generally low due to human error. Furthermore, existing voice quality inspection systems on the market first transcribe the recordings of customer service conversations into text data, then match the text data according to pre-set keywords and rules, and finally output the quality inspection results of the customer service voice. This process has the following problems: 1. Keyword settings that do not consider semantics lead to a large number of misjudgments; 2. Quality inspection rules have high requirements for the accuracy of transcription; 3. Deleting misjudged keywords or rules results in more missed detections, significantly impacting business operations. Summary of the Invention

[0004] Therefore, it is necessary to provide a voice quality inspection method, device, computer equipment, and storage medium to address the aforementioned technical problems. This method uses text matching quality inspection and model quality inspection to perform secondary quality inspection on the audio data to be inspected, thereby reducing the probability of misjudgment.

[0005] Firstly, this application provides a voice quality inspection method, including:

[0006] Obtain the audio data to be inspected from the target user's voice call;

[0007] Convert the audio data to be inspected into text information;

[0008] The text information is subjected to text matching quality inspection to obtain the first quality inspection result of the audio data to be inspected.

[0009] If the first quality inspection result is that the text information contains preset text information, then the text information is subjected to model quality inspection to obtain the second quality inspection result of the audio data to be inspected.

[0010] In some embodiments of this application, after performing model quality inspection on the text information to obtain a second quality inspection result for the audio data to be inspected, if the first quality inspection result indicates that the text information contains preset text information, the process includes:

[0011] If the second quality inspection result indicates that the text information contains preset text information, then the text information is sent to the data source.

[0012] When a request for re-inspection of the second quality inspection result is received from the data source, the text information is re-inspected to obtain the re-inspection result.

[0013] In some embodiments of this application, the step of sending the text information to the data source if the second quality inspection result indicates that the text information contains preset text information includes:

[0014] When a negative re-inspection request for the second quality inspection result is received from the data source, a quality inspection database is created by acquiring several text messages and the second quality inspection result.

[0015] In some embodiments of this application, after receiving a re-inspection request for the second quality inspection result from the data source, the step of re-inspecting the text information to obtain a re-inspection result includes:

[0016] A re-inspection database is created by acquiring several pieces of the aforementioned text information and the re-inspection results;

[0017] The preset initial model is trained based on the data in the quality inspection database and the re-inspection database to obtain a self-learning model, which is used to perform model quality inspection on the text information.

[0018] In some embodiments of this application, the step of training a preset initial model based on data from the quality inspection database and the re-inspection database to obtain a self-learning model includes:

[0019] The data in the quality inspection database and the re-inspection database are processed according to a preset data assembly strategy to obtain a training sample set;

[0020] The self-learning model is obtained by training the preset initial model based on the training sample set.

[0021] In some embodiments of this application, after training a preset initial model based on data from the quality inspection database and the re-inspection database to obtain a self-learning model, the process includes:

[0022] Obtain the status information of the currently used self-learning model, the re-inspection database, and the quality inspection database;

[0023] If the state information conforms to the preset model training strategy, the currently used self-learning model is retrained based on the data in the re-inspection database and the quality inspection database to obtain an updated self-learning model.

[0024] Based on the currently used self-learning model and the updated self-learning model, a next self-learning model is determined, which is used to replace the currently used self-learning model for model quality inspection of the text information.

[0025] In some embodiments of this application, determining the next self-learning model to be used based on the currently used self-learning model and the updated self-learning model includes:

[0026] Calculate the first harmonic mean of the currently used self-learning model and the second harmonic mean of the updated self-learning model;

[0027] The next self-learning model to be used is determined based on the first harmonic mean and the second harmonic mean.

[0028] Secondly, this application provides a voice quality inspection device, comprising:

[0029] The information acquisition module is used to acquire the audio data to be inspected from the voice calls of the target user;

[0030] The text conversion module is communicatively connected to the information acquisition module and is used to convert the audio data to be inspected into text information.

[0031] The matching quality inspection module is communicatively connected to the text conversion module and is used to perform text matching quality inspection on the text information to obtain the first quality inspection result of the audio data to be inspected.

[0032] The model quality inspection module is communicatively connected to the matching quality inspection module. If the first quality inspection result indicates that the text information contains preset text information, then the model quality inspection is performed on the text information to obtain the second quality inspection result of the audio data to be inspected.

[0033] In some embodiments of this application, the model quality inspection module is further configured to send the text information to the data source if the second quality inspection result is that the text information contains preset text information; and when receiving a re-inspection affirmation request for the second quality inspection result sent by the data source, to re-inspect the text information to obtain a re-inspection result.

[0034] In some embodiments of this application, the model quality inspection module is further configured to, when receiving a re-inspection rejection request for the second quality inspection result sent by the data source, acquire several text information and the second quality inspection result to create a quality inspection database.

[0035] In some embodiments of this application, the model quality inspection module is further used to obtain several text information and the re-inspection results to create a re-inspection database; and to train a preset initial model based on the data in the quality inspection database and the re-inspection database to obtain a self-learning model, wherein the self-learning model is used to perform model quality inspection on the text information.

[0036] In some embodiments of this application, the model quality inspection module is further configured to process the data in the quality inspection database and the re-inspection database according to a preset data assembly strategy to obtain a training sample set; and to train a preset initial model according to the training sample set to obtain the self-learning model.

[0037] In some embodiments of this application, the model quality inspection module is further configured to obtain the status information of the currently used self-learning model, the re-inspection database, and the quality inspection database; if the status information conforms to a preset model training strategy, the currently used self-learning model is retrained based on the data in the re-inspection database and the quality inspection database to obtain an updated self-learning model; based on the currently used self-learning model and the updated self-learning model, the next self-learning model to be used is determined, and the next self-learning model to be used is used to replace the currently used self-learning model to perform model quality inspection on the text information.

[0038] In some embodiments of this application, the model quality inspection module is further configured to calculate the first harmonic mean of the currently used self-learning model and the second harmonic mean of the updated self-learning model; and determine the next self-learning model to be used based on the first harmonic mean and the second harmonic mean.

[0039] Thirdly, this application also provides a computer device, the computer device comprising:

[0040] One or more processors;

[0041] The memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the voice quality inspection method described in any of the preceding claims.

[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps in the voice quality inspection method described in any of the preceding claims.

[0043] Fifthly, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in the first aspect described above.

[0044] The aforementioned speech quality inspection method, device, computer equipment, and storage medium first perform text matching quality inspection on the converted text information, and then perform model quality inspection on the text information that was judged to contain preset text information in the first quality inspection. The text matching quality inspection is a hard rule for keyword matching, while the model quality inspection is a quality inspection based on the algorithm model, which can understand and judge certain semantics. The combination of the two reduces the probability of misjudgment. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of a scenario for the voice quality inspection method in an embodiment of this application;

[0047] Figure 2 This is a flowchart illustrating the voice quality inspection method in the embodiments of this application;

[0048] Figure 3 This is a flowchart illustrating a voice quality inspection method in another embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the self-learning model training architecture in the embodiments of this application;

[0050] Figure 5 This is a schematic diagram of the voice quality inspection device in the embodiments of this application;

[0051] Figure 6 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0054] In the description of this application, the word "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0055] In the embodiments of this application, it should be noted that since the voice quality inspection method provided in this application is executed in a computer device, the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It is understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the computer device can process them. The specifics will not be elaborated here.

[0056] In the embodiments of this application, it should also be noted that the voice quality inspection method provided in the embodiments of this application can be applied to, for example, Figure 1The voice quality inspection system shown includes a terminal 100 and a server 200. The terminal 100 can be a device that includes both receiving and transmitting hardware, meaning it has receiving and transmitting hardware capable of performing bidirectional communication over a two-way communication link. Such a device can include cellular or other communication devices with single-line displays, multi-line displays, or no multi-line displays. Specifically, the terminal 100 can be a desktop terminal or a mobile terminal, specifically a mobile phone, tablet computer, laptop computer, or a camera installed at the monitoring site for information collection, storage, and transmission. The server 200 can be a standalone server or a server network or server cluster, including but not limited to computers, network hosts, single network servers, multiple network server sets, or cloud servers composed of multiple servers. The cloud server consists of a large number of computers or network servers based on cloud computing.

[0057] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one server (200) is shown; it is understood that this voice quality inspection system may also include one or more other servers, which are not specified here. Additionally, as... Figure 1 As shown, the voice quality inspection system may also include a memory for storing data, such as storing text information to be inspected.

[0058] It should also be noted that, Figure 1 The schematic diagram of the voice quality inspection system shown is merely an example. The voice quality inspection system and scenario described in this embodiment are intended to more clearly illustrate the technical solutions of this embodiment and do not constitute a limitation on the technical solutions provided by this embodiment. As those skilled in the art will know, with the evolution of voice quality inspection systems and the emergence of new business scenarios, the technical solutions provided by this embodiment are also applicable to similar technical problems.

[0059] See Figure 2 This application provides a voice quality inspection method, which is mainly applied to the above-mentioned... Figure 1 Taking server 200 as an example, the method includes steps S201 to S204, as follows:

[0060] S201, Obtain the audio data to be inspected from the target user's voice call.

[0061] Specifically, the audio data to be inspected refers to audio data that requires quality control, such as whether the target user used inappropriate language during a voice call. This includes, but is not limited to, customer service personnel's voice communication data and the target user's voice data. Any voice data that needs quality control can be used. Furthermore, the audio data to be inspected can also be a segment of the target user's voice call, obtained by trimming the target user's voice call to reduce the amount of data requiring quality control processing. The target user refers to the object whose voice information needs to be evaluated, including but not limited to various customer service personnel and staff.

[0062] The audio data to be inspected can be automatically captured from the network or server at regular intervals and in quantitative quantities by smart devices, or it can be relevant data input by the user. This embodiment does not make specific limitations.

[0063] In this embodiment, the acquisition of audio data to be inspected can be carried out by fetching data at certain time intervals, automatically fetching data when the stored audio data reaches a certain level, or acquiring data according to the user's needs. This embodiment does not impose any specific limitations.

[0064] Specifically, before server 200 performs the voice quality inspection task, the user can send a task request to server 200 through terminal 100. The task request carries the audio data to be inspected. After receiving the task request, server 200 can perform quality inspection analysis based on the audio data to be inspected according to the voice quality inspection method. Alternatively, server 200 does not need to receive the task request sent by terminal 100 before performing the voice quality inspection task. In this case, terminal 100 is a device with recording function. After terminal 100 periodically or in real time collects the audio data to be inspected, it can send it to server 200 to perform the voice quality inspection task. Furthermore, terminal 100 can be equipped with a voice acquisition device. Terminal 100 periodically or in real time collects audio, extracts the audio data to be inspected, and sends it to server 200 so that server 200 can perform the voice quality inspection task.

[0065] S202, the audio data to be inspected is converted into text information.

[0066] Specifically, the audio data to be inspected is audio data. To reduce the workload of quality inspection and lower the error rate, the audio data is transcribed into text information, such as text string data, using methods such as a speech-to-text engine. This embodiment does not specifically limit the method of converting the audio data to be inspected into text information.

[0067] In addition, for data that needs to be quality inspected, such as online comments or online chat conversations between e-commerce customer service representatives, the text information can be directly obtained for quality inspection and analysis without the need for text conversion.

[0068] S203, perform text matching quality inspection on the text information to obtain the first quality inspection result of the audio data to be inspected.

[0069] Specifically, text matching quality inspection is performed on the text information, matching the transcribed text information using pre-defined regular expressions and logical business rules. For example, "I want to hear from you (High-level Manager | Leader | Headquarters | Regional Department | District General Manager | Person in Charge | The One Who Can Make Decisions | The One Who Manages Things | Superior | Director | Manager | Manager | Supervisor)" and "I want to file a complaint against (High-level Manager | Leader | Headquarters | Regional Department | District General Manager | Person in Charge | The One Who Can Make Decisions | The One Who Manages Things | Superior | Director | Manager | Manager | Supervisor)" have the corresponding regular expressions: re_execu1=re.compile('I want to hear from you (High-level Manager | Leader | Headquarters | Regional Department | District General Manager | Person in Charge | The One Who Can Make Decisions | The One Who Manages Things | Superior | Director | Manager | Manager | Supervisor)') and re_execu2=re.compile('I want to file a complaint against (High-level Manager | Leader | Headquarters | Regional Department | District General Manager | Person in Charge | The One Who Can Make Decisions | The One Who Manages Things | Superior | Director | Manager | Manager | Supervisor)'). Logical business rules are the rules that text information cannot violate. For example, mechanical responses are defined as follows: (When a customer service representative mechanically replies to a customer in a communication conversation, repeatedly using the same phrase, and the keyword appears more than 3 times in a single conversation, it violates the logical business rules, and the system will deduct points.) Keywords include: (Cannot be delivered; cannot be arranged; cannot be arranged; cannot be delivered; cannot arrive today; we have tried our best; cannot be arranged; cannot be satisfied; etc.).

[0070] The system sets keywords for matching and determines whether the audio data contains preset text information, thus obtaining the first quality inspection result. This first result may or may not contain the preset text information.

[0071] The preset text information can include sensitive words, complaint statements, personal attacks, etc., and different settings can be made based on the needs of quality inspection. In addition, multiple violation categories can be set for the preset text information, and keywords or logical business rules can be set under each violation category. The keywords and logical business rules written for different violation categories can be the same or different.

[0072] Furthermore, the first quality inspection result should include at least several key fields, such as [text, violation category, whether a violation occurred]. Here, text refers to the text information to be inspected; violation category mainly refers to business items that have been defined as violating regulations (e.g., endangering personal safety, high-risk complaints, dissatisfaction, grievances, etc.; this field is empty for text information without warning); and whether a violation occurred is mainly an enumerable data value (e.g., 1 - violation, 0 - no violation). In addition, the first quality inspection result can also extract and include the specific violation portions of the text information.

[0073] S204, if the first quality inspection result is that the text information contains preset text information, then perform model quality inspection on the text information to obtain the second quality inspection result of the audio data to be inspected.

[0074] Specifically, if the first quality inspection result indicates that the text information contains preset text information, it suggests that the audio data to be inspected contains illegal content. To reduce false positives, a second quality inspection result is obtained by further performing model quality inspection on the text information, for example, by using a self-learning model to perform model quality inspection on the text information. The rules for model quality inspection and text matching quality inspection differ. Model quality inspection is based on an algorithm model and can understand and judge certain semantics; while text matching quality inspection matches keywords, which are hard rules and prone to false positives. Combining the two quality inspections can reduce the probability of false positives in the final quality inspection result. Furthermore, using different methods for the two quality inspections allows them to complement each other based on their respective characteristics, improving the accuracy of the quality inspection. In addition, since model quality inspection is based on an algorithm model and is relatively complex, performing a preliminary screening through text matching quality inspection helps to improve the speed of the quality inspection process.

[0075] Similarly, the second quality inspection result may or may not include preset text information. The second quality inspection result must contain at least several key fields, such as [text, matched violation category, whether a violation occurred]. Here, text refers to the text information to be inspected; violation category mainly refers to business items that have been defined as violating regulations (e.g., endangering personal safety, high-risk complaints, dissatisfaction, grievances, etc.; this field is empty for text information without warning); and whether a violation occurred is mainly an enumerable data value (e.g., 1 - violation, 0 - no violation). The second quality inspection result can also extract and include the specific violation portion of the text information. Furthermore, the first and second quality inspection results can also have fields set to represent the inspection method, such as: 0 - text matching inspection, 1 - model inspection.

[0076] Furthermore, if the first quality inspection result indicates that the text information does not contain the preset text information, it means that the audio data to be inspected does not contain any illegal content. In this case, the audio data to be inspected, the text information, and the corresponding first quality inspection result are stored in the quality inspection database. In the quality inspection database, data can be categorized based on different fields in the first quality inspection result for easy searching and statistics. Specifically, for audio data to be inspected where the first quality inspection result indicates that the text information does not contain the preset text information, random checks can be conducted according to certain rules, such as every certain random check period or when the corresponding category of data in the quality inspection database accumulates to a preset quantity. This allows for timely detection of potential quality inspection problems and reduces the probability of misjudgments.

[0077] In one embodiment, this step is followed by: S301, if the second quality inspection result is that the text information contains preset text information, then the text information is sent to the data source; S302, when a re-inspection affirmation request for the second quality inspection result is received from the data source, the text information is re-inspected to obtain a re-inspection result.

[0078] Specifically, if the second quality inspection result obtained through the model still determines that the text information contains preset text information, it indicates that the secondary search still analyzes the audio data to be inspected and finds non-compliant content. Therefore, the text information is sent to the data source.

[0079] The data source is the acquisition end of the audio data to be inspected, which is associated with the target user. The target user receives the second inspection result through the data source and determines whether to accept it. If accepted, the user sends a request for a negative re-inspection through the data source or other smart devices. If rejected, and the user believes the second inspection result is a misjudgment, the user sends a request for a positive re-inspection through the data source or other smart devices. The positive re-inspection request includes, but is not limited to, the reasons for the re-inspection.

[0080] When a request for confirmation of the second quality inspection result is received from the data source, the text information is re-inspected to obtain a re-inspection result. This re-inspection can be performed using a neural network model or manually. Specifically, upon receiving the request for confirmation of the second quality inspection result from the data source, the audio data to be inspected, the first quality inspection result, the second quality inspection result, and the confirmation request are all sent to the re-inspection client. Then, the re-inspection client receives the re-inspection result in response to the confirmation request. Finally, regardless of the re-inspection result, it can also be sent back to the data source so that the target user can be informed.

[0081] Similarly, the re-inspection result may or may not include preset text information. The re-inspection result should contain at least several key fields, such as [text, matched violation category, whether a violation occurred]. Here, text refers to the text information to be inspected; violation category mainly refers to business items with many violations defined according to business needs (e.g., endangering personal safety, high-risk complaints, dissatisfaction, grievances, etc.; this field is empty for text information without warning); and whether a violation occurred is mainly an enumerable data value (e.g., 1 - violation, 0 - no violation). The re-inspection result can also extract and include the specific violation portion of the text information. Furthermore, the re-inspection result can also set fields to represent the inspection method, such as: 0 - text matching inspection, 1 - model inspection, 2 - re-inspection.

[0082] It should be noted that if the second quality inspection result indicates that the text information does not contain the preset text information, it means that the audio data to be inspected does not contain any illegal content. In other words, the first quality inspection result of the text matching quality inspection was a misjudgment. In this case, the audio data to be inspected, the text information, and the corresponding first and second quality inspection results are stored in the quality inspection database. In the quality inspection database, data can be categorized based on different fields in the first and second quality inspection results for easy searching and statistics. Specifically, for audio data to be inspected where the second quality inspection result indicates that the text information does not contain the preset text information, random checks can be performed according to certain rules, such as every certain random check period or when the corresponding category of data in the quality inspection database accumulates to a preset quantity. This allows for timely detection of potential quality inspection problems and reduces the probability of misjudgments.

[0083] In one embodiment, step S301, if the second quality inspection result is that the text information contains preset text information, then after sending the text information to the data source, includes: S401, when receiving a re-inspection rejection request for the second quality inspection result sent by the data source, acquiring several pieces of the text information and the second quality inspection result to create a quality inspection database.

[0084] Specifically, if the target user accepts the second quality inspection result, that is, acknowledges that there is illegal content in the audio data to be inspected and determines that a re-inspection is not necessary, then a re-inspection rejection request will be sent back through the data source or other smart devices. When a re-inspection rejection request for the second quality inspection result is received from the data source, several similar feedback messages are obtained, including the text information of the re-inspection rejection request and the quality inspection database created from the second quality inspection result.

[0085] Similarly, the quality inspection database can be categorized based on different fields in the first and second quality inspection results, facilitating searching and statistics. For audio data awaiting quality inspection, random checks can be conducted according to certain rules, such as every specific inspection cycle or when a preset quantity of data of the corresponding category accumulates in the quality inspection database. This allows for the timely detection of potential quality inspection problems and reduces the likelihood of misjudgments.

[0086] In one embodiment, step S302, when receiving a request for re-inspection of the second quality inspection result from the data source, re-inspecting the text information to obtain a re-inspection result includes: S501, acquiring several pieces of the text information and the re-inspection result to create a re-inspection database; S502, training a preset initial model based on the data in the quality inspection database and the re-inspection database to obtain a self-learning model, wherein the self-learning model is used to perform model quality inspection on the text information.

[0087] Specifically, after the text information is re-checked, a re-check database is created by acquiring several re-checked text information entries and their corresponding re-check results. The re-check results may or may not contain preset text information. It should be noted that the re-check database stores data that has undergone re-checking, while the quality inspection database stores data that has not undergone re-checking. The quality inspection database contains text information whose first quality inspection result is that it does not contain preset text information, and text information corresponding to the second quality inspection result where the data source sent a re-check rejection request. The re-check database and the quality inspection database can also be merged and then categorized; this embodiment does not impose specific limitations on this.

[0088] In addition, data in the re-inspection database and the quality inspection database can be randomly checked according to certain rules, such as every certain sampling period or when the corresponding category of data in the quality inspection database accumulates to a preset quantity, so as to promptly discover potential problems in quality inspection and reduce the chance of misjudgment in quality inspection.

[0089] A self-learning model is obtained by training a preset initial model using data from the re-examination database and the quality inspection database. The preset initial model can be architecturally configured based on quality inspection requirements, such as being set as a binary classification training model; this embodiment does not impose specific limitations. Furthermore, the self-learning model is used for model quality inspection of text information. However, since the self-learning model can be subsequently trained and updated, the model used for quality inspection of text information may be an updated version of the self-learning model.

[0090] In one embodiment, step S502, training a preset initial model based on data from the quality inspection database and the re-inspection database to obtain a self-learning model, includes: S601, processing data from the quality inspection database and the re-inspection database according to a preset data assembly strategy to obtain a training sample set; S602, training the preset initial model based on the training sample set to obtain the self-learning model.

[0091] Specifically, the data assembly strategy is the allocation strategy for the training sample set. The training sample set includes a training sample set and a test sample set. The data assembly strategy can be set as a balanced allocation strategy, so that the number of positive and negative samples in the training sample set is relatively balanced. For example, if the quality inspection database and the re-inspection database contain 100 positive samples (determined to be without violations) and 900 negative samples (determined to be violations), then the total number of training samples is min(positive samples, negative samples) * 200 = 200. For negative samples, 100 samples are randomly sampled from the 900 samples to serve as negative samples in the training sample set. The ratio of the training sample set to the test sample set can be set based on the training requirements of the self-learning model. For example, the ratio of the training sample set to the test sample set is 8:2. This embodiment does not impose a specific limitation. The main purpose of the training sample set is to train the model; the main purpose of the test sample set is to calculate the evaluation indicators of the model and output the performance of the self-learning model on the test sample set.

[0092] It should be noted that the data in the re-inspection database and the data corresponding to the re-inspection rejection requests obtained from the quality inspection database have undergone multiple quality inspections to ensure their accuracy, resulting in a low probability of misjudgment. However, the data in the quality inspection database whose first quality inspection result does not contain the preset relationship has only undergone text matching quality inspection, which may lead to misjudgment. Therefore, a training sample set can be selected from the data in the re-inspection database and the data corresponding to the re-inspection rejection requests obtained from the quality inspection database based on the data assembly strategy to ensure the accuracy of the training data and thereby improve the accuracy of the self-learning model.

[0093] In one embodiment, after training a preset initial model based on the data in the quality inspection database and the re-inspection database to obtain a self-learning model, step S502 includes: S701, obtaining the status information of the currently used self-learning model, the re-inspection database, and the quality inspection database; S702, if the status information conforms to a preset model training strategy, then training the currently used self-learning model again based on the data in the re-inspection database and the quality inspection database to obtain an updated self-learning model; S703, determining the next self-learning model to be used based on the currently used self-learning model and the updated self-learning model, wherein the next self-learning model is used to replace the currently used self-learning model for model quality inspection of the text information.

[0094] Specifically, since the self-learning model used for model quality inspection of text information may adjust and update its parameters while maintaining the basic architecture, it is not necessarily the same self-learning model all the time. Therefore, it is necessary to obtain the status information of the currently used self-learning model and the re-inspection database. The status information includes, but is not limited to, the last learning time of the currently used self-learning model, the data storage time and storage capacity of the re-inspection database and the quality inspection database.

[0095] The model training strategies include scheduled training, data accumulation training, and custom training. Scheduled training involves pulling data at regular intervals and training the self-learning model according to a data assembly strategy. Data accumulation training automatically trains the self-learning model based on a certain volume of data from the review database and / or quality inspection database. Custom training allows users to train the self-learning model according to their needs. Pre-set model training strategies can be selected based on actual requirements; this embodiment does not impose specific limitations.

[0096] If the state information conforms to the preset model training strategy, the currently used self-learning model will be trained and updated again based on the data in the re-examination database and the quality inspection database to obtain the updated self-learning model.

[0097] It should be noted that as the quality inspection time increases, the amount of data in the re-inspection database and the quality inspection database also increases. In order to use more effective data to retrain and update the currently used self-learning model, we can determine the new data in the re-inspection database and the quality inspection database based on the last learning time of the currently used self-learning model. Then, we can use the new data to retrain and update the currently used self-learning model to obtain the updated self-learning model. Using the new data to update the self-learning model helps to correct the deficiencies of the parameters of the currently used self-learning model.

[0098] The updated self-learning model might actually decrease in accuracy due to factors such as data bias. Therefore, by comparing the analysis results of the currently used self-learning model and the updated self-learning model on a test sample set, the next self-learning model to be used is determined. This next self-learning model is then used to replace the current one for quality control of the text information. In other words, after determining the next self-learning model, subsequent quality control will use the next self-learning model until a new next self-learning model is determined. It should be noted that the next self-learning model could be either the currently used model or an updated one.

[0099] In one embodiment, step S703, determining the next self-learning model to be used based on the currently used self-learning model and the updated self-learning model, includes: S801, calculating the first harmonic mean of the currently used self-learning model and the second harmonic mean of the updated self-learning model; S802, determining the next self-learning model to be used based on the first harmonic mean and the second harmonic mean.

[0100] The self-learning model is essentially a binary classifier, and the main evaluation metric is F1 (the harmonic mean of precision and recall). Therefore, the first harmonic mean of the currently used self-learning model and the second harmonic mean of the updated self-learning model are calculated. Then, the accuracy between the currently used self-learning model and the updated self-learning model is compared based on the first and second harmonic means, and the next self-learning model to be used is determined.

[0101] In this embodiment, the model's secondary quality inspection reduces false positives and the time required for manual review, saving labor costs. The data accumulated from the re-inspection database and the quality inspection database can form a closed loop, allowing for more effective use of this data to train the model, improve work efficiency, and reduce the false positive rate of quality inspection rules.

[0102] See Figure 3 This application provides a voice quality inspection method, including the following steps:

[0103] Step 1: Collect audio data from the target user's voice calls to be inspected, such as customer service call audio files. The speech-to-text engine then converts the audio data into text information.

[0104] Step 2: Perform text matching quality inspection on the transcribed text information according to the string of the quality inspection rules. If the match is successful and the business rule is hit (including the pre-set text information violation trigger warning), proceed to Step 3. If there is no warning, send the first quality inspection result to the business system and save the corresponding data (including but not limited to the audio data to be inspected, the corresponding text information, and the first quality inspection result, etc.) to the quality inspection database (not shown in the figure).

[0105] Step 3: The warning text information is then subjected to a second quality check using the self-learning model, and the results of this second quality check are sent to the business system.

[0106] Step 4: Distribution of quality inspection results by the business system;

[0107] ① The business system sends the data containing preset text information to the data source of the corresponding target user, such as customer service staff, for the second quality inspection result of text matching and self-learning model.

[0108] ② Customer service staff provide feedback on the second quality inspection result through the data source end, indicating whether a re-inspection is needed. If a re-inspection is needed, a re-inspection affirmative request is sent through the data source end. The business system will then send the audio data to be inspected, the first quality inspection result, the second quality inspection result, and the re-inspection affirmative request to the review staff's re-inspection client. If no re-inspection is needed, a re-inspection negative request is sent through the data source end, and the corresponding data (including but not limited to the audio data to be inspected, the corresponding text information, the first quality inspection result, the second quality inspection result, and the re-inspection negative request, etc.) is saved to the quality inspection database.

[0109] ③ The review staff manually checks the data to see if there are any violations. The review results are sent to the business system through the review client and the review data (including but not limited to the audio data to be inspected, the corresponding text information, the first quality inspection result, the second quality inspection result, the review confirmation request, and the review result, etc.) are saved to the review database. At the same time, the review results can be sent to the data source end through the review client and the first-level business system.

[0110] ④ The business system will send the review results back to customer service staff;

[0111] The above four steps complete the business system quality inspection logic.

[0112] Step 5: Data storage after quality inspection. Data that is hit by quality inspection (both text matching quality inspection and model quality inspection include preset text information) is divided into two parts. One part is the data that needs to be reviewed and is saved to the review database. The other part is the data that is hit by the rule but receives a negative review request and does not need to be manually reviewed and is saved to the quality inspection database to prepare for the subsequent model self-training.

[0113] Step 6: Model training. Data in the verification database is labeled 0 or 1 (based on the annotation results of the verification staff; text messages that trigger warnings are labeled 1, and those that do not trigger warnings are labeled 0). Data in the quality inspection database that matches the rule but receives a negative verification request and does not require manual verification is labeled 1. The model is trained using this labeled data for binary classification. After the self-learning model is deployed, iterative updates can be used in the next model quality inspection.

[0114] Self-learning model training architecture such as Figure 4As shown, a training sample set is selected from the quality inspection database and the re-inspection database using a preset data assembly strategy. The training sample set includes a training sample set and a test sample set. The main purpose of the training sample set is to train the model; the test sample set is mainly used to calculate the model's evaluation metrics and output the self-learning model's performance on the test sample set. The currently used self-learning model is retrained according to the preset model training strategy and the training sample set to obtain an updated self-learning model. The self-learning model is a binary classification model, such as the BERT model. Based on the currently used self-learning model and the updated self-learning model, the next self-learning model to be used is determined. This next self-learning model replaces the currently used self-learning model for model quality inspection of text information. Text information is obtained through the model interface of the next self-learning model for model quality inspection, resulting in the second quality inspection result of the audio data to be inspected.

[0115] Model training strategies include scheduled training, data accumulation training, and custom training. Scheduled training involves pulling data at regular intervals and training the self-learning model according to a data assembly strategy. Data accumulation training automatically trains the self-learning model based on a certain volume of data from the review and / or quality inspection databases. Custom training allows users to train the self-learning model according to their specific needs.

[0116] The data assembly strategy is the allocation strategy for the training sample set. This strategy can be set as a balanced allocation strategy, ensuring a relatively even distribution of positive and negative samples in the training sample set. For example, if the quality inspection database and the re-inspection database contain 100 positive samples (determined no violation) and 900 negative samples (determined violation), then the total number of training samples is min(positive samples, negative samples) * 200 = 200. For negative samples, 100 samples are randomly selected from the 900 samples to form the negative samples in the training sample set. The ratio of the training sample set to the test sample set can be set based on the training requirements of the self-learning model; for example, the ratio of the training sample set to the test sample set is 8:2. This embodiment does not impose a specific limitation.

[0117] Since the quality inspection rules are based on string matching using regular expressions, they are poor at understanding and recognizing the semantics of text. Therefore, the model performs a second quality inspection on the data that triggers the quality inspection rules. The self-learning model can understand the semantics of the text based on the reviewed sample data and natural language processing, which can reduce errors caused by rule misjudgment.

[0118] By employing deep learning and supervised self-learning methods, and using data from feedback from reviewers as labeled sample data, a binary classification model is used for classification learning to reduce misjudgments, improve the accuracy of voice quality inspection, and save labor costs. Furthermore, a closed loop is formed from the perspectives of voice quality inspection, model self-learning, and effective data utilization.

[0119] To better implement the speech quality inspection method in the embodiments of this application, based on the speech quality inspection method, the embodiments of this application also provide a speech quality inspection device, such as... Figure 5 As shown, the voice quality inspection device 900 includes:

[0120] The information acquisition module 910 is used to acquire the audio data to be inspected from the voice call of the target user;

[0121] The text conversion module 920 is communicatively connected to the information acquisition module 910 and is used to convert the audio data to be inspected into text information.

[0122] The matching quality inspection module 930 is communicatively connected to the text conversion module 920 and is used to perform text matching quality inspection on the text information to obtain the first quality inspection result of the audio data to be inspected.

[0123] The model quality inspection module 940 is communicatively connected to the matching quality inspection module 930. If the first quality inspection result is that the text information contains preset text information, then the model quality inspection is performed on the text information to obtain the second quality inspection result of the audio data to be inspected.

[0124] In one embodiment, the model quality inspection module 940 is further configured to send the text information to the data source if the second quality inspection result is that the text information contains preset text information; and when it receives a re-inspection affirmation request for the second quality inspection result sent by the data source, it re-inspects the text information to obtain a re-inspection result.

[0125] In one embodiment, the model quality inspection module 940 is further configured to, when receiving a re-inspection rejection request for the second quality inspection result sent by the data source, acquire several text information and the second quality inspection result to create a quality inspection database.

[0126] In one embodiment, the model quality inspection module 940 is further configured to acquire several pieces of the text information and the re-inspection results to create a re-inspection database; and to train a preset initial model based on the data in the quality inspection database and the re-inspection database to obtain a self-learning model, wherein the self-learning model is used to perform model quality inspection on the text information.

[0127] In one embodiment, the model quality inspection module 940 is further configured to process the data in the quality inspection database and the re-inspection database according to a preset data assembly strategy to obtain a training sample set; and to train a preset initial model according to the training sample set to obtain the self-learning model.

[0128] In one embodiment, the model quality inspection module 940 is further configured to acquire the status information of the currently used self-learning model, the re-inspection database, and the quality inspection database; if the status information conforms to a preset model training strategy, the currently used self-learning model is retrained based on the data in the re-inspection database and the quality inspection database to obtain an updated self-learning model; based on the currently used self-learning model and the updated self-learning model, the next self-learning model to be used is determined, and the next self-learning model to be used is used to replace the currently used self-learning model to perform model quality inspection on the text information.

[0129] In one embodiment, the model quality inspection module 940 is further configured to calculate the first harmonic mean of the currently used self-learning model and the second harmonic mean of the updated self-learning model; and determine the next self-learning model to be used based on the first harmonic mean and the second harmonic mean.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0131] In some embodiments of this application, the voice quality inspection device 900 can be implemented as a computer program, which can be implemented in, for example... Figure 6 The computer device shown operates on this device. The computer device's memory can store the various program modules that make up the voice quality inspection device 900, for example, Figure 5 The information acquisition module 910, text conversion module 920, matching quality inspection module 930, and model quality inspection module 940 are shown. The computer program comprised of these modules causes the processor to execute the steps in the speech quality inspection methods of the various embodiments of this application described in this specification.

[0132] For example, Figure 6 The computer device shown can be used as follows Figure 5The information acquisition module 910 of the voice quality inspection device 900 shown executes step S201. The computer device can execute step S202 via the text conversion module 920. The computer device can execute step S203 via the matching quality inspection module 930. The computer device can execute step S204 via the model quality inspection module 940. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with external computer devices via a network connection. When the computer program is executed by the processor, it implements a voice quality inspection method.

[0133] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0134] In some embodiments of this application, a computer device is provided, including one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processors using the steps of the above-described voice quality inspection method. The steps of the voice quality inspection method here may be steps from the voice quality inspection methods of the various embodiments described above.

[0135] In some embodiments of this application, a computer-readable storage medium is provided, storing a computer program. The computer program is loaded by a processor, causing the processor to execute the steps of the aforementioned speech quality inspection method. The steps of this speech quality inspection method may be those found in the speech quality inspection methods of the various embodiments described above.

[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The above provides a detailed description of a voice quality inspection method, apparatus, computer device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A voice quality inspection method, characterized by, The method comprises the following steps: obtaining audio data to be inspected of a target user voice call; converting the audio data to be inspected into text information; performing text matching inspection on the text information to obtain a first inspection result of the audio data to be inspected; if the first inspection result is that the text information does not contain preset text information, performing spot check on the first inspection result based on a spot check period, or performing spot check on the first inspection result when data of a corresponding category in a quality inspection database accumulates to a preset quantity; if the first inspection result is that the text information contains the preset text information, performing model inspection on the text information to obtain a second inspection result of the audio data to be inspected; the model inspection is used for understanding and judging semantics; the quality inspection database contains text information with the first inspection result of not containing preset text information, and text information corresponding to a second inspection result of a data source end sending a re-inspection negative request.

2. The voice quality monitoring method of claim 1, wherein, After the step of if the first inspection result is that the text information contains preset text information, performing model inspection on the text information to obtain a second inspection result of the audio data to be inspected, the method further comprises the following steps: if the second inspection result is that the text information contains preset text information, sending the text information to the data source end; when receiving a re-inspection positive request for the second inspection result sent by the data source end, performing re-inspection on the text information to obtain a re-inspection result.

3. The voice quality monitoring method of claim 2, wherein, After the step of if the second inspection result is that the text information contains preset text information, sending the text information to the data source end, the method further comprises the following steps: when receiving a re-inspection negative request for the second inspection result sent by the data source end, obtaining a plurality of text information and the second inspection result to create a re-inspection database.

4. The voice quality monitoring method of claim 3, wherein, After the step of when receiving a re-inspection request for the second inspection result sent by the data source end, performing re-inspection on the text information to obtain a re-inspection result, the method further comprises the following steps: obtaining a plurality of text information and the re-inspection result to create a re-inspection database; training a preset initial model according to data in the quality inspection database and the re-inspection database to obtain a self-learning model, the self-learning model being used for model inspection on the text information.

5. The voice quality monitoring method of claim 4, wherein, The step of training a preset initial model according to data in the quality inspection database and the re-inspection database to obtain a self-learning model comprises the following steps: processing data in the quality inspection database and the re-inspection database according to a preset data assembly strategy to obtain a training sample set; training a preset initial model according to the training sample set to obtain the self-learning model.

6. The voice quality monitoring method of claim 5, wherein, After the step of training a preset initial model according to data in the quality inspection database and the re-inspection database to obtain a self-learning model, the method further comprises the following steps: obtaining state information of a currently used self-learning model, the re-inspection database and the quality inspection database; if the state information meets a preset model training strategy, re-training the currently used self-learning model according to data in the re-inspection database and the quality inspection database to obtain an updated self-learning model; According to the current self-learning model in use and the updated self-learning model, a next self-learning model in use is determined, the next self-learning model in use being used to replace the current self-learning model in use to perform model quality inspection on the text information.

7. The voice quality monitoring method of claim 6, wherein, According to the current self-learning model in use and the updated self-learning model, a next self-learning model in use is determined, including: calculating a first harmonic mean of the current self-learning model in use and a second harmonic mean of the updated self-learning model; determining the next self-learning model in use according to the first harmonic mean and the second harmonic mean.

8. A voice quality monitoring device, characterized by, including: an information acquisition module, configured to acquire to-be-inspected audio data of a target user voice call; a text conversion module, in communication connection with the information acquisition module, configured to convert the to-be-inspected audio data into text information; a matching inspection module, in communication connection with the text conversion module, configured to perform text matching inspection on the text information to obtain a first inspection result of the to-be-inspected audio data; if the first inspection result is that the text information does not contain preset text information, the first inspection result is subjected to spot check based on a spot check period, or the first inspection result is subjected to spot check when data of a corresponding category in a quality inspection database accumulates to a preset amount; a model inspection module, in communication connection with the matching inspection module, configured to, if the first inspection result is that the text information contains the preset text information, perform model inspection on the text information to obtain a second inspection result of the to-be-inspected audio data; the model inspection is used to understand and judge semantics; the quality inspection database contains text information with the first inspection result being that the text information does not contain preset text information, and text information corresponding to a second inspection result of a data source end sending a re-inspection negative request.

9. A computer device, comprising: The computer device includes: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the voice quality inspection method in any one of claims 1 to 7.

10. A computer readable storage medium having stored thereon a computer program, the computer program being loaded by a processor to perform steps in the voice quality inspection method in any one of claims 1 to 7.

Images