Voice data processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a speech data processing method and device, electronic equipment and storage medium, the method comprising: obtaining recognition information corresponding to the speech data; wherein the recognition information is obtained by a speech recognition model, and the recognition information contains timestamp information; determining the target text contained in the recognition information and the starting timestamp and the ending timestamp corresponding to the target text in the recognition information; according to the starting timestamp and the ending timestamp, the speech data is cut and processed to obtain a speech segment; the speech segment is desensitized, and the desensitized speech segment is used to adjust the speech recognition model. The speech data processing method provided by the present disclosure reduces the data volume of the speech data, thereby reducing the time consumption of desensitization and the subsequent data transmission data volume, and greatly improving the subsequent processing efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, apparatus, electronic device and storage medium for processing voice data. Background Technology

[0002] Voice data is involved in various business scenarios, such as telephone calls and online meetings, generating large amounts of voice data. However, due to the large volume and long playback time of voice data, it is not conducive to people quickly obtaining useful information. Therefore, speech-to-text technology has emerged. Through speech recognition models, voice data can be automatically converted into text data, making it easier to store and analyze.

[0003] However, speech recognition models inevitably produce various recognition errors during the speech recognition process. In order for the speech recognition model to learn and correct based on these errors, it is necessary to provide the speech recognition model with erroneous speech data to enable iterative updates. However, since the amount of erroneous speech data is usually large, there are many problems in providing this data to the speech recognition model, such as inconvenient data transmission and time-consuming processing. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and storage medium for processing voice data, which solves the problems of inconvenient voice data transmission and time-consuming processing.

[0005] In a first aspect, this disclosure provides a method for processing voice data, including:

[0006] Acquire the speech data to be processed, and the recognition information corresponding to the speech data; wherein the recognition information is obtained according to the speech recognition model, and the recognition information includes the text information corresponding to the speech data and the timestamp information corresponding to the text information;

[0007] The target text in the text information is determined, along with the start and end timestamps corresponding to the target text, wherein the target text is the text in the text information that was incorrectly identified.

[0008] The speech data is processed based on the start timestamp and the end timestamp to obtain the speech segment corresponding to the target text;

[0009] The speech segment is desensitized, and the desensitized speech segment is used to adjust the speech recognition model.

[0010] Secondly, this disclosure provides a voice data processing apparatus, including:

[0011] The acquisition module is adapted to acquire the voice data to be processed and the recognition information corresponding to the voice data; wherein the recognition information is obtained according to the voice recognition model, and the recognition information includes the text information corresponding to the voice data and the timestamp information corresponding to the text information;

[0012] The determining module is adapted to determine the target text in the text information, as well as the start timestamp and end timestamp corresponding to the target text, wherein the target text is the text in the text information that was incorrectly identified;

[0013] The processing module is adapted to process the speech data according to the start timestamp and the end timestamp to obtain the speech segment corresponding to the target text;

[0014] The desensitization module is adapted to desensitize the speech segment, and the desensitized speech segment is used to adjust the speech recognition model.

[0015] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the above-described method.

[0016] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-described method when executed by a processor / processor core.

[0017] The embodiments provided in this disclosure, after obtaining recognition information corresponding to speech data, can identify target text (i.e., misidentified text) contained in the recognition information, thereby determining the start and end timestamps corresponding to the target text. Then, the speech data is processed based on the start and end timestamps to obtain speech segments. After desensitization processing of the processed speech segments, they can be used to adjust the speech recognition model. Because this embodiment can identify target text contained in the recognition information and process speech data into speech segments based on timestamp information, the amount of speech data is significantly reduced. Therefore, the time spent on desensitization processing and the amount of subsequent data transmission are reduced, greatly improving subsequent processing efficiency.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 A flowchart of a voice data processing method provided as an embodiment of this disclosure;

[0021] Figure 2 A flowchart of a method for processing voice data provided in another embodiment of this disclosure;

[0022] Figure 3 A flowchart of a method for processing voice data, provided as a specific example of this disclosure;

[0023] Figure 4 A block diagram of a voice data processing apparatus provided in an embodiment of this disclosure;

[0024] Figure 5 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0027] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0029] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0030] The voice data processing method according to embodiments of this disclosure can be executed by electronic devices such as terminal devices or servers. Terminal devices can be in-vehicle devices, user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Specifically, the method can be implemented by a processor calling a computer program stored in memory.

[0031] Figure 1 A flowchart illustrating a method for processing voice data according to an embodiment of this disclosure. (Refer to...) Figure 1 The method includes:

[0032] Step S110: Obtain the speech data to be processed and the recognition information corresponding to the speech data; wherein, the recognition information is obtained according to the speech recognition model, and the recognition information includes the text information corresponding to the speech data and the timestamp information corresponding to the text information.

[0033] The voice data includes call recordings, meeting recordings, and course audio data, etc., and this disclosure does not limit the specific types of voice data. The recognition information corresponding to the voice data is obtained through a voice recognition model. This voice recognition model is used to perform speech-to-text conversion on the voice data, thereby obtaining the recognition information corresponding to the voice data. Therefore, the recognition information is mainly in text form; that is, the recognition information contains text information corresponding to the voice data. Furthermore, to facilitate the localization processing of the recognition information, timestamp information corresponding to the text information is further included in the recognition information.

[0034] Step S120: Determine the target text in the text information, as well as the start and end timestamps corresponding to the target text. The target text is the text in the text information that was incorrectly identified.

[0035] The target text refers to the text content containing transcription errors within the text information, specifically including incorrectly transcribed characters or phrases. Characters include Chinese characters and foreign language characters, with Chinese characters further including words and phrases. Identifying the target text within the text information can be achieved through various methods, such as manual or machine recognition. For example, target text within the text information can be identified using regular expression matching rules, speech quality inspection models, and / or error recognition models. In short, this disclosure does not limit the specific method for determining the target text.

[0036] After identifying the target text contained in the text information, the start and end timestamps of the target text in the recognition information are further determined in order to locate the position of the target text in the recognition information.

[0037] Step S130: Process the speech data according to the start timestamp and end timestamp to obtain the speech segment corresponding to the target text.

[0038] Specifically, various tools such as voice editors can be used to trim the voice data based on the start and end timestamps to obtain processed voice segments. These voice segments must contain at least the voice content corresponding to the target text.

[0039] Step S140: Desensitize the speech segments. The desensitized speech segments are used to adjust the speech recognition model.

[0040] In order to avoid the leakage of user information, it is necessary to perform desensitization processing before sending the voice segments to the server corresponding to the speech recognition model. Since the cropping operation has already been performed, the scope of desensitization processing is smaller and the processing speed is faster.

[0041] In the embodiments provided in this disclosure, after obtaining the recognition information corresponding to the speech data, the target text (i.e., the text that is misidentified) contained in the recognition information can be identified, thereby determining the start and end timestamps corresponding to the target text. Then, the speech data is processed based on the start and end timestamps to obtain speech segments. After desensitization processing of the processed speech segments, they can be used to adjust the speech recognition model. Because this embodiment can identify the target text contained in the recognition information and process the speech data into speech segments based on the timestamp information, the amount of speech data is significantly reduced. Therefore, the time consumption of desensitization processing and the amount of subsequent data transmission are reduced, significantly improving subsequent processing efficiency.

[0042] Figure 2 A flowchart illustrating a method for processing voice data, provided as another embodiment of this disclosure. (Refer to...) Figure 2 The method includes:

[0043] Step S210: Obtain the speech data to be processed and the recognition information corresponding to the speech data; wherein, the recognition information is obtained by the speech recognition model and includes the text information corresponding to the speech data and the timestamp information corresponding to the text information.

[0044] In this embodiment, the voice data can be call recording data. Of course, it can also be various types, such as real-time call data. By processing the voice data using a voice recognition model, recognition information in text form can be obtained. The recognition information includes the text information corresponding to the voice data and the timestamp corresponding to the text information. Furthermore, since the voice data is call data, the recognition information, in addition to the timestamp information, also further includes vocal tract information.

[0045] In practice, Automatic Speech Recognition (ASR) technology can be used to obtain recognition information corresponding to the speech data. Accordingly, the speech recognition model is the ASR model.

[0046] To facilitate understanding, a specific example of identification information is given below:

[0047]

[0048]

[0049]

[0050] As can be seen from the above example, "beginTime" in the identification information is used to identify the start time of a sentence, "endTime" is used to identify the end time of a sentence, and "channelId" is used to identify the vocal tract information corresponding to the speaker of a sentence.

[0051] Step S220: Determine the target text contained in the text information.

[0052] The target text refers to the text content containing transcription errors within the text information, specifically including incorrectly transcribed characters or phrases. Characters include Chinese characters and foreign language characters, with Chinese characters further including words and phrases. Therefore, the target text is the text in the text information where errors are identified. In specific implementation, this can be achieved through at least one of the following methods:

[0053] In the first implementation, regular expression matching rules are used to perform regular expression matching on the text information, and the target text in the text information is determined based on the regular expression matching results. In this regular expression matching, multiple regular expressions can be pre-defined to determine the target text contained in the text information.

[0054] For example, regular expression matching rules include keyword-based matching rules. Accordingly, when determining the target text contained in text information, matching can be performed based on pre-defined keywords. Specifically, this can be achieved as follows: First, determine a set of type keywords corresponding to the business type of the text information; each type keyword set corresponds one-to-one with a business type. Then, match the text information with the type keyword set, and determine the target text in the text information based on the matching results. Here, the business type is used to characterize business features related to the call. The method and specific number of business types can be set by those skilled in the art according to actual needs. To improve the accuracy of keyword matching, different type keyword sets are configured for each business type. The keywords stored in each type keyword set are determined according to the business characteristics of the corresponding business type. By dividing business types and setting different type keyword sets for each, the accuracy of target text recognition can be improved.

[0055] The aforementioned business types can be categorized in various ways. For example, they can be classified by business stage: consultation business, information push business, and follow-up business; or by business type: financial business, commodity business, and education business. In short, different business types have different call format specifications, which are used to define standard call content and scripts. Correspondingly, based on these call format specifications, a set of type keywords corresponding to the current call business type can be determined. For example, in the information push business type, the type keyword set is determined based on the type and content of the information to be pushed. For instance, the type keyword set corresponding to the information push business type should at least include the main content of the information to be pushed, such as the name of the information, the acquisition channel, and related descriptive information. Similarly, for the follow-up business type, the type keyword set is determined based on the follow-up content, such as the greetings, summary, and closing remarks. In conclusion, setting different type keyword sets for different business types helps improve the accuracy of identification.

[0056] Optionally, multiple sets of interval keywords can be set according to the process nodes corresponding to different process nodes based on the business type. This divides the text information into multiple text intervals, and each text interval is matched with its corresponding interval keyword set. By leveraging the process nodes and interval keyword sets, keywords for each process node can be flexibly set, thereby improving recognition accuracy.

[0057] For example, in one implementation, the business type includes at least one process node. When matching text information with a set of type keywords and determining the target text in the text information based on the matching results, this is achieved as follows: The text information is divided into at least two text intervals based on at least one process node of the business type; a set of interval keywords is determined for each text interval, with a one-to-one correspondence between the text interval and the interval keyword set; each text interval is matched with its corresponding interval keyword set, and the target text in the text information is determined based on the matching results. The number and names of the process nodes corresponding to different business types usually vary. For example, in a follow-up call business type, there are typically three process nodes: the first process node introduces the purpose of the call, indicating the intention to follow up; the second process node is used to communicate specifically with the user to achieve the follow-up purpose; and the third process node is used to summarize the content of the follow-up call to end the call. Correspondingly, each process node corresponds one-to-one with a set of interval keywords, and the set of interval keywords corresponding to each process node represents the standard script for that process node. In practical implementation, the text information can be divided into at least two text intervals based on call format specifications, and a set of keywords can be set for each text interval. This allows for keyword matching on a text interval basis, further improving matching accuracy. Different text intervals correspond to different keywords. For example, a typical phone call includes a greeting, a content introduction, and a closing. Correspondingly, the text information corresponding to the call data can be divided into three text intervals, corresponding to the greeting, content introduction, and closing. In the greeting, the corresponding keywords include the company name of the salesperson making the call and polite phrases; in the content introduction, the corresponding keywords include the product name (i.e., the content of the push notification).

[0058] In the second implementation, a voice quality inspection model is used to obtain voice quality inspection results corresponding to the recognized information. Based on these results, the target text contained within the text information is determined. The voice quality inspection model is used to inspect the recognized information to evaluate the caller's business capabilities. For example, the voice quality inspection model can be trained on pre-annotated voice data. Alternatively, the voice quality inspection model can be implemented based on the keyword matching method described above. For instance, if the voice quality inspection model matches a keyword, the quality inspection result is determined to be satisfactory; if it does not match a keyword, the result is determined to be unsatisfactory. Correspondingly, a quality inspection score is given based on the results. Therefore, in the process of quality inspection of voice data using a voice quality inspection model, the target text contained within the text information can also be indirectly obtained based on the quality inspection score. Generally, if the quality inspection result is satisfactory, it indicates that the speech data conforms to relevant standards, and therefore, the recognition information can be presumed to be correct. If the quality inspection result is unsatisfactory, it indicates that the speech data does not conform to relevant standards. In cases where the speech data does not conform to relevant standards, it may be due to recognition errors in the recognition information. Therefore, the focus should be on processing the recognition information whose output of the speech quality inspection model is unsatisfactory to obtain the text content containing transcription errors. Thus, by using a speech quality inspection model to obtain the quality inspection result corresponding to the text information, the target text within the text information can be determined based on the quality inspection result.

[0059] In the third implementation, an error detection model is used to obtain the conversion errors contained in the identification information, and the target text contained in the identification information is then identified based on these conversion errors. The error detection model is trained on labeled error recordings. Therefore, it is evident that the target text within text information can also be obtained through an error detection model. The above three methods can be used individually or in combination, and this invention does not limit their use.

[0060] Furthermore, considering the potential errors in the aforementioned identification methods, a review process is implemented to improve accuracy and reduce the error rate. Specifically, when determining the target text contained in the text information, firstly, candidate target texts are identified within the text information, and a review request is generated based on these candidate target texts. Then, in response to the review operation triggered by the review request, the target text contained in the text information is determined. The candidate target text can be determined using at least one of the three methods mentioned above. Correspondingly, the candidate target text consists of potentially erroneous text units obtained through regular expression matching rules, speech quality inspection models, and / or error recognition models.

[0061] If the review operation is a confirmation-type review operation that includes confirmation information, it means the user has confirmed that the candidate target text is indeed the target text (i.e., it contains error information). Therefore, the candidate target text is confirmed as the target text contained in the text information. If the review operation is an error-correction-type review operation that includes error correction information, it means that the candidate target text has an inaccurate recognition problem. Therefore, the target text in the text information is determined based on the error correction information; the error correction information is used to determine whether the candidate target text contains erroneous content. By using the review operation, the accuracy of target text recognition can be significantly improved, and recognition errors can be avoided.

[0062] Step S230: Determine the start and end timestamps of the target text in the recognition information, and determine the vocal tract information of the target text in the recognition information.

[0063] After identifying the target text contained in the identification information, the start and end timestamps of the target text in the identification information are further determined in order to locate the position of the target text in the identification information.

[0064] Specifically, when determining the start and end timestamps of the target text in the recognition information, this can be based on the unit attributes of the target text. These unit attributes include character attributes and sentence attributes. Character-attributed target text typically contains misspelled characters or words, while sentence-attributed target text typically contains erroneous sentences. Accordingly, when the target text's unit attribute is a character attribute, the start and end timestamps in the recognition information are obtained as follows: determine the sentence to which the erroneous character belongs, and based on the sentence's start and end times, determine the target text's start and end timestamps in the recognition information. When the target text's unit attribute is a sentence attribute, the start and end timestamps in the recognition information are obtained as follows: determine the context sentence of the erroneous sentence, and based on the context sentence's start and end times, determine the target text's start and end timestamps in the recognition information.

[0065] Furthermore, when there are multiple target texts, at least two target texts can be identified as an erroneous text combination based on their distribution. The start and end timestamps of this erroneous text combination can then be determined. The speech data can then be trimmed based on these timestamps to ensure that multiple closely related target texts can be trimmed into the same speech segment. The distribution of the multiple target texts includes factors such as the spacing between them and their distribution density. This erroneous text combination segmentation method avoids splitting two adjacent and closely related target texts during trimming and prevents incomplete content due to the lack of contextual information during the trimming process.

[0066] In an alternative implementation, the inventors discovered that if the length of the speech corresponding to the erroneous text combination is too long, it may result in an excessively large amount of speech segment data obtained from the final trimming, thus causing difficulties for the final desensitization and data transmission operations. To address this issue, when there are multiple target texts, when determining the start and end timestamps of the target texts in the recognition information, the multiple target texts are further divided into at least two erroneous text sets based on the interval distance and / or semantic association between them. For each erroneous text set, the start and end timestamps corresponding to that set in the recognition information are determined, thus establishing the start and end timestamps of each erroneous text set as the start and end timestamps of the target texts. Then, in subsequent steps, when trimming the speech data based on the start and end timestamps, the speech data is trimmed according to the start and end timestamps of each erroneous text set in the recognition information, resulting in at least two speech segments. Therefore, it is evident that target texts within an error text set are closely spaced or semantically related. Dividing the text into error text sets prevents semantically related target texts from being cut into different speech segments, ensuring the integrity of a single speech segment. Furthermore, dividing the text into multiple error text sets ensures that each cut speech segment is relatively short, preventing any single segment from becoming excessively long. Thus, cutting speech segments using error text sets balances semantic integrity and data volume.

[0067] The aforementioned methods for cropping based on error text combinations and error text sets can be used individually or in combination. When used in combination, an error text set can contain at least one error text combination.

[0068] Step S240: Based on the start timestamp, end timestamp, and vocal tract information, the speech data is trimmed to obtain speech segments.

[0069] Specifically, various tools such as voice editors can be used to trim voice data based on start and end timestamps to obtain processed voice segments. These processed voice segments are then used to refine the speech recognition model. For example, FFmpeg can be used to perform voice trimming.

[0070] For example, in one implementation, a trimming instruction containing vocal tract information, a start timestamp, and an end timestamp is generated, so that the speech editing tool can trim the speech data according to the trimming instruction to obtain a speech segment corresponding to the vocal tract information, the start timestamp, and the end timestamp.

[0071] Step S250: De-identify the speech segment and send the de-identified speech segment to the server corresponding to the speech recognition model so that the server can correct the speech recognition model based on the de-identified speech segment.

[0072] Because voice clips may contain sensitive data such as user privacy information, they need to be anonymized to prevent privacy leaks. This anonymization process can be implemented using preset rules. For example, by analyzing the subject-verb-object structure of a sentence, the subject is extracted and its identity is determined based on name composition rules. If it is a person's name, the subject needs anonymization. Similarly, extracting numerical content from a sentence and matching it against preset sensitive number formats (such as six-digit birthdays, 11-digit phone numbers, and ID card numbers) determines if an anonymization is required. Furthermore, to avoid inaccuracies in automatic anonymization, candidate anonymization content can be identified using preset rules. A confirmation request containing this candidate content is generated, and the anonymization operation is performed based on the confirmation command triggered by the request. Adding a confirmation request allows for accurate anonymization through manual verification.

[0073] In one specific implementation, a voice editor is used to trim or silence the voice content in the voice clips containing sensitive user privacy information (such as customer name, ID card, phone number, etc.) to prevent the leakage of user privacy information.

[0074] In addition to anonymization, annotation can be performed on speech segments to identify errors and their corresponding correct content. The anonymized and / or annotated speech segments are then used as training samples for the speech recognition model, thereby refining the model.

[0075] In summary, because this embodiment can identify target text contained in the identified information and trim the speech data into speech segments based on timestamp information, it significantly reduces the amount of speech data. Therefore, it reduces the amount of data transmitted, shortens the processing time for subsequent processing of speech segments, and greatly improves subsequent processing efficiency. Furthermore, this method can accurately trim based on vocal tract information, improving the accuracy of speech segments. Specifically, through keyword matching, speech quality inspection models, error detection models, and other methods, it can quickly and accurately identify target text in the identified information.

[0076] To facilitate understanding, a specific example is used below to detail the implementation of the above embodiments. In this example, an intelligent customer service quality inspection system is used to inspect the ASR transcription results of call recording data. This intelligent customer service quality inspection system uses detection models and algorithms to detect the text content of data such as voice, video, and dialogue text, and can detect the call behavior of agents.

[0077] In related technologies, when using ASR services, the transcribed recognition information may exhibit low accuracy or anomalies in certain specific scenarios. The conventional approach is to collect a batch of audio files, anonymize all content, and then provide it to the ASR server to resolve or improve recognition accuracy. Improving ASR recognition generally requires collecting a large amount of specific, anomalous audio file data. The average recording duration varies from 1 minute to half an hour depending on the business requirements. Furthermore, due to user privacy policies, sensitive information in the audio files provided to the ASR server needs to be anonymized. The conventional method is to manually listen to the entire recording and perform the anonymization process. Each recording contains a lot of irrelevant information, but the anonymization process requires listening to the entire recording and manual verification, making it very labor-intensive.

[0078] To address the aforementioned issues, this example employs an intelligent customer service quality inspection system to perform quality checks on the ASR transcription results of call recordings. Based on the inspection results, erroneous data is located, and the recordings are trimmed. This allows for desensitization processing only on the trimmed recording segments, significantly reducing the processing workload.

[0079] like Figure 3 As shown, this example specifically includes the following steps:

[0080] Step S301: Transcribe the call recording file into the corresponding identification information using the ASR service.

[0081] Step S302: During the processing of the transcribed text data, identify any transcription errors contained therein.

[0082] Specifically, in the process of using the recognition information obtained from audio transcription, errors in the transcription of sentences are identified. For example, in an intelligent customer service quality inspection system, an intelligent quality inspection model is used to inspect the transcribed text. If the quality inspection result of an audio recording is abnormal, a re-inspection is required. The re-inspection determines that the abnormality is due to the failure to match specific keywords (polite language, company or product, etc.) or semantics (identity verification, customer reassurance, etc.). Accordingly, the intelligent customer service quality inspection system provides error prompts, and staff verify the information based on the error prompts. If the verification determines that the reason for the failure to match keywords is an error in ASR transcription, then... For example, in the recognition information listed above, the underlined parts represent the content with transcription errors.

[0083] Step S303: Determine the timestamp and channel information of the abnormal speech segment with transcription errors.

[0084] Specifically, the ASR transcription result of the audio file carries the time offset of the current sentence transcription. Therefore, sentences with ASR transcription errors can be marked based on this time offset, thereby obtaining the channel, start, and end time offsets of the sentence. Repeatedly performing the above operation can collect a large amount of content marked with ASR transcription error data.

[0085] Step S304: Perform trimming based on timestamp information and audio channel information to obtain abnormal audio segments.

[0086] The collected data is edited using recording and editing tools such as FFmpeg to cut the specified recording files. Based on the marked recording channels, start and end times, and other information, corresponding recording segment files are generated to obtain abnormal voice segments.

[0087] Step S305: Desensitize abnormal audio segments.

[0088] Specifically, abnormal audio segments can be manually anonymized and verified.

[0089] Step S306: Provide the de-identified abnormal voice segments to the ASR server, which will then optimize and update the ASR service to perform subsequent recording transcription operations based on the optimized and updated ASR service.

[0090] Among these methods, the processed abnormal voice segments can be provided to the ASR service provider, who can then improve the effectiveness of the ASR service.

[0091] Therefore, in the example above, the data provided to the ASR server for optimization and improvement is reduced from the entire audio file to audio segment files within a specific anomaly range. This reduction in recording duration significantly decreases the time spent on manual desensitization labeling and verification, reduces data transmission bandwidth consumption, and lowers the risk of sensitive information leakage during data processing. Furthermore, it avoids the problem of increased error rates due to fatigue during manual desensitization caused by excessively long recordings.

[0092] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0093] In addition, this disclosure also provides a voice data processing apparatus, an electronic device, and a computer-readable storage medium, all of which can be used to implement any of the voice data processing methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section, and will not be repeated here.

[0094] Figure 4 This is a block diagram of a voice data processing apparatus provided in an embodiment of the present disclosure.

[0095] Reference Figure 4 This disclosure provides a voice data processing apparatus 40, which includes:

[0096] The acquisition module is adapted to acquire the voice data to be processed and the recognition information corresponding to the voice data; wherein the recognition information is obtained according to the voice recognition model, and the recognition information includes the text information corresponding to the voice data and the timestamp information corresponding to the text information;

[0097] The determining module is adapted to determine the target text in the text information, as well as the start timestamp and end timestamp corresponding to the target text, wherein the target text is the text in the text information that was incorrectly identified;

[0098] The processing module is adapted to process the speech data according to the start timestamp and the end timestamp to obtain the speech segment corresponding to the target text;

[0099] The desensitization module is adapted to desensitize the speech segment, and the desensitized speech segment is used to adjust the speech recognition model.

[0100] Optionally, the identification information may also include vocal tract information;

[0101] The processing module is specifically adapted to: generate a trimming instruction that includes the audio channel information, the start timestamp, and the end timestamp;

[0102] According to the cropping instruction, the speech segment corresponding to the target text is cropped from the speech data.

[0103] Optionally, the determining module is specifically adapted to:

[0104] The text information is subjected to regular expression matching rules, and the target text in the text information is determined based on the regular expression matching results.

[0105] The voice quality inspection model is used to obtain the quality inspection results corresponding to the text information, and the target text in the text information is determined based on the quality inspection results.

[0106] The target text in the text information is obtained through an error detection model.

[0107] Optionally, the determining module is specifically adapted to:

[0108] Identify candidate target text contained in the text information, and generate a review request based on the candidate target text;

[0109] In response to a review operation triggered by the review request, the target text in the text information is determined.

[0110] Optionally, determine which module is specifically suitable for:

[0111] If the review operation is a confirmation-type review operation that includes confirmation information, then the candidate target text is determined as the target text;

[0112] If the review operation is an error correction type review operation that includes error correction information, then the target text in the text information is determined according to the error correction information; wherein, the error correction information is used to determine whether the candidate target text contains erroneous content.

[0113] Optionally, if the regular expression matching rules include keyword-based matching rules, then the determining module is specifically adapted to:

[0114] Determine a set of type keywords corresponding to the business type of the text information, wherein the set of type keywords corresponds one-to-one with the business type;

[0115] The text information is matched with the set of keywords of the type, and the target text in the text information is determined based on the matching result.

[0116] Optionally, the business type includes at least one process node, and the determining module is specifically adapted to:

[0117] The text information is divided into at least two text intervals based on at least one process node of the business type;

[0118] Determine the set of keywords corresponding to each text interval, wherein the text interval and the set of keywords correspond one-to-one;

[0119] Each text interval is matched with the set of interval keywords corresponding to each text interval, and the target text in the text information is determined based on the matching results.

[0120] Optionally, when there are multiple target texts, the determining module is specifically adapted to:

[0121] Based on the spacing and / or semantic relationships between multiple target texts, the multiple target texts are divided into at least two sets of error texts;

[0122] Determine the start and end timestamps for each set of error texts, and set the start and end timestamps for each set of error texts as the start and end timestamps for the target text.

[0123] In the apparatus provided in this disclosure, after acquiring recognition information corresponding to speech data, it can identify the target text contained in the recognition information, thereby determining the start and end timestamps of the target text in the recognition information. Then, based on the start and end timestamps, the speech data is trimmed to obtain speech segments, and the speech recognition model is corrected using the processed speech segments. Because this embodiment can identify the target text contained in the recognition information and trim the speech data into speech segments based on the timestamp information, the amount of speech data is significantly reduced. Therefore, the amount of data transmitted is reduced, the processing time for subsequent processing of the speech segments is shortened, and the subsequent processing efficiency is greatly improved.

[0124] Figure 5 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0125] Reference Figure 5 This disclosure provides an electronic device, which includes: at least one processor 501; at least one memory 502; and one or more I / O interfaces 503 connected between the processor 501 and the memory 502; wherein the memory 502 stores one or more computer programs that can be executed by the at least one processor 501, and the one or more computer programs are executed by the at least one processor 501 to perform the above-mentioned voice data processing method.

[0126] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor / processor core, implements the aforementioned method for processing voice data. The computer-readable storage medium may be volatile or non-volatile.

[0127] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described voice data processing method.

[0128] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0129] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0130] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0131] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0132] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0133] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0135] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0137] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for processing voice data, characterized in that, include: Acquire the speech data to be processed, and the recognition information corresponding to the speech data; wherein the recognition information is obtained according to the speech recognition model, and the recognition information includes the text information corresponding to the speech data and the timestamp information corresponding to the text information; Determine the target text in the text information, as well as the start and end timestamps corresponding to the target text. The target text is the text that was incorrectly identified in the text information. When there are multiple target texts, determining the start and end timestamps corresponding to the target texts in the text information includes: Based on the spacing and / or distribution density between multiple target texts, at least two target texts are identified as an error text combination; based on the spacing and / or semantic association between multiple target texts, an error text combination is divided into at least two error text sets; the start timestamp and end timestamp corresponding to each error text set are determined respectively, and the start timestamp and end timestamp corresponding to each error text set are determined as the start timestamp and end timestamp corresponding to the target text; The speech data is processed based on the start timestamp and the end timestamp to obtain the speech segment corresponding to the target text; The speech segment is desensitized, and the desensitized speech segment is used to adjust the speech recognition model.

2. The method according to claim 1, characterized in that, The identification information also includes vocal tract information; The step of processing the speech data according to the start timestamp and the end timestamp to obtain the speech segment corresponding to the target text includes: Generate a trimming instruction that includes the audio channel information, the start timestamp, and the end timestamp; According to the cropping instruction, the speech segment corresponding to the target text is cropped from the speech data.

3. The method according to claim 1, characterized in that, Determining the target text in the text information includes at least one of the following methods: The text information is subjected to regular expression matching rules, and the target text in the text information is determined based on the regular expression matching results. The voice quality inspection model is used to obtain the quality inspection results corresponding to the text information, and the target text in the text information is determined based on the quality inspection results. The target text in the text information is obtained through an error detection model.

4. The method according to claim 1, characterized in that, Determining the target text in the text information includes: Identify candidate target text contained in the text information, and generate a review request based on the candidate target text; In response to a review operation triggered by the review request, the target text in the text information is determined.

5. The method according to claim 4, characterized in that, In response to a review operation triggered by the review request, the target text in the text information is determined, including: If the review operation is a confirmation-type review operation that includes confirmation information, then the candidate target text is determined as the target text; If the review operation is an error correction type review operation that includes error correction information, then the target text in the text information is determined according to the error correction information; wherein, the error correction information is used to determine whether the candidate target text contains erroneous content.

6. The method according to any one of claims 3-5, characterized in that, The regular expression matching rules include keyword-based matching rules. Therefore, the step of performing regular expression matching on the text information and determining the target text in the text information based on the regular expression matching results includes: Determine a set of type keywords corresponding to the business type of the text information, wherein the set of type keywords corresponds one-to-one with the business type; The text information is matched with the set of keywords of the type, and the target text in the text information is determined based on the matching result.

7. The method according to claim 6, characterized in that, The business type includes at least one process node, and the step of matching the text information with the set of keywords for the type, and determining the target text in the text information based on the matching result, includes: The text information is divided into at least two text intervals based on at least one process node of the business type; Determine the set of keywords corresponding to each text interval, wherein the text interval and the set of keywords correspond one-to-one; Each text interval is matched with the set of interval keywords corresponding to each text interval, and the target text in the text information is determined based on the matching results.

8. A voice data processing apparatus, characterized in that, include: The acquisition module is adapted to acquire the voice data to be processed and the recognition information corresponding to the voice data; wherein the recognition information is obtained according to the voice recognition model, and the recognition information includes the text information corresponding to the voice data and the timestamp information corresponding to the text information; The determining module is adapted to determine the target text in the text information, and the start and end timestamps corresponding to the target text, wherein the target text is the text that was incorrectly identified in the text information. When there are multiple target texts, determining the start and end timestamps corresponding to the target texts in the text information includes: Based on the spacing and / or distribution density between multiple target texts, at least two target texts are identified as an error text combination; based on the spacing and / or semantic association between multiple target texts, an error text combination is divided into at least two error text sets; the start timestamp and end timestamp corresponding to each error text set are determined respectively, and the start timestamp and end timestamp corresponding to each error text set are determined as the start timestamp and end timestamp corresponding to the target text; The processing module is adapted to process the speech data according to the start timestamp and the end timestamp to obtain the speech segment corresponding to the target text; The desensitization module is adapted to desensitize the speech segment, and the desensitized speech segment is used to adjust the speech recognition model.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-7.

Images