Training data acquisition method, device, equipment and storage medium

Abstract

The present application discloses a method, device, equipment and storage medium for acquiring training data, which belongs to the field of artificial intelligence. The method includes: cutting the audio data collected in the target scene to obtain a training audio set; each audio in the training audio set includes a wake-up command word; determining the execution order of multiple filtering methods according to the environment type of the target scene; different filtering methods are used to filter the audio in the training audio set based on different filtering conditions; when filtering each audio according to the execution order, in response to any audio meeting the filtering condition corresponding to the current filtering method, the audio is deleted from the training audio set; the filtered training audio set is used as training data for training the acoustic model. The present application can obtain high-quality training data, and model training based on high-quality training data can improve the model training effect, thereby improving the accuracy of speech recognition.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method, apparatus, device, and storage medium for acquiring training data. Background Art

[0002] Speech recognition is a major breakthrough in the development of artificial intelligence. Broadly speaking, speech recognition focuses on speech, with the goal of enabling natural language-based interaction between humans and machines. In a narrower sense, speech recognition is a technology that allows machines to convert speech into text or commands through a process of recognition and understanding.

[0003] Speech recognition technology is currently widely used in industries such as industry, home appliances, communications, automotive, electronics, healthcare, and home services. For example, in smart elevator call systems, speech recognition based on acoustic models enables intelligent elevator calling, where the user wakes up after recognizing the wake-up command word.

[0004] The acoustic model is a machine learning model in the field of artificial intelligence. Before applying the acoustic model for speech recognition, it is necessary to obtain labeled training data for model training. However, the quality of the training data directly affects the accuracy of model training, which in turn affects the performance of speech recognition. Therefore, how to obtain high-quality training data for model training to improve model training accuracy and thus ensure speech recognition accuracy has become a focus of technical personnel in this field. Summary of the Invention

[0005] The embodiments of the present application provide a training data acquisition method, apparatus, device, and storage medium that can acquire high-quality training data. Acoustic model training based on high-quality training data not only ensures model training accuracy and improves model training results, but also significantly improves speech recognition accuracy by performing speech recognition based on the trained acoustic model. The technical solution is as follows:

[0006] In one aspect, a method for obtaining training data is provided, the method comprising:

[0007] Cutting the audio data collected in the target scene to obtain a training audio set; wherein each audio in the training audio set includes a wake-up command word;

[0008] Determining an execution order of multiple filtering methods according to the environment type of the target scene; wherein different filtering methods are used to filter the audio in the training audio set based on different filtering conditions;

[0009] When filtering and determining each of the audios in the execution order, in response to any audio satisfying a filtering condition corresponding to the current filtering mode, deleting the audio from the training audio set;

[0010] The filtered training audio set is used as training data for training the acoustic model.

[0011] In a possible implementation, determining the execution order of multiple filtering methods according to the environment type of the target scene includes:

[0012] Obtaining the location type and surrounding infrastructure information corresponding to the target scene;

[0013] Obtaining a collection period of the audio data;

[0014] The execution order of the multiple filtering methods is determined according to the type of place corresponding to the target scene, the surrounding infrastructure information, and the collection period of the audio data.

[0015] In one possible implementation, determining the execution order of the multiple filtering methods according to the type of venue corresponding to the target scene, surrounding infrastructure information, and the collection period of the audio data includes:

[0016] determining, based on the type of the venue, an initial execution order of the plurality of filtering methods;

[0017] The initial execution order is modified according to the surrounding infrastructure information and the collection period of the audio data to obtain a final execution order of the multiple filtering methods.

[0018] In a possible implementation, in response to any audio satisfying a filtering condition corresponding to a current filtering mode, deleting the audio from the training audio set includes:

[0019] For any audio, obtain the audio waveform of the audio;

[0020] Performing waveform recognition on the audio waveform graph to obtain a waveform recognition result;

[0021] In response to determining that the audio is truncation-prone based on the waveform recognition result, deleting the audio from the training audio set;

[0022] The truncation phenomenon refers to the lack of some audio frames corresponding to the wake-up command word in the audio.

[0023] In one possible implementation, the method further includes at least one of the following:

[0024] In response to the waveform feature indicated by the waveform recognition result matching the first waveform feature, determining that the audio is truncation-prone; wherein the first waveform feature corresponds to the first audio, and the audio starting position of the first audio is truncation-prone;

[0025] In response to the waveform feature indicated by the waveform recognition result matching the second waveform feature, determining that the audio is truncation; wherein the second waveform feature corresponds to the second audio, and the audio end position of the second audio is truncation;

[0026] In response to the waveform feature indicated by the waveform recognition result matching the third waveform feature, it is determined that the audio has a truncation phenomenon; wherein, the third waveform feature corresponds to a third audio, and the audio middle position of the third audio has a truncation phenomenon.

[0027] In a possible implementation, in response to any audio satisfying a filtering condition corresponding to a current filtering mode, deleting the audio from the training audio set includes:

[0028] For any audio, perform voiceprint recognition on the audio to obtain a voiceprint recognition result;

[0029] In response to determining, based on the voiceprint recognition result, that the audio corresponds to multiple speakers and the number of audio frames corresponding to at least two speakers is greater than a frame number threshold, performing speech recognition on the audio;

[0030] In response to determining that chat speech exists in the audio based on the speech recognition result, the audio is deleted from the training audio set.

[0031] In a possible implementation, in response to any audio satisfying a filtering condition corresponding to a current filtering mode, deleting the audio from the training audio set includes:

[0032] For any audio, obtaining the signal-to-noise ratio of the audio;

[0033] In response to a signal-to-noise ratio of the audio being less than a signal-to-noise ratio threshold, the audio is deleted from the training audio set.

[0034] In a possible implementation, the clipping of audio data collected in the target scene to obtain a training audio set includes:

[0035] Performing audio truncation on the audio data collected in the target scene to obtain an initial audio set; wherein each audio in the initial audio set includes a wake-up command word;

[0036] Performing voice endpoint detection on each audio in the initial audio set;

[0037] Based on the voice endpoint detection result, the silent segments of each audio in the initial audio set are eliminated to obtain the training audio set; wherein the duration of the silent segments is greater than the target duration.

[0038] In another aspect, a training data acquisition device is provided, the device comprising:

[0039] A first processing module is configured to cut audio data collected in a target scene to obtain a training audio set; wherein each audio in the training audio set includes a wake-up command word;

[0040] A second processing module is configured to determine an execution order of multiple filtering methods according to the environment type of the target scene; wherein different filtering methods are used to filter the audio in the training audio set based on different filtering conditions;

[0041] The third processing module is configured to, when filtering and judging each of the audios in accordance with the execution order, delete the audio from the training audio set in response to any audio satisfying the filtering condition corresponding to the current filtering method; and use the filtered training audio set as training data for training the acoustic model.

[0042] In a possible implementation, the second processing module is configured to:

[0043] Obtaining the location type and surrounding infrastructure information corresponding to the target scene;

[0044] Obtaining a collection period of the audio data;

[0045] The execution order of the multiple filtering methods is determined according to the type of place corresponding to the target scene, the surrounding infrastructure information, and the collection period of the audio data.

[0046] In a possible implementation, the second processing module is configured to:

[0047] determining, based on the type of the venue, an initial execution order of the plurality of filtering methods;

[0048] The initial execution order is modified according to the surrounding infrastructure information and the collection period of the audio data to obtain a final execution order of the multiple filtering methods.

[0049] In a possible implementation, the third processing module is configured to:

[0050] For any audio, obtain the audio waveform of the audio;

[0051] Performing waveform recognition on the audio waveform graph to obtain a waveform recognition result;

[0052] In response to determining that the audio is truncation-prone based on the waveform recognition result, deleting the audio from the training audio set;

[0053] The truncation phenomenon refers to the lack of some audio frames corresponding to the wake-up command word in the audio.

[0054] In a possible implementation, the third processing module is configured to:

[0055] In response to the waveform feature indicated by the waveform recognition result matching the first waveform feature, determining that the audio is truncation-prone; wherein the first waveform feature corresponds to the first audio, and the audio starting position of the first audio is truncation-prone;

[0056] In response to the waveform feature indicated by the waveform recognition result matching the second waveform feature, determining that the audio is truncation; wherein the second waveform feature corresponds to the second audio, and the audio end position of the second audio is truncation;

[0057] In response to the waveform feature indicated by the waveform recognition result matching the third waveform feature, it is determined that the audio has a truncation phenomenon; wherein, the third waveform feature corresponds to a third audio, and the audio middle position of the third audio has a truncation phenomenon.

[0058] In a possible implementation, the third processing module is configured to:

[0059] For any audio, perform voiceprint recognition on the audio to obtain a voiceprint recognition result;

[0060] In response to determining, based on the voiceprint recognition result, that the audio corresponds to multiple speakers and the number of audio frames corresponding to at least two speakers is greater than a frame number threshold, performing speech recognition on the audio;

[0061] In response to determining that chat speech exists in the audio based on the speech recognition result, the audio is deleted from the training audio set.

[0062] In a possible implementation, the third processing module is configured to:

[0063] For any audio, obtaining the signal-to-noise ratio of the audio;

[0064] In response to a signal-to-noise ratio of the audio being less than a signal-to-noise ratio threshold, the audio is deleted from the training audio set.

[0065] In a possible implementation, the first processing module is configured to:

[0066] Performing audio truncation on the audio data collected in the target scene to obtain an initial audio set; wherein each audio in the initial audio set includes a wake-up command word;

[0067] Performing voice endpoint detection on each audio in the initial audio set;

[0068] Based on the voice endpoint detection result, the silent segments of each audio in the initial audio set are eliminated to obtain the training audio set; wherein the duration of the silent segments is greater than the target duration.

[0069] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the above-mentioned training data acquisition method.

[0070] On the other hand, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement the above-mentioned training data acquisition method.

[0071] On the other hand, a computer program product or computer program is provided, which includes computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the above-mentioned training data acquisition method.

[0072] After obtaining the training audio set, the embodiment of the present application will filter the audio in the training audio set to remove low-quality audio. This training data acquisition method can obtain high-quality training data, and acoustic model training based on high-quality training data can not only ensure the accuracy of model training and improve the model training effect, but also perform speech recognition based on the trained acoustic model, which can greatly improve the accuracy of speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0074] Figure 1 Schematic diagram of an implementation environment involved in a training data acquisition method provided in an embodiment of the present application;

[0075] Figure 2This is a flow chart of a method for obtaining training data provided in an embodiment of the present application;

[0076] Figure 3 Schematic diagram of the structure of a training data acquisition device provided in an embodiment of the present application;

[0077] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0078] Figure 5 It is a structural diagram of another computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0080] In this application, the terms "first," "second," and the like are used to distinguish identical or similar items having substantially the same role and function. It should be understood that "first," "second," and "nth" do not have a logical or temporal dependency, nor do they limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," and the like to describe various elements, these elements should not be limited by these terms.

[0081] These terms are simply used to distinguish one element from another. For example, without departing from the scope of various examples, the first element can be referred to as the second element, and similarly, the second element can also be referred to as the first element. Both the first element and the second element can be elements, and in some cases, can be separate and different elements. Wherein, at least one refers to one or more, for example, at least one element can be an element, two elements, three elements, etc., any integer greater than or equal to one. And a plurality of refers to two or more, for example, a plurality of elements can be two elements, three elements, etc., any integer greater than or equal to two.

[0082] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the initial text and interest tags involved in this application were obtained with full authorization.

[0083] Figure 1 This is a schematic diagram of an implementation environment involved in a training data acquisition method provided in an embodiment of the present application.

[0084] The training data acquisition method provided in the embodiment of the present application is applied to a training data acquisition device.

[0085] See also Figure 1 The training data acquisition device 101 is a computer device with machine learning capabilities. For example, the computer device can be a fixed computer device such as a personal computer or a server, or a mobile computer device such as a tablet computer or a smart phone. This application does not limit this.

[0086] Taking the smart elevator as an example, in a smart elevator's call system, the user's wake-up command (or call command in the smart elevator scenario) is generally short. Furthermore, before, during, or after the user calls the elevator, there may be other passengers chatting or the sound of the elevator's advertising screen playing.

[0087] Furthermore, when iteratively training the acoustic model of the elevator call system, actual elevator audio is required as training data. The elevator call system collects elevator audio over a long period of time and immediately initiates the elevator wake-up operation upon recognizing the elevator call command. The training data is segmented and annotated from this long-collected elevator audio. However, this training data may contain excessive noise, truncation, or idle chatter. The annotation results only reflect the actual elevator call command words required by the elevator call system. Therefore, using this training data to train the acoustic model would have a significant negative impact on the model and result in poor model training accuracy.

[0088] To address this issue, the present invention proposes a new method for acquiring training data to improve the quality of training data.

[0089] It should be noted that, in addition to the above-mentioned smart elevator scenario, the training data acquisition method provided in the embodiment of the present application can also be applied to other voice wake-up scenarios, such as home, restaurant, shopping mall and other public places, and this application does not limit this.

[0090] In one possible implementation, the present embodiment uses image recognition technology, voiceprint recognition technology, and signal-to-noise ratio calculation to improve the quality of training audio. For example, the present embodiment includes but is not limited to the following steps:

[0091] 1. Use VAD (Voice Activity Detection) technology to eliminate long periods of silence in the audio. This can reduce the audio size and thereby improve the subsequent feature extraction speed and model training speed.

[0092] In another possible implementation, the audio mentioned in this step is segmented by annotators from audio collected over a long period of time. These segmented audio constitute the training audio set. For example, the audio collection scenario is a smart elevator, but this application does not limit this.

[0093] 2. For any audio, if the noise level is too high, using it to train the acoustic model will negatively impact the model. To this end, the present embodiment calculates the signal-to-noise ratio of each audio. If the signal-to-noise ratio is lower than the set threshold, the audio is removed from the training audio set.

[0094] 3. Since each audio in the training audio set is cut out from audio collected over a long period of time, the audio may be truncated. In addition, sometimes the audio is truncate due to problems with audio collection. Truncation refers to the lack of some audio frames corresponding to the wake-up command words in the audio. In other words, the truncated audio lacks some pronunciation frames of the wake-up command words, which is very detrimental to the training of the acoustic model. For this reason, the embodiment of the present application will determine whether there is truncation for each audio, and delete the audio determined to be truncated from the training audio set.

[0095] 4. Voiceprint recognition technology is used to identify audio that contains idle chatter. If there are multiple speakers in an audio clip, and the number of audio frames corresponding to at least two speakers exceeds the set frame count threshold, then the audio clip may contain idle chatter. The audio clip is then fed into the speech recognition system, and the speech recognition results are used to further determine whether the audio clip contains idle chatter. If the audio clip is found to contain idle chatter, it is removed from the training audio set.

[0096] Through the above processing method, high-quality training data (labeled audio) that includes wake-up command words can be screened out, reducing the negative effects of low-quality audio on the acoustic model.

[0097] In another possible implementation, for the several filtering methods shown in steps 2-4 above, the embodiment of the present application also proposes a solution for determining the order of execution of multiple filtering methods according to the application scenario. In other words, different application scenarios may execute audio filtering in different orders. For example, for public places such as smart elevators, there is a greater possibility of excessive noise or people chatting, so each audio can be first subjected to a noise filtering judgment or a chat filtering judgment; if an audio does not pass the above filtering judgment, it is directly deleted from the training audio set. If the truncation filtering judgment is performed first and passes this filtering judgment, then the noise filtering judgment or the chat filtering judgment must continue to be executed, which is a waste of resources. See below for details. Figure 2 The embodiment shown.

[0098] Figure 2 This is a flow chart of a training data acquisition method provided by an embodiment of the present application. The execution subject of this method is a computer device, see Figure 2 The method flow of the embodiment of the present application includes the following steps.

[0099] 201. A computer device cuts audio data collected in a target scenario to obtain a training audio set; wherein each audio in the training audio set includes a wake-up command word.

[0100] In the embodiment of the present application, the target scene is any voice wake-up scene, such as home, restaurant, elevator or shopping mall and other public places, and the present application does not limit this. For example, the embodiment of the present application takes the target scene as a smart elevator as an example for explanation.

[0101] In one possible implementation, the audio data collected in the target scene is cut to obtain a training audio set, including but not limited to the following methods: performing audio truncation on the audio data collected in the target scene to obtain an initial audio set; wherein each audio in the initial audio set includes a wake-up command word; performing voice endpoint detection on each audio in the initial audio set; based on the obtained voice endpoint detection results, eliminating the silent segments of each audio in the initial audio set to obtain a training audio set; wherein the duration of the eliminated silent segments is greater than the target duration.

[0102] Taking the target scenario as an intelligent elevator as an example, the audio data collected in the target scenario is the elevator audio collected over a long period of time. Among them, the target duration can be an arbitrarily set time threshold, such as 30ms, which is not limited in this application.

[0103] This step reduces the audio size by eliminating long silent segments in the audio, thereby improving the subsequent feature extraction and model training speed.

[0104] 202. The computer device determines an execution order of multiple filtering methods according to an environment type of a target scene; wherein different filtering methods are used to filter the audio in the training audio set based on different filtering conditions.

[0105] In one possible implementation, the execution order of multiple filtering methods is determined according to the environment type of the target scene, including but not limited to the following methods:

[0106] Obtain the type of venue and surrounding infrastructure information corresponding to the target scene; obtain the collection time period of the audio data collected in the target scene; determine the execution order of multiple filtering methods based on the type of venue and surrounding infrastructure information corresponding to the target scene and the collection time period of the above audio data.

[0107] Among them, the type of place can be divided into private places and public places, etc.; the surrounding infrastructure information can be used to indicate basic suggestions in the surrounding area, such as whether there are overpasses, subways, construction sites or shopping malls in the surrounding area; the collection time period includes but is not limited to: daytime commuting time, daytime non-commuting time, night, etc., and this application does not limit this.

[0108] Exemplarily, the above-mentioned determination of the execution order of multiple filtering methods based on the venue type and surrounding infrastructure information corresponding to the target scene and the collection period of the above-mentioned audio data can be implemented based on the following method: determining the initial execution order of multiple filtering methods based on the venue type corresponding to the target scene; correcting the initial execution order based on the surrounding infrastructure information corresponding to the target scene and the collection period of the audio data collected in the target scene to obtain the final execution order of multiple filtering methods.

[0109] Taking the target scenario of a smart elevator as an example, since smart elevators are public places, the initial execution order of multiple filtering methods can be set to chat filtering → noise filtering → truncation filtering; assuming that the surrounding infrastructure information is a construction site and the collection period is during daytime non-working hours, the above initial execution order may be revised to noise filtering → chat filtering → truncation filtering.

[0110] 203. When the computer device performs filtering determination on each audio piece according to the execution order, in response to any audio piece meeting the filtering condition corresponding to the current filtering method, the audio piece is deleted from the training audio set.

[0111] 2031. For any audio piece in the training audio set, obtain a signal-to-noise ratio of the audio piece; in response to the signal-to-noise ratio of the audio piece being less than a signal-to-noise ratio threshold, delete the audio piece from the training audio set.

[0112] In one possible implementation, for any audio frame in the audio, the energy of the audio frame is obtained, and the ratio between the energy of the audio frame and a reference energy of noise is used as the signal-to-noise ratio of the audio frame. For example, the average signal-to-noise ratio of all audio frames in the audio can be compared with a set signal-to-noise ratio threshold, which is not limited in this application.

[0113] For example, each audio frame corresponds to an energy value, such as the root mean square energy of the audio signal, which represents the average energy of the audio signal waveform over a short period of time. Alternatively, a noise estimation algorithm can be used to estimate the energy of the noise (referred to herein as reference energy); for example, the noise estimation algorithm is a minimum tracking algorithm.

[0114] 2032. For any audio in the training audio set, obtain an audio waveform of the audio; perform waveform recognition on the audio waveform of the audio to obtain a waveform recognition result; in response to determining that the audio is truncation based on the waveform recognition result, delete the audio from the training audio set.

[0115] Audio is often represented as a waveform graph; for example, the horizontal axis is time and the vertical axis is amplitude. Alternatively, the horizontal axis can be the sampling rate. Truncation in audio has obvious characteristics on a waveform graph, so image recognition technology can be used to determine whether there is truncation in the audio. Specifically, the following methods can be used to determine whether a particular audio track has truncation:

[0116] 2032-1. In response to the waveform feature indicated by the waveform recognition result matching the first waveform feature, determine that the audio is truncation-prone.

[0117] The first waveform feature corresponds to the first audio, and the first audio has a truncation phenomenon at the audio start position. In other words, the truncation phenomenon at the audio start position is manifested in the waveform graph as a waveform consisting of sampling points with relatively high amplitudes instead of a silent segment at the beginning.

[0118] 2032-2. In response to the waveform feature indicated by the waveform recognition result matching the second waveform feature, it is determined that the audio is truncation-prone.

[0119] The second waveform feature corresponds to the second audio, and the second audio has a truncation phenomenon at the end of the audio. In other words, the truncation phenomenon at the end of the audio is manifested in the waveform as a waveform consisting of sampling points with relatively high amplitudes instead of a silent segment at the end.

[0120] 2032-3. In response to the waveform feature indicated by the waveform recognition result matching the third waveform feature, it is determined that the audio is truncation-prone.

[0121] The third waveform feature corresponds to the third audio frequency, which has a truncation phenomenon in the middle of the audio frequency. In other words, the truncation phenomenon in the middle of the audio frequency is manifested in a waveform consisting of a small silent segment followed by relatively high-amplitude sampling points.

[0122] 2033. For any audio in the training audio set, perform voiceprint recognition on the audio to obtain a voiceprint recognition result; in response to determining, based on the voiceprint recognition result, that the audio corresponds to multiple speakers, and the number of audio frames corresponding to at least two speakers is greater than a frame number threshold, perform speech recognition on the audio; in response to determining, based on the speech recognition result, that the audio contains chat speech, delete the audio from the training audio set.

[0123] The frame number threshold can be any set frame number, which is not limited in this application. In addition, the audio is subjected to voiceprint recognition using voiceprint recognition technology, and the audio is input into a speech recognition model to realize speech recognition of the audio.

[0124] 204. The computer device uses the filtered training audio set as training data for training an acoustic model.

[0125] Through the above processing method, high-quality training data that includes wake-up command words can be screened out, reducing the negative effects of low-quality audio on the acoustic model.

[0126] To sum up, after obtaining the training audio set, the embodiment of the present application will filter the audio in the training audio set to remove low-quality audio. This method of obtaining training data can obtain high-quality training data, and acoustic model training based on high-quality training data can not only ensure the accuracy of model training and improve the model training effect, but also perform speech recognition based on the trained acoustic model, which can greatly improve the accuracy of speech recognition.

[0127] In another possible implementation, in addition to Figure 2 In addition to the implementation shown, the following methods may be used to obtain training data.

[0128] Method 1: Cut the audio data collected in the target scene to obtain a training audio set; wherein, each audio in the training audio set includes a wake-up command word; determine the weights corresponding to multiple filtering methods according to the environment type of the target scene; for any audio in the training audio set, use multiple filtering methods to filter and judge the audio respectively, and obtain multiple filtering judgment results; in response to determining that the audio meets the filtering conditions according to the weights corresponding to the multiple filtering methods and the multiple filtering judgment results, delete the audio from the training audio set; use the filtered training audio set as training data for training the acoustic model. Taking the target scene as an intelligent elevator as an example, since the intelligent elevator is a public place, the order of the weights of the multiple filtering methods from large to small can be chat filtering → noise filtering → truncation filtering, and this application does not limit this.

[0129] Method 2: Cut the audio data collected in the target scenario to obtain a training audio set; wherein, each audio in the training audio set includes a wake-up command word; for any audio in the training audio set, use multiple filtering methods to filter and judge the audio separately to obtain multiple filtering judgment results; in response to determining that the audio meets the filtering condition based on any one of the multiple filtering judgment results, delete the audio from the training audio set; and use the filtered training audio set as training data for training the acoustic model.

[0130] Method three: cut the audio data collected in the target scene to obtain a training audio set; wherein, each audio in the training audio set includes a wake-up command word; according to the environment type of the target scene, determine the target filtering method that matches the environment type from multiple filtering methods; for any audio in the training audio set, use the target filtering method to filter and judge the audio; in response to determining that the audio meets the filtering conditions based on the filtering judgment result, delete the audio from the training audio set; and use the filtered training audio set as training data for training the acoustic model.

[0131] Figure 3 This is a structural diagram of a training data acquisition device provided in an embodiment of the present application. Figure 3 , the device comprises:

[0132] The first processing module 301 is configured to cut the audio data collected in the target scene to obtain a training audio set; wherein each audio in the training audio set includes a wake-up command word;

[0133] The second processing module 302 is configured to determine the execution order of multiple filtering methods according to the environment type of the target scene; wherein different filtering methods are used to filter the audio in the training audio set based on different filtering conditions;

[0134] The third processing module 303 is configured to, when filtering and judging each audio in accordance with the execution order, delete the audio from the training audio set in response to any audio satisfying the filtering condition corresponding to the current filtering method; and use the filtered training audio set as training data for training the acoustic model.

[0135] After obtaining the training audio set, the embodiment of the present application will filter the audio in the training audio set to remove low-quality audio. This training data acquisition method can obtain high-quality training data, and acoustic model training based on high-quality training data can not only ensure the accuracy of model training and improve the model training effect, but also perform speech recognition based on the trained acoustic model, which can greatly improve the accuracy of speech recognition.

[0136] In a possible implementation, the second processing module 302 is configured to:

[0137] Obtaining the location type and surrounding infrastructure information corresponding to the target scene;

[0138] Obtaining a collection period of the audio data;

[0139] The execution order of the multiple filtering methods is determined according to the type of place corresponding to the target scene, the surrounding infrastructure information, and the collection period of the audio data.

[0140] In a possible implementation, the second processing module 302 is configured to:

[0141] determining, based on the type of the venue, an initial execution order of the plurality of filtering methods;

[0142] The initial execution order is modified according to the surrounding infrastructure information and the collection period of the audio data to obtain a final execution order of the multiple filtering methods.

[0143] In a possible implementation, the third processing module 303 is configured to:

[0144] For any audio, obtain the audio waveform of the audio;

[0145] Performing waveform recognition on the audio waveform graph to obtain a waveform recognition result;

[0146] In response to determining that the audio is truncation-prone based on the waveform recognition result, deleting the audio from the training audio set;

[0147] The truncation phenomenon refers to the lack of some audio frames corresponding to the wake-up command word in the audio.

[0148] In a possible implementation, the third processing module 303 is configured to:

[0149] In response to the waveform feature indicated by the waveform recognition result matching the first waveform feature, determining that the audio is truncation-prone; wherein the first waveform feature corresponds to the first audio, and the audio starting position of the first audio is truncation-prone;

[0150] In response to the waveform feature indicated by the waveform recognition result matching the second waveform feature, determining that the audio is truncation; wherein the second waveform feature corresponds to the second audio, and the audio end position of the second audio is truncation;

[0151] In response to the waveform feature indicated by the waveform recognition result matching the third waveform feature, it is determined that the audio has a truncation phenomenon; wherein, the third waveform feature corresponds to a third audio, and the audio middle position of the third audio has a truncation phenomenon.

[0152] In a possible implementation, the third processing module 303 is configured to:

[0153] For any audio, perform voiceprint recognition on the audio to obtain a voiceprint recognition result;

[0154] In response to determining, based on the voiceprint recognition result, that the audio corresponds to multiple speakers and the number of audio frames corresponding to at least two speakers is greater than a frame number threshold, performing speech recognition on the audio;

[0155] In response to determining that chat speech exists in the audio based on the speech recognition result, the audio is deleted from the training audio set.

[0156] In a possible implementation, the third processing module 303 is configured to:

[0157] For any audio, obtaining the signal-to-noise ratio of the audio;

[0158] In response to a signal-to-noise ratio of the audio being less than a signal-to-noise ratio threshold, the audio is deleted from the training audio set.

[0159] In a possible implementation, the first processing module 301 is configured to:

[0160] Performing audio truncation on the audio data collected in the target scene to obtain an initial audio set; wherein each audio in the initial audio set includes a wake-up command word;

[0161] Performing voice endpoint detection on each audio in the initial audio set;

[0162] Based on the voice endpoint detection result, the silent segments of each audio in the initial audio set are eliminated to obtain the training audio set; wherein the duration of the silent segments is greater than the target duration.

[0163] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0164] It should be noted that the training data acquisition device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate the acquisition of training data. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the training data acquisition device provided in the above embodiment and the training data acquisition method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0165] Figure 4 4 is a schematic diagram of a computer device provided in an embodiment of the present application. For example, the computer 400 can be a training data acquisition device.

[0166] Typically, computer device 400 includes: a processor 401 and a memory 402. Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented in at least one hardware form: a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In one possible implementation, processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In one possible implementation, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computing operations related to machine learning.

[0167] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In one possible implementation, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one program code, which is executed by the processor 401 to implement the training data acquisition method provided in the method embodiment of the present application.

[0168] In one possible implementation, computer device 400 may optionally include a peripheral device interface 403 and at least one peripheral device. Processor 401, memory 402, and peripheral device interface 403 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 403 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 404, a display screen 405, a camera assembly 406, an audio circuit 407, and a power supply 408.

[0169] The peripheral device interface 403 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 401 and the memory 402. In one possible implementation, the processor 401, the memory 402, and the peripheral device interface 403 are integrated on the same chip or circuit board. In some other embodiments, any one or two of the processor 401, the memory 402, and the peripheral device interface 403 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0170] The radio frequency circuit 404 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 404 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency circuit 404 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 404 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In one possible implementation, the radio frequency circuit 404 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0171] Display screen 405 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, or any combination thereof. When display screen 405 is a touch screen display, it is also capable of collecting touch signals on or above the surface of display screen 405. This touch signal can be input as a control signal to processor 401 for processing. In this case, display screen 405 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In one possible implementation, there can be one display screen 405, disposed on the front panel of computer device 400. In another possible implementation, there can be at least two display screens 405, disposed on different surfaces of computer device 400 or in a foldable design. In another possible implementation, display screen 405 can be a flexible display, disposed on a curved or foldable surface of computer device 400. Display screen 405 can also be configured as a non-rectangular irregular shape, i.e., a special-shaped screen. Display screen 405 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0172] The camera assembly 406 is used to capture images or videos. Optionally, the camera assembly 406 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In one possible implementation, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In one possible implementation, the camera assembly 406 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0173] The audio circuit 407 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 401 for processing, or input into the radio frequency circuit 404 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the computer device 400. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 401 or the radio frequency circuit 404 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as distance measurement. In one possible implementation, the audio circuit 407 may also include a headphone jack.

[0174] Power supply 408 is used to power various components in computer device 400. Power supply 408 can be AC power, DC power, a disposable battery, or a rechargeable battery. When power supply 408 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be configured to support fast charging technology.

[0175] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the computer device 400, and the computer device 400 may include more or fewer components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0176] Figure 5 FIG2 is a schematic diagram of the structure of another computer device 500 provided in an embodiment of the present application. For example, the computer 500 may be a training data acquisition device.

[0177] The computer device 500 may vary significantly due to different configurations or performances, and may include one or more processors (Central Processing Units, CPUs) 501 and one or more memories 502, wherein the memory 502 stores at least one program code, which is loaded and executed by the processor 501 to implement the training data acquisition method provided by the above-mentioned various method embodiments. Of course, the computer device 500 may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The computer device 500 may also include other components for implementing device functions, which will not be described in detail here.

[0178] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the above-mentioned training data acquisition method.

[0179] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0180] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for obtaining training data, characterized in that: The method comprises: Cutting the audio data collected in the target scene to obtain a training audio set; wherein each audio in the training audio set includes a wake-up command word; Determining an execution order of multiple filtering methods according to the environment type of the target scene; wherein different filtering methods are used to filter the audio in the training audio set based on different filtering conditions; When filtering and determining each of the audios in the execution order, in response to any audio satisfying a filtering condition corresponding to the current filtering mode, deleting the audio from the training audio set; The filtered training audio set is used as training data for training the acoustic model.

2. The method according to claim 1, characterized in that The step of determining the execution order of the plurality of filtering methods according to the environment type of the target scene includes: Obtaining the location type and surrounding infrastructure information corresponding to the target scene; Obtaining a collection period of the audio data; The execution order of the multiple filtering methods is determined according to the type of place corresponding to the target scene, the surrounding infrastructure information, and the collection period of the audio data.

3. The method according to claim 2, characterized in that The determining of the execution order of the multiple filtering methods according to the type of venue corresponding to the target scene, the surrounding infrastructure information, and the collection period of the audio data includes: determining, based on the type of the venue, an initial execution order of the plurality of filtering methods; The initial execution order is modified according to the surrounding infrastructure information and the collection period of the audio data to obtain a final execution order of the multiple filtering methods.

4. The method according to claim 1, wherein In response to any audio satisfying a filtering condition corresponding to the current filtering mode, deleting the audio from the training audio set includes: For any audio, obtain the audio waveform of the audio; Performing waveform recognition on the audio waveform graph to obtain a waveform recognition result; In response to determining that the audio is truncation-prone based on the waveform recognition result, deleting the audio from the training audio set; The truncation phenomenon refers to the lack of some audio frames corresponding to the wake-up command word in the audio.

5. The method according to claim 4, characterized in that The method further comprises at least one of the following: In response to the waveform feature indicated by the waveform recognition result matching the first waveform feature, determining that the audio is truncation-prone; wherein the first waveform feature corresponds to the first audio, and the audio starting position of the first audio is truncation-prone; In response to the waveform feature indicated by the waveform recognition result matching the second waveform feature, determining that the audio is truncation; wherein the second waveform feature corresponds to the second audio, and the audio end position of the second audio is truncation; In response to the waveform feature indicated by the waveform recognition result matching the third waveform feature, it is determined that the audio has a truncation phenomenon; wherein, the third waveform feature corresponds to a third audio, and the audio middle position of the third audio has a truncation phenomenon.

6. The method according to claim 1, characterized in that In response to any audio satisfying a filtering condition corresponding to the current filtering mode, deleting the audio from the training audio set includes: For any audio, perform voiceprint recognition on the audio to obtain a voiceprint recognition result; In response to determining, based on the voiceprint recognition result, that the audio corresponds to multiple speakers and the number of audio frames corresponding to at least two speakers is greater than a frame number threshold, performing speech recognition on the audio; In response to determining that chat speech exists in the audio based on the speech recognition result, the audio is deleted from the training audio set.

7. The method according to claim 1, characterized in that In response to any audio satisfying a filtering condition corresponding to the current filtering mode, deleting the audio from the training audio set includes: For any audio, obtaining the signal-to-noise ratio of the audio; In response to a signal-to-noise ratio of the audio being less than a signal-to-noise ratio threshold, the audio is deleted from the training audio set.

8. The method according to any one of claims 1 to 7, characterized in that The audio data collected in the target scene is cut to obtain a training audio set, including: Performing audio truncation on the audio data collected in the target scene to obtain an initial audio set; wherein each audio in the initial audio set includes a wake-up command word; Performing voice endpoint detection on each audio in the initial audio set; Based on the voice endpoint detection result, the silent segments of each audio in the initial audio set are eliminated to obtain the training audio set; wherein the duration of the silent segments is greater than the target duration.

9. A training data acquisition device, characterized in that: The device comprises: A first processing module is configured to cut audio data collected in a target scene to obtain a training audio set; wherein each audio in the training audio set includes a wake-up command word; A second processing module is configured to determine an execution order of multiple filtering methods according to the environment type of the target scene; wherein different filtering methods are used to filter the audio in the training audio set based on different filtering conditions; The third processing module is configured to, when filtering and judging each of the audios in accordance with the execution order, delete the audio from the training audio set in response to any audio satisfying the filtering condition corresponding to the current filtering method; and use the filtered training audio set as training data for training the acoustic model.

10. A computer device, characterized in that: The device includes a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the training data acquisition method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the training data acquisition method according to any one of claims 1 to 8.

12. A computer program product or a computer program, characterized in that The computer program product or computer program includes computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the training data acquisition method according to any one of claims 1 to 8.

Images