A method and device for voice endpoint detection based on frame classification

Abstract

The present application relates to a method and device for detecting speech endpoints based on frame classification, the method comprising: obtaining first speech data to be detected; performing frame processing on the first speech data to obtain multiple first speech data frames, each of which has the same time length; performing feature extraction on the multiple first speech data frames to obtain a first feature of each first speech data frame; inputting the first feature into a pre-trained classification model to obtain a first label corresponding to each first speech data frame; the classification model is used to classify the first speech data frame, and the first label is used to indicate whether the first speech data frame contains valid speech; and determining the speech endpoint of the first speech data based on the first speech data frame and the corresponding first label. Through the present application, the problem of inaccurate VAD detection in the prior art is solved, and the accuracy of VAD detection is improved.

Description

Technical Field

[0001] The present application relates to the field of speech detection, and in particular to a speech endpoint detection method and device based on frame classification. Background Art

[0002] The purpose of VAD (Voice Activity Detection) is to identify and remove silence from speech signals. As a pre-processing tool for ASR (Automatic Speech Recognition), VAD detects silence and only sends the speech signal to the ASR system for recognition when a human voice is detected, saving computing resources. Furthermore, VAD can segment long audio clips into multiple short clips, enabling sentence segmentation in the ASR output and facilitating the addition of punctuation in post-processing.

[0003] The difficulty with VAD lies in the fact that speech signals are often not pure human voices or silence. Microphones capture both human voices and ambient noise, significantly impacting VAD accuracy. Silence and pure noise are generally considered invalid speech, while human voices and noisy human voices are considered valid speech.

[0004] Existing threshold-based VAD achieves the purpose of distinguishing speech from non-speech by extracting time domain (short-term energy, short-term zero-crossing rate, etc.) or frequency domain (MFCC, spectral entropy, etc.) features and setting a reasonable threshold.

[0005] Existing threshold-based VAD methods can produce both false positives and false negatives. A false positive occurs when invalid speech is recognized as valid, resulting in wasted ASR computing resources. A false negative occurs when valid speech is recognized as invalid, leading to incomplete ASR results. VAD accuracy directly impacts ASR performance and accuracy.

[0006] With regard to the problem of inaccurate VAD detection in related technologies, no effective solution has been proposed so far. Summary of the Invention

[0007] In this embodiment, a voice endpoint detection method and apparatus based on frame classification are provided to solve the problem of inaccurate VAD detection in related technologies.

[0008] In a first aspect, a method for detecting speech endpoints based on frame classification is provided in this embodiment. The method includes:

[0009] Acquire first voice data to be detected;

[0010] Performing frame processing on the first voice data to obtain a plurality of first voice data frames, each of which has the same time length;

[0011] Performing feature extraction on the plurality of first speech data frames to obtain a first feature of each first speech data frame;

[0012] Inputting the first feature into a pre-trained classification model to obtain a first label corresponding to each first speech data frame; the classification model is used to classify the first speech data frame, and the first label is used to indicate whether the first speech data frame contains valid speech;

[0013] Determine a voice endpoint of the first voice data according to the first voice data frame and the corresponding first tag.

[0014] In some embodiments, determining the voice endpoint of the first voice data according to the first voice data frame and the corresponding first tag includes:

[0015] If M consecutive first voice data frames contain valid voice information, determining the first first voice data frame containing valid voice information as a voice start frame of the first voice data segment; wherein M is a natural number greater than or equal to 5;

[0016] If N consecutive first voice data frames do not contain valid voice information, determining the first first voice data frame that does not contain valid voice information as a voice end frame of the first voice data; wherein N is a natural number greater than or equal to 5;

[0017] The voice start frame and the voice end frame are voice endpoints of the first voice data.

[0018] In some embodiments, before obtaining the first speech data to be detected, the method includes constructing a second speech data frame for training the classification model;

[0019] Performing feature extraction on the second speech data frame to obtain a second feature;

[0020] Constructing the classification model, the classification model including two gated recurrent unit layers and three fully connected layers, the classification model being used to classify the second speech data frame;

[0021] The classification model is trained according to the second feature.

[0022] In some embodiments, the constructing of the second speech data frame for the classification model training includes:

[0023] Acquire a voice data set, wherein the voice data set includes a plurality of third voice data, and the plurality of third voice data includes valid voices and invalid voices;

[0024] Performing pure voice extraction on the third voice data to obtain a pure voice data set, wherein the pure voice data set includes a plurality of pure voice data;

[0025] Performing frame processing on the plurality of pure voice data to obtain a plurality of pure voice data frames corresponding to each pure voice data, wherein each of the pure voice data frames has the same time length;

[0026] The second voice data frame is constructed according to the multiple pure voice data frames.

[0027] In some embodiments, constructing the second voice data frame according to the plurality of pure voice data frames includes:

[0028] splicing the plurality of pure voice data, inserting K silence data frames between two adjacent pure voice data to obtain a second voice data frame; the second voice data frame is composed of the pure voice data frame and the silence data frame; wherein K is a natural number from 10 to 50;

[0029] The time length of the silence data frame is the same as the time length of the second voice data frame.

[0030] In some embodiments, after constructing the second voice data frame according to the plurality of pure voice data frames, the method further comprises:

[0031] The second speech data frame is labeled to obtain a second label, where the second label is used to verify the classification model.

[0032] In some embodiments, the extracting features from the second speech data frame to obtain the second features includes:

[0033] Feature extraction is performed on the current second voice data frame and the first L second voice data frames of the current second voice data frame, and the extracted features are used as the second features of the current second voice data frame; wherein L is a natural number greater than or equal to 5.

[0034] In some embodiments, the training of the classification model based on the second feature includes:

[0035] Before each iteration of training the classification model, noise is added to the second speech data frame.

[0036] In some embodiments, before each iteration of training the classification model, adding noise to the second speech data frame includes:

[0037] Before each iteration of training the classification model, the classification model generates a random number n, where n is a random number between 0 and 1; if the random number n is greater than a preset value, noise of -3db to 3db is added to the second speech data frame.

[0038] In a second aspect, a speech endpoint detection device based on frame classification is provided in this embodiment, characterized in that the device includes:

[0039] an acquiring unit, configured to acquire first voice data to be detected;

[0040] A framing processing unit, configured to perform framing processing on the first voice data to obtain a plurality of first voice data frames, each of which has the same time length;

[0041] a feature extraction unit, configured to perform feature extraction on the plurality of first speech data frames to obtain a first feature of each first speech data frame;

[0042] a frame classification unit, configured to input the first feature into a pre-trained classification model to obtain a first label corresponding to each first speech data frame; the classification model is configured to classify the first speech data frame, and the first label is configured to indicate whether the first speech data frame contains valid speech;

[0043] An endpoint determination unit is used to determine the voice endpoint of the first voice data according to the first voice data frame and the corresponding first label.

[0044] According to a third aspect, an electronic device is provided in this embodiment, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the frame classification-based speech endpoint detection method described in any one of the first aspects.

[0045] In a fourth aspect, a computer-readable storage medium is provided in this embodiment, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the frame classification-based speech endpoint detection method described in any one of the first aspects are implemented.

[0046] Compared with the related art, the present embodiment provides a method and device for voice endpoint detection based on frame classification, which obtains multiple first voice data frames by dividing the voice data into frames, performs feature extraction on the multiple first voice data frames, obtains the first feature of each first voice data frame, inputs the first feature into a pre-trained classification model, and obtains the first label corresponding to each first voice data frame. The first label is used to indicate whether the first voice data frame contains valid voice. According to the first voice data frame and the corresponding first label, the voice endpoint of the first voice data is determined, which solves the problem of inaccurate VAD detection in the prior art and improves the accuracy of VAD detection.

[0047] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0049] Figure 1 This is a hardware structure block diagram of a terminal that executes a voice endpoint detection method based on frame classification according to this embodiment;

[0050] Figure 2 is a flow chart of a voice endpoint detection method based on frame classification of this embodiment;

[0051] Figure 3 is a flow chart of a classification model construction and training method of this embodiment;

[0052] Figure 4 This is a flowchart of a method for constructing a second speech data frame for classification model training in this embodiment. DETAILED DESCRIPTION

[0053] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0054] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0055] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a block diagram of the hardware structure of a terminal that executes a method for detecting speech endpoints based on frame classification according to this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0056] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the voice endpoint detection method based on frame classification in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0057] The transmission device 106 is used to receive or send data via a network. The network may include a wireless network provided by the terminal's telecommunications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0058] In this embodiment, a voice endpoint detection method based on frame classification is provided. Figure 2 This is a flow chart of a voice endpoint detection method based on frame classification in this embodiment. Figure 2 As shown, the process includes the following steps:

[0059] Step S202: Acquire first voice data to be detected.

[0060] First voice data to be detected is obtained. This first voice data is the voice data for which a voice endpoint is to be detected. The first voice data includes valid voice and invalid voice, which are mixed together to form the first voice data. Generally, silence and pure noise are defined as invalid voice, while human voice and noisy human voice are defined as valid voice. The first voice data can be collected by a microphone, and the processor 102 obtains the first voice data through the microphone.

[0061] Step S204 , performing frame processing on the first voice data acquired in step S202 to obtain a plurality of first voice data frames, each of which has the same time length.

[0062] The first voice data obtained in step S202 is subjected to framing processing. The framing processing here can be to divide the first voice data into data frames of the same time length. The time length of the data frame can be 10ms, 20ms, 30ms, etc. Taking the time length of the data frame as 30ms as an example, the framing processing is to frame the first voice data with 30ms as a frame, that is, to divide the first voice data into multiple data frames with a time length of 30ms, and obtain multiple first voice data frames with a time length of 30ms. Before the framing processing, the first voice data can be processed. When the time length of the first voice data is not an integer multiple of the time length of the first voice data frame, the first voice data is padded. Silence data can be added at the end of the first voice data so that the time length of the padded first voice data is an integer multiple of the time length of the first voice data frame.

[0063] Step S206: performing feature extraction on the plurality of first speech data frames to obtain a first feature of each first speech data frame.

[0064] When extracting features, the previous N frames of speech of the current frame can be added as reference information. When there are less than N frames before the current frame, the silence signal of the corresponding number of frames is supplemented, and finally the first feature of the audio of the continuous N+1 frames of the first speech data frame is extracted. The first feature can be an Fbank (MelFilter bank, Mel filter bank) feature. Wherein, N is a natural number greater than or equal to 5. The Fbank feature is designed based on the auditory perception characteristics of the human ear, and mainly refers to the low-frequency characteristics of the sound. The first feature can also be other features of the first speech data frame, including but not limited to the frame energy of the data frame, the zero-crossing rate (the number of times each frame of data passes through the zero point), etc.

[0065] Step S208: input the first feature into a pre-trained classification model to obtain a first label corresponding to each first speech data frame; the classification model is used to classify the first speech data frame, and the first label is used to indicate whether the first speech data frame contains valid speech.

[0066] The first feature corresponding to the first speech data frame is input into a pre-trained classification model. The classification model outputs a first label corresponding to the first speech data frame. The classification model is used to classify the first speech data frame. The first label is used to indicate whether the first speech data frame contains valid speech. For example, if the first label is 1, it indicates that the first speech data frame contains valid speech. If the first label is 0, it indicates that the first speech data frame does not contain valid speech.

[0067] Step S210: Determine the voice endpoint of the first voice data according to the first voice data frame and the corresponding first tag.

[0068] If the first label of M consecutive first voice data frames is 1, it means that the M consecutive first voice data frames contain valid voice information, and the first first voice data frame containing valid voice information is determined to be the start frame of a voice segment of the first voice data. If the first label of N consecutive first voice data frames is 0, it means that the N consecutive first voice data frames do not contain valid voice information, and the first first voice data frame that does not contain valid voice information is determined to be the end frame of a voice segment of the first voice data; wherein M and N are natural numbers greater than or equal to 5. The voice start frame and the voice end frame are the voice endpoints of the first voice data. A voice start frame and its adjacent next voice end frame are the voice endpoints of a voice data segment. A first voice data segment may include multiple voice endpoints.

[0069] A voice endpoint detection method based on frame classification is provided in this embodiment. The method performs frame processing on the voice data to obtain multiple first voice data frames, performs feature extraction on the multiple first voice data frames to obtain a first feature of each first voice data frame, inputs the first feature into a pre-trained classification model to obtain a first label corresponding to each first voice data frame, and the first label is used to indicate whether the first voice data frame contains valid voice. The voice endpoint of the first voice data is determined based on the first voice data frame and the corresponding first label, which solves the problem of inaccurate VAD detection in the prior art and improves the accuracy of VAD detection.

[0070] This embodiment also provides a classification model construction and training method. Figure 3 This is a flow chart of a classification model construction and training method of this embodiment. Figure 3 As shown, the process includes the following steps:

[0071] Step S302: construct a second speech data frame for classification model training.

[0072] The second voice data frame includes a pure voice data frame and a silent data frame, wherein the pure voice data frame contains only pure voice, and the pure voice may be a pure human voice.

[0073] Step S304: extract features from the second speech data frame to obtain second features.

[0074] The second feature is used for parameter training of the classification model.

[0075] When extracting features, the first N frames of speech of the current frame can be added as reference information. When there are less than N frames before the current frame, the silence signal of the corresponding number of frames is supplemented, and finally the second feature of the audio of the continuous N+1 frames of the second speech data frame is extracted. The second feature can be an Fbank (MelFilter bank) feature. Wherein, N is a natural number greater than or equal to 5. The Fbank feature is designed based on the auditory perception characteristics of the human ear, and mainly refers to the low-frequency characteristics of the sound. The second feature can also be other features of the second speech data frame, including but not limited to the frame energy of the data frame, the zero-crossing rate (the number of times each frame of data passes through the zero point), etc.

[0076] Step S306: construct a classification model, which includes 2 gated recurrent unit layers and 3 fully connected layers. The classification model is used to classify the second speech data frame.

[0077] This classification model is a five-layer deep learning model, consisting of two gated recurrent unit layers and three fully connected layers. Because only valid and invalid speech are classified, a sigmoid function can be added to the last fully connected layer. Sigmoid is a nonlinear activation function.

[0078] Step S308: training the classification model according to the second feature.

[0079] Select BCELoss as the loss function, Adam as the optimization function, and a batch size of 1024. Train the network until the loss converges. BCELoss is a binary cross entropy loss function, Adam is a deep learning gradient optimization algorithm, batch size is the number of samples per iteration, and loss is the loss function.

[0080] A classification model construction and training method provided in this embodiment pre-trains the classification model, thereby improving the accuracy of the classification model frame classification.

[0081] This embodiment also provides a method for constructing a second speech data frame for classification model training. Figure 4 This is a flow chart of a method for constructing a second speech data frame for classification model training according to this embodiment. Figure 4 As shown, the process includes the following steps:

[0082] Step S402: Acquire a speech data set, where the speech data set includes a plurality of third speech data, and the plurality of third speech data includes valid speech and invalid speech.

[0083] Step S404: performing pure voice extraction on the third voice data to obtain a pure voice data set, where the pure voice data set includes a plurality of pure voice data.

[0084] You can process speech datasets using the Webrtcvad and sox tools to generate a large number of short audio clips containing only pure human voices. These short audio clips are known as pure speech datasets, which contain multiple pure speech data points. Webrtcvad is a web-based instant messaging voice activity detector, and Sox is an open-source audio processing tool.

[0085] Step S406 : performing frame processing on the plurality of pure voice data to obtain a plurality of pure voice data frames corresponding to each pure voice data, wherein each pure voice data frame has the same time length.

[0086] The framing process here can be to divide the pure voice data into 30ms frames, i.e., to divide the pure voice data into multiple data frames with a time length of 30ms, thereby obtaining multiple pure voice data frames with a time length of 30ms. Before the framing process, the pure voice data can be processed. If the time length of the pure voice data is not an integer multiple of the time length of the pure voice data frame, the pure voice data can be padded. Silence data can be added to the end of the pure voice data so that the time length of the padded pure voice data is an integer multiple of the time length of the pure voice data frame.

[0087] Step S408: construct a second voice data frame according to the multiple pure voice data frames.

[0088] Multiple pure voice data frames are concatenated, and K silence data frames are inserted between two adjacent pure voice data frames to obtain a second voice data frame. The second voice data frame is composed of a pure voice data frame and a silence data frame. The duration of the silence data frame is the same as the duration of the second voice data frame. K is a natural number between 10 and 50.

[0089] A method for constructing a second speech data frame for classification model training provided in this embodiment improves the accuracy of classification model frame classification by performing frame processing on the classification model training data.

[0090] In some embodiments, each second speech data frame is marked to obtain a second label, and the second label is used to verify the classification model. The second label of the pure speech data frame is marked as 1, and the second label of the silent data frame is marked as 0. The marked second speech data frames are shuffled by a shuffling algorithm, and 80% are extracted as a training set, 10% as a validation set, and 10% as a test set. The data in the training set is used to train the classification model to obtain corresponding model parameters. The validation set and the test set are used to verify the trained classification model. After feature extraction, the data in the validation set and the test set are input into the trained classification model, and the result output by the classification model is compared with the corresponding second label. If the two are the same, it means that the classification result of the classification model is correct. If the two are different, it is necessary to continue training the classification model.

[0091] In some embodiments, noise is added to the second speech data frame before each iteration of training the classification model. Specifically, before each iteration of training the classification model, the classification model generates a random number n, where n is a random number between 0 and 1; if the random number n is greater than a preset value, noise of -3db to 3db is added to the second speech data frame. When noise is added, the original label corresponding to each frame remains unchanged, that is, the label remains 0 after noise is added to silence, and the label remains 1 after noise is added to pure speech. Noise sources are collected from multiple scenes such as kitchens, roads, bedrooms, and restaurants. Adding noise training can effectively enhance the model's noise resistance and distinguish noise from valid speech.

[0092] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0093] In this embodiment, a speech endpoint detection device based on frame classification is also provided. The device is used to implement the above-mentioned embodiments and preferred embodiments. The details that have been described will not be repeated here. The terms "module," "unit," "subunit," etc. used below may refer to a combination of software and / or hardware that implements the predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0094] The device includes an acquisition unit, configured to acquire first voice data to be detected;

[0095] A framing processing unit, configured to perform framing processing on the first voice data to obtain a plurality of first voice data frames, each of which has the same time length;

[0096] a feature extraction unit, configured to perform feature extraction on the plurality of first speech data frames to obtain a first feature of each first speech data frame;

[0097] a frame classification unit, configured to input the first feature into a pre-trained classification model to obtain a first label corresponding to each first speech data frame; the classification model is configured to classify the first speech data frame, and the first label is configured to indicate whether the first speech data frame contains valid speech;

[0098] The endpoint determination unit is used to determine the voice endpoint of the first voice data according to the first voice data frame and the corresponding first label.

[0099] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0100] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0101] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0102] Optionally, in this embodiment, the processor may be configured to execute the steps of any one of the frame classification-based speech endpoint detection methods in the above embodiments through a computer program.

[0103] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0104] In addition, in conjunction with the frame classification-based voice endpoint detection method provided in the above embodiment, this embodiment may also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the frame classification-based voice endpoint detection methods in the above embodiment.

[0105] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0106] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0107] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0108] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A speech endpoint detection method based on frame classification, characterized in that: The method comprises, Constructing a second speech data frame for classification model training; Performing feature extraction on the second speech data frame to obtain a second feature; Constructing the classification model, the classification model including two gated recurrent unit layers and three fully connected layers, the classification model being used to classify the second speech data frame; Training the classification model according to the second feature includes: Before each iteration of training the classification model, adding noise to the second speech data frame includes: Before each iteration of training the classification model, the classification model generates a random number n, where n is a random number between 0 and 1; if the random number n is greater than a preset value, adding a noise of -3dB to 3dB to the second speech data frame; Acquire first voice data to be detected; Performing frame processing on the first voice data to obtain multiple first voice data frames, each of which has the same time length; before the frame processing, processing the first voice data, when the time length of the first voice data is not an integer multiple of the time length of the first voice data frame, padding the first voice data by adding silence data to the end of the first voice data, so that the time length of the first voice data after the padding is an integer multiple of the time length of the first voice data frame; Performing feature extraction on the plurality of first speech data frames to obtain a first feature of each first speech data frame, wherein the first feature is an Fbank feature; Inputting the first feature into the pre-trained classification model to obtain a first label corresponding to each first speech data frame; the classification model is used to classify the first speech data frame, and the first label is used to indicate whether the first speech data frame contains valid speech; Determine a voice endpoint of the first voice data according to the first voice data frame and the corresponding first tag.

2. The speech endpoint detection method based on frame classification according to claim 1, characterized in that determining a voice endpoint of the first voice data according to the first voice data frame and the corresponding first tag, include, If M consecutive first voice data frames contain valid voice information, determining the first first voice data frame containing valid voice information as a voice start frame of the first voice data segment; wherein M is a natural number greater than or equal to 5; If N consecutive first voice data frames do not contain valid voice information, determining the first first voice data frame that does not contain valid voice information as a voice end frame of the first voice data; wherein N is a natural number greater than or equal to 5; The voice start frame and the voice end frame are voice endpoints of the first voice data.

3. The speech endpoint detection method based on frame classification according to claim 1, characterized in that The constructing of the second speech data frame for the classification model training includes: Acquire a voice data set, wherein the voice data set includes a plurality of third voice data, and the plurality of third voice data includes valid voices and invalid voices; Performing pure voice extraction on the third voice data to obtain a pure voice data set, wherein the pure voice data set includes a plurality of pure voice data; Performing frame processing on the plurality of pure voice data to obtain a plurality of pure voice data frames corresponding to each pure voice data, wherein each of the pure voice data frames has the same time length; The second voice data frame is constructed according to the multiple pure voice data frames.

4. The method for speech endpoint detection based on frame classification according to claim 3, wherein: said constructing the second voice data frame according to said plurality of pure voice data frames, include, splicing the plurality of pure voice data, inserting K silence data frames between two adjacent pure voice data to obtain a second voice data frame; the second voice data frame is composed of the pure voice data frame and the silence data frame; wherein K is a natural number from 10 to 50; The time length of the silence data frame is the same as the time length of the second voice data frame.

5. The method for speech endpoint detection based on frame classification according to claim 4, characterized in that: After constructing the second voice data frame according to the plurality of pure voice data frames, the method includes: The second speech data frame is labeled to obtain a second label, where the second label is used to verify the classification model.

6. The method for speech endpoint detection based on frame classification according to claim 4, characterized in that: The step of extracting features from the second speech data frame to obtain a second feature includes: Feature extraction is performed on the current second voice data frame and the first L second voice data frames of the current second voice data frame, and the extracted features are used as the second features of the current second voice data frame; wherein L is a natural number greater than or equal to 5.

7. A speech endpoint detection device based on frame classification, characterized in that: The device comprises, an acquisition unit, configured to construct a second speech data frame for training a classification model; Performing feature extraction on the second speech data frame to obtain a second feature; Constructing the classification model, the classification model including two gated recurrent unit layers and three fully connected layers, the classification model being used to classify the second speech data frame; Training the classification model according to the second feature includes: Before each iteration of training the classification model, adding noise to the second speech data frame includes: Before each iteration of training the classification model, the classification model generates a random number n, where n is a random number between 0 and 1; if the random number n is greater than a preset value, adding a noise of -3dB to 3dB to the second speech data frame; Acquire first voice data to be detected; a framing processing unit, configured to perform framing processing on the first voice data to obtain a plurality of first voice data frames, each of which has the same time length; and, before the framing processing, to process the first voice data, and, when the time length of the first voice data is not an integer multiple of the time length of the first voice data frame, to pad the first voice data by adding silence data at the end of the first voice data, so that the time length of the supplemented first voice data is an integer multiple of the time length of the first voice data frame; A feature extraction unit, configured to perform feature extraction on the plurality of first speech data frames to obtain a first feature of each first speech data frame; the first feature being an Fbank feature; a frame classification unit, configured to input the first feature into the pre-trained classification model to obtain a first label corresponding to each first speech data frame; the classification model is configured to classify the first speech data frame, and the first label is configured to indicate whether the first speech data frame contains valid speech; An endpoint determination unit is used to determine the voice endpoint of the first voice data according to the first voice data frame and the corresponding first label.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the voice endpoint detection method based on frame classification according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the voice endpoint detection method based on frame classification according to any one of claims 1 to 6 are implemented.

Images