Voice data processing method and device, electronic equipment, medium and vehicle
By using the target voice frame end prediction model in voice data processing, the end frame of non-voice frames is dynamically determined, which solves the problem of long wait time caused by fixed waiting frames in the prior art, and improves the user experience.
Patent Information
- Application Number
- CN202311169722.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-11
- Publication Date
- 2025-03-11
AI Technical Summary
In the prior art, since the number of preset fixed non-voice frames waiting for a large number of frames, the user waits for a long time for voice reply during human-computer interaction, which reduces the user experience.
When the starting non-speech frame in the sound data to be detected, the target frame number of consecutive non-speech frames is counted. When the target frame number is equal to the preset frame number, the probability value of the non-speech frame is obtained using the target speech frame end prediction model, and the target end frame of the sound data to be detected is determined based on the probability value.
By dynamically adjusting the waiting frame number of non-voice frames, the time for users to wait for voice reply is reduced and the user experience is improved.
Smart Images
Figure CN119601032A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of voice detection technology, and in particular to a voice data processing method, device, electronic device, medium and vehicle. Background Art
[0002] At present, human-computer interaction is mainly achieved by identifying user voice data, performing natural language understanding on the recognition results, and then making voice replies based on the understood results. In the process of identifying user voice data, in the prior art, voice activity detection (VAD) technology is first used to determine the starting voice frame and the ending voice frame of the user voice data, so as to obtain valid data in the user voice data, and then the valid data is recognized and processed to obtain the recognition result.
[0003] However, using the existing technology, in order to ensure the accuracy of detecting the end voice frame, a fixed non-voice frame waiting frame number, such as 50 frames, is pre-set. Specifically, when the first non-voice frame of the user's voice data is determined by VAD technology, the number of consecutive non-voice frames after the first non-voice frame is counted. When the number of non-voice frames is greater than or equal to the fixed non-voice frame waiting frame number, such as 50 frames, the user's voice data is considered to be finished, and subsequent natural language understanding and voice response operations are performed. Based on this, since the pre-set fixed non-voice frame waiting frame number, such as 50 frames, is relatively large, during the human-computer interaction process, the user has to wait for a longer time for a voice response, thereby reducing the user experience. Summary of the invention
[0004] In order to solve the above technical problems, the present disclosure provides a voice data processing method, device, electronic device, medium and vehicle.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for processing speech data, comprising: when a starting non-speech frame in the sound data to be detected is detected, counting a target number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected; wherein the starting non-speech frame is the starting non-speech frame after the speech frame in the sound data to be detected;
[0006] When the target number of frames is equal to the preset number of frames, the feature vector of each frame of the sound data to be detected is calculated through the target speech frame end prediction model to obtain the probability value of the non-speech frame corresponding to the target number of frames, wherein the preset number of frames is less than the initial non-speech frame waiting number of frames, the initial non-speech frame waiting number of frames is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target number of frames is the end frame of the sound data to be detected;
[0007] The target end frame of the sound data to be detected is determined according to the probability value of the non-speech frame.
[0008] Optionally, determining a target end frame of the sound data to be detected according to the probability value of the non-speech frame includes:
[0009] When it is determined that the probability value of the non-speech frame is greater than the preset probability value, the initial non-speech frame waiting frame number is adjusted to the target non-speech frame waiting frame number, and the non-speech frame corresponding to the target non-speech frame waiting frame number is determined as the target end frame, wherein the initial non-speech frame waiting frame number is used to determine the initial end frame of the sound data to be detected, and the target non-speech frame waiting frame number is less than the initial non-speech frame waiting frame number.
[0010] Optionally, when the target number of frames is equal to the preset number of frames, the target speech frame end prediction model is used to calculate the feature vector of each frame of the sound frame to be detected in the sound data to be detected, and before obtaining the probability value of the non-speech frame corresponding to the target number of frames, the method further includes:
[0011] When a starting speech frame in the sound data to be detected is detected, the sound data to be detected after the starting speech frame is input into the target speech frame end prediction model to obtain a probability value corresponding to each sound frame to be detected included in the sound data to be detected, wherein the sound frames to be detected include speech frames and non-speech frames.
[0012] Optionally, determining a target end frame of the sound data to be detected according to the probability value of the non-speech frame includes:
[0013] When it is determined that the probability value of the non-speech frame is less than or equal to the preset probability value, and the current preset frame number is less than the initial non-speech frame waiting frame number, the preset frame number and the preset value are summed, and the preset frame number is updated according to the summation result, wherein the preset value refers to the parameter value set for updating the preset frame number;
[0014] After determining to update the preset frame number according to the summation result, when the target frame number is equal to the updated preset frame number, based on the target speech frame end prediction model, the probability value of the non-speech frame corresponding to the target frame number is obtained, and the probability value of the non-speech frame is compared with the preset probability value to determine the target end frame of the sound data to be detected.
[0015] Optionally, the method further includes:
[0016] After determining the target end frame of the sound data to be detected, extracting the sound data to be detected between the start speech frame and the target end frame as target valid sound data;
[0017] Perform speech recognition processing on the target effective sound data to obtain a target speech result corresponding to the target effective sound data.
[0018] Optionally, before the prediction model is ended through the target speech frame, the feature vector of each frame of the sound frame to be detected in the sound data to be detected is calculated to obtain the probability value of the non-speech frame corresponding to the target number of frames, including:
[0019] Acquire a training set, wherein the training set includes a plurality of sound data samples and label data corresponding to each of the sound data samples, wherein the label data is used to indicate whether each sound frame included in each of the sound data samples is a non-end frame or an end frame;
[0020] The initial speech frame end prediction model is trained according to the training set to obtain the target speech frame end prediction model.
[0021] Optionally, the method further includes:
[0022] When, based on the probability value of the non-speech frame, before determining the target end frame of the sound data to be detected, if a speech frame is detected in the sound data to be detected, the target frame number is updated to 0, and the step of counting the target frame number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected when the starting non-speech frame in the sound data to be detected is returned to execute.
[0023] In a second aspect, an embodiment of the present disclosure provides a voice data processing device, including:
[0024] A target frame number counting module is used to count the target frame number of continuous non-speech frames after the starting non-speech frame in the sound data to be detected when the starting non-speech frame in the sound data to be detected is detected; wherein the starting non-speech frame is the starting non-speech frame after the speech frame in the sound data to be detected;
[0025] A probability value acquisition module, used for calculating the feature vector of each sound frame to be detected in the sound data to be detected by using a target speech frame end prediction model when the target frame number is equal to the preset frame number, and obtaining the probability value of the non-speech frame corresponding to the target frame number, wherein the preset frame number is less than the initial non-speech frame waiting frame number, the initial non-speech frame waiting frame number is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target frame number is the end frame of the sound data to be detected;
[0026] The target end frame determination module is used to determine the target end frame of the sound data to be detected according to the probability value of the non-speech frame.
[0027] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
[0028] one or more processors;
[0029] a storage device for storing one or more programs,
[0030] When the one or more programs are executed by the one or more processors, the one or more processors implement any method as described in the first aspect.
[0031] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any method described in the first aspect.
[0032] In a fifth aspect, an embodiment of the present disclosure provides a vehicle, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements any method described in the first aspect.
[0033] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:
[0034] The speech data processing method provided by the embodiment of the present disclosure comprises the following steps: when a starting non-speech frame in the sound data to be detected is detected, a target frame number of continuous non-speech frames after the starting non-speech frame in the sound data to be detected is counted; wherein the starting non-speech frame is a starting non-speech frame after a speech frame in the sound data to be detected; when the target frame number is equal to a preset frame number, a feature vector of each sound frame to be detected in the sound data to be detected is calculated through a target speech frame end prediction model to obtain a probability value of the non-speech frame corresponding to the target frame number, wherein the preset frame number is less than the initial non-speech frame waiting frame number, the initial non-speech frame waiting frame number is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target frame number is the end frame of the sound data to be detected; and according to the probability value of the non-speech frame, the target end frame of the sound data to be detected is determined. In the above process, when the target number of consecutive non-speech frames after the starting non-speech frame is equal to the preset number of frames, the probability value of the non-speech frame corresponding to the target number of frames is obtained. Since the probability value is used to indicate the probability that the non-speech frame corresponding to the target number of frames is the end frame of the sound data to be detected, the target end frame of the sound data to be detected can be determined by the probability value of the non-speech frame, thereby solving the problem in the prior art of a large number of waiting frames for the pre-set initial non-speech frame, shortening the time the user waits for a voice reply and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0037] Figure 1 It is a flowchart of a voice data processing method provided by an embodiment of the present disclosure;
[0038] Figure 2 is a flowchart of another voice data processing method provided by an embodiment of the present disclosure;
[0039] Figure 3 is a flowchart of another voice data processing method provided by an embodiment of the present disclosure;
[0040] Figure 4 is a flowchart of another voice data processing method provided by an embodiment of the present disclosure;
[0041] Figure 5 is a structural diagram of a voice data processing device provided by an embodiment of the present disclosure;
[0042] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0043] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0045] Figure 1 is a flow chart of a voice data processing method provided by an embodiment of the present disclosure, such as Figure 1 As shown, the voice data processing method provided by the embodiment of the present disclosure includes:
[0046] S11 . When a starting non-speech frame in the sound data to be detected is detected, a target number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected is counted.
[0047] Among them, the sound data to be detected refers to the sound data input by the user into a terminal device such as a smart phone during the human-computer interaction process. For example, the sound data to be detected is "How is the weather today", but it is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can make specific settings according to actual conditions.
[0048] The above-mentioned initial non-speech frame is the initial non-speech frame after the speech frame in the sound data to be detected.
[0049] Specifically, for the sound data to be detected, when the starting non-speech frame after the speech frame in the sound data to be detected is detected, the continuous non-speech frames after the starting non-speech frame in the sound data to be detected are statistically processed to obtain the target number of continuous non-speech frames after the starting non-speech frame.
[0050] Optionally, the above-mentioned counting of the target number of consecutive non-speech frames after the initial non-speech frame in the sound data to be detected can be achieved by setting a counter, but is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.
[0051] Optionally, based on the above embodiments, in some embodiments of the present disclosure, a target voice activity detection model can be used to detect whether each sound frame to be detected included in the sound data to be detected is a speech frame or a non-speech frame.
[0052] The target voice activity detection model is a detection model for detecting the starting voice frame and the starting non-voice frame of the sound data to be detected, and can determine the valid sound data in the sound data to be detected according to the detection results. The target voice activity detection model can be, for example, voice activity detection (Voice Activity Detection, VAD), but is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.
[0053] Specifically, when the sound data to be detected is input by the user, the sound data to be detected is input into the trained target voice activity detection model, and each frame of sound frame to be detected included in the sound data to be detected is detected by the target voice activity detection model to obtain the predicted probability value corresponding to each frame of sound frame to be detected, and the predicted probability value corresponding to each frame of sound frame to be detected is compared with the preset predicted probability value. When the predicted probability value is less than the preset predicted probability value, it is determined that the current sound frame to be detected is a non-speech frame. Correspondingly, when the predicted probability value is greater than or equal to the preset predicted probability value, it is determined that the current sound frame to be detected is a speech frame.
[0054] Optionally, based on the above embodiments, in some embodiments of the present disclosure, the target voice activity detection model is a neural network model based on deep learning, based on which the target voice activity detection model is obtained by training according to a sample training set, the sample training set includes multiple training sound data samples, and training label data corresponding to each training sound data sample, the training label data is used to indicate that each sound frame included in each training sound data sample is a speech frame, or a non-speech frame. Among them, the training label data can be obtained by a manual calibration method, but is not limited to this, the present disclosure is not specifically limited, and those skilled in the art can set it according to actual conditions.
[0055] Specifically, a sample training set corresponding to the initial voice activity detection model, including multiple training sound data samples and training label data corresponding to each training sound data sample, is obtained, the sample training set is input into the initial voice activity detection model, feature parameters corresponding to the multiple training sound data samples are extracted, and the training label data corresponding to each training sound data sample is used as supervision data for supervised training, and the parameter information in the initial voice activity detection model is continuously adjusted until the training function corresponding to the initial voice activity detection model converges, thereby obtaining a trained target voice activity detection model.
[0056] S12. When the target number of frames is equal to the preset number of frames, the prediction model is terminated through the target speech frame, and the feature vector of each sound frame to be detected in the sound data to be detected is calculated to obtain the probability value of the non-speech frame corresponding to the target number of frames.
[0057] The preset frame number is smaller than the initial non-voice frame waiting frame number.
[0058] Among them, the initial non-voice frame waiting frame number is used to determine the initial end frame of the sound data to be detected. The initial non-voice frame waiting frame number can be determined by users or manufacturers pre-setting different initial non-voice frame waiting frame numbers for different terminal devices, such as 50 frames. The preset frame number can be, for example, 5 frames or 10 frames, but is not limited to this. The present disclosure does not specifically limit it. The value of the preset frame number is determined by summarizing multiple historical experimental data.
[0059] The above-mentioned initial end frame is used to determine the effective sound data truncation point of the sound data to be detected, that is, when the starting non-speech frame of the sound data to be detected is detected, the number of consecutive non-speech frames after the starting non-speech frame is obtained. When the number of non-speech frames reaches 50 frames, the sound frame corresponding to the initial non-speech frame waiting frame number is determined to be the initial end frame, that is, for the sound data to be detected, the sound data between the starting speech frame and the initial end frame is extracted as the effective sound data.
[0060] The probability value is used to indicate the probability that the non-speech frame corresponding to the target frame number is the end frame of the sound data to be detected; the probability value ranges from 0 to 1, and the larger the probability value, the greater the possibility that the non-speech frame corresponding to the current target frame number is the end frame of the sound data to be detected.
[0061] Specifically, when the target number of consecutive non-speech frames after the starting non-speech frame is obtained, the target number of frames is compared with the preset number of frames. When the target number of frames is equal to the preset number of frames, the feature vector of each frame of the sound data to be detected is calculated through the target speech frame end prediction model to obtain the probability value indicating that the non-speech frame is the end frame of the sound data to be detected.
[0062] Exemplarily, following the above embodiment, the preset number of frames is set to 5 frames. When the starting non-speech frame is detected, the target number of consecutive non-speech frames after the starting non-speech frame is counted. When the target number of frames is equal to 5 frames, the probability value of the 5th non-speech frame is predicted to be, for example, 0.9 based on the trained target speech frame end prediction model, but is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.
[0063] Optionally, through the target speech frame end prediction model, it is possible to predict the probability value corresponding to each frame of the sound frame to be detected in the sound data to be detected after the starting speech frame, which is used to indicate that the sound frame to be detected is the end frame. The specific implementation process includes: extracting the multidimensional feature vector corresponding to the sound frame to be detected through the feature extraction layer in the target speech frame end prediction model, thereby using the multidimensional feature vector to obtain contextual semantic information, and further, using the fully connected layer in the target speech frame end prediction model, linearly transforming the multidimensional feature vectors of all the sound frames to be detected before the non-speech frame corresponding to the target frame number to obtain a corresponding two-dimensional feature vector, and using an activation function such as a softmax function to perform probability calculation on the two-dimensional feature vector, and the probability is used to represent the probability value that the non-speech frame is the end frame of the sound data to be detected. For the target speech frame end prediction model, for example, it can be a long short-term neural network (Long Short Term Memory, LSTM), but is not limited to this, the present disclosure is not specifically limited, and those skilled in the art can set it according to actual conditions.
[0064] Optionally, based on the above embodiment, in this embodiment, since the target speech frame end prediction model is a prediction model for predicting that each sound frame to be detected included in the sound data to be detected is an end frame, the target speech frame end prediction model is a neural network model based on deep learning, based on this, before executing S12, it also includes:
[0065] Get the training set.
[0066] The training set includes multiple sound data samples and label data corresponding to each sound data sample, and the label data is used to indicate whether each sound frame included in each sound data sample is a non-end frame or an end frame. The label data can be obtained by a manual calibration method, but is not limited to this. The present disclosure does not specifically limit this, and those skilled in the art can set it according to actual conditions.
[0067] The initial speech frame end prediction model is trained according to the training set to obtain a target speech frame end prediction model.
[0068] Specifically, a training set including multiple sound data samples and label data corresponding to each sound data sample corresponding to the initial speech frame end prediction model is obtained, the training set is input into the initial speech frame end prediction model, feature parameters corresponding to the multiple sound data samples are extracted, and the label data corresponding to each sound data sample is used as supervision data for supervised training, and the parameter information in the initial speech frame end prediction model is continuously adjusted until the training function corresponding to the initial speech frame end prediction model converges, thereby obtaining a trained target speech frame end prediction model.
[0069] Optional, Figure 2 is a flow chart of another voice data processing method provided by an embodiment of the present disclosure, which further includes, before executing S12:
[0070] S21. When a starting speech frame in the sound data to be detected is detected, the sound data to be detected after the starting speech frame is input into the target speech frame end prediction model to obtain a probability value corresponding to each sound frame to be detected included in the sound data to be detected.
[0071] The sound frames to be detected include speech frames and non-speech frames.
[0072] Specifically, for the sound data to be detected input by the user, when the starting speech frame in the sound data to be detected is detected, the sound data to be detected after the starting speech frame in the sound data to be detected is input into the target speech frame end prediction model to obtain the probability value of each sound frame to be detected after the starting speech frame.
[0073] In this way, the speech data processing method provided by the embodiment of the present disclosure can obtain the probability value of each sound frame to be detected after the starting speech frame, avoid invalid prediction of the sound frame to be detected before the starting speech frame in the sound data to be detected, and save resources.
[0074] S13. Determine the target end frame of the sound data to be detected according to the probability value of the non-speech frame.
[0075] Specifically, after the probability value of the non-speech frame corresponding to the target frame number is obtained, the target end frame of the sound data to be detected is determined according to the probability value of the non-speech frame.
[0076] Optionally, based on the above embodiment, in this embodiment, an implementation manner of S13 may be:
[0077] When it is determined that the probability value of the non-speech frame is greater than the preset probability value, the initial non-speech frame waiting frame number is adjusted to the target non-speech frame waiting frame number, and the non-speech frame corresponding to the target non-speech frame waiting frame number is determined as the target end frame.
[0078] The above-mentioned preset probability value is a parameter set to determine whether the non-speech frame corresponding to the target frame number is the end frame of the sound data to be detected. The preset probability value can be 0.8, for example, but is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.
[0079] The above-mentioned target non-voice frame waiting number of frames is less than the initial non-voice frame waiting number of frames. Exemplarily, following the above-mentioned embodiment, if the initial non-voice frame waiting number of frames is 50 frames, the target non-voice frame waiting number of frames can be, for example, 20 frames, but is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.
[0080] Specifically, after obtaining the probability value of the non-speech frame corresponding to the target frame number, the probability value of the non-speech frame corresponding to the target frame number is compared with the preset probability value. When it is determined that the probability value is greater than the preset probability value, the initial non-speech frame waiting frame number is adjusted to the target non-speech frame waiting frame number, and the non-speech frame corresponding to the target non-speech frame waiting frame number is determined as the non-speech frame as the target end frame.
[0081] Exemplarily, following the above embodiment, for the target non-speech frame waiting frame number is 20 frames, the 20th consecutive non-speech frame after the starting non-speech frame in the sound data to be detected is determined as the target end frame, but it is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.
[0082] In this way, the speech data processing method provided by the embodiment of the present disclosure counts the target frame number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected when the starting non-speech frame in the sound data to be detected is detected; wherein the starting non-speech frame is the starting non-speech frame after the speech frame in the sound data to be detected; when the target frame number is equal to the preset frame number, the feature vector of each frame of the sound frame to be detected in the sound data to be detected is calculated through the target speech frame end prediction model to obtain the probability value of the non-speech frame corresponding to the target frame number, wherein the preset frame number is less than the initial non-speech frame waiting frame number, the initial non-speech frame waiting frame number is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target frame number is the end frame of the sound data to be detected; according to the probability value of the non-speech frame, the target end frame of the sound data to be detected is determined. In the above process, when the target number of consecutive non-speech frames after the starting non-speech frame is equal to the preset number of frames, the probability value of the non-speech frame corresponding to the target number of frames is obtained. Since the probability value is used to indicate the probability that the non-speech frame corresponding to the target number of frames is the end frame of the sound data to be detected, the target end frame of the sound data to be detected can be determined by the probability value of the non-speech frame, thereby solving the problem in the prior art of a large number of waiting frames for the pre-set initial non-speech frame, shortening the time the user waits for a voice reply and improving the user experience.
[0083] Optionally, based on the above embodiment, in some embodiments of the present disclosure, an implementation manner of S13 may also be:
[0084] When it is determined that the probability value is less than or equal to the preset probability value, and the preset frame number is less than the initial non-speech frame waiting frame number, the preset frame number and the preset value are summed and the preset frame number is updated according to the summation result.
[0085] After determining to update the preset frame number based on the summation result, when the target frame number is equal to the updated preset frame number, based on the target speech frame end prediction model, obtain the probability value of the non-speech frame corresponding to the target frame number, compare the probability value with the preset probability value, and determine the target end frame of the sound data to be detected.
[0086] The preset value is used to update the current preset frame number. The preset value may be 5, for example, but is not limited thereto. The present disclosure does not specifically limit this, and those skilled in the art may set it according to actual conditions.
[0087] Specifically, when the target frame number is equal to the current preset frame number, since the probability value of the non-speech frame corresponding to the target frame number is less than or equal to the preset probability value at this time, the initial non-speech frame waiting frame number cannot be adjusted to the target non-speech frame waiting frame number, and the current preset frame number is less than the initial non-speech frame waiting frame number, it is possible to perform a summation process based on the preset value and the current preset frame number, and use the summation result to update the preset frame number. When the target frame number is once again equal to the updated preset frame number, the probability value of the non-speech frame corresponding to the target frame number is obtained, and the probability value is compared with the preset probability value to determine the target end frame of the sound data to be detected.
[0088] Exemplarily, following the above embodiment, for the initial non-speech frame waiting frame number such as 50 frames, the current preset frame number is 5 frames. When the target frame number corresponding to the continuous non-speech frame is equal to 5 frames, and the probability value corresponding to the 5th non-speech frame is 0.2, which is less than the preset probability value 0.8, it is characterized that the 5th non-speech frame is not the end frame of the sound data to be detected. Therefore, the initial non-speech frame waiting frame number of 50 frames cannot be adjusted to the target non-speech frame waiting frame number of 20 frames. At this time, the preset frame number 5 and the preset value 5 are summed, that is, the sum result 10 is obtained, and the preset frame number 5 is updated to 10 frames. In this way, when the target frame number of the non-speech frame is equal to the updated 10 frames, the probability value of the current target frame number, that is, the 10th non-speech frame, is obtained and compared with the preset probability value. However, it is not limited to this, and the present disclosure is not specifically limited. Those skilled in the art can set it according to actual conditions.
[0089] In this way, the voice data processing method provided by the embodiment of the present disclosure can adjust the initial non-voice frame waiting frame number to the target non-voice frame waiting frame number when the probability value of the non-voice frame corresponding to the target frame number is less than or equal to the preset probability value, and can continuously update the preset frame number when the current preset frame number is less than the initial non-voice frame waiting frame number. When the target frame number is equal to the preset frame number after the update, the probability value of the non-voice frame corresponding to the target frame number can still be obtained and compared with the preset probability value, so as to adjust the initial non-voice frame waiting frame number, thereby solving the problem in the prior art that the pre-set initial non-voice frame waiting frame number is large, shortening the time users wait for voice replies, and improving user experience.
[0090] Figure 3 is a flow chart of another voice data processing method provided by an embodiment of the present disclosure, Figure 3 is Figure 2 Based on the embodiment shown, Figure 3 As shown, the method also includes:
[0091] S31, after determining the target end frame of the sound data to be detected, extracting the sound data to be detected between the start speech frame and the target end frame as target valid sound data.
[0092] S32, performing speech recognition processing on the target effective sound data to obtain a target speech result corresponding to the target effective sound data.
[0093] Among them, the target voice result is used to determine the reply sentence of the sound data to be detected, and the user can also execute the instruction operation of the sound data to be detected, but it is not limited to this and the present disclosure does not specifically limit it.
[0094] Specifically, after determining the target end frame corresponding to the sound data to be detected, determine that the sound data to be detected between the start voice frame and the target end frame in the sound data to be detected is the target valid sound data. After obtaining the target sound data, perform speech recognition processing on the target sound data to obtain the target voice result corresponding to the target valid sound data. Use the target voice result to determine the reply sentence of the sound data to be detected, or execute the instruction operation of the sound data to be detected.
[0095] The above-mentioned speech recognition processing of the target valid sound data can refer to the existing technology, which will not be elaborated here.
[0096] In this way, the voice data processing method provided by the embodiment of the present disclosure determines the target end frame of the sound data to be detected by adjusting the target non-voice frame waiting frame number, thereby obtaining the target valid sound data according to the starting voice frame and the target end frame, and performing recognition processing, and then determining the reply sentence of the sound data to be detected, thereby shortening the time the user waits for the voice reply and improving the user experience.
[0097] Optionally, based on the above embodiments, in some embodiments of the present disclosure, the following further includes:
[0098] When, based on the probability value of the non-speech frame, before determining the target end frame of the sound data to be detected, if a speech frame is detected in the sound data to be detected, the target frame number is updated to 0, and the process returns to the step of counting the target frame number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected when the starting non-speech frame in the sound data to be detected is detected.
[0099] Specifically, based on the probability value of the non-speech frame, before determining the target end frame of the sound data to be detected, if a speech frame is detected in the sound data to be detected, it indicates that the user's voice data has not ended, so the target frame number is updated to 0, and when the starting non-speech frame in the sound data to be detected is detected again, the target frame number of continuous non-speech frames is re-counted.
[0100] In this way, the voice data processing method provided by the embodiment of the present disclosure can, based on the probability value of the non-voice frame, update the target frame number to 0 if a voice frame is detected in the sound data to be detected before determining the target end frame of the sound data to be detected. When the starting non-voice frame is detected again, the target frame number of continuous non-voice frames is re-counted, so as to avoid incomplete acquisition of the user's voice data.
[0101] Optionally, in combination with the above embodiment, Figure 4 FIG. 1 is a flow chart of another method for processing voice data provided by an embodiment of the present disclosure. Figure 4 As shown, the following steps are included:
[0102] S41, inputting the sound data to be detected into the target voice activity detection model;
[0103] S42, determining whether a starting non-speech frame is detected in the sound data to be detected.
[0104] S43, when it is determined that a starting non-speech frame is detected in the sound data to be detected, the sound data to be detected is input into a target speech frame end prediction model, and a target frame number of consecutive non-speech frames is counted.
[0105] S44, determining whether the target frame number is equal to the preset frame number.
[0106] S45, when it is determined that the target frame number is equal to the preset frame number, determine whether the probability value of the target frame number corresponding to the non-speech frame is greater than the preset probability value.
[0107] S46, when it is determined that the probability value of the target number of frames corresponding to the non-speech frame is greater than the preset probability value, adjusting the initial number of non-speech frame waiting frames to the target number of non-speech frame waiting frames.
[0108] S47, determining that the non-speech frame corresponding to the target non-speech frame waiting frame number is the target end frame.
[0109] Optionally, based on the above embodiment, when it is determined that the probability value of the target number of frames corresponding to the non-speech frame is less than or equal to the preset probability value, the method further includes:
[0110] S48, updating the preset frame number, and returning to execute S44 to S47.
[0111] In the embodiments of the present disclosure, the specific implementation of S41 to S48 may refer to the above embodiments, which will not be described in detail here.
[0112] Figure 5 is a structural diagram of a voice data processing device provided by an embodiment of the present disclosure, such as Figure 5 As shown, the speech data processing device includes: a target frame number counting module 11, a probability value obtaining module 12 and a target end frame determining module 13.
[0113] The target frame number counting module 11 is used to count the target frame number of continuous non-speech frames after the starting non-speech frame in the sound data to be detected when the starting non-speech frame in the sound data to be detected is detected; wherein the starting non-speech frame is the starting non-speech frame after the speech frame in the sound data to be detected;
[0114] A probability value acquisition module 12 is used to calculate the feature vector of each sound frame to be detected in the sound data to be detected by using a target speech frame end prediction model when the target frame number is equal to the preset frame number, and obtain the probability value of the non-speech frame corresponding to the target frame number, wherein the preset frame number is less than the initial non-speech frame waiting frame number, the initial non-speech frame waiting frame number is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target frame number is the end frame of the sound data to be detected;
[0115] The target end frame determination module 13 is used to determine the target end frame of the sound data to be detected according to the probability value of the non-speech frame.
[0116] The speech data processing device provided by the embodiment of the present disclosure includes a target frame number counting module, which is used to count the target frame number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected when the starting non-speech frame in the sound data to be detected is detected; wherein the starting non-speech frame is the starting non-speech frame after the speech frame in the sound data to be detected; a probability value acquisition module, which is used to calculate the feature vector of each frame of the sound frame to be detected in the sound data to be detected through the target speech frame end prediction model when the target frame number is equal to the preset frame number, and obtain the probability value of the non-speech frame corresponding to the target frame number, wherein the preset frame number is less than the initial non-speech frame waiting frame number, the initial non-speech frame waiting frame number is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target frame number is the end frame of the sound data to be detected; a target end frame determination module, which is used to determine the target end frame of the sound data to be detected according to the probability value of the non-speech frame. In the above process, when the target number of consecutive non-speech frames after the starting non-speech frame is equal to the preset number of frames, the probability value of the non-speech frame corresponding to the target number of frames is obtained. Since the probability value is used to indicate the probability that the non-speech frame corresponding to the target number of frames is the end frame of the sound data to be detected, the target end frame of the sound data to be detected can be determined by the probability value of the non-speech frame, thereby solving the problem in the prior art of a large number of waiting frames for the pre-set initial non-speech frame, shortening the time the user waits for a voice reply and improving the user experience.
[0117] Optionally, a target end frame determination module 13 is specifically used to adjust the initial non-speech frame waiting frame number to the target non-speech frame waiting frame number when it is determined that the probability value of the non-speech frame is greater than a preset probability value, and determine the non-speech frame corresponding to the target non-speech frame waiting frame number as the target end frame, wherein the initial non-speech frame waiting frame number is used to determine the initial end frame of the sound data to be detected, and the target non-speech frame waiting frame number is less than the initial non-speech frame waiting frame number.
[0118] The device also includes: a processing module, which is used to input the sound data to be detected after the starting speech frame into the target speech frame end prediction model when a starting speech frame in the sound data to be detected is detected, and obtain a probability value corresponding to each frame of the sound frame to be detected included in the sound data to be detected, wherein the sound frame to be detected includes a speech frame and a non-speech frame.
[0119] Optionally, the device further includes: an updating module, for summing the preset number of frames and a preset value when it is determined that the probability value of the non-speech frame is less than or equal to a preset probability value, and when the current preset number of frames is less than the initial non-speech frame waiting number of frames, and updating the preset number of frames according to the summation result, wherein the preset value refers to a parameter value set for updating the preset number of frames;
[0120] After determining to update the preset frame number according to the summation result, when the target frame number is equal to the updated preset frame number, based on the target speech frame end prediction model, the probability value of the non-speech frame corresponding to the target frame number is obtained, and the probability value of the non-speech frame is compared with the preset probability value to determine the target end frame of the sound data to be detected.
[0121] Optionally, the processing module is further used to extract the sound data to be detected between the starting speech frame and the target end frame as target valid sound data after determining the target end frame of the sound data to be detected;
[0122] Perform speech recognition processing on the target effective sound data to obtain a target speech result corresponding to the target effective sound data.
[0123] Optionally, the device further comprises: a training module, configured to obtain a training set, wherein the training set comprises a plurality of sound data samples and label data corresponding to each of the sound data samples, wherein the label data is used to indicate whether each sound frame included in each of the sound data samples is a non-end frame or an end frame;
[0124] The initial speech frame end prediction model is trained according to the training set to obtain the target speech frame end prediction model.
[0125] Optionally, the update module is also used to update the target frame number to 0 and return to execute the steps executed by the target frame number statistics module 11 when a speech frame is detected in the sound data to be detected before the target end frame of the sound data to be detected is determined based on the probability value of the non-speech frame.
[0126] The device provided by the embodiment of the present invention can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0127] It is worth noting that in the embodiment of the above-mentioned device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0128] Figure 6 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, such as Figure 6 As shown, the electronic device includes a processor 610, a memory 620, an input device 630, and an output device 640; the number of processors 610 in the computer device can be one or more. Figure 6 A processor 610 is taken as an example; the processor 610, the memory 620, the input device 630 and the output device 640 in the electronic device can be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.
[0129] The memory 620 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present invention. The processor 610 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 620, that is, implementing the method provided in the embodiment of the present invention.
[0130] The memory 620 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 620 may further include a memory remotely arranged relative to the processor 610, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0131] The input device 630 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, and may include a keyboard, a mouse, etc. The output device 640 may include a display device such as a display screen.
[0132] The present disclosure also provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, the method provided by the present disclosure includes:
[0133] When a starting non-speech frame in the sound data to be detected is detected, a target number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected is counted; wherein the starting non-speech frame is the starting non-speech frame after the speech frame in the sound data to be detected;
[0134] When the target number of frames is equal to the preset number of frames, the feature vector of each frame of the sound data to be detected is calculated through the target speech frame end prediction model to obtain the probability value of the non-speech frame corresponding to the target number of frames, wherein the preset number of frames is less than the initial non-speech frame waiting number of frames, the initial non-speech frame waiting number of frames is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target number of frames is the end frame of the sound data to be detected;
[0135] The target end frame of the sound data to be detected is determined according to the probability value of the non-speech frame.
[0136] Of course, the computer executable instructions of a storage medium including computer executable instructions provided by an embodiment of the present invention are not limited to the operations of the method described above, but can also execute related operations in the method provided by any embodiment of the present invention.
[0137] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FLASH), a hard disk or an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0138] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0139] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for processing speech data, characterized in that: include: When a starting non-speech frame in the sound data to be detected is detected, a target number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected is counted; wherein the starting non-speech frame is the starting non-speech frame after the speech frame in the sound data to be detected; When the target number of frames is equal to the preset number of frames, the feature vector of each frame of the sound data to be detected is calculated through the target speech frame end prediction model to obtain the probability value of the non-speech frame corresponding to the target number of frames, wherein the preset number of frames is less than the initial non-speech frame waiting number of frames, the initial non-speech frame waiting number of frames is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target number of frames is the end frame of the sound data to be detected; The target end frame of the sound data to be detected is determined according to the probability value of the non-speech frame.
2. The method according to claim 1, characterized in that The step of determining the target end frame of the sound data to be detected according to the probability value of the non-speech frame comprises: When it is determined that the probability value of the non-speech frame is greater than the preset probability value, the initial non-speech frame waiting frame number is adjusted to the target non-speech frame waiting frame number, and the non-speech frame corresponding to the target non-speech frame waiting frame number is determined as the target end frame, wherein the target non-speech frame waiting frame number is less than the initial non-speech frame waiting frame number.
3. The method according to claim 1, characterized in that When the target number of frames is equal to the preset number of frames, the target speech frame end prediction model is used to calculate the feature vector of each frame of the sound data to be detected, and before obtaining the probability value of the non-speech frame corresponding to the target number of frames, the method further includes: When a starting speech frame in the sound data to be detected is detected, the sound data to be detected after the starting speech frame is input into the target speech frame end prediction model to obtain a probability value corresponding to each sound frame to be detected included in the sound data to be detected, wherein the sound frames to be detected include speech frames and non-speech frames.
4. The method according to claim 1, characterized in that The step of determining the target end frame of the sound data to be detected according to the probability value of the non-speech frame comprises: When it is determined that the probability value of the non-speech frame is less than or equal to the preset probability value, and the current preset frame number is less than the initial non-speech frame waiting frame number, the preset frame number and the preset value are summed, and the preset frame number is updated according to the summation result, wherein the preset value refers to the parameter value set for updating the preset frame number; After determining to update the preset frame number according to the summation result, when the target frame number is equal to the updated preset frame number, based on the target speech frame end prediction model, the probability value of the non-speech frame corresponding to the target frame number is obtained, and the probability value of the non-speech frame is compared with the preset probability value to determine the target end frame of the sound data to be detected.
5. The method according to claim 2, characterized in that: The method further comprises: After determining the target end frame of the sound data to be detected, extracting the sound data to be detected between the start speech frame and the target end frame as target valid sound data; Perform speech recognition processing on the target effective sound data to obtain a target speech result corresponding to the target effective sound data.
6. The method according to claim 1, characterized in that Before the prediction model is ended through the target speech frame, the feature vector of each sound frame to be detected in the sound data to be detected is calculated to obtain the probability value of the non-speech frame corresponding to the target number of frames, including: Acquire a training set, wherein the training set includes a plurality of sound data samples and label data corresponding to each of the sound data samples, wherein the label data is used to indicate whether each sound frame included in each of the sound data samples is a non-end frame or an end frame; The initial speech frame end prediction model is trained according to the training set to obtain the target speech frame end prediction model.
7. The method according to claim 1, characterized in that The method further comprises: When, based on the probability value of the non-speech frame, before determining the target end frame of the sound data to be detected, if a speech frame is detected in the sound data to be detected, the target frame number is updated to 0, and the step of counting the target frame number of consecutive non-speech frames after the starting non-speech frame in the sound data to be detected when the starting non-speech frame in the sound data to be detected is returned to execute.
8. A voice data processing device, characterized in that: include: A target frame number counting module is used to count the target frame number of continuous non-speech frames after the starting non-speech frame in the sound data to be detected when the starting non-speech frame in the sound data to be detected is detected; wherein the starting non-speech frame is the starting non-speech frame after the speech frame in the sound data to be detected; A probability value acquisition module, used for calculating the feature vector of each sound frame to be detected in the sound data to be detected by using a target speech frame end prediction model when the target frame number is equal to the preset frame number, and obtaining the probability value of the non-speech frame corresponding to the target frame number, wherein the preset frame number is less than the initial non-speech frame waiting frame number, the initial non-speech frame waiting frame number is used to determine the initial end frame of the sound data to be detected, and the probability value is used to indicate the probability that the non-speech frame corresponding to the target frame number is the end frame of the sound data to be detected; The target end frame determination module is used to determine the target end frame of the sound data to be detected according to the probability value of the non-speech frame.
9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the voice data processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the speech data processing method according to any one of claims 1 to 7 is implemented.
11. A vehicle, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the speech data processing method according to any one of claims 1 to 7.