Speech processing method, model training method and device for speech processing
The feature vectors of audio frames are extracted through the convolutional neural network model, which solves the problem of difficulty in determining the speakers to which the audio belongs in the prior art, realizes more accurate and stable speech recognition, improves the security of identity verification and customized services. supply.
Patent Information
- Application Number
- CN202011400222.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-02
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-12-02
AI Technical Summary
Existing voice technologies are difficult to determine the speaker to which the audio belongs, affecting the security and the provision of customized services.
The Singer function convolution layer and residual network layer in the convolutional neural network model are used to extract the feature vectors of the audio frame, determine the feature vectors of the target speech by matching the reference speech, and confirm the object to which the speech belongs.
It improves the accuracy and stability of voice recognition, enhances the security of identity verification, and provides customized services to different users.
Smart Images

Figure CN114582331B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of speech recognition technology, and in particular to a speech processing method, a model training method, a device and an electronic device for speech processing. Background Art
[0002] In recent years, artificial intelligence technology has become popular in daily life, and voice-related technology has become one of the important directions of development. Currently, most existing voice technology applications focus on converting speech to text (Speech To Text, STT).
[0003] But on top of that, based on security considerations and the need to provide customized services to different users, how to determine the speaker to whom the audio belongs has become a technical issue that needs to be solved urgently. Summary of the invention
[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.
[0005] The present application proposes a speech processing method to confirm whether a target speech and a reference speech belong to the same speaking object based on a feature vector of the target speech, thereby achieving the purpose of confirming the object to which the speech belongs based on recognizing the speech of different users.
[0006] The first embodiment of the present application provides a speech processing method, including:
[0007] Get the target voice;
[0008] Extracting a plurality of audio frames from the target speech;
[0009] Performing low-dimensional feature extraction on each of the multiple audio frames using a Singe function convolution layer in a convolutional neural network model to obtain an intermediate feature vector of each of the audio frames;
[0010] Inputting the intermediate feature vector of each of the audio frames into the residual network layer in the convolutional neural network model to perform high-dimensional feature extraction to obtain a feature vector of each of the audio frames;
[0011] Determining a feature vector of the target speech according to the feature vectors of the multiple audio frames;
[0012] According to the feature vector of the target speech, a matching reference speech is determined, so that the object to which the reference speech belongs is determined as the object of the target speech.
[0013] The speech processing method of the embodiment of the present application extracts multiple audio frames from the target speech after acquiring the target speech. In this way, when the feature vector of the target speech is subsequently determined based on the feature vector of a set number of audio frames, the fluctuation of the feature expression performance of the feature vector of the target speech due to the fluctuation of the number of audio frames is avoided, so that the feature vector of the target speech can stably express the characteristics of the target speech. Therefore, the accuracy of the recognition result of confirming whether the target speech and the reference speech belong to the same speaking object is more stable, thereby realizing the confirmation of the object according to the recognition of the speech of different users, which can not only provide customized services for different users, but also improve the security of identity authentication.
[0014] The second aspect of the present application provides a model training method for speech processing, including:
[0015] Obtain training audio of multiple known objects;
[0016] Extracting at least one training frame from the training audio of each known object, and labeling the corresponding known object for the extracted multiple training frames;
[0017] Using the labeled multiple training frames to train the convolutional neural network;
[0018] The convolutional neural network model includes a sinc function convolution layer for performing low-dimensional feature extraction on an input training frame to obtain an intermediate feature vector, and a residual network layer for performing high-dimensional feature extraction on the intermediate feature vector to output a feature vector.
[0019] The model training method for speech processing of the embodiment of the present application obtains training audios of multiple known objects, extracts at least one training frame from the training audio of each known object, and labels the corresponding known objects for the extracted multiple training frames. Then, the convolutional neural network is trained using the labeled multiple training frames, thereby improving the accuracy of the feature vector of the extracted audio frame of the convolutional neural network model.
[0020] The third aspect of the present application provides a speech processing device, including:
[0021] An acquisition module, used to acquire target speech;
[0022] An audio frame extraction module, used to extract multiple audio frames from the target speech;
[0023] A feature extraction module, configured to perform low-dimensional feature extraction on each of the multiple audio frames using a Singer function convolution layer in a convolutional neural network model to obtain an intermediate feature vector of each of the audio frames;
[0024] A processing module, used for inputting the intermediate feature vector of each audio frame into the residual network layer in the convolutional neural network model to perform high-dimensional feature extraction to obtain a feature vector of each audio frame;
[0025] A first determining module, configured to determine a feature vector of the target speech according to the feature vectors of the plurality of audio frames;
[0026] The second determination module is used to determine a matching reference speech according to the feature vector of the target speech, so as to determine the object to which the reference speech belongs as the object of the target speech.
[0027] The fourth aspect of the present application provides a model training device for speech processing, including:
[0028] An acquisition module, used for acquiring training audios of multiple known objects;
[0029] A labeling module, used to extract at least one training frame from the training audio of each known object, and label the corresponding known object for the extracted multiple training frames;
[0030] A training module, used to train a convolutional neural network using the labeled multiple training frames;
[0031] The convolutional neural network model includes a sinc function convolution layer for performing low-dimensional feature extraction on an input training frame to obtain an intermediate feature vector, and a residual network layer for performing high-dimensional feature extraction on the intermediate feature vector to output a feature vector.
[0032] The fifth aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the speech processing method of the first aspect, or implements the model training for speech processing as described in the second aspect.
[0033] The sixth aspect embodiment of the present application proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the speech processing method of the first aspect embodiment, or implements the model training for speech processing as described in the second aspect embodiment.
[0034] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0036] Figure 1 A flowchart of a speech processing method provided in an embodiment of the present application;
[0037] Figure 2 A schematic diagram of a sub-process for determining an object of a target speech provided in an embodiment of the present application;
[0038] Figure 3 An example diagram of a speech processing method provided in an embodiment of the present application;
[0039] Figure 4 A flowchart of a model training method for speech processing provided in an embodiment of the present application;
[0040] Figure 5 A schematic diagram of the structure of a speech processing device provided in an embodiment of the present application;
[0041] Figure 6 A schematic diagram of the structure of a model training device for speech processing provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0043] The following describes the speech processing method, model training method, device, electronic device and storage medium for speech processing in embodiments of the present application with reference to the accompanying drawings.
[0044] Figure 1 A flowchart of a speech processing method provided in an embodiment of the present application.
[0045] like Figure 1 As shown, the speech processing method comprises the following steps:
[0046] Step 101, obtaining target speech.
[0047] The target speech refers to the speech information sent by the object to be confirmed. The object to which the target speech belongs in this application can be the same or multiple. For example, speech containing three speaking objects can be obtained. The number of objects to which the target speech belongs is not limited in this application.
[0048] It is understandable that in order to confirm the object to which the target speech belongs, it is first necessary to obtain the target speech, that is, the speech emitted by the object to which the target speech belongs. The target speech can come from the original audio data or the original video data; and the duration of the target speech can vary based on needs. This application does not limit the source and length of the target speech. For example: the target speech can be an audio of about 5 seconds.
[0049] Step 102: extract multiple audio frames from the target speech.
[0050] The number of audio frames extracted from the target speech may be preset.
[0051] In the embodiment of the present application, in order to maintain the consistency of the speech processing speed, multiple audio frames with the same number of frames can be extracted from the target speech.
[0052] It should be explained that audio data is streamed, and there is no clear concept of frames. In practical applications, for the convenience of audio algorithm processing / transmission, a frame of audio is generally taken as a data volume of 2.5 milliseconds to 60 milliseconds. There is no special standard for the length of an audio frame, which is determined according to the requirements of the codec and the specific application. In this application, each speech part obtained after the target speech is segmented according to a fixed time length is called an audio frame. For example, the fixed length of each frame played by MP3 can be 26ms.
[0053] As a possible implementation method, the frame interval corresponding to the target speech can be first determined according to the length of the target speech, and further, a set number of audio frames can be extracted from the target speech according to the frame interval corresponding to the target speech. The frame interval refers to the number of frames between the audio frames extracted from the target speech. Thus, the extracted audio frames are set in number through different frame intervals, so that when the feature vector of the target speech is subsequently determined based on the feature vector of the set number of audio frames, the fluctuation of the feature expression performance of the feature vector of the target speech due to the fluctuation of the number of audio frames is avoided, so that the feature vector of the target speech can stably express the characteristics of the target speech.
[0054] In the specific implementation process, the segment lengths corresponding to the multiple speech lengths can be determined according to the multiple speech lengths and the set number. For each speech length, the difference between the corresponding segment length and the set audio frame length is used as the frame interval corresponding to each speech length. Then, according to the frame intervals corresponding to the multiple speech lengths, a mapping relationship between the speech length and the frame interval is established.
[0055] The multiple speech lengths here can be the speech lengths corresponding to multiple speech of the same object, or can be the speech lengths corresponding to multiple speech of different corresponding objects. In this application, there is no limitation on the number of objects to which the speech corresponding to the multiple speech lengths belongs.
[0056] The segment length refers to the length corresponding to the speech being divided into a set number of segments according to the speech length of the speech. For example, assuming that the speech length is 10 milliseconds and the set number is 2, the segment length corresponding to the speech length is 5 milliseconds.
[0057] Assuming that the audio frame length is set to 50 milliseconds, the speech length is 5 seconds, the number is set to 5, and the corresponding segment length is 1 second, the frame interval corresponding to the speech length determined by the difference between the corresponding segment length and the set audio frame length is 950 milliseconds, thereby determining the mapping relationship between the speech length and the frame interval.
[0058] It should be explained that the above-mentioned mapping relationship between the speech length and the frame interval is only described as an example.
[0059] Further, after determining the mapping relationship between the speech length and the frame interval, the mapping relationship between the speech length and the frame interval can be queried according to the speech length of the target speech to determine the frame interval corresponding to the speech length of the target speech.
[0060] For example, assuming that the mapping relationship between speech length and frame interval is: the speech length is 5 seconds, the corresponding frame interval is 950 milliseconds, etc. When it is determined that the speech length of the target speech is 5 seconds, by querying the mapping relationship between the speech length and the frame interval, it can be determined that the frame interval between multiple audio frames extracted from the target speech is 950 milliseconds.
[0061] As another possible implementation, the length and number of audio frames may be preset, and then the frame interval between multiple audio frames extracted from the target speech may be calculated according to the speech length of the target speech.
[0062] Step 103: Perform low-dimensional feature extraction on each audio frame in the multiple audio frames using the Single function convolution layer in the convolutional neural network model to obtain an intermediate feature vector of each audio frame.
[0063] Step 104: Input the intermediate feature vector of each audio frame into the residual network layer in the convolutional neural network model to perform high-dimensional feature extraction to obtain the feature vector of each audio frame.
[0064] Among them, Convolutional Neural Network (CNN) is a feedforward neural network whose artificial neurons can respond to surrounding units within a part of the coverage area.
[0065] It can be understood that the Sinc function convolution layer (SincNet Filters) is an ideal electronic filter that removes all signal components above a given bandwidth and retains only low-frequency signals. In the frequency domain, it is shaped like a rectangular function, and in the time domain, it is shaped like a Sinc function.
[0066] Since the residual network layer (ResNet) has a deep network structure, when the residual network layer is used in the present application to perform high-dimensional feature extraction on the intermediate feature vector of each audio frame, more information of different levels is extracted, and more combinations of hierarchical information between different levels are also found, thereby avoiding the problem of feature loss.
[0067] In the embodiment of the present application, after extracting multiple audio frames from the target speech, the Sinc function convolution layer in the CNN model can be used to perform low-dimensional feature extraction on each of the multiple audio frames to obtain an intermediate feature vector of each audio frame. Furthermore, the intermediate feature vector of each audio frame is input into the ResNet network layer in the CNN model for high-dimensional feature extraction to obtain a feature vector of each audio frame.
[0068] It should be noted that the dimension of the intermediate feature vector input to the residual network layer in this application matches the dimension of the intermediate feature vector output by the sinc function convolution layer.
[0069] Among them, low-dimensional feature extraction can be performed on the audio frame based on the Sinc function convolution layer to obtain the intermediate feature vector of the audio frame. Furthermore, the ResNet network layer is used to perform high-dimensional feature extraction on the intermediate feature vector to obtain the feature values of the feature vector of the audio frame in multiple vector dimensions. Therefore, the convolution operation based on the Sinc convolution layer and the ResNet network can be used as a new audio code to determine the feature vector of the target speech.
[0070] The original SincNet architecture is a layer of filtering plus a two-layer CNN. Although increasing the number of layers can effectively enhance the feature extraction ability of the model. However, when there are too many layers, it is easy to cause feature loss. Therefore, in this application, the ResNet network layer is used to extract the intermediate feature vector of the audio frame to effectively avoid the problem of feature loss.
[0071] Step 105: Determine a feature vector of the target speech according to the feature vectors of the multiple audio frames.
[0072] In the present application, the feature vector may include some or all of the features in multiple dimensions, such as autocorrelation coefficient, subband energy (0-4kHz) feature, zero-crossing rate, Linear Prediction Cepstrum Coefficient (LPCC) feature and Mel Frequency Cepstrum Coefficient (MFCC).
[0073] As a possible implementation method, multiple audio frames extracted from the target speech can be respectively input into a trained convolutional neural network to extract the feature vector of each audio frame, and then the eigenvalues of the feature vector of each audio frame in multiple vector dimensions (d-vectors) can be determined. Then, according to the eigenvalues of the feature vector of each audio frame in multiple vector dimensions, the mean eigenvalues of each audio frame in each vector dimension can be determined, and then the mean eigenvalues in each vector dimension can be used as the feature vector of the target speech.
[0074] As a possible scenario of the present application, after determining the feature vector of the target speech, the feature vector of the target speech can be subjected to dimensionality reduction processing to generate a feature vector of the target speech with a smaller dimension, thereby not only reducing the amount of calculation and improving the rate of speech processing, but also reducing noise.
[0075] Exemplarily, a principal component analysis (PCA) method may be used to reduce the dimension of the feature vector of the target speech to obtain a feature vector of the target speech with a smaller dimension.
[0076] Exemplarily, a linear discriminant analysis model (Latent Dirichlet Allocation, LDA for short) may also be used to reduce the dimension of the feature vector of the target speech to obtain a feature vector of the target speech with a smaller dimension.
[0077] It should be noted that the above-mentioned method of reducing the dimension of the feature vector of the target speech is only described as an example, and other achievable dimensionality reduction methods are also applicable to the present application and will not be introduced one by one here.
[0078] Step 106: determine a matching reference speech according to the feature vector of the target speech, so as to determine the object to which the reference speech belongs as the object of the target speech.
[0079] The reference voice may be a voice of a known object, for example, a voice of a registered user.
[0080] In an embodiment of the present application, after determining the feature vector of the target speech, the difference between the feature vector of the target speech and the feature vector of the reference speech can be compared, and after determining the reference speech that matches the target speech, the object to which the reference speech belongs can be determined as the object of the target speech.
[0081] The method for extracting the feature vector of the reference speech may refer to the method for extracting the feature vector of the target speech in the above embodiment, which will not be described in detail here.
[0082] As a possible implementation method, the feature vector of the target speech and the feature vector of the reference speech can be input into a trained discriminant analysis model to determine whether the target speech and the reference speech belong to the same speaker based on the output of the model.
[0083] For example, the feature vector of the target speech and the feature vector of the reference speech can be input into a probabilistic linear discriminant analysis model (PLDA) to determine whether the target speech and the reference speech belong to the same speaker based on the probability that the target speech and the reference speech belong to the same speaker output by the PLDA model.
[0084] The speech processing method of the embodiment of the present application extracts multiple audio frames from the target speech after acquiring the target speech. In this way, when the feature vector of the target speech is subsequently determined based on the feature vector of a set number of audio frames, the fluctuation of the feature expression performance of the feature vector of the target speech due to the fluctuation of the number of audio frames is avoided, so that the feature vector of the target speech can stably express the characteristics of the target speech. Therefore, the accuracy of the recognition result of confirming whether the target speech and the reference speech belong to the same speaking object is more stable, thereby realizing the confirmation of the object according to the recognition of the speech of different users, which can not only provide customized services for different users, but also improve the security of identity authentication.
[0085] Based on the above embodiment, it has been mentioned in the above step 106 that a reference speech matching the target speech is determined according to the feature vector of the target speech and the feature vector of the reference speech to determine the object of the target speech. Figure 2 Provide detailed introduction, Figure 2 A schematic diagram of a sub-process for determining an object of a target speech provided in an embodiment of the present application.
[0086] like Figure 2 As shown, the above step 106 may further include the following sub-steps:
[0087] Step 201: Obtain a feature vector of at least one reference speech.
[0088] The reference voice may be the voice of a known object. For example, the voice of a registered user in different environments may be collected as the reference voice.
[0089] It should be noted that the length of the voice collected from the registered user can be 5 seconds, 10 seconds, etc., and is not limited here.
[0090] In the embodiment of the present application, the collected voices of each registered user can be determined as reference voices, and the frame interval corresponding to each reference voice can be determined according to the length of each reference voice, so as to extract a set number of reference frames from each reference voice according to the frame interval corresponding to each reference voice. Further, the feature vector of the corresponding reference voice is determined according to the feature vectors of the multiple reference frames.
[0091] When determining the feature vector of the reference frame, each audio frame in the multiple reference frames can be subjected to low-dimensional feature extraction using the Singer network convolution layer in the convolutional neural network to obtain an intermediate feature vector of each reference frame. Furthermore, the intermediate feature vector of each reference frame is input into the residual network layer in the convolutional neural network model to perform high-dimensional feature extraction to obtain a feature vector of each reference frame.
[0092] When determining the feature vector of the corresponding reference speech based on the feature vectors of multiple reference frames, the mean feature value of each reference frame on each vector dimension can be determined based on the feature values of the feature vector of each reference frame on multiple vector dimensions, so that the mean feature value on each vector dimension can be used as the feature vector of the reference speech.
[0093] In the specific implementation process, when determining the frame interval corresponding to each reference speech according to the length of each reference speech, the frame interval corresponding to the speech length of the reference speech can be determined by querying the mapping relationship between the speech length and the frame interval. The process of determining the mapping relationship between the speech length and the frame interval can refer to the implementation process in the above embodiment, which will not be repeated here.
[0094] Step 202: Use a dimensionality reduction model to reduce the dimension of each feature vector of the reference speech and the feature vector of the target speech.
[0095] In the embodiment of the present application, after obtaining the feature vector of at least one reference speech, a dimensionality reduction model can be used to reduce the dimension of the feature vector of each reference speech and the feature vector of the target speech. Thus, by performing dimensionality reduction processing on the feature vector of the reference speech and the feature vector of the target speech to generate a feature vector of smaller dimension, not only the amount of calculation is reduced, the rate of speech processing is increased, but also noise can be reduced.
[0096] Exemplarily, the PCA method may be used to reduce the dimension of the feature vectors of the reference speech and the target speech to obtain a feature vector of the reference speech with a smaller dimension.
[0097] Among them, the dimension reduction principle of the PCA method is to merge high-dimensional feature vectors into low-dimensional feature attributes, which is an unsupervised dimension reduction method. The algorithm goal is to map high-dimensional data to a low-dimensional space through a certain linear projection, and it is expected that the variance of the data in the projected dimension is the largest, so as to use fewer data dimensions while retaining more characteristics of the original data points.
[0098] Exemplarily, LDA may be used to reduce the dimension of the feature vectors of the reference speech and the target speech to obtain a feature vector of the reference speech with a smaller dimension.
[0099] Among them, LDA is an operation based on the classification model to merge feature attributes, and is a supervised dimensionality reduction method. The principle of LDA is to project the labeled data or points into a space with lower dimensions through the projection method, so that the projected points can be distinguished by category, so as to achieve the effect of extracting classification information and compressing the dimension of feature space.
[0100] It should be noted that the above-mentioned method of reducing the dimensionality of the feature vector of the target speech and the feature vector of the reference speech is only described as an example, and other achievable dimensionality reduction methods are also applicable to the present application and will not be introduced one by one here.
[0101] Step 203: Input the feature vector of each reference speech after dimension reduction and the feature vector of the target speech after dimension reduction into a probabilistic linear discriminant analysis model respectively.
[0102] Among them, the probabilistic linear discriminant analysis model has been trained using training samples and can accurately determine whether two input voices belong to the same speaker.
[0103] As a possible implementation method, the feature vector of the target speech and the feature vector of the reference speech can be input into a trained PLDA model to calculate the similarity between the feature vector of the target speech and the feature vector of the reference speech according to PLDA to output the probability that the target speech and the reference speech belong to the same speaker.
[0104] The following formula may be used to calculate the similarity between the feature vector of the target speech and the feature vector of the reference speech.
[0105]
[0106] Among them, score is the similarity between the feature vector of the target speech and the feature vector of the reference speech, η1 and η2 are the feature vectors of the target speech and the reference speech respectively, and the assumption that these two speech come from the same space is H s , the hypothesis from different spaces is Hd , p(η1,η2|H s ) is the likelihood function that the target speech and the reference speech come from the same space, p(η1|H d ) and p(η2|H d ) are the likelihood functions of η1 and η2 from different spaces.
[0107] From the calculation results of the above formula, it can be seen that the higher the ratio, the greater the probability that the target speech and the reference speech belong to the same speaker; the lower the ratio, the smaller the probability that the target speech and the reference speech belong to the same speaker.
[0108] As another possible implementation, the cosine scoring method can also be used to calculate the similarity between the feature vector of the target speech and the feature vector of the reference speech to determine whether the target speech and the reference speech belong to the same speaker. The specific calculation formula is as follows:
[0109]
[0110] Where S is the angle between the feature vector of the target speech and the feature vector of the reference speech, and W tar is the feature vector of the target speech, W test is the feature vector of the reference speech. The angle between the feature vector of the target speech and the feature vector of the reference speech reflects the correlation between the two. When the value of S is large, the correlation between the feature vector of the target speech and the feature vector of the reference speech is small; when the value of S is small, the correlation between the feature vector of the target speech and the feature vector of the reference speech is large.
[0111] It should be noted that, compared with the cosine scoring method used to determine the similarity between the feature vector of the target speech and the feature vector of the reference speech, the PLDA model used to determine the similarity between the feature vector of the target speech and the feature vector of the reference speech has the advantage of high accuracy.
[0112] Step 204: If it is confirmed that the target speech matches the input reference speech according to the output of the probabilistic linear discriminant analysis model, the object to which the input reference speech belongs is determined as the object of the target speech.
[0113] In an embodiment of the present application, the feature vector of each reference speech after dimensionality reduction and the feature vector of the target speech are input into the PLDA model. Based on the probability that the target speech and the reference speech belong to the same speaker output by the model, it can be confirmed whether the target speech matches the input reference speech to confirm whether the target speech and the reference speech belong to the same speaker.
[0114] As a possible implementation method, a probability threshold can be set in advance, and the probability output by the PLDA model can be compared with the probability threshold. When the probability output by the PLDA model that the target speech and the reference speech belong to the same speaking object is greater than the probability threshold, it can be determined that the target speech matches the reference speech, and then the object to which the input reference speech belongs can be determined as the object of the target speech.
[0115] The probability threshold may be pre-set according to actual conditions and is used to determine the maximum probability value of the target speech matching the reference speech.
[0116] When the probability that the target speech and the reference speech output by the PLDA model belong to the same speaking object is less than or equal to the probability threshold, it can be determined that the target speech and the reference speech do not belong to the same speaking object. In other words, the target speech does not match the input reference speech, and the object to which the input reference speech belongs is not the object of the target speech. Further, the target speech can be matched with the next reference speech to determine whether the target speech matches the next reference speech based on the probability output by the PLDA model.
[0117] It should be explained that the feature vector of each reference speech is input into the PLDA model together with the feature vector of the target speech after dimensionality reduction. The PLDA model can simultaneously output the probability of whether each reference speech matches the target speech, or can sequentially output the probability of whether each reference speech matches the target speech. There is no limitation here.
[0118] The feature vector of the reference speech input into the PLDA model can be the feature vector of the original speech of the known object collected, or the feature vector after the feature vector of the original speech of the known object is processed by dimensionality reduction, which is not limited here. If it is the feature vector of the original speech, it needs to be processed by dimensionality reduction before matching with the target speech; if it is the feature vector of the reduced dimension, it can be directly matched with the target speech.
[0119] In an embodiment of the present application, by obtaining a feature vector of at least one reference speech, a dimension reduction model is used to reduce the dimension of the feature vector of each reference speech and the feature vector of the target speech, and then the feature vector of each reference speech after dimension reduction and the feature vector of the target speech are input into a PLDA model to determine whether the object to which the input reference speech belongs is the object to which the target speech belongs according to the output of the PLDA model. Compared with the existing calculation method for determining vector similarity, the present application determines whether the target speech matches the input reference speech based on the PLDA model, which is conducive to improving the accuracy of confirming the object to which the target speech belongs.
[0120] As an example, Figure 3 As shown, Figure 3An example diagram of a speech processing method provided in an embodiment of the present application, such as Figure 3 As shown, the speech processing method may include the following steps:
[0121] Step 301: extract multiple audio frames from speech.
[0122] In the present application, after acquiring the speech, after determining the corresponding frame interval according to the length of each speech, multiple audio frames can be extracted from the speech.
[0123] In step 302, a low-dimensional feature extraction is performed on each audio frame of the multiple audio frames using a sinc function convolution layer to obtain an intermediate feature vector of each audio frame, and the intermediate feature vector of each audio frame is input into a residual network layer in a convolutional neural network model to perform high-dimensional feature extraction to obtain a feature vector of each audio frame.
[0124] Step 303: determine the eigenvalues of the eigenvector of each audio frame in multiple vector dimensions.
[0125] Step 304, according to the eigenvalues of the eigenvector of each audio frame in multiple vector dimensions, determine the mean eigenvalue of each audio frame in each vector dimension, and determine the mean eigenvalue in each vector dimension as the eigenvector of the speech.
[0126] Step 305: Use a dimensionality reduction model to reduce the dimension of the speech feature vector.
[0127] The dimensionality reduction model may be a PCA model.
[0128] Step 306: Input the feature vector of the reduced-dimensional speech and the feature vector of the reference speech into a probabilistic linear discriminant analysis model.
[0129] Step 307: confirm whether the speech matches the input reference speech based on the output of the probabilistic linear discriminant analysis model.
[0130] It should be noted that the detailed implementation process of step 301 to step 307 can be found in the above embodiment and will not be repeated here.
[0131] On the basis of the above embodiment, when a convolutional neural network model is used to extract feature vectors of audio frames, in order to improve the accuracy of the model, the model can be trained using training audio. Figure 3 Provide detailed introduction, Figure 4 A flowchart of a model training method for speech processing provided in an embodiment of the present application.
[0132] like Figure 4 As shown, the model training method for speech processing may include the following steps:
[0133] Step 401, obtaining training audio of multiple known objects.
[0134] The training audio is the audio whose belonging object has been determined, for example, the training audio 1 belongs to Zhang San, the training audio 2 belongs to Li Si, and so on.
[0135] In the embodiment of the present application, the voices of multiple registered users can be collected as training audio, and the audio of known objects can be downloaded from the audio library as training audio, etc. The present application does not limit the method for obtaining the training audio. Among them, the audio library can store the audio of multiple known objects.
[0136] Furthermore, after obtaining the training audios of multiple known objects, the multiple training audios can be classified according to the known objects to which each training audio belongs, so as to identify each training audio with a corresponding classification label. The classification label is used to classify the multiple training audios according to whether the known objects to which the training audios belong belong to the same object.
[0137] Step 402: extract at least one training frame from the training audio of each known object, and label the corresponding known objects for the extracted multiple training frames.
[0138] In an embodiment of the present application, after obtaining training audios of multiple known objects, at least one training frame can be extracted from the training audio of each known object.
[0139] As a possible scenario, after obtaining training audios of multiple known objects, the frame interval corresponding to each training audio can be determined according to the length of each training audio, so as to extract a set number of training frames from each training audio according to the frame interval corresponding to each training audio.
[0140] Optionally, when determining the frame interval corresponding to each training audio according to the length of each training audio, the frame interval corresponding to the speech length of the training audio can be determined by querying the mapping relationship between the speech length and the frame interval. The process of determining the mapping relationship between the speech length and the frame interval can refer to the implementation process in the above embodiment, which will not be repeated here.
[0141] In an embodiment of the present application, at least one training frame is extracted from the training audio of each known object. After obtaining multiple training frames, the corresponding known objects can be labeled for the extracted multiple training frames.
[0142] It should be noted that at least one training frame can be randomly extracted from the training audio of each known object, or at least one training frame can be extracted according to a set rule. The present application does not limit the method of extracting the training frames.
[0143] Step 403: Use the labeled multiple training frames to train the convolutional neural network.
[0144] The convolutional neural network model includes a sinc function convolution layer for performing low-dimensional feature extraction on the input training frame to obtain an intermediate feature vector, and a residual network layer for performing high-dimensional feature extraction on the intermediate feature vector to output a feature vector.
[0145] In an embodiment of the present application, after labeling the corresponding known objects for multiple training frames extracted from the training audio of each known object, the labeled multiple training frames can be used to train the Sigmund function convolution layer and the residual network layer of the convolutional neural network model.
[0146] In a specific implementation process, multiple training frames of annotated objects can be input into a convolutional neural network. After obtaining the feature vectors of each training frame, on the one hand, the first matching degree between any two training frames is determined based on the feature vectors of each training frame. On the other hand, based on the annotations of each training frame, the second matching degree between any two training frames is determined. Furthermore, based on the difference between the first matching degree and the second matching degree between any two training frames, the model parameters of the convolutional neural network are adjusted.
[0147] For the sake of distinction, in this application, the matching degree between any two training frames determined based on the feature vectors between the training frames is named the first matching degree; the matching degree between any two training frames determined based on the known objects annotated in the training frames is named the second matching degree. Of course, other naming methods can also be used, which are not limited here.
[0148] In one possible case, if a first matching degree between two training frames is determined based on the feature vectors between any two training frames, and a second matching degree between the two training frames is determined based on the known objects annotated by the two training frames, and it is determined that there is no difference between the first matching degree and the second matching degree, then the output of the convolutional neural network is relatively accurate and there is no need to adjust the model parameters of the convolutional neural network.
[0149] In another possible case, if the first matching degree between the two training frames is determined based on the feature vectors between any two training frames, and the second matching degree between the two training frames is determined based on the known objects annotated by the two training frames, and it is determined that the difference between the first matching degree and the second matching degree is large, then the error of the convolutional neural network is large, and the model parameters of the convolutional neural network can be adjusted to improve the accuracy of the model output.
[0150] Exemplarily, the training frames of multiple labeled objects can be input into the convolution layer of the convolutional neural network model, and low-dimensional feature extraction is performed on the input multiple training frames to obtain the intermediate feature vector of each training frame, and then the intermediate feature vector of each training frame is input into the residual network layer for high-dimensional feature extraction to obtain the feature vector of each training frame. Further, based on the feature vectors of any two training frames, the first matching degree between the two training frames is determined, and based on the annotations of the two training frames, the second matching degree between the two training frames is determined, so as to adjust the model parameters of the convolutional neural network according to the difference between the first matching degree and the second matching degree between any two training frames.
[0151] The model training method for speech processing of the embodiment of the present application obtains training audios of multiple known objects, extracts at least one training frame from the training audio of each known object, and labels the corresponding known objects for the extracted multiple training frames. Then, the convolutional neural network is trained using the labeled multiple training frames, thereby improving the accuracy of the feature vector of the extracted audio frame of the convolutional neural network model.
[0152] In order to implement the above embodiment, the present application also proposes a speech processing device.
[0153] Figure 5 A schematic diagram of the structure of a speech processing device provided in an embodiment of the present application.
[0154] like Figure 5 As shown, the speech processing device 500 may include: an acquisition module 510 , an audio frame extraction module 520 , a feature extraction module 530 , an input module 540 , a first determination module 550 and a second confirmation module 560 .
[0155] The acquisition module 510 is used to acquire the target speech.
[0156] The audio frame extraction module 520 is used to extract multiple audio frames from the target speech.
[0157] The feature extraction module 530 is used to perform low-dimensional feature extraction on each audio frame of the multiple audio frames using the Single function convolution layer in the convolutional neural network model to obtain an intermediate feature vector of each audio frame.
[0158] The processing module 540 is used to input the intermediate feature vector of each audio frame into the residual network layer in the convolutional neural network model to perform high-dimensional feature extraction to obtain the feature vector of each audio frame.
[0159] The first determination module 550 is configured to determine a feature vector of a target speech according to feature vectors of a plurality of audio frames.
[0160] The second determination module 560 is used to determine a matching reference speech according to the feature vector of the target speech, so as to determine the object to which the reference speech belongs as the object of the target speech.
[0161] As a possible situation, the first determining module 550 may also be specifically configured to:
[0162] According to the eigenvalues of the eigenvector of each audio frame in multiple vector dimensions, the mean eigenvalue of each audio frame in each vector dimension is determined; and the mean eigenvalue in each vector dimension is determined as the eigenvector of the target speech.
[0163] As another possible case, the audio frame extraction module 520 may include:
[0164] The determination unit is used to determine the frame interval corresponding to the target speech according to the speech length of the target speech.
[0165] The extraction unit is used to extract multiple audio frames from the target speech according to the frame interval corresponding to the target speech, and the number of the multiple audio frames is a set number.
[0166] As another possible situation, the determination unit may also be specifically used for:
[0167] According to the mapping relationship between the speech length and the frame interval, the frame interval corresponding to the speech length of the target speech is determined.
[0168] As another possible situation, the determination unit may also be specifically used for:
[0169] According to multiple speech lengths and set numbers, the segment lengths corresponding to the multiple speech lengths are determined; for each speech length, the difference between the corresponding segment length and the set audio frame length is used as the frame interval corresponding to each speech length; according to the frame intervals corresponding to the multiple speech lengths, a mapping relationship between the speech lengths and the frame intervals is established.
[0170] As another possible situation, the second determining module 560 may include:
[0171] The acquisition unit is used to acquire a feature vector of at least one reference speech.
[0172] The dimension reduction unit is used to reduce the dimension of each feature vector of the reference speech and the feature vector of the target speech by using a dimension reduction model.
[0173] The input unit is used to input the feature vector of each reference speech after dimension reduction and the feature vector of the target speech after dimension reduction into the probabilistic linear discriminant analysis model respectively.
[0174] The confirmation unit is used to determine the object to which the input reference speech belongs as the object of the target speech if it is confirmed that the target speech matches the input reference speech according to the output of the probabilistic linear discriminant analysis model.
[0175] As another possible case, the acquisition unit may also be specifically used for:
[0176] The collected voices of each registered user are determined as reference voices; the frame interval corresponding to each reference voice is determined according to the voice length of each reference voice; a set number of reference frames are extracted from each reference voice according to the frame interval corresponding to each reference voice; and a feature vector of the corresponding reference voice is determined according to the feature vectors of the multiple reference frames.
[0177] As another possible situation, the dimension of the intermediate feature vector input to the residual network layer matches the dimension of the intermediate feature vector output by the sinc function convolution layer. It should be noted that the above explanation of the speech processing method embodiment is also applicable to the speech processing device of this embodiment, which will not be repeated here.
[0178] The speech processing device of the embodiment of the present application extracts multiple audio frames from the target speech after acquiring the target speech. In this way, when the feature vector of the target speech is subsequently determined based on the feature vector of a set number of audio frames, the fluctuation of the feature expression performance of the feature vector of the target speech due to the fluctuation of the number of audio frames is avoided, so that the feature vector of the target speech can stably express the characteristics of the target speech. Therefore, the accuracy of the recognition result of confirming whether the target speech and the reference speech belong to the same speaking object is more stable, thereby realizing the confirmation of the object according to the recognition of the speech of different users, which can not only provide customized services for different users, but also improve the security of identity authentication.
[0179] In order to implement the above embodiments, the present application proposes a model training device for speech processing.
[0180] Figure 6 A schematic diagram of the structure of a model training device for speech processing provided in an embodiment of the present application.
[0181] like Figure 6 As shown, the model training device 600 for speech processing may include: an acquisition module 610, a labeling module 620 and a training module 630.
[0182] The acquisition module 610 is used to acquire training audio of multiple known objects.
[0183] The labeling module 620 is used to extract at least one training frame from the training audio of each known object, and label the corresponding known objects for the extracted multiple training frames.
[0184] A training module 630 is used to train the convolutional neural network using the labeled multiple training frames;
[0185] The convolutional neural network model includes a sinc function convolution layer for performing low-dimensional feature extraction on the input training frame to obtain an intermediate feature vector, and a residual network layer for performing high-dimensional feature extraction on the intermediate feature vector to output a feature vector.
[0186] As a possible scenario, the training module 630 may also be specifically used for:
[0187] Input multiple training frames into a convolutional neural network to obtain a feature vector of each training frame; determine a first matching degree between any two training frames based on the feature vector of each training frame; determine a second matching degree between any two training frames based on the annotations of each training frame; and adjust the model parameters of the convolutional neural network based on the difference between the first matching degree and the second matching degree between any two training frames.
[0188] It should be noted that the aforementioned explanation of the embodiment of the model training method for speech processing is also applicable to the model training device for speech processing of this embodiment, and will not be repeated here.
[0189] The model training device for speech processing of the embodiment of the present application obtains training audio of multiple known objects, extracts at least one training frame from the training audio of each known object, and labels the corresponding known objects for the extracted multiple training frames. Then, the convolutional neural network is trained using the labeled multiple training frames, thereby improving the accuracy of the feature vector of the extracted audio frame of the convolutional neural network model.
[0190] In order to implement the above embodiments, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the speech processing method of the above embodiments is implemented, or the model training method for speech processing of the above embodiments is implemented.
[0191] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the speech processing method of the above embodiments, or implements the model training method for speech processing of the above embodiments.
[0192] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0193] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0194] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0195] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0196] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0197] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0198] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0199] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A speech processing method, characterized in that: The method comprises: Get the target voice; Extracting a plurality of audio frames from the target speech; Performing low-dimensional feature extraction on each of the multiple audio frames using a Singe function convolution layer in a convolutional neural network model to obtain an intermediate feature vector of each of the audio frames; Inputting the intermediate feature vector of each of the audio frames into the residual network layer in the convolutional neural network model to perform high-dimensional feature extraction to obtain a feature vector of each of the audio frames; Determining a feature vector of the target speech according to the feature vectors of the multiple audio frames; According to the feature vector of the target speech, a matching reference speech is determined, so that the object to which the reference speech belongs is determined as the object of the target speech.
2. The speech processing method according to claim 1, characterized in that: The step of determining the feature vector of the target speech according to the feature vectors of the plurality of audio frames comprises: Determine, according to the eigenvalues of the eigenvectors of each of the audio frames in multiple vector dimensions, a mean eigenvalue of each audio frame in each vector dimension; The mean of the eigenvalues on each vector dimension is determined as the eigenvector of the target speech.
3. The speech processing method according to claim 1, characterized in that: The step of extracting a plurality of audio frames from the target speech comprises: Determining a frame interval corresponding to the target speech according to the speech length of the target speech; Extracting a plurality of audio frames from the target speech according to a frame interval corresponding to the target speech, wherein the number of the plurality of audio frames is a set number.
4. The speech processing method according to claim 3, characterized in that: The step of determining the frame interval corresponding to the target speech according to the speech length of the target speech comprises: According to the mapping relationship between the speech length and the frame interval, the frame interval corresponding to the speech length of the target speech is determined.
5. The speech processing method according to claim 4, characterized in that: The mapping relationship is determined in the following manner: Determining the segment lengths corresponding to the multiple speech lengths according to the multiple speech lengths and the set number; For each of the speech lengths, the difference between the corresponding segment length and the set audio frame length is used as the frame interval corresponding to each of the speech lengths; According to the frame intervals corresponding to the multiple speech lengths, a mapping relationship between the speech lengths and the frame intervals is established.
6. The speech processing method according to any one of claims 1 to 5, characterized in that: The step of determining a matching reference speech according to the feature vector of the target speech, so as to determine the object to which the reference speech belongs as the object of the target speech, comprises: Acquire a feature vector of at least one of the reference speech; Using a dimensionality reduction model to reduce the dimension of each feature vector of the reference speech and the feature vector of the target speech; Inputting the feature vector of each reference speech after dimension reduction and the feature vector of the target speech after dimension reduction into a probabilistic linear discriminant analysis model respectively; If it is confirmed according to the output of the probabilistic linear discriminant analysis model that the target speech matches the input reference speech, the object to which the input reference speech belongs is determined as the object of the target speech.
7. The speech processing method according to claim 6, characterized in that: The step of obtaining a feature vector of at least one reference speech comprises: Determining the collected voices of each registered user as each of the reference voices; Determining the frame interval corresponding to each of the reference voices according to the voice length of each of the reference voices; Extracting the set number of reference frames from each of the reference speech according to the frame interval corresponding to each of the reference speech; According to the feature vectors of the multiple reference frames, a feature vector of the corresponding reference speech is determined.
8. The speech processing method according to any one of claims 1 to 5, characterized in that: The dimension of the intermediate feature vector input to the residual network layer matches the dimension of the intermediate feature vector output by the sinc function convolution layer.
9. A model training method based on the speech processing method according to claim 1, characterized in that: The method comprises: Obtain training audio of multiple known objects; Extracting at least one training frame from the training audio of each known object, and labeling the corresponding known object for the extracted multiple training frames; Using the labeled multiple training frames to train the convolutional neural network; The convolutional neural network model includes a sinc function convolution layer for performing low-dimensional feature extraction on an input training frame to obtain an intermediate feature vector, and a residual network layer for performing high-dimensional feature extraction on the intermediate feature vector to output a feature vector.
10. The model training method according to claim 9, characterized in that: The step of training the convolutional neural network by using the labeled plurality of training frames includes: Inputting a plurality of the training frames into the convolutional neural network to obtain a feature vector of each of the training frames; Determining a first matching degree between any two of the training frames according to the feature vectors of the training frames; Determining a second matching degree between any two of the training frames according to the annotations of the training frames; Adjust the model parameters of the convolutional neural network according to the difference between the first matching degree and the second matching degree between any two of the training frames.
11. A speech processing device, characterized in that: include: An acquisition module, used to acquire target speech; An audio frame extraction module, used to extract multiple audio frames from the target speech; A feature extraction module, configured to perform low-dimensional feature extraction on each of the multiple audio frames using a Singer function convolution layer in a convolutional neural network model to obtain an intermediate feature vector of each of the audio frames; A processing module, used for inputting the intermediate feature vector of each audio frame into the residual network layer in the convolutional neural network model to perform high-dimensional feature extraction to obtain a feature vector of each audio frame; A first determining module, configured to determine a feature vector of the target speech according to the feature vectors of the plurality of audio frames; The second determination module is used to determine a matching reference speech according to the feature vector of the target speech, so as to determine the object to which the reference speech belongs as the object of the target speech.
12. The speech processing device according to claim 11, characterized in that: The first determining module is specifically configured to: Determine, according to the eigenvalues of the eigenvectors of each of the audio frames in multiple vector dimensions, a mean eigenvalue of each audio frame in each vector dimension; The mean of the eigenvalues on each vector dimension is determined as the eigenvector of the target speech.
13. The speech processing device according to claim 11, characterized in that: The audio frame extraction module comprises: A determination unit, configured to determine a frame interval corresponding to the target speech according to a speech length of the target speech; The extraction unit is used to extract multiple audio frames from the target speech according to the frame interval corresponding to the target speech, and the number of the multiple audio frames is a set number.
14. The speech processing device according to claim 13, characterized in that: The determining unit is specifically configured to: According to the mapping relationship between the speech length and the frame interval, the frame interval corresponding to the speech length of the target speech is determined.
15. The speech processing device according to claim 14, characterized in that: The determining unit is further configured to determine the mapping relationship in the following manner: Determining the segment lengths corresponding to the multiple speech lengths according to the multiple speech lengths and the set number; For each of the speech lengths, the difference between the corresponding segment length and the set audio frame length is used as the frame interval corresponding to each of the speech lengths; According to the frame intervals corresponding to the multiple speech lengths, a mapping relationship between the speech lengths and the frame intervals is established.
16. The speech processing device according to any one of claims 11 to 15, characterized in that: The second determining module includes: An acquiring unit, configured to acquire a feature vector of at least one of the reference speech; A dimension reduction unit, used for reducing the dimension of each feature vector of the reference speech and the feature vector of the target speech by using a dimension reduction model; An input unit, used for inputting the feature vector of each reference speech after dimension reduction and the feature vector of the target speech after dimension reduction into a probabilistic linear discriminant analysis model; A confirmation unit is used to determine the object to which the input reference speech belongs as the object of the target speech if it is confirmed that the target speech matches the input reference speech according to the output of the probabilistic linear discriminant analysis model.
17. The speech processing device according to claim 16, characterized in that: The acquisition unit is specifically used for: Determining the collected voices of each registered user as each of the reference voices; Determining the frame interval corresponding to each of the reference voices according to the voice length of each of the reference voices; Extracting the set number of reference frames from each of the reference speech according to the frame interval corresponding to each of the reference speech; According to the feature vectors of the multiple reference frames, a feature vector of the corresponding reference speech is determined.
18. The speech processing device according to any one of claims 11 to 15, characterized in that: The dimension of the intermediate feature vector input to the residual network layer matches the dimension of the intermediate feature vector output by the sinc function convolution layer.
19. A model training device based on the speech processing device according to claim 11, characterized in that: The device comprises: The acquisition module is further used to acquire training audio of multiple known objects; A labeling module, used to extract at least one training frame from the training audio of each known object, and label the corresponding known object for the extracted multiple training frames; A training module, used to train a convolutional neural network using the labeled multiple training frames; The convolutional neural network model includes a sinc function convolution layer for performing low-dimensional feature extraction on an input training frame to obtain an intermediate feature vector, and a residual network layer for performing high-dimensional feature extraction on the intermediate feature vector to output a feature vector.
20. The model training device according to claim 19, characterized in that: The training module is specifically used for: Inputting a plurality of the training frames into the convolutional neural network to obtain a feature vector of each of the training frames; Determining a first matching degree between any two of the training frames according to the feature vectors of the training frames; Determining a second matching degree between any two of the training frames according to the annotations of the training frames; Adjust the model parameters of the convolutional neural network according to the difference between the first matching degree and the second matching degree between any two of the training frames.
21. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the speech processing method according to any one of claims 1 to 10 is implemented.
22. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the speech processing method as described in any one of claims 1 to 10 is implemented.
