Method, device, electronic device and storage medium for extracting speech features
By obtaining the first feature vector and covariance matrix of speech data, the posterior mean vector is calculated and the importance parameter vector is spliced to generate speech features, which solves the problem of factor interference in the process of extracting speech features and improves the extraction accuracy.
Patent Information
- Application Number
- CN202111272479.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-29
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-10-29
AI Technical Summary
In voiceprint recognition technology, the process of extracting speech features is interfered by the speaker and the environmental factors, which affects the accuracy rate and leads to inaccurate feature extraction.
By obtaining the first eigenvector and covariance matrix of speech data, the posterior mean vector is calculated, and spliced with the importance parameter vector to generate speech features to improve the accuracy of extraction.
It improves the accuracy of speech feature extraction, can more fully extract the voice features of the speaker, and reduces the impact of environmental interference.
Smart Images

Figure CN114005452B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of voiceprint recognition, and specifically to a method, device, electronic device, computer-readable storage medium and computer program product for extracting voice features. Background Art
[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.
[0003] In voiceprint recognition technology, voice feature extraction is crucial. Voice feature extraction generally includes modules for extracting front-end features (also called low-level features), extracting speaker features (also called high-level features), and back-end classification. After the voice data is processed by the above modules, the corresponding recognition results can be obtained.
[0004] However, in the process of extracting speech features, it is often interfered by many factors such as the speaker himself and the environment, which has a certain impact on the accuracy of extracting speech features. How to eliminate these influences and extract the features of speech data more accurately has become a hot research field in speech feature extraction technology in recent years.
[0005] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention
[0006] The present disclosure provides a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product for extracting speech features.
[0007] According to one aspect of the present disclosure, a method for extracting speech features is provided, comprising: obtaining speech to be processed, and dividing the speech into frames to obtain multiple frames of speech data; performing feature extraction on the multiple frames of speech data to obtain a first feature vector of the speech, wherein the first feature vector includes multiple elements corresponding to corresponding features of the multiple frames of speech data respectively; obtaining a covariance matrix associated with multiple elements in the first feature vector; obtaining a second feature vector of the speech based on the first feature vector and the covariance matrix; obtaining a parameter vector characterizing the importance of multiple elements in the first feature vector; and concatenating the second feature vector with the parameter vector to obtain speech features of the speech.
[0008] According to another aspect of the present disclosure, a method for training a speech feature extraction model is provided, comprising: obtaining sample speech that has been sample-labeled, and dividing the sample speech into frames to obtain multiple frames of sample speech data; performing feature extraction on the multiple frames of sample speech data to obtain a first sample feature vector of the sample speech, wherein the first sample feature vector includes multiple sample elements that respectively correspond to corresponding features of the multiple frames of sample speech data; obtaining a sample covariance matrix associated with the multiple sample elements in the first sample feature vector; obtaining a second sample feature vector of the sample speech based on the first sample feature vector and the sample covariance matrix; obtaining a sample parameter vector that characterizes the importance of multiple sample elements in the sample first feature vector; concatenating the second sample feature vector with the sample parameter vector to obtain speech features of the sample speech; and obtaining network parameters for updating the speech feature extraction model based on the speech features to train the speech feature extraction model.
[0009] According to another aspect of the present disclosure, a device for extracting speech features is provided, comprising: a first module, configured to: obtain speech to be processed, and divide the speech into frames to obtain multiple frames of speech data; a second module, configured to: perform feature extraction on the multiple frames of speech data to obtain a first feature vector of the speech, wherein the first feature vector includes multiple elements corresponding to corresponding features of the multiple frames of speech data respectively; a third module, configured to: obtain a covariance matrix associated with multiple elements in the first feature vector; a fourth module, configured to: obtain a second feature vector of the speech based on the first feature vector and the covariance matrix; a fifth module, configured to: obtain a parameter vector characterizing the importance of multiple elements in the first feature vector; and a sixth module, configured to: concatenate the second feature vector with the parameter vector to obtain speech features of the speech.
[0010] According to another aspect of the present disclosure, a device for training a speech feature extraction model is provided, comprising: a first module, configured to: obtain sample speech that has been sample-annotated, and divide the sample speech into frames to obtain multiple frames of sample speech data; a second module, configured to: perform feature extraction on the multiple frames of sample speech data to obtain a first sample feature vector of the sample speech, wherein the first sample feature vector includes multiple sample elements that respectively correspond to corresponding features of the multiple frames of sample speech data; a third module, configured to: obtain a sample covariance matrix associated with multiple sample elements in the first sample feature vector; a fourth module, configured to: obtain a second sample feature vector of the sample speech based on the first sample feature vector and the sample covariance matrix; a fifth module, configured to: obtain a sample parameter vector that characterizes the importance of multiple sample elements in the sample first feature vector; a sixth module, configured to: concatenate the second sample feature vector with the sample parameter vector to obtain speech features; and a seventh module, configured to: obtain network parameters for updating the speech feature extraction model based on the speech features to train the speech feature extraction model.
[0011] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above method.
[0012] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above method.
[0013] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.
[0014] According to one or more embodiments of the present disclosure, features of speech data may be extracted more accurately.
[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.
[0017] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;
[0018] Figure 2 A flow chart of a method for extracting speech features according to an embodiment of the present disclosure is shown;
[0019] Figure 3 A flow chart of a method for training a speech feature extraction model according to an embodiment of the present disclosure is shown;
[0020] Figure 4 A schematic diagram of a speech extraction model according to an embodiment of the present disclosure is shown;
[0021] Figure 5 A block diagram of a device for extracting speech features according to an embodiment of the present disclosure is shown;
[0022] Figure 6 A block diagram of a device for extracting speech features according to another embodiment of the present disclosure is shown;
[0023] Figure 7 A block diagram of a device for training a speech feature extraction model according to an embodiment of the present disclosure is shown;
[0024] Figure 8 A structural block diagram of an electronic device applicable to the embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0026] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.
[0027] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.
[0028] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
[0029] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.
[0030] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable the method of extracting speech features of the present disclosure to be performed.
[0031] In some embodiments, server 120 may also provide other services or software applications that may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.
[0032] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.
[0033] The user may provide voice data using client devices 101, 102, 103, 104, 105, and / or 106. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.
[0034] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing a variety of different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.
[0035] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.
[0036] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.
[0037] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.
[0038] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.
[0039] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.
[0040] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The data repository 130 may reside in various locations. For example, the data repository used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data repository 130 may be of different types. In some embodiments, the data repository used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to a command.
[0041] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.
[0042] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.
[0043] Figure 2 2 is a flow chart showing a method 200 for extracting speech features according to an exemplary embodiment of the present disclosure. Figure 2 As shown, the method 200 may include the following steps:
[0044] Step S201, obtaining the speech to be processed, and dividing the speech into frames to obtain multiple frames of speech data;
[0045] Step S203, performing feature extraction on the multiple frames of speech data to obtain a first feature vector of the speech, wherein the first feature vector includes a plurality of elements corresponding to corresponding features of the multiple frames of speech data;
[0046] Step S205, obtaining a covariance matrix associated with a plurality of elements in the first eigenvector;
[0047] Step S207, obtaining a second eigenvector of the speech based on the first eigenvector and the covariance matrix;
[0048] Step S209, obtaining a parameter vector representing the importance of multiple elements in the first feature vector; and
[0049] Step S211, concatenating the second feature vector with the parameter vector to obtain the speech feature of the speech.
[0050] According to the method for extracting speech features disclosed in the present invention, in order to more accurately extract the features of the speaker's speech data, on the basis of generating the first feature vector of the speaker's speech, the covariance matrix that can characterize the correlation between the first feature vector in different feature dimensions (for example, the corresponding feature dimensions related to the speaker's own physiological characteristics such as the oral cavity and vocal cords, the corresponding feature dimensions related to the environment and background when speaking, etc.) can be used to obtain the posterior mean vector of the initial feature vector according to the Gaussian distribution. The posterior mean vector can contain richer speech feature information so that the speaker's speech features can be extracted as much as possible and fully. At the same time, based on the first feature vector, a parameter vector can be obtained to characterize the importance of each frame of data in the multi-frame speech data. Therefore, the speech feature obtained by splicing the posterior mean vector and the parameter vector is an updated feature vector based on the first feature vector, which can be used to accurately extract the features of the speaker's speech, thereby improving the accuracy of speech feature extraction.
[0051] The various steps of the method 200 for extracting speech features according to the present disclosure will be described in detail below.
[0052] In step S201, the speech to be processed may be the speech of any speaker. After obtaining the speech data of the speaker, the speech data may be preprocessed first, including removing environmental noise, busy tone, ring tone, etc. Then, the preprocessed speech data is framed according to a certain duration to obtain multiple frames of speech data.
[0053] According to some embodiments, at step S203, multiple frames of speech data may be input into a first neural network to perform feature extraction through the first neural network.
[0054] The first neural network can be any known type of neural network. In some embodiments, the first neural network can be composed of a basic network and a fully connected layer, wherein the basic network is a 5-layer time-delay neural network layer (TDNN).
[0055] Since neural networks can fit any distribution, there is no need to pre-set the distribution that needs to be satisfied, but to rely on the learning ability of the neural network itself to obtain the corresponding calculation results. With the help of multiple mappings of feature data by the neural network, the accuracy of calculating the initial feature vector can be improved.
[0056] According to some embodiments, at step S205, the first eigenvector may be input into the second neural network to calculate a covariance matrix through the second neural network.
[0057] The second neural network can also be any known type of neural network. Preferably, the second neural network is a two-layer feedforward neural network, wherein the size of the first layer is 1500×256, and the activation function of the first layer is the RELU activation function; the size of the second layer is 256×1500, and the activation function of the second layer is the softplus activation function, for logarithmic estimation. In this way, the second neural network takes the first eigenvector as input, and auxiliary outputs the covariance matrix to prepare for the subsequent application of the normalized exponential function (Softmax). In addition, by means of logarithmic estimation, the multiplication calculation can be converted into an addition calculation, thereby reducing the amount of calculation.
[0058] Accordingly, the covariance matrix may have second elements corresponding to the multiple first elements of the first eigenvector. Since covariance can be used to represent the correlation between data of different dimensions, each second element in the covariance matrix may characterize the correlation between the corresponding first elements in the initial eigenvector in different feature dimensions. Here, the different feature dimensions of the initial eigenvector may refer to, for example, corresponding feature dimensions related to the speaker's own physiological characteristics such as the oral cavity and vocal cords, corresponding feature dimensions related to the environment and background when speaking, etc. For example, each second element in the covariance matrix may characterize the correlation between the pronunciation features of the corresponding first element in the initial eigenvector in the oral cavity and the pronunciation features on the vocal cords.
[0059] According to some embodiments, in step S207, the second eigenvector is a posterior mean vector of the first eigenvector according to a Gaussian distribution.
[0060] Here, the posterior mean vector can be understood as an accurate value obtained based on the actual speech feature data of the speaker. In other words, under the assumption that the speech feature of the speaker conforms to the Gaussian distribution, the specific form of the Gaussian distribution cannot be reflected only from the prior probability.
[0061] In this way, when the posterior mean vector is obtained as described above, the speech features of the speaker can be accurately expressed with a Gaussian distribution, thereby facilitating accurate extraction of the speech features of the speaker.
[0062] According to some embodiments, a normalized exponential function may be applied to the covariance matrix to obtain a normalized exponential value with respect to the covariance matrix; and the normalized exponential value may be multiplied by the first eigenvector to obtain the second eigenvector.
[0063] After applying the normalized exponential function to the covariance matrix, the obtained normalized exponential value is multiplied by the first eigenvector, that is, weighted averaging is performed, so that the posterior mean vector of the first eigenvector according to the Gaussian distribution can be obtained.
[0064] In one example, in order to obtain the posterior mean vector, the prior mean and covariance may also be initialized (for example, initialized to 0) to be added to the above weighted average calculation.
[0065] In this way, since Softmax is similar to a normalization operation, after applying Softmax to the covariance matrix, the obtained Softmax value is multiplied by the first eigenvector, that is, weighted averaging is performed, so that the posterior mean vector of the first eigenvector according to the Gaussian distribution can be obtained.
[0066] According to some embodiments, in step S209, the first feature vector may be input into a third neural network, wherein the third neural network includes a first sub-neural network and a second sub-neural network, and wherein the first feature vector is input into the first sub-neural network to obtain a normalized weight vector of the first feature vector; and the normalized weight vector and the first feature vector are input into the second sub-neural network to calculate a parameter vector.
[0067] The first sub-neural network and the second sub-neural network may also be any known type of neural network.
[0068] A normalized weight vector of the first eigenvector is obtained through the first sub-neural network, and the weight vector can be represented as a score for the importance of each element in the first eigenvector. By using the normalized weight vector for calculation in the second sub-neural network, a parameter vector can be obtained, and the parameter vector can evaluate the importance of each element in the first eigenvector.
[0069] According to some embodiments, an initial weight vector of the first feature vector may be obtained through the first sub-neural network; and a normalized exponential function may be applied to the initial weight vector to obtain a normalized weight vector.
[0070] By normalizing the exponential function, each element in the weight vector can be made to be between (0, 1), and the sum of all elements is 1, so as to represent the importance of each frame of data in the multi-frame speech data in a probabilistic manner.
[0071] According to some embodiments, the parameter vector may include at least one of a weighted mean vector and a weighted standard deviation vector.
[0072] By obtaining a normalized weight vector, and then obtaining a weighted mean vector and a weighted standard deviation vector, the weighted mean vector and the weighted standard deviation vector can be used to characterize the importance of each frame of data in the multi-frame speech data. By using these two parameter vectors, the ability of the model to select frames can be improved, so that the importance of the frames selected by the model is higher.
[0073] According to some embodiments, in step S211, the second feature vector has multiple dimensions, and the parameter vector has multiple dimensions, and wherein concatenating the second feature vector with the parameter vector to obtain speech features of the speech includes: generating speech features having a target dimension, wherein the target dimension is the sum of the dimensions of the second feature vector and the parameter vector.
[0074] According to some examples, the weighted mean vector and the weighted standard deviation vector may have 512 dimensions to 1500 dimensions, respectively, and the second eigenvector may have 300 dimensions to 512 dimensions.
[0075] In an example, the second feature vector may have 300 dimensions, the weighted mean vector and the weighted standard deviation vector may have 512 dimensions respectively, and the speech feature obtained by concatenating the three may have 1324 dimensions.
[0076] The speech feature is obtained by concatenating the parameter vector and the second feature vector. The speech feature not only includes the characteristic components of the initial speech data, but also includes the components of the importance of each frame of data in multiple frames of speech data, thereby better characterizing the characteristics of the speech data.
[0077] As described above, according to the method for extracting speech features disclosed in the present invention, in order to more accurately extract the features of the speaker's speech data, on the basis of generating the first feature vector of the speaker's speech, the covariance matrix that can characterize the correlation between the first feature vector in different feature dimensions (for example, the corresponding feature dimensions related to the speaker's own physiological characteristics such as the oral cavity and vocal cords, the corresponding feature dimensions related to the environment and background when speaking, etc.) can be used to obtain the posterior mean vector of the initial feature vector according to the Gaussian distribution. The posterior mean vector can contain richer speech feature information so that the speaker's speech features can be extracted as much as possible and fully. At the same time, based on the first feature vector, a parameter vector can be obtained to characterize the importance of each frame of data in the multi-frame speech data. Therefore, the speech feature obtained by splicing the posterior mean vector and the parameter vector is an updated feature vector based on the initial feature vector, which can be used to accurately extract the features of the speaker's speech, thereby improving the accuracy of speech feature extraction.
[0078] According to an embodiment of the present disclosure, a method for training a speech feature extraction model is also provided.
[0079] Figure 3 FIG. 3 is a flow chart of a method 300 for training a speech feature extraction model according to an embodiment of the present disclosure. Figure 3 As shown, method 300 may include the following steps:
[0080] Step S301, obtaining sample speech that has been sample-labeled, and dividing the sample speech into frames to obtain multiple frames of sample speech data;
[0081] Step S303, performing feature extraction on the multiple frames of sample speech data to obtain a first sample feature vector of the sample speech, wherein the first sample feature vector includes a plurality of sample elements corresponding to corresponding features of the multiple frames of sample speech data;
[0082] Step S305, obtaining a sample covariance matrix associated with a plurality of sample elements in the first sample feature vector;
[0083] Step S307, obtaining a second sample feature vector of the sample speech based on the first sample feature vector and the sample covariance matrix;
[0084] Step S309, obtaining a sample parameter vector representing the importance of multiple sample elements in the first feature vector of the sample;
[0085] Step S311, concatenating the second sample feature vector and the sample parameter vector to obtain speech features of the sample speech; and
[0086] Step S313, obtaining network parameters for updating the speech feature extraction model based on the speech features to train the speech feature extraction model.
[0087] It can be noted that the operations of steps S303 to S311 in the method 300 for training a speech feature extraction model are combined with Figure 2 The operations of steps S203 to S211 in the speech feature extraction method 200 are similar, and are all used to extract the speech features of the speaker.
[0088] The difference is that in the method 300 for training the speech feature extraction model, the data used for training in step S303 is for a predetermined speaker, that is, sample speech data, so that the model training can be performed for a specific speaker. Accordingly, the audio data needs to be speaker labeled so that it can be determined which specific speaker the audio data is for. In addition, when the sample data is not enough, the data can be enhanced in the time domain by methods such as aliasing echo and changing rate to increase the sample data.
[0089] In addition, the method 300 for training the speech feature extraction model also includes updating the network parameters of the speech feature extraction model in step S313. In other words, the network parameters are updated using the speech features extracted in step S311, so that multiple rounds of training can be iterated repeatedly until the network converges, thereby completing the training of the model. Accordingly, the speech features concatenated in step S311 can be input into, for example, two fully connected layers (with ReLU activation function) and then input into the output layer, the loss is calculated by the CE (cross entropy) criterion, and finally the network parameters are updated in reverse according to SGD (stochastic gradient descent), so that multiple rounds of iterations are performed until the network converges to complete the model training.
[0090] Figure 4 A schematic diagram of a speech feature extraction model 400 according to an embodiment of the present disclosure is shown.
[0091] The following will be combined Figures 2 to 4 To further illustrate the method for extracting speech features and the method for training a speech feature extraction model according to the present disclosure.
[0092] like Figure 4 As shown, the speech feature extraction model 400 may include an encoder 410 , a Gaussian posterior inference module 420 , a third neural network 430 , and a decoder 440 .
[0093] The encoder 410 may include a first neural network (consisting of a TDNN layer 411 and a fully connected layer 412 ) and a second neural network 413 . Figure 4 The second neural network 413 is schematically shown to include a neural network layer 413-1 and a neural network layer 413-2. However, the number of layers of the second neural network 413 can be appropriately selected according to actual conditions. The TDNN layer 411 can also be a neural network including multiple layers, which can be appropriately selected according to actual conditions.
[0094] TDNN layer 411 and fully connected layer 412 together may correspond to combining Figure 2 The first neural network, and the second neural network 413 may correspond to the combination of Figure 2 The second neural network. Figure 4 As shown, a plurality of frames of speech data X extracted from audio data about a speaker may be input into the speech extraction model 400, wherein the plurality of frames of speech data X may include data x1, x2, ..., x3 having a temporal contextual relationship. T , the multiple frame data x1, x2, ..., x T It may correspond to T frames after the audio data is divided into frames.
[0095] It is worth noting that, during the training phase of the speech feature extraction model 400, since the audio data is labeled with speakers, the multiple frames of speech data X are sample initial speech feature data corresponding to a specific speaker. After the training of the speech extraction model 400 is completed, when the speech extraction model 400 is used to extract speech features, the audio data can be about any speaker, so the multiple frames of speech data X are also about any speaker.
[0096] like Figure 4 As shown, the multi-frame speech data X={x1,x2,…,x T After passing through the TDNN layer 411 and the fully connected layer 412 (the first neural network), the first feature vector {z1,z2,…,z T In addition, the first eigenvector {z1,z2,…,z T After the second neural network 413, the first feature vector {z1, z2, …, z T The covariance matrix log{L1,L2,…,L T}(As mentioned above, for ease of calculation, the covariance matrix is calculated using a logarithmic method).
[0097] The first eigenvector {z1,z2,…,z T Each first element z1,z2,…,z T to contain feature information on different feature dimensions. For example, z1,z2,…,z T Each of (for example, taking z1 as an example) may contain feature information on the corresponding feature dimension related to the speaker's oral cavity, vocal cords, and other physiological characteristics. Therefore, the corresponding second element {L1, L2, ..., L T} (for example, taking L1 as an example) can characterize the correlation between the pronunciation features on the oral cavity and the pronunciation features on the vocal cords.
[0098] The operation of Gaussian posterior inference module 420 may correspond to the above combined Figure 2 , Figure 3 The second feature vector of the generated speaker is as follows: Figure 4 In the Gaussian posterior inference module 420, the covariance matrix log{L1,L2,…,L T} performs Softmax, and then combines it with the first eigenvector {z1,z2,…,z T} are multiplied to obtain the second eigenvector φs. Through the covariance matrix log{L1,L2,…,L T} and the Softmax value of the first eigenvector {z1,z2,…,z T} multiplication is also a weighted average operation, so the second eigenvector φ s That is, the first eigenvector is a posterior mean vector according to the Gaussian distribution, which is a more accurate expression of the speaker's speech features in the Gaussian distribution.
[0099] In one example, we can also use the prior mean and covariance (μ p , L p ) is initialized to 0 to be added to the weighted average calculation as above, that is, L0=0, z0=0.
[0100] At the same time, in the multi-frame speech data X={x1,x2,…,x T} includes multiple frames of speech data {x1, x2, ..., x T}, the above process of obtaining the second eigenvector φs can be expressed as the first eigenvector {z1,z2,…,z T} with a temporal contextual relationship of multiple first elements z1, z2, ..., z T It aggregates in time, so it can also be called time aggregation.
[0101] It is noted that, in the training phase of the speech extraction model 400, since the input multi-frame speech data X is sample initial speech feature data about a specific speaker, the first feature vector {z1, z2, ..., z T}、Covariance matrix log{L1,L2,…,L T} and the second eigenvector φ s They are also the first sample feature vector, the sample covariance matrix and the second sample feature vector for the specific speaker respectively.
[0102] The operation of the third neural network 430 may correspond to the above combined Figure 2 , Figure 3 The parameter vector representing the importance of multiple elements in the first feature vector is obtained, such as Figure 4 The weighted mean vector shown and the weighted standard deviation vector The third neural network 430 includes a first sub-neural network 431 and a second sub-neural network 432. The initial weight vector α of the first feature vector can be obtained through the first sub-neural network 431 t , the initial weight vector α t and the first eigenvector {z1,z2,…,z T} is input to the second sub-neural network 432 to obtain the weighted mean vector and the weighted standard deviation vector Concatenate the second eigenvector φs and the weighted mean vector and the weighted standard deviation vector The speech features can be obtained.
[0103] The decoder 440 may include an embedding layer 441, a fully connected layer 442, and an output layer 443. The embedding layer 441 may map the speech features to generate embedded speech features of the speaker.
[0104] It is noted that in the training phase of the speech extraction model 400, a fully connected layer 442 and an output layer 443 may be further included. The speech features extracted in the embedding layer 441 may be input into the fully connected layer 442 and then into the output layer 443 (for speaker classification), the loss is calculated by the CE (cross entropy) criterion, and finally the network parameters are updated in reverse according to the SGD (stochastic gradient descent), so that multiple rounds of iterations are performed until the network converges. In the testing phase, the speech features may be extracted in the embedding layer 441, and then the pairwise similarity is calculated by the PLDA scoring method.
[0105] Combination of the above Figures 2 to 4 The method for extracting speech features and the method for training a speech feature extraction model of the present disclosure are described. In order to more fully understand the present disclosure, the model assumptions on which the principles of the present disclosure are based are briefly described below.
[0106] First, we make a prior distribution assumption about the speaker’s characteristics:
[0107] 1) Model: z t =h+∈ t
[0108] 2) Hidden variables:
[0109] 3) Uncertainty:
[0110] Among them, z t Generate variables for the model, representing the speaker's feature vector; h is a hidden variable, representing the speaker's basis vector; ∈ t is the residual variable, representing the difference vector of different speakers; μ and L are the mean and covariance in the Gaussian distribution, respectively, and z t , h and ∈ t All obey Gaussian distribution.
[0111] The basis vector h can reflect the average value of several characteristic attributes of the population (such as several characteristic dimensions related to the oral cavity, vocal cords, tongue, lips, etc.). t The disturbance vector that can reflect the change in some of these characteristic attributes (such as the vibration frequency of the vocal cords) can be used to characterize the characteristics of any speaker based on this model.
[0112] Since the basis vector h follows a Gaussian distribution, the posterior probability of the basis vector h can be calculated for a specific population (e.g., 1,000 Asians) to reflect the specific form of the Gaussian distribution of the basis vector h for the population. That is, the present disclosure obtains the posterior mean vector of the first eigenvector according to the Gaussian distribution.
[0113] Given the input data about a specific population, the posterior probability distribution of the basis vector h can be derived as:
[0114]
[0115] in:
[0116]
[0117]
[0118] Here, we can obtain as well as Among them, fenc() and genc() can correspond to the first neural network and the second neural network as described above, respectively.
[0119] Calculate the score e for each element in the encoder {z1,z2,…,zT} t :
[0120] e t =v T f(Wz t +b)+k (Formula 4)
[0121] Where f() is the nonlinear activation function tanh, vT and k are the weight and bias of the linear layer, and W and b are the weight and bias in the activation function of the nonlinear layer.
[0122] E t Normalize across all frames using the softmax function:
[0123]
[0124] Calculate the weighted mean vector
[0125]
[0126] Calculate the weighted standard deviation vector
[0127]
[0128] Among them, Formula 4 and Formula 5 correspond to the first sub-neural network as described above, and Formula 6 and Formula 7 correspond to the second sub-neural network as described above.
[0129] In addition, the training process of the speech feature extraction model can also be expressed by the following pseudo code:
[0130]
[0131] Among them, θ can represent the network parameters to be updated, such as weights and biases. θ can be calculated by multiplying the parameter g and the learning force, where the parameter g can reflect the amount of change or update learned from a batch of data each time, and the learning force can represent the magnitude of the change. After obtaining the parameter g, it can be added to θ (such as weights and biases) to produce corresponding changes, thereby realizing the training process.
[0132] According to another aspect of the present disclosure, a voiceprint feature extraction device is also provided. Figure 5 A block diagram of an apparatus 500 for extracting speech features according to an embodiment of the present disclosure is shown.
[0133] like Figure 5 As shown, the device 500 may include: a framing module 501, configured to: obtain speech to be processed, and frame the speech to obtain multiple frames of speech data; a first acquisition module 502, configured to: perform feature extraction on the multiple frames of speech data to obtain a first feature vector of the speech, wherein the first feature vector includes multiple elements corresponding to corresponding features of the multiple frames of speech data; a second acquisition module 503, configured to: obtain a covariance matrix associated with multiple elements in the first feature vector; a third acquisition module 504, configured to: obtain a second feature vector of the speech based on the first feature vector and the covariance matrix; a fourth acquisition module 505, configured to: obtain a parameter vector representing the importance of multiple elements in the first feature vector; and a splicing module 506, configured to: splice the second feature vector with the parameter vector to obtain speech features.
[0134] According to some embodiments, the multiple frames of speech data are extracted from audio data about a speaker, and the multiple frames of speech data include data of multiple frames having a temporal contextual relationship, wherein the data of the multiple frames correspond to multiple frames of audio data.
[0135] The operations performed by the above modules 501 to 506 are similar to those of the reference Figure 2 The described steps S201 to S211 correspond to each other, so the details of each aspect thereof will not be repeated.
[0136] Figure 6 A block diagram of a voiceprint feature extraction device 600 according to another embodiment of the present disclosure is shown. Figure 6 The modules 601 to 606 shown may correspond to Figure 5Modules 501 to 506 are shown. In addition, modules 602 to 606 may also include further sub-functional modules, which will be described in detail below.
[0137] According to some embodiments, the first acquisition module includes 602: a first submodule 6021, configured to: input multiple frames of speech data into a first neural network to perform feature extraction through the first neural network.
[0138] According to some embodiments, the second acquisition module 603 includes: a second submodule 6031 configured to: input the first feature vector into the second neural network to calculate the covariance matrix through the second neural network.
[0139] According to some embodiments, the third acquisition module 604 includes: a third submodule 6041, configured to: apply a normalized exponential function to the covariance matrix to obtain a normalized exponential value about the covariance matrix; and a fourth submodule 6042, configured to: multiply the normalized exponential value with the first eigenvector to obtain a second eigenvector.
[0140] According to some embodiments, the fourth acquisition module 605 includes: a fifth sub-module 6051, configured to: input the first feature vector into the third neural network, wherein the third neural network includes a first sub-neural network and a second sub-neural network, and wherein the fifth sub-module includes: a sixth sub-module, configured to: input the first feature vector into the first sub-neural network to obtain a normalized weight vector of the first feature vector; and a seventh sub-module, configured to: input the normalized weight vector and the first feature vector into the second sub-neural network to calculate a parameter vector.
[0141] According to some embodiments, the parameter vector includes at least one of a weighted mean vector and a weighted standard deviation vector
[0142] According to some embodiments, the sixth submodule includes: an eighth submodule, configured to: obtain an initial weight vector of the first feature vector through the first sub-neural network; and a ninth submodule, configured to: apply a normalized exponential function to the initial weight vector to obtain a normalized weight vector.
[0143] According to some embodiments, the second feature vector has multiple dimensions, and the parameter vector has multiple dimensions, and wherein the splicing module 606 includes: a tenth submodule 6061, configured to: generate speech features having a target dimension, wherein the target dimension is the sum of the dimensions of the second feature vector and the parameter vector.
[0144] According to another aspect of the present disclosure, a device for training a speech feature extraction model is also provided. Figure 7 A block diagram of an apparatus 700 for training a speech feature extraction model according to an embodiment of the present disclosure is shown.
[0145] like Figure 7 As shown, the device 700 may include: a sample framing module 701, configured to: obtain sample speech that has been sampled and to frame the sample speech to obtain multiple frames of sample speech data; a first sample acquisition module 702, configured to: perform feature extraction on the multiple frames of sample speech data to obtain a first sample feature vector of the sample speech, wherein the first sample feature vector includes multiple sample elements corresponding to corresponding features of the multiple frames of sample speech data; a second sample acquisition module 703, configured to: obtain a sample covariance matrix associated with the multiple sample elements in the first sample feature vector; a third sample acquisition module 704, configured to: extract the first sample feature vector from the multiple frames of sample speech data to obtain a first sample feature vector of the sample speech, wherein the first sample feature vector includes multiple sample elements corresponding to corresponding features of the multiple frames of sample speech data; a second sample acquisition module 705, configured to: obtain a sample covariance matrix associated with the multiple sample elements in the first sample feature vector; and a third sample acquisition module 706. The acquisition module 704 is configured to: acquire the second sample feature vector of the sample speech based on the first sample feature vector and the sample covariance matrix; the fourth sample acquisition module 705 is configured to: acquire the sample parameter vector representing the importance of the multiple sample elements in the sample first feature vector; the sample splicing module 706 is configured to: splice the second sample feature vector with the sample parameter vector to obtain the speech feature of the sample speech; and the parameter adjustment module 707 is configured to: acquire the network parameters for updating the speech feature extraction model based on the speech feature to train the speech feature extraction model.
[0146] According to another aspect of the present disclosure, an electronic device is also provided, comprising at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions that can be executed by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor executes the method as described above.
[0147] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable a computer to execute the method as described above.
[0148] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, wherein the computer program implements the method as described above when executed by a processor.
[0149] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0150] refer to Figure 8, a block diagram of an electronic device 800 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0151] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0152] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. The input unit 806 may be any type of device capable of inputting information to the device 800, the input unit 806 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device, and may include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 807 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 808 may include but is not limited to a disk, an optical disk. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.
[0153] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as method 200 and method 300. For example, in some embodiments, method 200 and method 300 may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of method 200 and method 300 described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the method 200 and the method 300 in any other appropriate manner (eg, by means of firmware).
[0154] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0155] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0156] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0157] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0158] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0159] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0160] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0161] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.
Claims
1. A method for extracting speech features, comprising: Acquire the speech to be processed, and divide the speech into frames to obtain multiple frames of speech data; Performing feature extraction on the multiple frames of speech data to obtain a first feature vector of the speech, wherein the first feature vector includes a plurality of elements corresponding to corresponding features of the multiple frames of speech data respectively; Obtaining a covariance matrix associated with the plurality of elements in the first eigenvector; Based on the first eigenvector and the covariance matrix, obtaining a second eigenvector of the speech; Obtaining a parameter vector representing the importance of the plurality of elements in the first feature vector; and The second feature vector is concatenated with the parameter vector to obtain speech features of the speech.
2. The method according to claim 1, wherein: The second eigenvector is a posterior mean vector of the first eigenvector according to a Gaussian distribution.
3. The method according to claim 1 or 2, wherein: Based on the first eigenvector and the covariance matrix, obtaining a second eigenvector of the speech includes: applying a normalized exponential function to the covariance matrix to obtain a normalized exponential value for the covariance matrix; and The normalized exponent value is multiplied by the first eigenvector to obtain the second eigenvector.
4. The method according to claim 1 or 2, wherein: Obtaining a parameter vector characterizing the importance of the plurality of elements in the first feature vector comprises: The first feature vector is input into a third neural network, wherein the third neural network includes a first sub-neural network and a second sub-neural network, and wherein, Inputting the first feature vector into the first sub-neural network to obtain a normalized weight vector of the first feature vector; and The normalized weight vector and the first feature vector are input into the second sub-neural network to calculate the parameter vector.
5. The method according to claim 4, wherein: Inputting the first feature vector into the first sub-neural network to obtain a normalized weight vector of the first feature vector includes: Acquiring an initial weight vector of the first feature vector through the first sub-neural network; and A normalized exponential function is applied to the initial weight vector to obtain the normalized weight vector.
6. The method according to claim 1 or 2, wherein: The parameter vector includes: at least one of a weighted mean vector and a weighted standard deviation vector.
7. The method according to claim 1 or 2, wherein: The second feature vector has multiple dimensions, and the parameter vector has multiple dimensions, and wherein concatenating the second feature vector with the parameter vector to obtain the speech feature of the speech comprises: Generate speech features with a target dimension, wherein the target dimension is the sum of the dimensions of the second feature vector and the parameter vector.
8. A method for training a speech feature extraction model, comprising: Acquire sample speech that has been sample-labeled, and divide the sample speech into frames to obtain multiple frames of sample speech data; Performing feature extraction on the multiple frames of sample speech data to obtain a first sample feature vector of the sample speech, wherein the first sample feature vector includes a plurality of sample elements respectively corresponding to corresponding features of the multiple frames of sample speech data; Obtaining a sample covariance matrix associated with the plurality of sample elements in the first sample feature vector; Based on the first sample feature vector and the sample covariance matrix, obtaining a second sample feature vector of the sample speech; Obtaining a sample parameter vector representing the importance of the plurality of sample elements in the sample first feature vector; concatenating the second sample feature vector with the sample parameter vector to obtain the speech feature; and Based on the speech features, network parameters for updating the speech feature extraction model are acquired to train the speech feature extraction model.
9. A device for extracting speech features, comprising: A framing module is configured to: obtain the speech to be processed, and frame the speech to obtain multiple frames of speech data; A first acquisition module is configured to: perform feature extraction on the multiple frames of speech data to obtain a first feature vector of the speech, wherein the first feature vector includes a plurality of elements corresponding to corresponding features of the multiple frames of speech data respectively; A second acquisition module is configured to: acquire a covariance matrix associated with the plurality of elements in the first eigenvector; A third acquisition module is configured to: acquire a second eigenvector of the speech based on the first eigenvector and the covariance matrix; A fourth acquisition module is configured to: acquire a parameter vector representing the importance of the plurality of elements in the first feature vector; and The concatenation module is configured to concatenate the second feature vector with the parameter vector to obtain the speech feature of the speech.
10. The device according to claim 9, wherein: The second eigenvector is a posterior mean vector of the first eigenvector according to a Gaussian distribution.
11. The device according to claim 9 or 10, wherein: The third acquisition module includes: A third submodule is configured to: apply a normalized exponential function to the covariance matrix to obtain a normalized exponential value for the covariance matrix; and The fourth submodule is configured to: multiply the normalized exponent value by the first eigenvector to obtain the second eigenvector.
12. The device according to claim 9 or 10, wherein: The fourth acquisition module includes: A fifth submodule is configured to: input the first feature vector into a third neural network, wherein the third neural network includes a first sub-neural network and a second sub-neural network, and wherein the fifth submodule includes: A sixth submodule is configured to: input the first feature vector into the first sub-neural network to obtain a normalized weight vector of the first feature vector; and The seventh submodule is configured to: input the normalized weight vector and the first feature vector into the second sub-neural network to calculate the parameter vector.
13. The device according to claim 12, wherein: The sixth submodule comprises: An eighth submodule is configured to: obtain an initial weight vector of the first feature vector through the first sub-neural network; and A ninth submodule is configured to: apply a normalized exponential function to the initial weight vector to obtain the normalized weight vector.
14. The device according to claim 13, wherein: The parameter vector includes: at least one of a weighted mean vector and a weighted standard deviation vector.
15. The device according to claim 9 or 10, wherein: The second feature vector has multiple dimensions, and the parameter vector has multiple dimensions, and wherein the stitching module includes: The tenth submodule is configured to: generate speech features with a target dimension, wherein the target dimension is the sum of the dimensions of the second feature vector and the parameter vector.
16. A device for training a speech feature extraction model, comprising: The sample framing module is configured to: obtain sample speech that has been sample-labeled, and frame the sample speech to obtain multiple frames of sample speech data; A first sample acquisition module is configured to: perform feature extraction on the multiple frames of sample speech data to acquire a first sample feature vector of the sample speech, wherein the first sample feature vector includes a plurality of sample elements corresponding to corresponding features of the multiple frames of sample speech data; A second sample acquisition module is configured to: acquire a sample covariance matrix associated with the plurality of sample elements in the first sample feature vector; A third sample acquisition module is configured to: acquire a second sample feature vector of the sample speech based on the first sample feature vector and the sample covariance matrix; A fourth sample acquisition module is configured to: acquire a sample parameter vector representing the importance of the plurality of sample elements in the sample first feature vector; A sample concatenation module is configured to: concatenate the second sample feature vector with the sample parameter vector to obtain the speech feature of the sample speech; and The parameter adjustment module is configured to: obtain network parameters for updating the speech feature extraction model based on the speech features to train the speech feature extraction model.
17. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.
18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.
19. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
