Method and apparatus for generating voice feature, storage medium and electronic device

Abstract

The application discloses a speech feature generation method and device, a storage medium and an electronic device, relates to the technical field of smart homes, and comprises the following steps: acquiring a plurality of speech samples; using the plurality of speech samples to train an initial feature generation network and an initial recognition network connected in sequence to obtain a target feature generation network and a target recognition network connected in sequence; using the target feature generation network to generate speech features corresponding to each speech file in a plurality of speech files, and obtaining a plurality of speech features; and extracting a voiceprint from the plurality of speech features to obtain one or more user voiceprints. By using the technical solution, the problem of low accuracy of speech features of speech files in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to a method and apparatus for generating voice features, a storage medium, and an electronic device. Background Technology

[0002] In intelligent voice dialogue systems, voiceprint recognition and applications are becoming increasingly widespread, but they are mostly based on registered voiceprints. Traditional registered voiceprints are the most widely used in current devices. The method typically involves a user registering a specific voiceprint using a device, such as a mobile phone. Subsequent use can then be confirmed by comparing features from the registered audio files. However, today's society has increasingly higher demands for intelligent voice technology, and users increasingly expect to receive services provided by smart homes in a seamless manner. Unregistered voiceprints involve collecting and accumulating users' voice files using voice acquisition devices. These files are then clustered based on the user's voice characteristics to obtain the core features of each cluster, which are used as the voiceprint features for a particular user. However, due to the widespread use of smart devices, the collected voice files can be obtained from different voice acquisition devices—that is, cross-channel device acquisition. This affects the clustering of voice and text, leading to inaccurate clustering results and impacting subsequent user voice recognition, severely affecting the user experience.

[0003] There is still no effective solution to the problem of low accuracy of speech features in the generated speech files in related technologies. Summary of the Invention

[0004] This application provides a method and apparatus for generating speech features, a storage medium, and an electronic device to at least solve the problem of low accuracy of speech features in generated speech files in related technologies.

[0005] According to one embodiment of this application, a method for generating speech features is provided, comprising: acquiring multiple speech samples, wherein each of the multiple speech samples includes speech emitted by a single user; inputting the multiple speech samples into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result; adjusting the network parameters of the initial feature generation network and the initial recognition network according to the initial recognition result and sample labels to obtain a target feature generation network and a target recognition network connected in sequence, wherein the target feature generation network is used to generate target speech features corresponding to each of the speech samples, the target recognition network is used to identify that the target speech features belong to a single user, and the sample labels are used to indicate the number of users present in the speech samples; using the target feature generation network to generate speech features corresponding to each of the multiple speech files to obtain multiple speech features; and performing voiceprint extraction on the multiple speech features to obtain one or more user voiceprints, wherein each user voiceprint is used to identify a reference user.

[0006] Optionally, inputting multiple speech samples into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result includes: inputting the speech samples into the initial feature generation network to obtain initial speech features corresponding to the speech samples; inputting the initial speech features into an initial attractor generation network to obtain an initial attractor generated by the initial attractor generation network, wherein the initial recognition network includes the initial attractor generation network, the initial attractor is used to indicate the user present in the speech sample, and the initial recognition result includes the initial attractor; adjusting the network parameters of the initial feature generation network and the initial recognition network according to the initial recognition result and sample labels to obtain a target feature generation network and a target recognition network connected in sequence includes: calculating the target probability corresponding to the initial attractor, wherein the target probability is the probability that the initial attractor indicates that the user present in the speech sample is a single user; adjusting the network parameters of the initial feature generation network and the initial attractor generation network according to the relationship between the target probability and the reference probability to obtain the target feature generation network and the target attractor generation network, wherein the reference probability is the probability that the user present in the speech sample is a single user, and the sample label includes the reference probability.

[0007] Optionally, adjusting the network parameters of the initial feature generation network and the initial attractor generation network based on the relationship between the target probability and the reference probability to obtain the target feature generation network and the target attractor generation network includes: obtaining a set of target probabilities and reference probabilities with corresponding relationships for each speech sample, obtaining multiple sets of target probabilities and reference probabilities with corresponding relationships; calculating the target loss value of the multiple speech samples based on the multiple sets of target probabilities and reference probabilities with corresponding relationships; adjusting the network parameters of the initial feature generation network and the initial attractor generation network according to the target loss value until the target loss value converges, thereby obtaining the target feature generation network and the target attractor generation network.

[0008] Optionally, calculating the target loss value of the plurality of speech samples based on the plurality of sets of corresponding target probabilities and reference probabilities includes: substituting each set of reference probabilities and target probabilities into a target loss function to obtain an initial loss value corresponding to each speech sample; determining a weight value corresponding to the initial loss value of each speech sample, wherein the weight value is used to indicate the degree of outlier of the initial loss value of the corresponding speech sample among the plurality of initial loss values ​​of the plurality of speech samples, and the higher the degree of outlier, the smaller the corresponding weight value; and determining the target loss value as the weighted sum of the initial loss value and the corresponding weight value.

[0009] Optionally, determining the weight value corresponding to the initial loss value of each speech sample includes: determining the maximum value of the difference between the reference probability and the corresponding target probability as the gradient magnitude; dividing the gradient magnitude into a continuous gradient interval of a target number, wherein the target number is determined based on the number of multiple speech samples; and determining the weight value corresponding to the speech sample in each gradient interval as the ratio of the interval length of each gradient interval to the number of speech samples falling within each gradient interval.

[0010] Optionally, generating speech features corresponding to each speech file in multiple speech files using the target feature generation network includes: dividing each speech file into frames to obtain multiple speech frames; inputting the multiple speech frames into the target feature generation network to obtain initial sub-speech features output by the target feature generation network corresponding to each speech frame; and performing dimensionality reduction processing on each initial sub-speech feature to obtain multiple target sub-speech features, wherein the speech features corresponding to each speech file include multiple target sub-speech features.

[0011] Optionally, after extracting voiceprints from the multiple speech features, the method further includes: when a target speech is acquired, matching the target speech with one or more user voiceprints; if the target speech matches the target user voiceprint among the one or more user voiceprints, registering the user corresponding to the target user voiceprint; if the target speech does not match the one or more user voiceprints, identifying the target speech as the speech file.

[0012] According to another embodiment of this application, a speech feature generation apparatus is also provided, comprising: an acquisition module for acquiring multiple speech samples, wherein each of the multiple speech samples includes speech emitted by a single user; an input module for inputting the multiple speech samples into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result; an adjustment module for adjusting the network parameters of the initial feature generation network and the initial recognition network according to the initial recognition result and sample labels to obtain a target feature generation network and a target recognition network connected in sequence, wherein the target feature generation network is used to generate target speech features corresponding to each of the speech samples, the target recognition network is used to identify that the target speech features belong to a single user, and the sample labels are used to indicate the number of users present in the speech samples; a generation module for using the target feature generation network to generate speech features corresponding to each of the multiple speech files to obtain multiple speech features; and an extraction module for performing voiceprint extraction on the multiple speech features to obtain one or more user voiceprints, wherein each user voiceprint is used to identify a reference user.

[0013] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described speech feature generation method at runtime.

[0014] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the speech feature generation method through the computer program.

[0015] In this embodiment, multiple speech samples are acquired, each of which includes speech from a single user. The multiple speech samples are input into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result. The network parameters of the initial feature generation network and the initial recognition network are adjusted according to the initial recognition result and sample labels to obtain a target feature generation network and a target recognition network connected in sequence. The target feature generation network generates target speech features corresponding to each speech sample, and the target recognition network identifies that the target speech features belong to a single user. The sample labels indicate the number of users present in the speech samples. The target feature generation network is used to generate speech features corresponding to each of the multiple speech files, resulting in multiple speech features. Voiceprint extraction is performed on the multiple speech features to obtain one or more user voiceprints. The user's voiceprint is used to identify a reference user. Each speech sample includes speech from a single user. A target feature generation network trained on multiple speech samples generates speech features corresponding to each speech sample in multiple speech files. During the training phase, the input values ​​of the speech samples are sequentially connected to an initial feature generation network and an initial recognition network. The network parameters of the initial feature generation network and the initial recognition network are adjusted based on the obtained initial recognition results and sample labels, thus completing the training of the network model. The trained target feature generation network can generate target speech features corresponding to each speech sample, and the trained target recognition network can identify that the target speech features belong to a single user, ensuring the recognizability of the speech features output by the target feature generation network. Therefore, using this target feature generation model to generate speech features corresponding to each speech file in multiple speech files, and extracting voiceprints from multiple speech features, results in more accurate voiceprints. This technical solution solves the problem of low accuracy in generating speech features for speech files in related technologies, achieving the technical effect of improving the accuracy of speech features in generated speech files. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1This is a schematic diagram of the hardware environment for a method of generating speech features according to an embodiment of this application;

[0019] Figure 2 This is a flowchart of a method for generating speech features according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of an optional network model according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of another optional network model according to an embodiment of this application;

[0022] Figure 5 This is an optional unregistered voiceprint recognition flowchart according to an embodiment of this application;

[0023] Figure 6 This is a structural block diagram of a speech feature generation apparatus according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] According to one aspect of the embodiments of this application, a method for generating voice features is provided. This method is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligencehouse ecosystems. Optionally, in this embodiment, the above method can be applied to, for example... Figure 1The hardware environment shown consists of terminal device 102 and server 104. Figure 1 This is a schematic diagram of the hardware environment for a speech feature generation method according to an embodiment of this application. Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0027] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0028] This embodiment provides a method for generating voice features, applied to the aforementioned device terminal. Figure 2 This is a flowchart of a method for generating speech features according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0029] Step S202: Obtain multiple voice samples, wherein each of the multiple voice samples includes voice spoken by a single user;

[0030] Step S204: Input the multiple speech samples into the sequentially connected initial feature generation network and initial recognition network to obtain the initial recognition result;

[0031] Step S206: Adjust the network parameters of the initial feature generation network and the initial recognition network according to the initial recognition result and the sample label to obtain a target feature generation network and a target recognition network connected in sequence. The target feature generation network is used to generate target speech features corresponding to each speech sample. The target recognition network is used to identify that the target speech features belong to a single user. The sample label is used to indicate the number of users present in the speech sample.

[0032] Step S208: Use the target feature generation network to generate speech features corresponding to each speech file in multiple speech files, thereby obtaining multiple speech features;

[0033] Step S210: Extract voiceprints from the multiple speech features to obtain one or more user voiceprints, wherein each user voiceprint is used to identify a reference user.

[0034] Through the above steps, each speech sample includes speech from a single user. A target feature generation network trained on multiple speech samples generates speech features corresponding to each speech sample in multiple speech files. During the training phase, the input values ​​of the speech samples are sequentially connected to an initial feature generation network and an initial recognition network. The network parameters of the initial feature generation network and the initial recognition network are adjusted based on the obtained initial recognition results and sample labels, thus completing the training of the network model. The trained target feature generation network can generate target speech features corresponding to each speech sample, and the trained target recognition network can identify that the target speech features belong to a single user, ensuring the recognizability of the speech features output by the target feature generation network. Therefore, using this target feature generation model to generate speech features corresponding to each speech file in multiple speech files, and extracting voiceprints from multiple speech features, results in more accurate voiceprints. This technical solution solves the problem of low accuracy in generating speech features for speech files in related technologies, achieving the technical effect of improving the accuracy of speech features in generated speech files.

[0035] In the technical solution provided in step S202 above, multiple voice samples can be multiple voice data collected by a voice acquisition device, or they can be obtained by transforming the voice data collected by a voice acquisition device. For example, voice feature adjustment or timbre adjustment can be performed on the voice collected by a voice acquisition device to simulate the voice data collected by multiple different voice acquisition devices (i.e., cross-channel devices) from the same user.

[0036] Optionally, in this embodiment, multiple voice samples may be voice data from the same user, or they may be voice data from multiple different users. This solution does not limit this.

[0037] In the technical solution provided in step S204 above, the initial feature generation network and the initial recognition network that connect multiple speech sample input values ​​in sequence can first use the speech sample input values ​​to the initial feature generation network to obtain the features of the speech samples output by the initial feature generation network, and then input the features output by the initial feature generation network to the initial recognition network to obtain the initial recognition result.

[0038] Optionally, in this embodiment, the initial recognition network is used to assist in training the initial feature generation model, thereby improving the feature recognition accuracy of the target speech features output by the target feature generation network.

[0039] In the technical solution provided in step S206 above, the initial identification result and the network of the initial identification network...

[0040] Optionally, in this embodiment, adjusting the network parameters based on the initial recognition results and sample labels can be achieved by constructing a loss function using the initial recognition results and sample labels, and adjusting the network parameters of the initial feature generation network and the initial recognition network according to a method that matches the loss function value. For example, a loss function can be constructed based on the output of the initial recognition network and the sample labels of the speech samples. The value of the loss function can be used to determine whether the initial feature generation network and the initial recognition network have converged, whether the network parameters of the network model need to be adjusted, and / or, based on the first network parameter adjustment direction and first adjustment step size of the initial feature generation network that matches the value of the loss function, and based on the second network parameter adjustment direction and second adjustment step size of the initial recognition network that matches the value of the loss function, the network parameters of the initial feature generation network can be adjusted according to the first network parameter adjustment direction and first adjustment step size, and the network parameters of the initial recognition network can be adjusted according to the second network parameter adjustment direction and second step size.

[0041] Optionally, in this embodiment, the voice features are features carried in the voice data that indicate the user to whom the voice data belongs, and may include, but are not limited to, timbre, pitch, frequency, rhythm changes, etc.

[0042] Optionally, in this embodiment, the initial feature generation network may be, but is not limited to, an SA-EEND (End-to-End Neural Speaker Diarization with Self-Attention) network, a Transformer network, etc., and this solution does not limit it.

[0043] Optionally, in this embodiment, the initial recognition network may be, but is not limited to, a network model composed of one or more sub-networks. For example, the initial recognition network may include an initial attractor recognition network and / or an initial user recognition network. The initial attractor recognition network generates initial attractors based on the initial speech features output by the initial feature generation network. The initial attractors indicate the users present in the speech samples. The initial user recognition network predicts the users corresponding to the speech samples based on the initial speech features generated by the initial feature generation network and the initial attractors generated by the initial attractor generation network. When the initial recognition network includes both the initial attractor generation network and the initial user recognition network, the network parameters are adjusted accordingly. During correction, a first loss value can be calculated using the users predicted by the initial user identification network in the speech samples and the users labeled in the speech samples. Based on this loss value, the adjustment direction and adjustment step size of the network parameters of the initial feature generation network and the initial attractor generation network are determined. The network parameters of the initial feature generation network and the initial attractor generation network are adjusted according to the adjustment direction and adjustment step size. Furthermore, the target probability corresponding to the initial attractor can be calculated by calculating the initial attractor. This target probability represents the probability that there is a single user in the speech sample. A second loss value is calculated based on the target probability and the number of users labeled in the speech samples to determine whether the initial feature generation network and the initial attractor generation network have converged. For example, when the initial recognition network is only the initial attractor generation network, when correcting the network parameters, the target probability corresponding to the initial attractor can be calculated by calculating the initial attractor. This target probability represents the probability that there is a single user in the speech sample. The third loss value is calculated based on the target probability and the number of users labeled in the speech sample. The network parameters of the initial feature generation network and the initial attractor generation network are adjusted based on the third loss value. Finally, the convergence of the initial feature generation network and the initial attractor generation network is determined based on the third loss value.

[0044] Optionally, in this embodiment, the target recognition network outputs the result of identifying that the target speech feature belongs to a single user. For example, the target recognition network may output the attractor corresponding to the target speech feature, or it may output the probability value of the target speech feature belonging to a single user, or it may output the user to which the target speech feature belongs. This solution does not limit this.

[0045] In the technical solution provided in step S208 above, multiple voice files can be collected by one or more voice acquisition devices, and multiple voice files can belong to one or more users.

[0046] Optionally, in this embodiment, the target feature generation network can generate speech features corresponding to each speech file by directly inputting each speech file into the target feature generation network, thereby determining the output of the target feature generation network as a speech feature. Alternatively, the speech file can be divided into multiple frames according to time order, and the multiple speech frames can be input into the target feature generation network in time order to obtain multiple sub-speech features output by the target feature generation network. The multiple sub-speech features corresponding to each speech file are then subjected to dimensionality reduction processing, and feature transformation is performed on the reduced multiple sub-speech features to obtain a speech feature used to indicate the speech text. This solution does not limit this.

[0047] In the technical solution provided in step S210 above, the method of extracting voiceprints from multiple speech features may include, but is not limited to, clustering multiple speech features and taking a set of speech features with the same feature type obtained after clustering as a user voiceprint, or each speech feature may include multiple dimensions of sub-features, and extracting the target sub-feature of the target dimension from the multiple dimensions of sub-features as the user voiceprint. For example, the speech features may include multiple dimensions of sub-features such as pitch features, frequency features, rhythm change features, and timbre features, and timbre features may be selected from these sub-features as the user voiceprint.

[0048] As an optional embodiment, multiple speech samples are input into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result, including: inputting the speech samples into the initial feature generation network to obtain initial speech features corresponding to the speech samples; inputting the initial speech features into an initial attractor generation network to obtain initial attractors generated by the initial attractor generation network, wherein the initial recognition network includes the initial attractor generation network, the initial attractors are used to indicate the users present in the speech samples, and the initial recognition result includes the initial attractors;

[0049] The step of adjusting the network parameters of the initial feature generation network and the initial recognition network based on the initial recognition result and sample labels to obtain a target feature generation network and a target recognition network connected in sequence includes: calculating the target probability corresponding to the initial attractor, wherein the target probability is the probability that the initial attractor indicates that the user present in the speech sample is a single user; adjusting the network parameters of the initial feature generation network and the initial attractor generation network according to the relationship between the target probability and the reference probability to obtain the target feature generation network and the target attractor generation network, wherein the reference probability is the probability that the user present in the speech sample is a single user, and the sample labels include the reference probability.

[0050] Optionally, in this embodiment, inputting speech samples into the initial feature generation network can be done by directly inputting each speech sample into the initial feature generation network, thereby determining the output of the initial feature generation network as the initial speech feature. Alternatively, each speech sample can be divided into multiple frames according to time sequence, and the multiple speech frames of each speech sample can be sequentially input into the initial feature generation network according to time sequence to obtain multiple sub-speech features output by the target feature generation network. The multiple sub-speech features corresponding to each speech file are then subjected to dimensionality reduction processing, and the multiple sub-speech features after dimensionality reduction are used as the initial speech features corresponding to the speech sample. This solution does not limit this.

[0051] Optionally, in this embodiment, the relationship between the target probability and the reference probability can be used to determine whether the initial feature generation network and the initial attractor generation network have converged (whether the network parameters need to be adjusted), and / or how to adjust the network parameters of the network model. For example, the target high probability and the reference probability are calculated to obtain the loss values ​​of the initial feature generation network and the initial attractor generation network. Based on the loss values, it is determined whether the initial feature generation network and the initial attractor generation network have converged, and / or, the first adjustment direction and the first adjustment step size of the network parameters of the initial feature generation network corresponding to the loss value are determined, as well as the second adjustment direction and the second adjustment step size of the initial attractor generation network. The network parameters of the initial feature generation network are adjusted according to the first adjustment direction and the first adjustment step size, and the network parameters of the second feature generation network are adjusted according to the second adjustment direction and the second adjustment step size.

[0052] Figure 3 This is a schematic diagram of an optional network model according to an embodiment of this application, such as... Figure 3 As shown, the initial feature generation network is the SA-EEND network, and the initial recognition network is the initial attractor generation network, i.e., the encoder-decoder based attractor calculation network in the figure. Each speech sample is divided into T frames (i.e., X1 to X2 in the figure). T The input is fed into the initial feature generation network to obtain the speech features (i.e., the feature vectors e1 to e2 in the figure). TThe initial speech features are input into the LSTM encoder (Long Short-Term Memory encoder) of the initial attractor generation network. The LSTM encoder encodes the speech features with contextual relevance in the initial speech features, and obtains the target encoded data (h0 and c0) output by the LSTM encoder. Here, h0 is the hidden state short-term memory data, which is used to indicate the recent memory between the frames included in the initial speech features, and c0 is the cell state long-term memory data, which is used to indicate the long-term memory between the frames included in the initial speech features. The target encoded data (h0 and c0) is transmitted to the LSTM decoder (Long Short-Term Memory decoder). Since the LSTM decoder has no initial input, zero vectors (0, 0, ..., 0, 0) are added below the LSTM decoder as initial input, resulting in the initial attractors (a1, a2, ..., as, as+1) output by the LSTM decoder. In this real-time example, since the sample speech only includes the speech of one user, only one initial attractor is included. The target probability (Attractor existence) corresponding to each attractor vector is calculated through an activation function. Since the sample speech contains only one user's speech, there is only one target probability. The initial attractor of this target probability represents the probability that the user in the speech sample is a single user. The speech sample is marked with the reference probability at the corresponding position, which is the Label at the corresponding position in the figure (the Label can be 0 or 1). Then, the target loss value of the network model is constructed based on the target probability and the reference probability of all speech samples. The target loss value is used to determine whether the initial feature generation network and the initial attractor generation network have converged, and / or the adjustment method of the network parameters of the initial feature generation network and the initial attractor generation network.

[0053] This embodiment employs the GHM (gradient harmonizing mechanism) loss function to calculate the target loss value. This loss function addresses the problem in related technologies where excessive focus on outliers leads to inaccurate network model training. By introducing a gradient density formula, this problem is harmonized and resolved. The GHM loss function is based on the principle of Focal loss, introducing the idea of ​​reducing the loss of samples with confidence (p). However, Focal loss models have problems due to their excessive focus on outlier samples. Furthermore, the values ​​of γ and α are entirely experimental and therefore difficult to adjust. This function introduces a large number of samples for attenuation. This has little impact on the probability of a large number of samples, but it plays a significant role in adjusting the data for samples with a small amount of data, similar to the ARPA model training principle of LM. In this embodiment, attenuation is performed on a certain number of samples with confidence p, rather than attenuating all samples; the large number of samples are attenuated.

[0054] Gradient density GD(g):

[0055]

[0056] δ ∈ (g k (g) represents the gradient magnitude distribution in samples 1 to N. Number of samples within the range; ∈ (g) indicates The length of the interval (gradient interval), in this embodiment, is a single l ∈ The length of (g) is the value obtained by dividing the gradient magnitude equally according to the number of samples;

[0057] Gradient magnitude g:

[0058]

[0059] p is the target probability predicted by the model, p * These are the labels (reference probabilities) of the ground-truth in the graph; p * The value can be 0 or 1; therefore, the physical meaning of gradient density GD(g) is: the number of samples per unit gradient magnitude g.

[0060] GHM's loss function:

[0061]

[0062] Figure 4 This is a schematic diagram of another optional network model according to an embodiment of this application, such as... Figure 4As shown, the initial feature generation network is an SA-EEND network, and the initial recognition network includes an initial attractor generation network (i.e., an encoder-decoder based attractor calculation network in the figure) and an initial user recognition network. Each speech sample is divided into T frames (i.e., X1 to X2 in the figure). T The input is fed into the initial feature generation network to obtain the speech features (i.e., the feature vectors e1 to e2 in the figure). TThe initial speech features are input into the LSTM encoder (Long Short-Term Memory encoder) of the initial attractor generation network. The LSTM encoder encodes the speech features with contextual relevance in the initial speech features, and obtains the target encoded data (h0 and c0) output by the LSTM encoder. Here, h0 is the hidden state short-term memory data, which is used to indicate the recent memory between the frames included in the initial speech features, and c0 is the cell state long-term memory data, which is used to indicate the long-term memory between the frames included in the initial speech features. The target encoded data (h0 and c0) are transmitted to the LSTM decoder (Long Short-Term Memory decoder). Since the LSTM decoder has no initial input, zero vectors (0, 0, ..., 0, 0) are added below the LSTM decoder as initial input, resulting in the initial attractors (a1, a2, ..., as, as+1) output by the LSTM decoder. In this real-time example, since the sample speech only includes the speech of one user, there is only one initial attractor. The target probability (Attractor existence probabilities (p1, p2, ..., ps, ps+1) corresponding to each attractor vector is calculated through the activation function. Since the sample speech only includes the speech of one user, there is only one target probability. This target probability initial attractor represents the probability that the user in the speech sample is a single user. The speech sample is marked with the reference probability at the corresponding position, i.e., the Labels at the corresponding position in the figure (here, the Labels can take the value 0 or 1). The initial user identification network multiplies the initial speech features and the initial attractor, and activates the result using an activation function to predict the user corresponding to the speech sample, i.e., the Diarization results in the figure. Then, based on the users finally predicted from all speech samples and the users labeled on the speech samples, the accuracy of the model can be determined, and the loss value of the network model can be constructed. This determines the first adjustment direction and first adjustment step size of the network parameters of the initial feature generation network, and the second adjustment direction and second adjustment step size of the network parameters of the initial attractor generation network. The network parameters of the initial feature generation network and the initial attractor generation network are adjusted, and the target loss value of the network model is constructed based on the target probability and reference probability of all speech samples. The target loss value is used to determine whether the initial feature generation network and the initial attractor generation network have converged and whether parameter adjustment is needed.

[0063] As an optional embodiment, adjusting the network parameters of the initial feature generation network and the initial attractor generation network based on the relationship between the target probability and the reference probability to obtain the target feature generation network and the target attractor generation network includes:

[0064] Obtain a set of target probabilities and reference probabilities with corresponding relationships for each speech sample, and obtain multiple sets of target probabilities and reference probabilities with corresponding relationships;

[0065] The target loss value of the multiple speech samples is calculated based on the multiple sets of target probabilities and reference probabilities that have corresponding relationships.

[0066] The network parameters of the initial feature generation network and the initial attractor generation network are adjusted according to the target loss value until the target loss value converges, thereby obtaining the target feature generation network and the target attractor generation network.

[0067] Optionally, in this embodiment, the reference probability can be determined based on the sample label of the voice sample. The sample label of the voice sample can be the user information corresponding to the voice sample, or the label value of the voice sample belonging to a single user. For example, the label value can be "0" and "1", where "0" indicates that the voice sample does not belong to a single user, and "1" indicates that the voice sample belongs to a single user.

[0068] Optionally, in this embodiment, the target loss value may include, but is not limited to, cross-entropy loss value, GHM (gradient harmonizing mechanism) loss value, MSE (mean squared error) loss value, etc., and this solution does not limit it.

[0069] As an optional embodiment, calculating the target loss value of the plurality of speech samples based on the plurality of sets of target probabilities and reference probabilities with corresponding relationships includes:

[0070] Substitute each set of reference probabilities and target probabilities into the target loss function to obtain the initial loss value corresponding to each speech sample;

[0071] Determine a weight value corresponding to the initial loss value of each speech sample, wherein the weight value is used to indicate the degree of outlier of the initial loss value of the corresponding speech sample among the multiple initial loss values ​​of multiple speech samples, and the higher the degree of outlier, the smaller the corresponding weight value;

[0072] The target loss value is determined by the weighted sum of the initial loss value and the corresponding weight value.

[0073] Optionally, in this embodiment, the target loss function may be, but is not limited to, the cross-entropy loss function, the mean squared error loss function, etc., and this solution does not limit it.

[0074] Optionally, in this embodiment, the weight value can be determined based on the initial loss value of each speech sample, or it can be determined based on the difference between the reference probability and the target probability of each speech sample. That is, the difference between the reference probability and the target probability of each speech sample indicates the initial loss value of the speech sample. For example, the method of determining the weight value based on the initial loss value can be by clustering the initial loss values ​​of each speech sample to obtain the cluster centers of multiple speech samples, determining the outliers or the degree of outlier of multiple speech samples by calculating the distance from the cluster centers to the initial loss values ​​of each speech sample, and determining the weight value of each speech sample from the corresponding outliers and weight values. The weight value corresponding to the outlier, or the target distance value between the cluster center and the initial loss value of each speech sample can be calculated, and the weight value corresponding to the target distance value can be determined from the corresponding distance values ​​and weight values; the method of determining the weight value based on the difference between the reference probability and the target probability of each pseudo-sample can be to determine the gradient magnitude of multiple speech samples (the absolute value of the difference between the reference probability and the target probability), select the largest gradient magnitude from multiple samples as the target gradient magnitude, divide the target gradient length into multiple segments, calculate the gradient density of each segment (the ratio of the number of speech samples falling into each segment to the length of each segment), and determine the weight value corresponding to each speech sample based on the gradient density.

[0075] As an optional embodiment, determining the weight value corresponding to the initial loss value for each of the speech samples includes:

[0076] The maximum value of the difference between the reference probability and the corresponding target probability is determined as the gradient magnitude;

[0077] The gradient magnitude is divided into a target number of continuous gradient intervals, wherein the target number is determined based on the number of multiple speech samples;

[0078] The ratio of the length of each gradient interval to the number of speech samples falling within each gradient interval is determined as the weight value corresponding to the speech sample within each gradient interval.

[0079] Optionally, in this embodiment, the target number can be the number of samples from multiple speech samples.

[0080] Optionally, in this embodiment, the difference between the reference probability and the target probability of different speech samples is different. Therefore, the gradient magnitude with the largest difference between the reference probability and the target probability is selected from each speech sample as the gradient magnitude of all speech samples.

[0081] As an optional embodiment, generating speech features for each speech file in multiple speech files using the target feature generation network includes:

[0082] Each of the audio files is divided into frames to obtain multiple audio frames;

[0083] Multiple speech frames are input into the target feature generation network to obtain initial sub-speech features output by the target feature generation network corresponding to each speech frame;

[0084] Each of the initial sub-speech features is subjected to dimensionality reduction processing to obtain multiple target sub-speech features, wherein the speech features corresponding to each speech file include multiple target sub-speech features.

[0085] Optionally, in this embodiment, the audio file can be framed at fixed intervals, that is, the audio file can be divided into multiple segments of equal size, or it can be framed based on the sudden change in the volume of the audio in the audio text. For example, when the difference in volume between two consecutive frames is detected to be greater than a target threshold, framing begins.

[0086] As an optional embodiment, after extracting the voiceprints from the plurality of speech features, the method further includes:

[0087] When the target speech is collected, the target speech is matched with the voiceprints of one or more users;

[0088] If the target speech matches the target user's voiceprint in one or more user voiceprints, the user corresponding to the target user's voiceprint is registered.

[0089] If the target speech does not match the voiceprints of one or more users, the target speech is identified as the speech file.

[0090] Optionally, in this embodiment, when the target voice is consistent with the target user's voiceprint in one or more user voiceprints, it can be determined that the target user who sent the target voice and the reference user corresponding to the target user's voiceprint are the same user. At this time, it is detected whether the reference user has completed voiceprint registration. If the voiceprint registration has not been completed, the target user is registered (for example, first play the script "Hello, please complete voiceprint registration", and then execute the registration process to guide the target user to complete voiceprint registration). If the voiceprint registration is completed, the target user's target voice is responded to. For example, if the target voice is a voice that asks a smart device A to perform a certain operation, then "Hello, xxx, device A is performing XX operation" can be replied.

[0091] Figure 5 This is an optional unregistered voiceprint recognition flowchart according to an embodiment of this application, such as... Figure 5 As shown, it includes at least the following steps:

[0092] Step S501: When a user uses the smart home system, they will interact with home devices via voice. The voice recordings used during these interactions will be temporarily stored in the cloud according to the specific home.

[0093] In step S502, for each user interaction, the cloud performs filtering based on the number of characters and signal-to-noise ratio of the recognition results; if the signal-to-noise ratio is negative or the number of recognized characters is less than 4, the audio file will be deleted by the cloud.

[0094] Step S503: Detect whether the voice file is greater than the target number. For example, if the voice text is greater than 6 sentences, then execute step S504; if it is less than 6 sentences, then execute step S501.

[0095] Step S504: Use the target feature generation network trained above to recognize each speech file, thereby outputting the speech features corresponding to each speech file, thus obtaining multiple speech features;

[0096] Step S505: Clustering algorithms will be used to cluster multiple speech features to obtain one or more user voiceprints (t-SNE clustering algorithm can be used here. t-SNE is an unsupervised clustering algorithm and a high-dimensional data dimensionality reduction mode. This algorithm has a good classification effect on samples after high-dimensional data dimensionality reduction).

[0097] Step S506: When the target speech is collected, match the target speech with one or more user voiceprints, and detect whether the target speech and user voiceprints are consistent. If they are consistent, proceed to step S507; otherwise, treat the target speech as a speech file and proceed to step S501.

[0098] In step S507, if the match is consistent, it is considered to be the same user, and the voiceprint recognition process is completed by replying with the phrase "Hello, ***".

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0100] Figure 6 This is a structural block diagram of a speech feature generation apparatus according to an embodiment of this application; as shown below. Figure 6 As shown, the system includes: an acquisition module 602, used to acquire multiple speech samples, wherein each of the multiple speech samples includes speech emitted by a single user; an input module 604, used to input the multiple speech samples into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result; an adjustment module 606, used to adjust the network parameters of the initial feature generation network and the initial recognition network according to the initial recognition result and sample labels to obtain a target feature generation network and a target recognition network connected in sequence, wherein the target feature generation network is used to generate target speech features corresponding to each speech sample, the target recognition network is used to identify that the target speech features belong to a single user, and the sample labels are used to indicate the number of users present in the speech samples; a generation module 608, used to use the target feature generation network to generate speech features corresponding to each speech file in multiple speech files to obtain multiple speech features; and an extraction module 610, used to extract voiceprints from the multiple speech features to obtain one or more user voiceprints, wherein each user voiceprint is used to identify a reference user.

[0101] In the above embodiments, each speech sample includes speech from a single user. A target feature generation network trained on multiple speech samples generates speech features corresponding to each speech sample in multiple speech files. During the training phase, the input values ​​of the speech samples are sequentially connected to an initial feature generation network and an initial recognition network. The network parameters of the initial feature generation network and the initial recognition network are adjusted based on the obtained initial recognition results and sample labels, thereby completing the training of the network model. The trained target feature generation network can generate target speech features corresponding to each speech sample, and the trained target recognition network can identify that the target speech features belong to a single user, thus ensuring the recognizability of the speech features output by the target feature generation network. Therefore, using this target feature generation model to generate speech features corresponding to each speech file in multiple speech files, and extracting voiceprints from multiple speech features, results in more accurate voiceprints. This technical solution solves the problem of low accuracy in generating speech features for speech files in related technologies, achieving the technical effect of improving the accuracy of speech features in generated speech files.

[0102] Optionally, the input module includes: a first input unit, configured to input the speech sample into the initial feature generation network to obtain initial speech features corresponding to the speech sample; a second input unit, configured to input the initial speech features into the initial attractor generation network to obtain an initial attractor generated by the initial attractor generation model, wherein the initial recognition network includes the initial attractor generation network, the initial attractor is used to indicate the user present in the speech sample, and the initial recognition result includes the initial attractor; the adjustment module includes: a calculation unit, configured to calculate the target probability corresponding to the initial attractor, wherein the target probability is the probability that the initial attractor indicates that the user present in the speech sample is a single user; and an adjustment unit, configured to adjust the network parameters of the initial feature generation network and the initial attractor generation network according to the relationship between the target probability and the reference probability to obtain the target feature generation network and the target attractor generation network, wherein the reference probability is the probability that the user present in the speech sample is a single user, and the sample label includes the reference probability.

[0103] Optionally, the adjustment unit is configured to: obtain a set of target probabilities and reference probabilities with corresponding relationships for each speech sample, thereby obtaining multiple sets of target probabilities and reference probabilities with corresponding relationships; calculate the target loss value of the multiple speech samples based on the multiple sets of target probabilities and reference probabilities with corresponding relationships; adjust the network parameters of the initial feature generation network and the initial attractor generation network according to the target loss value until the target loss value converges, thereby obtaining the target feature generation network and the target attractor generation network.

[0104] Optionally, the adjustment unit is configured to: substitute each set of reference probabilities and target probabilities into a target loss function to obtain an initial loss value corresponding to each speech sample; determine a weight value corresponding to the initial loss value of each speech sample, wherein the weight value is used to indicate the degree of outlier of the initial loss value of the corresponding speech sample among the multiple initial loss values ​​of multiple speech samples, and the higher the degree of outlier, the smaller the corresponding weight value; and determine the target loss value by weighting the initial loss value and the corresponding weight value.

[0105] Optionally, the adjustment unit is configured to: determine the maximum value of the difference between the reference probability and the corresponding target probability as the gradient magnitude; divide the gradient magnitude into consecutive gradient intervals of a target number, wherein the target number is determined based on the number of multiple speech samples; and determine the weight value corresponding to the speech sample in each gradient interval as the ratio of the interval length of each gradient interval to the number of speech samples falling within each gradient interval.

[0106] Optionally, the generation module includes: a framing unit for framing each speech file to obtain multiple speech frames; a third input unit for inputting the multiple speech frames into the target feature generation network to obtain initial sub-speech features output by the target feature generation network corresponding to each speech frame; and a processing unit for performing dimensionality reduction processing on each initial sub-speech feature to obtain multiple target sub-speech features, wherein the speech features corresponding to each speech file include multiple target sub-speech features.

[0107] Optionally, the device further includes: a matching module, configured to match the target speech with one or more user voiceprints after the voiceprint extraction of the multiple speech features is performed, when the target speech is acquired; a registration module, configured to register the user corresponding to the target user voiceprint when the target speech matches the target user voiceprint among the one or more user voiceprints; and a determination module, configured to determine the target speech as the speech file when the target speech does not match the one or more user voiceprints.

[0108] Embodiments of this application also provide a storage medium including a stored program, wherein the program, when executed, performs the speech feature generation method of any of the above-mentioned methods.

[0109] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps: acquiring multiple speech samples, wherein each of the multiple speech samples includes speech emitted by a single user; inputting the multiple speech samples into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result; adjusting the network parameters of the initial feature generation network and the initial recognition network according to the initial recognition result and sample labels to obtain a target feature generation network and a target recognition network connected in sequence, wherein the target feature generation network is used to generate target speech features corresponding to each of the speech samples, the target recognition network is used to identify that the target speech features belong to a single user, and the sample labels are used to indicate the number of users present in the speech samples; using the target feature generation network to generate speech features corresponding to each of the multiple speech files to obtain multiple speech features; performing voiceprint extraction on the multiple speech features to obtain one or more user voiceprints, wherein each user voiceprint is used to identify a reference user.

[0110] Embodiments of this application also provide an electronic device including a memory and a processor, the memory storing a computer program, the processor being configured to run the computer program to perform the steps in any of the above-described speech feature generation method embodiments.

[0111] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0112] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: acquiring multiple speech samples, wherein each of the multiple speech samples includes speech emitted by a single user; inputting the multiple speech samples into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result; adjusting the network parameters of the initial feature generation network and the initial recognition network according to the initial recognition result and sample labels to obtain a target feature generation network and a target recognition network connected in sequence, wherein the target feature generation network is used to generate target speech features corresponding to each of the speech samples, the target recognition network is used to identify that the target speech features belong to a single user, and the sample labels are used to indicate the number of users present in the speech samples; using the target feature generation network to generate speech features corresponding to each of the multiple speech files to obtain multiple speech features; performing voiceprint extraction on the multiple speech features to obtain one or more user voiceprints, wherein each user voiceprint is used to identify a reference user.

[0113] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0114] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0115] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0116] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for generating speech features, characterized in that, include: Acquire multiple voice samples, wherein each of the multiple voice samples comprises voice spoken by a single user; Multiple speech samples are input into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result; Based on the initial recognition result and sample labels, the network parameters of the initial feature generation network and the initial recognition network are adjusted to obtain a target feature generation network and a target recognition network connected in sequence. The target feature generation network is used to generate target speech features corresponding to each speech sample, the target recognition network is used to identify that the target speech features belong to a single user, and the sample labels are used to indicate the number of users present in the speech sample. The target feature generation network is used to generate speech features corresponding to each speech file in multiple speech files, resulting in multiple speech features; Voiceprint extraction is performed on multiple of the aforementioned speech features to obtain one or more user voiceprints, wherein each user voiceprint is used to identify a reference user; The initial recognition network includes an initial attractor generation network, where the initial attractor is used to indicate the user present in the speech sample. The initial recognition result includes the initial attractor. The step of adjusting the network parameters of the initial feature generation network and the initial recognition network based on the initial recognition result and sample labels to obtain a sequentially connected target feature generation network and target recognition network includes: calculating the target probability corresponding to the initial attractor, where the target probability is the probability that the initial attractor indicates that the user present in the speech sample is a single user; and adjusting the network parameters of the initial feature generation network and the initial attractor generation network based on the relationship between the target probability and a reference probability to obtain the target feature generation network and the target attractor generation network, where the reference probability is the probability that the user present in the speech sample is a single user, and the sample labels include the reference probability.

2. The method according to claim 1, characterized in that, Multiple speech samples are input into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result, including: inputting the speech samples into the initial feature generation network to obtain initial speech features corresponding to the speech samples; and inputting the initial speech features into an initial attractor generation network to obtain an initial attractor generated by the initial attractor generation network.

3. The method according to claim 2, characterized in that, The step of adjusting the network parameters of the initial feature generation network and the initial attractor generation network according to the relationship between the target probability and the reference probability to obtain the target feature generation network and the target attractor generation network includes: Obtain a set of target probabilities and reference probabilities with corresponding relationships for each speech sample, and obtain multiple sets of target probabilities and reference probabilities with corresponding relationships; The target loss value of the multiple speech samples is calculated based on the multiple sets of target probabilities and reference probabilities that have corresponding relationships. The network parameters of the initial feature generation network and the initial attractor generation network are adjusted according to the target loss value until the target loss value converges, thereby obtaining the target feature generation network and the target attractor generation network.

4. The method according to claim 3, characterized in that, The step of calculating the target loss value of the multiple speech samples based on the multiple sets of target probabilities and reference probabilities with corresponding relationships includes: Substitute each set of reference probabilities and target probabilities into the target loss function to obtain the initial loss value corresponding to each speech sample; Determine a weight value corresponding to the initial loss value of each speech sample, wherein the weight value is used to indicate the degree of outlier of the initial loss value of the corresponding speech sample among the multiple initial loss values ​​of multiple speech samples, and the higher the degree of outlier, the smaller the corresponding weight value; The target loss value is determined by the weighted sum of the initial loss value and the corresponding weight value.

5. The method according to claim 4, characterized in that, Determining the weight value corresponding to the initial loss value for each speech sample includes: The maximum value of the difference between the reference probability and the corresponding target probability is determined as the gradient magnitude; The gradient magnitude is divided into a target number of continuous gradient intervals, wherein the target number is determined based on the number of multiple speech samples; The ratio of the length of each gradient interval to the number of speech samples falling within each gradient interval is determined as the weight value corresponding to the speech sample within each gradient interval.

6. The method according to claim 1, characterized in that, The step of generating speech features for each speech file in multiple speech files using the target feature generation network includes: Each of the audio files is divided into frames to obtain multiple audio frames; Multiple speech frames are input into the target feature generation network to obtain initial sub-speech features output by the target feature generation network corresponding to each speech frame; Each of the initial sub-speech features is subjected to dimensionality reduction processing to obtain multiple target sub-speech features, wherein the speech features corresponding to each speech file include multiple target sub-speech features.

7. The method according to claim 1, characterized in that, After performing speakerprint extraction on the multiple speech features, the method further includes: When the target speech is collected, the target speech is matched with the voiceprints of one or more users; If the target speech matches the target user's voiceprint in one or more user voiceprints, the user corresponding to the target user's voiceprint is registered. If the target speech does not match the voiceprints of one or more users, the target speech is identified as the speech file.

8. A speech feature generation apparatus, characterized in that, include: An acquisition module is used to acquire multiple voice samples, wherein each of the multiple voice samples includes voice spoken by a single user; The input module is used to input multiple speech samples into an initial feature generation network and an initial recognition network connected in sequence to obtain an initial recognition result; An adjustment module is used to adjust the network parameters of the initial feature generation network and the initial recognition network according to the initial recognition result and the sample label, so as to obtain a target feature generation network and a target recognition network connected in sequence. The target feature generation network is used to generate target speech features corresponding to each speech sample, the target recognition network is used to identify that the target speech features belong to a single user, and the sample label is used to indicate the number of users present in the speech sample. The generation module is used to generate speech features corresponding to each speech file in multiple speech files using the target feature generation network, thereby obtaining multiple speech features; The extraction module is used to extract voiceprints from multiple speech features to obtain one or more user voiceprints, wherein each user voiceprint is used to identify a reference user; The initial recognition network includes an initial attractor generation network. The initial attractor is used to indicate the user present in the speech sample. The initial recognition result includes the initial attractor. The adjustment module includes: a calculation unit, used to calculate the target probability corresponding to the initial attractor, wherein the target probability is the probability that the initial attractor indicates that the user present in the speech sample is a single user; and an adjustment unit, used to adjust the network parameters of the initial feature generation network and the initial attractor generation network according to the relationship between the target probability and the reference probability, to obtain the target feature generation network and the target attractor generation network, wherein the reference probability is the probability that the user present in the speech sample is a single user, and the sample label includes the reference probability.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 7 through the computer program.

Images