A voice detection method and device, electronic equipment and storage medium

Abstract

The application discloses a speech detection method and device, an electronic device and a storage medium. The method comprises the following steps: obtaining to-be-detected speech, and obtaining a speech detection model; wherein a teacher model used for speech feature extraction is subjected to knowledge distillation to obtain a pre-trained feature extraction network; an initial detection model is obtained based on at least the pre-trained feature extraction network and an initial feature mapping network; and the network parameters of the feature mapping network are adjusted based on the feature difference of the feature mapping network before and after mapping in the training process of the initial detection model, so as to obtain the speech detection model; and the to-be-detected speech is detected based on the speech detection model to obtain a detection result representing whether the to-be-detected speech is real speech. In the foregoing manner, the accuracy of speech detection can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a speech detection method and device, an electronic device and a storage medium. BACKGROUND

[0002] Automatic speaker verification is a low-cost and flexible biometric technology, and the identity of a speaker is verified by verifying whether the speech of the speaker is real speech. However, with the development of technology, a fraudster can deceive the verification system by generating a fake voice to imitate the identity of another person. There are many ways to generate a fake voice, such as through imitation, playback, speech synthesis and speech conversion technologies. Therefore, how to accurately detect whether the speech is real speech has become a problem to be solved. SUMMARY

[0003] The technical problem solved by the present application is to provide a speech detection method and device, an electronic device and a storage medium, which can improve the accuracy of speech detection.

[0004] To solve the above technical problem, the first aspect of the present application provides a speech detection method, which comprises: acquiring a to-be-detected speech and acquiring a speech detection model; wherein a teacher model for speech feature extraction is knowledge distilled to obtain a pre-trained feature extraction network, and at least based on the pre-trained feature extraction network and an initial feature mapping network, an initial detection model is obtained, and based on the feature difference before and after the feature mapping network is mapped in the training process of the initial detection model, the network parameters of the feature mapping network are adjusted to obtain the speech detection model; the to-be-detected speech is detected based on the speech detection model to obtain a detection result representing whether the to-be-detected speech is real speech.

[0005] To solve the above technical problem, the second aspect of the present application provides a speech detection device, which comprises an acquisition module and a detection module, the acquisition module is used for acquiring a to-be-detected speech and acquiring a speech detection model; wherein a teacher model for speech feature extraction is knowledge distilled to obtain a pre-trained feature extraction network, and at least based on the pre-trained feature extraction network and an initial feature mapping network, an initial detection model is obtained, and based on the feature difference before and after the feature mapping network is mapped in the training process of the initial detection model, the network parameters of the feature mapping network are adjusted to obtain the speech detection model; the detection module is used for detecting the to-be-detected speech based on the speech detection model to obtain a detection result representing whether the to-be-detected speech is real speech.

[0006] To solve the above technical problem, the third aspect of the present application provides an electronic device, which comprises a memory and a processor coupled with each other, the memory stores program instructions; the processor is used for executing the program instructions stored in the memory to realize the method provided in the first aspect.

[0007] To solve the above technical problems, the fourth aspect of the present application provides a computer readable storage medium for storing program instructions capable of being executed to implement the method provided in the first aspect.

[0008] The beneficial effects of the present application are: different from the prior art, the present application obtains the to-be-detected voice and the voice detection model, detects the to-be-detected voice based on the voice detection model, and obtains a detection result representing whether the to-be-detected voice is real voice. Wherein, based on the feature difference of the initial detection model before and after mapping through the feature mapping network in the training process, the network parameters of the feature mapping network are adjusted to obtain the voice detection model; the initial to-be-detected model is obtained based on at least the pre-trained feature extraction network and the initial feature mapping network, and the trained feature extraction network is obtained by knowledge distillation on the teacher model for voice feature extraction. First, the teacher model is distilled to obtain the trained feature extraction network, so that the trained feature extraction network can learn the voice feature extraction capability of the teacher model, that is, it can reduce the parameter amount and the calculation amount while ensuring the voice feature extraction capability of the voice detection model. Further, based on the feature difference before and after mapping through the feature mapping network, the network parameters of the feature mapping network are adjusted, which can minimize the information loss before and after mapping through the feature mapping network, and further improve the accuracy of the voice detection model. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a flowchart of an embodiment of the voice detection model provided by the present application;

[0010] Figure 2 is a framework diagram of an embodiment of the feature extraction network and the teacher model provided by the present application;

[0011] Figure 3 is a framework diagram of an embodiment of the voice detection model provided by the present application;

[0012] Figure 4 is a framework diagram of an embodiment of the voice detection device provided by the present application;

[0013] Figure 5 is a framework structure diagram of an embodiment of the electronic device provided by the present application;

[0014] Figure 6 is a framework diagram of an embodiment of the computer readable storage medium provided by the present application. DETAILED DESCRIPTION

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0016] It should be noted that the embodiments of this application contain descriptions involving "first," "second," etc., which are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0017] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0018] Please refer to the following: Figures 1-3 , Figure 1 This is a flowchart illustrating one embodiment of the speech detection method provided in this application. Figure 2 This is a schematic diagram illustrating the framework of one implementation of the feature extraction network and teacher model provided in this application. Figure 3 This is a schematic diagram of the framework of an implementation method of the speech detection model provided in this application; the method includes:

[0019] S11: Obtain the speech to be detected and obtain the speech detection model.

[0020] Specifically, the speech to be detected can be fake speech (e.g., obtained through imitation, playback, speech synthesis, speech conversion, etc.) or real speech, without any limitation.

[0021] In an embodiment, the speech detection model is trained based on an initial detection model, which is based on at least a pre-trained feature extraction network and an initial feature mapping network, i.e., the initial detection model can include the pre-trained feature extraction network and the initial feature mapping network. The pre-trained feature extraction network is obtained by knowledge distillation of a teacher model for speech feature extraction, i.e., the pre-trained feature extraction network can learn the speech feature extraction capability of the teacher model as a student model. The initial feature mapping network is used to map the features. Based on the difference between the features before and after the feature mapping network in the training process of the initial detection model, the network parameters of the feature mapping network are adjusted to obtain the speech detection model. In other embodiments, the initial feature mapping network can also be replaced by a pooling layer. Compared with mapping the features by using the pooling layer, the embodiment maps the features by using the feature mapping network, which can measure the loss of the features before and after the mapping and reduce the loss, so that the distinguishability of true and false voice is greater.

[0022] In an embodiment, the speech detection model is trained based on an initial detection model, which is based on at least a pre-trained feature extraction network and an initial feature mapping network, i.e., the initial detection model can include the pre-trained feature extraction network and the initial feature mapping network. The pre-trained feature extraction network is obtained by knowledge distillation of a teacher model for speech feature extraction, i.e., the pre-trained feature extraction network can learn the speech feature extraction capability of the teacher model as a student model. The initial feature mapping network is used to map the features. Based on the difference between the features before and after the feature mapping network in the training process of the initial detection model, the network parameters of the feature mapping network are adjusted to obtain the speech detection model. In other embodiments, the initial feature mapping network can also be replaced by a pooling layer. Compared with mapping the features by using the pooling layer, the embodiment maps the features by using the feature mapping network, which can measure the loss of the features before and after the mapping and reduce the loss, so that the distinguishability of true and false voice is greater.

[0023] In a specific embodiment, as Figure 2As shown, the first sample speech can be input into the teacher model and the feature extraction network respectively, and the first sample speech can include gigaspeech, LibriLight, and other data sets, and also includes voice data generated by imitation, playback, speech synthesis, and speech conversion technologies. The teacher model and the feature extraction network both include convolutional layers. The convolutional layers in the teacher model perform feature extraction on the first sample speech to obtain first initial features of each audio frame in the first sample speech. The transformer in the teacher model performs feature extraction on the first initial features of each audio frame to obtain first features of each audio frame. The convolutional layers of the feature extraction network perform feature extraction on the first sample speech to obtain second initial features of each audio frame in the first sample speech. Further, the second initial features of part of the audio frames in the first sample speech are masked to obtain third initial features of each audio frame. For example, for the audio frames whose second initial features are masked, the third initial features of the audio frames can be the masking features obtained after the second initial features are masked, and for the audio frames whose second initial features are not masked, the third initial features of the audio frames can be the second initial features. Specifically, the masking features can be empty (i.e., Null characters), or can also be mask features (e.g., represented as [mask]), which are not limited herein. The transformer in the feature extraction network performs feature extraction on the third initial features of each audio frame to obtain second features of each audio frame. The second features of each audio frame include third features obtained by the feature extraction network performing feature extraction on the masking features, and fourth features obtained by the feature extraction network performing feature extraction on the second initial features of the audio frames except for the part of the audio frames. The first loss is obtained by using the difference between the first features and the third features of the same audio frame, and the second loss is obtained by using the difference between the first features and the fourth features of the same audio frame. Finally, the network parameters of the feature extraction network are adjusted based on the first loss and the second loss to obtain the pre-trained feature extraction network. The number of transformers in the teacher model can be greater than the number of transformers in the feature extraction network, for example, the teacher model uses a network with 24 layers and 1024 nodes, and the feature extraction network uses a network with 12 layers and 256 nodes. The teacher model can be a general voice pre-training model such as Hubert, wavLM, wav2vec2.0, etc.It should be noted that, in the process of performing feature extraction, the feature extraction network and the teacher model both refer to the initial features of each audio frame and the initial features of other audio frames. Therefore, by performing feature extraction on the feature extraction network and using the difference between the features extracted by the feature extraction network and the features extracted by the teacher model for the same audio frame, the network parameters of the feature extraction network can be adjusted to force the feature extraction network to extract features as close as possible to the features extracted by the teacher model for the same audio frame. On the one hand, the feature extraction network can learn the feature extraction capability of the teacher model as much as possible, and on the other hand, the feature extraction network can be forced to improve the ability to reconstruct features based on the temporal correlation between audio frames. Therefore, the above method helps to improve the accuracy of the feature extraction network as much as possible.

[0024] In other embodiments, the teacher model and the feature extraction network can also not include convolutional layers, and the first initial features and the second initial features can be obtained by a speech feature extraction algorithm or other means.

[0025] Please continue to refer to Figure 2 In another specific embodiment, the transformation layer of the teacher model performs feature extraction on the first initial features of each audio frame. After the transformation layer of the teacher model performs feature extraction on the first initial features of each audio frame to obtain the first features of each audio frame, the first features of each audio frame can be clustered, and the clustered features can be used as new first features. Based on the difference between the new first features and the second features, the network parameters of the feature extraction network are adjusted to obtain a trained feature extraction network. Specifically, based on the difference between the new first features and the third features, a first loss is obtained; based on the difference between the new first features and the fourth features, a second loss is obtained; based on the first loss and the second loss, the network parameters of the feature extraction network are adjusted to obtain a pre-trained feature extraction network. The first loss and the second loss can be cross-entropy loss. In this embodiment, since the teacher model can convert the speech signal into discrete speech units, the k-means clustering method can be used to cluster the discrete speech units into a predetermined number of categories, and each category can represent a sample center of a phoneme corresponding to an audio frame. By adjusting the network parameters of the feature extraction network using the new first features, the feature extraction network can learn the discrete speech units of the teacher model.

[0026] The network parameters of the feature mapping network are adjusted based on the feature difference before and after the feature mapping network is mapped in the training process of the initial detection model, and a speech detection model is obtained, including: obtaining a relationship function of a mapping matrix and a mapping loss matrix; wherein the relationship function represents the difference between the mapping matrix and the mapping loss matrix; the mapping loss matrix represents the feature difference; and the mapping matrix is the network parameter of the feature mapping network. For the current moment, the mapping loss matrix of the current moment can be determined based on the feature difference before and after the feature mapping network is mapped in the training process of the initial detection model of the current moment; the mapping matrix of the next moment is determined based on the relationship function and the mapping loss matrix of the current moment; the current moment is taken as the next moment, and the determination of the mapping loss matrix of the current moment based on the feature difference before and after the feature mapping network is mapped in the training process of the initial detection model of the current moment and the determination of the mapping matrix of the next moment based on the relationship function and the mapping loss matrix are repeated until the difference between the latest mapping matrix and the latest mapping loss matrix is within a preset range; and the final network parameter of the feature mapping network is determined based on the latest mapping matrix. The preset range can be set by the user according to actual needs, which is not limited here. In other embodiments, the number of iterations can also be set, for example, if the number of iterations is set to 10, the mapping matrix obtained for the 10th time is the final network parameter of the feature mapping network.

[0027] In a specific embodiment, the relationship function is the difference between the inner product of the mapping matrix and the mapping loss matrix and a regularization function. Considering the invariance of entropy, the regularization function can be designed as an entropy-based regularization function. The relationship function is specifically shown in the following formula 1.

[0028]

[0029] wherein, represents the relationship function; C is the mapping loss matrix; P is the mapping matrix, a is the feature before being mapped by the feature mapping network; b is the feature after being mapped by the feature mapping network, a and b have the same dimension; ε is a regularization coefficient; H(P) = -∑ i,j P i,j (log(P i,j -1)) is the regularization function.

[0030] The mapping loss matrix of the current moment and the relationship function are obtained, and the mapping matrix of the next moment is obtained. Specifically, first, the initial detection model is used to obtain the first parameter corresponding to the next moment by using the feature before the feature mapping network mapping in the training process, the mapping loss matrix of the current moment, and the second parameter corresponding to the current moment. In a specific embodiment, the quotient of the mapping loss matrix and the regularization coefficient can be obtained; the quotient of the mapping loss matrix and the regularization coefficient is taken as the index of the exponential function to obtain the third parameter; wherein the third parameter is a matrix; the quotient of the feature before the feature mapping network mapping and the product of the third parameter and the second parameter corresponding to the current moment is taken as the first parameter corresponding to the next moment. Secondly, the initial detection model is used to determine the second parameter corresponding to the next moment by using the feature after the feature mapping network mapping in the training process, the mapping loss matrix, and the first parameter corresponding to the next moment. In a specific embodiment, the product of the transpose of the third parameter and the first parameter corresponding to the next moment is taken as the fourth parameter; the quotient of the feature after the feature mapping network mapping and the fourth parameter is taken as the second parameter corresponding to the next moment. Finally, based on the first parameter and the second parameter corresponding to the next moment, and the mapping loss matrix of the current moment, the mapping matrix of the next moment is determined.

[0031] Specifically, formula 1 is operated to obtain the relationship between the mapping matrix and the mapping loss matrix. Make PI m = a, P T I n = b, I n and I m are all-1 vectors of length n and m, introduce f ∈ R n and g ∈ R m two dual variables, formula (1) can be further expanded to formula 2.

[0032] L(P,f,g) = <C,P> - εH(P) - <f,PI m -a> - <g,P T I n -b> (2)

[0033] Where <f,PI m -a> and <g,P T I n -b> are 0, so the results of formula 1 and formula 2 are the same.

[0034] The derivative of formula 2 is taken, and the derivative is equal to 0 to obtain formula 3.

[0035]

[0036] Where P i,j represents the element of the i-th row and the j-th column of the mapping matrix; C i,jrepresents the element of the i-th row and j-th column of the mapping loss matrix; f i represents the element of the i-th row of the dual variable f; g j represents the element of the j-th row of the dual variable g.

[0037] By formula 3, we can get For the convenience of calculation, let Then P i,j = u i K i,j v j . Write P i,j = u i K i,j v j in matrix form, and substitute PI m = a, P T I n = b into formula 4.

[0038] diag(u)Kdiag(v)I m = a, diag(v)Kdiag(u)I n = b (4)

[0039] By the associative law of matrices, formula 4 is written in the form of matrix multiplication and dot product, and formula 5 is obtained.

[0040] u⊙(Kv)=a,v⊙(K T u)=b (5)

[0041] By formula 5, we can get formula 6.

[0042]

[0043] Where u (l+1) is the first parameter corresponding to the l+1 moment, v (l+1) is the second parameter corresponding to the l+1 moment, a is the feature before being mapped by the feature mapping network; b is the feature after being mapped by the feature mapping network at the l moment; K = e -C / ε , C is the mapping loss matrix at the l moment; v represents the second parameter corresponding to the l moment, v (0) = I m ; K is the third parameter; K T u (l+1) is the fourth parameter.

[0044] By the above formula, the mapping matrix P l+1 = u l+1 K l v l+1 at the l+1 moment can be obtained. In the l+1 moment, the mapping matrix P l+1The feature a at time l+1 is mapped to b through a feature mapping network. Then, the first and second parameters at time l+2 are obtained again using the above formula 6. The mapping matrix at time l+2 is then obtained. By analogy, the mapping matrix at each time can be obtained.

[0045] The above method, by determining the final network parameters of the feature mapping network through the latest mapping matrix, can ensure that the information loss before and after the feature mapping network is minimized, making the distinction between real and false voices greater, thereby improving the false voice detection and recognition rate of the speech detection model.

[0046] In other implementations, such as Figure 3 As shown, the initial detection model also includes a first linear layer, a bidirectional Long Short-Term Memory (LSTM) network, and a second linear layer. The step of adjusting the network parameters of the feature mapping network based on the feature differences before and after mapping during the initial detection model's training process further includes: obtaining a third loss based on the initial detection model's detection results and annotation results for the second sample speech; and adjusting the network parameters of the initial detection model based on the third loss to obtain the speech detection model. Specifically, the second sample speech can be fake speech data. The initial detection model can detect whether the second sample speech is fake speech data. The third loss is calculated based on the initial detection model's detection results and annotation results for the second sample speech. The third loss can be cross-entropy loss or cosine similarity. Finally, the network parameters of the entire initial detection model are adjusted based on the third loss to obtain the speech detection model. In this embodiment, fake speech training data is used to fine-tune the initial speech detection model to further improve its performance.

[0047] S12: Detect the speech to be detected based on the speech detection model, and obtain the detection result representing whether the speech to be detected is real speech.

[0048] The speech to be detected is input into a trained speech detection model to obtain a detection result representing whether the speech is real speech. Specifically, the speech to be detected is input into the speech detection model, first passing through a feature extraction network to obtain the second feature of the speech to be detected; the second feature is then passed through a first linear layer for dimensionality transformation, converting it into a dimension that a bidirectional long short-term memory network can process, resulting in a sixth feature; the sixth feature is then input into the bidirectional long short-term memory network, allowing the bidirectional long short-term memory network to process the sixth feature, resulting in a seventh feature; the seventh feature is then passed through a feature mapping network to obtain the mapped feature; the second linear layer performs dimensionality transformation on the mapped feature, thus obtaining the detection result representing whether the speech to be detected is real speech.

[0049] The method comprises the following steps: obtaining a to-be-detected voice and a voice detection model; detecting the to-be-detected voice based on the voice detection model to obtain a detection result representing whether the to-be-detected voice is real voice. The voice detection model is obtained by adjusting network parameters of a feature mapping network based on feature differences before and after the feature mapping network is mapped in a training process of an initial detection model. The initial detection model is obtained based on at least a pre-trained feature extraction network and an initial feature mapping network. The trained feature extraction network is obtained by performing knowledge distillation on a teacher model for voice feature extraction. First, the knowledge distillation is performed on the teacher model to obtain the trained feature extraction network, so that the trained feature extraction network can learn the voice feature extraction capability of the teacher model, that is, the voice feature extraction capability of the voice detection model can be ensured while reducing the parameter quantity and the calculation quantity. Further, the network parameters of the feature mapping network are adjusted based on the feature differences before and after the feature mapping network is mapped, so that the information loss before and after the feature mapping network is mapped is minimized, and the accuracy of the voice detection model is further improved.

[0050] Referring to Figure 4 , Figure 4 is a schematic diagram of an embodiment of a voice detection device provided by the present application. The voice detection device 40 comprises an acquisition module 41 and a detection module 42. The acquisition module 41 is configured to acquire a to-be-detected voice and acquire a voice detection model. The voice detection model is obtained based on at least a pre-trained feature extraction network and an initial feature mapping network, and the pre-trained feature extraction network is obtained by performing knowledge distillation on a teacher model for voice feature extraction. The network parameters of the feature mapping network are adjusted based on feature differences before and after the feature mapping network is mapped in a training process of an initial detection model, and the voice detection model is obtained. The detection module 42 is configured to detect the to-be-detected voice based on the voice detection model to obtain a detection result representing whether the to-be-detected voice is real voice.

[0051] The method comprises the following steps: obtaining a to-be-detected voice and a voice detection model; detecting the to-be-detected voice based on the voice detection model to obtain a detection result representing whether the to-be-detected voice is real voice. The voice detection model is obtained by adjusting network parameters of a feature mapping network based on feature differences before and after the feature mapping network is mapped in a training process of an initial detection model. The initial detection model is obtained based on at least a pre-trained feature extraction network and an initial feature mapping network. The trained feature extraction network is obtained by performing knowledge distillation on a teacher model for voice feature extraction. First, the knowledge distillation is performed on the teacher model to obtain the trained feature extraction network, so that the trained feature extraction network can learn the voice feature extraction capability of the teacher model, that is, the voice feature extraction capability of the voice detection model can be ensured while reducing the parameter quantity and the calculation quantity. Further, the network parameters of the feature mapping network are adjusted based on the feature differences before and after the feature mapping network is mapped, so that the information loss before and after the feature mapping network is mapped is minimized, and the accuracy of the voice detection model is further improved.

[0052] In an embodiment, the voice detection apparatus 40 further comprises a first training module configured to perform knowledge distillation on a teacher model for voice feature extraction to obtain a pre-trained feature extraction network. Specifically, the first training module is configured to perform feature extraction on first initial features of audio frames in a first sample voice based on the teacher model to obtain first features of the audio frames, mask second initial features of part of the audio frames in the first sample voice to obtain third initial features of the audio frames, and perform feature extraction on the third initial features of the audio frames based on the feature extraction network to obtain second features of the audio frames; and adjust network parameters of the feature extraction network based on a difference between the first features and the second features of the same audio frames to obtain a trained feature extraction network.

[0053] Therefore, by using the knowledge distillation technology, the pre-trained large model for voice feature extraction is used as a teacher model to guide the training of the feature extraction network, so that the accuracy of the feature extraction network is ensured while reducing the parameter amount and the calculation amount. Further, by masking the second initial features of part of the audio frames to obtain third initial features of the audio frames, and performing feature extraction on the third initial features of the audio frames based on the feature extraction network to obtain second features of the audio frames, the feature extraction network can be prompted to predict the features of the masked part based on the features of the unmasked part, thereby further improving the performance of the model.

[0054] In an embodiment, the third initial features are masked features obtained by masking the second initial features of part of the audio frames, or the second initial features of the audio frames except the part of the audio frames; and the second features include third features and fourth features, the third features being obtained by performing feature extraction on the masked features based on the feature extraction network, and the fourth features being obtained by performing feature extraction on the second initial features of the audio frames except the part of the audio frames based on the feature extraction network.

[0055] The first training module is configured to adjust the network parameters of the feature extraction network based on a difference between the first features and the second features, specifically including: the first training module is configured to obtain a first loss based on a difference between the first features and the third features, and obtain a second loss based on a difference between the first features and the fourth features; and adjust the network parameters of the feature extraction network based on the first loss and the second loss.

[0056] Therefore, by calculating the first loss and the second loss respectively, and adjusting the network parameters of the feature extraction network based on the first loss and the second loss, the feature extraction network can learn different capabilities of the teacher model, for example, the discrete voice units of the teacher model for the masked features, and the posterior probability of the teacher model for the unmasked features.

[0057] In an embodiment, the first training module is further configured to cluster the first features, and take the clustered features as new first features; and adjust network parameters of the feature extraction network based on a difference between the new first features and the second features to obtain the trained feature extraction network.

[0058] In an embodiment, the voice detection apparatus 40 further comprises a second training module configured to obtain a third loss based on a detection result of the second sample voice by the initial detection model and a label of the second sample voice; and adjust network parameters of the initial detection model based on the third loss to obtain the voice detection model.

[0059] Therefore, the initial detection model is fine-tuned by using the second sample voice, and the detection performance of the voice detection model on the falsetto is further improved.

[0060] In an embodiment, the network parameters of the feature mapping network are represented as a mapping matrix, and the voice detection apparatus 40 further comprises a third training module configured to adjust the network parameters of the feature mapping network based on a feature difference of the initial detection model before and after being mapped by the feature mapping network during the training process to obtain the voice detection model. Specifically, the third training module comprises a relationship function obtaining module, a mapping loss matrix determining module, a mapping matrix determining module, and a parameter determining module. The relationship function obtaining module is configured to obtain a relationship function of the mapping matrix and a mapping loss matrix. The relationship function represents a difference between the mapping matrix and the mapping loss matrix. The mapping loss matrix represents the feature difference. The mapping loss matrix determining module is configured to determine the mapping loss matrix at a current time based on the feature difference of the initial detection model before and after being mapped by the feature mapping network during the training process. The mapping matrix determining module is configured to determine a mapping matrix at a next time based on the relationship function and the mapping loss matrix at the current time. The determination of the mapping loss matrix at the current time based on the feature difference of the initial detection model before and after being mapped by the feature mapping network during the training process and the determination of the mapping matrix at the next time based on the relationship function and the mapping loss matrix at the current time are repeatedly performed until the difference between the latest mapping matrix and the latest mapping loss matrix is within a preset range. The parameter determining module is configured to determine the final network parameters of the feature mapping network based on the latest mapping matrix.

[0061] Therefore, the feature mapping network is trained to ensure that the information loss before and after mapping is minimized, and the distinction between the real and falsetto voice is greater, so that the recognition rate of falsetto detection is improved.

[0062] In an embodiment, the relationship function is a difference between an inner product of the mapping matrix and the mapping loss matrix and a regularization function.

[0063] In an embodiment, the mapping matrix determination module comprises a first mapping matrix determination submodule, a second mapping matrix determination submodule and a third mapping matrix determination submodule, the first mapping matrix determination submodule is configured to obtain the first parameter corresponding to the next moment by using the feature before being mapped by the feature mapping network, the mapping loss matrix and the second parameter corresponding to the current moment in the training process of the initial detection model; the second mapping matrix determination submodule is configured to determine the second parameter corresponding to the next moment by using the feature after being mapped by the feature mapping network, the mapping loss matrix and the first parameter corresponding to the next moment in the training process of the initial detection model; and the third mapping matrix determination submodule is configured to determine the mapping matrix of the next moment based on the first parameter and the second parameter corresponding to the next moment and the mapping loss matrix of the current moment.

[0064] In an embodiment, the first mapping matrix determination submodule is further configured to obtain a quotient of the mapping loss matrix and the regularization coefficient; obtain the third parameter by taking the quotient of the mapping loss matrix and the regularization coefficient as an index of an exponential function; wherein the third parameter is a matrix; and obtain the first parameter corresponding to the next moment by taking a quotient of the feature before being mapped by the feature mapping network and a product of the third parameter and the second parameter corresponding to the current moment.

[0065] In an embodiment, the second mapping matrix determination submodule is further configured to obtain the fourth parameter by taking a product of a transpose of the third parameter and the first parameter corresponding to the next moment; and obtain the second parameter corresponding to the next moment by taking a quotient of the feature after being mapped by the feature mapping network and the fourth parameter.

[0066] Please refer to Figure 5 , Figure 5 is a schematic diagram of a frame structure of an embodiment of the electronic device provided in the present application.

[0067] The electronic device 50 comprises a memory 51 and a processor 52 coupled with each other, the memory 51 stores program instructions, and the processor 52 is configured to execute the program instructions stored in the memory 51 to implement the steps of any of the above method embodiments. In a specific implementation scenario, the electronic device 50 can include but is not limited to a microcomputer, a server, in addition, the electronic device 50 can also include a notebook computer, a tablet computer and other mobile devices, which are not limited herein.

[0068] Specifically, the processor 52 is configured to control itself and the memory 51 to implement the steps of any of the above method embodiments. The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 can be an integrated circuit chip having a processing capability of signals. The processor 52 can also be a general processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. In addition, the processor 52 can be jointly implemented by integrated circuit chips.

[0069] Please refer to Figure 6 , Figure 6 is a schematic diagram of an embodiment of the computer readable storage medium provided in the present application.

[0070] The computer readable storage medium 60 stores program instructions 61, which are executed by the processor to implement the steps of any of the above method embodiments.

[0071] The computer readable storage medium 60 can specifically be a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, etc. which can store a computer program, or can also be a server storing the computer program, which can send the stored computer program to other devices for running, or can also run the stored computer program by itself.

[0072] The above description of various embodiments tends to emphasize the differences between various embodiments, and the same or similar parts can be referred to each other. For the sake of brevity, the same or similar parts will not be described herein again.

[0073] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely an example, and the division of the modules or units can be different, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, and can be electrical, mechanical or other forms.

[0074] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0075] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0076] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0077] If the technical solutions of the present application involve personal information, the product applying the technical solutions of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solutions of the present application involve sensitive personal information, the product applying the technical solutions of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that it has entered the personal information collection range and will collect personal information. If the individual voluntarily enters the collection range, it is considered to agree to collect personal information. Or, on the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the individual to upload his / her personal information. The personal information processing rules can include personal information processor, personal information processing purpose, processing method and personal information type, etc.

[0078] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A speech detection method, characterized in that, include: Acquire the speech to be detected and acquire the speech detection model; wherein, knowledge distillation is performed on the teacher model used for speech feature extraction to obtain a pre-trained feature extraction network, and an initial detection model is obtained based at least on the pre-trained feature extraction network and an initial feature mapping network, and the network parameters of the feature mapping network are adjusted based on the feature differences of the initial detection model before and after mapping by the feature mapping network during the training process to obtain the speech detection model. The speech to be detected is detected based on the speech detection model to obtain a detection result that characterizes whether the speech to be detected is real speech.

2. The method according to claim 1, characterized in that, The knowledge distillation of the teacher model used for speech feature extraction to obtain a pre-trained feature extraction network includes: Based on the teacher model, the first initial features of the audio frames in the first sample speech are extracted to obtain the first features of each audio frame. The second initial features of some audio frames in the first sample speech are masked to obtain the third initial features of each audio frame. The third initial features of the audio frames are extracted based on the feature extraction network to obtain the second features of each audio frame. Based on the difference between the first feature and the second feature of the same audio frame, the network parameters of the feature extraction network are adjusted to obtain a trained feature extraction network.

3. The method according to claim 2, characterized in that, The third initial feature is a masked feature obtained by masking the second initial feature of a portion of the audio frames, or the second initial feature of audio frames other than a portion of the audio frames; the second feature includes a third feature and a fourth feature, the third feature is obtained by the feature extraction network extracting the masked feature, and the fourth feature is obtained by the feature extraction network extracting the second initial feature of audio frames other than a portion of the audio frames; The step of adjusting the network parameters of the feature extraction network based on the difference between the first feature and the second feature includes: A first loss is obtained based on the difference between the first feature and the third feature; and a second loss is obtained based on the difference between the first feature and the fourth feature. Based on the first loss and the second loss, the network parameters of the feature extraction network are adjusted.

4. The method according to claim 2, characterized in that, After extracting the first initial features of the audio frames in the first sample speech based on the teacher model to obtain the first features of each audio frame, the process further includes: Cluster the first feature and use the clustered features as the new first feature; The step of adjusting the network parameters of the feature extraction network based on the difference between the first feature and the second feature of the same audio frame to obtain a trained feature extraction network includes: Based on the difference between the new first feature and the second feature, the network parameters of the feature extraction network are adjusted to obtain a trained feature extraction network.

5. The method according to claim 1, characterized in that, In the step of adjusting the network parameters of the feature mapping network based on the feature differences before and after mapping by the feature mapping network during the training of the initial detection model, the method further includes: The third loss is obtained based on the detection results of the second sample speech by the initial detection model and the annotation results of the second sample speech; Based on the third loss, the network parameters of the initial detection model are adjusted to obtain the speech detection model.

6. The method according to claim 1, characterized in that, The network parameters of the feature mapping network are represented as a mapping matrix. The process of adjusting the network parameters of the feature mapping network based on the feature differences before and after mapping by the feature mapping network during the training of the initial detection model to obtain the speech detection model includes: Obtain the relationship function between the mapping matrix and the mapping loss matrix; wherein the relationship function characterizes the difference between the mapping matrix and the mapping loss matrix; and the mapping loss matrix characterizes the feature difference. Based on the feature differences of the initial detection model before and after mapping by the feature mapping network during the training process, the mapping loss matrix at the current time is determined; Based on the relation function and the mapping loss matrix at the current time, determine the mapping matrix at the next time step; The process of repeatedly performing the following steps—based on the feature differences of the initial detection model before and after mapping by the feature mapping network during training at the current time, determining the mapping loss matrix at the current time; and determining the mapping matrix at the next time step based on the relation function and the mapping loss matrix—continues until the difference between the latest mapping matrix and the latest mapping loss matrix is ​​within a preset range. Based on the latest mapping matrix, the final network parameters of the feature mapping network are determined.

7. The method according to claim 6, characterized in that, The relation function is the difference between the inner product of the mapping matrix and the mapping loss matrix and the regularization function.

8. The method according to claim 6, characterized in that, Determining the mapping matrix for the next time step based on the relation function and the mapping loss matrix includes: Using the features of the initial detection model before it is mapped by the feature mapping network during training, the mapping loss matrix, and the second parameter corresponding to the current time step, the first parameter corresponding to the next time step is obtained. Using the features mapped by the feature mapping network during the training of the initial detection model, the mapping loss matrix, and the first parameter corresponding to the next time step, the second parameter corresponding to the next time step is determined. Based on the first and second parameters corresponding to the next time step, and the mapping loss matrix at the current time step, the mapping matrix at the next time step is determined.

9. The method according to claim 8, characterized in that, The step of obtaining the first parameter corresponding to the next time step using the features of the initial detection model before mapping by the feature mapping network during training, the mapping loss matrix, and the second parameter corresponding to the current time step includes: Obtain the quotient of the mapping loss matrix and the regularization coefficient; The quotient of the mapping loss matrix and the regularization coefficient is used as the exponent of the exponential function to obtain the third parameter; wherein, the third parameter is a matrix; The quotient of the product of the features before mapping by the feature mapping network, the third parameter, and the second parameter corresponding to the current time step is used as the first parameter corresponding to the next time step.

10. The method according to claim 9, characterized in that, The step of determining the second parameter corresponding to the next time step using the features mapped by the feature mapping network during the training of the initial detection model, the mapping loss matrix, and the first parameter corresponding to the next time step includes: The product of the transpose of the third parameter and the first parameter corresponding to the next time step is taken as the fourth parameter; The quotient of the features mapped by the feature mapping network and the fourth parameter is used as the second parameter corresponding to the next time step.

11. A voice detection device, characterized in that, include: An acquisition module is used to acquire the speech to be detected and to acquire a speech detection model; wherein, a pre-trained feature extraction network is obtained by knowledge distillation of the teacher model used for speech feature extraction, and an initial detection model is obtained based at least on the pre-trained feature extraction network and an initial feature mapping network, and the network parameters of the feature mapping network are adjusted based on the feature differences of the initial detection model before and after mapping by the feature mapping network during the training process to obtain the speech detection model. The detection module is used to detect the speech to be detected based on the speech detection model, and obtain a detection result that characterizes whether the speech to be detected is real speech.

12. An electronic device, characterized in that, The method includes a memory and a processor coupled to each other, the memory storing program instructions and the processor executing the program instructions to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the method according to any one of claims 1 to 10.

Images