Speech authentic identification method based on feature clustering and related equipment

Abstract

The invention belongs to the field of artificial intelligence and big data, is applied to the field of smart cities, and relates to a voice authentic identification method based on feature clustering, which comprises the following steps: acquiring a voice training data set, training a pre-trained voice authentic identification model through the voice training data set to obtain a trained voice authentic identification model, the voice training data set comprises audio in a track data set; performing feature sequence extraction processing on a target audio through the trained voice authentic identification model to obtain a target audio feature sequence; and performing authentic identification processing on the target audio feature sequence by using a K-nearest neighbor method, and determining an authentic identification processing result of the target audio. The invention further provides a voice authentic identification device based on feature clustering, computer equipment and a storage medium. The problem that an existing voice authentic identification method is low in operation efficiency and accuracy is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and big data technology, and in particular to a method for detecting fake voice based on feature clustering and related equipment. Background Art

[0002] With the development of deepfake technology, it has become easier to generate highly realistic fake human voice audio. The mainstream speech generation methods are mainly divided into two forms. One is to synthesize the speech audio of the target task through a file, that is, TTS. The other is to convert the source speaker's voice into the target speaker's voice and keep the audio content unchanged, that is, voice conversion VC. The development of convolutional neural networks has made generative speech more and more realistic. While these technologies bring convenience and entertainment, they also increase the risk of voice fraud and voice fraud. It is not ruled out that criminals use these technologies to maliciously forge other people's audio to commit fraud, illegally obtain other people's information, or breed trouble and other illegal acts.

[0003] The existing speech authentication method usually extracts certain features of the audio to be detected and then feeds them into the classification network for classification. This results in the model's effect being largely dependent on the design of the feature extraction module and the classification network. Therefore, there is an urgent need for a speech authentication method based on feature clustering to solve the problems of low operating efficiency and accuracy of the existing speech authentication methods. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a speech authentication method and related equipment based on feature clustering to solve the problems of low operating efficiency and accuracy of existing speech authentication methods. Its main purpose is to improve the accuracy of speech authentication by using feature fusion and multi-classification network to process input features, and use K-nearest neighbor algorithm to perform feature calculation and recognition, thereby improving the generalization ability and detection accuracy.

[0005] In order to solve the above technical problems, the present application provides a method for voice authentication based on feature clustering, which adopts the following technical solution:

[0006] Acquire a speech training data set, and train a pre-trained speech authentication model using the speech training data set to obtain a trained speech authentication model, wherein the speech training data set includes audio in the track data set;

[0007] The trained speech authentication model is used to extract the feature sequence of the target audio to obtain the target audio feature sequence;

[0008] The target audio feature sequence is subjected to authentication processing using the K nearest neighbor method to determine an authentication processing result of the target audio.

[0009] Furthermore, after obtaining the speech training data set, the method further includes:

[0010] Performing feature sequence extraction processing on the speech training data set to obtain a feature sequence of the speech training data set;

[0011] The K nearest neighbor method is used to perform clustering calculation processing on the feature sequence of the speech training data set to construct a feature retrieval library.

[0012] Furthermore, the K-nearest neighbor method is used to perform clustering calculation processing on the feature sequence of the speech training data set to construct a feature retrieval library, including:

[0013] Using the K nearest neighbor method to calculate and process the feature sequence of the speech training data set to obtain a category label of the feature sequence of the speech training data set;

[0014] The category labels are clustered to construct a feature retrieval library.

[0015] Furthermore, a loss function is used to constrain the model during the training process of the speech authentication model. The formula is as follows:

[0016] Loss = -[ylog(σ(x n ))+(1-y n )log(1-σ(x n ))]

[0017] Among them, y is the category label to which the audio belongs, σ(x n ) is the sigmoid function, which maps x to the (0,1) interval: y n To predict the category label that the audio belongs to.

[0018] Furthermore, the using of the K nearest neighbor method to perform authentication processing on the target audio feature sequence to determine the authentication processing result of the target audio includes:

[0019] The target audio feature sequence is calculated and processed using a K nearest neighbor method to obtain an average value of the K feature frames closest to each frame in the target audio feature sequence in the feature retrieval library;

[0020] Using the average value of the K most adjacent feature frames to perform replacement processing to obtain a new target audio feature sequence;

[0021] Perform authentication processing on the new target audio feature sequence to obtain an authentication result of the target audio.

[0022] Furthermore, the replacing process using the average value of the K most adjacent feature frames to obtain a new target audio feature sequence includes:

[0023] The corresponding frames of the target audio feature sequence are replaced by the average value of the K most adjacent feature frames to obtain a new target feature sequence.

[0024] Furthermore, performing authentication processing on the new target audio feature sequence to obtain an authentication result of the target audio includes:

[0025] Performing feature extraction processing on the new target audio feature sequence to obtain new target audio features;

[0026] Perform authentication processing on the new target audio feature to obtain an authentication processing result of the target audio.

[0027] In order to solve the above technical problems, the present application also provides a voice authentication device based on feature clustering, which adopts the following technical solution:

[0028] An acquisition module is used to acquire a speech training data set, and train a pre-trained speech authentication model through the audio training data set to obtain a trained speech authentication model, wherein the speech training data set includes the audio in the track data set;

[0029] The first extraction processing module is used to perform feature sequence extraction processing on the target audio through the trained speech authentication model to obtain a target audio feature sequence;

[0030] The authentication processing module is used to perform authentication processing on the target audio feature sequence using the K nearest neighbor method to determine the authentication processing result of the target audio.

[0031] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0032] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for speech authentication based on feature clustering described in any one of the embodiments of the present application are implemented.

[0033] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0034] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method for speech authentication based on feature clustering described in any one of the embodiments of the present invention are implemented.

[0035] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: obtaining a speech training data set, training a pre-trained speech authentication detection model through the speech training data set to obtain a trained speech authentication detection model, and performing feature sequence extraction processing on the target audio through the trained speech authentication detection model to obtain a target audio feature sequence, using the K nearest neighbor method to perform authentication processing on the target audio feature sequence, and determining the authentication processing result of the target audio, thereby solving the problems of low operating efficiency and accuracy of the existing speech authentication detection methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0038] Figure 2 is a flow chart of an embodiment of a method for detecting false voices based on feature clustering according to the present application;

[0039] Figure 3 yes Figure 2 A flowchart of a specific implementation method after step S201;

[0040] Figure 4 yes Figure 3 A flowchart of a specific implementation of step S302;

[0041] Figure 5 yes Figure 2 A flowchart of a specific implementation method of step S203;

[0042] Figure 6 yes Figure 5 A flowchart of a specific implementation method of step S2033;

[0043] Figure 7 It is a structural diagram of the voice authentication method based on feature clustering according to the present application;

[0044] Figure 8 It is a structural schematic diagram of a specific embodiment of a voice authentication device based on feature clustering according to the present application;

[0045] Fig. 9 yes Figure 8A structural diagram of another specific implementation of a voice authentication device 800 based on feature clustering;

[0046] Fig.10 yes Fig. 9 A schematic diagram of a specific implementation of the computing processing module 805;

[0047] Fig.11 yes Figure 8 A schematic diagram of a specific implementation of the authentication processing module 803;

[0048] Fig.12 yes Fig.11 A schematic diagram of a specific implementation of the counterfeit identification processing submodule 8033;

[0049] Fig.13 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0051] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0053] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0054] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0055] The terminal devices 101, 102, and 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0056] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0057] It should be noted that the voice authentication method based on feature clustering provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the voice authentication device based on feature clustering is generally arranged in a server / terminal device.

[0058] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0059] Continue to refer Figure 2 , shows a flow chart of an embodiment of a voice authentication method based on feature clustering according to the present application. The voice authentication method based on feature clustering comprises the following steps:

[0060] Step S201: Acquire a speech training data set, and train a pre-trained speech authentication model using the speech training data set to obtain a trained speech authentication model.

[0061] In this embodiment, the electronic device (eg, Figure 1The server / terminal device shown in the figure can receive the voice authentication request based on feature clustering from the terminal device through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, Wi-Fi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (Ultra Wi-Fi Band) connection, and other wireless connection methods currently known or to be developed in the future.

[0062] The above-mentioned terminal devices may be terminal devices equipped with remote interaction functions such as personal assistants and intelligent customer service.

[0063] The above-mentioned voice authentication method based on feature clustering can be applied to online file authentication, online file management and other businesses.

[0064] Specifically, the electronic device can be used in different industries such as medical voice authentication, property insurance voice authentication, finance voice authentication, and store voice authentication.

[0065] The above-mentioned speech training data set includes the audio in the track data set; the above-mentioned track data set is a data set that calls for voiceprint recognition with a large amount of audio data.

[0066] The pre-trained speech authentication model is a tool that calls a speech task, and the speech task includes speech recognition and the like.

[0067] The above training can be supervised training. Supervised training is a training method in machine learning. Its core is to use a set of data with known labels to train the model, and optimize the model parameters so that the model can predict the labels of new data or make decisions based on the characteristics of existing data. In the training process of supervised learning, first, the data set needs to be divided into a training set and a test set. The training set is used for model training, and the test set is used to evaluate the performance of the model. Then, the features useful to the model, that is, the input features, are extracted from the original data. Then, the parameters of the model are adjusted by using the data of the training set through the optimization algorithm so that the model can fit the data well. Finally, the model is verified using the test set data to test the generalization ability of the model. After completing the training and verification, a trained speech authentication model is obtained.

[0068] The above-trained speech authentication model is a model obtained through learning from training data of a large number of track data sets. It is a speech authentication model that can accurately judge new speech data.

[0069] In one embodiment, 125,000 audios may be randomly selected from the track data set as a training set, 100,000 audios may be selected to train the model during training, and the remaining 25,000 audios may be used for model testing after each round of training.

[0070] It should be noted that during the training process, the parameters of the model will be adjusted according to the supervisory signals on the speech dataset, so that the model can better identify the authenticity of different speech.

[0071] Step S202: Perform feature sequence extraction processing on the target audio through the trained speech authentication model to obtain a target audio feature sequence.

[0072] In this embodiment, the target audio refers to the audio that needs to be authenticated by voice.

[0073] The above feature sequence extraction process can be understood as the process of extracting a set of information that can reflect the signal characteristics from the original signal and converting it into a digital feature sequence that can be understood and processed. Feature extraction can convert complex speech signals into a set of representative feature vector sequences, thereby greatly simplifying the complexity of subsequent signal processing methods and models.

[0074] Step S203: Use the K nearest neighbor method to perform authentication processing on the target audio feature sequence to determine the authentication processing result of the target audio.

[0075] In this embodiment, the idea of ​​the above K-Nearest Neighbors (KNN) method is that if most of the k nearest neighboring samples of a sample in the feature space belong to a certain category, then the sample is also classified into this category.

[0076] The above-mentioned authentication process can be understood as a process of extracting a representative feature vector representation from the target audio.

[0077] Specifically, the K nearest neighbor method can be used to average the K nearest feature frames in the feature retrieval library for each frame in the target audio feature sequence, and use the average value to replace the corresponding frame in the target audio feature sequence. K can be a number of values ​​such as [2-10] for experimental comparison. Using the K nearest neighbor method to replace feature frames will not change the content of the audio, but only replace other features of the audio with feature information similar to it in the training set, which is more conducive to accurate classification of the network.

[0078] Furthermore, the target audio feature sequence is replaced to obtain a new feature sequence, and the new feature sequence is integrated into five consecutive two-bit convolutional layers for feature extraction. Each convolutional layer is connected to a maximum pooling layer, and the output of the last pooling layer is used as the input of the fully connected layer. Then the last fully connected layer outputs the authentication result of the target audio.

[0079] The feature retrieval library is a database containing a large number of extracted feature vectors. The pre-trained model extracts feature sequences from the speech training data set, and uses the K nearest neighbor method to perform clustering calculations on the feature sequences of the speech training data set to construct the feature retrieval library.

[0080] Specifically, the purpose of the convolution layer is to extract more abstract and higher-level features from the input feature sequence. Each convolution layer uses different filters (or convolution kernels) to scan the input data to generate a new feature map. The convolution layer extracts local features by sliding small convolution kernels on the input sequence and integrates these features into a more abstract feature representation. Each convolution layer is followed by a maximum pooling layer to reduce the spatial dimension of the features, reduce the amount of computation and increase the robustness of the model. The fully connected layer maps the feature vector obtained in the previous step to a high-dimensional space, making the differences between different categories more obvious.

[0081] The present application obtains a speech training data set, trains a pre-trained speech authentication model through the speech training data set to obtain a trained speech authentication model, performs feature sequence extraction processing on the target audio through the trained speech authentication model to obtain a target audio feature sequence, and uses the K nearest neighbor method to perform authentication processing on the target audio features to determine the authentication processing result of the target audio, thereby solving the problems of low operating efficiency and accuracy of existing speech authentication methods.

[0082] Continue to refer Figure 3 , shown in Figure 2 The method for identifying false voices based on feature clustering further comprises the following steps:

[0083] Step S301: Perform feature sequence extraction processing on the speech training data set to obtain the feature sequence of the speech training data set.

[0084] In this embodiment, the pre-training model may be used to perform feature sequence extraction processing on the speech training data set to obtain the feature sequence of the speech training data set.

[0085] The above pre-trained models are tools with speech tasks, and speech tasks include tasks such as speech recognition.

[0086] The above pre-trained model uses a large amount of speech training data for pre-training, thereby learning the characteristics and rules of speech signals.

[0087] Step S302: Use the K nearest neighbor method to perform clustering calculation processing on the feature sequence of the speech training data set to construct a feature retrieval library.

[0088] In this embodiment, the above clustering calculation process can be understood as a process of classifying the samples to be classified into the known category with the closest distance by calculating the distance between the samples to be classified and the samples of the known category.

[0089] The above-mentioned feature retrieval library is a database with a large number of speech feature vectors.

[0090] Specifically, the feature sequence of the speech training data set is used as input, the distance between each feature sample and other samples is calculated by the K nearest neighbor method, and clustering is performed according to the size of the distance, and finally multiple clusters are obtained, namely the feature retrieval library.

[0091] It should be noted that the k-nearest neighbor algorithm is used to calculate the feature sequences of all training data. The number of clusters can be defined as 10,000. These sequences are shuffled to construct a large self-supervised feature vector pool, namely the feature retrieval library.

[0092] The present application obtains the feature sequence of the speech training data set by performing feature sequence extraction processing on the speech training data set, and uses the K nearest neighbor method to perform clustering calculation processing on the feature sequence of the speech training data set to construct a feature retrieval library, thereby improving recognition accuracy and efficiency.

[0093] Continue to refer Figure 4 , showing that in Figure 3 Flow chart of a specific implementation of step S302 in FIG. 1 . Step S2014 specifically includes the following steps:

[0094] Step S3021: Use the K nearest neighbor method to calculate and process the feature sequence of the speech training data set to obtain the category label of the feature sequence of the speech training data set.

[0095] In this embodiment, the idea of ​​the K nearest neighbor algorithm is that if most of the k nearest samples of a sample in the feature space belong to a certain category, then the sample is also classified into this category.

[0096] Furthermore, for each feature sequence, its K nearest neighbors in the known categories are calculated, and then a category label is assigned to the feature sequence based on the category labels of these K neighbors.

[0097] Step S3022: cluster the category labels to construct a feature retrieval library.

[0098] In this embodiment, the above clustering process can be understood as a process of classifying data objects with similar characteristics into the same category for processing.

[0099] The K-nearest neighbor method can be used to analyze and process category labels, classify similar category labels into the same cluster, and then establish a feature retrieval library for subsequent feature retrieval and application.

[0100] The present application uses the K-nearest neighbor method to calculate and process the feature sequence of a speech training data set, obtains the category labels of the feature sequence of the speech training data set, and clusters the category labels to construct a feature retrieval library. The feature retrieval library can be used to quickly find other feature sequences similar to a given feature sequence, thereby achieving efficient speech recognition.

[0101] Optionally, a loss function is used to constrain the model during the training process of the speech authentication model. The formula is as follows:

[0102] Loss = -[ylog(σ(x n ))+(1-y n )log(1-σ(x n ))]

[0103] Among them, y is the category label to which the audio belongs, σ(x n ) is the si gmod function, which maps x to the interval (0,1): y n To predict the category label that the audio belongs to.

[0104] In this embodiment, the above loss function is used to measure the gap between the model prediction value and the true value. By optimizing the loss function, the model can be made more accurate and reliable. The BCEWithLog itLoss loss function can be used to constrain the model, and the multi-classification loss function can also be used to constrain the model. The above BCEWithLogi tLoss loss function combines the functions of the Sigmo id activation function and the binary cross entropy loss (Bi nary Cross Entropy Loss), so it is improved in numerical stability and gradient calculation. The above multi-classification loss function optimizes the model by calculating the cross entropy between the probability of each sample belonging to each category and the true label.

[0105] The above si gmod function is used to map the input x to the interval (0,1), which can make the output smoother and more interpretable.

[0106] It should be noted that the gradient of the loss function is used to adjust the model parameters of the classification network to minimize the error between the predicted value and the actual label.

[0107] This application uses a loss function to constrain the model during the training process of the voice authentication model. The formula is as follows:

[0108] Loss = -[ylog(σ(x n ))+(1-y n )log(1-σ(x n ))]

[0109] Among them, y is the category label to which the audio belongs, σ(x n ) is the si gmod function, which maps x to the interval (0,1): y n To predict the category label that the audio belongs to. The si gmo id function is used to map the input x to the interval (0,1), which can make the output smoother and more interpretable. During the training process, the value of the loss function is minimized, and the model can better fit the data and improve the accuracy of authentication.

[0110] Continue to refer Figure 5 , showing Figure 2 Flow chart of a specific implementation manner of step S203 in FIG. Step S203 specifically includes the following steps:

[0111] Step S2031: Use the K nearest neighbor method to calculate and process the target audio feature sequence to obtain the average value of the K feature frames closest to each frame in the target audio feature sequence in the feature retrieval library.

[0112] In this embodiment, the K nearest neighbor method is used for each vector in the target audio feature sequence, each frame in the target audio feature sequence is compared with all frames in the feature retrieval library, the K closest feature frames are found, and then the average value of these K feature frames is calculated as the feature value of the current frame in the target audio feature sequence.

[0113] The average value of the K nearest feature frames can be understood as the result of finding the K closest frames to each frame in an audio feature sequence, and then averaging the feature values ​​of these K frames. This average value can be regarded as the feature representation of the frame and is used for subsequent audio recognition, classification and other tasks.

[0114] Step S2032: Use the average value of the most adjacent K feature frames to perform replacement processing to obtain a new target audio feature sequence.

[0115] In this embodiment, the above replacement process can be understood as a process of replacing the feature value of each frame in the target audio feature sequence with the average value of the nearest K feature frames. K can be obtained by experimental comparison using multiple sets of values ​​such as [2-10].

[0116] It should be noted that performing the K-nearest neighbor method to replace the feature frame will not change the content of the audio, but only replace other features of the audio with feature information similar to it in the training set, which is more conducive to the accurate classification of the network.

[0117] Step S2033: perform authentication processing on the new target audio feature sequence to obtain an authentication processing result of the target audio.

[0118] In this embodiment, the above-mentioned authentication process can be understood as a process of extracting a representative feature vector representation from a new target audio feature sequence.

[0119] The present application calculates and processes the target audio feature sequence by using the K nearest neighbor method, obtains the average value of the K feature frames closest to each frame in the target audio feature sequence in the feature retrieval library, uses the average value of the K closest feature frames for replacement processing, obtains a new target audio feature sequence, performs authentication processing on the new target audio feature sequence, obtains the authentication processing result of the target audio, and improves the quality and availability of audio data.

[0120] Optionally, step S2032 includes replacing corresponding frames of the target audio feature sequence with an average value of the most adjacent K feature frames to obtain a new target feature sequence.

[0121] In this embodiment, the K most adjacent feature frames may be understood as K frames closest to the current frame, and the difference between them is the smallest.

[0122] The above average value can be understood as the weighted average of the feature values ​​of these K feature frames, where the weight of each feature value is inversely proportional to its distance from the current frame.

[0123] The present application obtains a new target feature sequence by replacing the corresponding frames of the target audio feature sequence with the average value of the nearest K feature frames. Executing the K nearest neighbor regression method to replace the feature frames will not change the content of the audio, but only replaces other features of the audio with feature information similar to it in the training set, which is beneficial to the accurate classification of the network.

[0124] Continue to refer Figure 6 , showing Figure 5 Flow chart of a specific implementation manner of step S2033 in FIG. Step S2033 specifically includes the following steps:

[0125] Step S20331: perform feature extraction processing on the new target audio feature sequence to obtain new target audio features.

[0126] In this embodiment, the new target audio feature sequence can be fed into five consecutive two-dimensional convolutional layers for feature extraction to obtain new target audio features. The purpose of the convolutional layer is to extract more abstract and higher-level features from the input feature sequence. Each convolutional layer uses a different filter (or convolution kernel) to scan the input data to generate a new feature map.

[0127] The above feature extraction process can be understood as a process of extracting useful information or features from a new target audio feature sequence.

[0128] Step S20332: perform authentication processing on the new target audio feature to obtain an authentication processing result of the target audio.

[0129] In this embodiment, the above-mentioned authentication process can be understood as a process of identifying and eliminating false or untrue audio features.

[0130] The authentication results of the above-mentioned target audio can help users identify the authenticity and authenticity of the audio, thereby protecting intellectual property rights and preventing the spread of false information.

[0131] The present application performs feature extraction processing on a new target audio feature sequence to obtain a new target audio feature, performs authentication processing on the new target audio feature to obtain an authentication processing result of the target audio, and can effectively improve the accuracy and reliability of feature extraction.

[0132] like Figure 7 As shown, a structural diagram of a speech authentication method based on feature clustering according to the present application is shown. Specifically, a speech training data set is selected, the speech training data set includes the audio in the track data set, and the pre-training model is trained through the speech training data set to obtain a trained speech authentication model; the pre-training model is used to extract feature sequences from all speech training data sets, and the feature sequences of all speech training data sets are clustered using the K nearest neighbor algorithm, and the feature sequences of all speech training data sets are shuffled and constructed into a feature retrieval library; the feature sequence of the audio to be tested is extracted through the trained speech authentication model, and the K nearest neighbor algorithm is used to calculate the average value of the K nearest feature frames of each frame in the feature sequence of the audio to be tested in the feature retrieval library; after replacement, a new feature sequence is obtained, and the new feature sequence is sent to five consecutive two-dimensional convolutional layers for further feature extraction, each convolutional layer is connected to a maximum pooling layer, and the output of the last pooling layer is used as the input of the fully connected layer, and then the last fully connected layer outputs the authentication result of the audio to be tested.

[0133] In this embodiment, the speech training data set includes audio from a track data set; the track data set is a data set that calls for voiceprint recognition with a large amount of audio data.

[0134] The above-mentioned pre-trained models are tools with speech tasks, and the speech tasks include speech recognition and the like.

[0135] The above training can be supervised training. Supervised training is a training method in machine learning. Its core is to use a set of data with known labels to train the model, and optimize the model parameters so that the model can predict the labels of new data or make decisions based on the characteristics of existing data. In the training process of supervised learning, first, the data set needs to be divided into a training set and a test set. The training set is used for model training, and the test set is used to evaluate the performance of the model. Then, the features useful to the model, that is, the input features, are extracted from the original data. Then, the parameters of the model are adjusted by using the data of the training set through the optimization algorithm so that the model can fit the data well. Finally, the model is verified using the test set data to test the generalization ability of the model. After completing the training and verification, a trained speech authentication model is obtained.

[0136] The above-trained speech authentication model is a model obtained through learning from training data of a large number of track data sets. It is a speech authentication model that can accurately judge new speech data.

[0137] In one embodiment, 125,000 audios may be randomly selected from the track data set as a training set, 100,000 audios may be selected to train the model during training, and the remaining 25,000 audios may be used for model testing after each round of training.

[0138] The above-mentioned feature sequence extraction can be understood as a process of extracting a set of information that can reflect the signal characteristics from the original signal and converting it into a digital feature sequence that can be understood and processed.

[0139] The above-mentioned audio to be tested refers to the audio that needs to be subjected to voice authentication.

[0140] The above clustering calculation can be understood as a process of classifying data objects with similar characteristics into the same category for processing.

[0141] The idea of ​​the above K-nearest neighbor algorithm is that if most of the k nearest samples of a sample in the feature space belong to a certain category, then the sample is also classified into this category.

[0142] Executing the K-nearest neighbor method to replace the feature frame will not change the content of the audio, but only replace other features of the audio with similar feature information in the training set, which is more conducive to the accurate classification of the network.

[0143] The loss function is used to constrain the model during the training process. The formula is as follows:

[0144] Loss = -[ylog(σ(x n))+(1-y n )log(1-σ(x n ))]

[0145] Where y is the category label to which the audio belongs, σ(x n ) is the si gmo id function, which maps x to the (0,1) interval: y n To predict the category label that the audio belongs to.

[0146] The loss function is used to measure the gap between the model's predicted value and the true value. By optimizing the loss function, the model can be made more accurate and reliable. The model can be constrained using the BCEW i thLog i tLoss loss function, which combines the functions of the Sigmoid activation function and the Binary CrossEnt r opy Loss, thus improving numerical stability and gradient calculation. The gradient of the loss function is used to adjust the model parameters of the classification network to minimize the error between the predicted value and the actual label.

[0147] The feature retrieval library is a database containing a large number of extracted feature vectors. The pre-trained model extracts feature sequences from the speech training data set, and uses the K nearest neighbor method to perform clustering calculations on the feature sequences of the speech training data set to construct the feature retrieval library.

[0148] The purpose of the above convolutional layers is to extract more abstract and higher-level features from the input feature sequence. Each convolutional layer uses different filters (or convolution kernels) to scan the input data to generate a new feature map. The convolutional layer extracts local features by sliding small convolution kernels on the input sequence and integrates these features into a more abstract feature representation. Each convolutional layer is followed by a maximum pooling layer to reduce the spatial dimension of the features, reduce the amount of computation and increase the robustness of the model. The fully connected layer maps the feature vector obtained in the previous step to a high-dimensional space, making the differences between different categories more obvious.

[0149] This application uses a pre-trained model to extract features, and organically combines the extracted features through the attention mechanism, so that the final features have richer information and higher representation quality; three different network structures are used in the classification network to perform high-level feature processing on the input features. The classification network consists of a convolutional layer, a pooling layer, and a fully connected layer. The purpose of the convolutional layer is to extract more abstract and higher-level features from the input feature sequence. Each convolutional layer uses different filters (or convolution kernels) to scan the input data to generate a new feature map. The pooling layer can effectively reduce the dimension of the feature map, so that the subsequent network layer can process smaller input data. Each convolutional layer is connected to a maximum pooling layer to reduce the spatial dimension of the feature, reduce the amount of calculation and increase the robustness of the model. The fully connected layer maps the feature vector obtained in the previous step to a high-dimensional space, making the differences between different categories more obvious.

[0150] In this embodiment, the non-Company software tools or components that appear in the embodiments of this application are merely examples and do not represent actual use.

[0151] This embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0152] The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning. This application can be applied to the fields of artificial intelligence and big data, thereby promoting the construction of smart cities.

[0153] This application belongs to the field of smart cities, and this solution can promote the construction of smart cities.

[0154] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0155] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0156] Further references Figure 8 , as a response to the above Figure 2 The present application provides an embodiment of a voice authentication device based on feature clustering, and the embodiment of the voice authentication device based on feature clustering is similar to Figure 2 Corresponding to the embodiment of the voice authentication method based on feature clustering shown, the voice authentication device based on feature clustering can be specifically applied to various electronic devices.

[0157] like Figure 8 As shown, the speech authentication device 800 based on feature clustering described in this embodiment includes: a training module 801, a first extraction processing module 802 and an authentication processing module 803. Among them:

[0158] A training module 801 is used to obtain a speech training data set, and train a pre-trained speech authentication model through the audio training data set to obtain a trained speech authentication model, wherein the speech training data set includes the audio in the track data set;

[0159] The first extraction processing module 802 is used to perform feature sequence extraction processing on the target audio through the trained speech authentication model to obtain a target audio feature sequence;

[0160] The authentication processing module 803 is used to perform authentication processing on the target audio feature sequence using the K nearest neighbor method to determine the authentication processing result of the target audio.

[0161] In this embodiment, a speech training data set is obtained, a pre-trained speech authentication model is trained with the speech training data set to obtain a trained speech authentication model, a feature sequence extraction process is performed on the target audio with the trained speech authentication model to obtain a target audio feature sequence, and the K nearest neighbor method is used to perform authentication processing on the target audio features to determine the authentication processing result of the target audio, thereby solving the problems of low operating efficiency and accuracy of existing speech authentication methods.

[0162] See also Fig. 9 ,for Figure 8 The structure diagram of a specific implementation of a speech authentication device 800 based on feature clustering is shown in FIG. The speech authentication device 800 based on feature clustering further includes a second extraction processing module 804 and a calculation processing module 805. Among them:

[0163] The second extraction processing module 804 is used to perform feature sequence extraction processing on the speech training data set to obtain a feature sequence of the speech training data set;

[0164] The calculation processing module 805 is used to perform clustering calculation processing on the feature sequence of the speech training data set using the K nearest neighbor method to construct a feature retrieval library.

[0165] In this embodiment, a feature sequence of the speech training data set is obtained by performing feature sequence extraction processing on the speech training data set, and the feature sequence of the speech training data set is clustered using the K nearest neighbor method to construct a feature retrieval library, thereby improving recognition accuracy and efficiency.

[0166] See also Fig.10 ,for Fig. 9 The schematic diagram of the structure of a specific implementation of the calculation processing module 805 is shown in FIG. 8 , wherein the calculation processing module 805 includes a first calculation processing submodule 8051 and a clustering processing submodule 8052.

[0167] The first calculation processing submodule 8051 is used to calculate and process the feature sequence of the speech training data set using the K nearest neighbor method to obtain a category label of the feature sequence of the speech training data set;

[0168] The clustering processing submodule 8052 is used to perform clustering processing on the category labels to construct a feature retrieval library.

[0169] In this embodiment, the feature sequence of the speech training data set is calculated and processed by using the K-nearest neighbor method to obtain the category labels of the feature sequence of the speech training data set, and the category labels are clustered to construct a feature retrieval library. The feature retrieval library can be used to quickly find other feature sequences similar to the given feature sequence, thereby achieving efficient speech recognition.

[0170] Optionally, a loss function is used to constrain the model during the training process of the speech authentication model. The formula is as follows:

[0171] Loss = -[ylog(σ(x n ))+(1-y n )log(1-σ(x n ))]

[0172] Among them, y is the category label to which the audio belongs, σ(x n ) is the sigmoid function, which maps x to the (0,1) interval: y n To predict the category label that the audio belongs to.

[0173] In this embodiment, the loss function is used to constrain the model during the training process of the speech authentication model. The formula is as follows:

[0174] Loss = -[ylog(σ(x n ))+(1-y n )log(1-σ(x n ))]

[0175] Among them, y is the category label to which the audio belongs, σ(x n ) is the si gmod function, which maps x to the interval (0,1): y n In order to predict the category label to which the audio belongs, the si gmo id function is used to map the input x to the interval of (0,1), which can make the output smoother and more interpretable. During the training process, the value of the loss function is minimized, and the model can better fit the data and improve the accuracy of authentication.

[0176] See also Fig.11 ,for Figure 8 The structural diagram of a specific implementation of the authentication processing module 803 is shown in FIG. 8 , wherein the authentication processing module 803 includes a second calculation processing submodule 8031, a replacement processing submodule 8032, and an authentication processing submodule 8033.

[0177] The second calculation processing submodule 8031 ​​is used to calculate and process the target audio feature sequence using the K nearest neighbor method to obtain the average value of the K feature frames closest to each frame in the target audio feature sequence in the feature retrieval library;

[0178] A replacement processing submodule 8032 is used to perform replacement processing using the average value of the K most adjacent feature frames to obtain a new target audio feature sequence;

[0179] The authentication processing submodule 8033 is used to perform authentication processing on the new target audio feature sequence to obtain the authentication processing result of the target audio.

[0180] In this embodiment, the target audio feature sequence is calculated and processed by using the K nearest neighbor method to obtain the average value of the K feature frames closest to each frame in the target audio feature sequence in the feature retrieval library, and the average value of the K feature frames closest to each frame is used for replacement processing to obtain a new target audio feature sequence, and the new target audio feature sequence is subjected to authentication processing to obtain the authentication processing result of the target audio, thereby improving the quality and availability of the audio data.

[0181] Optionally, the replacement processing submodule 8032 is further configured to use the average value of the K most adjacent feature frames to replace the corresponding frames of the target audio feature sequence to obtain a new target feature sequence.

[0182] In this embodiment, a new target feature sequence is obtained by replacing the corresponding frame of the target audio feature sequence with the average value of the nearest K feature frames. Performing the K nearest neighbor regression method to replace the feature frame does not change the content of the audio, but only replaces other features of the audio with feature information similar to it in the training set, which is beneficial to the accurate classification of the network.

[0183] See also Fig.12 ,for Fig.11 The structural diagram of a specific implementation of the authentication processing submodule 8033 is shown in FIG. 8 , wherein the authentication processing submodule 8033 includes a feature extraction unit 80331 and an authentication processing unit 80332.

[0184] A feature extraction unit 80331 is used to perform feature extraction processing on the new target audio feature sequence to obtain a new target audio feature;

[0185] The authentication processing unit 80332 is used to perform authentication processing on the new target audio feature to obtain an authentication processing result of the target audio.

[0186] In this embodiment, the new target audio feature sequence is subjected to feature extraction processing to obtain new target audio features, and the new target audio features are subjected to authentication processing to obtain authentication results of the target audio, which can effectively improve the accuracy and reliability of feature extraction.

[0187] To solve the above technical problems, the present application also provides a computer device. Fig.13 , Fig.13 This is a basic structural block diagram of the computer device in this embodiment.

[0188] The computer device 13 includes a memory 131, a processor 132, and a network interface 133 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 13 having components 131-133, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0189] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0190] The memory 131 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc.

[0191] In some embodiments, the memory 131 may be an internal storage unit of the computer device 13, such as a hard disk or memory of the computer device 13. In other embodiments, the memory 131 may also be an external storage device of the computer device 13, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 13. Of course, the memory 131 may also include both the internal storage unit of the computer device 13 and its external storage device. In this embodiment, the memory 131 is generally used to store the operating system and various application software installed on the computer device 13, such as computer-readable instructions of a voice authentication method based on feature clustering, etc. In addition, the memory 131 may also be used to temporarily store various types of data that have been output or are to be output. In some embodiments, the processor 132 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 132 is generally used to control the overall operation of the computer device 13. In this embodiment, the processor 132 is used to run computer-readable instructions or process data stored in the memory 131, such as computer-readable instructions for running the voice authentication method based on feature clustering.

[0192] The network interface 133 may include a wireless network interface or a wired network interface. The network interface 133 is generally used to establish a communication connection between the computer device 13 and other electronic devices.

[0193] In this embodiment, the impact data of the system in the knowledge base can be used to generate a feature clustering-based voice authentication that is affected by the target system to be tested for the user, so that the user can intuitively obtain the impact range of the target system to be tested. According to the impact range of the target system to be tested, the impact of the change can be evaluated more accurately and quickly, irrelevant interference points can be eliminated, and regression use cases can be streamlined to avoid missing impact points in the early stage of the evaluation, resulting in the subsequent development of various systems. When they are jointly debugged, it is found that some associations have not been processed. In the mildest case, it will take up development time to supplement the logic. In the most serious case, the solution design may need to be overturned and redone, thereby improving the efficiency of project development.

[0194] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned feature clustering-based speech authentication method.

[0195] In this embodiment, the impact data of the system in the knowledge base can be used to generate a feature clustering-based voice authentication that is affected by the target system to be tested for the user, so that the user can intuitively obtain the impact range of the target system to be tested. According to the impact range of the target system to be tested, the impact of the change can be evaluated more accurately and quickly, irrelevant interference points can be eliminated, and regression use cases can be streamlined to avoid missing impact points in the early stage of the evaluation, resulting in the subsequent development of various systems. When they are jointly debugged, it is found that some associations have not been processed. In the mildest case, it will take up development time to supplement the logic. In the most serious case, the solution design may need to be overturned and redone, thereby improving the efficiency of project development.

[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0197] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for detecting fake speech based on feature clustering, characterized in that: The steps include: Acquire a speech training data set, and train a pre-trained speech authentication model using the speech training data set to obtain a trained speech authentication model, wherein the speech training data set includes audio in the track data set; The trained speech authentication model is used to extract the feature sequence of the target audio to obtain the target audio feature sequence; The target audio feature sequence is subjected to authentication processing using the K nearest neighbor method to determine an authentication processing result of the target audio.

2. The method for identifying false voice based on feature clustering according to claim 1, characterized in that: After acquiring the speech training data set, the method further includes: Performing feature sequence extraction processing on the speech training data set to obtain a feature sequence of the speech training data set; The K nearest neighbor method is used to perform clustering calculation processing on the feature sequence of the speech training data set to construct a feature retrieval library.

3. The method for identifying false voice based on feature clustering according to claim 2, characterized in that: The method of using the K nearest neighbor method to perform clustering calculation processing on the feature sequence of the speech training data set to construct a feature retrieval library includes: Using the K nearest neighbor method to calculate and process the feature sequence of the speech training data set to obtain a category label of the feature sequence of the speech training data set; The category labels are clustered to construct a feature retrieval library.

4. The method for identifying counterfeit speech based on feature clustering according to claim 3, characterized in that: The loss function is used to constrain the model during the training process of the speech authentication model. The formula is as follows: Loss=-[ylog(σ(x n ))+(1-y n )log(1-σ(x n ))] Among them, y is the category label to which the audio belongs, σ(x n ) is the sigmoid function, which maps x to the (0,1) interval: y n To predict the category label that the audio belongs to.

5. The method for identifying false voice based on feature clustering according to claim 4, characterized in that: The method of using the K nearest neighbor method to perform authentication processing on the target audio feature sequence to determine the authentication processing result of the target audio includes: The target audio feature sequence is calculated and processed using a K nearest neighbor method to obtain an average value of the K feature frames closest to each frame in the target audio feature sequence in the feature retrieval library; Using the average value of the K most adjacent feature frames to perform replacement processing to obtain a new target audio feature sequence; Perform authentication processing on the new target audio feature sequence to obtain an authentication result of the target audio.

6. The method for identifying false voices based on feature clustering according to claim 5, characterized in that: The replacing process using the average value of the most adjacent K feature frames to obtain a new target audio feature sequence includes: The corresponding frames of the target audio feature sequence are replaced by the average value of the K most adjacent feature frames to obtain a new target feature sequence.

7. The method for identifying false voices based on feature clustering according to claim 6, characterized in that: The performing authentication processing on the new target audio feature sequence to obtain the authentication processing result of the target audio includes: Performing feature extraction processing on the new target audio feature sequence to obtain new target audio features; Perform authentication processing on the new target audio feature to obtain an authentication processing result of the target audio.

8. A voice authentication device based on feature clustering, characterized in that: include: A training module, used to obtain a speech training data set, train a pre-trained speech authentication model through the speech training data set, and obtain a trained speech authentication model based on feature clustering, wherein the speech training data set includes audio in the track data set; The first feature extraction processing module is used to perform feature sequence extraction processing on the target audio through the trained speech authentication model to obtain the target audio feature sequence; The authentication processing module is used to perform authentication processing on the target audio feature sequence using the K nearest neighbor method to determine the authentication processing result of the target audio.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the method for identifying counterfeit speech based on feature clustering as claimed in any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the voice authentication method based on feature clustering as described in any one of claims 1 to 7 are implemented.

Images