Voice authentic identification method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, finance and medical health, and discloses a voice authentication method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring to-be-identified voice information; inputting the voice information to be subjected to authentic identification into the self-supervised large model for feature extraction to obtain an extraction result; inputting the potential speech representation features into a classification model for deep coding to obtain a coding result, converting the coding result into low-dimensional representation, and calculating the probability distribution of speech authenticity to obtain a classification result; and outputting a classification result. By implementing the method provided by the invention, the capability of identifying the forged voice in high-imitation audio and noise environments can be remarkably improved, and reliable guarantee is provided for voice safety in key fields such as finance, medical health and the like.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, finance, and medical health technology, and more specifically to a voice authentication method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the rapid advancement of speech synthesis and voice conversion technologies, especially the significant improvement in the quality of deepfake voices, this has posed an unprecedented challenge to voice authentication technology. Traditional voice authentication methods mainly rely on hand-crafted feature extraction and shallow machine learning models, which are powerless in the face of increasingly complex forgeries. In recent years, although deep learning technology has made significant progress in the field of voice authentication, it still faces a series of problems. For example, current methods perform poorly in identifying highly realistic forged voices generated by advanced technologies such as generative adversarial networks. Traditional models perform poorly in noisy environments and are easily affected by background noise and differences in recording equipment. Supervised learning methods are highly dependent on large amounts of accurately labeled data, and the high cost of obtaining high-quality forged voice samples limits the generalization ability and practicality of the models.

[0003] The application of voice authentication technology is crucial in the financial industry. For example, in areas such as remote identity verification and customer service automation, ensuring the security and authenticity of voice interactions is key to preventing fraud. However, the development of deepfake voice technology poses significant risks to identity verification conducted over the phone or through online services. Financial institutions require more advanced voice authentication solutions to protect customer information and assets while ensuring an uncompromising user experience. In healthcare, voice technology is used for patient monitoring, remote diagnosis, and health management. For example, analyzing a patient's voice patterns can help detect diseases such as Parkinson's disease and depression at an early stage. However, with the advancement of voice authentication technology, the risk of false medical consultations or misdiagnoses increases. Ensuring the authenticity and reliability of voice interactions is crucial for maintaining the quality of healthcare services and patient safety. Therefore, developing voice authentication systems that can effectively defend against deepfake attacks has become a key issue in the healthcare sector.

[0004] Therefore, it is necessary to design a new method to significantly improve the ability to recognize high-imitation audio and forged voices in noisy environments, and provide reliable protection for voice security in key areas such as finance and healthcare. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a voice authentication method, device, computer equipment and storage medium.

[0006] To achieve the above object, the present invention adopts the following technical solution: a voice authentication method, comprising:

[0007] Obtain the voice information to be authenticated;

[0008] Inputting the speech information to be detected as false into the self-supervised large model for feature extraction to obtain an extraction result;

[0009] Inputting the extraction result into a classification model for deep encoding to obtain an encoding result, converting the encoding result into a low-dimensional representation, and calculating the probability distribution of the true and false speech to obtain a classification result;

[0010] The classification result is output.

[0011] Its further technical solution is: the classification model is obtained by enhancing the speech information and extracting features from the self-supervised large model, and then using the extracted features as a sample set to train the ConvNeXt v2 network.

[0012] A further technical solution is: inputting the speech information to be detected as false into the self-supervised large model for feature extraction to obtain an extraction result, including:

[0013] Using a multi-layer convolutional network of a self-supervised large model to convert the speech information to be detected into a potential speech representation;

[0014] Masking the latent speech representation in partial time steps and generating a contextual representation using a contextual network of a Transformer architecture;

[0015] A true quantized latent speech representation is identified according to the context representation to obtain an extraction result.

[0016] A further technical solution is: inputting the extraction result into a classification model for deep encoding to obtain an encoding result, converting the encoding result into a low-dimensional representation, and calculating the probability distribution of the authenticity of the speech to obtain a classification result, including:

[0017] Normalizing the extraction results to obtain normalized results;

[0018] Inputting the normalized result into a classification model for deep encoding to obtain an encoding result;

[0019] Extract high-dimensional feature representation from the encoding results;

[0020] Converting the high-dimensional feature representation into a low-dimensional feature representation through a fully connected layer;

[0021] The probability distribution of the authenticity of the speech is calculated based on the low-dimensional feature representation to obtain a classification result.

[0022] Its further technical solution is: the classification model includes a multi-layer structure, each layer of the structure contains multiple convolution blocks, and each of the convolution blocks includes a depth-separable convolution and an inverted bottleneck structure.

[0023] A further technical solution is: calculating the probability distribution of speech authenticity based on the low-dimensional feature representation to obtain a classification result, including:

[0024] The softmax function is used to calculate the probability distribution of the true and false speech according to the low-dimensional feature representation to obtain a classification result.

[0025] Its further technical solution is: the classification model optimizes model parameters by combining the cross entropy loss function and the contrastive learning method.

[0026] The present invention also provides a voice authentication device, comprising:

[0027] An information acquisition unit, configured to acquire voice information to be authenticated;

[0028] A feature extraction unit is used to input the speech information to be detected as false into the self-supervised large model to perform feature extraction to obtain an extraction result;

[0029] A classification unit, configured to input the extraction result into a classification model for deep encoding to obtain an encoding result, convert the encoding result into a low-dimensional representation, and calculate a probability distribution of the authenticity of the speech to obtain a classification result;

[0030] An output unit is used to output the classification result.

[0031] The present invention further provides a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0032] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0033] The beneficial effects of the present invention compared with the prior art are as follows: the present invention obtains the voice information to be authenticated and uses a large self-supervised model to perform deep feature extraction, captures the rich semantic and acoustic features in the voice, and then inputs these features into a specially designed classification model for further deep encoding and classification discrimination, and finally outputs the probability distribution of the voice authenticity as the classification result. This method fully utilizes the advantages of self-supervised learning in processing large-scale unlabeled data, and combines the efficient convolutional structure to enhance the recognition ability of high-imitation audio and forged voice in noisy environments, significantly improving the robustness and accuracy of the voice authentication system, providing solid and reliable voice security for key areas such as financial transaction verification and telemedicine services, and ensuring the security and reliability of the user identity authentication process; through this technical means, it can not only effectively prevent security threats caused by deep fake audio, but also maintain a high degree of adaptability and accuracy in complex application scenarios.

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 A schematic diagram of an application scenario of the voice authentication method provided by an embodiment of the present invention;

[0037] Figure 2 A flow chart of a voice authentication method according to an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a sub-process of a voice authentication method according to an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of a sub-process of a voice authentication method according to an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of a convolutional block of a ConvNeXt v2 network provided in an embodiment of the present invention;

[0041] Figure 6 A schematic block diagram of a voice authentication device provided by an embodiment of the present invention;

[0042] Figure 7 A schematic block diagram of a feature extraction unit of a voice authentication device according to an embodiment of the present invention;

[0043] Figure 8 A schematic block diagram of a classification unit of a voice authentication device according to an embodiment of the present invention;

[0044] Figure 9 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0046] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0047] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0048] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0049] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the voice authentication method provided by an embodiment of the present invention. Figure 2A schematic flow chart of a voice authentication method provided in an embodiment of the present invention. The voice authentication method is applied in a server. The server interacts with the terminal for data, extracts features through a large self-supervisory model, and uses a specially designed classification model such as the ConvNeXt v2 network for deep encoding, combining multi-layer convolution blocks, depth-separable convolutions, and inverted bottleneck structures to improve feature expression capabilities; at the same time, normalization processing, context representation enhancement, and partial time step masking strategies are adopted to enhance robustness to high-imitation audio and noisy environments. Finally, the probability distribution of the authenticity of the voice is calculated by the softmax function to obtain the classification result, and the model parameters are optimized with the help of the cross-entropy loss function and contrastive learning, thereby significantly improving the recognition ability of forged voices, and providing reliable protection for voice security in key areas such as finance and healthcare.

[0050] Figure 2 FIG. 1 is a flow chart of a voice authentication method according to an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S140.

[0051] S110: Acquire voice information to be authenticated.

[0052] In this embodiment, the voice information to be authenticated refers to the voice data that requires authentication. This voice data can come from a variety of sources and scenarios, such as phone recordings, audio clips from video conferences, and audio recorded by smart devices. The voice information to be authenticated can be unprocessed audio files directly obtained from the original recording device, or it can be audio data that has undergone certain preprocessing steps, such as format conversion and sampling rate adjustment.

[0053] Specifically, voice data is collected from a designated source using appropriate recording equipment or interfaces. This may involve the use of specialized recording software or hardware to ensure voice quality and integrity. Preliminary organization and preparation of the collected voice data is performed, including but not limited to removing silence, normalizing volume levels, and adjusting the sampling rate to match model input requirements, to ensure data consistency and applicability.

[0054] Depending on actual needs, enhancement techniques can be applied to the voice data at this stage, such as adding noise, time stretching or pitch shifting, to simulate different real-world environmental conditions and improve the model's ability to identify forged voices in complex environments.

[0055] Convert the prepared speech data into a format suitable for subsequent steps. Typically, this means converting the speech signal into a digital representation and organizing it into a specific data structure to facilitate feature extraction using the self-supervised large model Wav2Vec 2.0.

[0056] The speech information obtained through the above steps, as the first step in the entire speech authentication process, provides the foundation for subsequent feature extraction using Wav2Vec 2.0 and deep encoding using the ConvNeXt v2 network. This process is crucial to ensuring the accuracy and reliability of the final authentication results.

[0057] Furthermore, when processing the input speech signal, various data augmentation techniques are introduced to simulate real-world speech variations. These include adding Gaussian noise, time stretching, and pitch shifting. First, the noise addition process involves randomly generating white noise that conforms to a Gaussian distribution and mixing it into the original audio signal at a specific ratio to simulate random interference in a real-world environment. Second, time stretching simulates speech variations at different speeds by adjusting the speech rate (faster or slower), with a control range of 0.8 to 1.2 times. This helps improve the model's adaptability to changes in speech rhythm. Finally, pitch shifting simulates the variations in audio pitch. Using the pitch shift function in the audio processing library, the audio is randomly raised or lowered by two semitones to reflect speech under different pitch conditions. Through these data augmentation techniques, the diversity of speech data can be significantly expanded, thereby improving the model's generalization and robustness.

[0058] S120: Input the speech information to be authenticated into the self-supervised large model for feature extraction to obtain an extraction result.

[0059] In this example, the extraction result refers to the final speech feature representation generated by the Wav2Vec 2.0 self-supervised large model. This feature representation not only contains rich semantic information and acoustic features, but also has high discriminative and robust properties, making it suitable for subsequent feature encoding and classification stages. It is crucial for achieving high-precision and robust speech authentication.

[0060] In one embodiment, see Figure 3 , the above-mentioned step S120 may include steps S121 to S123.

[0061] S121. Use a multi-layer convolutional network of a self-supervised large model to convert the speech information to be identified as false speech into a potential speech representation.

[0062] In this embodiment, the latent speech representation is an intermediate representation generated by processing the input speech information to be authenticated using a multi-layer convolutional network in Wav2Vec 2.0. This representation is designed to capture the rich acoustic and semantic information in the speech signal, including global context and local details. As the basis for subsequent processing, the latent speech representation effectively reflects the essential properties of speech.

[0063] S122. Perform partial time step masking on the potential speech representation, and generate a contextual representation using a context network of a Transformer architecture.

[0064] In this embodiment, the contextual representation is generated by masking some of the time steps in the latent speech representation and inputting them into a context network based on the Transformer architecture. The contextual representation not only contains information about the original speech signal but also incorporates information about surrounding time steps, thereby better capturing the temporal dynamics and long- and short-term dependencies of the speech signal. This approach helps enhance the model's adaptability to changes in speech rhythm and improves its generalization performance in complex scenarios.

[0065] S123. Identify the true quantized potential speech representation according to the context representation to obtain an extraction result.

[0066] In this embodiment, in this step, based on the context representation, the model needs to complete a comparison task, that is, to identify the true quantized potential speech representation from a set of K+1 candidate representations. These candidate representations include the true quantized potential speech representation and the remaining K interference terms, which are uniformly sampled from different masked time steps of the same input. By calculating the cosine similarity between the context representation and each candidate representation, and adjusting the dimension through the projection layer, the model can learn the ability to distinguish between real samples and negative samples, and finally output accurate extraction results. This process encourages the model to generate high similarity scores for the correct target, while penalizing the similarity with the wrong target, making the model's predictions more and more accurate.

[0067] Specifically, deep feature extraction is performed on preprocessed speech data using the pretrained self-supervised large model Wav2Vec 2.0. This pretrained Wav2Vec 2.0 is an open-source self-supervised learning model from Facebook. Its core is a feature encoder consisting of a multi-layer convolutional network, which converts the input audio into a latent speech representation z. Subsequently, a quantization module discretizes z into a finite set of speech representations q, which serve as the target in the self-supervised objective. Simultaneously, some time steps of z are masked and fed into a context network based on the Transformer architecture to construct the context representation c.

[0068] In the pre-training phase, the model learns audio representations by solving contrast tasks, that is, at each masked time step, the self-supervised large model needs to identify the correct quantized potential audio representation from a set of K+1 candidate representations (including the true quantized potential speech representation and other K interference items). The interference items are uniformly sampled from different masked time steps of the same input. The model calculates the cosine similarity between and (the dimension needs to be matched through a projection layer), where the contrast loss encourages high similarity with the true positive sample target and penalizes high similarity scores with the negative sample target, thereby continuously improving the accuracy of the prediction. Specifically, for a given context network output c centered at the masked time step t, t , the model needs to contain a set of K+1 quantized candidate representations q'∈Q t Identify the true quantized latent speech representation q t (Q t Including q t and K interference terms, where the interference terms are uniformly sampled from other masked time steps of the same input), model comparison c t With q'∈Q t The cosine similarity between t A projection layer is required to match q t The contrastive loss encourages high similarity with the true positive target and penalizes high similarity scores with the negative target, making the prediction more and more accurate.

[0069] Since the Wav2Vec 2.0 structure has demonstrated good feature representation capabilities in large-scale unlabeled speech data, it can capture the global context information and local detail features of speech signals and generate speech feature representations rich in semantic information.

[0070] Therefore, in this embodiment, the Wav2Vec 2.0 pre-trained model is used as a feature extractor to extract the context representation of the audio and provide input features for the subsequent authentication method.

[0071] S130: Input the extraction result into a classification model for deep encoding to obtain an encoding result, convert the encoding result into a low-dimensional representation, and calculate the probability distribution of the authenticity of the speech to obtain a classification result.

[0072] In this embodiment, the classification result refers to the probability of whether the speech is true or false.

[0073] The classification model includes a multi-layer structure, each layer of the structure includes multiple convolution blocks, and each of the convolution blocks includes a depth-separable convolution and an inverted bottleneck structure.

[0074] Specifically, the classification model adopts a multi-layer design, with each layer containing multiple convolutional blocks. This design allows the model to extract features at different levels of the speech signal layer by layer.

[0075] Depthwise separable convolution decomposes standard convolution into depthwise convolution and pointwise convolution. The former is responsible for single-channel information extraction, and the latter is responsible for cross-channel information integration, reducing computational complexity and parameter quantity.

[0076] The inverted bottleneck structure enhances the expressiveness of the model through nonlinear activation functions, while reducing the number of parameters and computational complexity, effectively improving the model's ability to learn different features.

[0077] The classification model is obtained by training a ConvNeXt v2 network using the sample set after enhancing the speech information and extracting features from the self-supervised large model.

[0078] The classification model optimizes model parameters by combining the cross entropy loss function and contrastive learning method.

[0079] First, the original speech data is enhanced (such as adding noise, time stretching, pitch shifting, etc.), and then the pre-trained self-supervised large model Wav2Vec 2.0 is used to extract features. These features are used as sample sets to train the ConvNeXtv2 network.

[0080] The cross-entropy loss function is used to optimize the model's performance on the classification task, ensuring that the model can accurately distinguish between true and false speech.

[0081] By calculating the similarity matrix and constructing the positive and negative sample mask matrix, the model parameters are optimized, the distance between similar samples is shortened, the distance between heterogeneous samples is increased, and the discriminative power of the model is improved.

[0082] Specifically, the feature extraction results obtained from the Wav2Vec 2.0 model are normalized to ensure that all features are within the same scale range for subsequent processing. The normalized features are then fed into the ConvNeXt v2 model for deep encoding. Its unique convolutional block structure (including depthwise separable convolutions and an inverted bottleneck structure) further enhances the robustness and discriminability of the features, generating an encoded result. Highly discriminative high-dimensional feature representations are extracted from the encoded result, which contain rich semantic information and acoustic properties. A fully connected layer converts the high-dimensional feature representation into a low-dimensional feature representation suitable for classification tasks, simplifying the data dimension while retaining key classification information. Finally, a softmax function is used to calculate the probability distribution of the speech authenticity based on the low-dimensional feature representation to obtain the final classification result. This step provides a probability estimate of whether the speech is authentic or forged, supporting decision-making.

[0083] In summary, step S130 achieves efficient conversion from raw speech signals to final classification results through a series of carefully designed data processing and model manipulation processes. This method not only improves recognition accuracy but also enhances the model's ability to cope with various complex situations, providing strong technical support for the field of speech security.

[0084] In one embodiment, see Figure 4 , the above-mentioned step S130 may include steps S131 to S135.

[0085] S131 , normalizing the extraction result to obtain a normalized result.

[0086] In this embodiment, the normalized result refers to the result obtained after normalizing the extraction result.

[0087] Normalization ensures that the input data has the same scale for ease of subsequent processing. This can be done by subtracting the mean from each feature value in the extracted data and dividing it by its standard deviation (Z-score normalization), or by scaling it to a specific range (such as [0, 1]). This eliminates the dimensionality effects between features and improves the stability and efficiency of model training.

[0088] S132: Input the normalized result into a classification model for deep coding to obtain a coding result.

[0089] In this embodiment, the encoding result refers to a high-dimensional vector containing multi-level abstract features of the input data obtained after processing by a deep model.

[0090] The normalized data is then fed into a classification model (such as a ConvNeXt v2 network) for deep encoding. This process involves applying multiple convolutional layers and nonlinear activation functions, enabling the model to learn a multi-level, abstract representation of the input data. The resulting encoding is a feature vector, processed through the model's multi-layered structure. It contains the high-dimensional features of the original data and is optimized for subsequent feature extraction and classification tasks.

[0091] S133. Extract high-dimensional feature representation from the encoding result.

[0092] In this embodiment, the high-dimensional feature representation refers to a feature vector extracted from the encoding result that can reflect the core characteristics of the input data and rich semantic information.

[0093] High-dimensional feature representations are highly discriminative feature vectors extracted from the encoding results. These feature vectors typically contain rich semantic information and details, reflecting the core characteristics of the input data. At this stage, additional techniques, such as pooling or attention mechanisms, are applied to further refine and enhance these features.

[0094] S134. Convert the high-dimensional feature representation into a low-dimensional feature representation through a fully connected layer.

[0095] In this embodiment, the low-dimensional feature representation refers to a simplified feature vector after mapping high-dimensional features to a low-dimensional space through a fully connected layer, which retains key information and is easy to classify and process.

[0096] To simplify model complexity and reduce the risk of overfitting, high-dimensional feature representations are mapped to a lower-dimensional space through fully connected layers. This process involves multiplying the weight matrix by the input feature vector, adding a bias term, and possibly passing it through an activation function. This low-dimensional feature representation not only retains the key information of the original high-dimensional features but also makes it easier to perform tasks such as classification.

[0097] S135 . Calculate the probability distribution of the speech authenticity based on the low-dimensional feature representation to obtain a classification result.

[0098] In this embodiment, a softmax function is used to calculate the probability distribution of speech authenticity based on the low-dimensional feature representation to obtain a classification result.

[0099] The final step is to predict whether the speech is genuine based on the low-dimensional feature representation. This is typically achieved by applying the softmax function, which converts the scores for each category into probabilities such that the sum of all category probabilities equals 1. For a binary classification problem (true / false), the softmax function produces two probabilities, one representing the likelihood that the input sample belongs to the "true" or "false" category. Ultimately, the category with the greater probability is selected as the classification result.

[0100] In this embodiment, step S130 further enhances the speech features extracted from the self-supervised large model to improve their robustness and stability. First, a normalization step is performed to eliminate inconsistencies in the data distribution, ensuring consistency and smoothness in subsequent processing. Next, the enhanced speech features are input into the ConvNeXt v2 network for encoding.

[0101] The ConvNeXt v2 architecture consists of multiple stages, such as Figure 5As shown in the figure, LN represents normalization; GELU is a Gaussian error linear unit (GELU), an activation function; GRN is a layer consisting of depthwise separable convolution and an inverted bottleneck structure; and each stage contains several convolution blocks. Each convolution block is primarily composed of depthwise separable convolution and an inverted bottleneck structure. Depthwise separable convolution significantly reduces computational complexity and parameter count by decomposing the standard convolution into two steps: depthwise convolution and pointwise convolution. Specifically, depthwise convolution applies the convolution kernel independently to each input channel, extracting information only in the spatial dimension, similar to the effect of the self-attention mechanism. Subsequently, pointwise convolution weightedly combines the results of the depthwise convolution in the channel direction to generate a new feature map. This design not only reduces the amount of computation and parameters but also enhances the model's feature representation capability. The inverted bottleneck structure, by introducing a nonlinear activation function, improves the model's nonlinear representation capability while maintaining low computational cost, thereby capturing more effective feature information.

[0102] ConvNeXt v2 can generate more discriminative high-dimensional feature representations, providing strong support for subsequent classification tasks. During the classification process, a fully connected layer first reduces and maps the high-dimensional features output by ConvNeXt v2, converting them into low-dimensional feature representations suitable for classification. A softmax function is then used to calculate the probability distribution of the authenticity of the speech, ultimately outputting the classification result.

[0103] During model training, a cross-entropy loss function is used to optimize model parameters, and contrastive learning is used to further improve the model's discriminative capabilities. Within a batch, samples with the same label are considered positive, while samples with different labels are considered negative. When calculating contrastive loss, a similarity matrix is ​​first constructed based on the input feature representation. Next, a positive sample mask matrix is ​​created based on the input label matrix. Contrastive loss is calculated by calculating the log-probability matrix for the similarity matrix and, combined with the mask matrix, eliminating the self-product. The formula is as follows: where z i is the feature representation of the input, P(i) is the index set of samples of the same category as sample i, A(i) is the index set of all samples except i, and τ is the temperature parameter that controls the sharpness of the similarity distribution. The numerator of the function exp(z i ·z p / τ) is the similarity between sample i and all positive samples p, and the denominator ∑ a∈A(i) exp(z i ·z a / τ) contains the similarity of all samples (positive and negative). By minimizing the entire fraction, the numerator is maximized (that is, the similarity between sample i and all positive samples p) and the denominator is minimized (that is, the similarity between sample i and negative samples). Contrastive learning improves the model's ability to distinguish forged speech by narrowing the similarity between similar samples and widening the similarity between different samples, thereby improving classification accuracy.

[0104] S140: Output the classification result.

[0105] In this embodiment, the classification result is output to the terminal for display.

[0106] The method of this embodiment combines the advantages of the self-supervised learning model Wav2Vec2 and the ConvNeXt V2 network. First, the Wav2Vec2 model uses self-supervised learning on large-scale unlabeled speech data to deeply mine and extract rich semantic information and acoustic features from speech, including global contextual relationships and local details. Next, the ConvNeXt network is used to further encode these features at multiple levels, leveraging its efficient convolutional structure and powerful global modeling capabilities to enhance the discriminability and robustness of the features.

[0107] This method not only excels in identifying highly authentic audio (such as forged speech generated by generative adversarial networks), but also demonstrates excellent generalization capabilities in complex scenarios such as noisy environments and cross-device recordings. As a result, it achieves highly accurate and robust voice authentication, providing strong technical support for ensuring voice security.

[0108] Specifically, the method of this embodiment includes the following steps: first, after augmenting the speech data, feature extraction is performed using the wav2vec2 model; then, the extracted features are input into a ConvNeXt V2 network for encoding and classification; finally, during training, the entire model is jointly optimized by integrating cross-entropy loss and contrastive learning loss. This system is both efficient and highly robust, effectively addressing the challenges posed by deepfake audio and ensuring technical reliability in the field of voice security. This approach improves the performance of existing voice authentication systems.

[0109] For example, a large commercial bank decided to introduce a voice authentication system based on the self-supervised learning model Wav2Vec 2.0 and ConvNeXt V2 network in its telephone banking service to improve the security of customer service.

[0110] When a customer calls the bank's customer service hotline, the system automatically records the voice clips of the call.

[0111] These voice clips have undergone preliminary processing, including removing silent segments and standardizing volume levels, to ensure the consistency and applicability of the data.

[0112] If needed, enhancement techniques such as noise, time stretching or pitch shifting can be added to these voice clips to simulate different real-world environmental conditions and improve the model's ability to identify forged voices in complex environments.

[0113] Use the pre-trained Wav2Vec 2.0 model to perform deep feature extraction on the pre-processed speech data to generate a representation containing rich semantic information and acoustic features.

[0114] The context network of the Transformer architecture is used to further process the potential speech representation to capture the temporal dynamics and long-term and short-term dependencies of the speech signal.

[0115] The extracted features are input into the ConvNeXt V2 network for encoding and classification.

[0116] The high-dimensional features are mapped to a low-dimensional space through the fully connected layer, and the probability distribution of the authenticity of the speech is calculated to obtain the final classification result.

[0117] During the training process, the cross-entropy loss and contrastive learning loss are integrated to jointly optimize the entire model and enhance the model's ability to distinguish forged speech.

[0118] Based on the classification results, the system determines whether the voice in the call is the real customer voice and provides corresponding financial services accordingly.

[0119] After deploying this voice authentication system, the bank successfully intercepted several fraud attempts to invade customer accounts through voice impersonation, significantly improving the security of financial services and user experience.

[0120] To give another example: A leading telemedicine company uses the voice authentication technology based on the present invention as part of its service platform for patient identity verification.

[0121] Before each session, the system asks the patient to read a randomly generated text message to collect a voice sample. The collected voice sample undergoes preliminary processing, such as removing background noise and adjusting the sampling rate, to ensure data quality.

[0122] The Wav2Vec 2.0 model is used to perform deep feature extraction on speech samples, generating a representation that reflects the patient's unique voiceprint characteristics. The extracted features are then fed into a ConvNeXt V2 network for encoding and classification.

[0123] By comparing the voice features in the current conversation with the previously stored patient voiceprint template, it is determined whether the current speaker is the registered patient himself.

[0124] The system confirms the patient's identity based on the classification results and allows them to continue to receive online diagnosis and treatment services.

[0125] This approach significantly improves the security of remote diagnosis and treatment, reduces the risk of misdiagnosis due to identity theft, strengthens patient trust in the platform, and promotes the healthy development of telemedicine services. This approach not only ensures the authenticity and security of doctor-patient communication, but also provides patients with a more convenient and efficient medical service experience.

[0126] The aforementioned voice authentication method obtains the voice information to be authenticated and uses a large self-supervised model for deep feature extraction, capturing the rich semantic and acoustic features in the voice. These features are then input into a specially designed classification model for further deep encoding and classification, ultimately outputting a probability distribution of the voice authenticity as the classification result. This method fully leverages the advantages of self-supervised learning in processing large-scale unlabeled data. Combined with an efficient convolutional structure, it enhances the ability to recognize high-fidelity audio and forged speech in noisy environments, significantly improving the robustness and accuracy of the voice authentication system. This provides solid and reliable voice security for key areas such as financial transaction verification and telemedicine services, ensuring the security and reliability of the user identity verification process. This technical approach not only effectively prevents security threats caused by deep fake audio, but also maintains a high degree of adaptability and accuracy in complex application scenarios.

[0127] Figure 6 FIG is a schematic block diagram of a voice authentication device 300 provided by an embodiment of the present invention. Figure 6 As shown, corresponding to the above voice authentication method, the present invention also provides a voice authentication device 300. The voice authentication device 300 includes a unit for executing the above voice authentication method, and the device can be configured in a server. Figure 6 The voice authentication device 300 includes an information acquisition unit 301 , a feature extraction unit 302 , a classification unit 303 and an output unit 304 .

[0128] The information acquisition unit 301 is used to obtain the speech information to be authenticated; the feature extraction unit 302 is used to input the speech information to be authenticated into the self-supervised large model for feature extraction to obtain an extraction result; the classification unit 303 is used to input the extraction result into the classification model for deep encoding to obtain an encoding result, and convert the encoding result into a low-dimensional representation, and calculate the probability distribution of the authenticity of the speech to obtain a classification result; the output unit 304 is used to output the classification result.

[0129] In one embodiment, if Figure 7 As shown, the feature extraction unit 302 includes a first conversion subunit 3021 , a first generation subunit 3022 and an identification subunit 3023 .

[0130] The first conversion subunit 3021 is used to use the multi-layer convolutional network of the self-supervised large model to convert the speech information to be identified into a potential speech representation; the first generation subunit 3022 is used to perform partial time step masking on the potential speech representation and generate a context representation using the context network of the Transformer architecture; the recognition subunit 3023 is used to identify the true quantized potential speech representation based on the context representation to obtain an extraction result.

[0131] In one embodiment, if Figure 8 As shown, the classification unit 303 includes a normalization subunit 3031 , an encoding subunit 3032 , an extraction subunit 3033 , a second conversion subunit 3034 and a calculation subunit 3035 .

[0132] The normalization subunit 3031 is used to normalize the extraction result to obtain a normalized result; the encoding subunit 3032 is used to input the normalized result into the classification model for deep encoding to obtain an encoding result; the extraction subunit 3033 is used to extract a high-dimensional feature representation from the encoding result; the second conversion subunit 3034 is used to convert the high-dimensional feature representation into a low-dimensional feature representation through a fully connected layer; the calculation subunit 3035 is used to calculate the probability distribution of the authenticity of the speech based on the low-dimensional feature representation to obtain a classification result.

[0133] In one embodiment, the calculation subunit 3035 is configured to use a softmax function to calculate the probability distribution of the speech authenticity based on the low-dimensional feature representation to obtain a classification result.

[0134] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned voice authentication device 300 and each unit can refer to the corresponding description in the above-mentioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0135] The above-mentioned voice authentication device 300 can be implemented in the form of a computer program. The computer program can be used in Figure 9 Runs on the computer device shown.

[0136] See also Figure 9 , Figure 91 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0137] See Figure 9 The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0138] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, can enable the processor 502 to execute a voice authentication method.

[0139] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0140] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a voice authentication method.

[0141] The network interface 505 is used to communicate with other devices through the network. Figure 9 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0142] The processor 502 is configured to execute a computer program 5032 stored in the memory to implement the following steps:

[0143] Acquire speech information to be authenticated; input the speech information to be authenticated into a large self-supervised model for feature extraction to obtain an extraction result; input the extraction result into a classification model for deep encoding to obtain an encoding result, and convert the encoding result into a low-dimensional representation, and calculate the probability distribution of the speech authenticity to obtain a classification result; output the classification result.

[0144] The classification model is obtained by enhancing speech information and extracting features from a self-supervised large model, and then using the extracted features as a sample set to train a ConvNeXt v2 network.

[0145] In one embodiment, when the processor 502 implements the step of inputting the speech information to be detected as false into the self-supervised large model for feature extraction to obtain an extraction result, the processor 502 specifically implements the following steps:

[0146] The multi-layer convolutional network of the self-supervised large model is used to convert the false speech information to be identified into a potential speech representation; the potential speech representation is partially time-step masked, and a contextual representation is generated using the context network of the Transformer architecture; and the true quantized potential speech representation is identified based on the contextual representation to obtain an extraction result.

[0147] In one embodiment, when the processor 502 implements the step of inputting the extraction result into the classification model for deep encoding to obtain an encoding result, converting the encoding result into a low-dimensional representation, and calculating the probability distribution of the speech authenticity to obtain a classification result, the processor 502 specifically implements the following steps:

[0148] The extraction result is normalized to obtain a normalized result; the normalized result is input into a classification model for deep encoding to obtain an encoding result; a high-dimensional feature representation is extracted from the encoding result; the high-dimensional feature representation is converted into a low-dimensional feature representation through a fully connected layer; and the probability distribution of the authenticity of the speech is calculated based on the low-dimensional feature representation to obtain a classification result.

[0149] The classification model includes a multi-layer structure, each layer of the structure includes multiple convolution blocks, and each of the convolution blocks includes a depth-separable convolution and an inverted bottleneck structure.

[0150] The classification model optimizes model parameters by combining the cross entropy loss function and contrastive learning method.

[0151] In one embodiment, when the processor 502 implements the step of calculating the probability distribution of speech authenticity based on the low-dimensional feature representation to obtain a classification result, it specifically implements the following steps:

[0152] The softmax function is used to calculate the probability distribution of the true and false speech according to the low-dimensional feature representation to obtain a classification result.

[0153] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0154] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0155] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0156] Acquire speech information to be authenticated; input the speech information to be authenticated into a large self-supervised model for feature extraction to obtain an extraction result; input the extraction result into a classification model for deep encoding to obtain an encoding result, and convert the encoding result into a low-dimensional representation, and calculate the probability distribution of the speech authenticity to obtain a classification result; output the classification result.

[0157] The classification model is obtained by enhancing speech information and extracting features from a self-supervised large model, and then using the extracted features as a sample set to train a ConvNeXt v2 network.

[0158] In one embodiment, when the processor executes the computer program to implement the step of inputting the speech information to be detected as false into the self-supervised large model for feature extraction to obtain an extraction result, the processor specifically implements the following steps:

[0159] The multi-layer convolutional network of the self-supervised large model is used to convert the false speech information to be identified into a potential speech representation; the potential speech representation is partially time-step masked, and a contextual representation is generated using the context network of the Transformer architecture; and the true quantized potential speech representation is identified based on the contextual representation to obtain an extraction result.

[0160] In one embodiment, when the processor executes the computer program to implement the steps of inputting the extraction result into the classification model for deep encoding to obtain an encoding result, converting the encoding result into a low-dimensional representation, and calculating the probability distribution of the true or false speech to obtain a classification result, the processor specifically implements the following steps:

[0161] The extraction result is normalized to obtain a normalized result; the normalized result is input into a classification model for deep encoding to obtain an encoding result; a high-dimensional feature representation is extracted from the encoding result; the high-dimensional feature representation is converted into a low-dimensional feature representation through a fully connected layer; and the probability distribution of the authenticity of the speech is calculated based on the low-dimensional feature representation to obtain a classification result.

[0162] The classification model includes a multi-layer structure, each layer of the structure includes multiple convolution blocks, and each of the convolution blocks includes a depth-separable convolution and an inverted bottleneck structure.

[0163] The classification model optimizes model parameters by combining the cross entropy loss function and contrastive learning method.

[0164] In one embodiment, when the processor executes the computer program to implement the step of calculating the probability distribution of speech authenticity based on the low-dimensional feature representation to obtain a classification result, the processor specifically implements the following steps:

[0165] The softmax function is used to calculate the probability distribution of the true and false speech according to the low-dimensional feature representation to obtain a classification result.

[0166] The storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0167] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0168] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the various units is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0169] The steps in the methods of the embodiments of the present invention may be adjusted in order, combined, or deleted as needed. The units in the devices of the embodiments of the present invention may be combined, divided, or deleted as needed. Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0170] If this integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present invention.

[0171] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A voice authentication method, characterized in that: include: Obtain the voice information to be authenticated; Inputting the speech information to be detected as false into the self-supervised large model for feature extraction to obtain an extraction result; Inputting the extraction result into a classification model for deep encoding to obtain an encoding result, converting the encoding result into a low-dimensional representation, and calculating the probability distribution of the true and false speech to obtain a classification result; The classification result is output.

2. The voice authentication method according to claim 1, wherein: The classification model is obtained by training a ConvNeXt v2 network using the sample set after enhancing the speech information and extracting features from the self-supervised large model.

3. The voice authentication method according to claim 1, wherein: The step of inputting the to-be-detected speech information into the self-supervised large model for feature extraction to obtain an extraction result includes: Using a multi-layer convolutional network of a self-supervised large model to convert the speech information to be detected into a potential speech representation; Masking the latent speech representation in partial time steps and generating a contextual representation using a contextual network of a Transformer architecture; A true quantized latent speech representation is identified according to the context representation to obtain an extraction result.

4. The voice authentication method according to claim 1, wherein: The extraction result is input into the classification model for deep encoding to obtain an encoding result, and the encoding result is converted into a low-dimensional representation, and the probability distribution of the true and false speech is calculated to obtain a classification result, including: Normalizing the extraction results to obtain normalized results; Inputting the normalized result into a classification model for deep encoding to obtain an encoding result; Extract high-dimensional feature representation from the encoding results; Converting the high-dimensional feature representation into a low-dimensional feature representation through a fully connected layer; The probability distribution of the authenticity of the speech is calculated based on the low-dimensional feature representation to obtain a classification result.

5. The voice authentication method according to claim 4, wherein: The classification model includes a multi-layer structure, each layer of the structure includes multiple convolution blocks, and each of the convolution blocks includes a depth-separable convolution and an inverted bottleneck structure.

6. The voice authentication method according to claim 4, characterized in that: Calculating the probability distribution of speech authenticity based on the low-dimensional feature representation to obtain a classification result includes: The softmax function is used to calculate the probability distribution of the true and false speech according to the low-dimensional feature representation to obtain the classification result.

7. The voice authentication method according to claim 2, wherein: The classification model optimizes model parameters by combining the cross entropy loss function and the contrastive learning method.

8. A voice authentication device, characterized in that: include: An information acquisition unit, configured to acquire voice information to be authenticated; A feature extraction unit is used to input the speech information to be detected as false into the self-supervised large model to perform feature extraction to obtain an extraction result; A classification unit, configured to input the extraction result into a classification model for deep encoding to obtain an encoding result, convert the encoding result into a low-dimensional representation, and calculate the probability distribution of the speech authenticity to obtain a classification result; An output unit is used to output the classification result.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images