Voice anti-counterfeiting verification method, device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a voice anti-counterfeiting verification method, which comprises the following steps: generating a unique private key of a user, obtaining a registered voice sample, generating a noisy private key after adding noise, generating a signature audio in combination with the registered voice sample, and encrypting and storing the signature audio. When a user initiates a service request, the system collects a request voice sample and extracts voice features, the voice features are matched with the signature audio by using the pre-trained verification model, and the matching degree is calculated to determine whether the service request passes verification. According to the method, the private key information is protected through differential privacy, the risk of private key leakage is avoided, the voice features are accurately matched in combination with the deep learning model, and the accuracy and safety of voice anti-counterfeiting verification are effectively improved. Meanwhile, through matching calculation of signature audio and request voice features, voice forgery attacks can be efficiently dealt with, the method is suitable for high-security scenes such as finance, and potential risks caused by voice forgery are reduced.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a voice anti-counterfeiting verification method, device, equipment and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, voice deep fake technology has made significant progress. In particular, driven by deep learning algorithms, the ability to synthesize or generate voice has been greatly improved. Forged voices have made continuous breakthroughs in naturalness, realism, and diversity, which has made it possible to apply voice deep fake technology. However, it is precisely because of these technological advances that forged voices have brought unprecedented security challenges in the financial field.

[0003] Voice deepfake technology may be used by criminals in the financial sector to impersonate financial executives, securities analysts or traders, publish misleading information, and thus influence investors' decisions. Fake voices can manipulate markets and stock prices by forging important announcements, false trading orders or deceiving investors, causing serious economic losses. This risk has become more serious with the advancement of technology, especially on social media and online financial platforms, where the use of fake voices is not only difficult to identify by hearing, but can also bypass existing multi-layer identity verification mechanisms.

[0004] Existing anti-counterfeiting measures, such as traditional watermark embedding technology, can embed logos in audio to verify the authenticity of the audio, but their limitations are also very obvious. The concealment of watermarks is often insufficient and can be easily detected or removed. In actual financial applications, the lag of watermark technology also makes it impossible to provide real-time and efficient anti-counterfeiting functions, especially in high-frequency trading scenarios and real-time financial data transmission. As the complexity of forged voice increases, watermarks or single authentication methods cannot effectively identify forged audio, resulting in limited applicability in the financial field.

[0005] In addition, traditional biometric technology also shows obvious shortcomings when dealing with deep voice fakes. Deep voice fake technology, especially speech synthesis (SS) and voice conversion (VC), can greatly improve the realism of fake voices by maintaining the original semantic content and changing the voice characteristics of the speaker, which brings great challenges to voice-based identity authentication. Existing voice detection and authentication technologies usually rely on predefined biometric features for matching, but due to the increased similarity of fake voices in multi-dimensional features, traditional detection technologies are difficult to effectively prevent the potential risks brought by fake voices.

[0006] Therefore, in the face of increasingly realistic voice deepfakes and increasingly complex financial frauds, existing technologies still have significant deficiencies in accuracy and real-time performance, making it difficult to provide effective protection. How to improve the reliability and efficiency of voice anti-counterfeiting verification and effectively resist the threat of fake voices to financial transactions and market security has become an urgent problem to be solved. Summary of the invention

[0007] The main purpose of the present invention is to provide a voice anti-counterfeiting verification method, device, equipment and storage medium, aiming to solve the technical problem that the existing voice anti-counterfeiting technology lacks the ability to protect privacy and efficiently and accurately detect forged audio when facing forged voice.

[0008] To achieve the above object, the present invention provides a voice anti-counterfeiting verification method, comprising:

[0009] Generate a unique private key for the user and obtain the user's registered voice sample;

[0010] Adding noise to the private key to generate a noisy private key for differential privacy protection;

[0011] Using a signature module based on the U-Net architecture, the noisy private key is combined with the registered voice sample to generate a signed audio;

[0012] Encrypting the signed audio and storing it in a database;

[0013] When a user initiates a service request through voice interaction, collect the user's request voice sample;

[0014] Extracting the user's voice features from the request voice sample;

[0015] Using a pre-trained verification model, the voice feature is matched with the signature audio to determine the degree of match between the voice feature and the signature audio;

[0016] If the matching degree between the voice feature and the signature audio is higher than the matching degree threshold, the service request is confirmed to be verified and authorized.

[0017] Furthermore, in order to achieve the above-mentioned object, the present invention provides a voice anti-counterfeiting verification device, comprising:

[0018] The private key generation and voice collection module is used to generate a unique private key for the user and obtain the user's registered voice sample;

[0019] A differential privacy processing module, used to add noise to the private key to generate a noisy private key for differential privacy protection;

[0020] A signature generation module, used to combine the noisy private key with the registered voice sample to generate a signed audio using a signature module based on a U-Net architecture;

[0021] An encryption module, used to encrypt the signed audio and store it in a database;

[0022] The voice interaction module is used to collect the user's request voice sample when the user initiates a service request through voice interaction;

[0023] A feature extraction module, used to extract the user's voice features from the request voice sample;

[0024] A verification model module, used to match the voice feature with the signature audio using a pre-trained verification model, and determine the degree of match between the voice feature and the signature audio;

[0025] The matching degree analysis and authorization module is used to confirm that the service request has passed the verification and authorize the service request if the matching degree between the voice feature and the signature audio is higher than the matching degree threshold.

[0026] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a voice anti-counterfeiting verification program stored in the memory and executable on the processor, and the voice anti-counterfeiting verification program, when executed by the processor, implements the steps of the voice anti-counterfeiting verification method as described above.

[0027] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a voice anti-counterfeiting verification program is stored, and when the voice anti-counterfeiting verification program is executed by a processor, the steps of the voice anti-counterfeiting verification method described above are implemented.

[0028] Beneficial effects: The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a voice anti-counterfeiting verification method, which generates a unique private key for the user and obtains a registered voice sample, generates a noisy private key after adding noise, generates a signature audio in combination with the registered voice sample, and encrypts and stores it. When a user initiates a service request, the system collects the request voice sample and extracts voice features, uses a pre-trained verification model to match the voice features with the signature audio, and calculates the degree of match to determine whether the service request has passed the verification. The present invention protects private key information through differential privacy, avoids the risk of private key leakage, and combines a deep learning model to accurately match voice features, effectively improving the accuracy and security of voice anti-counterfeiting verification. At the same time, through the matching calculation of the signature audio and the request voice features, it can efficiently deal with voice forgery attacks, is suitable for high-security scenarios such as finance, and reduces the potential risks brought by forged voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0030] Figure 1 A schematic diagram of an application environment of a voice anti-counterfeiting verification method in an embodiment of the present invention;

[0031] Figure 2 A flow chart of an embodiment of a voice anti-counterfeiting verification method of the present invention;

[0032] Figure 3 A schematic diagram of functional modules of a preferred embodiment of the voice anti-counterfeiting verification device of the present invention;

[0033] Figure 4 A schematic diagram of the structure of a computer device in one embodiment of the present invention;

[0034] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0035] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0036] The voice anti-counterfeiting verification method provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment, the user terminal communicates with the server terminal through the network. The server terminal can generate a unique private key of the user through the user terminal and obtain a registered voice sample, generate a noisy private key after adding noise, generate a signature audio in combination with the registered voice sample, and encrypt and store it. When the user initiates a service request, the system collects the request voice sample and extracts the voice features, uses the pre-trained verification model to match the voice features with the signature audio, and calculates the matching degree to determine whether the service request is verified. The present invention protects the private key information through differential privacy, avoids the risk of private key leakage, and combines the deep learning model to accurately match the voice features, effectively improving the accuracy and security of voice anti-counterfeiting verification. At the same time, through the matching calculation of the signature audio and the request voice features, it can efficiently deal with voice forgery attacks, is suitable for high-security scenarios such as finance, and reduces the potential risks brought by forged voice. Among them, the user terminal can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0037] See also Figure 2 , Figure 2This is a flow chart of an embodiment of a voice anti-counterfeiting verification method provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0038] like Figure 2 As shown, the voice anti-counterfeiting verification method proposed by the present invention comprises the following steps:

[0039] S10, generating a unique private key for the user and obtaining the user's registered voice sample;

[0040] In this embodiment, a unique private key is generated for each user, ensuring that the authentication process for each user is unique. Private key generation typically uses modern encryption algorithms such as RSA or elliptic curve cryptography (ECC). A private key is essentially a randomly generated, unique long sequence of bits used for encryption and signing operations. The uniqueness of the private key is achieved through encryption algorithms and random number generation mechanisms.

[0041] Generate private keys through encryption algorithms, such as RSA or ECC algorithms. The key generator sets relevant parameters (such as key length) to ensure the randomness and security of each private key. The generated private keys need to be securely stored in the system. Common storage methods include hardware security modules (HSM) or encrypted databases. Key management tools (such as KMS) can be used to securely access and manage private keys.

[0042] When a user registers, a voice sample of the user needs to be collected through a microphone or other audio input device. The collected voice sample is used for subsequent voice verification and anti-counterfeiting operations. The quality and length of the voice sample should meet the requirements of the system for subsequent processing. Usually, the user reads a preset phrase to maintain consistency and improve verification results.

[0043] Use an audio recording device (such as a microphone) to collect user voice samples in real time. The system will prompt the user to pronounce and record the sound, ensuring that the voice sample is clear and without excessive background noise. The voice sample needs to be stored in a high-quality format (such as WAV or other lossless audio formats) to ensure that the voice features can be accurately extracted and modeled later.

[0044] Example description: During a financial transaction, a user initiates a high-value transaction request through voice. To ensure transaction security, the system first generates a unique private key for the user to ensure the uniqueness and non-tamperability of their identity verification. During the registration phase, the system collects the user's voice sample for subsequent identity verification. This technology effectively reduces the risk of financial fraud that may be caused by forged voice, ensures that the initiator of each transaction is a legitimate user, and that the privacy and security of the transaction process are guaranteed.

[0045] By generating a unique private key for each user, the authentication process of each user is ensured to be unique and highly secure, effectively preventing identity impersonation and forgery. The generation of private keys is combined with differential privacy protection technology to further enhance data security. By obtaining registered voice samples, the system can perform anti-counterfeiting verification based on the user's biometrics, thereby improving the security and accuracy of voice interaction, which is particularly suitable for secure transactions and identity authentication in financial scenarios.

[0046] S20, adding noise to the private key to generate a noisy private key for differential privacy protection;

[0047] In this embodiment, noise is added to the generated private key in order to protect the user's private key information using differential privacy technology. The principle of differential privacy is to prevent malicious actors from inferring the value of the original private key by analyzing the output results by adding noise to the input data (private key). Common noise addition methods include noise generation based on Laplace distribution or Gaussian distribution. The intensity of the noise is determined by the global sensitivity and the privacy budget (epsilon parameter).

[0048] The generated private key is essentially a numerical vector, which is processed by differential privacy and adds noise in each dimension to form a noisy private key. In this way, even if a malicious actor obtains the noisy private key, it is difficult to accurately restore the original private key.

[0049] According to the differential privacy requirements, the system first selects the type of noise distribution. Generally, Laplace distribution or Gaussian distribution is selected. The choice of these distributions depends on the privacy protection requirements of the system.

[0050] Global sensitivity indicates the maximum impact of a single dimension change in the input data on the output result. The system needs to analyze the impact that each dimension change in the private key may have on the output (such as signed audio) based on the nature of the input private key. The privacy budget (epsilon) controls the intensity of the noise. The smaller the epsilon value, the greater the noise intensity and the stronger the privacy protection.

[0051] After the noise is generated, the system injects it into each dimension of the private key to form a noisy private key. This noisy private key is used in the subsequent steps of generating the signed audio to ensure that the private key is not leaked during the entire process.

[0052] Example: In a financial transaction scenario, the user needs to complete the transaction authorization through the voice interaction system. The system first generates a unique private key for the user and adds noise to protect the privacy of the private key. Even if an attacker obtains the noisy private key, it is impossible to reversely infer the user's original private key by analyzing the private key. The noise protection of differential privacy ensures that each financial transaction has a high degree of privacy security during the execution process, especially in high-risk financial activities, effectively preventing transaction fraud or forgery that may be caused by private key leakage.

[0053] By adding noise to the private key, private key protection based on differential privacy is achieved, effectively preventing malicious attackers from deducing the private key by analyzing the output results. The privacy protection level of the system is improved to ensure that the user's private key information will not be leaked or speculated during the entire authentication and signing process. The application of differential privacy ensures that the user's private key is always securely protected in high-security fields such as financial transactions, reduces the risk of reverse speculation of the private key, and ensures the anti-counterfeiting accuracy of the system.

[0054] S30, using a signature module based on a U-Net architecture, combining the noisy private key with the registered voice sample to generate a signed audio;

[0055] In this embodiment, U-Net is a deep learning network architecture commonly used for feature extraction and fusion. It was originally applied to image processing and is applied to speech processing in this embodiment. Its advantage is that it can retain both global and local features and is suitable for processing multimodal input data. The U-Net architecture is used to combine the features of the noisy private key and the registered speech sample to generate a unique signature audio.

[0056] The noisy private key is preprocessed and converted into a numerical feature representation suitable for processing by the U-Net architecture. This feature representation is usually input into the encoder part of the U-Net architecture in the form of a vector or matrix, and the encoder is responsible for multi-level extraction and compression of the private key features.

[0057] The registered speech samples are converted into frequency domain features or spectrograms through speech feature extraction tools such as MFCC or Fourier transform. These feature representations can better capture the frequency information of the speech, making it easier to fuse with the private key features in the U-Net architecture. The converted speech feature representations are also passed as input to the encoder part of the U-Net architecture.

[0058] The characteristic of the U-Net architecture is that it can fuse data features from different inputs at multiple levels. In the encoding stage, the features of the noisy private key and the registered voice sample are extracted separately. The jump connection mechanism of U-Net ensures that the features are combined and integrated at different levels. The fused feature map contains the unique information of the private key and the biometric information of the voice sample.

[0059] After feature fusion, the decoder part of the U-Net architecture performs reverse operations on the fused features to restore the generated audio signal layer by layer. The final generated signature audio contains the user's voice features and the unique signature features of the private key. This signature audio can not only ensure the security of voice anti-counterfeiting, but also can be used for subsequent identity verification.

[0060] The noisy private key is encoded as a feature vector as input. The registered speech sample is processed using Fourier transform or Mel-Frequency Cepstral Coefficients (MFCC) to convert the time domain signal into frequency domain features or generate a spectrogram. The encoder processes the private key features and speech sample features separately, reduces the dimensionality of high-dimensional features and extracts core features. The U-Net skip connection mechanism is used to integrate features at different levels to ensure that the signature information of the private key is fully integrated with the biometric information of the speech sample. The decoder gradually restores the fused features to generate the signature audio.

[0061] The decoder of the U-Net architecture decodes the feature map into an audio signal and generates audio data with signature features. The generated signature audio contains the user's voice features and the unique signature information of the private key.

[0062] Example description: In a financial scenario, users complete identity authentication and transaction authorization through a voice interaction system. The system uses the signature audio generated by the U-Net architecture to ensure the authenticity of the voice. The user's private key information is protected by differential privacy technology, and the fusion of their voice sample and private key ensures the accuracy and security of anti-counterfeiting. Even if an attacker attempts to forge a voice, the private key features in the generated signature audio make it difficult for them to successfully pass the verification, ensuring the security of financial transactions.

[0063] By using a signature module based on the U-Net architecture, the system can efficiently combine the user's noisy private key with the features of the registered voice sample to generate a unique signature audio. The signature audio contains both the user's biometric features and the signature information of the private key, which greatly improves the accuracy and security of voice anti-counterfeiting. Combined with the multi-level feature extraction and fusion capabilities of the U-Net architecture, the system can prevent the leakage of private key information while ensuring the authenticity of the voice.

[0064] S40, encrypting the signed audio and storing it in a database;

[0065] In this embodiment, after the signature audio is generated, it must be encrypted to ensure that it will not be tampered with or accessed without authorization during transmission and storage. Common encryption methods include symmetric encryption and asymmetric encryption. Symmetric encryption methods (such as AES, Advanced Encryption Standard) use a single key to encrypt and decrypt data, which is suitable for efficient encryption during data transmission. Asymmetric encryption methods (such as RSA, Rivest-Shamir-Adleman) use public key encryption and private key decryption, which are suitable for storage scenarios that require high security.

[0066] The encrypted signed audio needs to be securely stored in the database. The database usually needs to support encrypted storage to ensure that the data cannot be accessed or tampered with externally during the storage process. The database can be a traditional relational database (such as MySQL, PostgreSQL) or a cloud storage service (such as Amazon S3, Google Cloud Storage). During the storage process, you can also use the database's encryption function (such as full disk encryption or field-level encryption) to further improve security.

[0067] For example, a symmetric encryption algorithm such as AES (Advanced Encryption Standard) can be used to encrypt the generated signature audio. Symmetric encryption has a high encryption speed and is suitable for scenarios with high real-time requirements. The system generates a random encryption key for encrypting and decrypting the signature audio. The encryption key is used to encrypt the signature audio to generate ciphertext.

[0068] If the scenario requires higher security, an asymmetric encryption algorithm such as RSA can be used. In this case, the system uses the public key to encrypt the signed audio, and only the corresponding private key can decrypt it. The signed audio is encrypted using the public key generated by the system. It can be decrypted using the private key to ensure the integrity and security of the data.

[0069] Store the encrypted signed audio in a secure database. The database should support encrypted storage to ensure that the encrypted data cannot be accessed by unauthorized persons. During the storage process, you can use the encryption function provided by the database, such as Transparent Data Encryption (TDE) supported by MySQL or encryption options provided by cloud services (such as AWS KMS, GCP's Cloud KMS). The database should also support access control and permission management to ensure that only authorized systems or users can access the stored signed audio.

[0070] Example description: In a financial transaction scenario, a high-value transaction initiated by a user will generate an audio with a private key signature. The signed audio is processed by AES symmetric encryption and stored in a secure cloud database. The database of the financial system has strict access rights, and only authenticated administrators or system services can access the encrypted audio data. Even if a malicious attacker attempts to steal the audio data, the attacker cannot decrypt or tamper with the signed audio because it is encrypted, ensuring data security and user privacy protection during financial transactions.

[0071] By encrypting the generated signature audio, the system effectively ensures the security of audio data during storage and transmission, preventing unauthorized access or tampering. Using symmetric encryption can improve the speed and efficiency of encryption processing, while asymmetric encryption provides higher security and is suitable for sensitive scenarios. The encrypted signature audio is stored in a secure database and equipped with access control measures, further improving the security and data integrity of the system.

[0072] S50, when the user initiates a service request through voice interaction, collecting the user's request voice sample;

[0073] In this embodiment, when a user initiates a request through a voice interaction system (such as a phone or voice recognition application), the system needs to collect a voice sample of the user in real time. The voice sample is used for subsequent identity verification to ensure that the user's request is legitimate. This process requires a microphone or other audio collection device, and the user enters commands through voice commands, such as requesting transaction authorization or access to sensitive information.

[0074] The system captures the user's voice input through a microphone or other audio input device (such as a built-in microphone on a mobile phone or computer). The collected voice samples are usually short-term voice streams, and the system needs to pre-process them, such as removing background noise and echo interference to ensure the clarity and accuracy of the voice samples. The collected voice samples will be used for feature extraction and subsequent verification processes.

[0075] When a user initiates a service request through the voice interaction system, the system will start the voice collection function. The user can trigger the operation by saying a preset command, such as "please authorize the transaction" or "check the account balance". The system monitors the user's voice input and captures the user's complete voice command.

[0076] Use microphone devices to collect user voice samples. The system ensures that the collected voice samples can accurately reflect the user's identity characteristics and have no obvious noise interference. During collection, voice processing technology can be used for real-time processing to remove background noise and echo to ensure sample quality.

[0077] The collected voice samples are stored in a specific format (such as WAV or other lossless formats) for subsequent voice feature extraction and verification. The storage of voice data can be managed through local cache or cloud storage to ensure that the voice data can be safely processed or deleted after verification is completed.

[0078] Example description: In a financial transaction scenario, when a user initiates a high-value transaction request through telephone banking or a voice recognition system, the system will start the voice collection function to capture the transaction command issued by the user. To ensure the security of the transaction, the system will process the audio signal in real time when collecting voice samples to remove noise and echo. The collected voice samples are encrypted and stored in the cloud server, and then used to verify the user's identity. Through this process, the system can accurately recognize the user's voice, prevent forged voice or identity impersonation, and ensure the security and legality of each financial transaction.

[0079] By collecting user request voice samples in real time, the system can authenticate users based on their biometrics to ensure that the service request is made by a legitimate user. This can effectively avoid malicious requests initiated by forged voices or third parties impersonating users. The collected voice samples are of higher quality after preprocessing, which facilitates subsequent feature extraction and verification, further improving the security and recognition accuracy of the system.

[0080] S60, extracting the user's voice features from the request voice sample;

[0081] In this embodiment, extracting voice features from the collected request voice samples is a crucial step in voice anti-counterfeiting verification. The extraction of voice features usually converts the audio signal from the time domain to the frequency domain, and extracts key features that can reflect the speaker's identity and biometrics. Commonly used voice feature extraction methods include Mel Frequency Cepstral Coefficients (MFCC), Linear Prediction Cepstral Coefficients (LPCC), Short Time Fourier Transform (STFT), etc.

[0082] Speech signals are usually collected in the time domain. Speech feature extraction requires converting the signal into a frequency domain signal because frequency domain features (such as pitch, frequency energy distribution, etc.) can more effectively describe the uniqueness of speech. The most commonly used method is short-time Fourier transform (STFT), which divides the speech into short-time windows, performs Fourier transform on each window, and extracts its frequency features.

[0083] Common speech feature extraction methods include MFCC, which simulates the human auditory system to extract key speech features that can distinguish different people. MFCC is widely used in speech recognition and identity authentication. The extracted features are usually a matrix or vector, which represents the energy distribution of the speech sample at a specific frequency.

[0084] Before extracting speech features, the collected speech samples need to be preprocessed, including denoising, normalization, and framing operations.

[0085] Denoising: Use noise filtering algorithms to remove background noise and improve the clarity of speech samples.

[0086] Normalization: Normalize the amplitude of the speech signal to make the signal amplitude consistent, which is convenient for subsequent feature extraction.

[0087] Framing: The speech signal is divided into multiple short time frames, each frame usually contains a duration of about 20ms to 30ms, so as to extract the features of each frame.

[0088] The short-time Fourier transform (STFT) is used to convert the time domain signal of each frame into a frequency domain signal to obtain the frequency component of the speech frame. The spectrum information of each frame can be used for subsequent feature extraction.

[0089] Feature extraction can be done in the following ways:

[0090] Mel-Frequency Cepstral Coefficient (MFCC): This is one of the most commonly used speech feature extraction methods. MFCC extracts a set of coefficients representing speech features by simulating the sensitivity of the human ear to different frequencies. Usually 12 to 13 MFCC coefficients are extracted and combined with energy information to form a feature vector representing the entire speech sample.

[0091] Linear Prediction Cepstral Coefficients (LPCC): Another commonly used feature extraction method, which predicts the value of the future speech signal and calculates the linear prediction error to obtain a set of coefficients that describe the characteristics of the speech.

[0092] The extracted features can be represented as feature vectors or matrices, which contain the most critical information in the user's voice sample and can be used for subsequent identity verification. The extracted features need to be stored in temporary memory or encrypted in a secure database for comparison with previously registered voice features.

[0093] Example description: In a financial service scenario, when a user initiates a transaction request through a voice interaction system, the system collects the user's voice sample and extracts features in real time. By using MFCC to extract the user's voice features, the system can effectively capture the user's unique voice features. The extracted features are then matched with the user's registered voice features to confirm the legitimacy of the request. In this way, the system can ensure the security of each transaction, prevent identity impersonation or voice forgery, and ensure the security of financial transactions.

[0094] By extracting voice features from the request voice sample, the system can accurately obtain the user's biometric information, which plays a vital role in the identity verification process. By using efficient feature extraction methods (such as MFCC or LPCC), the system can maintain high recognition accuracy under a variety of noise and interference conditions, improving the reliability and security of the system.

[0095] S70, using a pre-trained verification model, matching the voice feature with the signature audio to determine a matching degree between the voice feature and the signature audio;

[0096] In this embodiment, the verification model is a model trained based on machine learning or deep learning, which is used to compare the user's voice features with the stored signature audio features. Pre-training means that the model has been trained on a large amount of voice data and signature audio, and has strong generalization ability and high matching accuracy. The pre-trained model can quickly identify the difference between the input voice features and the signature audio, ensuring the real-time and accuracy of the verification process.

[0097] During the voice interaction process, the system extracts voice features from the user's request voice sample and matches it with the stored signature audio features. The matching process is completed by the verification model, which calculates the similarity (i.e., matching degree) between the two to determine whether the user's request voice is consistent with the stored signature audio. The matching degree is usually calculated based on distance measurement or similarity calculation, such as cosine similarity, Euclidean distance, or similarity measurement in vector space.

[0098] The matching degree is a quantitative result obtained by calculating the similarity between the user's voice features and the stored signature audio features. The matching degree value is usually a floating point number between 0 and 1, and the higher the value, the more similar the two features are. Based on the preset matching degree threshold, the system determines whether the user's voice matches the voice at the time of registration, thereby confirming the user's identity.

[0099] The system first extracts the features of the user's requested voice sample (such as MFCC, LPCC, etc.), which represent the user's current voice. The signature audio is generated when the user registers and contains the user's voice features and private key information to ensure uniqueness. Use a pre-trained deep learning verification model (such as CNN, RNN or DNN), which has been trained on a large number of voice data sets and can handle the matching task between voice features and signature audio features. The verification model compares the input voice features with the stored signature audio features to analyze the similarity between the two.

[0100] The system calculates the matching degree between the user's voice features and the signature audio features through a preset distance measurement method. Common matching degree calculation methods are:

[0101] Cosine similarity: Calculate the cosine angle of two feature vectors. The closer the value is to 1, the more similar they are.

[0102] Euclidean distance: Calculates the Euclidean distance between two feature vectors. The smaller the value, the closer they are.

[0103] The generated matching degree is used to determine whether the user's voice features match the signature audio.

[0104] The system sets a matching degree threshold (such as 0.85). If the matching degree is higher than the threshold, the system determines that the user's identity authentication has passed; if the matching degree is lower than the threshold, the identity authentication has failed, and the user is prompted to re-authenticate or the request is rejected.

[0105] Example description: In a financial transaction scenario, a user initiates a high-value transfer request through a phone or voice interaction system. The system first collects the user's request voice sample and extracts voice features. Then, the pre-trained verification model is used to match the user's voice features with the stored signature audio. The system confirms whether the user's voice is consistent with the voice at the time of registration by calculating the matching degree between the two. When the matching degree is higher than the preset threshold (such as 0.85), the system passes the identity verification and approves the transaction. If the matching degree is lower than the threshold, the system rejects the transaction request and prompts the user to re-verify. This method effectively prevents forged voice or identity impersonation, and ensures the security and legality of financial transactions.

[0106] By using the pre-trained verification model, the system can quickly and accurately compare the user's request voice features with the stored signature audio features to verify the user's identity. The calculation of the matching degree ensures the reliability of identity authentication and can effectively prevent forged voice or other attacks. The setting of the matching degree threshold ensures the anti-counterfeiting security of the system while maintaining a high verification accuracy rate, which is suitable for scenarios with high security requirements such as finance and identity authentication. The pre-trained model greatly improves the real-time and scalability of the verification process, enabling the system to efficiently handle a large number of user requests.

[0107] S80: If the matching degree between the voice feature and the signature audio is higher than the matching degree threshold, confirm that the service request has passed the verification and authorize the service request.

[0108] In this embodiment, the system generates a matching value after matching the voice features and the signature audio features. The matching value usually ranges from 0 to 1, indicating the similarity between the two. The system sets a matching threshold, which is used to determine whether the service request is authenticated. The common matching threshold can be set to 0.85 or higher depending on the security requirements. If the matching value is greater than or equal to the threshold, it means that the similarity between the two is high enough, and the system considers that the user identity authentication is passed.

[0109] When the matching degree is higher than the matching degree threshold, the system confirms that the user's identity authentication is successful. A successful verification result means that the user's current voice characteristics match the signature audio during registration, proving that the user is a legitimate requester.

[0110] After the user's identity is authenticated, the system will authorize the service request initiated by the user. For example, if the user requests to conduct a financial transaction or access sensitive data, the system will allow the request to proceed and record the operation. Authorization means that the system will execute the user's operation request and the request will be marked as initiated by a legitimate user.

[0111] The system calculates the matching degree between the user's voice features and the signature audio through the verification model, and compares the matching degree with the preset threshold. If the matching degree is higher than the threshold (for example, the threshold is 0.85), the system confirms that the identity authentication is successful.

[0112] When the match degree is higher than the threshold, the system will mark the verification process as successful. This status will be recorded in the system memory for subsequent tracking and analysis.

[0113] After the identity verification is passed, the system will authorize the user's request. For different types of service requests, the system will perform corresponding operations:

[0114] Financial transaction scenario: If the user initiates a transaction request, the system will allow the transaction to proceed.

[0115] Data access scenario: If a user requests access to sensitive data, the system will allow the user to access the required data.

[0116] The system will also log these operations to ensure that each authorization operation has an audit record for future traceability. The system will record the entire authorization process, including the matching calculation, the time and result of the verification, and the details of the authorization operation. This data can be used for subsequent auditing, monitoring and analysis.

[0117] Example: In the financial field, a user initiates a high-value transfer request through a voice recognition system. The system extracts the user's voice features and matches them with the stored signature audio to generate a match. When the match is higher than the preset threshold (such as 0.9), the system confirms that the user's identity verification is successful and authorizes the user to transfer the money. After that, the transaction is successfully executed, and the system records the verification and authorization process in the log for audit purposes. This mechanism effectively prevents financial fraud through forged voice and ensures the legitimacy and security of each transaction.

[0118] By setting the matching threshold, the system can effectively identify legitimate users and authorize their operations based on the similarity between voice features and signature audio. This ensures the efficiency and accuracy of identity authentication, while being able to flexibly adjust the matching threshold according to business needs. This mechanism has a significant anti-counterfeiting effect in highly secure scenarios (such as financial transactions, sensitive data access, etc.), greatly reducing the risk of identity impersonation and voice forgery. Authorization operations after verification can be automatically executed, greatly improving the system's response speed and user experience.

[0119] The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a voice anti-counterfeiting verification method, which generates a unique private key for a user and obtains a registered voice sample, generates a noisy private key after adding noise, generates a signature audio in combination with the registered voice sample, and encrypts and stores it. When a user initiates a service request, the system collects the request voice sample and extracts voice features, uses a pre-trained verification model to match the voice features with the signature audio, and calculates the matching degree to determine whether the service request is verified. The present invention protects private key information through differential privacy, avoids the risk of private key leakage, and combines a deep learning model to accurately match voice features, effectively improving the accuracy and security of voice anti-counterfeiting verification. At the same time, through the matching calculation of the signature audio and the request voice features, it can efficiently deal with voice forgery attacks, is suitable for high-security scenarios such as finance, and reduces the potential risks brought by forged voice.

[0120] In one embodiment, the above S20 includes:

[0121] S201, determining a distribution type of noise, where the distribution type includes Laplace distribution or Gaussian distribution;

[0122] S202, performing numerical encoding and vectorization processing on the private key to generate a private key vector, analyzing the maximum impact value of a single dimension change in the private key vector on the signature audio feature representation to be generated, and using the maximum impact value as the global sensitivity;

[0123] S203, determining the intensity of the noise based on the global sensitivity and a preset privacy budget;

[0124] S204, generating complete noise based on the distribution type of the noise and the intensity of the noise;

[0125] S205: Inject the complete noise into the private key to generate a noisy private key.

[0126] In this embodiment, differential privacy protection masks the real data by adding noise to the private key. Common noise distribution types are Laplace distribution and Gaussian distribution. Laplace distribution is often used in scenarios that require simpler noise generation, while Gaussian distribution is used in more complex situations and is suitable for protection methods that require smoother noise. By selecting different distribution types, the strength of privacy protection and the accuracy of the model can be adjusted.

[0127] The private key can be represented as a multi-dimensional vector, where each dimension represents a part of the private key. Assume that the private key is a vector containing multiple values, each of which corresponds to an independent part of the private key. These values ​​are within a specific range and can be used to generate specific signature audio features.

[0128] For each dimension in the private key vector, when the dimension changes, it will affect the final generated signature audio features. Analyzing these changes can help us understand the sensitivity of the private key and ensure that a small change in a certain dimension does not lead to a large change in the features. Through this analysis process, the system can evaluate the impact of private key changes in different dimensions on the final audio features.

[0129] Each dimension in the private key vector affects the generated signature audio features differently. Changes in some dimensions may only produce minor changes in the generated audio features, while changes in other dimensions may result in significant feature differences. To protect privacy and avoid speculation attacks, the system needs to find the dimension that has the greatest impact on the signature audio features and use its change impact as the global sensitivity. This analysis ensures that even in the most extreme cases (maximum impact value), the private key can remain private through noise protection.

[0130] Global sensitivity refers to the maximum difference that may occur to the signature audio feature representation when a dimension in the private key vector changes. By calculating and determining the global sensitivity, the system can ensure that the noise intensity is large enough to protect the privacy of the private key when adding noise. The higher the global sensitivity, the greater the impact of private key changes in certain dimensions on the output features, so stronger noise protection is required.

[0131] The system calculates the intensity of the noise by combining global sensitivity and the privacy budget (usually expressed as epsilon). The privacy budget is a preset parameter used to control the size of the noise. The smaller the epsilon value, the greater the noise intensity and the stronger the privacy protection. After the noise intensity is calculated by the formula, it will be used to generate specific noise data. Epsilon (ε) can be understood as a unit of measurement for controlling the privacy budget, which quantifies the "cost" of privacy for each query. For example, if ε = 1, the privacy leakage allowed by the system is a certain standard value; if ε = 0.1, then the privacy leakage will be very strictly limited, which means that less budget is "spent". When multiple queries or operations are performed, each operation will consume part of the ε value, and the final accumulated ε value represents the overall privacy budget consumption.

[0132] In a differential privacy system, each data access (such as query, analysis) consumes a certain privacy budget. When the system queries the data multiple times, the privacy budget will gradually decrease with the number of queries until it reaches a certain critical point and the privacy budget is exhausted. Epsilon is a specific parameter that measures how much privacy budget is consumed by each query.

[0133] Based on the selected noise distribution type (such as Laplace distribution or Gaussian distribution) and the calculated noise intensity, the system generates complete noise. The generated noise is then injected into each dimension of the private key vector to form a noisy private key. This private key is used in the subsequent signature audio generation step to ensure that the entire signing process meets the requirements of differential privacy protection.

[0134] For example, first, the private key vector is decomposed into multiple independent dimensions. For example, the private key vector may be x1,x2,x3,...,xn, where each xi represents a dimension.

[0135] Simulate a single dimension change. During the simulation, the system changes the value of each dimension one by one. For example, x1 is slightly increased or decreased from the initial value to observe its effect on the generated signature audio feature representation. For each dimension, the amount of change in the generated signature audio feature when the dimension changes is recorded.

[0136] The system will analyze the change results of all dimensions and find the dimension that produces the largest feature change. Assuming that the change in the xk dimension has the greatest impact on the signature audio feature, this maximum change is defined as the global sensitivity. The maximum impact value is used as the global sensitivity. Global sensitivity is a key parameter in the subsequent noise generation and addition process to ensure the effectiveness of differential privacy technology.

[0137] Example description: Assume that a user uses a private key for voice signature in a financial transaction. In order to protect the user's private key, the system needs to use differential privacy technology in the signature audio generation process.

[0138] Assume that the user's private key is represented as a three-dimensional vector x1, x2, x3, where each dimension represents a part of the information in the private key. This vector will be used to generate the signature audio.

[0139] The system gradually changes the values ​​of x1, x2, and x3 and generates different signature audios. For example, the system increases x1 from 1.0 to 1.1 and observes the changes in the generated audio feature representation. Then similar operations are performed on x2 and x3 respectively.

[0140] By comparison, the system found that changes in x1 had little effect on the generated audio features, changes in x2 had a moderate effect on the audio features, and changes in x3 would cause significant changes in the audio features. After analysis, the system determined that changes in x3 had the greatest impact on the signature audio features.

[0141] The system uses the maximum feature change value generated when x3 changes as the global sensitivity. This global sensitivity is used to calculate the noise intensity required when adding noise, thereby ensuring that the private key cannot be guessed when generating the signed audio.

[0142] The system generates noise based on the global sensitivity and privacy budget, and injects noise into each dimension of the private key, especially adding stronger noise in the x3 dimension. The final signature audio contains the signature information of the private key, while preventing the leakage of the private key through noise protection.

[0143] In this embodiment, by analyzing the maximum impact value of a single dimension in the private key vector on the signature audio feature, the system can accurately determine the global sensitivity, thereby generating noise of appropriate intensity based on the sensitivity. This ensures that the system can accurately generate signature audio and perform identity authentication while protecting the privacy of the user's private key. Through effective differential privacy technology, the system can be applied in a variety of high-security scenarios, including financial transactions, identity authentication, etc., to ensure user data security.

[0144] In one embodiment, the above S30 includes:

[0145] S301, performing numerical encoding and vectorization processing on the noisy private key to generate a numerical feature vector;

[0146] S302, converting the registered speech sample into a frequency domain feature representation through short-time Fourier transform;

[0147] S303, the signature module of the U-Net architecture extracts features from the numerical feature vector and the frequency domain feature representation respectively, and fuses the extracted features to generate a fused feature map;

[0148] S304, reconstructing and generating a signature audio from the fused feature map, wherein the signature audio includes a signature feature of the private key and a biometric feature of the registered voice sample.

[0149] In this embodiment, the noisy private key first needs to be numerically encoded and vectorized to convert the private key data into a format suitable for processing by the machine learning model. The private key vector is usually a collection of numerical features representing the dimensions of the private key. The encoding process converts the private key from the original format into a numerical form and forms a feature vector according to a specific structure so that it can be input into the model for processing in the subsequent steps.

[0150] The registered speech samples need to be converted from time domain signals to frequency domain signals, which is usually done through short-time Fourier transform (STFT). STFT is a technique that divides the time domain signal into multiple short time windows and performs Fourier transform on each time window. Frequency domain feature representation can better capture the frequency information in speech, reflect the characteristics of speech, and facilitate feature extraction and fusion of subsequent models.

[0151] The core of the U-Net architecture is its encoder-decoder structure. The encoder part of the module extracts features from the numerical feature vector (from the noisy private key) and the frequency domain feature representation (from the enrollment speech sample). The encoder compresses these two inputs and extracts low-dimensional feature representations.

[0152] The characteristic of U-Net is that it combines the high-dimensional and low-dimensional features extracted during the encoding process through skip connections, ensuring that the feature information is not lost in multiple compressions.

[0153] After extracting the numerical feature vector and frequency domain feature representation, the signature module of the U-Net architecture will fuse the features of the two. The fused feature map is a multimodal information map that contains the signature information of the private key and the biometric features of the voice sample. This fusion ensures that the generated signature audio contains both the user's voice information and the private key signature features, providing a unique identity.

[0154] The decoder part of the U-Net architecture reconstructs the final signature audio from the fused feature map. The reconstruction process is to reverse the compressed features, restore them layer by layer to high-dimensional representation, and finally generate a playable audio signal. The generated signature audio contains the signature features of the private key and the user's biometric voice features, ensuring security and uniqueness.

[0155] In this embodiment, through the signature module based on the U-Net architecture, the system can effectively fuse the signature features of the private key with the user's voice biometric features to generate a unique signature audio. The feature extraction and fusion capabilities of the U-Net architecture ensure the accuracy and security of the generated audio, making the voice anti-counterfeiting process more robust. At the same time, the use of short-time Fourier transform to extract the frequency domain features of the voice can more accurately describe the user's voice characteristics, and the numerically encoded private key features ensure the uniqueness of each signature audio. Through this process, the system can effectively prevent forgery and identity impersonation.

[0156] In one embodiment, the above S70 includes:

[0157] S701, extracting features from the signature audio to generate corresponding signature audio features;

[0158] S702, inputting the speech feature and the signature audio feature into a pre-trained verification model;

[0159] S703, the verification model compares the voice features with the signature audio features layer by layer to determine the feature similarity between the voice features and the signature audio features;

[0160] S704: Analyze the matching score between the speech feature and the signature audio by using cosine similarity according to the feature similarity.

[0161] In this embodiment, the signature audio is generated by the U-Net architecture and contains the signature information of the user's private key and voice biometrics. Before matching, the system needs to extract features from the signature audio. Similar to the way feature extraction is performed on voice samples, this process usually uses methods such as Mel Frequency Cepstral Coefficients (MFCC) to convert audio signals into feature vectors. These feature vectors can represent important information in the audio, which is convenient for subsequent model comparison.

[0162] The extracted speech features and signature audio features are input into a pre-trained validation model trained with deep learning. This model has been trained on a large amount of data and has the ability to recognize speech features and audio features. The main task of the pre-trained model is to determine the similarity of the input features by comparing them.

[0163] In the verification model, the speech features and the signature audio features are compared in a multi-layer neural network. Each layer of the neural network in the model processes the input features and generates feature representations at different levels. By comparing layer by layer, the verification model can capture the differences and similarities between the input features more deeply. Different layers of the neural network extract features at different levels, such as low layers capturing detailed features and high layers capturing global features. This layer-by-layer comparison method can provide more accurate similarity analysis.

[0164] Finally, the verification model calculates the similarity between the extracted voice features and the signature audio features. The commonly used method is cosine similarity, which measures the similarity between two vectors by calculating the angle between them. The value of cosine similarity is between -1 and 1, where 1 means that the two vectors are exactly the same and 0 means that they are not similar. After the calculation is completed, the system will generate a matching score based on the cosine similarity to determine whether the voice verification has passed.

[0165] In this embodiment, through layer-by-layer comparison and cosine similarity calculation of the pre-trained verification model, the system can accurately determine the similarity between the user's voice features and the signature audio, thereby performing identity authentication. Cosine similarity provides an efficient way to measure the similarity between two feature vectors, and the matching score reflects the consistency of the user's current voice with the registered voice. The layer-by-layer comparison method can compare at different feature levels, capture subtle changes and global features, and thus improve the accuracy and security of the verification process.

[0166] In one embodiment, after the above S70, the method further includes:

[0167] S801, if the matching degree between the voice feature and the signature audio is not higher than the matching degree threshold, confirm that the service request has not passed the verification, and prompt for re-verification;

[0168] S802, obtaining a re-verification voice sample of the user, and extracting a re-verification voice feature from the re-verification voice sample;

[0169] S803, matching the re-verification voice feature with the signature audio to determine the matching degree between the re-verification voice feature and the signature audio;

[0170] S804: Determine whether the service request is verified based on whether the matching degree between the re-verification voice feature and the signature audio is higher than the matching degree threshold.

[0171] In this embodiment, after completing the initial match between the voice feature and the signature audio, the system will determine the similarity between the two by calculating the matching degree. If the matching degree is lower than the set matching degree threshold (for example, the threshold is 0.85), the system will determine that the verification has failed. At this time, the system will generate a prompt to inform the user that the current service request has failed to pass the verification and require the user to re-verify. This process ensures that the service request will only be authorized when the voice feature matches the signature audio successfully.

[0172] When the user receives the re-verification prompt, they need to re-record and submit the voice sample. The system will process the re-submitted voice sample and extract the voice features. The extraction process of the re-verification voice sample is the same as the initial extraction process. Usually, feature extraction methods such as Mel Frequency Cepstral Coefficients (MFCC) are used to obtain feature vectors that can represent the voice.

[0173] The system will match the re-verified voice features with the user's signature audio features again. This process is the same as the initial verification process. The re-verified voice features are input into the pre-trained verification model, compared with the stored signature audio features layer by layer, and the matching degree is calculated. The matching degree calculation result is used to determine whether this verification is successful.

[0174] The system determines whether the service request can be verified based on the re-verified matching result. If the matching degree of the re-verified voice features and the signature audio is higher than the set threshold, the system confirms that the service request is verified and allows the user to continue. If the re-verified matching degree is still lower than the threshold, the system may reject the service request or further prompt other identity verification methods.

[0175] By providing a re-verification mechanism, this embodiment can effectively handle verification failures caused by environmental changes, equipment problems, or user pronunciation differences. The re-verification process allows users to submit voice samples again to ensure the accuracy and flexibility of verification. In this way, the system can improve the user experience and avoid excessive rejection of legitimate requests while maintaining high security standards. In high-security application scenarios, this mechanism can effectively prevent fraudulent behavior of forged voices and protect user privacy and data security.

[0176] In one embodiment, after the above S804, the method further includes:

[0177] S805, monitoring the user's verification behavior and recording the time and result of each verification;

[0178] S806, when the user fails to verify multiple times within a preset time window, extracting failed voice features from all request voice samples that failed verification;

[0179] S807, analyzing the average feature value of all failed speech features, and comparing the average feature value with the speech features of a preset normal speech sample to determine the feature difference between the average feature value and the speech features of the normal speech sample;

[0180] S808: If the feature difference exceeds a preset difference threshold, mark the service request as a suspicious request.

[0181] In this embodiment, the system continuously monitors each verification operation of the user during the entire verification process. The monitoring content includes information such as the time, result (pass or fail) and matching score of each verification. By recording these verification behaviors, the system can audit the user's behavior and provide basic data for subsequent anomaly detection.

[0182] If a user submits multiple voice verification requests within a preset time window (e.g., within 5 minutes) and all of them fail, the system will extract voice features from each failed request voice sample. The feature extraction process at this time is similar to that of regular verification. The system extracts feature vectors of all failed samples for subsequent analysis.

[0183] After extracting the features of the failed voice samples, the system calculates the average of all failed feature vectors. The average feature value represents the trend of the user's voice features when multiple verifications fail. In order to determine whether there is an abnormality, the system compares this average feature value with the features of normal voice samples (such as voice samples during registration) and calculates the feature difference between the two.

[0184] The system has a preset difference threshold, which is used to determine whether the current user's voice features significantly deviate from normal features. If the calculated average feature value and the feature difference of the normal voice sample exceed the preset difference threshold, the system will mark the current service request as a suspicious request. Suspicious requests mean that the system suspects that the current user behavior may be risky or offensive, such as forging voice or impersonating an identity.

[0185] The system continuously monitors each user's verification operation in the background, recording the time, result (pass or fail), and matching information. This data will be used for subsequent analysis, especially for in-depth detection in the case of multiple failures. When a user fails to verify multiple times within a time window, the system extracts voice features from each failed request and saves these features. This step ensures that the system can perform further analysis on the failed samples. The system averages the voice feature vectors of multiple failed requests to generate a comprehensive average feature value. This average feature value reflects the common features of the user's multiple failed samples, which is convenient for comparison with normal voice features. The system compares the average feature value with the preset normal voice sample (usually the voice features entered when the user registered). By calculating the feature difference, the system can determine the degree of deviation of the current user behavior from normal behavior. If the feature difference exceeds the preset threshold, the system will mark the current request as a suspicious request. At this time, the system may trigger additional verification steps or reject the user's request.

[0186] By monitoring the user's verification behavior, the system in this embodiment can better prevent the security risks that may be caused by multiple verification failures. By extracting and analyzing the average features of failed voice samples, the system can effectively identify potential abnormal situations, such as voice forgery or identity impersonation. Feature difference analysis provides a mechanism for quickly identifying abnormalities, allowing the system to respond in time when user behavior deviates from normal patterns and prevent possible security threats.

[0187] In one embodiment, after the above S808, the method further includes:

[0188] S809, determining an abnormality level of the suspicious request based on a feature difference between the average feature value and a voice feature of a normal voice sample, wherein the abnormality level includes multiple levels;

[0189] S810, configuring corresponding exception handling operations for each level of exception level;

[0190] S811, if the abnormality level of the suspicious request is the lowest abnormality level, prompting to perform a verification operation other than voice verification;

[0191] S812: If the abnormality level of the suspicious request is the highest abnormality level, reject the suspicious request and notify the security processing end to process the suspicious request.

[0192] In this embodiment, after finding that the feature difference exceeds the preset threshold and is marked as a suspicious request, the system further evaluates the abnormality of the request. The abnormality level is determined based on the difference between the average feature value and the normal voice sample feature. Generally, the greater the feature difference, the higher the abnormality. The system classifies the difference into different abnormality levels, such as low level, medium level and high level, indicating the suspicious degree of the request.

[0193] The system presets corresponding processing strategies according to each abnormality level. Each level corresponds to different security response operations. For lower abnormality levels, the system may prompt the user to perform additional verification steps; for higher abnormality levels, the system will take more stringent measures, such as directly rejecting the request or notifying the security processing department to intervene. These operations ensure that the system can take appropriate measures according to different risk levels to prevent potential security threats.

[0194] If the system evaluates a suspicious request as the lowest level of abnormality, it means that although the feature difference exists, it may be caused by accidental factors and the risk is relatively low. In this case, the system will not directly reject the request, but prompt the user to perform other verification methods other than voice verification, such as providing a password, answering security questions, or confirming the identity through a text message verification code. This method can effectively improve the flexibility of verification and reduce user experience problems caused by misjudgment.

[0195] When the abnormality level of a suspicious request reaches the highest level, the system considers that the request is very likely to be forged or impersonated. At this point, the system will directly reject the request and not allow it to pass. At the same time, the system will notify the security processing end or relevant administrators and require further investigation and processing. This mechanism ensures that high-risk requests will not affect the security of the system and can respond to potential security threats in a timely manner.

[0196] After calculating the feature differences, the system divides the request into different abnormality levels according to the set threshold. The greater the feature difference, the higher the abnormality level. Common classification methods include low-level abnormalities (small differences), medium-level abnormalities (obvious differences) and high-level abnormalities (extreme differences). For each abnormality level, the system presets a corresponding processing flow. For example, for low-level abnormalities, the system prompts the user to perform secondary verification, while for high-level abnormalities, the system directly rejects the request. Each processing operation strictly corresponds to the abnormality level. For low-level abnormalities, the system will not immediately reject the user request, but will guide the user to perform additional verification methods. For example, the system may prompt the user to enter the account password or complete the verification through SMS verification code. This ensures security without excessively affecting the user experience. In the case of the highest level of abnormality, the system rejects the request and immediately notifies the security department. This can ensure that high-risk operations can receive timely attention and processing by triggering alarms or sending notifications to a dedicated security management team.

[0197] This embodiment introduces the classification of abnormal levels and corresponding processing operations, so that the system can flexibly respond to different levels of security risks. Requests with lower risks will not be rejected immediately, but additional verification methods will be used to ensure the user's identity and improve the user experience; for high-risk requests, the system can quickly reject and issue a security alert to ensure the security of the system and user data. The hierarchical processing mechanism can effectively reduce the misjudgment rate and improve the system's security protection capabilities, which is of great significance in high-risk application scenarios such as financial transactions and identity authentication.

[0198] In one embodiment, a voice anti-counterfeiting verification device is provided, and the voice anti-counterfeiting verification device corresponds one-to-one to the voice anti-counterfeiting verification method in the above embodiment. Figure 3 , Figure 3This is a functional module diagram of a preferred embodiment of the voice anti-counterfeiting verification device of the present invention. Private key generation and voice collection module 10, differential privacy processing module 20, signature generation module 30, encryption module 40, voice interaction module 50, feature extraction module 60, verification model module 70 and matching degree analysis and authorization module 80. The functional modules are described in detail as follows:

[0199] The private key generation and voice collection module 10 is used to generate a unique private key for the user and obtain the user's registered voice sample;

[0200] A differential privacy processing module 20, used to add noise to the private key to generate a noisy private key for differential privacy protection;

[0201] The signature generation module 30 is used to combine the noisy private key with the registered voice sample to generate a signature audio by using a signature module based on the U-Net architecture;

[0202] An encryption module 40 is used to encrypt the signature audio and store it in a database;

[0203] The voice interaction module 50 is used to collect the user's request voice sample when the user initiates a service request through voice interaction;

[0204] A feature extraction module 60, configured to extract the user's voice features from the request voice sample;

[0205] A verification model module 70, for matching the speech feature with the signature audio using a pre-trained verification model, and determining a degree of match between the speech feature and the signature audio;

[0206] The matching degree analysis and authorization module 80 is used to confirm that the service request has passed the verification and authorize the service request if the matching degree between the voice feature and the signature audio is higher than the matching degree threshold.

[0207] In one embodiment, the differential privacy processing module 20 is specifically used to:

[0208] determining a distribution type of the noise, wherein the distribution type comprises a Laplace distribution or a Gaussian distribution;

[0209] Numerical encoding and vectorization processing are performed on the private key to generate a private key vector, and a maximum impact value of a single dimension change in the private key vector on the signature audio feature representation to be generated is analyzed, and the maximum impact value is used as the global sensitivity;

[0210] Determining the intensity of the noise based on the global sensitivity and a preset privacy budget;

[0211] generating a complete noise based on the distribution type of the noise and the intensity of the noise;

[0212] The complete noise is injected into the private key to generate a noisy private key.

[0213] In one embodiment, the signature generation module 30 is specifically used to:

[0214] Numerical encoding and vectorization processing are performed on the noisy private key to generate a numerical feature vector;

[0215] Converting the registered speech sample into a frequency domain feature representation through short-time Fourier transform;

[0216] The signature module of the U-Net architecture extracts features from the numerical feature vector and the frequency domain feature representation respectively, and fuses the extracted features to generate a fused feature map;

[0217] The signature audio is reconstructed and generated from the fused feature map, where the signature audio contains the signature feature of the private key and the biometric feature of the registered voice sample.

[0218] In one embodiment, the verification model module 70 is specifically used to:

[0219] Extracting features from the signature audio to generate corresponding signature audio features;

[0220] Inputting the speech features and the signature audio features into a pre-trained verification model;

[0221] The verification model compares the speech features with the signature audio features layer by layer to determine the feature similarity between the speech features and the signature audio features;

[0222] Based on the feature similarity, the matching score between the speech feature and the signature audio is analyzed by cosine similarity.

[0223] In one embodiment, the verification model module 70 is specifically used to:

[0224] If the matching degree between the voice feature and the signature audio is not higher than the matching degree threshold, the service request is confirmed to have failed the verification and a re-verification prompt is given;

[0225] Obtaining a re-verification voice sample of the user, and extracting a re-verification voice feature from the re-verification voice sample;

[0226] Matching the re-verification voice feature with the signature audio to determine the degree of match between the re-verification voice feature and the signature audio;

[0227] Whether the service request is verified is determined based on whether the matching degree between the re-verification voice feature and the signature audio is higher than the matching degree threshold.

[0228] In one embodiment, the verification model module 70 is specifically used to:

[0229] Monitor the user's verification behavior and record the time and results of each verification;

[0230] When the user fails to verify multiple times within a preset time window, the failed voice features are extracted from all the request voice samples that failed verification;

[0231] Analyzing an average feature value of all failed speech features, and comparing the average feature value with a speech feature of a preset normal speech sample to determine a feature difference between the average feature value and the speech feature of the normal speech sample;

[0232] If the feature difference exceeds a preset difference threshold, the service request is marked as a suspicious request.

[0233] In one embodiment, the verification model module 70 is specifically used to:

[0234] Determining an abnormality level of the suspicious request based on a feature difference between the average feature value and a voice feature of a normal voice sample, wherein the abnormality level includes multiple levels;

[0235] Configure corresponding exception handling operations for each level of exception level;

[0236] If the abnormality level of the suspicious request is the lowest abnormality level, a prompt is given to perform verification operations other than voice verification;

[0237] If the abnormality level of the suspicious request is the highest abnormality level, the suspicious request is rejected, and the security processing end is notified to process the suspicious request.

[0238] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal via a network connection. When the computer program is executed by the processor, a function or step on the service side of a voice anti-counterfeiting verification method is implemented.

[0239] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a user side of a voice anti-counterfeiting verification method

[0240] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0241] Generate a unique private key for the user and obtain the user's registered voice sample;

[0242] Adding noise to the private key to generate a noisy private key for differential privacy protection;

[0243] Using a signature module based on the U-Net architecture, the noisy private key is combined with the registered voice sample to generate a signed audio;

[0244] Encrypting the signed audio and storing it in a database;

[0245] When a user initiates a service request through voice interaction, collect the user's request voice sample;

[0246] Extracting the user's voice features from the request voice sample;

[0247] Using a pre-trained verification model, the voice feature is matched with the signature audio to determine the degree of match between the voice feature and the signature audio;

[0248] If the matching degree between the voice feature and the signature audio is higher than the matching degree threshold, the service request is confirmed to be verified and authorized.

[0249] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0250] Generate a unique private key for the user and obtain the user's registered voice sample;

[0251] Adding noise to the private key to generate a noisy private key for differential privacy protection;

[0252] Using a signature module based on the U-Net architecture, the noisy private key is combined with the registered voice sample to generate a signed audio;

[0253] Encrypting the signed audio and storing it in a database;

[0254] When a user initiates a service request through voice interaction, collect the user's request voice sample;

[0255] Extracting the user's voice features from the request voice sample;

[0256] Using a pre-trained verification model, the voice feature is matched with the signature audio to determine the degree of match between the voice feature and the signature audio;

[0257] If the matching degree between the voice feature and the signature audio is higher than the matching degree threshold, the service request is confirmed to be verified and authorized.

[0258] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0259] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0260] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0261] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A voice anti-counterfeiting verification method, characterized in that: The following steps are involved: Generate a unique private key for the user and obtain the user's registered voice sample; Adding noise to the private key to generate a noisy private key for differential privacy protection; Using a signature module based on the U-Net architecture, the noisy private key is combined with the registered voice sample to generate a signed audio; Encrypting the signed audio and storing it in a database; When a user initiates a service request through voice interaction, collect the user's request voice sample; Extracting the user's voice features from the request voice sample; Using a pre-trained verification model, the voice feature is matched with the signature audio to determine the degree of match between the voice feature and the signature audio; If the matching degree between the voice feature and the signature audio is higher than the matching degree threshold, the service request is confirmed to be verified and authorized.

2. The voice anti-counterfeiting verification method as claimed in claim 1, characterized in that: Adding noise to the private key to generate a noisy private key for differential privacy protection, including: determining a distribution type of the noise, wherein the distribution type comprises a Laplace distribution or a Gaussian distribution; Numerical encoding and vectorization processing are performed on the private key to generate a private key vector, and a maximum impact value of a single dimension change in the private key vector on the signature audio feature representation to be generated is analyzed, and the maximum impact value is used as the global sensitivity; Determining the intensity of the noise based on the global sensitivity and a preset privacy budget; generating a complete noise based on the distribution type of the noise and the intensity of the noise; The complete noise is injected into the private key to generate a noisy private key.

3. The voice anti-counterfeiting verification method according to claim 1, characterized in that: Using a signature module based on the U-Net architecture, the noisy private key is combined with the registered voice sample to generate a signed audio, including: Numerical encoding and vectorization processing are performed on the noisy private key to generate a numerical feature vector; Converting the registered speech sample into a frequency domain feature representation through short-time Fourier transform; The signature module of the U-Net architecture extracts features from the numerical feature vector and the frequency domain feature representation respectively, and fuses the extracted features to generate a fused feature map; The signature audio is reconstructed and generated from the fused feature map, where the signature audio contains the signature feature of the private key and the biometric feature of the registered voice sample.

4. The voice anti-counterfeiting verification method according to claim 1, characterized in that: Using a pre-trained verification model, matching the voice feature with the signature audio, and determining a matching degree between the voice feature and the signature audio, including: Extracting features from the signature audio to generate corresponding signature audio features; Inputting the speech features and the signature audio features into a pre-trained verification model; The verification model compares the speech features with the signature audio features layer by layer to determine the feature similarity between the speech features and the signature audio features; Based on the feature similarity, the matching score between the speech feature and the signature audio is analyzed by cosine similarity.

5. The voice anti-counterfeiting verification method according to claim 1, characterized in that: After determining the match between the voice features and the signature audio, it also includes: If the matching degree between the voice feature and the signature audio is not higher than the matching degree threshold, the service request is confirmed to have failed the verification and a re-verification prompt is given; Obtaining a re-verification voice sample of the user, and extracting a re-verification voice feature from the re-verification voice sample; Matching the re-verification voice feature with the signature audio to determine the degree of match between the re-verification voice feature and the signature audio; Whether the service request is verified is determined based on whether the matching degree between the re-verification voice feature and the signature audio is higher than the matching degree threshold.

6. The voice anti-counterfeiting verification method according to claim 5, characterized in that: After determining whether the service request is verified based on whether the matching degree between the re-verification voice feature and the signature audio is higher than the matching degree threshold, the method further includes: Monitor the user's verification behavior and record the time and results of each verification; When the user fails to verify multiple times within a preset time window, the failed voice features are extracted from all the request voice samples that failed verification; Analyzing an average feature value of all failed speech features, and comparing the average feature value with a speech feature of a preset normal speech sample to determine a feature difference between the average feature value and the speech feature of the normal speech sample; If the feature difference exceeds a preset difference threshold, the service request is marked as a suspicious request.

7. The voice anti-counterfeiting verification method according to claim 6, characterized in that: If the feature difference exceeds a preset difference threshold, after marking the service request as a suspicious request, the method further includes: Determining an abnormality level of the suspicious request based on a feature difference between the average feature value and a voice feature of a normal voice sample, wherein the abnormality level includes multiple levels; Configure corresponding exception handling operations for each level of exception level; If the abnormality level of the suspicious request is the lowest abnormality level, a prompt is given to perform verification operations other than voice verification; If the abnormality level of the suspicious request is the highest abnormality level, the suspicious request is rejected, and the security processing end is notified to process the suspicious request.

8. A voice anti-counterfeiting verification device, characterized in that: The voice anti-counterfeiting verification device comprises: The private key generation and voice collection module is used to generate a unique private key for the user and obtain the user's registered voice sample; A differential privacy processing module, used to add noise to the private key to generate a noisy private key for differential privacy protection; A signature generation module, used to combine the noisy private key with the registered voice sample to generate a signed audio using a signature module based on a U-Net architecture; An encryption module, used to encrypt the signed audio and store it in a database; The voice interaction module is used to collect the user's request voice sample when the user initiates a service request through voice interaction; A feature extraction module, used to extract the user's voice features from the request voice sample; A verification model module, used to match the voice feature with the signature audio using a pre-trained verification model, and determine the degree of match between the voice feature and the signature audio; The matching degree analysis and authorization module is used to confirm that the service request has passed the verification and authorize the service request if the matching degree between the voice feature and the signature audio is higher than the matching degree threshold.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a voice anti-counterfeiting verification program stored in the memory and executable on the processor. When the voice anti-counterfeiting verification program is executed by the processor, the steps of the voice anti-counterfeiting verification method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a voice anti-counterfeiting verification program, which, when executed by the processor, implements the steps of the voice anti-counterfeiting verification method according to any one of claims 1 to 7.

Images