Audio evaluation method and device based on artificial intelligence, computer equipment and medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to an artificial intelligence-based audio evaluation method and device, computer equipment and a storage medium, and the method comprises the steps: carrying out the preprocessing of an initial audio input by a user, and obtaining a specified audio; performing feature extraction on the specified audio based on the target speaker coding model to obtain speaker embedding; based on the classifier, performing antagonism identification on speaker embedding to obtain an antagonism identification result; if the adversarial identification result is that adversarial interference occurs, performing purification processing on the initial audio based on a diffusion model to obtain a target audio; performing feature extraction on the target audio based on the target speaker coding model to obtain target speaker embedding; calculating the similarity between the target speaker embedding and the known speaker embedding contained in a preset speaker embedding set; and generating a fraud evaluation result corresponding to the initial audio based on the similarity. In addition, fraud evaluation results may be stored in a blockchain. According to the invention, the accuracy of audio fraud identification is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence development technology and the field of financial technology, and in particular to an audio evaluation method, device, computer equipment and storage medium based on artificial intelligence. Background Art

[0002] With the rapid development of artificial intelligence technology, zero-shot text-to-speech (TTS) technology has become a hot topic in research and application in recent years. Zero-shot TTS technology means that the model can generate speech content with the same sound characteristics based on a short voice prompt on a new voice sample that has never been seen. This feature makes it show great application potential in the fields of personalized speech synthesis and voice imitation.

[0003] At present, although zero-sample TTS technology has made significant progress, it still faces many challenges. From a security perspective, the widespread use of zero-sample TTS technology makes malicious attacks possible. Since the TTS system needs to process and synthesize various voice content, attackers may take advantage of this feature and try to deceive the system or mislead the audience by inputting carefully designed audio content. For example, attackers can generate voices that are highly similar to real people to commit illegal acts such as telephone fraud and forged testimony. However, the current TTS system cannot accurately identify whether the input audio is fraudulent, which seriously threatens the security of the system. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose an audio evaluation method, apparatus, computer device and storage medium based on artificial intelligence to solve the technical problem that the existing TTS system cannot accurately identify whether the input audio is fraudulent, which seriously threatens the security of the system.

[0005] In order to solve the above technical problems, the present application embodiment provides an audio evaluation method based on artificial intelligence, which adopts the following technical solution:

[0006] Preprocess the initial audio input by the user to obtain the corresponding specified audio;

[0007] Extracting features of the specified audio based on a target speaker encoding model to obtain a corresponding speaker embedding; wherein the target speaker encoding model is a model generated by adversarially training a preset initial speaker encoding model based on pre-constructed speech sample data;

[0008] Performing adversarial recognition on the speaker embedding based on a preset classifier to obtain a corresponding adversarial recognition result;

[0009] If the adversarial recognition result is adversarial interference, purifying the initial audio based on a preset diffusion model to obtain a corresponding target audio;

[0010] Extracting features of the target audio based on the target speaker encoding model to obtain a corresponding target speaker embedding;

[0011] Calculating the similarity between the target speaker embedding and known speaker embeddings included in a preset speaker embedding set;

[0012] A fraud assessment result corresponding to the initial audio is generated based on the similarity.

[0013] Furthermore, before the step of extracting features from the designated audio based on the target speaker encoding model to obtain the corresponding speaker embedding, the method further includes:

[0014] Obtain a pre-collected raw speech data set;

[0015] Performing attack processing on the original speech data set based on a preset adversarial attack method to generate corresponding adversarial samples;

[0016] Generate corresponding speech sample data based on the original speech data set and the adversarial sample;

[0017] Calling the initial speaker encoding model and obtaining a preset adversarial loss function;

[0018] Based on the adversarial loss function, using the speech sample data to perform adversarial training on the initial speaker encoding model to obtain a trained designated speaker encoding model;

[0019] The target speaker coding model is generated based on the designated speaker coding model.

[0020] Furthermore, the step of attacking the original speech data set based on a preset adversarial attack method to generate a corresponding adversarial sample specifically includes:

[0021] Performing attack processing on the original speech data set based on a preset first adversarial attack method to generate a corresponding first adversarial sample;

[0022] Performing an attack on the original speech data set based on a preset second adversarial attack method to generate a corresponding second adversarial sample;

[0023] Integrate the first adversarial sample and the second adversarial sample to obtain a corresponding integrated sample;

[0024] The integrated sample is used as the adversarial sample.

[0025] Furthermore, the step of purifying the initial audio based on a preset diffusion model to obtain the corresponding target audio specifically includes:

[0026] Inputting the initial audio as a noise signal into the diffusion model;

[0027] Performing reverse sampling processing on the initial audio based on the diffusion model to obtain corresponding reconstructed audio;

[0028] Optimizing the reconstructed audio to obtain a processed designated reconstructed audio;

[0029] The designated reconstructed audio is used as the target audio.

[0030] Furthermore, the step of generating a fraud assessment result corresponding to the initial audio based on the similarity specifically includes:

[0031] Filter out a designated similarity with the highest value from all the similarities;

[0032] Determining whether the specified similarity is less than a preset similarity threshold;

[0033] If so, generating a first fraud assessment result indicating that the initial audio is a fraudulent activity;

[0034] If not, a second fraud assessment result is generated indicating that the initial audio does not belong to fraudulent activity.

[0035] Furthermore, the step of preprocessing the initial audio input by the user to obtain the corresponding specified audio specifically includes:

[0036] Acquire the initial audio input by the user;

[0037] Performing noise reduction processing on the initial audio to obtain a corresponding first audio;

[0038] Performing standardization processing on the first audio to obtain a corresponding second audio;

[0039] Performing sampling rate conversion processing on the second audio to obtain corresponding third audio;

[0040] The third audio is used as the designated audio.

[0041] Furthermore, after the step of generating a fraud assessment result corresponding to the initial audio based on the similarity, the method further includes:

[0042] Determining whether the fraud assessment result indicates that the initial audio is a fraudulent activity;

[0043] If the fraud assessment result is that the initial audio is a fraudulent activity, restricting speech synthesis processing of the audio;

[0044] generating corresponding alarm information based on the initial audio;

[0045] Sending the alarm information to relevant users;

[0046] If the fraud assessment result is that the initial audio does not belong to fraudulent activities, speech synthesis processing is performed on the initial audio.

[0047] In order to solve the above technical problems, the embodiment of the present application further provides an audio evaluation device based on artificial intelligence, which adopts the following technical solution:

[0048] A preprocessing module, used to preprocess the initial audio input by the user to obtain the corresponding specified audio;

[0049] A first extraction module is used to extract features from the specified audio based on a target speaker encoding model to obtain a corresponding speaker embedding; wherein the target speaker encoding model is a model generated by adversarially training a preset initial speaker encoding model based on pre-constructed speech sample data;

[0050] A recognition module, used for performing adversarial recognition on the speaker embedding based on a preset classifier to obtain a corresponding adversarial recognition result;

[0051] A purification module, configured to purify the initial audio based on a preset diffusion model to obtain a corresponding target audio if the adversarial recognition result is adversarial interference;

[0052] A second extraction module, configured to extract features of the target audio based on the target speaker encoding model to obtain a corresponding target speaker embedding;

[0053] A calculation module, configured to calculate a similarity between the target speaker embedding and known speaker embeddings included in a preset speaker embedding set;

[0054] The first generating module is used to generate a fraud assessment result corresponding to the initial audio based on the similarity.

[0055] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0056] Preprocess the initial audio input by the user to obtain the corresponding specified audio;

[0057] Extracting features of the specified audio based on a target speaker encoding model to obtain a corresponding speaker embedding; wherein the target speaker encoding model is a model generated by adversarially training a preset initial speaker encoding model based on pre-constructed speech sample data;

[0058] Performing adversarial recognition on the speaker embedding based on a preset classifier to obtain a corresponding adversarial recognition result;

[0059] If the adversarial recognition result is adversarial interference, purifying the initial audio based on a preset diffusion model to obtain a corresponding target audio;

[0060] Extracting features of the target audio based on the target speaker encoding model to obtain a corresponding target speaker embedding;

[0061] Calculating the similarity between the target speaker embedding and known speaker embeddings included in a preset speaker embedding set;

[0062] A fraud assessment result corresponding to the initial audio is generated based on the similarity.

[0063] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0064] Preprocess the initial audio input by the user to obtain the corresponding specified audio;

[0065] Extracting features of the specified audio based on a target speaker encoding model to obtain a corresponding speaker embedding; wherein the target speaker encoding model is a model generated by adversarially training a preset initial speaker encoding model based on pre-constructed speech sample data;

[0066] Performing adversarial recognition on the speaker embedding based on a preset classifier to obtain a corresponding adversarial recognition result;

[0067] If the adversarial recognition result is adversarial interference, purifying the initial audio based on a preset diffusion model to obtain a corresponding target audio;

[0068] Extracting features of the target audio based on the target speaker encoding model to obtain a corresponding target speaker embedding;

[0069] Calculating the similarity between the target speaker embedding and known speaker embeddings included in a preset speaker embedding set;

[0070] A fraud assessment result corresponding to the initial audio is generated based on the similarity.

[0071] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0072] The present application first pre-processes the initial audio input by the user to obtain the corresponding specified audio; then, based on the target speaker coding model, feature extraction is performed on the specified audio to obtain the corresponding speaker embedding; wherein, the target speaker coding model is a model generated by adversarially training a preset initial speaker coding model based on pre-constructed speech sample data; then, adversarial recognition is performed on the speaker embedding based on a preset classifier to obtain the corresponding adversarial recognition result; if the adversarial recognition result is adversarially interfered, the initial audio is purified based on a preset diffusion model to obtain the corresponding target audio; and then, based on the target speaker coding model, the speaker embedding is subjected to adversarial interference. The encoding model performs feature extraction on the target audio to obtain a corresponding target speaker embedding; further calculates the similarity between the target speaker embedding and the known speaker embeddings contained in the preset speaker embedding set; and finally generates a fraud assessment result corresponding to the initial audio based on the similarity. In this way, the present application can quickly and accurately identify whether there is fraud in the input audio by combining the use of the target speaker encoding model, classifier and diffusion model generated after adversarial training, thereby improving the processing efficiency and accuracy of audio fraud identification, ensuring the reliability of the fraud assessment result corresponding to the generated initial audio, and thus ensuring the security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0074] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0075] Figure 2 A flowchart of an embodiment of an artificial intelligence-based audio evaluation method according to the present application;

[0076] Figure 3 is a structural schematic diagram of an embodiment of an audio evaluation device based on artificial intelligence according to the present application;

[0077] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0078] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0079] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0080] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0081] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0082] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0083] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0084] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0085] It should be noted that the artificial intelligence-based audio evaluation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based audio evaluation device is generally arranged in the server / terminal device.

[0086] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0087] Continue to refer Figure 2 , showing a flow chart of an embodiment of an audio evaluation method based on artificial intelligence according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The audio evaluation method based on artificial intelligence provided in the embodiment of the present application can be applied to any scenario that requires audio evaluation, and the audio evaluation method based on artificial intelligence can be applied to products in these scenarios, for example, audio evaluation in the field of finance and insurance. The audio evaluation method based on artificial intelligence comprises the following steps:

[0088] Step S201, pre-processing the initial audio input by the user to obtain the corresponding designated audio.

[0089] In this embodiment, the electronic device (e.g. Figure 1The server / terminal device shown in the figure) can obtain the initial audio input by the user through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future. The executor of the present application is specifically a zero-sample text-to-speech (TTS) system based on speaker embedding, or simply referred to as the system. The system is responsible for generating speech based on given text and speaker features. The above-mentioned initial audio is the audio data responsible for providing speaker features. Among them, the above-mentioned specific implementation process of preprocessing the initial audio input by the user to obtain the corresponding specified audio will be further described in detail in the subsequent specific embodiments of the present application, and will not be elaborated on here.

[0090] Step S202: extract features of the specified audio based on the target speaker encoding model to obtain a corresponding speaker embedding.

[0091] In this embodiment, the preprocessed designated audio is input into the target speaker encoding model to extract the speaker embedding in the designated audio through the target speaker encoding model. The target speaker encoding model is a model generated by adversarially training a preset initial speaker encoding model based on pre-constructed speech sample data. The specific construction process will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here.

[0092] Step S203, performing adversarial recognition on the speaker embedding based on a preset classifier to obtain a corresponding adversarial recognition result.

[0093] In this embodiment, the classifier is a pre-trained binary classifier, which is used to distinguish normal speech from speech after adversarial attack. The features used for classification (which may be the embedding vector itself or some statistics thereof) are extracted from the speaker embedding. The extracted features are then input into the binary classifier to obtain the corresponding classification results, i.e., adversarial recognition results (including normal or adversarially interfered).

[0094] Step S204: If the adversarial recognition result is adversarial interference, the initial audio is purified based on a preset diffusion model to obtain a corresponding target audio.

[0095] In this embodiment, the specific implementation process of purifying the initial audio based on the preset diffusion model to obtain the corresponding target audio will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated on here.

[0096] Step S205: extract features of the target audio based on the target speaker encoding model to obtain a corresponding target speaker embedding.

[0097] In this embodiment, the purified target audio is input into a target speaker encoding model so as to extract the target speaker embedding in the target audio through the target speaker encoding model.

[0098] Step S206, calculating the similarity between the target speaker embedding and the known speaker embeddings included in the preset speaker embedding set.

[0099] In this embodiment, the speaker embedding set is a pre-constructed data set containing multiple known speaker embeddings. The similarity between the extracted target speaker embedding and the known speaker embedding can be calculated by using a similarity measurement method (such as cosine similarity, Euclidean distance, etc.).

[0100] Step S207: generating a fraud assessment result corresponding to the initial audio based on the similarity.

[0101] In this embodiment, the specific implementation process of generating the fraud assessment result corresponding to the initial audio based on the similarity will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0102] The present application first pre-processes the initial audio input by the user to obtain the corresponding specified audio; then, based on the target speaker coding model, feature extraction is performed on the specified audio to obtain the corresponding speaker embedding; wherein, the target speaker coding model is a model generated by adversarially training a preset initial speaker coding model based on pre-constructed speech sample data; then, adversarial recognition is performed on the speaker embedding based on a preset classifier to obtain the corresponding adversarial recognition result; if the adversarial recognition result is adversarially interfered, the initial audio is purified based on a preset diffusion model to obtain the corresponding target audio; and then, based on the target speaker coding model, the speaker embedding is subjected to adversarial interference. The encoding model performs feature extraction on the target audio to obtain a corresponding target speaker embedding; further calculates the similarity between the target speaker embedding and the known speaker embeddings contained in the preset speaker embedding set; and finally generates a fraud assessment result corresponding to the initial audio based on the similarity. In this way, the present application can quickly and accurately identify whether there is fraud in the input audio by combining the use of the target speaker encoding model, classifier and diffusion model generated after adversarial training, thereby improving the processing efficiency and accuracy of audio fraud identification, ensuring the reliability of the fraud assessment result corresponding to the generated initial audio, and thus ensuring the security of the system.

[0103] In some optional implementations, before step S202, the electronic device may further perform the following steps:

[0104] Get a pre-collected raw speech dataset.

[0105] In this embodiment, a large amount of speech data, including speech samples of different speakers, is collected in advance and used as an original speech data set for subsequent training of a speaker encoder.

[0106] The original speech data set is attacked based on a preset adversarial attack method to generate corresponding adversarial samples.

[0107] In this embodiment, adversarial samples are obtained by adding small perturbations to the original input, which are intended to maximize the difference between the encoder output and the real speaker embedding. The specific implementation process of attacking the original speech dataset based on the preset adversarial attack method to generate the corresponding adversarial samples will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here.

[0108] Generate corresponding speech sample data based on the original speech data set and the adversarial sample.

[0109] In this embodiment, the original speech data set and the adversarial sample may be processed by data combination to generate speech sample data including the original speech data set and the adversarial sample.

[0110] The initial speaker encoding model is called, and a preset adversarial loss function is obtained.

[0111] In this embodiment, there is no specific limitation on the selection of the initial speaker encoder model. For example, a neural network architecture such as a deep neural network (DNN), a convolutional neural network (CNN) or a recurrent neural network (RNN) can be selected to construct the corresponding initial speaker encoder model. In addition, the adversarial loss function is used to measure the degree of performance degradation of the model when it is subjected to adversarial perturbations. For the speaker encoding model, the adversarial loss function can be defined as the difference between the encoder output and the real speaker embedding.

[0112] Based on the adversarial loss function, the initial speaker coding model is adversarially trained using the speech sample data to obtain a trained designated speaker coding model.

[0113] In this embodiment, the initial speaker encoder model is trained by using speech sample data including adversarial samples and original speech data sets, and during the training process, the original loss function and the adversarial loss function are optimized simultaneously to improve the robustness of the model, thereby obtaining a trained designated speaker encoder model. Among them, adversarial training is an efficient method to enhance the ability of the model to resist adversarial attacks, and its core lies in integrating adversarial samples into the training process.

[0114] The target speaker coding model is generated based on the designated speaker coding model.

[0115] In this embodiment, the performance of the designated speaker encoding model on the test set can be regularly evaluated, including indicators such as recognition accuracy and robustness. The model architecture, training parameters or adversarial loss function of the designated speaker encoding model can be adjusted according to the evaluation results to further improve the model performance of the designated speaker encoding model, and the adjusted designated speaker encoding model can be used as the final target speaker encoding model.

[0116] This application obtains a pre-collected original speech data set; then attacks the original speech data set based on a preset adversarial attack method to generate corresponding adversarial samples; then generates corresponding speech sample data based on the original speech data set and the adversarial samples; subsequently calls the initial speaker coding model and obtains a preset adversarial loss function; further based on the adversarial loss function, uses the speech sample data to conduct adversarial training on the initial speaker coding model to obtain a trained designated speaker coding model; finally, generates the target speaker coding model based on the designated speaker coding model. This application attacks the original speech data set based on the use of an adversarial attack method to generate corresponding adversarial samples, then generates corresponding speech sample data based on the original speech data set and the adversarial samples, then uses the speech sample data to conduct adversarial training on the initial speaker coding model based on a preset adversarial loss function, thereby achieving rapid and accurate training to obtain a target speaker coding model that meets the requirements, improving the model construction efficiency of the target speaker coding model, and ensuring the model effect of the obtained target speaker coding model, thereby effectively improving the system's resistance to adversarial attacks.

[0117] In some optional implementations of this embodiment, the attack processing on the original speech data set based on the preset adversarial attack method to generate a corresponding adversarial sample includes the following steps:

[0118] The original speech data set is attacked based on a preset first adversarial attack method to generate a corresponding first adversarial sample.

[0119] In this embodiment, the above-mentioned first adversarial attack method can be specifically a PGD (projected gradient descent) method. Among them, the core idea of ​​the PGD method is to iteratively update the input sample according to the gradient sign of the loss function while controlling the amplitude of the perturbation. Specifically, given the input speech, the target label and a well-trained speaker encoder model. The predicted label can be obtained by calculating the index of the maximum cosine similarity between the speaker embedding of the source speech and the speaker embedding of the adversarial sample extracted by the speaker encoder. Specifically, in order to generate adversarial samples, projected gradient descent (PGD) is used to find the optimal perturbation. The PGD algorithm iteratively updates the perturbation through the following steps: 1. Initialize the perturbation to 0. 2. In each round of iteration, calculate the gradient of the loss function with respect to the input. 3. Update the perturbation according to the direction of the gradient while ensuring that the size of the perturbation does not exceed a predetermined threshold. 4. Project the updated perturbation back to the allowed perturbation space (for example, by clipping the perturbation to ensure that the input is still within the valid range). 5. Repeat steps 2-4 until a predetermined number of iterations is reached or other stop conditions are met. The PGD method can generate adversarial samples that can effectively deceive the speaker encoder, thereby evaluating and enhancing the robustness of the system.

[0120] The original speech data set is attacked based on a preset second adversarial attack method to generate a corresponding second adversarial sample.

[0121] In this embodiment, the above-mentioned second adversarial attack method can specifically be a method based on the Adam optimizer. Unlike the PGD method, the Adam optimizer adopts an adaptive learning rate strategy. During each iteration, Adam not only considers the current gradient data, but also integrates the momentum information of the previous iteration to adjust the disturbance. Specifically, for the Adam optimizer, the relevant parameters (such as momentum, learning rate, etc.) are first initialized. Then Adam is used to maximize the loss function (for example, the cosine similarity difference between the speaker embedding and the target speaker embedding) by iteratively updating the disturbance of the input speech. In each iteration, the gradient of the loss function with respect to the input is calculated, and the disturbance is updated according to the gradient direction. Ensure that the size of the disturbance does not exceed a predetermined threshold to keep the disturbance imperceptible. After reaching a predetermined number of iterations or meeting other stopping conditions, the final adversarial sample is generated. In this way, Adam can quickly adapt to gradient changes while maintaining stability, thereby more effectively generating adversarial disturbances.

[0122] The first adversarial sample and the second adversarial sample are integrated to obtain a corresponding integrated sample.

[0123] In this embodiment, the integrated sample includes the first adversarial sample and the second adversarial sample.

[0124] The integrated sample is used as the adversarial sample.

[0125] In this embodiment, adversarial samples refer to data instances that are intentionally introduced with imperceptible small changes. These changes are obtained by solving specific optimization problems, with the goal of causing a well-trained model to produce incorrect predictions. The PGD (projected gradient descent) method and the Adam optimizer-based method used in this embodiment are both gradient-based white-box attack strategies. The goal of the processing is to attack the speaker encoder of a zero-shot TTS system based on speaker embedding. By adding small perturbations to the input speech, the output speaker embedding changes, thereby causing the TTS system to generate speech that imitates the target speaker.

[0126] The present application attacks the original speech data set based on a preset first adversarial attack method to generate a corresponding first adversarial sample; and attacks the original speech data set based on a preset second adversarial attack method to generate a corresponding second adversarial sample; then integrates the first adversarial sample and the second adversarial sample to obtain a corresponding integrated sample; and subsequently uses the integrated sample as the adversarial sample. The present application attacks the original speech data set based on a first adversarial attack method to generate a corresponding first adversarial sample, and attacks the original speech data set based on a second adversarial attack method to generate a corresponding second adversarial sample, and then integrates the first adversarial sample and the second adversarial sample, thereby realizing the intelligent and accurate generation of adversarial samples that can effectively deceive the speaker encoding model, improving the diversity of the generated adversarial samples and ensuring the accuracy of the adversarial samples.

[0127] In some optional implementations, step S204 includes the following steps:

[0128] The initial audio is input into the diffusion model as a noise signal.

[0129] In this embodiment, the adversarial purification processing of the diffusion model can be used for adversarial attacks, which uses the diffusion model to remove adversarial perturbations in the input data, thereby restoring the original clear speech to provide effective defense. As a plug-and-play component, the diffusion model effectively avoids the domain shift and secondary training problems associated with adversarial training. In addition, they do not need to be trained on predefined adversarial samples, which makes them have strong generalization capabilities and can cope with a variety of attack methods.

[0130] Among them, the diffusion model usually includes a forward diffusion process and a reverse sampling process. The forward diffusion process gradually adds Gaussian noise to the input speech until the distribution of the noisy speech converges to the standard Gaussian distribution. The reverse sampling process attempts to restore the original speech from the standard Gaussian distribution by gradually removing the noise. The objective function is to optimize the model to minimize the error of speech reconstruction, which is achieved by using the mean square error (MSE) as the loss function. By minimizing this loss function, the model is trained to reconstruct the input speech as accurately as possible, thereby improving the defense capability against adversarial attacks. In addition, considering that the adversarial purification process may have an adverse effect on uncontaminated audio, a binary classifier (i.e., the above-mentioned classifier) ​​is set before the diffusion model. The purpose of this classifier is to identify which audio samples have been adversarially disturbed and which are clean samples that have not been disturbed.

[0131] Based on the diffusion model, a reverse sampling process is performed on the initial audio to obtain a corresponding reconstructed audio.

[0132] In this embodiment, the initial audio is subjected to reverse sampling processing using a diffusion model to gradually remove noise from the audio to restore the original speech signal, thereby obtaining the corresponding reconstructed audio. During the reverse sampling process, the diffusion model uses the learned knowledge to predict how much noise should be removed in each time step to minimize the reconstruction error.

[0133] The reconstructed audio is optimized to obtain a processed designated reconstructed audio.

[0134] In this embodiment, the above optimization processing may include denoising, echo removal and other processing to further improve the sound quality.

[0135] The designated reconstructed audio is used as the target audio.

[0136] The present application performs reverse sampling processing on the initial audio as a noise signal into the diffusion model; then performs reverse sampling processing on the initial audio based on the diffusion model to obtain the corresponding reconstructed audio; then optimizes the reconstructed audio to obtain the processed designated reconstructed audio; and subsequently uses the designated reconstructed audio as the target audio. The present application performs reverse sampling processing on the initial audio based on the use of a diffusion model to obtain the reconstructed audio, and then optimizes the reconstructed audio, so as to quickly and accurately complete the purification processing of the initial audio, so as to restore the audio subjected to adversarial disturbance to its original clean state, thereby ensuring the quality and accuracy of the generated target audio.

[0137] In some optional implementations, step S207 includes the following steps:

[0138] A designated similarity with the highest value is selected from all the similarities.

[0139] In this embodiment, a numerical comparison can be performed on the similarities between the calculated target speaker embedding and all known speaker embeddings included in the above speaker embedding set to screen out the specified similarity with the highest value.

[0140] It is determined whether the specified similarity is less than a preset similarity threshold.

[0141] In this embodiment, there is no specific limitation on the value of the similarity threshold, which can be set according to actual business needs, for example, 0.8. If the similarity between the extracted speaker embedding and a known speaker embedding is lower than the threshold, it is considered that the comparison result does not meet expectations.

[0142] If so, a first fraud assessment result is generated indicating that the initial audio is a fraudulent activity.

[0143] In this embodiment, if the specified similarity is less than the similarity threshold, it means that the initial audio cannot be matched to a known speaker, thereby generating a first fraud assessment result indicating that the initial audio belongs to a fraudulent activity.

[0144] If not, a second fraud assessment result is generated indicating that the initial audio does not belong to fraudulent activity.

[0145] In this embodiment, if the specified similarity is greater than the similarity threshold, it means that the initial audio is successfully matched to a known speaker, thereby generating a second fraud assessment result indicating that the initial audio does not belong to a fraudulent activity.

[0146] The present application selects the specified similarity with the highest value from all the similarities; then determines whether the specified similarity is less than a preset similarity threshold; if so, generates a first fraud assessment result indicating that the initial audio belongs to a fraudulent activity; if not, generates a second fraud assessment result indicating that the initial audio does not belong to a fraudulent activity. The present application selects the specified similarity with the highest value from all the similarities, then compares the specified similarity with the similarity threshold, and then can quickly and accurately generate a fraud assessment result corresponding to the initial audio based on the obtained numerical comparison result, thereby improving the efficiency of generating the fraud assessment result and ensuring the accuracy of the obtained fraud assessment result.

[0147] In some optional implementations of this embodiment, step S201 includes the following steps:

[0148] Acquire the initial audio input by the user.

[0149] In this embodiment, the initial audio is the most original version of the audio input by the user that needs to be processed by speech synthesis.

[0150] The initial audio is subjected to noise reduction processing to obtain a corresponding first audio.

[0151] In this embodiment, background noise in the audio may be removed by using a noise reduction algorithm (such as spectral subtraction, Wiener filtering, etc.).

[0152] The first audio is standardized to obtain a corresponding second audio.

[0153] In this embodiment, the amplitude of the audio signal may be standardized to a uniform range to ensure the stability and consistency of subsequent processing.

[0154] Perform sampling rate conversion on the second audio to obtain corresponding third audio.

[0155] In this embodiment, the sampling rate conversion process refers to converting the sampling rate of the audio into the sampling rate required for internal processing of the system.

[0156] The third audio is used as the designated audio.

[0157] The present application obtains the initial audio input by the user; then performs noise reduction processing on the initial audio to obtain the corresponding first audio; then performs standardization processing on the first audio to obtain the corresponding second audio; subsequently performs sampling rate conversion processing on the second audio to obtain the corresponding third audio; and finally uses the third audio as the designated audio. The present application obtains the initial audio input by the user, and then performs noise reduction processing, standardization processing, and sampling rate conversion processing on the initial audio, so as to achieve efficient and accurate pre-processing of the initial audio, and ensure the accuracy and compliance of the generated initial audio.

[0158] In some optional implementations of this embodiment, after step S207, the electronic device may further perform the following steps:

[0159] Determine whether the fraud assessment result indicates that the initial audio is a fraudulent activity.

[0160] In this embodiment, the fraud assessment result may include that the initial audio belongs to a fraudulent activity, or that the initial audio does not belong to a fraudulent activity.

[0161] If the fraud assessment result is that the initial audio belongs to fraudulent activities, then speech synthesis processing of the audio is restricted.

[0162] In this embodiment, if the fraud assessment result is detected that the initial audio belongs to fraudulent activities, it means that an adversarial attack has been detected from the initial audio, and the system will automatically refuse to process the input initial audio into synthetic speech to prevent potential fraud.

[0163] Generate corresponding alarm information based on the initial audio.

[0164] In this embodiment, the corresponding alarm information can be generated by filling the audio information of the initial audio into a preset alarm information template, wherein the content of the alarm information template can be written and generated according to the actual alarm notification requirements.

[0165] The alarm information is sent to relevant users.

[0166] In this embodiment, the above-mentioned relevant user may refer to a system administrator. The above-mentioned alarm information may be sent to the relevant user via email, SMS, push notification, etc.

[0167] If the fraud assessment result is that the initial audio does not belong to fraudulent activities, speech synthesis processing is performed on the initial audio.

[0168] In this embodiment, if the fraud assessment result is detected to be that the initial audio does not belong to fraudulent activities, that is, no adversarial attack is detected from the initial audio, then the speech synthesis process of the initial audio will continue to be performed subsequently.

[0169] This application determines whether the fraud assessment result is that the initial audio belongs to a fraudulent activity; if the fraud assessment result is that the initial audio belongs to a fraudulent activity, then restricting the audio from being processed by speech synthesis; then generating a corresponding alarm message based on the initial audio; subsequently sending the alarm message to the relevant user; and if the fraud assessment result is that the initial audio does not belong to a fraudulent activity, then performing speech synthesis on the initial audio. After generating a fraud assessment result corresponding to the initial audio based on similarity, if the fraud assessment result is detected to be that the initial audio belongs to a fraudulent activity, this application will automatically restrict the audio from being processed by speech synthesis, and will generate a corresponding alarm message based on the initial audio and send it to the relevant user, thereby effectively preventing potential fraudulent behavior. Effectively protect the security of the system and the interests of users, and improve the user experience.

[0170] In some optional implementations, the user information obtained is subject to the user's consent and complies with relevant laws and policies.

[0171] In addition, the non-Company software tools or components that appear in the embodiments of the present application are merely examples and do not represent actual use.

[0172] In addition, this application focuses on the challenges of adversarial attacks and the construction of defense strategies in zero-shot TTS systems based on speaker embedding. At the attack level, white-box attack techniques driven by PGD and Adam optimizers are applied to attack the speaker encoder of the TTS system, with the intention of making the TTS system generate speech that is extremely similar to the target speaker. In order to deal with the risks brought by these attacks, not only the traditional active defense method of adversarial training is adopted, but also the innovative passive defense strategy of adversarial purification based on diffusion model is introduced. We evaluate the effectiveness of defense measures, their impact on model performance, and their impact on speech synthesis quality. Specifically, adversarial training enhances the model's anti-attack ability by introducing adversarial samples during the training process, thereby improving resistance to such attacks. Adversarial purification uses a diffusion probability model to restore adversarially perturbed audio to its original clean state. Experimental results confirm that these defense mechanisms can significantly reduce the impact of adversarial perturbations, thereby enhancing the security and reliability of zero-shot TTS systems based on speaker embedding in adversarial environments.

[0173] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0174] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned fraud assessment results, the above-mentioned fraud assessment results can also be stored in a node of a blockchain.

[0175] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0176] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0177] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0178] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0179] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of an audio evaluation device based on artificial intelligence, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0180] like Figure 3 As shown, the artificial intelligence-based audio evaluation device 300 described in this embodiment includes: a preprocessing module 301, a first extraction module 302, a recognition module 303, a purification module 304, a second extraction module 305, a calculation module 306 and a first generation module 307.

[0181] in:

[0182] The preprocessing module 301 is used to preprocess the initial audio input by the user to obtain the corresponding specified audio;

[0183] A first extraction module 302 is used to extract features from the specified audio based on a target speaker encoding model to obtain a corresponding speaker embedding; wherein the target speaker encoding model is a model generated by adversarially training a preset initial speaker encoding model based on pre-constructed speech sample data;

[0184] The recognition module 303 is used to perform adversarial recognition on the speaker embedding based on a preset classifier to obtain a corresponding adversarial recognition result;

[0185] A purification module 304 is used for, if the adversarial recognition result is adversarial interference, to purify the initial audio based on a preset diffusion model to obtain a corresponding target audio;

[0186] A second extraction module 305 is used to extract features of the target audio based on the target speaker encoding model to obtain a corresponding target speaker embedding;

[0187] A calculation module 306, configured to calculate a similarity between the target speaker embedding and known speaker embeddings included in a preset speaker embedding set;

[0188] The first generating module 307 is used to generate a fraud assessment result corresponding to the initial audio based on the similarity.

[0189] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the audio evaluation method based on artificial intelligence in the aforementioned implementation mode, and are not described in detail here.

[0190] In some optional implementations of this embodiment, the audio evaluation device based on artificial intelligence further includes:

[0191] A first acquisition module is used to acquire a pre-collected original speech data set;

[0192] An attack module, used for attacking the original speech data set based on a preset adversarial attack method to generate a corresponding adversarial sample;

[0193] A second generating module, configured to generate corresponding speech sample data based on the original speech data set and the adversarial sample;

[0194] A calling module, used to call the initial speaker encoding model and obtain a preset adversarial loss function;

[0195] A training module, configured to perform adversarial training on the initial speaker encoding model using the speech sample data based on the adversarial loss function to obtain a trained designated speaker encoding model;

[0196] The third generating module is used to generate the target speaker coding model based on the designated speaker coding model.

[0197] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the audio evaluation method based on artificial intelligence in the aforementioned implementation mode, and are not described in detail here.

[0198] In some optional implementations of this embodiment, the attack module includes:

[0199] A first attack submodule, configured to perform an attack process on the original speech data set based on a preset first adversarial attack method to generate a corresponding first adversarial sample;

[0200] A second attack submodule, used to attack the original speech data set based on a preset second adversarial attack method to generate a corresponding second adversarial sample;

[0201] An integration submodule, configured to integrate the first adversarial sample and the second adversarial sample to obtain a corresponding integrated sample;

[0202] The first determination submodule is used to use the integrated sample as the adversarial sample.

[0203] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the audio evaluation method based on artificial intelligence in the aforementioned implementation mode, and are not described in detail here.

[0204] In some optional implementations of this embodiment, the purification module 304 includes:

[0205] An input submodule, used for inputting the initial audio as a noise signal into the diffusion model;

[0206] A first processing submodule, configured to perform reverse sampling processing on the initial audio based on the diffusion model to obtain corresponding reconstructed audio;

[0207] A second processing submodule is used to optimize the reconstructed audio to obtain a processed designated reconstructed audio;

[0208] The third determination submodule is configured to use the designated reconstructed audio as the target audio.

[0209] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the audio evaluation method based on artificial intelligence in the aforementioned implementation mode, and are not described in detail here.

[0210] In some optional implementations of this embodiment, the first generating module 307 includes:

[0211] A screening submodule, used for screening out a designated similarity with the highest value from all the similarities;

[0212] A judgment submodule, used to judge whether the specified similarity is less than a preset similarity threshold;

[0213] A first generating submodule, configured to generate a first fraud assessment result indicating that the initial audio is a fraudulent activity;

[0214] The second generating submodule is used to generate a second fraud assessment result indicating that the initial audio does not belong to a fraudulent activity if no.

[0215] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the audio evaluation method based on artificial intelligence in the aforementioned implementation mode, and are not described in detail here.

[0216] In some optional implementations of this embodiment, the preprocessing module 301 includes:

[0217] An acquisition submodule, used to acquire the initial audio input by the user;

[0218] A third processing submodule, configured to perform noise reduction processing on the initial audio to obtain a corresponding first audio;

[0219] A fourth processing submodule, configured to perform standardization processing on the first audio to obtain a corresponding second audio;

[0220] a fifth processing submodule, configured to perform sampling rate conversion processing on the second audio to obtain a corresponding third audio;

[0221] The fourth determining submodule is used to use the third audio as the designated audio.

[0222] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the audio evaluation method based on artificial intelligence in the aforementioned implementation mode, and are not described in detail here.

[0223] In some optional implementations of this embodiment, the audio evaluation device based on artificial intelligence further includes:

[0224] A determination module, used for determining whether the fraud assessment result indicates that the initial audio is a fraudulent activity;

[0225] A first processing module, configured to restrict speech synthesis processing of the audio if the fraud assessment result indicates that the initial audio belongs to fraudulent activities;

[0226] A fourth generating module, used to generate corresponding alarm information based on the initial audio;

[0227] A sending module, used to send the alarm information to relevant users;

[0228] The second processing module is used to perform speech synthesis processing on the initial audio if the fraud assessment result is that the initial audio does not belong to fraudulent activities.

[0229] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the audio evaluation method based on artificial intelligence in the aforementioned implementation mode, and are not described in detail here.

[0230] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0231] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0232] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0233] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of an audio evaluation method based on artificial intelligence, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0234] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the artificial intelligence-based audio evaluation method.

[0235] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0236] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based audio evaluation method.

[0237] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0238] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. An audio evaluation method based on artificial intelligence, characterized in that: The steps include: Preprocess the initial audio input by the user to obtain the corresponding specified audio; Extracting features of the specified audio based on a target speaker encoding model to obtain a corresponding speaker embedding; wherein the target speaker encoding model is a model generated by adversarially training a preset initial speaker encoding model based on pre-constructed speech sample data; Performing adversarial recognition on the speaker embedding based on a preset classifier to obtain a corresponding adversarial recognition result; If the adversarial recognition result is adversarial interference, purifying the initial audio based on a preset diffusion model to obtain a corresponding target audio; Extracting features of the target audio based on the target speaker encoding model to obtain a corresponding target speaker embedding; Calculating the similarity between the target speaker embedding and known speaker embeddings included in a preset speaker embedding set; A fraud assessment result corresponding to the initial audio is generated based on the similarity.

2. The audio evaluation method based on artificial intelligence according to claim 1, characterized in that: Before the step of extracting features from the designated audio based on the target speaker encoding model to obtain a corresponding speaker embedding, the method further includes: Obtain a pre-collected raw speech data set; Performing attack processing on the original speech data set based on a preset adversarial attack method to generate corresponding adversarial samples; Generate corresponding speech sample data based on the original speech data set and the adversarial sample; Calling the initial speaker encoding model and obtaining a preset adversarial loss function; Based on the adversarial loss function, using the speech sample data to perform adversarial training on the initial speaker encoding model to obtain a trained designated speaker encoding model; The target speaker coding model is generated based on the designated speaker coding model.

3. The audio evaluation method based on artificial intelligence according to claim 2, characterized in that: The step of attacking the original speech data set based on a preset adversarial attack method to generate a corresponding adversarial sample specifically includes: Performing attack processing on the original speech data set based on a preset first adversarial attack method to generate a corresponding first adversarial sample; Performing an attack on the original speech data set based on a preset second adversarial attack method to generate a corresponding second adversarial sample; Integrate the first adversarial sample and the second adversarial sample to obtain a corresponding integrated sample; The integrated sample is used as the adversarial sample.

4. The audio evaluation method based on artificial intelligence according to claim 1, characterized in that: The step of purifying the initial audio based on a preset diffusion model to obtain the corresponding target audio specifically includes: Inputting the initial audio as a noise signal into the diffusion model; Performing reverse sampling processing on the initial audio based on the diffusion model to obtain corresponding reconstructed audio; Optimizing the reconstructed audio to obtain a processed designated reconstructed audio; The designated reconstructed audio is used as the target audio.

5. The audio evaluation method based on artificial intelligence according to claim 1, characterized in that: The step of generating a fraud assessment result corresponding to the initial audio based on the similarity specifically includes: Filter out a designated similarity with the highest value from all the similarities; Determining whether the specified similarity is less than a preset similarity threshold; If so, generating a first fraud assessment result indicating that the initial audio is a fraudulent activity; If not, a second fraud assessment result is generated indicating that the initial audio does not belong to fraudulent activity.

6. The audio evaluation method based on artificial intelligence according to claim 1, characterized in that: The step of preprocessing the initial audio input by the user to obtain the corresponding specified audio specifically includes: Acquire the initial audio input by the user; Performing noise reduction processing on the initial audio to obtain a corresponding first audio; Performing standardization processing on the first audio to obtain a corresponding second audio; Performing sampling rate conversion processing on the second audio to obtain corresponding third audio; The third audio is used as the designated audio.

7. The audio evaluation method based on artificial intelligence according to claim 1, characterized in that: After the step of generating a fraud assessment result corresponding to the initial audio based on the similarity, the method further includes: Determining whether the fraud assessment result indicates that the initial audio is a fraudulent activity; If the fraud assessment result is that the initial audio is a fraudulent activity, restricting speech synthesis processing of the audio; generating corresponding alarm information based on the initial audio; Sending the alarm information to relevant users; If the fraud assessment result is that the initial audio does not belong to fraudulent activities, speech synthesis processing is performed on the initial audio.

8. An audio evaluation device based on artificial intelligence, characterized in that: include: A preprocessing module, used to preprocess the initial audio input by the user to obtain the corresponding specified audio; A first extraction module is used to extract features from the specified audio based on a target speaker encoding model to obtain a corresponding speaker embedding; wherein the target speaker encoding model is a model generated by adversarially training a preset initial speaker encoding model based on pre-constructed speech sample data; A recognition module, used for performing adversarial recognition on the speaker embedding based on a preset classifier to obtain a corresponding adversarial recognition result; A purification module, configured to purify the initial audio based on a preset diffusion model to obtain a corresponding target audio if the adversarial recognition result is adversarial interference; A second extraction module, configured to extract features of the target audio based on the target speaker encoding model to obtain a corresponding target speaker embedding; A calculation module, configured to calculate a similarity between the target speaker embedding and known speaker embeddings included in a preset speaker embedding set; The first generating module is used to generate a fraud assessment result corresponding to the initial audio based on the similarity.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the audio evaluation method based on artificial intelligence as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the audio assessment method based on artificial intelligence are implemented as described in any one of claims 1 to 7.

Images