Model processing method, device and equipment

Abstract

The embodiment of the invention discloses a model processing method, device and equipment, and the method comprises the steps: obtaining voice sample data, carrying out the feature extraction of the voice sample data through a feature extraction network in a voice risk recognition model, obtaining a corresponding voice sample feature, and carrying out the adjustment of the voice sample feature through an adjustment network in the voice risk recognition model. Based on the voice sample features, determining the validity of each reference feature in a plurality of pre-constructed reference features for the voice sample features, based on the voice sample features and the plurality of reference features, determining the distribution difference information of the plurality of reference features among domains, and based on the validity and the distribution difference information, determining the validity of the plurality of reference features among the domains. And adjusting the number of the plurality of reference features, determining an identification result of the voice sample data based on the voice sample features and the reference features after the number is adjusted through a decision network in the voice risk identification model, and training the voice risk identification model based on the identification result and a preset loss function.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a model processing method, device and equipment. Background Art

[0002] The continuous development of big data and artificial intelligence (AI) has brought convenience to people's lives and work, but it also poses potential risks. Generative artificial intelligence (AIGC) is an AI technology that can autonomously learn from data and generate new content (including images, sounds, text, etc.). However, AI can also be used to forge voices, leading to potential risks such as the spread of false information, discrimination, privacy leaks, and the undermining of social trust. Therefore, speech synthesis or conversion technology also poses security risks in the AIGC era. Therefore, better measures are needed to mitigate these risks, prevent the misuse of this technology, and protect personal privacy data and social security. Summary of the Invention

[0003] The purpose of the embodiments of this specification is to provide a better measure to avoid risks in speech synthesis or conversion, prevent the abuse of technology, and protect personal privacy data and social security.

[0004] In order to implement the above technical solution, the embodiments of this specification are implemented as follows: An embodiment of the present specification provides a model processing method, the method comprising: obtaining speech sample data; performing feature extraction on the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data; determining, through an adjustment network in the speech risk identification model, the validity of each of a plurality of pre-constructed benchmark features for the speech sample features based on the speech sample features, and determining, based on the speech sample features and the plurality of benchmark features, distribution difference information of the plurality of benchmark features between domains, and adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the plurality of benchmark features between domains; determining, through a decision network in the speech risk identification model, a recognition result of whether the speech sample data is forged speech data based on the speech sample features and the adjusted number of benchmark features, and training the speech risk identification model based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

[0005] An embodiment of the present specification provides a model processing device, the device comprising: a speech acquisition module for acquiring speech sample data; a feature extraction module for extracting features from the speech sample data using a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data; an information determination module for determining, using an adjustment network in the speech risk identification model, the validity of each of a plurality of pre-constructed benchmark features for the speech sample features based on the speech sample features, and determining, based on the speech sample features and the plurality of benchmark features, information about distribution differences between the plurality of benchmark features across domains, and adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature for the speech sample features and the information about distribution differences between the plurality of benchmark features across domains; and a training module for determining, using a decision network in the speech risk identification model, a recognition result of whether the speech sample data is forged based on the speech sample features and the adjusted number of benchmark features, and training the speech risk identification model based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

[0006] An embodiment of the present specification provides a model processing device, the model processing device comprising: a processor; and a memory arranged to store computer-executable instructions, wherein the executable instructions, when executed, cause the processor to: obtain speech sample data; perform feature extraction on the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data; determine, through an adjustment network in the speech risk identification model, the validity of each of a plurality of pre-constructed benchmark features for the speech sample features based on the speech sample features, and determine, based on the speech sample features and the plurality of benchmark features, distribution difference information of the plurality of benchmark features between domains; and adjust the number of the plurality of benchmark features based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the plurality of benchmark features between domains; determine, through a decision network in the speech risk identification model, a recognition result of whether the speech sample data is forged speech data based on the speech sample features and the adjusted number of benchmark features, and train the speech risk identification model based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

[0007] An embodiment of the present specification also provides a storage medium, which is used to store computer-executable instructions, which implement the following process when executed by a processor: obtaining speech sample data; performing feature extraction on the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data; determining, through an adjustment network in the speech risk identification model, the validity of each of a plurality of pre-constructed benchmark features for the speech sample features based on the speech sample features, and determining, based on the speech sample features and the plurality of benchmark features, distribution difference information of the plurality of benchmark features between domains; adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the plurality of benchmark features between domains; determining, through a decision network in the speech risk identification model, a recognition result of whether the speech sample data is forged speech data based on the speech sample features and the adjusted number of benchmark features, and training the speech risk identification model based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

[0008] An embodiment of the present specification also provides a computer program product, including a computer program, which, when executed by a processor, implements the following process: obtaining speech sample data; performing feature extraction on the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data; determining, through an adjustment network in the speech risk identification model, the validity of each of a plurality of pre-constructed benchmark features for the speech sample features based on the speech sample features, and determining, based on the speech sample features and the plurality of benchmark features, distribution difference information of the plurality of benchmark features between domains, and adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the plurality of benchmark features between domains; determining, through a decision network in the speech risk identification model, a recognition result of whether the speech sample data is forged speech data based on the speech sample features and the adjusted number of benchmark features, and training the speech risk identification model based on the recognition result and a preset loss function to obtain a trained speech risk identification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some of the embodiments described in this specification. Those skilled in the art can also derive other drawings based on these drawings without inventive work. Figure 1 This is a schematic diagram of the processing process of a model with fixed reference features in this specification; Figure 2 This is a schematic diagram of the structure of a processing system of a model in this specification; Figure 3 A schematic diagram of the processing process of a model in this specification; Figure 4 This is a schematic diagram of a voice risk identification page in this manual; Figure 5 A schematic diagram of a processing process of a specific model in this specification; Figure 6 This is a schematic diagram of a processing process of a distribution difference model based on domain entropy in this specification; Figure 7 This is a schematic diagram of a processing process of a distribution difference model based on maximizing domain entropy in this specification; Figure 8 A schematic diagram of a processing procedure of a model including probability determination of a domain in this specification; Figure 9 A schematic diagram of a processing procedure of a model including category determination in this specification; Figure 10 A schematic diagram of a processing procedure of a model including effectiveness evaluation in this specification; Figure 11 A schematic diagram of a processing device of a model of this specification; Figure 12 A schematic diagram of a processing device of a model for this specification. DETAILED DESCRIPTION

[0010] The embodiments of this specification provide a model processing method, device and equipment.

[0011] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments derived by those skilled in the art based on the embodiments in this specification without creative effort shall fall within the scope of protection of this specification.

[0012] The embodiments of this specification provide an anti-counterfeiting attack detection mechanism based on adaptive benchmark data selection. Generative artificial intelligence (AIGC) is an artificial intelligence technology that can autonomously learn and generate new content (including images, sounds, text, etc.) from data. Speech synthesis or conversion technology also has security risks in the AIGC era. In practical applications, such as Figure 1 As shown, in the model training stage, the speech features of the speech data can be used as the input data of the speech risk identification model, and classification and identification are performed through the number of pre-fixed baseline features. The fixed number of baseline features may limit the generalization ability on unseen domains, especially when facing new or different attack methods. Therefore, the generalization ability of the model needs to be improved. The embodiment of this specification provides an anti-counterfeiting attack detection mechanism based on adaptive baseline data selection. The adaptive adjustment of the number of prototypes allows the system to dynamically adjust the number of prototypes according to the complexity and changes of the input speech; by adaptively adjusting the number of baseline features, overfitting caused by too many baseline features or underfitting caused by insufficient number of baseline features can be avoided, thereby helping to improve the classification performance and accuracy of the model. For specific processing, please refer to the specific content in the following embodiments.

[0013] The model processing method provided in one or more embodiments of this specification can be applied to the implementation environment of model processing, referring to Figure 2 , the implementation environment includes at least: The client 100 and the server 200. In addition, the server 200 may include various network models, among which: The client 100 can run on a terminal device, which can be a mobile phone, a personal computer, a tablet computer, an e-book reader, a wearable device, a device that interacts with information based on AR (Augmented Reality) or VR (Virtual Reality), a laptop computer, etc. The terminal device can install the client 100, which can be an application, a browser, or a subroutine installed in an application, etc.

[0014] The server 200 can run on a server, which can be one or more servers, a server cluster consisting of several servers, or a cloud server on a cloud computing platform. The server can be installed with the server 200, which can be an application or a subroutine within an application. Various network models (including voice risk identification models) can be integrated into the server 200, or the server 200 can call any one or more of the various network models.

[0015] In this implementation environment, the server 200 can obtain voice sample data provided by the client 100, etc., and then, feature extraction can be performed on the voice sample data through the feature extraction network in the voice risk identification model to obtain voice sample features corresponding to the voice sample data. The voice risk identification model is used to identify whether the input voice data is forged voice data. Afterwards, the adjustment network in the voice risk identification model can be used to determine the validity of each of the multiple pre-constructed benchmark features for the voice sample features based on the voice sample features, and based on the voice sample features and the multiple benchmark features, determine the distribution difference information of the multiple benchmark features between domains, and adjust the number of the multiple benchmark features based on the validity of each benchmark feature for the voice sample features and the distribution difference information of the multiple benchmark features between domains. Finally, the decision network in the voice risk identification model can be used to determine the recognition result of whether the voice sample data is forged voice data based on the voice sample features and the adjusted number of benchmark features. The voice risk identification model is trained based on the recognition result and the preset loss function to obtain a trained voice risk identification model.

[0016] like Figure 3 As shown, the embodiment of this specification provides a method for processing a model, and the execution subject of the method can be a terminal device or a server, etc., wherein the terminal device can be a mobile terminal device such as a mobile phone, a tablet computer, or a computer device such as a laptop or a desktop computer, or an IoT device (specifically such as a smart watch, a car-mounted device, etc.), etc., wherein the server can be an independent server, or a server cluster composed of multiple servers, etc. The server can be a background server in the financial field or the online shopping field, etc., or a background server of an application, etc. In this embodiment, the execution subject is a server as an example for detailed description. For the case where the execution subject is a terminal device, please refer to the following server case processing, which will not be repeated here. The method can specifically include the following steps: In step S302, voice sample data is acquired.

[0017] Among them, the voice sample data may include one or more different contents. For example, the voice sample data may only include the data of the voice to be recognized, or the voice sample data may include not only the data of the voice to be recognized, but also the user's demand information, etc. The demand information may include multiple types. For example, the demand information may include relevant information on how to process the voice to be recognized, such as identifying whether the aforementioned voice is forged voice data and explaining the reasons for the recognition results, etc. The specific settings can be based on actual conditions.

[0018] In practice, the scenarios of voice data recognition can include many, for example, users or technicians actively request to recognize a certain voice data, or the current system receives a voice data, but for the sake of subsequent data security, the system needs to identify the risk of the voice data, etc. The specific settings can be based on actual conditions. The uploading of voice data can be achieved in the following ways: the voice risk identification page can be obtained through the terminal device, such as Figure 4 As shown, the voice risk identification page may include a voice data input trigger button, a voice risk identification result output box, an OK button, and a Cancel button. The input trigger button can be used to input voice data in a variety of different ways. For example, the user can press and hold the input trigger button to input voice data. After input is completed, the user releases the pressed input trigger button. Alternatively, the user can click the input trigger button, at which point the voice risk identification page may prompt the user to begin inputting voice data. The user can then enter voice data according to the prompt and click the input trigger button again after input is completed. The specific settings can be customized according to actual circumstances. The user can use the above-mentioned input trigger button and the above-mentioned voice data input methods to input voice data, and the voice data can be sent to the server. The voice sample data can be historical voice data obtained from the above-mentioned voice data recognition scenarios, voice data authorized by the user, or voice data crawled from the network by a web crawler. The specific settings can be customized according to actual circumstances.

[0019] In step S304, feature extraction is performed on the speech sample data through a feature extraction network in the speech risk identification model to obtain speech sample features corresponding to the speech sample data.

[0020] Speech sample features can include a variety of features, such as user timbre and pronunciation habits, and can be set based on actual circumstances. Speech sample features can be represented in vector or matrix form, and can be set based on actual circumstances. The speech risk identification model is used to identify whether input speech data is forged.

[0021] In implementation, the risk identification of voice data can be performed through a voice risk identification model to determine whether the voice data is forged voice data. The voice risk identification model can be divided into multiple different components according to the functions and roles that the voice risk identification model needs to have, which can include a feature extraction network. The voice sample data can be input into the voice risk identification model. First, the voice sample data can be feature extracted through the feature extraction network to obtain the voice sample features corresponding to the voice sample data. Among them, the feature extraction network can pre-set a voice feature extraction algorithm, such as the Mel-frequency cepstral coefficient algorithm, the linear prediction coding algorithm, etc. In addition, the feature extraction network can also be constructed through a machine learning network. For example, the feature extraction network can be constructed through a convolutional neural network, or a recurrent neural network, or a feature extraction network can be constructed based on a Transformer module, or a feature extraction network can be constructed through a wav2vec model, etc. The specific settings can be based on actual conditions. For example, the above-mentioned speech sample data can be input into a feature extraction network constructed by the wav2vec model. Through the Transformer module in the wav2vec model, useful speech sample features can be learned from the waveform corresponding to the speech sample data. Specifically, if the dimension of the speech sample data input into the wav2vec model is: [14, 64000], the dimension of the output speech sample features can be: [14, 1024], etc., which can be set according to actual conditions.

[0022] In step S306, the adjustment network in the speech risk identification model is used to determine the effectiveness of each of the multiple pre-constructed benchmark features for the speech sample feature based on the above-mentioned speech sample feature, and based on the speech sample feature and the multiple benchmark features, the distribution difference information of the multiple benchmark features between domains is determined. Based on the effectiveness of each benchmark feature for the speech sample feature and the distribution difference information of the multiple benchmark features between domains, the number of the multiple benchmark features is adjusted.

[0023] Among them, the benchmark features can be the voice sample features corresponding to the benchmark voice sample data. The benchmark voice sample data can include real voice sample data (that is, the data of the voice uttered by the user (or person)). This part of the voice sample data can be collected in a variety of different ways. For example, voice sample data of specified content or any content can be collected from different users or technicians through equivalent exchange or purchase. It can also include forged voice sample data, among which the forged voice sample data can include voice sample data generated by generative artificial intelligence AIGC, voice sample data generated by generative adversarial network model, voice sample data constructed by expert experience, etc., which can be set specifically according to actual conditions. A domain can be a field, region, interval, or category, etc. In this embodiment, different deep fake speech generation algorithms (such as WaveNet, Tacotron, Vocoder-based, etc.) may generate speech sample data with unique fake traces or acoustic features, so that each unique fake trace or acoustic feature forms an independent "domain". Different domains may have different feature spaces, and the marginal probability distribution (or data distribution) between domain data may be different. Even if the feature spaces between domains are the same, the marginal probability distribution (or data distribution) between domain data is often different. The specific setting can be based on actual conditions.

[0024] In implementation, to adjust the number of baseline features, an adjustment network can be set up in the speech risk identification model. After obtaining speech sample features through the above method, multiple learnable prototype functions can be deployed in the last fully connected layer at the output of the feature extraction network to adjust for complex data distributions. The prototype functions can be presented in vector or matrix form and serve as baseline features. In this embodiment, the task can be a binary classification task (no forgery and forgery). The central number of baseline features for each category can be set to M (M is a positive integer greater than or equal to 1). Among the multiple benchmark features, there may be benchmark features that are valid for the voice sample feature, and there may also be benchmark features that are invalid for the voice sample feature. To this end, the validity between each benchmark feature in the multiple benchmark features and the voice sample feature can be calculated through validity verification, and the benchmark feature that passes the validity verification can be used as the benchmark feature that is valid for the voice sample feature. The validity verification may include multiple methods, such as validity verification methods based on covariance and covariance matrix, validity verification methods based on regression analysis (regression analysis is a statistical method for determining the relationship between two or more groups of variables), validity verification methods based on chart correlation analysis (visualizing data, that is, drawing charts. It is difficult to find trends and connections from the perspective of data alone, but after drawing data points into charts, trends and connections may be clearly apparent), etc. The specific settings can be made according to actual conditions, and the embodiments of this specification do not limit this.

[0025] In order to improve the model's ability to identify different types of attacks and thus enhance the model's generalization ability, it is also possible to determine the distribution difference information of multiple benchmark features between domains. Specifically, the model's optimization algorithm can be pre-set, and the model parameters can be analyzed based on the optimization algorithm by adjusting the network. For example, the model parameters can be analyzed using a gradient descent algorithm, so as to determine the distribution difference information of multiple benchmark features between domains by analyzing the changes in the gradient of the model parameters of the speech risk identification model.

[0026] The adjustment network can dynamically adjust the number of benchmark features (such as adding new benchmark features or reducing benchmark features, etc.) according to the effectiveness of each benchmark feature for the above-mentioned speech sample features and the distribution difference information of multiple benchmark features between domains, so as to adaptively adjust the number of benchmark features according to the complexity and changes of the input speech sample data, thereby avoiding the overfitting problem caused by too many benchmark features or the underfitting problem caused by insufficient number of benchmark features, which helps to improve the classification performance and accuracy of the model.

[0027] In step S308, the decision network in the speech risk identification model is used to determine whether the speech sample data is forged speech data based on the speech sample features and the adjusted number of baseline features. The speech risk identification model is trained based on the recognition results and a preset loss function to obtain a trained speech risk identification model.

[0028] In implementation, a decision network can be set up in advance in the voice risk identification model. Through the decision network, based on the voice sample features and the baseline features after the adjustment number, the voice sample features and the baseline features after the adjustment number are clustered using a clustering algorithm to obtain baseline features that are clustered into the same category as the voice sample features. The category obtained by clustering can be used as the classification basis for the voice sample features, thereby determining whether the voice sample data corresponding to the voice sample features is forged voice data. The above-mentioned processing of the model to determine whether the above-mentioned voice sample data is forged is only an optional processing method. In actual applications, it can also include a variety of different processing methods, which can be set specifically according to actual conditions. The embodiments of this specification do not limit this. The result of the above-mentioned determination of whether the voice sample data is forged voice data can be used as the recognition result of the voice sample data.

[0029] Based on the recognition results, corresponding loss information can be calculated using a preset loss function. The model parameters of the speech risk recognition model can be adjusted using the loss information. Then, the process of steps S304 to S308 can be repeated with more speech sample data to train the speech risk recognition model until the loss function converges. Ultimately, a trained speech risk recognition model can be obtained. The trained speech risk recognition model can more accurately identify whether the input speech data is forged.

[0030] The speech risk recognition model obtained after training with the above model has the number of baseline features reaching a relatively optimal state, and the corresponding speech risk recognition model has also been trained to a relatively optimal state. At this time, the speech risk recognition model can be used to identify whether the input speech data is forged speech data. Specifically, the speech data to be recognized is obtained, and the speech data can be input into the speech risk recognition model. The feature extraction network in the speech risk recognition model is used to extract features of the speech data to obtain corresponding speech features. The similarity between the speech features and the baseline features in the speech risk recognition model can be calculated to obtain the baseline features corresponding to the maximum similarity. The category corresponding to the obtained baseline features can be used as the category of the speech data, where the category can be forged or not forged.

[0031] The embodiment of this specification provides a model processing method, which obtains speech sample data, and then extracts features from the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data. After that, the adjustment network in the speech risk identification model can determine the validity of each of the multiple pre-constructed benchmark features for the speech sample features based on the speech sample features, and determine the distribution difference information of the multiple benchmark features between domains based on the speech sample features and the multiple benchmark features. Based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the multiple benchmark features between domains, the number of the multiple benchmark features is adjusted. Finally, the decision network in the speech risk identification model can determine the speech sample features based on the speech sample features and the adjusted number of benchmark features. The data is an identification result of whether it is forged voice data. The voice risk identification model is trained based on the identification result and a preset loss function to obtain a trained voice risk identification model. In this way, the number of baseline features can be dynamically adjusted according to the complexity of the input voice sample data and the characteristics of deep fake attacks. This adaptability makes the voice risk identification model more flexible and can better adapt to different data distributions and attack types. Moreover, by adaptively adjusting the number of baseline features, the overfitting problem caused by an excessive number of baseline features or the underfitting problem caused by an insufficient number of baseline features can be avoided, which helps to improve the classification performance and accuracy of the voice risk identification model. In addition, the feature extraction and adaptive baseline features are integrated together to form an end-to-end deep fake detection process. This integration helps to improve the efficiency and performance of the entire system.

[0032] In practical applications, to simplify the processing of the voice risk identification model, the length of the voice data can be set to a preset length, such as 4 seconds, 5 seconds, or 10 seconds, which can be set based on actual circumstances. Based on this, if the length of the voice sample data does not reach the preset length, the voice sample data is copied until the length of the copied voice sample data and the voice sample data reaches the preset length.

[0033] In implementation, for example, if the length of the voice sample data is 1 second, the voice sample data can be copied. In order to finally reach the preset length of 4 seconds, 3 voice sample data can be copied, and the 3 copied voice sample data and the initial voice sample data (i.e., 4 voice sample data) are spliced ​​into voice sample data with a length of 4 seconds.

[0034] In addition, if the length of the above-mentioned voice sample data exceeds the preset length, the voice sample data will be trimmed into voice sample data segments of the preset length. For the voice sample data segments that are less than the preset length remaining after the final trimming, the voice sample data segments that are less than the preset length remaining after the trimming can be copied until the length of the copied voice sample data segments and the voice sample data segments that are less than the preset length remaining after the trimming reaches the preset length.

[0035] In implementation, for example, if the length of the voice sample data is 8 seconds, the voice sample data can be trimmed into two 4-second voice sample data segments. For another example, if the length of the voice sample data is 10 seconds, the voice sample data can be trimmed into two 4-second voice sample data segments, leaving a 2-second voice sample data segment. The 2-second voice sample data segment can be copied, and the copied voice sample data segment and the remaining 2-second voice sample data segment can be spliced ​​into a voice sample data segment with a length of 4 seconds.

[0036] In practical applications, each of the above-mentioned reference features may also be optimized, which may specifically include the following steps A2 and A4.

[0037] In step A2, the correlation between the speech sample feature and each of the multiple benchmark features is determined through the adjustment network in the speech risk identification model.

[0038] During implementation, a correlation algorithm can be pre-set in the adjustment network, and the correlation algorithm can be used to calculate the correlation between each of the multiple baseline features and the voice sample feature. The baseline feature with a correlation greater than a preset threshold can be used as a valid baseline feature for the voice sample feature. The relevant information algorithms may include multiple ones, such as the Pearson correlation algorithm, the rank correlation coefficient algorithm, etc., which can be set specifically according to actual conditions, and the embodiments of this specification do not limit this.

[0039] In step A4, each reference feature is optimized based on the determined correlation between the speech sample feature and each reference feature.

[0040] In implementation, each baseline feature can be optimized and adjusted based on the determined correlation between the speech sample feature and each baseline feature, or a specified algorithm (such as a weighted algorithm, etc.) can be pre-set to calculate the correlation between the determined speech sample feature and each baseline feature. The calculated result can be used to optimize and adjust each baseline feature so that each baseline feature can maximize its correlation with the speech feature of the correct category and minimize its correlation with the speech features of irrelevant categories. The specific setting can be based on actual conditions.

[0041] In practical applications, the specific processing methods of the above step A2 can be various. An optional processing method is provided below, which can specifically include the processing of the following steps A22 and A24.

[0042] In step A22, the mutual information between the speech sample feature and each of the multiple reference features is calculated through the adjustment network in the speech risk identification model.

[0043] In implementation, Figure 5 As shown, the mutual information between the speech sample feature and each prototype function (i.e., benchmark feature) can be calculated through the adjustment network in the speech risk identification model to evaluate the correlation between them. Assuming that the speech sample feature is X and the benchmark feature is Y, the mutual information between the speech sample feature and each benchmark feature can be calculated by the following formula I ( X ; Y )

[0044] in, p ( x,y ) is the joint probability distribution function of X and Y, p ( x ) is the marginal probability distribution function of X, p ( y ) is the marginal probability distribution function of Y. The logarithm is usually to the base 2, in which case the unit of mutual information is bits. Alternatively, the natural logarithm (i.e., to the base e) can be used, in which case the unit of mutual information is nat. A larger mutual information value indicates a stronger correlation between the two variables (i.e., speech sample feature X and reference feature Y). If the two variables are completely independent, the mutual information value is 0.

[0045] In step A24, based on the calculated mutual information, the correlation between the speech sample feature and each of the multiple reference features is determined.

[0046] In implementation, Figure 5As shown, the information provided by each benchmark feature can be evaluated based on the mutual information calculated above, thereby determining the correlation between the above-mentioned speech sample feature and each benchmark feature in the multiple benchmark features. That is, if a benchmark feature is a benchmark feature of the "true" category (i.e., speech data produced by a user (or person), i.e., not forged), then its corresponding mutual information should be relatively high for all speech sample data with a true label of "true", and at the same time, its corresponding mutual information should be relatively low for all speech sample data with a true label of "fake" (i.e., synthesized speech data, i.e., forged speech sample data). Similarly, if a benchmark feature is a benchmark feature of the "fake" category, then its corresponding mutual information should be relatively high for all speech sample data with a true label of "fake", and at the same time, its corresponding mutual information should be relatively low for all speech sample data with a true label of "true". In this way, the speech risk identification model can optimize each benchmark feature during the training process, so that each benchmark feature can maximize its mutual information with speech features of the correct category while minimizing its mutual information with speech features of irrelevant categories. The above optimization objectives can also be incorporated into the overall loss function.

[0047] In practical applications, the specific processing methods for determining the distribution difference information of multiple reference features between domains based on the speech sample features and multiple reference features in step S306 can be varied. The following provides an optional processing method, which can specifically include the following steps S30602 to S30606. Based on this, in the above Figure 3 Based on this, the method specifically includes the following steps: Figure 6 shown.

[0048] In step S30602, based on the speech sample features, the probabilities of the domains corresponding to the multiple reference features are obtained.

[0049] In implementation, domain entropy can be used to ensure that there are certain distribution differences between the domains corresponding to multiple benchmark features. Specifically, the probabilities of the domains corresponding to the multiple benchmark features can be determined in advance based on the voice sample features through the adjustment network in the voice risk identification model. When it is necessary to calculate the domain entropy, the probabilities of the domains corresponding to the multiple benchmark features can be obtained from the probabilities of the domains determined above. The pre-determined probabilities of the domains corresponding to the multiple benchmark features can be achieved in a variety of ways. For example, the probabilities of the domains corresponding to the multiple benchmark features can be set based on expert experience in combination with the voice sample features, or the probabilities of the domains corresponding to the multiple benchmark features can be obtained by calculation and verification through a large amount of historical data and voice sample features, etc. The specific setting can be based on actual conditions.

[0050] In step S30604, domain entropies of the domains corresponding to the multiple reference features are determined based on the probabilities of the domains corresponding to the multiple reference features.

[0051] In implementation, the probabilities of domains corresponding to multiple benchmark features can be substituted into the following formula, and the domain entropy of the domains corresponding to multiple benchmark features can be determined by the following formula:

[0052] in, H ( D ) is the domain entropy, D is the set of all possible domains, p ( d ) is a data belonging to the domain d probability.

[0053] In step S30606 , based on the determined domain entropy, distribution difference information of the plurality of reference features between domains is determined.

[0054] In practice, an optimization algorithm can be pre-set through the adjustment network in the speech risk identification model. This optimization algorithm can be used to optimize the domain entropy calculated above to ensure that the predictions for each domain are as uniform as possible, thereby removing domain-specific information. Ultimately, after optimization, information about the distribution differences between multiple benchmark features across domains can be obtained. The optimization algorithm can include various methods, such as genetic algorithms, Monte Carlo methods, and physical particle optimization algorithms, and can be specifically selected based on actual circumstances.

[0055] In practical applications, the specific processing methods of the above step S30606 can be various. The following provides an optional processing method, which can specifically include the processing of the following steps S306062 and S306064. Based on this, in the above Figure 6 Based on this, the method specifically includes the following steps: Figure 7 shown.

[0056] In step S306062, the determined domain entropy is maximized by a preset optimization algorithm to obtain a maximized domain entropy, where the optimization algorithm includes a gradient descent algorithm.

[0057] In step S306064, based on maximizing domain entropy, distribution difference information of multiple benchmark features between domains is determined.

[0058] like Figure 5 As shown, the processing of the above steps S306062 and S306064 can ensure that the benchmark features have the greatest distribution difference between domains by maximizing the domain entropy.

[0059] In practical applications, the specific processing methods of the above step S30602 can be various. The following provides an optional processing method, which can specifically include the following steps S306022 to S306026. Based on this, in the above Figure 7 Based on this, the method specifically includes the following steps: Figure 8 shown.

[0060] In step S306022, a sample data set is obtained, where the sample data set includes a plurality of sample data.

[0061] In step S306024, the number of sample data belonging to each domain in the domains corresponding to the multiple reference features in the sample data set is obtained.

[0062] In step S306026, based on the number of sample data belonging to each domain in the sample data set and the total number of sample data included in the sample data set, the probability of each domain in the domains corresponding to the multiple reference features is determined.

[0063] In implementation, for example, the sample data set includes N sample data, and the domains corresponding to the multiple benchmark features include domains d , belongs to the domain d The number of sample data is Nd, then we can calculate the sample data belongs to the domain d Probability p ( d )= Nd / N, through the above method, the probability of each domain in the domains corresponding to multiple benchmark features can be calculated.

[0064] In practical applications, the specific processing methods for determining whether the voice sample data is forged voice data in the above step S308 by the decision network in the voice risk identification model based on the voice sample features and the adjusted number of benchmark features can be varied. The following is another optional processing method, which can specifically include the following steps S3082 and S3084. Based on this, in the above step S308 Figure 8 Based on this, the method specifically includes the following steps: Figure 9 shown.

[0065] In step S3082, the similarity between the speech sample feature and each of the adjusted number of reference features is calculated through the decision network in the speech risk identification model.

[0066] Among them, the similarity algorithm can be pre-set in the decision network, and the similarity algorithm can be used to calculate the similarity between the speech sample feature and each benchmark feature in the adjusted number of benchmark features. The similarity algorithm can include multiple types, such as Euclidean distance algorithm, cosine similarity algorithm, Pearson correlation coefficient, Jaccard similarity coefficient, etc., which can be set specifically according to actual conditions.

[0067] In step S3084, a baseline feature whose calculated similarity is greater than a preset similarity threshold is obtained, and based on the category corresponding to the obtained baseline feature, an identification result of whether the voice sample data is forged voice data is determined, and the category includes non-forged and forged.

[0068] The similarity threshold can be set according to actual conditions, such as 90% or 95%.

[0069] In implementation, the decision network in the voice risk identification model can be used to obtain the calculated benchmark features with a similarity greater than a preset similarity threshold, and the category corresponding to the obtained benchmark features can be used as the category corresponding to the voice sample data, so as to determine whether the voice sample data is forged voice data. If the obtained benchmark features include multiple, and the categories corresponding to the multiple benchmark features are different, at this time, the benchmark feature corresponding to the maximum value of the similarity between the voice sample features can be obtained, and the category corresponding to the benchmark feature can be determined as the category corresponding to the voice sample data, or the category with the largest number among the categories corresponding to the multiple benchmark features obtained above can be used as the category corresponding to the voice sample data, etc. The specific settings can be made according to actual conditions.

[0070] In practical applications, in step S306, the adjustment network in the voice risk identification model is used to determine the effectiveness of each of the multiple pre-built benchmark features for the voice sample features based on the voice sample features. There are many specific processing methods. The following provides an optional processing method, which may specifically include the following steps S30608 and S30610. Based on this, in the above Figure 6 Based on this, the method specifically includes the following steps: Figure 10 shown.

[0071] In step S30608, the adjustment network in the speech risk identification model is used to determine a first similarity between the speech sample feature and each benchmark feature, and to determine a second similarity between different benchmark features.

[0072] In practice, the similarity between speech sample features and each benchmark feature, as well as the similarity between different benchmark features, can be characterized by reconstruction loss information. Reconstruction loss information measures how well a given input speech sample feature is "represented" or "reconstructed" by a benchmark feature (or a set of benchmark features). If a benchmark feature can well represent the speech sample data it covers (i.e., the reconstruction loss information is small), then this benchmark feature is effective. Conversely, if the reconstruction loss information for speech sample data in a certain area is consistently high, it indicates that the existing benchmark features are unable to effectively capture the speech sample characteristics of this speech sample data. Thus, by adjusting the network, the first similarity between the speech sample feature and each benchmark feature, and the second similarity between different benchmark features, can be determined to determine the corresponding reconstruction loss information.

[0073] In step S30610, based on the first similarity and / or the second similarity, the validity of each reference feature in the plurality of reference features with respect to the speech sample feature is determined.

[0074] In practice, if the reconstruction loss information remains persistently high, during model training, if the speech risk identification model discovers a set of "forged" speech sample data, and after processing by the feature extraction network, the first similarity between these speech sample features and the baseline features corresponding to all existing "forged" speech sample data is relatively small (or the distance is relatively large), resulting in persistently high reconstruction loss information for these speech sample features, this indicates that the existing baseline features corresponding to the "forged" speech sample data cannot well represent or "cover" the speech sample features of this new batch of speech sample data. In other words, each of the multiple baseline features is also not effective for the speech sample features. In this case, it can be determined that this batch of speech sample data may represent a new, under-recognized, or unconventional forgery risk type. Then, one or more baseline features corresponding to the "forged" speech sample data can be added to the region of the feature space corresponding to the high reconstruction loss information. These new baseline features will be initialized from the speech sample data corresponding to the high reconstruction loss information to better capture the characteristics of this type of forgery risk, thereby reducing the reconstruction loss information of this portion of speech sample data and improving the ability to identify this type of forgery risk.

[0075] For the case of low reconstruction loss information, during the model training process, if the speech risk identification model finds that two benchmark features are very close to each other in the feature space (that is, the second similarity between different benchmark features is large (that is, exceeds the preset threshold)), and the speech sample data sets they represent are highly overlapping, resulting in the reconstruction loss information of these two benchmark features being very low, and covering a large amount of the same speech sample data, then it indicates that although each of the multiple benchmark features is effective for the speech sample features, the two benchmark features are redundant and can be merged into one benchmark feature to streamline the model structure, reduce the amount of calculation, and avoid overfitting.

[0076] In addition, if a certain baseline feature has low reconstruction loss information but attracts very few speech sample data, or the baseline feature it represents has been completely included by other baseline features, it can also be removed.

[0077] Based on the above-mentioned processing of maximizing domain entropy, and in combination with the evaluation of the effectiveness of each of the above-mentioned multiple benchmark features with respect to the speech sample features, the adjustment method of the benchmark features (such as increasing the benchmark features, reducing the benchmark features, merging the benchmark features, etc.) and the number of adjustments required can be comprehensively determined, and then the number of current benchmark features can be adjusted based on the adjustment method and the number of adjustments required. Among them, the above-mentioned comprehensive determination method can include multiple methods. For example, corresponding thresholds can be set for the processing of maximizing domain entropy and the evaluation of effectiveness, and the adjustment method of the benchmark features and the number of adjustments required can be determined by the corresponding thresholds. Alternatively, the results of the processing of maximizing domain entropy and the evaluation of effectiveness can be weighted, and the obtained calculation results can be compared with a pre-set specified threshold to determine the adjustment method of the benchmark features and the number of adjustments required. The specific setting can be based on actual conditions and is not limited in this embodiment of this specification.

[0078] The embodiment of this specification provides a model processing method, which obtains speech sample data, and then extracts features from the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data. After that, the adjustment network in the speech risk identification model can determine the validity of each of the multiple pre-constructed benchmark features for the speech sample features based on the speech sample features, and determine the distribution difference information of the multiple benchmark features between domains based on the speech sample features and the multiple benchmark features. Based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the multiple benchmark features between domains, the number of the multiple benchmark features is adjusted. Finally, the decision network in the speech risk identification model can determine the speech sample features based on the speech sample features and the adjusted number of benchmark features. The data is an identification result of whether it is forged voice data. The voice risk identification model is trained based on the identification result and a preset loss function to obtain a trained voice risk identification model. In this way, the number of baseline features can be dynamically adjusted according to the complexity of the input voice sample data and the characteristics of deep fake attacks. This adaptability makes the voice risk identification model more flexible and can better adapt to different data distributions and attack types. Moreover, by adaptively adjusting the number of baseline features, the overfitting problem caused by an excessive number of baseline features or the underfitting problem caused by an insufficient number of baseline features can be avoided, which helps to improve the classification performance and accuracy of the voice risk identification model. In addition, the feature extraction and adaptive baseline features are integrated together to form an end-to-end deep fake detection process. This integration helps to improve the efficiency and performance of the entire system.

[0079] Furthermore, when evaluating the correlation between baseline features and speech sample features, mutual information is used as a metric. Mutual information quantifies the dependency between two variables, helping to more accurately assess the effectiveness of baseline features. Furthermore, when adaptively adjusting the number of baseline features, the model is optimized by maximizing domain entropy. Maximizing domain entropy helps improve the model's ability to identify different attack types, thereby enhancing its generalization capabilities.

[0080] The above is a processing method of the model provided in the embodiment of this specification. Based on the same idea, the embodiment of this specification also provides a processing device of the model, such as Figure 11 shown.

[0081] The processing device of the model includes: a speech acquisition module 1101, a feature extraction module 1102, an information determination module 1103 and a training module 1104, wherein: The voice acquisition module 1101 acquires voice sample data; A feature extraction module 1102 extracts features from the speech sample data using a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data; An information determination module 1103 determines, through an adjustment network in a speech risk identification model, based on the speech sample feature, the validity of each of a plurality of pre-established benchmark features with respect to the speech sample feature, determines, based on the speech sample feature and the plurality of benchmark features, distribution difference information of the plurality of benchmark features between domains, and adjusts the number of the plurality of benchmark features based on the validity of each benchmark feature with respect to the speech sample feature and the distribution difference information of the plurality of benchmark features between domains. The training module 1104 determines the recognition result of whether the voice sample data is forged voice data based on the voice sample features and the adjusted number of baseline features through the decision network in the voice risk recognition model, and trains the voice risk recognition model based on the recognition result and a preset loss function to obtain a trained voice risk recognition model.

[0082] In the embodiment of this specification, the length of the voice sample data is a preset length, and the device further includes: The voice processing module copies the voice sample data if the length of the voice sample data does not reach a preset length until the length of the copied voice sample data and the voice sample data combined reaches a preset length.

[0083] In the embodiment of this specification, the device further includes: a correlation determination module, which determines the correlation between the speech sample feature and each of the multiple reference features through an adjustment network in the speech risk identification model; The optimization module optimizes each reference feature based on the determined correlation between the speech sample feature and each reference feature.

[0084] In an embodiment of the present specification, the correlation determination module calculates the mutual information between the speech sample feature and each of the multiple benchmark features through the adjustment network in the speech risk identification model; and determines the correlation between the speech sample feature and each of the multiple benchmark features based on the calculated mutual information.

[0085] In the embodiment of this specification, the information determination module 1103 includes: A domain probability acquisition unit, which acquires probabilities of domains corresponding to the plurality of reference features based on the speech sample features; a domain entropy determining unit, configured to determine domain entropies of domains corresponding to the plurality of reference features based on probabilities of the domains corresponding to the plurality of reference features; The distribution difference determining unit determines distribution difference information of the plurality of reference features between domains based on the determined domain entropy.

[0086] In an embodiment of the present specification, the distribution difference determination unit maximizes the determined domain entropy through a preset optimization algorithm to obtain a maximized domain entropy, and the optimization algorithm includes a gradient descent algorithm; based on the maximized domain entropy, the distribution difference information of the multiple benchmark features between domains is determined.

[0087] In an embodiment of the present specification, the domain probability acquisition unit acquires a sample data set, wherein the sample data set includes multiple sample data; obtains the number of sample data in the sample data set belonging to each domain in the domains corresponding to the multiple benchmark features; and determines the probability of each domain in the domains corresponding to the multiple benchmark features based on the number of sample data belonging to each domain in the sample data set and the total number of sample data contained in the sample data set.

[0088] In the embodiment of this specification, the information determination module 1103 includes: a similarity determination unit, which determines, through an adjustment network in a speech risk identification model, a first similarity between the speech sample feature and each reference feature, and determines a second similarity between different reference features; The validity evaluation unit determines the validity of each reference feature in a plurality of reference features with respect to the speech sample feature based on the first similarity and / or the second similarity.

[0089] In the embodiment of this specification, the training module 1104 includes: A similarity calculation unit, which calculates the similarity between the speech sample feature and each of the adjusted number of reference features through a decision network in the speech risk identification model; The recognition unit obtains a calculated reference feature whose similarity is greater than a preset similarity threshold, and determines a recognition result of whether the voice data is forged voice data based on the category corresponding to the obtained reference feature, wherein the category includes non-forged and forged.

[0090] The embodiment of this specification provides a model processing device, which obtains speech sample data, and then extracts features from the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data. After that, the adjustment network in the speech risk identification model can determine the validity of each of the multiple pre-constructed benchmark features for the speech sample features based on the speech sample features, and determine the distribution difference information of the multiple benchmark features between domains based on the speech sample features and the multiple benchmark features. Based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the multiple benchmark features between domains, the number of the multiple benchmark features is adjusted. Finally, the decision network in the speech risk identification model can determine the speech sample based on the speech sample features and the benchmark features after the adjustment. The data is an identification result of whether it is forged voice data. The voice risk identification model is trained based on the identification result and a preset loss function to obtain a trained voice risk identification model. In this way, the number of baseline features can be dynamically adjusted according to the complexity of the input voice sample data and the characteristics of deep fake attacks. This adaptability makes the voice risk identification model more flexible and can better adapt to different data distributions and attack types. Moreover, by adaptively adjusting the number of baseline features, the overfitting problem caused by an excessive number of baseline features or the underfitting problem caused by an insufficient number of baseline features can be avoided, which helps to improve the classification performance and accuracy of the voice risk identification model. In addition, the feature extraction and adaptive baseline features are integrated together to form an end-to-end deep fake detection process. This integration helps to improve the efficiency and performance of the entire system.

[0091] Furthermore, when evaluating the correlation between baseline features and speech sample features, mutual information is used as a metric. Mutual information quantifies the dependency between two variables, helping to more accurately assess the effectiveness of baseline features. Furthermore, when adaptively adjusting the number of baseline features, the model is optimized by maximizing domain entropy. Maximizing domain entropy helps improve the model's ability to identify different attack types, thereby enhancing its generalization capabilities.

[0092] The above is a processing device for the model provided in the embodiment of this specification. Based on the same idea, the embodiment of this specification also provides a processing device for the model, such as Figure 12 shown.

[0093] The processing device of the model may provide a terminal device or a server, etc. for the above-mentioned embodiment.

[0094] The model processing device may vary significantly due to configuration or performance differences. It may include one or more processors 1201 and memory 1202. Memory 1202 may store one or more applications or data. Memory 1202 may be either ephemeral or persistent. The applications stored in memory 1202 may include one or more modules (not shown), each of which may include a series of computer-executable instructions for the model processing device. Furthermore, processor 1201 may be configured to communicate with memory 1202 to execute the series of computer-executable instructions in memory 1202 on the model processing device. The model processing device may also include one or more power supplies 1203, one or more wired or wireless network interfaces 1204, one or more input / output interfaces 1205, and one or more keyboards 1206.

[0095] Specifically, in this embodiment, the model processing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions in the model processing device, and the one or more programs are configured to be executed by one or more processors, including computer-executable instructions for performing the following: Obtain voice sample data; Performing feature extraction on the voice sample data through a feature extraction network in a voice risk identification model to obtain voice sample features corresponding to the voice sample data; Determining, by an adjustment network in a speech risk identification model, based on the speech sample feature, the validity of each of a plurality of pre-established benchmark features with respect to the speech sample feature, and determining, based on the speech sample feature and the plurality of benchmark features, distribution difference information between the plurality of benchmark features across domains, and adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature with respect to the speech sample feature and the distribution difference information between the plurality of benchmark features across domains; Through the decision network in the speech risk identification model, based on the speech sample features and the adjusted number of baseline features, the recognition result of whether the speech sample data is forged speech data is determined, and the speech risk identification model is trained based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

[0096] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the model processing device embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, refer to the description of the method embodiment.

[0097] The embodiment of this specification provides a model processing device, which obtains speech sample data, and then extracts features from the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data. After that, the adjustment network in the speech risk identification model can determine the validity of each of the multiple pre-constructed benchmark features for the speech sample features based on the speech sample features, and determine the distribution difference information of the multiple benchmark features between domains based on the speech sample features and the multiple benchmark features. Based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the multiple benchmark features between domains, the number of the multiple benchmark features is adjusted. Finally, the decision network in the speech risk identification model can determine the speech sample based on the speech sample features and the adjusted number of benchmark features. The data is an identification result of whether it is forged voice data. The voice risk identification model is trained based on the identification result and a preset loss function to obtain a trained voice risk identification model. In this way, the number of baseline features can be dynamically adjusted according to the complexity of the input voice sample data and the characteristics of deep fake attacks. This adaptability makes the voice risk identification model more flexible and can better adapt to different data distributions and attack types. Moreover, by adaptively adjusting the number of baseline features, the overfitting problem caused by an excessive number of baseline features or the underfitting problem caused by an insufficient number of baseline features can be avoided, which helps to improve the classification performance and accuracy of the voice risk identification model. In addition, the feature extraction and adaptive baseline features are integrated together to form an end-to-end deep fake detection process. This integration helps to improve the efficiency and performance of the entire system.

[0098] Furthermore, based on the above Figures 3 to 10 One or more embodiments of this specification further provide a storage medium for storing computer-executable instruction information. In a specific embodiment, the storage medium may be a USB flash drive, an optical disk, a hard disk, etc. When the computer-executable instruction information stored in the storage medium is executed by a processor, the following process can be implemented: Obtain voice sample data; Performing feature extraction on the voice sample data through a feature extraction network in a voice risk identification model to obtain voice sample features corresponding to the voice sample data; Determining, by an adjustment network in a speech risk identification model, based on the speech sample feature, the validity of each of a plurality of pre-established benchmark features with respect to the speech sample feature, and determining, based on the speech sample feature and the plurality of benchmark features, distribution difference information between the plurality of benchmark features across domains, and adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature with respect to the speech sample feature and the distribution difference information between the plurality of benchmark features across domains; Through the decision network in the speech risk identification model, based on the speech sample features and the adjusted number of baseline features, the recognition result of whether the speech sample data is forged speech data is determined, and the speech risk identification model is trained based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

[0099] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from the other embodiments. In particular, the aforementioned storage medium embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, refer to the description of the method embodiment.

[0100] The embodiment of this specification provides a storage medium, which obtains speech sample data, and then extracts features from the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data. After that, the adjustment network in the speech risk identification model can determine the validity of each of the multiple pre-constructed benchmark features for the speech sample features based on the speech sample features, and determine the distribution difference information of the multiple benchmark features between domains based on the speech sample features and the multiple benchmark features. Based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the multiple benchmark features between domains, the number of the multiple benchmark features is adjusted. Finally, the decision network in the speech risk identification model can determine the speech sample based on the speech sample features and the benchmark features after the adjustment. The recognition result of whether the data is forged voice data is obtained, and the voice risk recognition model is trained based on the recognition result and the preset loss function to obtain the trained voice risk recognition model. In this way, the number of baseline features can be dynamically adjusted according to the complexity of the input voice sample data and the characteristics of deep fake attacks. This adaptability makes the voice risk recognition model more flexible and can better adapt to different data distributions and attack types. Moreover, by adaptively adjusting the number of baseline features, the overfitting problem caused by too many baseline features or the underfitting problem caused by insufficient number of baseline features can be avoided, which helps to improve the classification performance and accuracy of the voice risk recognition model. In addition, the feature extraction and adaptive baseline features are integrated together to form an end-to-end deep fake detection process. This integration helps to improve the efficiency and performance of the entire system.

[0101] Furthermore, based on the above Figures 3 to 10 One or more embodiments of this specification further provide a computer program product, including a computer program. When the computer program in the computer program product is executed by a processor, it can implement the following process: Obtain voice sample data; Performing feature extraction on the voice sample data through a feature extraction network in a voice risk identification model to obtain voice sample features corresponding to the voice sample data; Determining, by an adjustment network in a speech risk identification model, based on the speech sample feature, the validity of each of a plurality of pre-established benchmark features with respect to the speech sample feature, and determining, based on the speech sample feature and the plurality of benchmark features, distribution difference information between the plurality of benchmark features across domains, and adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature with respect to the speech sample feature and the distribution difference information between the plurality of benchmark features across domains; Through the decision network in the speech risk identification model, based on the speech sample features and the adjusted number of baseline features, the recognition result of whether the speech sample data is forged speech data is determined, and the speech risk identification model is trained based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

[0102] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the aforementioned computer program product embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, reference can be made to the description of the method embodiment.

[0103] The embodiment of the present specification provides a computer program product, which obtains speech sample data, and then extracts features from the speech sample data through a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data. Afterwards, the adjustment network in the speech risk identification model can determine, based on the speech sample features, the validity of each of the pre-constructed multiple benchmark features for the speech sample features, and based on the speech sample features and the multiple benchmark features, determine the distribution difference information of the multiple benchmark features between domains, and adjust the number of the multiple benchmark features based on the validity of each benchmark feature for the speech sample features and the distribution difference information of the multiple benchmark features between domains. Finally, the decision network in the speech risk identification model can determine the speech sample based on the speech sample features and the adjusted number of benchmark features. The data is an identification result of whether it is forged voice data. The voice risk identification model is trained based on the identification result and a preset loss function to obtain a trained voice risk identification model. In this way, the number of baseline features can be dynamically adjusted according to the complexity of the input voice sample data and the characteristics of deep fake attacks. This adaptability makes the voice risk identification model more flexible and can better adapt to different data distributions and attack types. Moreover, by adaptively adjusting the number of baseline features, the overfitting problem caused by an excessive number of baseline features or the underfitting problem caused by an insufficient number of baseline features can be avoided, which helps to improve the classification performance and accuracy of the voice risk identification model. In addition, the feature extraction and adaptive baseline features are integrated together to form an end-to-end deep fake detection process. This integration helps to improve the efficiency and performance of the entire system.

[0104] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using physical hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly performed using software called a "logic compiler." This is similar to the software compilers used during program development. Before compilation, the original code must be written in a specific programming language, called a Hardware Description Language (HDL). There are many types of HDL, including ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that simply by programming a method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0106] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the memory control logic. Those skilled in the art will also appreciate that, in addition to implementing the controller purely in computer-readable program code, the controller can also be implemented in the form of logic gates, switches, an application-specific integrated circuit, a programmable logic controller, an embedded microcontroller, etc. by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the means for implementing the various functions included therein can also be considered as structures within the hardware component. Alternatively, the means for implementing the various functions can be considered both a software module implementing the method and a structure within the hardware component.

[0107] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0108] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0109] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] The embodiments of this specification are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable fraud case serial and parallel device to produce a machine, so that the instructions executed by the processor of the computer or other programmable fraud case serial and parallel device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable fraud case serial and parallel device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, the instruction device being implemented in the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0112] These computer program instructions may also be loaded onto a computer or other programmable device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0113] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0114] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0115] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0116] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0117] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0118] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communications network. In distributed computing environments, program modules may be located in local and remote computer storage media, including storage devices.

[0119] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0120] The foregoing is merely an example of the present invention and is not intended to limit this document. Various modifications and variations are possible within the scope of this document. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this document are intended to be included within the scope of the claims of this document.

Claims

1. A method for processing a model, the method comprising: Obtain voice sample data; Performing feature extraction on the voice sample data through a feature extraction network in a voice risk identification model to obtain voice sample features corresponding to the voice sample data; Determining, by an adjustment network in a speech risk identification model, based on the speech sample feature, the validity of each of a plurality of pre-established benchmark features with respect to the speech sample feature, and determining, based on the speech sample feature and the plurality of benchmark features, distribution difference information between the plurality of benchmark features across domains, and adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature with respect to the speech sample feature and the distribution difference information between the plurality of benchmark features across domains; Through the decision network in the speech risk identification model, based on the speech sample features and the adjusted number of baseline features, the recognition result of whether the speech sample data is forged speech data is determined, and the speech risk identification model is trained based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

2. The method according to claim 1, wherein the length of the voice sample data is a preset length, and the method further comprises: If the length of the voice sample data does not reach the preset length, the voice sample data is copied until the length of the copied voice sample data and the voice sample data is concatenated to reach the preset length.

3. The method according to claim 2, further comprising: determining, by an adjustment network in a speech risk identification model, a correlation between the speech sample feature and each of the plurality of baseline features; Based on the determined correlation between the speech sample feature and each reference feature, each reference feature is optimized.

4. The method according to claim 3, wherein determining the correlation between the speech sample feature and each of the plurality of baseline features through the adjustment network in the speech risk identification model comprises: calculating, by means of an adjustment network in a speech risk identification model, mutual information between the speech sample feature and each of the plurality of reference features; Based on the calculated mutual information, a correlation between the speech sample feature and each of the multiple reference features is determined.

5. The method according to any one of claims 1 to 4, wherein determining, based on the speech sample feature and the multiple reference features, distribution difference information of the multiple reference features between domains comprises: Based on the speech sample features, obtaining probabilities of domains corresponding to the multiple reference features; determining domain entropies of the domains corresponding to the plurality of reference features based on probabilities of the domains corresponding to the plurality of reference features; Based on the determined domain entropy, distribution difference information of the multiple reference features between domains is determined.

6. The method according to claim 5, wherein obtaining the probabilities of the domains corresponding to the plurality of reference features based on the speech sample features comprises: Acquire a sample data set, where the sample data set includes a plurality of sample data; Obtaining the number of sample data in each domain of the domains corresponding to the multiple reference features in the sample data set; The probability of each of the domains corresponding to the plurality of reference features is determined based on the number of sample data belonging to each domain in the sample data set and the total number of sample data contained in the sample data set.

7. The method according to claim 5, wherein determining, based on the speech sample features, the effectiveness of each of a plurality of pre-established baseline features with respect to the speech sample features through an adjustment network in the speech risk identification model comprises: Determining, by means of an adjustment network in a speech risk identification model, a first similarity between the speech sample feature and each benchmark feature, and determining a second similarity between different benchmark features; Based on the first similarity and / or the second similarity, the validity of each reference feature in a plurality of reference features with respect to the speech sample feature is determined.

8. The method according to claim 7, wherein determining, by a decision network in a voice risk identification model, whether the voice sample data is forged voice data based on the voice sample features and the adjusted number of baseline features, comprises: Calculating, by a decision network in a speech risk identification model, a similarity between the speech sample feature and each of the adjusted number of baseline features; Obtain a calculated benchmark feature whose similarity is greater than a preset similarity threshold, and determine whether the voice data is a forged voice data recognition result based on the category corresponding to the obtained benchmark feature, wherein the category includes non-forged and forged.

9. A model processing device, comprising: Voice acquisition module, which obtains voice sample data; A feature extraction module extracts features from the speech sample data using a feature extraction network in a speech risk identification model to obtain speech sample features corresponding to the speech sample data; an information determination module that, based on the speech sample feature, determines, through an adjustment network in the speech risk identification model, the validity of each of a plurality of pre-established benchmark features with respect to the speech sample feature, determines, based on the speech sample feature and the plurality of benchmark features, distribution difference information between the plurality of benchmark features across domains, and adjusts the number of the plurality of benchmark features based on the validity of each benchmark feature with respect to the speech sample feature and the distribution difference information between the plurality of benchmark features across domains; The training module determines, through the decision network in the speech risk identification model, the recognition result of whether the speech sample data is forged speech data based on the speech sample features and the adjusted number of baseline features, and trains the speech risk identification model based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

10. A model processing device, comprising: processor; as well as a memory arranged to store computer-executable instructions which, when executed, cause the processor to: Obtain voice sample data; Performing feature extraction on the voice sample data through a feature extraction network in a voice risk identification model to obtain voice sample features corresponding to the voice sample data; Determining, by an adjustment network in a speech risk identification model, based on the speech sample feature, the validity of each of a plurality of pre-established benchmark features with respect to the speech sample feature, and determining, based on the speech sample feature and the plurality of benchmark features, distribution difference information between the plurality of benchmark features across domains, and adjusting the number of the plurality of benchmark features based on the validity of each benchmark feature with respect to the speech sample feature and the distribution difference information between the plurality of benchmark features across domains; Through the decision network in the speech risk identification model, based on the speech sample features and the adjusted number of baseline features, the recognition result of whether the speech sample data is forged speech data is determined, and the speech risk identification model is trained based on the recognition result and a preset loss function to obtain a trained speech risk identification model.

Images