A method and device for detecting a fake voice based on federated learning

Abstract

The application provides a method and device for detecting fake speech based on federated learning, constructs a speech data set, and divides the speech data set into at least a first data set and a second data set; uses the first data set to centrally train a fake speech detection backbone model to obtain an initialized global model; distributes the second data set to a plurality of client nodes and distributes the initialized global model to the plurality of client nodes, so that each client node locally trains the received global model based on the local data set allocated to the client node to obtain local model parameters and uploads the local model parameters; receives the local model parameters uploaded by each client node, performs weighted aggregation, and obtains a target federated fake speech detection model; obtains a speech to be detected, inputs the speech to be detected into the target federated fake speech detection model, and generates a fake speech detection result according to the classification confidence output by the model. The original speech sample is locally protected and the multi-node cooperative detection performance is improved.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing technology, and in particular to a method and apparatus for detecting fake speech based on federated learning. Background Technology

[0002] With the development of deep learning and neural speech generation technologies, the quality of forged speech generated based on speech synthesis, speech conversion, recording playback, and speech splicing is constantly improving. High-quality forged speech can pose a security threat to remote identity authentication, speaker verification, voice control, and intelligent voice interaction systems. Therefore, forged speech detection has become an important research direction in the field of speech security.

[0003] Existing methods for detecting spoofed speech typically employ a centralized training approach, where raw speech samples from various sources are aggregated onto a server and then used to train the detection model. While this method can achieve model learning in experimental environments, it has significant limitations in real-world deployments. Firstly, raw speech samples contain speaker voiceprints and speech content information; direct centralized transmission can easily lead to privacy breaches and is also affected by network bandwidth, data management regulations, and cross-domain sharing restrictions. Secondly, local speech samples held by different nodes vary in speaker distribution, spoofing attack types, collection conditions, and sample quantity, making centralized training unsuitable for the needs of distributed node collaborative modeling.

[0004] Federated learning allows clients to retain original data locally, uploading only local model parameters or parameter update results, while the server aggregates the models and updates the global model. This approach enables multi-node collaborative training without transmitting original speech samples. However, existing federated learning technologies are mainly applied to image classification, text analysis, and recommender systems, and solutions for forged speech detection tasks remain insufficient. Especially in forged speech detection scenarios, it is necessary not only to improve the accuracy of classifying real and fake speech but also to reduce the system-level detection cost when chained with an automatic speaker verification system.

[0005] Therefore, there is an urgent need for a method and device for detecting fake speech based on federated learning to improve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a method and apparatus for detecting fake speech based on federated learning, which can achieve local protection of original speech samples and improve the performance of multi-node collaborative detection.

[0007] In a first aspect, the present invention provides a method for detecting fake speech based on federated learning, comprising the steps of: acquiring real speech samples and fake speech samples, constructing a speech dataset, and dividing the speech dataset into at least a first dataset and a second dataset; using the first dataset to perform centralized training on a fake speech detection backbone model to obtain an initialized global model; and distributing the second dataset to... The system assigns one client node to each client node and distributes the initialized global model to each client node, enabling each client node to train the received global model locally based on its allocated local dataset, obtain local model parameters, and upload them. The parameter is a positive integer; it receives local model parameters uploaded by each client node, performs weighted aggregation to obtain the target federated speech forgery detection model; it obtains the speech to be detected, inputs the speech to be detected into the target federated speech forgery detection model, and generates a speech forgery detection result based on the classification confidence output by the model.

[0008] Optionally, it further includes: performing preprocessing on the speech samples in the speech dataset, wherein the preprocessing is used to convert each speech sample into a standardized form that meets the input requirements of the forged speech detection backbone model; the preprocessing includes at least: sampling rate adjustment processing, used to unify the sampling rate of each speech sample to a target sampling rate; sequence length alignment processing, used to adjust the number of sampling points of each speech sample to a target length; and label mapping processing, used to map real speech samples and forged speech samples to corresponding category labels respectively.

[0009] Optionally, the forged speech samples include forged speech generated through speech synthesis, speech conversion, recording playback, or speech splicing; and / or the forged speech detection backbone model is a neural network model based on the original speech waveform input.

[0010] Optionally, the process of using the first dataset to perform centralized training on the backbone model for spoofing speech detection to obtain the initial global model includes: using the Adam optimizer to iteratively train the backbone model for spoofing speech detection, and selecting the optimal model parameters as the initial global model based on the equal error rate on the validation set in the speech dataset.

[0011] Optionally, receiving local model parameters uploaded by each client node and performing weighted aggregation to obtain the target federated speech forgery detection model includes: receiving local model parameters uploaded by each client node, and performing weighted aggregation on each local model parameter according to the proportion of the sample size held by each client node to the total sample size to obtain an updated global model; if the stopping condition is met, it is used as the target federated speech forgery detection model; otherwise, the local training step and the weighted aggregation step are repeated until the stopping condition is met.

[0012] Optionally, the formula for the weighted aggregation is as follows:

[0013]

[0014] in, For the number of client nodes, For the first The number of samples held by each client node. For the first The client node in the ... The model parameters uploaded after local training. The first one obtained after aggregation Round global model parameters.

[0015] Secondly, the present invention provides a forged speech detection device based on federated learning, the device comprising modules / units for performing any of the possible design methods described in the first aspect above. These modules / units can be implemented in hardware or by hardware executing corresponding software.

[0016] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a program executable on the processor, and when the program is executed by the processor, the electronic device implements a method for performing any of the possible designs described above.

[0017] Fourthly, the present invention provides a readable storage medium storing a program, which, when executed, implements a method of any possible design of any of the above aspects.

[0018] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0019] The beneficial effects of the method of this invention are as follows: This invention does not transmit raw speech data during federated training, but only interacts with model parameters, thereby achieving speech data privacy protection. Simultaneously, it adopts a combination of centralized pre-training and federated aggregation, enabling each client to train from a uniformly initialized model, improving the stability of federated training. By modeling the model's output scores, it achieves efficient real / fake speech discrimination, and the model's performance can be comprehensively evaluated from both independent detection performance and system-level detection cost perspectives. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a method for detecting fake speech based on federated learning, provided in an embodiment of the present invention;

[0021] Figure 2A schematic diagram of a forged speech detection device based on federated learning provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word covers the element or object listed following the word and its equivalents, but do not exclude other elements or objects.

[0024] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. In the description of the embodiments of the present invention, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the present invention. The singular expressions “a,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of the present invention, “at least one” and “one or more” refer to one or more (including two). The term “and / or” is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0025] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the invention. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," and "in still other embodiments" appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically emphasized. The term "connection" includes both direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0026] In embodiments of the present invention, "exemplarily" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design described as "exemplarily" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0027] like Figure 1 As shown, a method for detecting fake speech based on federated learning includes the following steps:

[0028] S101, Obtain real speech samples and fake speech samples, construct a speech dataset, and divide the speech dataset into at least a first dataset and a second dataset.

[0029] In some embodiments, the method further includes: performing preprocessing on speech samples in the speech dataset, the preprocessing being used to convert each speech sample into a standardized form that meets the input requirements of the forged speech detection backbone model.

[0030] In some specific embodiments, the preprocessing includes at least: sampling rate adjustment processing, used to unify the sampling rate of each speech sample to a target sampling rate; sequence length alignment processing, used to adjust the number of sampling points of each speech sample to a target length; and label mapping processing, used to map real speech samples and fake speech samples to corresponding category labels respectively.

[0031] In other embodiments, the forged voice samples include forged voice generated through speech synthesis, speech conversion, recording playback, or speech splicing.

[0032] S102, use the first dataset to perform centralized training on the backbone model for fake speech detection to obtain the initialized global model.

[0033] In some embodiments, the forged speech detection backbone model is a neural network model based on the original speech waveform input.

[0034] In other embodiments, the process of using the first dataset to perform centralized training on the backbone model for spoofing speech detection to obtain an initial global model includes: using the Adam optimizer to iteratively train the backbone model for spoofing speech detection, and selecting the optimal model parameters as the initial global model based on the equal error rate on the validation set in the speech dataset.

[0035] S103, allocate the second dataset to The system assigns one client node to each client node and distributes the initialized global model to each client node, enabling each client node to train the received global model locally based on its allocated local dataset, obtain local model parameters, and upload them. It is a positive integer;

[0036] S104 receives the local model parameters uploaded by each client node, performs weighted aggregation, and obtains the target federated forged speech detection model.

[0037] In some embodiments, receiving local model parameters uploaded by each client node and performing weighted aggregation to obtain a target federated speech forgery detection model includes: receiving local model parameters uploaded by each client node, and performing weighted aggregation on each local model parameter according to the proportion of the sample size held by each client node to the total sample size to obtain an updated global model; if the stopping condition is met, it is used as the target federated speech forgery detection model; otherwise, the local training step and the weighted aggregation step are repeated until the stopping condition is met.

[0038] In other embodiments, the formula for the weighted aggregation is as follows:

[0039]

[0040] in, For the number of client nodes, For the first The number of samples held by each client node. For the first The client node in the ... The model parameters uploaded after local training. The first one obtained after aggregation Round global model parameters.

[0041] S105, acquire the speech to be detected, input the speech to be detected into the target federated speech forgery detection model, and generate a speech forgery detection result based on the classification confidence output by the model.

[0042] The advantages of this invention are as follows: A stable initial global model is obtained through centralized pre-training, effectively mitigating aggregation jitter caused by differences in client data distribution, making the subsequent federated learning process more stable and efficient. The adoption of a federated averaging strategy based on sample size ensures that clients with larger sample sizes contribute more to the global model, better reflecting the characteristics of actual distributed data distribution and improving the generalization ability of the final model. This invention completes federated collaborative learning during the training phase and only requires a single target model for inference during the testing phase, eliminating the need for multi-model parallelism, thus keeping memory usage and computational overhead controllable and suitable for practical deployment.

[0043] To facilitate understanding, this embodiment further elaborates on the specific implementation process of the above method in conjunction with a specific application scenario, which includes the following steps:

[0044] Step 1: Dataset Construction and Partitioning. Obtain real speech samples and fake speech samples to construct a speech dataset. Real speech refers to naturally collected, unaltered speech, while fake speech includes speech synthesized, converted, recorded, and spliced ​​speech. Divide the dataset into a pre-training subset (i.e., the first dataset), a federated training data pool (i.e., the second dataset), a validation set, and a test set.

[0045] Step 2, Speech Preprocessing. All speech samples undergo uniform preprocessing:

[0046] Sampling rate adjustment processing, resampling to 16000 Hz;

[0047] Sequence length alignment processing is performed to trim or pad the speech, so that the length of each speech is uniformly 64,600 sampling points;

[0048] Tag mapping is performed, with real speech tagged as bonafide and fake speech tagged as spoof.

[0049] Step 3, Centralized Pre-training. RawNet2 is used as the backbone network for fake speech detection, trained using the Adam optimizer with a learning rate of [missing information]. The weight decays to The batch size is 8, and the number of training iterations is 20. During training, the optimal model parameters are selected based on the equal error rate (EER) on the validation set, and these parameters are used as the initial global model for federated learning.

[0050] Step 4, Federated Data Allocation and Model Deployment. The federated training data pool is divided into... A local dataset for each client, and the initial global model is distributed to... One client node.

[0051] Step 5, Local Training on Clients. Each client updates the global model locally based on its local dataset. One round of local training is performed in each round of federated communication. After training is complete, the local model parameters are uploaded.

[0052] Step 6, Parameter Aggregation. Collect all local model parameters uploaded by clients, and aggregate them by weighted average according to the number of samples from each client to obtain a new global model, as shown in the following formula:

[0053]

[0054] in, For the number of client nodes, For the first The number of samples held by each client node. For the first The client node in the ... The model parameters uploaded after local training. The first one obtained after aggregation Round global model parameters.

[0055] Step 7: Iterate training until convergence. Repeat steps 5 and 6 until the preset number of federated communication rounds (e.g., 10 rounds) is reached or the validation set EER no longer decreases. Select the global model with the lowest EER on the validation set as the final target federated speech forgery detection model.

[0056] Step 8, Preprocessing of the speech to be detected. Perform the same resampling, cropping, and padding operations as in the training phase on the speech to be detected to make it meet the model input requirements.

[0057] Step 9, Model Inference and Score Calculation. Input the preprocessed speech into the target model to obtain the log probabilities of real and fake speech. Calculate the classification soft score using the following formula:

[0058]

[0059] in, Let be the probability that the speech to be detected belongs to the category of real speech. This represents the probability that the speech to be detected belongs to the category of fake speech.

[0060] Step 10, Result Determination. The classification soft score... (i.e., classification confidence) and preset threshold Compare:

[0061] like It was determined to be genuine voice recording;

[0062] like It was determined to be a forged voice recording.

[0063] Step 11, Performance Evaluation and Communication Statistics. Evaluate the target model, calculate the equal error rate (EER) and the minimum cascaded detection cost. Simultaneously, communication overhead is calculated: the amount of communication per round is based on... Calculate, where, For the number of client nodes, The parameter size for a single model is given; and the cumulative communication volume required to reach the optimal model and the total training communication volume are calculated.

[0064] The equal error rate (EER) represents the error rate at a certain decision threshold τ when the false acceptance rate equals the false rejection rate. Its calculation relationship is as follows:

[0065]

[0066] The false acceptance rate was:

[0067]

[0068] The false rejection rate is:

[0069]

[0070] Wherein, FP represents the number of samples that were misidentified as real speech, TN represents the number of samples that were correctly identified as fake speech, FN represents the number of samples that were misidentified as fake speech, and TP represents the number of samples that were correctly identified as real speech.

[0071] Minimum cascaded detection cost function The combined detection cost, used to measure the cascaded execution of the fake speech detection module and the automatic speaker verification module, is defined as follows:

[0072]

[0073]

[0074] in, The threshold for judgment, Indicates at the threshold The false rejection rate of the fake voice detection module against real voice is as follows. This indicates the false acceptance rate of the fake speech detection module for fake speech. , and This represents the cost coefficient. The smaller the value, the better the overall detection performance of the system.

[0075] This represents the overall precision and recall rate; a higher value indicates a stronger overall detection capability of the model.

[0076] The formula for calculating accuracy (ACC) is:

[0077]

[0078] in, This represents the number of samples that were correctly identified as real speech. This represents the number of samples that were correctly identified as fake speech. This represents the number of samples where fake voice was mistakenly identified as real voice. This represents the number of samples where real speech was mistakenly identified as fake speech. Accuracy. This represents the proportion of correctly classified samples out of the total number of samples.

[0079] The formula for calculating the F1 value is:

[0080]

[0081] in, , and The meanings are the same as above. The F1 score is used to comprehensively measure the precision and recall of the model, and can more comprehensively reflect the overall performance of the classification model in real speech and fake speech detection tasks.

[0082] To verify the effectiveness of the present invention in the task of detecting fake voice, the detection performance of the method of the present invention was compared based on the above implementation process, and the detection results and communication efficiency under different client numbers were statistically analyzed. The results are shown in Tables 1 and 2.

[0083] Table 1

[0084]

[0085] Table 1 shows the detection performance of the model on the development set (Dev) and evaluation set (Eval) under different training methods. Dev represents the development set, used for performance verification and model selection during training; Eval represents the evaluation set, used to finally test the model's detection performance. In the table, EER represents the equal error rate, with lower values ​​indicating better detection performance; ACC represents the classification accuracy, with higher values ​​indicating more accurate classification.

[0086] The results in the table show that the federated training method adopted in this invention outperforms the average results of models that are pre-trained only in a centralized manner and models trained individually on each client in terms of EER, ACC, and F1 scores on both the development and evaluation sets, indicating that the proposed method can effectively improve the performance of fake speech detection.

[0087] Table 2

[0088]

[0089] Table 2 further illustrates the performance improvement of the federated training model compared to the baseline model. The results in the table show that, compared to the model trained solely on a pre-trained basis and the model averaged by local clients, the federated training model further reduces the error rate on both the development and evaluation sets, indicating that multi-client parameter aggregation can effectively improve the model's detection capability and generalization performance.

[0090] The advantages of the embodiments of the present invention are as follows:

[0091] (1) Only the model parameters are interacted during the federated training process, and the original speech samples are not transmitted, which significantly improves the privacy protection of the speaker's speech data;

[0092] (2) A centralized pre-training combined with federated aggregation approach is adopted to improve the stability of distributed training and the performance of model detection;

[0093] (3) It can simultaneously calculate EER and min t-DCF, realizing a complete performance evaluation system from classification accuracy to system-level cost;

[0094] (4) It can quantify the amount of communication and training rounds, providing a basis for performance and resource consumption for the deployment of a real distributed fake speech detection system.

[0095] In summary, this invention enables multi-client collaborative training of a forged speech detection model through federated learning, which improves detection accuracy and system robustness while protecting speech privacy. It is suitable for real-world deployment scenarios involving cross-institutions, cross-devices, and where data cannot be centrally aggregated.

[0096] like Figure 2 As shown, based on the above method, this invention provides a forged speech detection device based on federated learning, comprising: a collection unit 201, used to acquire real speech samples and forged speech samples, construct a speech dataset, and divide the speech dataset into at least a first dataset and a second dataset; a training unit 202, used to perform centralized training on a forged speech detection backbone model using the first dataset to obtain an initialized global model; and an allocation unit 203, used to allocate the second dataset to... The system assigns one client node to each client node and distributes the initialized global model to each client node, enabling each client node to train the received global model locally based on its allocated local dataset, obtain local model parameters, and upload them. The value is a positive integer; the receiving unit 204 is used to receive the local model parameters uploaded by each client node, and perform weighted aggregation to obtain the target federated spoofed speech detection model; the detection unit 205 is used to input the speech to be detected into the target federated spoofed speech detection model, and generate spoofed speech detection results based on the classification confidence of the model output.

[0097] It should be understood that all relevant content of each step involved in the above method embodiments can be referenced to the functional description of the corresponding functional module, and will not be repeated here. Furthermore, the use of suffixes such as "module," "component," or "unit" to represent elements is merely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "component," or "unit" can be used interchangeably. Terminals can be implemented in various forms. For example, the terminals described in this invention may include mobile terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers. The following description will use mobile terminals as examples; those skilled in the art will understand that, in addition to elements specifically designed for mobile purposes, the construction according to embodiments of the present invention can also be applied to fixed-type terminals.

[0098] In other embodiments of the present invention, an electronic device 300 is disclosed, such as... Figure 3 As shown, the device may include: one or more processors 301; a memory 302; a display 303; one or more application programs (not shown); and one or more computer programs 304. These devices can be connected via one or more communication buses 305. The one or more computer programs 304 are stored in the memory 302 and configured to be executed by the one or more processors 301. The one or more computer programs 304 include instructions that can be used to perform actions such as... Figure 1 Each step in the corresponding embodiment.

[0099] Processor 301 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0100] The memory 302 can be an internal storage unit of the electronic device 300, such as a hard disk or RAM of the electronic device 300. The memory 302 can also be an external storage device of the electronic device 300, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or FlashCard equipped on the electronic device 300. Furthermore, the memory 302 can include both internal and external storage units of the electronic device 300. The memory 302 is used to store computer programs and other programs and data required by the electronic device. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0101] The computer program 304 can be divided into one or more modules / units. The one or more modules / units can be a series of computer program instruction segments that can perform a specific function. The instruction segments are used to describe the execution process of the computer program 304 in the electronic device 300.

[0102] In addition to the above-described structure, those skilled in the art will understand that Figure 3 This is merely an example of electronic device 300 and does not constitute a limitation on electronic device 300. Electronic device 300 may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0103] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0104] Based on the above embodiments, the present invention also discloses a computer-readable storage medium having at least one computer program stored thereon, wherein the computer program, when executed by a processor, implements the methods described in the foregoing embodiments.

[0105] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. This available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0106] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0107] Although the embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations can be made to these embodiments. The above descriptions are merely embodiments of the present invention and do not limit the patent scope of the present invention. However, it should be understood that such modifications and variations fall within the scope and spirit of the present invention. Moreover, the present invention described herein may have other embodiments and can be implemented or realized in various ways. All equivalent transformations made based on the description and drawings of the present invention, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for detecting forged speech based on federated learning, characterized in that, Including the following steps: Acquire real speech samples and fake speech samples, construct a speech dataset, and divide the speech dataset into at least a first dataset and a second dataset; The backbone model for fake speech detection is trained centrally using the first dataset to obtain the initial global model; Assign the second dataset to The system assigns one client node to each client node and distributes the initialized global model to each client node, enabling each client node to train the received global model locally based on its allocated local dataset, obtain local model parameters, and upload them. It is a positive integer; Receive local model parameters uploaded by each client node, perform weighted aggregation, and obtain the target federated forged speech detection model; The speech to be detected is obtained, and the speech to be detected is input into the target federated speech forgery detection model. The speech forgery detection result is generated based on the classification confidence score output by the model.

2. The method according to claim 1, characterized in that, Also includes: Preprocessing is performed on the speech samples in the speech dataset to convert each speech sample into a standardized form that meets the input requirements of the forgery detection backbone model; The preprocessing includes at least: Sampling rate adjustment processing is used to unify the sampling rate of each speech sample to the target sampling rate; Sequence length alignment is used to adjust the number of sampling points in each speech sample to the target length; Label mapping processing is used to map real speech samples and fake speech samples to corresponding category labels.

3. The method according to claim 1, characterized in that, The forged voice samples include forged voice generated through speech synthesis, speech conversion, recording playback, or speech splicing. And / or the forged speech detection backbone model is a neural network model based on the original speech waveform input.

4. The method according to claim 1, characterized in that, The backbone model for forged speech detection is trained centrally using the first dataset to obtain the initial global model, which includes: The Adam optimizer is used to iteratively train the forged speech detection backbone model, and the optimal model parameters are selected as the initial global model based on the equal error rate on the validation set in the speech dataset.

5. The method according to claim 1, characterized in that, The system receives local model parameters uploaded by each client node, performs weighted aggregation, and obtains the target federated speech forgery detection model, which includes: The system receives local model parameters uploaded by each client node and performs weighted aggregation on each local model parameter according to the proportion of the sample size held by each client node to the total sample size, to obtain an updated global model. If the stopping condition is met, then it is used as the target federated fake speech detection model; otherwise, the local training step and the weighted aggregation step are repeated until the stopping condition is met.

6. The method according to claim 1, characterized in that, The formula for the weighted aggregation is as follows: in, For the number of client nodes, For the first The number of samples held by each client node. For the first The client node in the ... The model parameters uploaded after local training. The first one obtained after aggregation Round global model parameters.

7. A forged speech detection device based on federated learning, used in the method of any one of claims 1-6, characterized in that, include: The acquisition unit is used to acquire real speech samples and fake speech samples, construct a speech dataset, and divide the speech dataset into at least a first dataset and a second dataset. The training unit is used to centrally train the backbone model for fake speech detection using the first dataset to obtain an initialized global model. Allocation unit, used to allocate the second dataset to The system assigns one client node to each client node and distributes the initialized global model to each client node, enabling each client node to train the received global model locally based on its allocated local dataset, obtain local model parameters, and upload them. It is a positive integer; The receiving unit is used to receive the local model parameters uploaded by each client node, and perform weighted aggregation to obtain the target federated fake speech detection model. The detection unit is used to input the speech to be detected into the target federalized speech forgery detection model and generate a speech forgery detection result based on the classification confidence score output by the model.

8. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a program that can run on the processor, and when the program is executed by the processor, causes the electronic device to perform the method of any one of claims 1-6.

9. A readable storage medium storing a program, characterized in that, When the program is executed, it implements the method of any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-6.

Images