Speech call privacy protection method and system fusing edge computing and federated learning

Abstract

This invention discloses a method and system for protecting privacy in voice calls by integrating edge computing and federated learning, specifically relating to the field of voice communication privacy protection technology. The method includes: edge nodes acquiring PCM audio streams, separating speaker identity and content semantics through adversarial decoupling autoencoders, outputting anonymous acoustic latent vectors, concatenating them with word embeddings, and adding Laplacian noise to obtain desensitized joint features; edge nodes training local models, pruning based on second-order gradient statistics to calculate contribution, and uploading the retained gradients after residual encoding, product quantization, Gaussian perturbation, and Paillier encryption; the cloud partitioning dynamic clusters through clustering, securely weighting and aggregating them in the encrypted domain, decrypting them through thresholding to recover the cluster center gradients, updating the meta-basic parameters using Reptile meta-learning, and generating personalized low-rank bias matrices. This invention achieves differential privacy protection, communication compression, and personalized federated learning across the entire voice call link, balancing security and practicality.

Description

Technical Field

[0001] This invention relates to the field of voice communication privacy protection technology, and more specifically, to a voice call privacy protection method and system that integrates edge computing and federated learning. Background Technology

[0002] With the widespread application of voice call services in customer service centers, remote work, smart homes, and other scenarios, call content often contains a large amount of sensitive personal information (such as identity, emotions, and health status). How to effectively protect user privacy while ensuring service quality has become an urgent technical challenge. Existing voice privacy protection methods mainly rely on transport layer encryption (such as TLS / SRTP) and simple voice anonymization (such as tone transformation and voice splicing). The former only protects the transmission process, and voice data is still processed in plaintext at edge nodes or the cloud, posing a risk of data leakage. The latter will seriously damage the semantic integrity and analyzability of voice, making it difficult to meet the business needs of subsequent quality inspection, emotion recognition, etc. In recent years, although differential privacy technology has been introduced into the voice feature publishing stage, existing methods mostly adopt a global noise-adding strategy, which does not fully consider the coupling characteristics of speaker identity and content semantics in voice features, making it difficult to balance privacy protection and utility.

[0003] The combination of edge computing and federated learning offers a new approach to protecting voice call privacy. Edge nodes can process voice streams in real time close to the data source, avoiding the uploading of raw audio to a central server; federated learning allows multiple edge nodes to collaboratively train models without sharing raw data. However, existing solutions have significant shortcomings: on the one hand, model gradients uploaded by edge nodes may still leak user privacy, and existing gradient perturbation or pruning methods lack refined evaluation of gradient contribution, making it difficult to balance communication overhead and model accuracy; on the other hand, the distribution of voice data across different edge nodes exhibits non-independent and identically distributed characteristics (such as differences in accents and business types in different customer service areas), and traditional federated average aggregation methods cannot provide personalized model adaptation for each node. Furthermore, gradients still exist in plaintext during cloud aggregation, posing a risk of eavesdropping for semi-honest cloud service providers, while fully homomorphic encryption schemes have excessive computational overhead and are difficult to implement in real-time voice call scenarios.

[0004] Therefore, there is an urgent need for a method that can balance privacy protection, communication efficiency, and model personalization throughout the entire voice call process. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method and system for protecting voice call privacy by integrating edge computing and federated learning.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for protecting the privacy of voice calls that integrates edge computing and federated learning includes the following steps: Step 1: Edge nodes acquire PCM audio streams from both parties in real time, perform streaming speech transcription to obtain word sequences, extract acoustic features, and output anonymous acoustic latent vectors through adversarial decoupling autoencoders. After concatenating word embeddings and latent vectors, L1 norm pruning and adding Laplace noise are performed to obtain desensitized joint features. Step 2: The edge nodes train a local model based on the desensitized joint features. After backpropagation, the contribution of each parameter is calculated using the second-order statistics of the gradient. The gradients are retained in descending order of contribution and the remaining gradients are set to zero. The retained gradients are then subjected to residual encoding, product quantization, Gaussian noise perturbation, and Paillier homomorphic encryption in sequence. An encrypted update package is generated and uploaded to the cloud. Step 3: The cloud receives the encrypted update packets and plaintext statistical features from each node, performs online clustering to divide the plaintext features into dynamic clusters after dimensionality reduction, uses Paillier additive homomorphism to perform secure weighted aggregation of the encrypted update packets within each cluster, and then recovers the cluster center gradient by decrypting through a threshold. Step 4: The cloud uses the Reptile algorithm to update the meta-basic parameters based on the gradient of the cluster center, and generates a low-rank bias matrix for the adaptive layer of each cluster. The updated meta-basic parameters and the corresponding low-rank bias matrix of the cluster are encrypted with the public key of each node and then sent down. Step 5: The edge node decrypts the received parameters using its local private key, projects the low-rank bias matrix to generate scaling and offset parameters, performs a channel-by-channel affine transformation on the adaptive layer input of the local model to complete the model update, and if the loss does not decrease after multiple consecutive rounds of validation, increases the Gaussian noise variance of the gradient upload and reduces the frequency of participation in federation.

[0007] Specifically, the training loss function of the adversarial decoupled autoencoder in step 1 includes Mel spectrum reconstruction loss, adversarial loss, and mutual information term; The adversarial loss is used to maximize the classification error of the speaker identity branch, and the mutual information term is used to minimize the dependency between the residual branch output and the speaker label; after training, the encoder weights are fixed and deployed on the edge nodes.

[0008] Specifically, the calculation of contribution in step 2 is as follows: Maintain the first-moment estimate of the gradient and the diagonal approximation of the Fisher information matrix. Define the contribution as the absolute value of the gradient divided by the square root of the diagonal approximation, and then multiplied by the direction cosine of the gradient and the first-moment estimate plus a preset bias term. After calculating the contribution, multiply it by the current round decay factor, and retain the top 10% of gradient positions in descending order.

[0009] Specifically, the residual encoding in step 2 is as follows: Calculate the residual between the preserved gradient and the actual uploaded value in the previous round. If it is the first round of training, the uploaded value in the previous round is assumed to be the zero vector. The residual vector is uniformly divided into four-dimensional sub-vectors, and each sub-vector is mapped to the nearest codeword index using a shared product quantization codebook. After dequantization, Gaussian noise that satisfies local differential privacy is added, then converted to an integer and Paillier element-wise encryption is performed.

[0010] Specifically, in step 3, the cloud recovers the cluster center gradient within a secure sandbox using the threshold Paillier scheme: The private key is split into three fragments and hosted on three independent key management nodes. Each node provides partial decryption in a trusted execution environment, and the sandboxes combine them to obtain the plaintext gradient.

[0011] Specifically, in step 4, the cloud dynamically adjusts the rank of the low-rank bias matrix for each cluster: The rank is adjusted based on the magnitude of the loss reduction and the number of participating nodes: the rank is increased when the magnitude of the loss reduction is lower than the preset threshold and the number of nodes is not lower than the minimum value; otherwise, the rank is decreased if the number of nodes is not lower than the minimum value. In other cases, the original rank is maintained and the rank is limited to the preset upper and lower bounds. The step size for each adjustment of the rank is a positive integer.

[0012] Specifically, in step 5, the edge nodes randomly sample desensitized features and labels from the replay buffer to evaluate the local verification loss. If the loss does not decrease for five consecutive rounds, the Gaussian noise variance is increased by 10% and the frequency of participation in federation is reduced to once every two rounds. The process is gradually restored after the accuracy recovers.

[0013] A voice call privacy protection system integrating edge computing and federated learning includes the following modules: The edge feature desensitization module is used to access PCM audio streams in real time on the call center side. It extracts anonymous acoustic latent vectors through streaming speech transcription and adversarial decoupling autoencoder, and adds Laplacian noise after concatenating them with semantic embeddings to output desensitized joint features that satisfy differential privacy. The edge gradient upload module is used to train a local model on edge nodes based on desensitized features, calculate the contribution using second-order gradient statistics and perform structured pruning, and then package and upload the retained gradients to the cloud after product quantization, Gaussian perturbation and Paillier encryption. The cloud aggregation and meta-learning module is used to receive encrypted update packets from each node, group the nodes through clustering, perform secure weighted aggregation in the ciphertext domain, recover the cluster center gradient through threshold decryption, and update the meta-basic parameters and the low-rank bias matrix of each cluster using the Reptile algorithm. The edge model update module is used by edge nodes to decrypt the encrypted meta-model and low-rank bias matrix sent from the cloud using their local private key and perform adaptive layer modulation using the FiLM method to obtain the final local model. If the loss no longer decreases after multiple consecutive rounds of verification, the module dynamically increases the uploaded noise variance and reduces the participation frequency.

[0014] The technical effects and advantages of this invention are as follows: This invention deploys an adversarial decoupling autoencoder at the edge to separate speaker identity information from content semantic information in speech features, retaining only anonymized acoustic latent vectors. Combined with L1 norm pruning and Laplacian noise addition, it achieves desensitized joint feature publishing that satisfies ε-differential privacy, severing the link between the original speech and speaker identity from an information theory perspective. Simultaneously, the edge nodes employ a contribution evaluation method based on second-order gradient statistics, dynamically pruning low-contribution gradients. Combined with product quantization, residual coding, and Gaussian noise perturbation, it significantly reduces uplink communication overhead. Furthermore, under the dual protection of local differential privacy and Paillier homomorphic encryption, it ensures the confidentiality and unlinkability of uploaded gradients.

[0015] The cloud-based system uses online clustering to divide non-independent, identically distributed nodes into dynamic clusters. A threshold Paillier scheme is used to perform secure weighted aggregation in the encrypted domain. After recovering the gradient at the cluster centers, the Reptile meta-learning algorithm generates personalized low-rank bias matrices. The rank is adaptively adjusted based on the loss reduction rate and the number of participating nodes, achieving a dynamic balance between communication efficiency and model expressive power. Edge nodes receive parameters and modulate their local models using FiLM, enabling hot switching and incremental iteration without interrupting the transcription service. This invention constructs an end-to-end privacy protection system throughout the entire voice call chain, balancing data privacy, communication efficiency, and personalized performance, and possesses significant engineering practical value. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, the steps of the voice call privacy protection method integrating edge computing and federated learning are as follows: Step 1: An edge processing module is deployed on the call center side, enabling real-time access to two PCM audio streams from both parties in a call via a TLS 1.3-based encrypted link. First, a streaming speech-to-text engine based on the Conformer-CTC architecture is run, incrementally recognizing the audio in 40ms blocks and outputting timestamped word sequences. Simultaneously, 80-dimensional Mel filter bank features are extracted with 25ms frame lengths and 10ms frame shifts, and the current frame plus two frames on either side are stacked to form a 5-frame context window, resulting in 400-dimensional initial acoustic features. This feature matrix is ​​then input into an adversarial decoupling autoencoder trained on a public multi-speaker dataset. This encoder consists of four layers of one-dimensional dilated convolutions, with the output divided into an identity branch and a residual branch: the identity branch contains fully connected layers and a softmax layer for predicting speaker labels; the residual branch outputs a 64-dimensional anonymized acoustic latent vector after a gradient inversion layer. The loss function during training is: ;in For the total loss function, For the loss in the reconstruction of the Mel spectrum, To balance the hyperparameters to combat the loss, To counteract the loss and maximize the classification error of the identity branch, the mutual information term minimizes the dependency between the residual vector and the speaker label through adversarial mutual information estimation, thereby ensuring that the residual branch only contains non-sensitive attributes such as channel, volume, and speech rate. For the balancing hyperparameters of the mutual information term, Input features With residual branch output Mutual information between them.

[0019] After training, fixed encoder weights are deployed on edge nodes, outputting anonymized acoustic latent vectors for each frame. To align with text semantics, a lightweight pre-trained BERT-mini model is used for word embedding in the transcribed text, mapping each word to a 384-dimensional vector. Average pooling is then performed on the word vectors across the entire speech segment to obtain segment-level semantic embeddings. For the acoustic latent vectors, speaker-independent attention pooling is used to aggregate the frame-level 64-dimensional vectors into a segment-level 64-dimensional representation. The two are concatenated according to feature dimensions to form a 448-dimensional original joint feature. Subsequently, L1 norm pruning is applied to this joint feature, with a pruning threshold C set to 3.0. The global sensitivity Δf = 2C is calculated, and Laplacian noise with a scale parameter b = Δf / ε is generated according to a preset privacy budget ε = 4, where C is the L1 norm pruning threshold (set to 3.0), Δf is the global sensitivity, ε is the differential privacy budget (set to 4), and b is the Laplacian noise scale parameter. Noise is added independently to each dimension to obtain desensitized joint features that satisfy ε-differential privacy. From an information theory perspective, these features can no longer be linked to the original speech and the speaker's identity.

[0020] Step 2: The local model maintained by the edge nodes adopts a "shared base encoder + specific adaptive layer" architecture. The base encoder is a 3-layer bidirectional Transformer, taking 448-dimensional desensitized features as input and outputting a 256-dimensional latent representation. The adaptive layer is a task-specific head for post-transcription processing, such as a speech sentiment classification or keyword detection head, with parameters tailored to individual training objects. Quality inspection business labels (such as customer service attitude ratings) are used as supervision signals, and training is performed using cross-entropy loss.

[0021] After backpropagation, a multidimensional evaluation method based on second-order gradient statistics is used to calculate the contribution: for the t-th round of local training of a certain edge node, let the t-th... Layer parameters The gradient is .

[0022] Maintain two exponential moving averages: First, the first moment estimation of the gradient. This is used to capture the directional trend of gradients; Second, the diagonal approximation of the Fisher information matrix. ,in Let be the decay rate. The expected contribution of the parameter to the decrease in loss is defined as: ;in For the first The contribution of the expected loss reduction of the wheel parameters, This is a diagonal approximation of the Fisher information matrix. To prevent small constants from being divided by zero, ∈[0,1] represents the directional consistency weight bias, used to ensure that the parameters still have a basic contribution even when the direction cosine is zero. This is a first-moment estimate of the gradient. Let L2 norm be the gradient vector. The L2 norm of the first-order moment vector. The directional consistency weight bias has a value range of [0,1]. This contribution incorporates three aspects of information: Reflects the current gradient strength, The historical sensitivity of the parameters is measured, while the direction cosine term assesses the consistency between the current gradient and the historical moving average direction, assigning higher scores to parameters that consistently point in the optimization direction. The calculation yields... Then, multiply by the current round's decay factor. ( This is the attenuation base value, with a range of (0,1). (For the current round of federated learning), and retain the top K% gradient positions in descending order of scores.

[0023] The gradient positions are retained in descending order of scores, with the remaining gradients set to zero. A binary sparse mask is generated to achieve element-wise structured pruning (unstructured pruning combined with sparse matrix encoding). For the retained gradients, the residual between them and the actual uploaded values ​​from the previous round is calculated; if it is the first round of training, the uploaded values ​​from the previous round are defaulted to a zero vector.

[0024] The residual vector is uniformly divided into 4-dimensional sub-vectors. Quantization is performed using a shared product quantization codebook (codebook size 256, trained in the stochastic gradient subspace by the cloud before deployment) obtained from the cloud during initialization at the edge nodes. Each sub-vector is mapped to the nearest codeword index, achieving high compression. After quantization, to satisfy local differential privacy, Gaussian noise with variance σ is added to the codeword vector (after dequantization) corresponding to the index. 2 Gaussian noise, σ according to the target local privacy budget (ε) l =2) and quantization distortion are comprehensively derived. Finally, the noisy codeword vector is converted back to floating-point gradient residuals. To adapt to Paillier homomorphic encryption, each gradient value is multiplied by 10. 6 The integer is then rounded down to a 32-bit signed integer and mapped to the positive integer field. Each integer ciphertext is then encrypted element-by-element using the Paillier public key pre-requested from the key management node, generating an encrypted update packet.

[0025] Meanwhile, edge nodes monitor uplink bandwidth and their own CPU load in real time. If the bandwidth is greater than 1Mbps and the call concurrency is less than 80% of the capacity, they decide to participate in this round of aggregation and upload the encrypted update package, sparse mask, codebook index, and a small amount of plaintext anonymized statistical features (adding Laplace noise with the same privacy budget to the mean and covariance diagonal elements of the anonymous acoustic latent vector and semantic embedding).

[0026] Step 3: The cloud-based federated aggregation module receives data packets from N active nodes. For the plaintext anonymized statistical features uploaded by each node, PCA is first used to reduce the dimension to 16, followed by online cosine similarity clustering: the similarity is compared with the existing K cluster centers; if the maximum similarity is below the threshold of 0.75, a new cluster is created, and the number of clusters K is dynamically adjusted; otherwise, it is assigned to the nearest cluster, and the cluster center is updated to an exponential moving average. After clustering, secure weighted aggregation is performed on the encrypted updates within each cluster in the ciphertext domain: using Paillier additive homomorphism, the encrypted updates of each node i are weighted... Multiply by data volume weight And sum them up to obtain the cluster-level encrypted aggregation gradient. Since there is no private key in the cloud, a secure aggregation decryption protocol needs to be initiated—using the threshold Paillier scheme, the private key is divided into m=3 fragments, each hosted on a separate key management node. Within a trusted execution environment sandbox, each key node provides partial decryption via a secure channel, and the sandbox recovers the plaintext cluster center gradient. Subsequently, the cloud performs model-independent meta-learning based on the cluster center model replica: using the Reptile algorithm, each cluster is treated as a task, the cluster model is updated using the cluster center gradient, and the difference between the cluster model and the previous meta-model is calculated as the meta-gradient; meta-basic parameters... according to Update; among which The meta-learning rate, The current number of dynamic clusters, The model gradient for each cluster.

[0027] Simultaneously, a low-rank bias matrix is ​​generated for each cluster: two low-rank matrices are learned for the weight matrix W of the adaptive layer. and ,satisfy Bias The number of parameters is only O(r(d+k)), where Here, is the bias of the weight matrix, and d is the input dimension of the adaptive layer. is the output dimension of the adaptive layer, and r is the low-rank number.

[0028] Let the current round of federated training be t, and the cluster The rank of the current bias matrix is Its value satisfies ,in and These are the lower and upper bounds of the rank, respectively. Clusters are obtained in the cloud within a secure sandbox. After implementing the plaintext cluster-centric model, the loss of the model is evaluated on the global validation set. And calculate the decrease compared to the previous round's loss: t represents the federal training round; Simultaneously, statistics on the participating clusters in this round were compiled. Number of nodes in the aggregation When the decrease in loss is below a preset threshold And the number of participating nodes is not less than If the current low-rank capacity is limited, the rank should be increased to enhance personalized expression capabilities; conversely, if the loss has been sufficiently reduced but communication resources are strained, the rank can be decreased to reduce the overhead of the next round of transmission. The rank adjustment method is as follows: ;in For clusters In the The rank of the low-rank matrix of the wheel, The threshold for loss reduction, For the amount of loss, The minimum number of nodes required to allow rank adjustment. , For the lower and upper bounds of rank, The rank step size for each adjustment, For symbolic functions, For indicator functions, Limit the results to [ , ]Inside. when When the sign function outputs +1, the rank increases. Otherwise, output -1, and the rank decreases. If the number of nodes is insufficient, the original rank is maintained. This rank update is performed after the meta-learning parameter update, and only affects the dimension of the low-rank matrix AB in the next round of training, without changing the content of the bias matrix in the current round.

[0029] Through this adaptive mechanism, the personalized capacity and communication overhead of the cluster are dynamically balanced, enabling the meta-personalization model to be both compact and highly adaptable in non-independent and identically distributed scenarios.

[0030] The final cloud-based distribution strategy is as follows: using the basic parameters, each node's public key is used to encrypt and distribute the data, and the corresponding cluster's A and B matrices are encrypted and distributed in the same way, while updating the cluster allocation information.

[0031] Step 4: After receiving the encrypted meta-fundamental model and low-rank bias matrix, the edge node decrypts them using its local private key. The linear modulation of the adaptive layer employs the FiLM method: a channel-by-channel affine transformation is performed on the input x of the adaptive layer. ,in The modulated output, This is a channel-by-channel offset parameter vector. For element-wise multiplication, and From the bias matrix Projection generation (flattens AB into a vector and generates scaling and offset parameters through a linear layer).

[0032] The modulated parameters form the final local model parameters. The edge nodes load the model and continue to randomly sample from the replay buffer (which stores the most recent 1000 desensitized features and labels) to evaluate the local validation loss. If the loss no longer decreases after 5 consecutive rounds, the Gaussian noise variance during gradient upload is increased by 10% (equivalent to reducing the local privacy budget ε). l The frequency of participation in federation will be reduced to once every two rounds; if accuracy recovers, it will be gradually restored. The transcription service will not be interrupted during the model hot-swapping process.

[0033] The following are the modules of a voice call privacy protection system that integrates edge computing and federated learning: The edge feature desensitization module receives PCM audio streams in real time on the call center side, extracts anonymous acoustic latent vectors through streaming speech transcription and adversarial decoupling autoencoders, concatenates them with semantic embeddings and adds Laplacian noise, and outputs desensitized joint features that satisfy differential privacy.

[0034] The edge gradient upload module trains a local model on edge nodes based on desensitized features, calculates the contribution using second-order gradient statistics and performs structured pruning, and packages and uploads the retained gradients to the cloud after product quantization, Gaussian perturbation and Paillier encryption.

[0035] The cloud aggregation and meta-learning module receives encrypted update packets from each node, groups the nodes through clustering, performs secure weighted aggregation in the ciphertext domain, recovers the cluster center gradient through threshold decryption, and uses the Reptile algorithm to update the meta-basic parameters and the low-rank bias matrix of each cluster.

[0036] The edge model update module receives the encrypted meta-model and low-rank bias matrix from the cloud, decrypts them with its local private key, and performs adaptive layer modulation using the FiLM method to obtain the final local model. If the loss no longer decreases after multiple consecutive rounds of verification, the uploaded noise variance is dynamically increased and the participation frequency is reduced to achieve hot switching and incremental iteration of the model.

[0037] The above formulas are all dimensionless calculations. Dimensionless calculations can be performed using various methods such as standardization, which will not be elaborated here. The formulas are derived from software simulations based on a large amount of collected data, and the preset parameters in the formulas can be set by those skilled in the art according to the actual situation.

[0038] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, ATA hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state ATA hard disk.

[0039] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0040] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0041] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0042] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0043] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0044] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable ATA hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0045] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A voice call privacy protection method integrating edge computing and federated learning, characterized in that, Includes the following steps: Step 1: Edge nodes acquire PCM audio streams from both parties in real time, perform streaming speech transcription to obtain word sequences, extract acoustic features, and output anonymous acoustic latent vectors through adversarial decoupling autoencoders. After concatenating word embeddings and latent vectors, L1 norm pruning and adding Laplace noise are performed to obtain desensitized joint features. Step 2: The edge nodes train a local model based on the desensitized joint features. After backpropagation, the contribution of each parameter is calculated using the second-order statistics of the gradient. The gradients are retained in descending order of contribution and the remaining gradients are set to zero. The retained gradients are then subjected to residual encoding, product quantization, Gaussian noise perturbation, and Paillier homomorphic encryption in sequence. An encrypted update package is generated and uploaded to the cloud. Step 3: The cloud receives the encrypted update packets and plaintext statistical features from each node, performs online clustering to divide the plaintext features into dynamic clusters after dimensionality reduction, uses Paillier additive homomorphism to perform secure weighted aggregation of the encrypted update packets within each cluster, and then recovers the cluster center gradient by decrypting through a threshold. Step 4: The cloud uses the Reptile algorithm to update the meta-basic parameters based on the gradient of the cluster center, and generates a low-rank bias matrix for the adaptive layer of each cluster. The updated meta-basic parameters and the corresponding low-rank bias matrix of the cluster are encrypted with the public key of each node and then sent down. Step 5: The edge node decrypts the received parameters using its local private key, projects the low-rank bias matrix to generate scaling and offset parameters, performs a channel-by-channel affine transformation on the adaptive layer input of the local model to complete the model update, and if the loss does not decrease after multiple consecutive rounds of validation, increases the Gaussian noise variance of the gradient upload and reduces the frequency of participation in federation.

2. The voice call privacy protection method integrating edge computing and federated learning according to claim 1, characterized in that, The training loss function of the adversarial decoupled autoencoder in step 1 includes Mel spectrum reconstruction loss, adversarial loss, and mutual information term; The adversarial loss is used to maximize the classification error of the speaker identity branch, and the mutual information term is used to minimize the dependency between the residual branch output and the speaker label. After training, the encoder weights are fixed and deployed on the edge nodes.

3. The voice call privacy protection method integrating edge computing and federated learning according to claim 1, characterized in that, The calculation of contribution in step 2 is specifically as follows: Maintain the first-moment estimate of the gradient and the diagonal approximation of the Fisher information matrix. Define the contribution as the absolute value of the gradient divided by the square root of the diagonal approximation, and then multiplied by the direction cosine of the gradient and the first-moment estimate plus a preset bias term. After calculating the contribution, multiply it by the current round decay factor, and retain the top 10% of gradient positions in descending order.

4. The voice call privacy protection method integrating edge computing and federated learning according to claim 1, characterized in that, The residual encoding in step 2 is as follows: Calculate the residual between the preserved gradient and the actual uploaded value in the previous round. If it is the first round of training, the uploaded value in the previous round is assumed to be the zero vector. The residual vector is uniformly divided into four-dimensional sub-vectors, and each sub-vector is mapped to the nearest codeword index using a shared product quantization codebook. After dequantization, Gaussian noise that satisfies local differential privacy is added, then converted to an integer and Paillier element-wise encryption is performed.

5. The voice call privacy protection method integrating edge computing and federated learning according to claim 1, characterized in that, In step 3, the cloud recovers the cluster center gradient within a secure sandbox using the threshold Paillier scheme. The private key is split into three fragments and hosted on three independent key management nodes. Each node provides partial decryption in a trusted execution environment, and the sandboxes combine them to obtain the plaintext gradient.

6. The voice call privacy protection method integrating edge computing and federated learning according to claim 1, characterized in that, In step 4, the cloud dynamically adjusts the rank of the low-rank bias matrix for each cluster: The rank is adjusted based on the magnitude of the loss reduction and the number of participating nodes: the rank is increased when the magnitude of the loss reduction is lower than the preset threshold and the number of nodes is not lower than the minimum value; otherwise, the rank is decreased if the number of nodes is not lower than the minimum value. In other cases, the original rank is maintained and the rank is limited to the preset upper and lower bounds. The rank is adjusted by a positive integer step size each time.

7. The voice call privacy protection method integrating edge computing and federated learning according to claim 1, characterized in that, In step 5, the edge nodes randomly sample desensitized features and labels from the replay buffer to evaluate the local verification loss. If the loss does not decrease for five consecutive rounds, the Gaussian noise variance is increased by 10% and the frequency of participation in federation is reduced to once every two rounds. The process is gradually restored after the accuracy recovers.

8. A system applied to the voice call privacy protection method integrating edge computing and federated learning as described in any one of claims 1-7, characterized in that, include: The edge feature desensitization module is used to access PCM audio streams in real time on the call center side. It extracts anonymous acoustic latent vectors through streaming speech transcription and adversarial decoupling autoencoder, and adds Laplacian noise after concatenating them with semantic embeddings to output desensitized joint features that satisfy differential privacy. The edge gradient upload module is used to train a local model on edge nodes based on desensitized features, calculate the contribution using second-order gradient statistics and perform structured pruning, and then package and upload the retained gradients to the cloud after product quantization, Gaussian perturbation and Paillier encryption. The cloud aggregation and meta-learning module is used to receive encrypted update packets from each node, group the nodes through clustering, perform secure weighted aggregation in the ciphertext domain, recover the cluster center gradient through threshold decryption, and update the meta-basic parameters and the low-rank bias matrix of each cluster using the Reptile algorithm. The edge model update module is used by edge nodes to decrypt the encrypted meta-model and low-rank bias matrix sent from the cloud using their local private key and perform adaptive layer modulation using the FiLM method to obtain the final local model. If the loss no longer decreases after multiple consecutive rounds of verification, then the upload noise variance is dynamically increased and the participation frequency is reduced.

Images