A distributed multi-task video feature compression method supporting federated learning and privacy protection

Abstract

The application discloses a distributed multi-task video feature compression method supporting federated learning and privacy protection. The method comprises the following steps: initializing a front-end feature extraction network, a VQ-VAE model and a multi-task back-end network on a cloud server, and distributing parameters to each edge device; each edge device uses a local video frame to extract intermediate features through the front-end network; the intermediate features are input into the local VQ-VAE for encoding, quantization and decoding to reconstruct a feature map; task-specific training is performed on the multi-task back-end network, and task header parameters are updated; differential privacy protection is performed before uploading the parameter update; the server aggregates each device update to generate a global model, and distributes the global model to the devices for iterative training; in actual deployment, the edge device is responsible for feature compression, the cloud end is responsible for decoding and multi-task reasoning. According to the application, multi-edge device collaborative training of high-performance multi-task video feature compression can be realized on the premise of protecting data privacy.

Description

Technical Field

[0001] This invention belongs to the fields of distributed artificial intelligence, federated learning, privacy computing and computer vision, and particularly relates to the field of multi-edge device collaboration, specifically a distributed multi-task video feature compression method that supports federated learning and privacy protection. Background Technology

[0002] Multi-task video feature compression and quantization algorithms based on variational autoencoders (VAEs) for efficient network transmission typically employ a typical edge-cloud collaborative architecture. In this framework, edge devices undertake some of the front-end feature extraction and compression tasks: first, intermediate features are generated through a front-end feature extraction network, then compressed by a VQ-VAE encoder before being transmitted to the cloud server; the cloud server then uses a VQ-VAE decoder in conjunction with a back-end multi-task processing network to complete the decoding, supporting subsequent model training and multi-task inference. This approach can alleviate the computational burden on edge devices to some extent and support multi-task parallel processing.

[0003] However, this type of architecture still has significant limitations in practical applications. First, the model training process often requires concentrating large amounts of video data from different devices and scenarios onto a cloud server for unified training. This not only incurs huge network transmission and storage overhead but also poses serious risks to privacy and security. For example, in scenarios such as home security monitoring, medical image analysis, and industrial production process monitoring, video data typically contains sensitive personal or commercial information and is subject to strict privacy protection and data compliance (such as GDPR) constraints, making it impossible for it to leave the local device directly.

[0004] Secondly, different edge devices may face different usage environments and task combinations. For example, one edge device may focus on pedestrian detection and action recognition, while another may focus more on face recognition and expression analysis. This diverse range of needs means that relying solely on a single cloud-based model is insufficient to achieve optimal performance. To accommodate the differentiated task requirements of different scenarios, a feature compression model that can generalize and adapt to various environments is needed.

[0005] Federated learning offers a new solution to this problem. It allows participating devices to upload only local model parameter updates or gradient information, rather than the original data, thereby collaboratively training a higher-performing global model while protecting data privacy. This approach not only meets privacy compliance requirements but also effectively aggregates data knowledge from multiple distributions, improving the model's generalization ability.

[0006] However, how to organically combine federated learning mechanisms with complex video feature compression models (such as vector quantization variational autoencoders, VQ-VAE) and multi-task learning remains a technical challenge that urgently needs to be addressed. Summary of the Invention

[0007] The purpose of this invention is to address the deficiencies or shortcomings of the existing technologies by providing a distributed multi-task video feature compression method based on privacy protection and federated learning. This method combines a federated learning framework with vector quantization variational autoencoder (VQ-VAE) feature compression and a modular multi-task network, enabling multiple edge devices to collaboratively train a high-performance video feature compression model and a backend multi-task analysis network without leaking local data.

[0008] The technical solution to achieve the purpose of this invention is: a distributed multi-task video feature compression method that supports federated learning and privacy protection. The method adopts a distributed training architecture of federated learning, which includes a cloud server and multiple edge devices. Throughout the process, the original video data is always kept locally on the edge devices and is never uploaded to the cloud server. Only the model parameters or the gradients of parameter updates are transmitted to the cloud server for aggregation during the training process to ensure data privacy and security.

[0009] Furthermore, the method includes the following steps:

[0010] Step 1, System Initialization: On the cloud server, initialize the front-end feature extraction network (front-end model), VQ-VAE model, and multi-task back-end network (back-end model).

[0011] Step 2, Global Model Distribution: Distribute the model parameters to each edge device as the local initialization model;

[0012] Step 3, Local Feature Extraction: Each edge device inputs local video frames into the front-end feature extraction network to generate intermediate feature maps;

[0013] Step 4, VQ-VAE feature compression: The edge device inputs the intermediate feature map into the VQ-VAE encoder to generate a latent representation and completes the quantization mapping in the codebook; the latent representation is then reconstructed into a feature map by the decoder for subsequent local multi-task training;

[0014] Step 5, Multi-task backend training: The reconstructed feature maps are input locally into the multi-task backend network to perform multi-task specific training;

[0015] Step 6, Differential Privacy Protection and Upload: Before updating the local training to the cloud server, each edge device perturbs the gradient or parameter updates to protect data privacy;

[0016] Step 7, Global Model Aggregation: The cloud server collects parameter updates from various edge devices and aggregates them using a federated averaging method to generate a new global model. The aggregated global model is then distributed to various edge devices based on requirements, and the next training iteration is initiated. Steps 2 through 7 are repeated until the application requirements are met.

[0017] Step 8, Edge-Cloud Collaborative Deployment and Inference: In actual deployment, edge devices are responsible for video frame processing and feature compression, and transmit the compressed features to the cloud; the cloud receives the latent representation, reconstructs the feature map through the VQ-VAE decoder, and then the multi-task backend network completes the task inference.

[0018] Furthermore, throughout step 4, only the latent representation or model parameter updates are uploaded; the original video and intermediate feature maps do not leave the edge device.

[0019] Furthermore, in step 5, each edge device calculates the loss function based on its own set of tasks and updates the corresponding task header parameters. At the same time, the shared feature layer also participates in training, but only generates gradients or updates parameters locally.

[0020] Furthermore, in step 6, the perturbed update only includes model information and does not include the original video data or intermediate features.

[0021] Furthermore, the system initialization in step 1 specifically includes:

[0022] The server initializes a front-end feature extraction network. This is used to extract intermediate features from the original video frames; a VQ-VAE model is initialized. Including encoders decoder and learnable codes This is used to perform feature compression and reconstruction; and to initialize a multi-task backend network. A modular task head was designed, which includes a shared feature layer and multiple task heads; after initialization, the server sends the model parameters to all edge devices as the starting point for local training.

[0023] Furthermore, each edge device only needs to download and update the task header module related to its own task, without having to carry the complete task set; in step 7, for modular task headers, the task headers updated by different edge devices are only aggregated with similar tasks, while the shared feature layer is uniformly optimized across all edge devices.

[0024] Furthermore, during the local training phase in steps 2 through 7, in each round of federated learning, each edge device... The following training process is performed independently based on its own dataset:

[0025] (1) Feature extraction: Using local private video data through the front-end network Extracting intermediate features ;

[0026] (2) VQ-VAE training: intermediate features Input to local VQ-VAE model Mapped to a latent representation by the encoder The codebook searches for the closest symbol, quantizes it, and then the decoder reconstructs the feature. In this process, the training objective of VQ-VAE is to minimize the reconstruction error, codebook update error, and encoder commitment error.

[0027] (3) Backend network fine-tuning: Edge devices will reconstruct features Input to the local backend network In the process, multi-task training is performed; each edge device performs training according to the task set. Calculate the corresponding loss function; the overall multi-task loss function is defined as follows: :

[0028]

[0029] In the formula, For the task The corresponding real data labels;

[0030] By minimizing this loss function, edge devices can locally adapt to the differences in feature distribution caused by compression;

[0031] (4) Differential privacy protection: Before uploading the model update, a differential privacy protection mechanism is introduced; specifically, after the edge device generates the gradient or parameter update, noise is added to it, that is:

[0032]

[0033] In the formula, Indicates edge device Local parameter updates, This is the updated result after the perturbation; This indicates that the mean is 0 and the variance is 0. Gaussian noise; It is the covariance matrix of a multidimensional Gaussian distribution;

[0034] (5) Model update and aggregation: During the aggregation phase, the cloud server aggregates the global model after collecting a preset number of updates from edge devices;

[0035] (6) Distribution and iteration: The cloud server distributes the aggregated global model to each edge device again, and the edge devices receive it and enter a new round of local training;

[0036] By continuously repeating the iterative process of training, uploading, aggregating, and distributing described above, the convergence of the global model is eventually achieved.

[0037] Furthermore, during the VQ-VAE training process, the loss function is defined as follows: :

[0038]

[0039] In the formula, the first term The reconstruction loss is used to maintain the closeness between the reconstructed features and the original features; the second term The codebook loss is used to update the codebook vector; the third term... To account for the loss, it is used to constrain the encoder output to be close to the selected codebook. For hyperparameters, This indicates that the gradient operation has stopped.

[0040] Furthermore, in the model update and aggregation, the aggregation method uses federated averaging, with the following formula:

[0041]

[0042] In the formula, For edge devices In the Parameters are updated in the upload round. For the first The parameter update results of the round aggregation, Indicates edge device The amount of data, This represents the total amount of data across all edge devices.

[0043] Compared with the prior art, the significant advantages of this invention are:

[0044] (1) Privacy and compliance protection: Through the federated learning framework and differential privacy mechanism, the original video data is always kept locally, and only parameter updates are uploaded, thereby avoiding the leakage of privacy information and complying with data security and compliance requirements (such as GDPR). This approach not only protects personal and sensitive data, but also meets the needs of application scenarios with extremely high privacy protection requirements, such as medical care and security.

[0045] (2) High communication efficiency: Compared with directly transmitting the original video or high-dimensional feature map, this invention only needs to transmit lightweight parameter updates, which greatly reduces bandwidth usage and communication overhead. Even in low bandwidth or unstable network environments, it can ensure the normal operation of model training and inference.

[0046] (3) Strong generalization ability: By using multi-source data distribution from different edge devices for joint training, the global model can adapt to various scenarios and task combinations, and has better generalization and robustness. This enables the model to maintain high performance when facing unknown environments or new tasks.

[0047] (4) Support for task heterogeneity: Through modular task header design, edge devices can select and update task modules as needed, achieving the sharing of general features while meeting personalized application requirements. This flexibility effectively reduces the computing and storage burden on the device side and improves the adaptability and scalability of the overall system.

[0048] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating a distributed multi-task video feature compression method that supports federated learning and privacy protection in one embodiment.

[0050] Figure 2 This is an overall architecture diagram of a distributed multi-task video feature compression method that supports federated learning and privacy protection in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] It should be noted that if the embodiments of the present invention involve descriptions such as "first" and "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0053] In one embodiment, combined Figure 1 and Figure 2This paper presents a distributed multi-task video feature compression method that supports federated learning and privacy protection. The method adopts a distributed training architecture of federated learning, which includes a cloud server and multiple edge devices. Throughout the process, the original video data is always kept locally on the edge devices and is never uploaded to the cloud server. Only the model parameters or the gradients of parameter updates are transmitted to the cloud server for aggregation during the training process to ensure data privacy and security.

[0054] Furthermore, in one embodiment, the method includes the following steps:

[0055] Step 1, System Initialization: On the cloud server, initialize the front-end feature extraction network (i.e., the front-end model), the VQ-VAE model (including encoder, decoder, and learnable codebook), and the multi-task back-end network (including shared feature layer and task head module) (i.e., the back-end model).

[0056] Step 2, Global Model Distribution: The model parameters are distributed to each edge device as the local initialization model; this stage only involves the transmission of model parameters.

[0057] Step 3, Local Feature Extraction: Each edge device inputs local video frames into the front-end feature extraction network to generate intermediate feature maps. This step is completed locally, and the original video frames are not uploaded to the cloud server to ensure data privacy. The intermediate feature maps will be used as input to VQ-VAE to provide data for subsequent compression and reconstruction.

[0058] Step 4, VQ-VAE feature compression: The edge device inputs the intermediate feature map into the VQ-VAE encoder to generate a latent representation and completes the quantization mapping in the codebook; the latent representation is then reconstructed into a feature map by the decoder for subsequent local multi-task training; throughout the process, only the latent representation or model parameter updates may be uploaded, and the original video and intermediate feature map do not leave the device.

[0059] Step 5, Multi-task backend training: The reconstructed feature maps are input locally into the multi-task backend network to perform multi-task specific training;

[0060] Step 6, Differential Privacy Protection and Upload: Before updating the local training to the cloud server, each edge device perturbs the gradient or parameter updates to protect data privacy;

[0061] Step 7, Global Model Aggregation: The cloud server collects parameter updates from various edge devices and aggregates them using a federated averaging method to generate a new global model. The aggregated global model is then distributed to various edge devices based on requirements, and the next training iteration is initiated. Steps 2 through 7 are repeated until the application requirements are met.

[0062] Step 8, Edge-Cloud Collaborative Deployment and Inference: In actual deployment, edge devices are responsible for video frame processing and feature compression, and transmit the compressed features to the cloud; the cloud receives the latent representation, reconstructs the feature map through the VQ-VAE decoder, and then the multi-task backend network completes task inference. This deployment method ensures that the original video data does not leave the edge device, while supporting multi-task analysis and heterogeneous task scenarios.

[0063] Furthermore, in one embodiment, throughout step 4, only the latent representation or model parameter updates are uploaded; the original video and intermediate feature maps do not leave the edge device.

[0064] Furthermore, in one embodiment, in step 5, each edge device calculates a loss function based on its own set of tasks and updates the corresponding task header parameters. Simultaneously, the shared feature layer also participates in training, but only generates gradients or parameter updates locally. In this way, each device can optimize for its own task while maintaining the consistency of the shared features.

[0065] Furthermore, in one embodiment, in step 6, the perturbed update only contains model information and does not contain the original video data or intermediate features, thereby preventing cloud servers or third parties from inferring sensitive information.

[0066] Furthermore, in one embodiment, the system initialization in step 1 specifically includes:

[0067] The server initializes a front-end feature extraction network. This is used to extract intermediate features from the original video frames; a VQ-VAE model is initialized. Including encoders decoder and learnable codes This is used to perform feature compression and reconstruction; and to initialize a multi-task backend network. A modular task head was designed, which includes a shared feature layer and multiple task heads; after initialization, the server sends the model parameters to all edge devices as the starting point for local training.

[0068] Furthermore, in one embodiment, each edge device only needs to download and update the task header module related to its own task, without having to carry the complete task set; in step 7, for modular task headers, the task headers updated by different edge devices are only aggregated with similar tasks, while the shared feature layer is uniformly optimized across all edge devices.

[0069] Furthermore, in one embodiment, during the local training phase of steps 2 through 7, in each round of federated learning, each edge device... The following training process is performed independently based on its own dataset:

[0070] (1) Feature extraction: Using local private video data through the front-end network Extracting intermediate features ;

[0071] (2) VQ-VAE training: intermediate features Input to local VQ-VAE model Mapped to a latent representation by the encoder The codebook searches for the closest symbol, quantizes it, and then the decoder reconstructs the feature. In this process, the training objective of VQ-VAE is to minimize the reconstruction error, codebook update error, and encoder commitment error.

[0072] (3) Backend network fine-tuning: Edge devices will reconstruct features Input to the local backend network In the process, multi-task training is performed; each edge device performs training according to the task set. Calculate the corresponding loss function; the overall multi-task loss function is defined as follows: :

[0073]

[0074] In the formula, For the task The corresponding real data labels;

[0075] By minimizing this loss function, edge devices can locally adapt to the differences in feature distribution caused by compression;

[0076] (4) Differential privacy protection: Before uploading the model update, a differential privacy protection mechanism is introduced; specifically, after the edge device generates the gradient or parameter update, noise is added to it, that is:

[0077]

[0078] In the formula, Indicates edge device Local parameter updates, This is the updated result after the perturbation; This indicates that the mean is 0 and the variance is 0. Gaussian noise; It is the covariance matrix of a multidimensional Gaussian distribution;

[0079] (5) Model update and aggregation: During the aggregation phase, the cloud server aggregates the global model after collecting a preset number of updates from edge devices;

[0080] (6) Distribution and iteration: The cloud server distributes the aggregated global model to each edge device again, and the edge devices receive it and enter a new round of local training;

[0081] By continuously repeating the iterative process of training, uploading, aggregating, and distributing described above, the convergence of the global model is eventually achieved.

[0082] Preferably, in some embodiments, during the VQ-VAE training process, the loss function is defined as :

[0083]

[0084] In the formula, the first term The reconstruction loss is used to maintain the closeness between the reconstructed features and the original features; the second term The codebook loss is used to update the codebook vector; the third term... To account for the loss, it is used to constrain the encoder output to be close to the selected codebook. For hyperparameters, This indicates that the gradient operation has stopped.

[0085] Preferably, in some embodiments, the aggregation method used in the model update and aggregation is federated average, with the formula:

[0086]

[0087] In the formula, For edge devices In the Parameters are updated in the upload round. For the first The parameter update results of the round aggregation, Indicates edge device The amount of data, This represents the total amount of data across all edge devices.

[0088] In one embodiment, a distributed multi-task video feature compression system supporting federated learning and privacy protection is provided, the system comprising:

[0089] The first module is used to implement system initialization: on the cloud server side, the front-end feature extraction network, i.e., the front-end model, the VQ-VAE model, and the multi-task back-end network, i.e., the back-end model, are initialized.

[0090] The second module is used to implement global model distribution: the model parameters are distributed to each edge device as the local initialization model;

[0091] The third module is used to implement local feature extraction: each edge device inputs local video frames into the front-end feature extraction network to generate intermediate feature maps;

[0092] The fourth module is used to implement VQ-VAE feature compression: the edge device inputs the intermediate feature map into the VQ-VAE encoder to generate a latent representation and completes the quantization mapping in the codebook; the latent representation is then reconstructed into a feature map by the decoder for subsequent local multi-task training;

[0093] The fifth module is used to implement multi-task backend training: the reconstructed feature maps are input locally into the multi-task backend network to perform multi-task specific training;

[0094] The sixth module is used to implement differential privacy protection and uploading: Before updating local training to the cloud server, each edge device perturbs the gradient or parameter updates to protect data privacy;

[0095] The seventh module is used to realize global model aggregation: the cloud server collects parameter updates from various edge devices, aggregates them through the federated averaging method, and generates a new global model; the aggregated global model determines whether to continue to be distributed to various edge devices according to the needs, and enters the next round of training iteration, repeating steps 2 to 7 until the application needs are met.

[0096] The eighth module is used to realize edge-cloud collaborative deployment and inference: In actual deployment, the edge device is responsible for video frame processing and feature compression, and transmits the compressed features to the cloud; the cloud receives the latent representation, reconstructs the feature map through the VQ-VAE decoder, and then the multi-task backend network completes the task inference.

[0097] Specific limitations regarding the distributed multi-task video feature compression system supporting federated learning and privacy protection can be found in the limitations of the distributed multi-task video feature compression method supporting federated learning and privacy protection described above, and will not be repeated here. Each module in the aforementioned distributed multi-task video feature compression system supporting federated learning and privacy protection can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0098] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements:

[0099] Step 1, System Initialization: On the cloud server, initialize the front-end feature extraction network (front-end model), VQ-VAE model, and multi-task back-end network (back-end model).

[0100] Step 2, Global Model Distribution: Distribute the model parameters to each edge device as the local initialization model;

[0101] Step 3, Local Feature Extraction: Each edge device inputs local video frames into the front-end feature extraction network to generate intermediate feature maps;

[0102] Step 4, VQ-VAE feature compression: The edge device inputs the intermediate feature map into the VQ-VAE encoder to generate a latent representation and completes the quantization mapping in the codebook; the latent representation is then reconstructed into a feature map by the decoder for subsequent local multi-task training;

[0103] Step 5, Multi-task backend training: The reconstructed feature maps are input locally into the multi-task backend network to perform multi-task specific training;

[0104] Step 6, Differential Privacy Protection and Upload: Before updating the local training to the cloud server, each edge device perturbs the gradient or parameter updates to protect data privacy;

[0105] Step 7, Global Model Aggregation: The cloud server collects parameter updates from various edge devices and aggregates them using a federated averaging method to generate a new global model. The aggregated global model is then distributed to various edge devices based on requirements, and the next training iteration is initiated. Steps 2 through 7 are repeated until the application requirements are met.

[0106] Step 8, Edge-Cloud Collaborative Deployment and Inference: In actual deployment, edge devices are responsible for video frame processing and feature compression, and transmit the compressed features to the cloud; the cloud receives the latent representation, reconstructs the feature map through the VQ-VAE decoder, and then the multi-task backend network completes the task inference.

[0107] For specific limitations on each step, please refer to the limitations on distributed multi-task video feature compression methods that support federated learning and privacy protection mentioned above, which will not be repeated here.

[0108] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being implemented when executed by a processor:

[0109] Step 1, System Initialization: On the cloud server, initialize the front-end feature extraction network (front-end model), VQ-VAE model, and multi-task back-end network (back-end model).

[0110] Step 2, Global Model Distribution: Distribute the model parameters to each edge device as the local initialization model;

[0111] Step 3, Local Feature Extraction: Each edge device inputs local video frames into the front-end feature extraction network to generate intermediate feature maps;

[0112] Step 4, VQ-VAE feature compression: The edge device inputs the intermediate feature map into the VQ-VAE encoder to generate a latent representation and completes the quantization mapping in the codebook; the latent representation is then reconstructed into a feature map by the decoder for subsequent local multi-task training;

[0113] Step 5, Multi-task backend training: The reconstructed feature maps are input locally into the multi-task backend network to perform multi-task specific training;

[0114] Step 6, Differential Privacy Protection and Upload: Before updating the local training to the cloud server, each edge device perturbs the gradient or parameter updates to protect data privacy;

[0115] Step 7, Global Model Aggregation: The cloud server collects parameter updates from various edge devices and aggregates them using a federated averaging method to generate a new global model. The aggregated global model is then distributed to various edge devices based on requirements, and the next training iteration is initiated. Steps 2 through 7 are repeated until the application requirements are met.

[0116] Step 8, Edge-Cloud Collaborative Deployment and Inference: In actual deployment, edge devices are responsible for video frame processing and feature compression, and transmit the compressed features to the cloud; the cloud receives the latent representation, reconstructs the feature map through the VQ-VAE decoder, and then the multi-task backend network completes the task inference.

[0117] For specific limitations on each step, please refer to the limitations on distributed multi-task video feature compression methods that support federated learning and privacy protection mentioned above, which will not be repeated here.

[0118] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A distributed multi-task video feature compression method supporting federated learning and privacy protection, characterized in that, The method employs a distributed training architecture of federated learning, which includes a cloud server and multiple edge devices. Throughout the process, the original video data remains locally on the edge devices and is never uploaded to the cloud server. Only the model parameters or the gradients of parameter updates are transmitted to the cloud server for aggregation during training to ensure data privacy and security. The method includes the following steps: Step 1, System Initialization: On the cloud server, initialize the front-end feature extraction network (front-end model), VQ-VAE model, and multi-task back-end network (back-end model). Step 2, Global Model Distribution: Distribute the model parameters to each edge device as the local initialization model; Step 3, Local Feature Extraction: Each edge device inputs local video frames into the front-end feature extraction network to generate intermediate feature maps; Step 4, VQ-VAE feature compression: The edge device inputs the intermediate feature map into the VQ-VAE encoder to generate a latent representation and completes the quantization mapping in the codebook; the latent representation is then reconstructed into a feature map by the decoder for subsequent local multi-task training; Step 5, Multi-task backend training: The reconstructed feature maps are input locally into the multi-task backend network to perform multi-task specific training; Step 6, Differential Privacy Protection and Upload: Before updating the local training to the cloud server, each edge device perturbs the gradient or parameter updates to protect data privacy; Step 7, Global Model Aggregation: The cloud server collects parameter updates from various edge devices and aggregates them using a federated averaging method to generate a new global model. The aggregated global model is then distributed to various edge devices based on requirements, and the next training iteration is initiated. Steps 2 through 7 are repeated until the application requirements are met. Step 8, Edge-Cloud Collaborative Deployment and Inference: In actual deployment, edge devices are responsible for video frame processing and feature compression, and transmit the compressed features to the cloud; the cloud receives the latent representation, reconstructs the feature map through the VQ-VAE decoder, and then the multi-task backend network completes the task inference. During the local training phase in steps 2 through 7, in each round of federated learning, each edge device... The following training process is performed independently based on its own dataset: (1) Feature extraction: Using local private video data through the front-end network Extracting intermediate features ; (2) VQ-VAE training: intermediate features Input to local VQ-VAE model Mapped to a latent representation by the encoder The codebook searches for the closest symbol, quantizes it, and then the decoder reconstructs the feature. In this process, the training objective of VQ-VAE is to minimize the reconstruction error, codebook update error, and encoder commitment error. (3) Backend network fine-tuning: Edge devices will reconstruct features Input to the local backend network In the process, multi-task training is performed; each edge device performs training according to the task set. Calculate the corresponding loss function; the overall multi-task loss function is defined as follows: : ； In the formula, For the task The corresponding real data labels; By minimizing this loss function, edge devices can locally adapt to the differences in feature distribution caused by compression; (4) Differential privacy protection: Before uploading model updates, a differential privacy protection mechanism is introduced; specifically, after the edge device generates gradients or updates parameters, noise is added to them, i.e.: ； In the formula, Indicates edge device Local parameter updates, This is the updated result after the perturbation; This indicates that the mean is 0 and the variance is 0. Gaussian noise; It is the identity matrix. It is the covariance matrix of a multidimensional Gaussian distribution; (5) Model update and aggregation: During the aggregation phase, the cloud server aggregates the global model after collecting a preset number of updates from edge devices; (6) Distribution and iteration: The cloud server distributes the aggregated global model to each edge device again, and the edge devices receive it and enter a new round of local training; By continuously repeating the iterative process of training, uploading, aggregating, and distributing described above, the convergence of the global model is eventually achieved.

2. The distributed multi-task video feature compression method supporting federated learning and privacy protection according to claim 1, characterized in that, Throughout step 4, only the latent representation or model parameter updates are uploaded; the original video and intermediate feature maps do not leave the edge device.

3. The distributed multi-task video feature compression method supporting federated learning and privacy protection according to claim 1, characterized in that, In step 5, each edge device calculates the loss function based on its own set of tasks and updates the corresponding task header parameters. At the same time, the shared feature layer also participates in training, but only generates gradients or updates parameters locally.

4. The distributed multi-task video feature compression method supporting federated learning and privacy protection according to claim 1, characterized in that, In step 6, the perturbed update only contains model information and does not include the original video data or intermediate features.

5. The distributed multi-task video feature compression method supporting federated learning and privacy protection according to claim 1, characterized in that, Step 1, system initialization, specifically includes: The server initializes a front-end feature extraction network. This is used to extract intermediate features from the original video frames; a VQ-VAE model is initialized. Including encoders decoder and learnable codes This is used to perform feature compression and reconstruction; and to initialize a multi-task backend network. A modular task head was designed, which includes a shared feature layer and multiple task heads; after initialization, the server sends the model parameters to all edge devices as the starting point for local training.

6. The distributed multi-task video feature compression method supporting federated learning and privacy protection according to claim 5, characterized in that, Each edge device only needs to download and update the task header module related to its own task, without having to carry the complete task set; in step 7, for modular task headers, the task headers updated by different edge devices are only aggregated with similar tasks, while the shared feature layer is uniformly optimized across all edge devices.

7. The distributed multi-task video feature compression method supporting federated learning and privacy protection according to claim 6, characterized in that, During the VQ-VAE training process, the loss function is defined as follows: : ； In the formula, the first term The reconstruction loss is used to maintain the closeness between the reconstructed features and the original features; the second term The codebook loss is used to update the codebook vector; the third term... To account for the loss, it is used to constrain the encoder output to be close to the selected codebook. For hyperparameters, This indicates that the gradient operation has stopped.

8. The distributed multi-task video feature compression method supporting federated learning and privacy protection according to claim 6, characterized in that, In the model update and aggregation, the aggregation method uses federated average, and the formula is as follows: ； In the formula, For edge devices The parameter update uploaded in round t+1 This is the parameter update result for the (t+1)th round of aggregation. Indicates edge device The amount of data, This represents the total amount of data across all edge devices.

9. A distributed multi-task video feature compression system based on the method of any one of claims 1 to 8, characterized in that, The system is used to implement the steps of the method according to any one of claims 1 to 8.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 8.

Images