Distributed data processing method and device, computer equipment and storage medium

A technology of distributed data and processing methods, which is applied in the fields of electrical digital data processing, computer security devices, computing, etc., and can solve problems affecting the integration effect of federated learning models

Active Publication Date: 2021-02-05
TENCENT TECH (SHENZHEN) CO LTD
View PDF4 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In the above technical solution, in order to protect the security of the training data, the model gradient needs to be encrypted, which leads to the central node device can only use the security aggregation algorithm for model fusion, which affects the model integration effect of federated learning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed data processing method and device, computer equipment and storage medium
  • Distributed data processing method and device, computer equipment and storage medium
  • Distributed data processing method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0076] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

[0077] It should be understood that "several" mentioned herein refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and / or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. The character " / " gener...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a distributed data processing method and device, computer equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the steps that model training information sent by at least two edge node devices is acquired, wherein the model training information is transmitted in a plaintext form, and the model training information is obtained by training a sub-model by the edge node equipment in a differential privacy mode; on the basis of the model training information sent by the at least two pieces of edge node equipmentrespectively, sub-models obtained by training the at least two pieces of edge node equipment respectively are obtained; and on the basis of a specified model integration strategy, model integration iscarried out on the sub-models obtained by training the at least two pieces of edge node equipment to obtain a global model. By means of the scheme, on the premise that data privacy safety is guaranteed, the model integration mode is expanded, and the model integration effect is improved.

Description

technical field [0001] The embodiments of the present application relate to the technical field of artificial intelligence, and in particular to a distributed data processing method, device, computer equipment, and storage medium. Background technique [0002] With the continuous development of artificial intelligence and the continuous improvement of user privacy requirements, the application of machine learning model training based on distributed systems is becoming more and more extensive. [0003] Federated learning is a machine learning method for distributed systems based on cloud technology. In the federated learning architecture, it includes a central node device and multiple edge node devices, and each edge node device stores its own training data locally. Federated learning includes horizontal federated learning. Horizontal federated learning is to obtain respective model gradients based on local training data in multiple edge node devices, encrypt each model grad...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F21/62G06N20/00
CPCG06F21/6245G06N20/00G06N3/098G06N3/063G06N3/09G06F21/64G06N20/20G06F21/602
Inventor 程勇陶阳宇
Owner TENCENT TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap