Data processing method and device, equipment and storage medium

A data processing and data technology, applied in the computer field, can solve problems that affect the progress of federated learning and low processing efficiency of federated learning tasks, and achieve the effects of accelerating learning progress, reducing computing pressure, and improving processing efficiency

Pending Publication Date: 2022-04-22
WEBANK (CHINA)
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The main purpose of the present invention is to provide a data processing method, device, device and storage medium, aiming to solve the problem of low processing efficiency of federated learning tasks and affecting the progress of federated learning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device, equipment and storage medium
  • Data processing method and device, equipment and storage medium
  • Data processing method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0046] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0047] At present, in the process of federated learning, multiple federated learning tasks may be involved (for example, by changing parameters to train multiple models at the same time, the training of each model can be called a federated learning task), the central processing unit Federated learning is finally realized by processing multiple federated learning tasks. However, when the central processing unit processes each federa...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a first data set corresponding to a first operation type from data corresponding to at least one federated learning task, and taking data except the data in the first data set as a second data set, aggregating the first data sets corresponding to the federated learning tasks, performing first operation processing on the aggregated first data sets in batches by calling a preset asynchronous channel to obtain a first operation result set, and performing non-first operation processing on the second data set to obtain a second operation result set, and returning the first operation result set and the second operation result set to the corresponding federated learning tasks to be called by the federated learning tasks so as to realize the federated learning tasks. According to the invention, the processing efficiency of the federal learning task is improved, and the learning progress of federal learning is accelerated.

Description

technical field [0001] The present invention relates to the field of computers, in particular to a data processing method, device, equipment and storage medium. Background technique [0002] With the development of computer technology, federated learning can effectively solve the problem of data islands, allowing participants to jointly model on the basis of not sharing data, which can technically break data islands and realize AI (Artificial Intelligence, Artificial Intelligence) collaboration Advantages are more and more popular with users. [0003] In the process of federated learning, multiple tasks may be involved, and the central processor finally realizes federated learning by processing multiple tasks. [0004] However, when the central processing unit processes each task, it performs calculations in series according to the preset execution order of the tasks. The central processing unit not only needs to coordinate the tasks, but also needs to calculate the data of...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N20/00
CPCG06N20/00
Inventor 黄启军陈瑞钦陈振南卓本刚陈天健
Owner WEBANK (CHINA)
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap