Data loading method and device, electronic equipment and storage medium

A data loading and data technology, applied in the fields of computer vision, deep learning, and artificial intelligence, can solve problems such as the lack of technical solutions, affecting hardware processing speed and processing accuracy, and increasing hardware costs.

Pending Publication Date: 2021-09-10
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, the model structure or model training is various, and the corresponding hardware performance will affect the processing speed and processing accuracy of the hardware. It is necessary to improve the model performance, and the model performance is related to the model training. In the case of large-scale training data, greater requirements will be placed on the storage capacity of the training machine during model training, which will increase hardware costs
In this regard, there is no effective solution in related technologies

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data loading method and device, electronic equipment and storage medium
  • Data loading method and device, electronic equipment and storage medium
  • Data loading method and device, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0046] Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0047] The term "and / or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. The term "at least one" herein means any one of a variety or any combination of at ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data loading method and device, electronic equipment and a storage medium, and relates to the technical field of computer vision and deep learning. The specific implementation scheme comprises the steps of dividing to-be-loaded data into multiple parts of data, and storing the multiple parts of data in corresponding storage nodes respectively; in response to a training operation of a neural network, dynamically loading the multiple copies of data on the storage node to a training machine; and under the condition that the hardware resources of the training machine do not reach the resource upper limit, circularly loading the multiple pieces of data until the training is finished. By adopting the method and the device, the super-large-scale training data can be automatically and efficiently processed.

Description

technical field [0001] This disclosure relates to the field of artificial intelligence. The present disclosure particularly relates to the technical fields of computer vision and deep learning, and can be specifically used in smart city and smart traffic scenarios. Background technique [0002] With the development of technology, artificial intelligence can be used to improve hardware performance, and it is applicable to a variety of application scenarios, such as hardware design involving graphic and text processing, image processing, video processing, face recognition, target positioning and other application scenarios. Using artificial intelligence technology, the trained model is deployed in the hardware to improve the processing speed and processing accuracy of the hardware. [0003] However, the model structure or model training is various, and the corresponding hardware performance will affect the processing speed and processing accuracy of the hardware. It is necess...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/71G06F16/51G06N3/04G06N3/08
CPCG06F16/71G06F16/51G06N3/04G06N3/08
Inventor 王之港王健孙昊丁二锐
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap