Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and device for reading training sample data, storage medium and electronic equipment

A technology of sample data and training samples, applied in the computer field, can solve problems such as unproposed solutions

Active Publication Date: 2020-12-29
TENCENT TECH (SHENZHEN) CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] For the above problems, no effective solution has been proposed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for reading training sample data, storage medium and electronic equipment
  • Method and device for reading training sample data, storage medium and electronic equipment
  • Method and device for reading training sample data, storage medium and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is an embodiment of a part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0032] It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and device for reading training sample data in a cloud technology scene, a storage medium and electronic equipment, and specifically relates to data query in a database technology scene. Among them, the method includes: reading the sample data blocks from the target data set stored in the storage space according to the order of the first data blocks, performing model training on the target model according to the read sample data blocks, and using the read samples The data blocks are cached to the target cache space; after the sample data blocks are read from the target data set according to the order of the first data blocks, the order of the first data blocks is adjusted to the order of the second data blocks; according to the order of the second data blocks, from The sample data blocks in the target data set are read in the target cache space and the storage space, and model training is performed on the target model according to the read sample data blocks in the target data set. The invention solves the technical problem that the hit rate of data reading is low.

Description

technical field [0001] The present invention relates to the field of computers, in particular to a method and device for reading training sample data, a storage medium and electronic equipment. Background technique [0002] In the data reading scenario, because the volume of the data set is often much larger than the memory, it is impossible to cache all the data sets by only using the memory as a cache. And the prior art proposes following two kinds of schemes in order to solve the above problem: [0003] Solution 1. When training a deep learning model, the volume of the data set is often much larger than the memory. Therefore, only memory is used as a cache, and all data sets cannot be cached. FSCache is a stand-alone caching system for remote file systems, which supports multi-level caching of memory and SSD. [0004] Solution 2: Memcached is a distributed caching system that also supports multi-level caching. It uses the Memcached protocol to provide caching accelerati...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F12/0866G06F12/0893
CPCG06F12/0866G06F12/0893G06F2212/1021
Inventor 薛磊阎姝含
Owner TENCENT TECH (SHENZHEN) CO LTD