Data storage method and device

A data storage device and data storage technology, which is applied in the field of data transmission, can solve problems such as user inability to use, data errors, etc., and achieve the effect of avoiding complex problems and improving flexibility

Pending Publication Date: 2020-04-03
BEIJING GRIDSUM TECH CO LTD
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In view of the above problems, the purpose of the embodiments of the present invention is to provide a data storage method and device to solve the problem that the data uploaded by the crawler end is wrong and the user cannot use the data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data storage method and device
  • Data storage method and device
  • Data storage method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Apparently, the described embodiments are some, but not all, embodiments of the present invention. For those skilled in the art, other embodiments can also be obtained based on these embodiments without making creative efforts.

[0026] An embodiment of the present invention provides a data storage system, figure 1 It is a schematic diagram of the data storage system architecture in the embodiment of the present invention, see figure 1 As shown, the system includes: a crawler end 101 and a server end 102 .

[0027] Wherein, the crawler end may be a tool with a data crawling function, such as a web crawler. The server can be a server capable of receiving, storing and processing data, for example: a data receiving server, that is, a server, and the data receiving server can receive the crawled raw data transmi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data storage method and device, and aims to solve the problem that a user cannot use data due to errors of the data uploaded by a crawler end. The method comprises the stepsof obtaining a confirmation submission request, wherein the confirmation submission request is used for indicating that a crawler end crawls data required by a crawling task; in response to the confirmation submission request, reading a plurality of partial data from a first storage area, the first storage area storing data crawled by the crawler end according to the crawling task; and obtaining data required by the crawling task according to the plurality of partial data, and storing the data required by the crawling task.

Description

technical field [0001] The invention relates to the field of data transmission, in particular to a data storage method and device. Background technique [0002] A web crawler, also known as a web spider or a web robot, is a program or script that automatically grabs information on the World Wide Web according to certain rules. [0003] In the process of crawling data, the crawler crawls according to the crawling task. After crawling the data according to the crawling task, the crawler will upload the data for the user to use. However, there may be dependencies between the data required in the crawling task. The data uploaded by the crawler at one time may not be the complete data required in the crawling task, or the crawler has an error during the crawling process. The wrong data fetched is also uploaded, which will affect the user's use of data. Contents of the invention [0004] In view of the above problems, the purpose of the embodiments of the present invention is ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/958G06F16/951G06F16/953
Inventor 张鑫
Owner BEIJING GRIDSUM TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap