Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data processing method and related equipment

A technology of data processing and data processing devices, which is applied in the field of data processing, can solve problems such as inability to recover from failures, and achieve the effects of realizing failure recovery and reducing resource consumption

Pending Publication Date: 2022-01-07
AGRICULTURAL BANK OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, existing technologies cannot effectively recover from failures based on checkpoints

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and related equipment
  • Data processing method and related equipment
  • Data processing method and related equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0046] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0047] Such as figure 1 As shown, this embodiment proposes the first data processing method. The method may include the steps of:

[0048] S101. Determine the allowable time for fault recovery when the target node executes the target job;

[0049] Wherein, the target node may be a certain node for processing the target job, such as a mobile phone and a tablet computer.

[0050] Optionally, the target job may be a batch job or a non-batch job...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data processing method and related equipment, which can determine a fault recovery allowable duration when a target node executes a target job, determine a fault recovery parallelism degree matched with the fault recovery allowable duration, divide check point data into M partition data when the target node outputs the check point data, wherein m is the same as the value of the fault recovery parallelism degree, and when the target node has a fault in the process of executing the target job, read the M partition data in parallel according to the fault recovery parallelism degree so as to perform fault recovery. The tolerable time of check point write-out and recovery can be estimated according to the dependency relationship and the execution duration of the historical jobs, and then the minimum parallelism degree capable of meeting the fault recovery requirement and the job response duration is estimated, so that the resource consumption is reduced to the maximum extent, and fault recovery based on the check point data can be effectively realized.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a data processing method and related equipment. Background technique [0002] With the development of computer technology, data processing technology continues to improve. [0003] Currently, nodes can provide fault tolerance by outputting checkpoints during job processing. Specifically, when a node fails in the process of processing a job, it can perform fault recovery based on the checkpoint to restore to the state corresponding to the checkpoint, and continue processing the job from the state corresponding to the checkpoint without restarting the job , can effectively improve job processing efficiency. [0004] However, the existing technology cannot effectively perform fault recovery based on checkpoints. Contents of the invention [0005] In view of the above problems, the present invention provides a data processing method and related equipment that ove...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F11/14
CPCG06F11/1458
Inventor 谷兰芝
Owner AGRICULTURAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com