A Batch Job-Resource Matching Method and System with Security Constraints
A security constraint and resource matching technology, applied in the field of distributed parallel computing, can solve the problem that the job scheduling method cannot well adapt to security constraints, batch processing, batch job scheduling, etc., to achieve a simple and direct strategy, faster completion time, and good practicality sexual effect
Inactive Publication Date: 2018-11-09
XIANGTAN UNIV
View PDF1 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0006] In summary, the existing job scheduling methods cannot be well adapted to batch job scheduling problems with security constraints in practical applications.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0029] Below in conjunction with specific embodiment, further illustrate the present invention, should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention, after having read the present invention, those skilled in the art will understand various equivalent forms of the present invention All modifications fall within the scope defined by the appended claims of the present application.
[0030] A traditional MapReduce system includes control nodes and data nodes, such as figure 1 As shown, the implementation system of the batch job-resource matching method with security constraints provided by the present invention adds a downtime detection node. The batch job-resource matching system with security constraints includes control nodes, data nodes and downtime detection nodes.
[0031] Control node:
[0032] Control node: receives jobs from users and receives security constraints a...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a batch job-resource matching method and system with a security constraint. A security constraint and load balance-based batch job and resource matching method is adopted; the batch job-resource matching method comprises the specific steps of (A) configuring the security at a preprocessing stage; (B) mapping a batch job set to an appropriate resource of meeting the security constraint and a load balance security constraint, sorting jobs on various nodes and sequentially executing the jobs; and (C) dynamically and repeatedly detecting a downtime fault and redistributing the jobs on a fault machine according to the security constraint and the load balance security constraint. The method disclosed by the invention is especially suitable for batch job scheduling applications, such as MapReduce-based police data job scheduling, traffic monitoring job scheduling and the like, with relatively high data security. Secure access of data can be ensured on the basis of obtaining relatively short batch job completion time.
Description
technical field [0001] The present invention relates to a batch job-resource matching method and system with safety constraints, especially suitable for batch job scheduling applications with high data security, such as police data job scheduling based on MapReduce, traffic monitoring job scheduling, etc. It belongs to the field of distributed parallel computing. Background technique [0002] As one of the core components of Hadoop, the MapReduce framework is widely used. During data processing, MapReduce divides large tasks into small calculation jobs. These jobs are typical batch jobs, including Map jobs and Reduce jobs. When a batch of tasks arrives, MapReduce divides them into many Map and Reduce jobs. At this time, different job scheduling strategies have different effects. The batch job scheduling strategy has a great impact on the resource utilization of the cluster and the execution efficiency of the system. influences. [0003] Existing Hadoop schedulers mainly i...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4881G06F9/505G06F2209/484G06F2209/502
Inventor 李逸博
Owner XIANGTAN UNIV