A Condor Task Mapping Method for Remote Sensing Image Processing Docker Cluster

Abstract

The invention relates to a condor task mapping method of a remote sensing image processing Docker cluster, which belongs to the field of cloud computing. During the entire processing process, the association between Docker, Condor and tasks is realized by the system automatically generating a Docker universe mode task submission description file that satisfies ClassAd resource matching, and starting Condor to execute the Docker container on the host to run the remote sensing image processing algorithm. Deploy the Condor cluster on the physical machine, deploy docker-engine on the condor execution client, and build the Condor cluster environment; store the metadata of the Docker image, and store the IP address information and image name of the physical machine in MySQL; according to Condor ClassAD The task description method, the information recorded in MySQL, and the visual page provided by the front-end page allow users to form a task submission file generation wizard that meets their needs through the visual page. The invention uses a container to improve the resource utilization of remote sensing image processing nodes in a distributed environment, so that multiple remote sensing tasks can run on the same physical machine.

Description

technical field [0001] The invention relates to a condor task mapping for remote sensing image processing Docker clusters, belonging to the field of cloud computing. Background technique [0002] Docker can package remote sensing professional software, remote sensing image processing algorithms and dependent tools into a portable container to achieve fast startup of Docker clusters and convenient migration of Docker. Compared with virtual machines, Docker is characterized by fast startup and small resource usage. It can start multiple Docker containers on a physical machine or virtual machine, and run multiple remote sensing image processing algorithms on the same physical machine or virtual machine at the same time. On the other hand, multiple remote sensing image processing tasks can be easily implemented in parallel on the same physical node with resource isolation, which greatly utilizes the resources of the physical machine. However, when remote sensing users keep pack...

AI Technical Summary

Problems solved by technology

However, when remote sensing users keep packing a large number of remote sensing algorithms and their operating environments into Docker containers, causing users to manually start different Docker containers to perform hundreds, thousands or even more remote sensing tasks, manual management and configuration are almost impossible. The correspondence between thousands of Docker image names and computing nodes on ten or even hundreds of computing nodes is unacceptable to them, so they need a tool to automatically manage computing nodes and containers, and quickly deploy Their requirements are visually mapped to different Docker containers for execution

Images