A remote sensing big data integration online computing system based on a distributed container cluster and a method thereof

Abstract

The application provides a remote sensing big data integrated online computing system based on a distributed container cluster and a method thereof, and belongs to the technical field of remote sensing big data processing systems, and comprises a management layer, a container arrangement layer, a programming computing layer and a data storage access layer, the data storage access layer is connected with the programming computing layer, and the container arrangement layer is connected with the management layer and the programming computing layer respectively. The application has extremely high system integration and access efficiency, realizes deep integration of GPU computing power resources and TB-level large-scale remote sensing original data sets, replaces the traditional data download mode through a NFS proxy re-export direct mounting mechanism, solves the delay problem of data pulling, and improves the data access efficiency by more than 10 times.

Description

Technical Field

[0001] This invention relates to the field of remote sensing big data integration online computing system and method based on distributed container clusters. Background Technology

[0002] With the dense deployment of high-resolution satellite constellations, remote sensing image data is characterized by massive volume, multiple sources, and high frequency. Traditional stand-alone processing models face bottlenecks such as limited storage space, restricted I / O throughput, and lengthy processing cycles when dealing with petabyte-scale data. Modern remote sensing big data processing aims to achieve automated image correction, feature extraction, and change detection through a combination of streaming and batch processing. However, the transmission latency between different storage nodes and how to achieve efficient and rapid indexing and streaming loading for unstructured remote sensing tiles remain the core challenges for achieving real-time processing in this field.

[0003] Distributed computing, through the collaborative work of multiple nodes, decomposes complex computational tasks into multiple subtasks for parallel execution, serving as the cornerstone for solving large-scale scientific computing problems. In remote sensing deep learning applications, distributed architectures can effectively schedule heterogeneous computing resources within a cluster (such as multi-GPU clusters), significantly shortening model training and large-scale image inference time. However, in multi-user concurrent environments, how to achieve dynamic and elastic allocation of computing resources, load balancing between tasks, and optimization of communication efficiency during multi-node parallelism directly affects the overall throughput performance of the cluster, making it a key research focus for distributed scheduling mechanisms.

[0004] Container cloud platforms leverage container virtualization technology to decouple applications from underlying hardware, providing developers with a lightweight, standardized, and portable runtime environment. Through a container orchestration engine, the system can automatically deploy, scale, and self-heal services. In multi-user shared computing centers, the core control of container cloud platforms lies in fine-grained resource isolation and security management. A key challenge in container cloud control technology is how to dynamically configure environments, inject environment variables, and mount privileged storage for specific deep learning images, while ensuring the independence of user environments and the security of the host machine.

[0005] Geographic big data analytics emphasizes extracting spatiotemporal evolution patterns from massive spatial coordinates and time-series data. With the integration of WebGIS and online programming environments, interactive and collaborative cloud-based analytics are gradually becoming mainstream. This model requires systems to directly access large-scale heterogeneous datasets (such as vector, raster, and point cloud datasets) and support complex spatiotemporal operators and interactive visualization. However, achieving a direct sharing and access mechanism in the cloud environment where "data is fixed and algorithms converge with data," and realizing efficient online interactive analysis while ensuring the security of original data access, are important development directions in the field of geographic information science.

[0006] With the dense deployment of high-resolution satellite constellations, remote sensing image data exhibits characteristics of massive quantity, multi-source, and high frequency. Traditional single-machine processing mode faces bottlenecks such as insufficient storage space, limited I / O throughput, and lengthy processing cycles when dealing with PB-level data. Currently, this field faces the following core challenges: (1) Distributed computing and resource scheduling: In a multi-user concurrent environment, how to achieve dynamic elastic allocation of computing resources, load balancing between tasks, and optimization of communication efficiency when multiple nodes are in parallel directly affects the throughput performance of the cluster; (2) Environment isolation and underlying security: Container cloud platforms need to perform dynamic environment configuration and privileged storage mounting for specific deep learning images, while ensuring the independence of user environments and the security of host machines when multiple users share computing centers; (3) Geospatial data sharing: Interactive cloud analysis requires the system to be able to directly access large-scale heterogeneous datasets; how to achieve a direct sharing access mechanism that fixes data and aligns algorithms with data while ensuring the security of the original data is an urgent problem to be solved; (4) Complex environment dependency conflicts: There are serious conflicts between different algorithm environments, and nested storage mounting in a distributed environment can easily lead to deadlock and other technical problems. Summary of the Invention

[0007] The purpose of this invention is to provide an integrated online computing system and method for remote sensing big data based on a distributed container cluster, which solves the technical problems in the existing remote sensing big data processing flow, such as high cost of cross-node transmission of massive amounts of data, long time consumption of full download, high risk of data security management, and disconnect between algorithm and data scheduling.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] An integrated online computing system for remote sensing big data based on a distributed container cluster includes a management layer, a container orchestration layer, a programming computing layer, and a data storage and access layer. The data storage and access layer is connected to the programming computing layer, and the container orchestration layer is connected to both the management layer and the programming computing layer.

[0010] The management system is built on JupyterHub to provide users with a visual web interface for operation and access, while also handling user authentication, access control, and receiving user requests for computing resources.

[0011] The container orchestration layer is built on a Docker Swarm cluster and is used to receive instructions from the management layer, complete the full lifecycle management of computing containers, schedule and allocate computing tasks, and limit the amount of computing resources used by each task.

[0012] The programming computing layer includes several worker nodes configured with GPU hardware acceleration units, which are used to run user computing containers and perform specific remote sensing data processing, analysis, and deep learning model training tasks.

[0013] The data storage access layer is built on NFS distributed storage and is used to provide three types of storage space.

[0014] Furthermore, the three types of storage space include a private workspace dedicated to each user, a public document space accessible to all users, and a dataset storage pool for storing massive amounts of remote sensing image data.

[0015] Furthermore, the data storage access layer is equipped with an NFS proxy re-export mechanism. The specific implementation process is as follows: First, a dedicated NFS proxy node is deployed on the cluster intranet. Through this proxy node, a one-time full mount of the remote sensing dataset in the remote storage server is completed. Then, in the NFS service configuration file of the proxy node, a fixed and unique fsid file system identifier is configured for the remote sensing dataset directory to be shared. At the same time, crossmnt cross file system access permissions are enabled for the corresponding shared directory. Finally, through the NFS proxy node, the remote remote sensing dataset that has been mounted is re-shared with read-only access permissions to all worker nodes of the programming computing layer, realizing secure read-only sharing of remote sensing data and second-level batch mounting of the entire cluster.

[0016] Furthermore, the fsid file system identifier is used to avoid file system ID drift and recognition conflicts during cross-node mounting, ensuring the long-term stability of the mount link;

[0017] crossmnt cross-file system access permissions are used to support direct, penetrating access to nested hierarchical storage directories without needing to configure sharing rules separately for each level of subdirectory, while avoiding access deadlock issues in nested mount scenarios.

[0018] Furthermore, by using NFS proxy nodes for unified relay, direct access to remote source storage by programming computing layer worker nodes is completely avoided, reducing the security risks of core remote sensing data being tampered with or leaked. At the same time, the cross-network segment mounting time of all cluster nodes is reduced from minutes to seconds, fundamentally solving the industry pain point of batch mounting timeout.

[0019] Furthermore, there are technical issues related to the timeout of batch mounting of remote storage directly to the programming computing layer worker nodes, restricted access to nested hierarchical directories, and the data security risks arising from the direct exposure of core source storage.

[0020] A method for an integrated online computing system for remote sensing big data based on a distributed container cluster, the method comprising the following steps:

[0021] Step 1: Manually start the base container image on the worker node of the programming computing layer, enter the container to complete the runtime environment configuration, installation of dependent components and remote sensing processing tools;

[0022] Step 2: Users log in through the management layer and issue a request. The management layer then passes the request to the container orchestration layer, which schedules the computing tasks.

[0023] Step 3: After the computing container starts up, it directly accesses the remote sensing dataset shared by the NFS proxy node of the data storage access layer through the pre-configured mount path, and performs online streaming reading and analysis processing of the raw remote sensing data. There is no need to download the remote sensing data to the container or user terminal throughout the process.

[0024] Step 4: Display interactive feedback to users via the web interface, allowing them to view the relevant attribute information of the remote sensing data captured by the satellite and preprocess the remote sensing data;

[0025] Step 5: Users use the management layer to call the pre-installed deep learning model in the system to analyze the pre-processed remote sensing data and uncover the hidden information in the remote sensing data;

[0026] Step 6: Automatically store the results of data computation and processing online on NAS, and define automatic archiving logic for the computation results;

[0027] Step 7: Set up independent image platform control and exception handling, link the management layer with the container orchestration layer, capture the CPU, memory usage and running status signals of each computing container in real time, and build an exception handling mechanism to realize exception early warning;

[0028] Step 8: Independent image exit, data saving and automatic backup to prevent data loss due to single point of failure in the storage layer.

[0029] Furthermore, in step 1, after verifying that the environment is available, the currently available container environment is saved as a fixed snapshot image using the docker commit command. This image is then uploaded to the cluster's private image repository and simultaneously updated to the available image list in the management layer for users to select and use.

[0030] In step 2, after logging in through the web interface of the management layer, the user selects the required computing node specifications, solidifies the snapshot image, and submits a computing resource application. The management layer triggers the pre-start hook module to complete the preprocessing of automatic image path matching, startup command injection, and computing resource reservation. After the preprocessing is completed, the management layer sends the task scheduling instruction to the container orchestration layer, which schedules the computing tasks to the corresponding worker nodes, creates and starts the computing container, and completes the dynamic allocation and isolation of resources.

[0031] Furthermore, in step 4, after the computation task begins, the user prepares for remote sensing data processing interactive programming and data analysis based on the JupyterLab environment within the container. Interactive feedback is displayed to the user through the web interface, allowing the user to view and export data online, thus achieving preprocessing before remote sensing analysis.

[0032] In step 5, users enter a dedicated computing container environment via a web URL provided by the management layer. Using the built-in JupyterLab interactive programming interface, they directly call the pre-installed GDAL, PDAL, and deep learning frameworks. Based on the read-only mount path established by the data storage access layer, the analysis program directly streams the raw remote sensing images from the remote NFS proxy pool in a zero-download manner, performs spatiotemporal analysis on the raw remote sensing images, and renders the intermediate results in real time through the built-in Leaflet or Plotly components, achieving high-precision geospatial interactive display on the browser.

[0033] Furthermore, in step 6, a read-write separation persistence mechanism is adopted. All source code written by the user is forcibly stored in the mounted user private workspace directory and synchronized in real time with the backend highly reliable NFS distributed storage to ensure that the code logic remains safe when the container is abnormally destroyed. Automatic archiving logic is defined for the computing results. After the remote sensing products, statistical reports and training models generated by the user are written to the specified output directory, they are persisted to the private storage space through the backend data channel, and a storage index is automatically generated according to the task ID to realize the automated online storage and version management of scientific research results.

[0034] In step 7, users enable the management layer and container orchestration layer to work together by setting up full lifecycle monitoring and anomaly self-healing capabilities for containers. This allows for real-time capture of CPU and memory usage and running status signals for each computing container. In response to common issues in remote sensing big data processing, such as memory overflow or network fluctuations in computing nodes, a built-in anomaly handling mechanism is implemented. When a container crash caused by non-human factors is detected, the orchestration layer will attempt to automatically restart the container on a node with sufficient resources and remap the original storage path and environment variables. The pre-startup hook module will audit privileged instructions executed by users in real time to prevent illegal operations from compromising host security and ensure the robustness and isolation security of each independent image platform in a multi-tenant environment.

[0035] In step 8, after the user completes the processing task, they can initiate an exit command or the system can detect a session timeout or the task completion. In this process, the environment cleanup process begins. The hook module is pre-started to execute the deregistration script, automatically unmounting all NFS remote directories mounted in the container and cleaning up the temporary slice cache generated during the processing. This releases system I / O resources. Before the container is officially destroyed, an incremental snapshot of the private workspace is triggered to automatically back up the latest changes, processing logs, and environment configurations to prevent data loss due to single point of failure in the storage layer. The container orchestration layer confirms that computing resources such as GPU memory and CPU cores have been fully recovered and re-entered into the available resource pool to ensure the high efficiency of cluster resource recycling.

[0036] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0037] (1) The present invention has extremely high system integration and access efficiency, realizes deep integration of GPU computing resources and TB-level large-scale remote sensing raw datasets, and replaces the traditional data download mode with the NFS proxy re-export direct mounting mechanism, which solves the data retrieval delay problem and improves data access efficiency by more than 10 times.

[0038] (2) Strong customization and resolution of environmental conflicts: Each user environment is completely isolated, supporting fine-grained software stack customization for specific remote sensing algorithms. The pre-startup hook module automatically identifies the image type and performs dynamic path alignment and command injection, solving the problem of environmental conflicts between different deep learning frameworks.

[0039] (3) High security and isolation protection: The container sandbox mechanism and read-only storage mounting logic are adopted. While providing root-level customization permissions, hardware-level isolation protection of the underlying server is achieved, ensuring the security of the host system and the integrity of massive remote sensing raw data.

[0040] (4) Efficient sharing and scientific research collaboration: A consistent public data and document access path has been established, a seamless knowledge and data sharing space has been built, and an efficient scientific research collaboration model with fixed data and online analysis has been realized.

[0041] (5) Optimize the lifecycle of the image. By using the image snapshot solidification strategy, the startup timeout of the complex software stack of large-scale remote sensing deep learning is solved. Attached Figure Description

[0042] Figure 1 This is a block diagram of the hierarchical architecture of the system of this invention;

[0043] Figure 2 This is a flowchart of the container deployment and data flow of the system of this invention;

[0044] Figure 3This is a flowchart of the dynamic adaptation process for multiple mirror environments in the system of this invention;

[0045] Figure 4 This is a flowchart of the system image lifecycle optimization process of this invention;

[0046] Figure 5 This is the logic diagram of the nested NFS mounting system of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are merely to provide the reader with a thorough understanding of one or more aspects of the present invention, and these aspects of the invention can be implemented even without these specific details.

[0048] like Figure 1 As shown, a remote sensing big data integrated online computing system based on a distributed container cluster adopts a top-down four-layer architecture. The hardware foundation is a Docker Swarm cluster composed of multiple high-performance servers. Each layer communicates through the cluster intranet to form a full-link collaborative operation architecture. The specific implementation of each layer is as follows: The management layer is deployed on the cluster management node, built on JupyterHub, and provides a Web UI interaction interface to the outside world; after user-submitted resource requests are authenticated and verified, the built-in pre-startup hook module intercepts and parses metadata such as image specifications and resource quotas, completes container startup command injection and storage path mapping preprocessing, and then issues a standardized scheduling request to the container orchestration layer.

[0049] The container orchestration layer is deployed in conjunction with the management layer. It is built on the native cluster capabilities of Docker Swarm and has built-in container orchestration scheduling and monitoring services. After receiving scheduling instructions, this layer schedules tasks to matching worker nodes based on the real-time resource status of cluster nodes, completes the full lifecycle management of computing containers, and implements hardware resource isolation and quota limits through the Linux kernel mechanism to avoid resource contention among multi-tenant tasks.

[0050] The programming computing layer is deployed on the cluster worker nodes and divided into two heterogeneous resource pools: GPU and CPU. The GPU nodes are equipped with high-performance NVIDIA accelerator cards and have built-in deep learning environments such as TensorFlow and PyTorch adapted for remote sensing processing, as well as a GEE online processing and analysis environment, to perform computationally intensive remote sensing image interpretation, model training and inference tasks. The CPU nodes are adapted for lightweight data preprocessing and code development tasks. When the computing container starts, the storage path mapping is automatically injected through the pre-startup hook to achieve seamless access to storage resources.

[0051] The data storage and access layer serves as the unified data foundation of the system, built upon NFS distributed storage. It comprises three main modules: user private space, public read-only space, and multi-source external remote sensing datasets, with dedicated NFS proxy servers deployed synchronously. The user private space assigns dedicated read / write directories to each user, the public read-only space stores shared documents and tools, and the multi-source external remote sensing datasets store massive amounts of remote sensing imagery. The NFS proxy server, through configuration of fsid and crossmnt parameters, enables nested storage kernel-level proxy sharing, providing a unified data access entry point for the programming computation layer via a read-only mount path. This allows for online reading of remote sensing data without downloading, while simultaneously avoiding direct connections between computing nodes and remote source storage, thus reducing data security risks.

[0052] See Figure 2 In this embodiment, the entire process of container deployment and data flow is implemented based on the aforementioned four-layer architecture. In the Docker Swarm cluster of this embodiment, the worker nodes are configured with exemplary NVIDIA high-performance GPUs to support deep learning inference and quantitative inversion tasks of remote sensing images; the management node deploys the JupyterHub central service as the core entry point for user access and resource scheduling. After the computation request initiated by the user is verified and preprocessed by the management layer, the container orchestration layer completes the container scheduling and deployment. After the container starts, it automatically mounts the shared directory of the data storage access layer, forming a complete data flow closed loop of "request access - scheduling and deployment - resource mounting - task execution".

[0053] See Figure 3 The core scheduling logic of this invention is the dynamic pre-startup hook module integrated in JupyterHub, used to achieve adaptive adaptation to multiple image environments. When a user submits a computing resource request form through the web interface, the pre-startup hook module intercepts the request in real time and performs environment adaptive configuration according to the image type selected by the user. The system's preset adaptation logic branches are as follows:

[0054] If the user selects NVIDIA NGC official images such as TensorFlow and PyTorch, the system will automatically map the working directory inside the container to the user's working path, and inject GPU optimization environment variables to achieve on-demand allocation of video memory and avoid waste of video memory resources.

[0055] If the user selects the standard GEE or traditional Python analysis image, the system will map the mount target path to the user's regular working path, ensuring consistency in usage habits across different software stacks and lowering the barrier to entry.

[0056] At the environmental control level, to address the environment deployment failure issue caused by the restrictions on system-level pip installation in the new version of Python (PEP 668 specification), this embodiment dynamically generates container startup instructions through a pre-startup hook module. The specific execution sequence is as follows: first, the instructions are executed to create the mount anchor directory; then, the installation instructions, combined with customized parameters and domestic mirror source addresses, force the adaptation and installation of the JupyterHub single-user runtime environment; finally, the JupyterLab process is started, completely resolving the dependency installation failure issue under the new version of Python.

[0057] See Figure 4 This invention addresses the industry pain point of startup timeouts for ultra-large remote sensing images by providing a method for image solidification and full lifecycle optimization. Traditional JupyterHub remote sensing computing environments employ a dynamic deployment model. Each time a user starts a container, they must dynamically install Python dependencies, remote sensing-specific processing libraries, and supporting components online within the JupyterHub environment using the `pip install` command. This deployment process is prone to timeouts, with overall startup times reaching the minute level. It suffers from poor stability and the core flaw of requiring repeated dependency installations with each startup. This makes it highly susceptible to startup timeouts for ultra-large remote sensing images (for example, initial startup time exceeding 600 seconds) and repeated restarts of the Docker Swarm service due to response timeouts during the preparation phase. To address the aforementioned pain points, this invention pre-fabricates and solidifies the runtime environment through a one-time solidification phase: First, a temporary container is created based on a basic public image (such as the official PyTorch base image), and JupyterLab and remote sensing-specific processing libraries are pre-installed and debugged within the container. Then, the currently running temporary container is solidified into a local private image containing a complete runtime environment using the `docker commit` command. This image is uploaded to the cluster's private image repository and simultaneously updated in the available image configuration of JupyterHub, completing the entire process with one-time solidification. In the subsequent reusable user phase, when users submit computing power requests for this custom image through JupyterHub, the time-consuming steps of installing dependencies via the network are skipped, achieving container startup in seconds. This effectively avoids the failure of Docker Swarm service repeatedly restarting due to response timeouts during the preparation phase.

[0058] See Figure 5This invention addresses the technical problem of difficult sharing and access of remote sensing big data in cross-network segment environments by proposing a data sharing method based on NFS proxy re-export. In a typical application scenario, large-scale remote sensing datasets are stored on remote NAS storage servers. Due to network topology isolation, computing nodes cannot directly mount this remote storage resource. To address this, this invention deploys a relay proxy within the intranet of the computing cluster. The proxy server first mounts the remote dataset. Subsequently, in the NFS service configuration of the proxy server, a unique file system identifier (fsid) is set for the shared directory, and the cross-file system boundary access flag (crossmnt) is enabled, mapping it to the network segment where the computing cluster resides. Each computing node in the cluster network segment mounts the NFS shared path of the proxy server via Docker named volumes, enabling direct access to massive amounts of remote sensing image data without data copying or downloading, thus achieving zero-download, read-only access.

[0059] The integrated system provided in this embodiment supports users to complete the entire online analysis of remote sensing big data within the JupyterLab interface: users can directly access real-time synchronized global / regional remote sensing image datasets in the rs-data directory within the container; through the pre-installed remote sensing processing operators, analysis tasks such as multispectral band calculation, vegetation cover extraction, and target detection can be directly performed; the calculation results can be rendered and displayed directly in the browser through the system's built-in interactive plotting components such as Leaflet and Plotly, realizing an online closed loop for the entire remote sensing analysis process.

[0060] Matters not covered in this invention are common knowledge.

[0061] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A remote sensing big data integrated online computing system based on a distributed container cluster, characterized in that: It includes a management layer, a container orchestration layer, a programming compute layer, and a data storage access layer. The data storage access layer is connected to the programming compute layer, and the container orchestration layer is connected to both the management layer and the programming compute layer. The management system is built on JupyterHub to provide users with a visual web interface for operation and access, while also handling user authentication, access control, and receiving user requests for computing resources. The container orchestration layer is built on a Docker Swarm cluster and is used to receive instructions from the management layer, complete the full lifecycle management of computing containers, schedule and allocate computing tasks, and limit the amount of computing resources used by each task. The programming computing layer includes several worker nodes configured with GPU hardware acceleration units, which are used to run user computing containers and perform specific remote sensing data processing, analysis, and deep learning model training tasks. The data storage access layer is built on NFS distributed storage and is used to provide three types of storage space.

2. The remote sensing big data integrated online computing system based on a distributed container cluster according to claim 1, characterized in that: The three types of storage space include a private workspace dedicated to each user, a public document space accessible to all users, and a dataset storage pool for storing massive amounts of remote sensing image data.

3. The remote sensing big data integrated online computing system based on a distributed container cluster according to claim 1, characterized in that: The data storage access layer is equipped with an NFS proxy re-export mechanism. The specific implementation process is as follows: First, a dedicated NFS proxy node is deployed on the cluster intranet. Through this proxy node, a one-time full mount of the remote sensing dataset in the remote storage server is completed. Then, in the NFS service configuration file of the proxy node, a fixed and unique fsid file system identifier is configured for the remote sensing dataset directory to be shared. At the same time, crossmnt cross file system access permissions are enabled for the corresponding shared directory. Finally, through the NFS proxy node, the remote remote sensing dataset that has been mounted is re-shared with read-only access permissions to all worker nodes of the programming computing layer, realizing secure read-only sharing of remote sensing data and full cluster second-level batch mounting.

4. The remote sensing big data integrated online computing system based on a distributed container cluster according to claim 3, characterized in that: The fsid file system identifier is used to prevent file system ID drift and recognition conflicts during cross-node mounting; crossmnt cross-file system access permissions are used to support direct, penetrating access to nested hierarchical storage directories without needing to configure sharing rules separately for each level of subdirectory, while avoiding access deadlocks in nested mount scenarios.

5. The remote sensing big data integrated online computing system based on a distributed container cluster according to claim 4, characterized in that: By using NFS proxy nodes for unified relay, direct access of programming computing layer worker nodes to remote source storage is blocked, preventing remote sensing data from being tampered with, stolen, or leaked. The time required for cross-network segment mounting of all cluster nodes is reduced from minutes to seconds.

6. The remote sensing big data integrated online computing system based on a distributed container cluster according to claim 1, characterized in that: Technical issues related to data security risks arising from batch mounting timeouts when directly mounting remote storage to worker nodes in the programming computing layer, restricted access to nested directory levels, and direct exposure of core source storage.

7. A method for an integrated online computing system for remote sensing big data based on a distributed container cluster, as described in any one of claims 1-6, characterized in that: The method includes the following steps: Step 1: Manually start the base container image on the worker node of the programming computing layer, enter the container to complete the runtime environment configuration, installation of dependent components and remote sensing processing tools; Step 2: Users log in through the management layer and issue a request. The management layer then passes the request to the container orchestration layer, which schedules the computing tasks. Step 3: After the computing container starts up, it directly accesses the remote sensing dataset shared by the NFS proxy node of the data storage access layer through the pre-configured mount path, and performs online streaming reading and analysis processing of the raw remote sensing data. There is no need to download the remote sensing data to the container or user terminal throughout the process. Step 4: Display interactive feedback to users via the web interface, allowing them to view the relevant attribute information of the remote sensing data captured by the satellite and preprocess the remote sensing data; Step 5: Users use the management layer to call the pre-installed deep learning model in the system to analyze the pre-processed remote sensing data and uncover the hidden information in the remote sensing data; Step 6: Automatically store the results of data computation and processing online on NAS, and define automatic archiving logic for the computation results; Step 7: Set up independent image platform control and exception handling, link the management layer with the container orchestration layer, capture the CPU, memory usage and running status signals of each computing container in real time, and build an exception handling mechanism to realize exception early warning; Step 8: Independent image exit, data saving and automatic backup to prevent data loss due to single point of failure in the storage layer.

8. The method for an integrated online computing system for remote sensing big data based on a distributed container cluster according to claim 7, characterized in that: In step 1, after verifying that the environment is available, the currently available container environment is saved as a fixed snapshot image using the docker commit command. This image is then uploaded to the cluster's private image repository and updated synchronously to the available image list in the management layer for users to select and use. In step 2, after logging in through the web interface of the management layer, the user selects the required computing node specifications, solidifies the snapshot image, and submits a computing resource application. The management layer triggers the pre-start hook module to complete the preprocessing of automatic image path matching, startup command injection, and computing resource reservation. After the preprocessing is completed, the management layer sends the task scheduling instruction to the container orchestration layer, which schedules the computing tasks to the corresponding worker nodes, creates and starts the computing container, and completes the dynamic allocation and isolation of resources.

9. The method for an integrated online computing system for remote sensing big data based on a distributed container cluster according to claim 8, characterized in that: In step 4, after the computation task begins, the user prepares for interactive programming and data analysis of remote sensing data processing based on the JupyterLab environment within the container. Interactive feedback is displayed to the user through the web interface, allowing the user to view and export data online, thus achieving preprocessing before remote sensing analysis. In step 5, users enter a dedicated computing container environment via a web URL provided by the management layer. Using the built-in JupyterLab interactive programming interface, they directly call the pre-installed GDAL, PDAL, and deep learning frameworks. Based on the read-only mount path established by the data storage access layer, the analysis program directly streams the raw remote sensing images from the remote NFS proxy pool in a zero-download manner, performs spatiotemporal analysis on the raw remote sensing images, and renders the intermediate results in real time through the built-in Leaflet or Plotly components, achieving high-precision geospatial interactive display on the browser.

10. The method for an integrated online computing system for remote sensing big data based on a distributed container cluster according to claim 1, characterized in that: In step 6, a read-write separation persistence mechanism is adopted. All source code written by users is forcibly saved in the mounted user private workspace directory and synchronized in real time with the backend highly reliable NFS distributed storage to ensure that the code logic remains safe when the container is abnormally destroyed. Automatic archiving logic is defined for the calculation results. After the remote sensing products, statistical reports and training models generated by users are written to the specified output directory, they are persisted to the private storage space through the backend data channel, and a storage index is automatically generated according to the task ID. In step 7, users can set up container lifecycle monitoring and anomaly self-healing capabilities to achieve linkage between the management layer and the container orchestration layer. This allows for real-time capture of CPU, memory usage, and running status signals of each computing container. In response to common issues in remote sensing big data processing such as memory overflow or network fluctuations of computing nodes, a built-in anomaly handling mechanism is implemented. When a container crash caused by non-human factors is detected, the orchestration layer will attempt to automatically restart the container on a node with sufficient resources and remap the original storage path and environment variables. The pre-startup hook module will audit the privileged instructions executed by the user in real time. In step 8, after the user completes the processing task, they can initiate an exit command or the system can detect a session timeout or the task ends. In this process, the environment cleanup process begins. The hook module is pre-started to execute the deregistration script, automatically unmounting all NFS remote directories mounted in the container and cleaning up the temporary slice cache generated during the processing. This releases system I / O resources. Before the container is officially destroyed, an incremental snapshot of the private workspace is triggered to automatically back up the latest changes, processing logs, and environment configuration.

Images