Remote sensing data storage system, storage and retrieval method combining Kubernetes and Ceph

Abstract

The embodiments of the present disclosure provide a remote sensing data storage system, storage and retrieval method combining Kubernetes and Ceph. The storage system includes an interaction layer for interacting with the client, receiving remote sensing data uploaded by the client and / or providing corresponding remote sensing data according to the user's remote sensing data query request; the remote sensing data includes metadata files and image files; the service layer is used to generate a queue for the remote sensing data uploaded by the client and distribute it to the data storage layer through the data cache layer; and / or query the corresponding remote sensing data in the data cache layer and the data storage layer according to the client query request; the data cache layer is used to cache the remote sensing data; the data storage layer includes a layer for storing the metadata file in the MongoDB database and the image file in the Ceph distributed storage system. In this way, efficient storage, sharing and backup of massive remote sensing data are achieved.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of storage technology, and more specifically, to remote sensing data storage and retrieval methods, devices, equipment, and computer-readable storage media in combination with Kubernetes and Ceph. Background Art

[0002] With the increasing capability of remote sensing data acquisition, the amount of data acquired by a single satellite every day is calculated in TB level, and these big data have spatial structured characteristics, that is, spatial data. Spatial data itself has high density value, and each data unit is associated with a spatial location, reflecting certain spatial attributes.

[0003] The early spatial data storage and management method was an extremely effective heterogeneous spatial data storage management system based on relational databases. However, due to the high consistency requirements of relational databases for data, the scalability of the underlying storage is weak, and there are problems with data segmentation and merging. The traditional relational data model and data organization structure only realize the integrity of data organization at the conceptual and logical levels, including the sequence snapshot model, base map correction model, space-time combination model and space-time cube model, etc., without considering the access weight of remote sensing image data. In addition, the physical implementation uniformly adopts local disk storage, resulting in disk I / O becoming a bottleneck for database scheduling performance, greatly increasing the disk I / O load and data scheduling time. In addition, the storage cost is high, the scalability is poor, there are bottlenecks in data sharing, the control granularity of data security is insufficient, online queries are slow, and the download time is greatly affected by the number of concurrent connections.

[0004] Later, the spatial data storage and management methods were optimized to use NoSQL, column storage and other types of databases. The underlying distributed file system was used to achieve storage fault tolerance and scalability. In order to solve data consistency and spatial indexing problems, on the one hand, the index was placed in a large memory or split into a large number of small index storage. On the other hand, a large number of computing nodes were used for distributed index calculations. However, the applicable computing mode is relatively limited. Summary of the invention

[0005] According to an embodiment of the present disclosure, a remote sensing data storage solution combining Kubernetes and Ceph is provided.

[0006] In the first aspect of the present disclosure, a remote sensing data storage system combining Kubernetes and Ceph is provided. The system includes an interaction layer, a service layer, a data cache layer and a data storage layer; wherein the interaction layer is used to interact with the client, receive the remote sensing data uploaded by the client and / or provide corresponding remote sensing data according to the user's remote sensing data query request; the remote sensing data includes metadata files and image files; the service layer is used to generate a queue for the remote sensing data uploaded by the client and distribute it to the data storage layer through the data cache layer; and / or query the corresponding remote sensing data in the data cache layer and the data storage layer according to the client query request; the data cache layer is used to cache the remote sensing data; the data storage layer includes a layer for storing the metadata file in the MongoDB database and storing the image file in the Ceph distributed storage system.

[0007] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein the interaction layer, the service layer and the data cache layer are deployed in the master node of the Kubernetes cluster; and the data storage layer is deployed in the worker node of the Kubernetes cluster.

[0008] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein the MongoDB database and the Ceph distributed storage system are respectively deployed in the worker nodes of the Kubernetes cluster.

[0009] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein the service layer includes a message queue module, a task scheduling module, a remote sensing metadata parsing module, a database operation module and a storage operation module.

[0010] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein the task scheduling module is used to establish a mapping relationship between queue data and worker nodes based on a consistent hashing algorithm.

[0011] In the second aspect of the present disclosure, a remote sensing data storage method combining Kubernetes and Ceph is provided. The method includes: the interaction layer receives remote sensing data submitted by the client through a universal interface defined by the PUSH method; the remote sensing data includes metadata files and image files; the service layer generates queue data for the remote sensing data and distributes it to the data storage layer through the data cache layer; the queue data includes metadata file queue data and image file queue data; the data cache layer caches the queue data in a memory-type cache; the data storage layer stores the metadata file queue data in a MongoDB database, and stores the image queue data in a Ceph distributed storage system.

[0012] In the third aspect of the present disclosure, a remote sensing data query method combining Kubernetes and Ceph is provided. The method includes: the interaction layer receives a data retrieval request submitted by a client through a universal interface defined in a PULL manner; the service layer receives the data retrieval request and queries the cache layer for the data; the cache layer queries the cache layer's memory based on the key value given in the query request to see whether the requested data information is included; if the cache is hit, the corresponding metadata file or image file information is returned to the client; if it is not hit, the data is queried in the MongoDB distributed storage system of the data storage layer and the Ceph-based distributed storage system, and the acquired required data information is returned to the client.

[0013] As described above and any possible implementation method, a further implementation method is provided, in which the required data information obtained is returned to the client, including: returning the corresponding metadata file or image file information to the client through the service layer and the interaction layer, and writing it back to the memory of the cache layer.

[0014] In a fourth aspect of the present disclosure, an electronic device is provided, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.

[0015] In a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0016] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0018] Figure 1 A schematic diagram showing an exemplary operating environment in which embodiments of the present disclosure can be implemented;

[0019] Figure 2 A block diagram of a storage system according to an embodiment of the present disclosure is shown;

[0020] Figure 3 A flow chart of a data storage method according to an embodiment of the present disclosure is shown;

[0021] Figure 4 A block diagram of a data query method according to an embodiment of the present disclosure is shown;

[0022] Figure 5 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0024] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0025] Terminology explanation:

[0026] Kubernetes: k8s for short, is an open source container orchestration engine from Google that supports automated deployment, large-scale scalability, and containerized application management.

[0027] Ceph: A unified distributed storage system designed to provide better performance, reliability, and scalability, while supporting block storage, object storage, and distributed file systems.

[0028] MongoDB: A distributed, column-oriented open source database suitable for storing massive amounts of data. MongoDB stores loosely structured BSON-formatted data similar to JSON, and supports MapReduce to ensure that it can perform complex relationships and analyses on data. In addition, MongoDB supports indexing and retrieval operations for geographic information.

[0029] Figure 1 A schematic diagram of an exemplary operating environment 100 in which embodiments of the present disclosure can be implemented is shown. The operating environment 100 includes a client 102, a network 104, and a storage system 106.

[0030] Figure 2FIG. 1 shows a block diagram of a remote sensing data storage system 106 combining Kubernetes and Ceph according to an embodiment of the present disclosure; Figure 2 As shown, the storage system 106 includes an interaction layer 210 , a service layer 220 , a data cache layer 230 and a data storage layer 240 .

[0031] The interaction layer 210 is located at the front end of the storage system 106 and is responsible for interacting with other systems or clients 102 .

[0032] In some embodiments, the interaction layer 210 provides two integration modes, PULL and PUSH, supports multiple transmission protocols, and is compatible with different heterogeneous systems. On the one hand, the client 102 can submit data (remote sensing data) to the storage system 106 through the universal interface defined by the PUSH mode, and on the other hand, the client 102 can obtain data in the storage system 106 through the universal interface defined by the PULL mode, which is convenient for integration with other systems.

[0033] In some embodiments, the remote sensing data includes remote sensing metadata files, remote sensing image files, elevation files, etc. The interaction layer 210 is responsible for providing the client with the format specification of the metadata file, the general interface (upload interface) defined by the PUSH method of the image file and the elevation file; the interaction layer 210 submits the remote sensing data submitted by the client 102 to the storage system 106 through the general interface (upload interface) defined by the PUSH method to the lower service layer 220 for further processing.

[0034] In some embodiments, remote sensing data (remote sensing images) generally have a data header file describing the information of the image, which includes basic information for establishing metadata. According to the metadata elements to be established, relevant field values ​​are extracted as the values ​​of the corresponding metadata elements. The client extracts metadata according to the format specification provided by the interaction layer 210 and generates a metadata file.

[0035] In some embodiments, the interaction layer 210 is deployed in the master node of the Kubernetes cluster. The Kubernetes cluster is configured with at least two master nodes and multiple worker nodes. The at least two master nodes are mutually backup, ensuring that after a master node goes down, another master node takes over, so that the multiple worker nodes can continue to access another master node and operate normally.

[0036] The service layer 220 is a Kubernetes service layer, which is the core layer of the storage system 106 architecture. It is used to generate queues and distribute data uploaded by clients; and / or query corresponding data according to client query requests.

[0037] In some embodiments, the service layer 220 mainly includes a message queue module, a task scheduling module, a remote sensing metadata parsing module, a database operation module and a storage operation module. The message queue module is responsible for receiving the data submitted by the interaction layer 210, and the service program implements the storage of the queue data. Through the queue design, the instantaneous pressure of the database can be reduced and the concurrency capability can be improved. In some embodiments, the message queue module establishes queues for the metadata files and image files submitted by the interaction layer 210. The task scheduling module is responsible for scheduling and managing the execution of various task processes in the system. The task scheduling module includes core modules such as task execution, scheduler and trigger. The task scheduling module cooperates with the message queue module to complete the distribution and load balancing of storage data. The database operation module is responsible for writing the storage tasks in the metadata file queue to the MongoDB database of the data storage layer 240 through the cache layer 230. The storage operation module is responsible for writing the storage tasks in the image file queue to the Ceph distributed storage system of the data storage layer 240 through the cache layer 230.

[0038] In some embodiments, the remote sensing metadata parsing module is used to parse remote sensing image metadata of different structures to achieve unified formatting.

[0039] In some embodiments, by introducing a consistent hashing algorithm, a mapping relationship between queue data and worker nodes is established, the amount of data between nodes is balanced, and the number of data migrations between nodes is reduced. Specifically, a mapping relationship between metadata file queue data and worker nodes that deploy MongoDB databases, as well as a mapping relationship between image file queue data and worker nodes that deploy Ceph distributed storage systems, are established. In the consistent hashing algorithm, when the number of nodes changes, very few objects have invalid mapping relationships, and the migration cost is also very small. The consistent hashing algorithm only needs to relocate a small part of the data in the ring space for the addition or removal of nodes, and has good fault tolerance and scalability.

[0040] In some embodiments, due to differences in metadata content and application strategies between different systems, the metadata content and format provided are also different. The metadata file generated according to the metadata template is saved as an XML file. By analyzing the correspondence between the metadata elements and the content, the substantive content that can be converted can be found, so a metadata file conversion program can be designed to achieve conversion between different metadata files. In the disclosed embodiment, remote sensing image metadata with different structures are parsed to achieve unified formatting and generate metadata files with unified content. .

[0041] In some embodiments, the service layer 220 is deployed in the master node of the Kubernetes cluster.

[0042] The cache layer 230 is used to cache remote sensing data, and a memory cache such as Memcache or Redis cache server may be used.

[0043] In some embodiments, frequently used data is kept resident in the memory through a high-speed memory-type cache, thereby improving data access efficiency.

[0044] In some embodiments, the cache layer 230 caches the queue data in a memory-type cache and further stores it in the data storage layer 240 .

[0045] In some embodiments, the cache layer 230 receives a data acquisition request sent by the service layer 220 and accesses the data; the basic steps are as follows:

[0046] ① receiving the query request sent by the service layer 220, and querying whether the memory of the cache layer 230 contains the data information (metadata or image file information) required by the request according to the key value given in the query request;

[0047] ② If the cache information is hit, the required data information is returned directly from the cache layer;

[0048] ③ If the cache information is not hit, then enter the MongoDB database in the data storage layer 240 or the Ceph-based distributed storage system to obtain the required data information;

[0049] ④ The acquired required data information is returned to the client 102 through the service layer 220 and the interaction layer 210 , and written back to the memory of the cache layer 230 .

[0050] In some embodiments, the cache layer 230 is deployed in the master node of the Kubernetes cluster.

[0051] The data storage layer 240 is used for data storage and backup of the storage system 106 , and stores the queue data in a corresponding database or distributed storage system.

[0052] In some embodiments, the data storage layer 240 includes a MongoDB database and a Ceph distributed storage system; wherein MongoDB is a database based on distributed file storage.

[0053] In some embodiments, the MongoDB database is used to store unstructured metadata, and the Ceph distributed storage system is responsible for storing large-scale image files.

[0054] In some embodiments, the data storage layer 240 is deployed in the worker node of the Kubernetes cluster. The MongoDB database and the Ceph distributed storage system are deployed on the Kubernetes cluster instead of deploying a Ceph cluster independently, which reduces the difficulty of management. Since the kubernetes cluster can support multiple worker nodes, for example, hundreds of them, it can support the storage of massive image files, and the storage capacity of the cluster can be easily expanded by adding worker nodes.

[0055] Figure 3 FIG. 3 is a flowchart of a data storage method 300 according to an embodiment of the present disclosure. The method 300 may be Figure 1 The storage system 106 in is executed.

[0056] In block 310 , the interaction layer 210 receives data (remote sensing data) submitted by the client 102 through the universal interface defined by the PUSH method;

[0057] In some embodiments, the remote sensing data includes remote sensing metadata files, remote sensing image files, elevation files, etc. The interaction layer 210 is responsible for providing the client with the format specification of metadata, the general interface (upload interface) defined by the PUSH method of image files and elevation files; the interaction layer 210 submits the remote sensing data submitted by the client 102 to the storage system 106 through the general interface (upload interface) defined by the PUSH method to the lower service layer 220 for further processing.

[0058] In block 320 , the service layer 220 generates a queue for the data and distributes it to the data storage layer through the data cache layer;

[0059] In some embodiments, the system message queue is responsible for receiving data submitted by the interaction layer 210, and the service program implements the storage of queue data. Through the queue design, the instantaneous pressure of the database can be reduced and the concurrency capability can be improved. The task scheduling module is responsible for scheduling and managing the execution of various task processes in the system. The task scheduling module includes core modules such as task execution queue, scheduler and trigger. The task scheduling module cooperates with the message queue to complete the distribution and load balancing of requests.

[0060] The queue data includes metadata file queue data and image file queue data.

[0061] The remote sensing metadata parsing module is used to parse remote sensing image metadata of different structures to achieve unified formatting.

[0062] In some embodiments, by introducing a consistent hashing algorithm, a mapping relationship between queue data and worker nodes is established to balance the amount of data between nodes and reduce the number of data migrations between nodes.

[0063] In block 330 , the data cache layer 230 caches the queue data in a memory-type cache for further storage in the data storage layer 240 .

[0064] In block 340 , the data storage layer 240 stores the queue data in a corresponding database or distributed storage system.

[0065] In some embodiments, the data storage layer 240 includes a MongoDB database and a Ceph distributed storage system; wherein MongoDB is a database based on distributed file storage.

[0066] In some embodiments, the MongoDB database is used to store unstructured metadata, namely metadata file queue data; the Ceph distributed storage system is responsible for storing large-scale image files, namely image file queue data.

[0067] Figure 4 FIG. 4 is a flow chart showing a data retrieval method 400 according to an embodiment of the present disclosure. The method 400 may be performed by Figure 1 The storage system 106 in is executed.

[0068] In block 410 , the interaction layer 210 receives a data retrieval request submitted by the client 102 through a universal interface defined in a PULL manner;

[0069] In some embodiments, the data retrieval request is used to obtain data in the storage system 106 to facilitate integration with other systems.

[0070] At block 420 , the service layer 220 receives the data retrieval request and queries the cache layer 230 for the data;

[0071] In block 430 , the cache layer 230 queries whether the memory of the cache layer 230 contains the data information required by the request according to the key value given in the query request;

[0072] In block 440 , if the cache is hit, the corresponding metadata file or image file information is returned to the client 102 through the service layer 220 and the interaction layer 210 ;

[0073] In box 450, if the data is not hit in the corresponding database (MongoDB database and Ceph-based distributed storage system) of the data storage layer 240, the data is queried; the required data information obtained is returned to the client 102 through the service layer 220 and the interaction layer 210, and the corresponding metadata file or image file information is written back to the memory of the cache layer 230.

[0074] In some embodiments, if the data cannot be found in the corresponding database of the data storage layer 240 , feedback information indicating the query failure is returned to the client 102 through the service layer 220 and the interaction layer 210 .

[0075] According to the embodiments of the present disclosure, the following technical effects are achieved:

[0076] By combining Kubernetes and Ceph, we can achieve efficient storage, sharing and backup of massive remote sensing data. One cluster supports hundreds of nodes, which can support massive data storage, and the storage capacity of the cluster can be easily expanded by adding nodes. We aggregate data resources with the meta-information library, build heterogeneous storage for different computing (application) modes at the bottom layer, effectively decompose data, aggregate resources, and provide unified and standard services to the outside world. Through the distributed storage platform, we can achieve efficient storage management of massive, heterogeneous and increasing spatial big data.

[0077] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0078] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described method can refer to the corresponding description in the aforementioned system embodiment, and will not be repeated here.

[0079] Figure 5 Schematic block diagram of an electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The device 500 can be used to implement Figure 1500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 502 or computer program instructions loaded from a storage unit 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0080] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0081] The processing unit 501 performs the various methods and processes described above, such as methods 300 and 400. For example, in some embodiments, the methods 300 and 400 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the CPU 501, one or more steps of the methods 300 and 400 described above may be performed. Alternatively, in other embodiments, the CPU 501 may be configured to perform the methods 300 and 400 in any other suitable manner (e.g., by means of firmware).

[0082] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0083] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0084] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0085] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0086] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A remote sensing data storage system combining Kubernetes and Ceph, characterized in that: It includes interaction layer, service layer, data cache layer and data storage layer; among them, The interaction layer is used to interact with the client, receive remote sensing data uploaded by the client and / or provide corresponding remote sensing data according to the user's remote sensing data query request; the remote sensing data includes metadata files and image files; The service layer is used to generate a queue for the remote sensing data uploaded by the client and distribute it to the data storage layer through the data cache layer; and / or query the corresponding remote sensing data in the data cache layer and the data storage layer according to the client query request; The data cache layer is used to cache the remote sensing data; The data storage layer includes a layer for storing the metadata file in a MongoDB database and storing the image file in a Ceph distributed storage system; The interaction layer, the service layer and the data cache layer are deployed in the master node of the Kubernetes cluster; the data storage layer is deployed in the worker node of the Kubernetes cluster; the MongoDB database and the Ceph distributed storage system are respectively deployed in the worker nodes of the Kubernetes cluster.

2. The system according to claim 1, characterized in that The service layer includes a message queue module, a task scheduling module, a remote sensing metadata parsing module, a database operation module and a storage operation module.

3. The system according to claim 2, characterized in that The task scheduling module is used to establish a mapping relationship between queue data and worker nodes according to a consistent hashing algorithm.

4. A remote sensing data storage method combining Kubernetes and Ceph, applied to the system as described in any one of claims 1-3, characterized in that: include: The interaction layer receives remote sensing data submitted by the client through a universal interface defined by the PUSH method; the remote sensing data includes metadata files and image files; The service layer generates queue data for the remote sensing data and distributes it to the data storage layer through the data cache layer; the queue data includes metadata file queue data and image file queue data; The data cache layer caches the queue data in a memory cache; The data storage layer stores the metadata file queue data in a MongoDB database, and stores the image queue data in a Ceph distributed storage system.

5. A remote sensing data query method combining Kubernetes and Ceph, applied to the system as described in any one of claims 1-3, characterized in that: include: The interaction layer receives data retrieval requests submitted by the client through the general interface defined in PULL mode; The service layer receives the data retrieval request and queries the cache layer for the data; The cache layer queries the memory of the cache layer according to the key value given in the query request to determine whether the data information required by the request is included; If the cache is hit, the corresponding metadata file or image file information is returned to the client; If there is no hit, the data is queried in the MongoDB distributed storage system and the Ceph-based distributed storage system of the data storage layer, and the acquired required data information is returned to the client.

6. The method according to claim 5, characterized in that The required data information obtained is returned to the client including: The required data information obtained is returned to the client through the service layer and the interaction layer as the corresponding metadata file or image file information, and is written back to the memory of the cache layer.

7. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 4 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 4 to 6 is implemented.

Images