Unattended remote sensing image processing method, system, equipment and medium

Abstract

The invention provides an unattended remote sensing image processing method, system and device and a medium, and relates to the technical field of remote sensing image processing, and the method comprises the steps: executing a data scanning operation through a scanning management module according to a scanning rule described by a scanning item, and obtaining a to-be-processed remote sensing image; creating an image processing task for a to-be-processed remote sensing image through a task center module, and calling a target remote sensing processing business process from a process management module based on a scanning item; computing resources are distributed for the image processing tasks through the resource center module; executing a target remote sensing processing business process for the to-be-processed remote sensing image through the task center module by utilizing the computing resources to obtain a target processing result; and executing a storage operation on the target processing result through the data management module. The universality of remote sensing image processing can be remarkably improved, the automation degree of remote sensing image processing can be further improved, manual intervention is reduced, and the efficiency of large-scale remote sensing image processing is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to an unattended remote sensing image processing method, system, equipment and medium. Background Art

[0002] With the rapid advancement and development of satellite technology, the amount of remote sensing image data has increased rapidly. Remote sensing images meet the application needs of various industries in different forms, and fully automated image processing has become a trend. However, the fully automated image processing technology provided by related technologies is usually a customized image processing process for original remote sensing images, L1A-level data, etc., resulting in poor versatility of fully automated image processing technology. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide an unattended remote sensing image processing method, system, equipment and medium, which can not only significantly improve the versatility of remote sensing image processing, but also further improve the degree of automation of remote sensing image processing, so as to reduce manual intervention and improve the efficiency of large-scale remote sensing image processing.

[0004] In a first aspect, the present invention provides an unattended remote sensing image processing method, which is applied to a remote sensing image processing system, wherein the remote sensing image processing system includes a process management module, a scanning management module, a task center module, a resource center module, and a data management module, and the method includes:

[0005] Through the scanning management module, the data scanning operation is performed according to the scanning rules described by the scanning items corresponding to the various business scenarios configured therein to obtain the remote sensing image to be processed;

[0006] Through the task center module, create image processing tasks for the remote sensing images to be processed, and call the target remote sensing processing business process corresponding to the image processing task from the process management module based on the association between the scanning rules described by the scanning items and the remote sensing processing business process;

[0007] Allocate computing resources for image processing tasks through the resource center module;

[0008] Through the task center module, computing resources are used to execute the target remote sensing processing business process for the remote sensing images to be processed to obtain the target processing results;

[0009] Through the data management module, the target processing results are put into storage.

[0010] In one embodiment, the scanning rules include scanning paths, data features, and scanning intervals; performing data scanning operations according to the scanning rules described by the scanning items corresponding to the various business scenarios configured therein to obtain remote sensing images to be processed includes:

[0011] For each business scenario, determine the current scanning time point based on the scanning interval corresponding to the business scenario, and perform a data scanning operation on the scanning path corresponding to the business scenario when the current scanning time point is reached;

[0012] If the data scanning operation scans data, and the data scanned by the data scanning operation meets the data characteristics corresponding to the business scenario, it is determined that the remote sensing image to be processed is scanned.

[0013] In one embodiment, the method further comprises:

[0014] When the data scanning operation fails to scan any data, or when the data scanned by the data scanning operation does not conform to the data characteristics corresponding to the business scenario, the next scanning time point is determined based on the scanning interval corresponding to the business scenario, and the data scanning operation is continued to be performed on the scanning path corresponding to the business scenario when the next scanning time point is reached.

[0015] In one embodiment, a remote sensing image processing system is configured with a plurality of computing nodes, each of which includes a plurality of containers; and computing resources are allocated to image processing tasks, including:

[0016] From the computing nodes, assign a target computing node to each remote sensing processing algorithm in the target remote sensing processing business process;

[0017] Allocate a target container and a resource amount of the target container to each remote sensing processing algorithm in the target remote sensing processing business process from the containers included in the target computing node;

[0018] The computing resources include the target container corresponding to each remote sensing processing algorithm and the resource quantity of the target container.

[0019] In one implementation, the process management module is configured with dependencies between each remote sensing processing algorithm in the target remote sensing processing business process; using computing resources to execute the target remote sensing processing business process for the remote sensing image to be processed to obtain the target processing result, including:

[0020] For any remote sensing processing algorithm in the target remote sensing processing business process, in the target container allocated to the remote sensing processing algorithm, the remote sensing processing algorithm is executed for the remote sensing image to be processed or the processing result output by the upstream remote sensing processing algorithm on which the remote sensing processing algorithm depends, so as to obtain the processing result output by the remote sensing processing algorithm;

[0021] The processing result output by the last remote sensing processing algorithm in the target remote sensing processing business is used as the target processing result.

[0022] In one embodiment, the method further comprises:

[0023] During the execution of the image processing task, the remote sensing processing algorithm currently executed in the target remote sensing processing business process is monitored through the task center module, and when the remote sensing processing algorithm is monitored to be completed, the downstream remote sensing processing algorithm in the target remote sensing processing business process is automatically started based on the dependency relationship until each remote sensing processing algorithm in the target remote sensing processing business process is executed.

[0024] In one embodiment, the method further comprises:

[0025] The data query request is received through the data management module, and the target processing results corresponding to the data query request are queried from the target processing results of the storage operation that have been executed; wherein the data query request includes a spatial query request and / or an attribute query request.

[0026] In a second aspect, the present invention further provides an unattended remote sensing image processing system, comprising a process management module, a scanning management module, a task center module, a resource center module and a data management module;

[0027] The scanning management module is used to perform data scanning operations according to the scanning rules described by the scanning items corresponding to the various business scenarios configured therein to obtain the remote sensing images to be processed;

[0028] The task center module is used to create image processing tasks for the remote sensing images to be processed, and based on the association between the scanning rules described by the scanning items and the remote sensing processing business process, call the target remote sensing processing business process corresponding to the image processing task from the process management module;

[0029] The resource center module is used to allocate computing resources for image processing tasks;

[0030] The task center module is also used to use computing resources to execute the target remote sensing processing business process for the remote sensing image to be processed to obtain the target processing results;

[0031] The data management module is used to perform warehousing operations on the target processing results.

[0032] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.

[0033] In a fourth aspect, the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0034] The present invention provides an unattended remote sensing image processing method, system, device and medium. First, through the scanning management module, the scanning rules of the scanning items corresponding to the various business scenarios configured therein are executed to obtain the remote sensing image to be processed; then, through the task center module, an image processing task is created for the remote sensing image to be processed, and based on the association between the scanning rules described by the scanning items and the remote sensing processing business process, the target remote sensing processing business process corresponding to the image processing task is called from the process management module; then, through the resource center module, computing resources are allocated to the image processing task; again, through the task center module, computing resources are used to execute the target remote sensing processing business process for the remote sensing image to be processed to obtain the target processing result; finally, through the data management module, the target processing result is stored in the warehouse. The above method can realize the automatic creation of tasks and automatic storage of processing results of remote sensing images in various business scenarios by presetting scanning items corresponding to various business scenarios and their associated remote sensing processing business processes, which can not only significantly improve the versatility of remote sensing image processing, but also further improve the automation of remote sensing image processing, so as to reduce manual intervention and improve the efficiency of large-scale remote sensing image processing.

[0035] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A schematic diagram of a flow chart of an unattended remote sensing image processing method provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of the structure of a remote sensing image processing system provided by an embodiment of the present invention;

[0040] Figure 3 A technical framework diagram of an unattended remote sensing image processing method provided by an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of a true color processing and storage process for orthorectification of GF1-WFV data provided by an embodiment of the present invention;

[0042] Figure 5 A flowchart of automatic true color uniformity processing of received data provided by an embodiment of the present invention;

[0043] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The following fully automated image processing technologies have been proposed:

[0046] (1) Fully automatic remote sensing image preprocessing device: A fully automatic remote sensing image preprocessing device is proposed to solve the problem of diverse image production requirements. It empowers the entire remote sensing preprocessing process with AI (Artificial Intelligence), builds a full-process, automated, and customized remote sensing data preprocessing device, and realizes large-scale automated collaborative image production. It reduces the requirements for manual intervention in remote sensing image preprocessing and improves the efficiency of remote sensing image preprocessing. The processing of this system is triggered manually, and the device is only applicable to Class 1A data of the Resource Satellite Center data classification.

[0047] (2) Remote sensing automated processing system and remote sensing automated processing method: Provide a remote sensing automated processing system and remote sensing automated processing method for crop growth status analysis that is simple to operate and can realize automatic downloading of computer satellite data. This system is a customized automated agricultural remote sensing data processing platform and does not have general remote sensing data processing capabilities.

[0048] (3) A portable remote sensing image real-time processing system with integrated hardware and software: A portable remote sensing image real-time processing system with integrated hardware and software is implemented; it includes a computer with multiple GPUs (Graphics Processing Units) and multiple CPUs (Central Processing Units), an embedded cluster scheduling manager and a data receiving program, a data formatting program, and a remote sensing image processing program; it can efficiently and quickly process a large amount of remote sensing image data, and realize the whole process of image production preprocessing and processing integration and real-time display of image products. The system mainly realizes the real-time reception and processing of raw remote sensing image data, and the fully automatic processing of satellite and drone visible light 0-3 level products.

[0049] However, the above fully automated image processing technology has the problem of poor versatility. Based on this, the present invention provides an unattended remote sensing image processing method, system, device and medium, which can not only significantly improve the versatility of remote sensing image processing, but also further improve the automation level of remote sensing image processing, so as to reduce manual intervention and improve the efficiency of large-scale remote sensing image processing.

[0050] To facilitate understanding of this embodiment, firstly, an unattended remote sensing image processing method disclosed in an embodiment of the present invention is introduced in detail. The method is applied to a remote sensing image processing system. The remote sensing image processing system includes a process management module, a scanning management module, a task center module, a resource center module and a data management module. Figure 1 The flowchart of an unattended remote sensing image processing method shown in FIG. 1 mainly includes the following steps S102 to S110:

[0051] Step S102: The scanning management module performs data scanning operations according to scanning rules described by scanning items corresponding to various business scenarios configured therein to obtain remote sensing images to be processed.

[0052] Among them, the scanning items corresponding to the various business scenarios configured in the scanning management module are used to describe the scanning rules of the business scenarios and the association between the scanning rules and the remote sensing processing business process. The scanning rules include scanning paths, data features, and scanning intervals. In one example, for any business scenario, the scanning management module can be triggered to perform a data scanning operation on the scanning path corresponding to the business scenario according to the scanning interval corresponding to the business scenario, and when the scanned data meets the data features corresponding to the business scenario, it is determined that the remote sensing image to be processed has been scanned.

[0053] Step S104, creating an image processing task for the remote sensing image to be processed through the task center module, and calling the target remote sensing processing business process corresponding to the image processing task from the process management module based on the association between the scanning rules described by the scanning items and the remote sensing processing business process.

[0054] Among them, the process management module is configured with multiple remote sensing processing algorithms, multiple remote sensing processing business processes and the dependency relationship between each remote sensing processing algorithm in the remote sensing processing business process (also known as the parameter transfer relationship). The remote sensing processing business process is obtained by arranging the required remote sensing processing algorithms. In one example, when the scanning management module scans a remote sensing image to be processed in a business scene, it can automatically trigger the task center to create an image processing task for the remote sensing image to be processed, and determine the target remote sensing processing business process from the multiple remote sensing processing business processes configured in the process management module according to the aforementioned association relationship, and call the target remote sensing processing business process.

[0055] Step S106, allocating computing resources to the image processing task through the resource center module.

[0056] The remote sensing image processing system is configured with multiple computing nodes, each computing node includes multiple containers, and the computing resources include a target container corresponding to each remote sensing processing algorithm and the resource amount of the target container. In one example, for each remote sensing processing algorithm in the target remote sensing processing business process, a target computing node is allocated to the remote sensing processing algorithm, and a target container and resource amount are allocated to the remote sensing processing algorithm from the container included in the target computing node.

[0057] Step S108, through the task center module, using computing resources to execute the target remote sensing processing business process for the remote sensing image to be processed to obtain the target processing results.

[0058] The target processing result is the processing result output by the last remote sensing processing algorithm in the target remote sensing processing business process. In one example, according to the dependency relationship between the aforementioned remote sensing processing algorithms, each remote sensing processing algorithm in the target remote sensing processing business process can be executed in turn in the corresponding target container until each remote sensing processing algorithm in the target remote sensing processing business process is executed, and the target processing result output for the remote sensing image to be processed can be obtained.

[0059] Step S110, performing a warehousing operation on the target processing results through the data management module.

[0060] In one example, the data management module is configured with a storage algorithm corresponding to the storage plan. The storage algorithm is connected in series to the target remote sensing processing business process to receive the target processing results and perform storage operations on them.

[0061] The embodiment of the present invention involves algorithm and process management, automatic task triggering, task execution, etc. Among them, the automatic task triggering link can have various forms, including timed scanning, message or interface call, etc., and the embodiment of the present invention mainly adopts timed scanning.

[0062] The unattended remote sensing image processing method provided by the embodiment of the present invention can realize the automatic creation of remote sensing image tasks and automatic storage of processing results in various business scenarios by presetting scanning items corresponding to various business scenarios and their associated remote sensing processing business processes. This can not only significantly improve the versatility of remote sensing image processing, but also further improve the degree of automation of remote sensing image processing, thereby reducing manual intervention and improving the efficiency of large-scale remote sensing image processing.

[0063] For ease of understanding, the present invention first explains the remote sensing image processing system. Figure 2 The schematic diagram of the structure of a remote sensing image processing system shown in the figure includes a process management module, a scanning management module, a task center module, a resource center module and a data management module. The task center module is connected to the scanning management module, the resource center module and the data management module respectively. The scanning management module is also connected to the process management module. Figure 2 The arrows shown in represent the direction of data transmission.

[0064] (I) Process management module: used to realize the arrangement of remote sensing processing business processes, including algorithm management and process arrangement. Algorithm management mainly manages various remote sensing processing algorithms, including algorithm names, algorithm packages, algorithm images, etc.; process editing mainly arranges multiple remote sensing algorithms into a remote sensing processing business process according to different business scenarios, and configures the parameter transfer relationship between each remote sensing processing algorithm in the remote sensing processing business process; the arranged remote sensing processing business process is the basis for subsequent data processing.

[0065] (ii) Scanning management module: mainly manages scanning items configured according to different scenarios, triggers scanning items at certain time intervals and performs data scanning of the corresponding scanning path (also known as the scanning directory). Each scanning item includes scanning rules and their associated remote sensing processing business processes, algorithm parameter configuration and other contents. Among them, the scanning rules are set according to the characteristics of the business, and the data features and scanning intervals that need to trigger the remote sensing processing business process are set, such as the characters contained in the data name, the sign of complete data transmission, how often the scan is triggered, etc. The remote sensing processing business process is the processing process used for the scanned remote sensing images to be processed, and is the remote sensing processing business process configured by the process management module; the algorithm parameter configuration is to configure the input and output parameters of the remote sensing processing business process.

[0066] (III) Task center module: used to create tasks, schedule resources during task execution, and monitor task execution. After the scanning management module scans the remote sensing image to be processed, it will trigger the automatic creation of the remote sensing processing business process; during the task execution, the task scheduling module schedules the corresponding computing resources (also called hardware resources) to execute the remote sensing processing algorithm according to the resource requirements of the remote sensing processing algorithm in the remote sensing processing business process; the task monitoring function monitors the execution of the entire image processing task, and automatically starts the next remote sensing processing algorithm after one remote sensing processing algorithm is completed; after the remote sensing processing algorithm fails to execute due to insufficient resources, the remote sensing processing algorithm is restarted; after all remote sensing processing algorithms are executed, the image processing task is terminated.

[0067] (IV) Resource Center Module: It is used to manage the nodes, storage resources and the scheduling of the entire resources in the system, including node management, storage management and resource scheduling. Among them, node management is the management of the system computing nodes, supporting the dynamic expansion of computing resources; storage management is the management of the system storage resources, supporting the expansion of storage; resource scheduling is to allocate the resources required for the execution of remote sensing processing algorithms according to the needs of executing remote sensing processing algorithms.

[0068] (V) Data management module: used to manage storage plans and storage data, and provide data query functions. Among them, the storage plan mainly serves the analysis of storage data, including various types of plans, the content of the plan includes the structure of the data to be stored, and the metadata file that needs to be parsed; in order to achieve unattended storage, the storage plan is packaged into a storage algorithm, which is connected to the remote sensing processing business process in series to receive the target processing results output by the remote sensing processing business process; the storage data refers to various data that have been output and are to be stored; the data query function includes various common spatial queries (administrative region query, drawing range query, import vector file query, etc.) and attribute queries (phase, resolution, cloud cover, etc.).

[0069] Based on the remote sensing image processing system, the embodiment of the present invention provides a specific implementation of an unattended remote sensing image processing method.

[0070] The embodiment of the present invention provides a specific implementation of the aforementioned step S102, including steps 1.1 to 1.3:

[0071] Step 1.1 For each business scenario, determine the current scanning time point based on the scanning interval corresponding to the business scenario, and perform a data scanning operation on the scanning path corresponding to the business scenario when the current scanning time point is reached. For example, if the scanning interval is once an hour and the scanning path is a certain disk, then perform a data scanning operation on the disk every hour.

[0072] Step 1.2: If the data scanning operation scans data, and the data scanned by the data scanning operation meets the data characteristics corresponding to the business scenario, it is determined that the remote sensing image to be processed is scanned. For example, if the data characteristics are "starting with GF1, ending with final, and in the .tar.gz format", it is determined whether the scanned data meets the above data characteristics. If the data characteristics are met, it can be determined that the scanned data is a remote sensing image to be processed.

[0073] Step 1.3, when the data scanning operation does not scan data, or when the data scanned by the data scanning operation does not meet the data characteristics corresponding to the business scenario, determine the next scanning time point based on the scanning interval corresponding to the business scenario, and continue to perform the data scanning operation on the scanning path corresponding to the business scenario when the next scanning time point is reached. For example, when the data scanning operation is performed this time, no data is scanned from the disk, or the scanned data does not meet the aforementioned data characteristics of "starting with GF1, ending with final, and in the format of .tar.gz", wait until the next scanning time point, scan the disk again, and once data meeting the aforementioned data characteristics is scanned, the task center module can be triggered to create an image processing task.

[0074] The embodiment of the present invention provides a specific implementation of the aforementioned step S104, including steps 2.1 to 2.2:

[0075] Step 2.1, create an image processing task for the remote sensing image to be processed. For example, create a task of "GF1-WFV data orthorectification true color processing and storage".

[0076] Step 2.2, based on the association between the scanning rule described in the scanning item and the remote sensing processing business process, call the target remote sensing processing business process corresponding to the image processing task from the process management module. For example, the association described in the scanning item is: the remote sensing processing business process corresponding to the scanning rule is "GF1-WFV data orthorectification true color processing and storage process", then call "GF1-WFV data orthorectification true color processing and storage process" from the process management module.

[0077] The embodiment of the present invention provides a specific implementation of the aforementioned step S106, including step 3.1 to step 3.1 to step 3.2:

[0078] Step 3.1, from the computing nodes, assign a target computing node to each remote sensing processing algorithm in the target remote sensing processing business process. In one example, a plurality of computing nodes are configured in the system, and a target computing node can be assigned to each remote sensing processing algorithm in the target remote sensing processing business process based on the load conditions of each computing node. The target computing nodes assigned to different remote sensing processing algorithms can be the same or different.

[0079] Step 3.2, from the containers included in the target computing node, allocate a target container and the resource amount of the target container to each remote sensing processing algorithm in the target remote sensing processing business process. In one example, each remote sensing processing algorithm is allocated a different container, and the resource amount of the target container is determined based on the computational amount of the remote sensing processing algorithm. The container can well isolate the algorithm's operating environment from the host server, and at the same time can limit the hardware usage of the remote sensing processing algorithm.

[0080] An embodiment of the present invention provides a specific implementation method of the aforementioned step S108, including: for any remote sensing processing algorithm in the target remote sensing processing business process, in a target container allocated to the remote sensing processing algorithm, for the remote sensing image to be processed or the processing result output by the upstream remote sensing processing algorithm on which the remote sensing processing algorithm depends, executing the remote sensing processing algorithm to obtain the processing result output by the remote sensing processing algorithm; and taking the processing result output by the last remote sensing processing algorithm in the target remote sensing processing business as the target processing result.

[0081] Furthermore, during the execution of the image processing task, the remote sensing processing algorithm currently executed in the target remote sensing processing business process is monitored through the task center module, and when the remote sensing processing algorithm is monitored to be completed, the downstream remote sensing processing algorithm in the target remote sensing processing business process is automatically started based on the dependency relationship until each remote sensing processing algorithm in the target remote sensing processing business process is executed.

[0082] An embodiment of the present invention provides a specific implementation of the aforementioned step S110, including: packaging the warehousing plan into a warehousing algorithm, and connecting the warehousing algorithm in series after the target remote sensing processing business process, thereby using the warehousing algorithm to receive the target processing results, and successfully executing the warehousing operation for the target processing.

[0083] Furthermore, through the data management module, a data query request is received, and the target processing results corresponding to the data query request are queried from the target processing results that have been put into storage; wherein the data query request includes a spatial query request and / or an attribute query request, the spatial query request includes query requests such as a political district query, a drawing range query, and an import vector file query, and the attribute query request includes query requests such as a phase, a resolution, and a cloud cover.

[0084] Further, the embodiments of the present invention provide Figure 3 The technical framework diagram of an unattended remote sensing image processing method shown in the figure includes multiple processes such as registration algorithm, process arrangement, configuration of scanning items, scanning data, creation of processing tasks, output of results, and storage of results.

[0085] (A) Registration algorithm: Register the remote sensing processing algorithm that meets the business scenario in the remote sensing image processing system for subsequent process orchestration.

[0086] (B) Process arrangement: Arrange multiple remote sensing processing algorithms into a process suitable for remote sensing data processing business (referred to as remote sensing processing business process), and configure the parameter transfer relationship between different remote sensing processing algorithms. Take the business scenario of GF1-WFV data orthorectification true color processing as an example, see Figure 4 The diagram of the true color processing and storage process of orthorectification of GF1-WFV data shown in the figure requires arranging the remote sensing processing algorithms involved in multiple processes such as data decompression, star source analysis, project creation, pyramid creation within the project, control point matching within the project / connection point matching within the project, block adjustment, orthorectification within the project, true color output, metadata generation, satellite type judgment, thumbnail storage, and creation of satellite data index into the remote sensing processing business process under this scenario.

[0087] (C) Configure scanning items: Configure scanning items that meet business needs, including scanning directory, data features, scanning interval, remote sensing processing business process and algorithm output path, etc. For example, the remote sensing image to be processed starts with GF1, ends with final, and is in the .tar.gz format. The scanning interval is once an hour, and the remote sensing processing business process is "GF1-WFV data orthorectification true color processing and storage process".

[0088] (D) Scanning data: Based on the configured scanning items, the system automatically scans the data in the specified scanning directory at a certain time interval and determines whether it meets the set data characteristics; if data meeting the data characteristics is scanned, the task creation is started; if data meeting the data characteristics is not scanned, wait for the next scan.

[0089] (E) Create processing tasks: Create image processing tasks based on the remote sensing processing business process set in the scan item and the scanned data, and execute the image processing tasks based on the set computing resources.

[0090] (F) Output results: Output the target processing results during the execution of the image processing task.

[0091] (G) Storage of results: According to the set storage plan, the target processing results are stored in the corresponding location. The target processing results after storage can also be queried through the data query function.

[0092] The embodiment of the present invention proposes an unattended remote sensing image processing method, which constructs an automated and customized remote sensing data production process by presetting processing procedures, scanning services, and warehousing solutions, thereby realizing large-scale automated image production; eliminating manual participation in the remote sensing image processing process and improving the efficiency of large-scale image processing.

[0093] The embodiment of the present invention takes "automatic true color uniformity processing of received data" as an example to provide a specific implementation method of an unattended remote sensing image processing method. Figure 5 The flowchart of automatic true color uniformity processing of received data shown in the figure shows that "automatic true color uniformity processing of received data" involves multiple processes such as registering a true color algorithm, registering a uniformity algorithm, arranging a true color uniformity process, configuring true color uniformity scanning items, scanning true color data to be processed, creating a true color uniformity processing task, outputting uniformity results and storing the uniformity results in storage.

[0094] (a) Registering true color algorithms: True color algorithms are packaged according to unified algorithm management specifications and registered.

[0095] (b) Registered color grading algorithm: The color grading algorithm is packaged according to the unified algorithm management specification and registered.

[0096] (c) Configure the storage plan: According to the output type of the true color and uniform color algorithms, configure the corresponding data structure and metadata content parsing plan; in order to facilitate the readability and modifiability of the storage plan, the storage plan is described in JSON structure, which describes the file types supported for parsing, metadata field sources, thumbnails, quick view files and other information. Supported file type configuration:

[0097]

[0098] Indicates that files starting with ZY3_ and ending with .tiff or .tif are file types supported by this import solution.

[0099] Metadata field source resolution configuration:

[0100]

[0101] Indicates that the meta-field attribute platform is of string type and is not a required field. Its value comes from the SatelliteID field under ProductMetaData in the metadata file.

[0102] Quick view: "quickview": "{PRIMARY}.jpg", the quick view file name is composed of the main file name, with the suffix "jpg".

[0103] Thumbnail: "thumbnail"{PRIMARY}thumb.jpg", the thumbnail file name starts with the main file name and contains _thumb, and the suffix is ​​jpg.

[0104] (d) Registered algorithms: Algorithms are packaged in accordance with unified algorithm management specifications and registered.

[0105] (e) Arrange the true color uniformity process: Arrange the true color uniformity processing process; the input of the true color algorithm comes from the scanned data, the input of the uniformity algorithm comes from the output of the true color algorithm, and the uniformity results are stored in the warehouse; the input of the storage algorithm comes from the output of the true color or uniformity algorithm. According to the different types of storage results, multiple storage algorithms can be configured, and different results can be distinguished by the storage scheme configured by the storage algorithm; the process uses XML format to describe the algorithms (Entity) in the process and the relationship between algorithms (Edge). The Entity contains the input parameters and output parameters of the algorithm. The Edge contains the upstream algorithm and the downstream algorithm, and the relationship between the algorithms is established.

[0106] (f) Configure the true color uniformity scanning item: the path to be scanned is a disk directory, the data format is .tif, the scanning interval is once an hour, and the processing flow is "automatic true color uniformity processing flow for receiving data"; the scanning item is described in JSON format, which includes the parameter information of the process operation.

[0107] (g) Scanning the true color data to be scanned: Scan the data in the specified path according to the scan item; describe the scanning time interval through the CRON expression. If the data is scanned, the scan data is combined with the process configured by the scan item to call the task creation module to create a processing task; if the data is not scanned, wait until the next scan time point to scan again.

[0108] (h) Create a true color uniformity processing task: Based on the path of the scanned data and the true color uniformity process, build the task and submit it to the task scheduling module to schedule resources to execute the task.

[0109] (i) Output color balancing results: During the task execution process, the color balancing algorithm is executed in the container according to the order in the process arrangement. The container can well isolate the algorithm's operating environment from the host server and limit the algorithm's hardware usage. After the color balancing algorithm is executed, the output result is fed back to the scheduling system through the message mechanism and passed as the input parameter of the next algorithm.

[0110] (j) Storage of color uniformity results: Data is stored and managed through storage algorithms.

[0111] To sum up, the embodiments of the present invention address the lack of automated processing of remote sensing images and propose an unattended remote sensing image processing method. The method is suitable for fully automatic processing of various types of remote sensing image data, and the processing results are stored in real time to meet subsequent query, distribution and other needs.

[0112] Based on the above-mentioned embodiments, the embodiments of the present invention provide an unattended remote sensing image processing system, including a process management module, a scanning management module, a task center module, a resource center module and a data management module;

[0113] The scanning management module is used to perform data scanning operations according to the scanning rules described by the scanning items corresponding to the various business scenarios configured therein to obtain the remote sensing images to be processed;

[0114] The task center module is used to create image processing tasks for the remote sensing images to be processed, and based on the association between the scanning rules described by the scanning items and the remote sensing processing business process, call the target remote sensing processing business process corresponding to the image processing task from the process management module;

[0115] The resource center module is used to allocate computing resources for image processing tasks;

[0116] The task center module is also used to use computing resources to execute the target remote sensing processing business process for the remote sensing image to be processed to obtain the target processing results;

[0117] The data management module is used to perform warehousing operations on the target processing results.

[0118] The unattended remote sensing image processing system provided by the embodiment of the present invention can realize the automatic creation of remote sensing image tasks and automatic storage of processing results in various business scenarios by presetting scanning items corresponding to various business scenarios and their associated remote sensing processing business processes. This can not only significantly improve the versatility of remote sensing image processing, but also further improve the degree of automation of remote sensing image processing, thereby reducing manual intervention and improving the efficiency of large-scale remote sensing image processing.

[0119] The implementation principle and technical effects of the system provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the system embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0120] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0121] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 60, a memory 61, a bus 62 and a communication interface 63, wherein the processor 60, the communication interface 63 and the memory 61 are connected via the bus 62; the processor 60 is used to execute an executable module stored in the memory 61, such as a computer program.

[0122] The memory 61 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 63 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0123] The bus 62 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0124] Among them, the memory 61 is used to store programs, and the processor 60 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 60 or implemented by the processor 60.

[0125] The processor 60 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 60. The above processor 60 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 61, and the processor 60 reads the information in the memory 61 and completes the steps of the above method in combination with its hardware.

[0126] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0127] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0128] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An unattended remote sensing image processing method, characterized in that: The method is applied to a remote sensing image processing system, which includes a process management module, a scanning management module, a task center module, a resource center module and a data management module. The method includes: Through the scanning management module, the data scanning operation is performed according to the scanning rules described in the scanning items corresponding to the various business scenarios configured therein to obtain the remote sensing image to be processed; Creating an image processing task for the remote sensing image to be processed through the task center module, and calling the target remote sensing processing business process corresponding to the image processing task from the process management module based on the association between the scanning rule described by the scanning item and the remote sensing processing business process; Allocating computing resources to the image processing task through the resource center module; Through the task center module, the computing resources are used to execute the target remote sensing processing business process for the remote sensing image to be processed to obtain the target processing result; The target processing results are put into storage through the data management module.

2. The unattended remote sensing image processing method according to claim 1, characterized in that: The scanning rules include scanning paths, data features and scanning intervals; performing data scanning operations according to the scanning rules described in the scanning items corresponding to the various business scenarios configured therein to obtain remote sensing images to be processed includes: For each business scenario, determine a current scanning time point based on the scanning interval corresponding to the business scenario, and perform a data scanning operation on the scanning path corresponding to the business scenario when the current scanning time point is reached; If the data scanning operation scans data, and the data scanned by the data scanning operation meets the data characteristics corresponding to the business scenario, it is determined that the remote sensing image to be processed is scanned.

3. The unattended remote sensing image processing method according to claim 2, characterized in that: The method further comprises: When the data scanning operation fails to scan any data, or when the data scanned by the data scanning operation does not conform to the data characteristics corresponding to the business scenario, the next scanning time point is determined based on the scanning interval corresponding to the business scenario, and when the next scanning time point is reached, the data scanning operation is continued to be performed on the scanning path corresponding to the business scenario.

4. The unattended remote sensing image processing method according to claim 1, characterized in that: The remote sensing image processing system is configured with a plurality of computing nodes, each of which includes a plurality of containers; Allocating computing resources for the image processing task includes: Allocating a target computing node from the computing nodes to each remote sensing processing algorithm in the target remote sensing processing business process; Allocating a target container and a resource amount of the target container to each remote sensing processing algorithm in the target remote sensing processing business process from the container included in the target computing node; The computing resources include the target container corresponding to each remote sensing processing algorithm and the resource amount of the target container.

5. The unattended remote sensing image processing method according to claim 4, characterized in that: The process management module is configured with the dependency relationship between each remote sensing processing algorithm in the target remote sensing processing business process; Utilizing the computing resources to execute the target remote sensing processing business process for the remote sensing image to be processed to obtain the target processing result includes: For any of the remote sensing processing algorithms in the target remote sensing processing business process, in the target container allocated to the remote sensing processing algorithm, the remote sensing processing algorithm is executed for the remote sensing image to be processed or the processing result output by the upstream remote sensing processing algorithm on which the remote sensing processing algorithm depends, so as to obtain the processing result output by the remote sensing processing algorithm; The processing result output by the last remote sensing processing algorithm in the target remote sensing processing business is used as the target processing result.

6. The unattended remote sensing image processing method according to claim 5, characterized in that: The method further comprises: During the execution of the image processing task, the remote sensing processing algorithm currently executed in the target remote sensing processing business process is monitored by the task center module, and when the remote sensing processing algorithm is monitored to have finished execution, the next remote sensing processing algorithm in the target remote sensing processing business process is automatically started based on the dependency relationship until each remote sensing processing algorithm in the target remote sensing processing business process has finished execution.

7. The unattended remote sensing image processing method according to claim 1, characterized in that: The method further comprises: The data management module receives a data query request, and queries the target processing results corresponding to the data query request from the target processing results that have executed the warehousing operation; wherein the data query request includes a spatial query request and / or an attribute query request.

8. An unattended remote sensing image processing system, characterized in that: It includes process management module, scanning management module, task center module, resource center module and data management module; The scanning management module is used to perform data scanning operations according to the scanning rules described in the scanning items corresponding to the various business scenarios configured therein to obtain the remote sensing images to be processed; The task center module is used to create an image processing task for the remote sensing image to be processed, and based on the association between the scanning rule described by the scanning item and the remote sensing processing business process, call the target remote sensing processing business process corresponding to the image processing task from the process management module; The resource center module is used to allocate computing resources for the image processing task; The task center module is also used to use the computing resources to execute the target remote sensing processing business process for the remote sensing image to be processed to obtain the target processing results; The data management module is used to perform a warehousing operation on the target processing results.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.

Images