Distributed-based pyramid construction method and apparatus

Abstract

The application discloses a pyramid construction method and device based on distribution, and relates to the technical field of remote sensing image processing. The method comprises the following steps: acquiring a target range and a maximum level under a target reference coordinate system, generating a coverage tile set according to the target range and the maximum level, and constructing a quadtree structure according to the coverage tile set; then, a plurality of sub-trees are determined on the quadtree structure, a deep-first scheduling sequence is adopted to execute the sub-trees one by one, each sub-tree is regarded as an independent distributed computing task, tiles are processed in parallel in the sub-trees by using a distributed computing framework, a sub-tree containing a highest resolution layer is executed first, then corresponding parent-child sub-trees are executed, and a level aggregation from bottom to top is completed. The pyramid level aggregation correctness can be maintained, the memory occupation is reduced from exponential growth to a constant order of magnitude related to the height of the sub-tree, and therefore, the efficient construction demand of large-scale raster images in a limited memory environment can be met.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, specifically to a distributed pyramid construction method and apparatus. Background Technology

[0002] With the continuous improvement of remote sensing image resolution and the rapid growth of image data scale, pyramid tile construction has become a key step in remote sensing image publishing and visualization services. Existing pyramid construction methods typically employ full-layer caching or batch parallel computing. As the number of layers increases, the number of tiles grows exponentially, leading to a rapid expansion of memory consumption and problems such as intermediate result backlog, unstable task scheduling, and decreased construction efficiency. Current common solutions often use bottom-up aggregation, relying on lower-layer tiles to generate upper-layer tiles, resulting in complex cross-layer dependencies. When processing terabyte (TB) level raster data, full-layer caching is difficult to stably complete in a limited memory environment. Some solutions use distributed computing frameworks for tile parallel processing, but these often use full-layer tiles as the parallel unit, causing intermediate results from all levels to reside simultaneously in the memory of distributed nodes, resulting in an exponential increase in total cluster memory consumption. In terabyte-level single-image scenarios, node memory is quickly exhausted, making stable construction difficult. Existing methods struggle to balance throughput and memory safety, necessitating a pyramid construction method that can control intermediate result residency, reduce memory spikes, and maintain correct aggregation relationships. Summary of the Invention

[0003] The purpose of this application is to provide a distributed pyramid construction method and apparatus, which solves the problems in large-scale raster image pyramid construction, such as the sharp increase in memory consumption, complex cross-layer dependencies, and insufficient scheduling stability caused by full-layer caching and batch parallelism, making it difficult to complete the construction efficiently in a limited memory environment.

[0004] This application is achieved through the following technical solution:

[0005] The first aspect of this application provides a distributed pyramid construction method, comprising:

[0006] Obtain the image association information corresponding to the raster image metadata, and obtain the target range and maximum level in the target reference coordinate system based on the image association information;

[0007] A set of covering tiles is generated based on the target range and the maximum level, and a quadtree structure is constructed based on the set of covering tiles.

[0008] On the quadtree structure, the root node of the subtree is marked with a fixed subtree height, and multiple subtrees are determined based on the root node of the subtree;

[0009] The algorithm employs a depth-first scheduling order to execute subtrees one by one, treating each subtree as an independent distributed computing task. Within each subtree, the distributed computing framework is used to process tiles in parallel. The algorithm executes the subtree containing the highest resolution layer first, followed by the corresponding parent and child trees, thus completing the bottom-up hierarchical aggregation and obtaining the raster image pyramid.

[0010] In one possible implementation, image association information corresponding to raster image metadata is obtained, and the target range and maximum level in the target reference coordinate system are obtained based on the image association information, including:

[0011] Obtain the source image range, coordinate system, number of pixel columns, and number of pixel rows corresponding to the raster image metadata to obtain image association information;

[0012] Based on the image association information, the target range and maximum level in the target reference coordinate system are obtained.

[0013] In one possible implementation, a set of overlay tiles is generated based on the target range and the maximum level, and a quadtree structure is constructed based on the set of overlay tiles, including:

[0014] A quadtree is constructed from top to bottom using a queue method with L0 level as the root node; L0 level refers to the top level.

[0015] For each level, the range of tile row and column numbers intersecting with the target range is calculated, and the tiles located within the tile row and column number range are constructed as a set of covering tiles. Quadtree nodes are generated only for the tiles in the set of covering tiles, and the quadtree continues to expand downward according to the parent-child relationship of the quadtree until the maximum level is reached, resulting in a quadtree structure that covers the target range and contains only intersecting tiles.

[0016] In one possible implementation, the root nodes of subtrees are marked on the quadtree structure at fixed subtree heights, and multiple subtrees are determined based on the root nodes of the subtrees, including: based on the quadtree structure, a root node of a subtree is determined every d-1 layers, and a continuous d layers of tiles are regarded as a subtree, thus obtaining multiple subtrees; where d is a pre-set positive integer.

[0017] In one possible implementation, execution is carried out subtree by subtree using a depth-first scheduling order, including:

[0018] The subtree is used as a distributed computing task. After one distributed computing task is completed, the next distributed computing task is executed to reduce the memory peak caused by parallel superposition. During the execution of the distributed computing task, the intermediate results are only cached in memory and not stored.

[0019] In one possible implementation, executing the next distributed computing task after the completion of one distributed computing task includes: caching the next distributed computing task in memory during the execution of one distributed computing task, and starting the next distributed computing task only after all computing nodes of the currently executed distributed computing task have completed and released memory.

[0020] In one possible implementation, each subtree is treated as an independent distributed computing task. Within the subtree, tiles are processed in parallel using a distributed computing framework. This includes distributing the reading, projection, and slicing operations of each tile within the subtree to multiple computing nodes for parallel processing, and releasing the cache of the corresponding lower-level tile after any upper-level tile has been processed.

[0021] In one possible implementation, during the process of reading the source image and constructing tiles in the lowest subtree of all subtrees, the method further includes: determining the source data window based on the lowest subtree, expanding the source data window, intersecting the source data window after the expansion operation with the source image range to obtain a source image set, and using the source image set to perform reading, projection, and slicing operations to obtain the highest resolution tile.

[0022] In one possible implementation, the method further includes:

[0023] For subtrees, only leaf layer tiles are cached in memory, and all intermediate layer tiles above the leaf layer are released immediately after generation. Output is only performed on the target level to be persisted.

[0024] The method also includes employing a batch reading and spatially sequential reading strategy during the reading of the lowest-level subtree.

[0025] A second aspect of this application provides a distributed pyramid construction device, comprising:

[0026] The initial processing module is used to obtain image association information corresponding to raster image metadata, and to obtain the target range and maximum level in the target reference coordinate system based on the image association information.

[0027] The quadtree construction module is used to generate a set of covering tiles based on the target range and the maximum level, and to construct a quadtree structure based on the set of covering tiles.

[0028] A subtree segmentation module is used to mark the root node of a subtree at a fixed subtree height on the quadtree structure, and to determine multiple subtrees based on the root node of the subtree.

[0029] The pyramid building module is used to execute subtrees in a depth-first scheduling order, treating each subtree as an independent distributed computing task. Within each subtree, the distributed computing framework is used to process tiles in parallel. The subtree containing the highest resolution layer is executed first, followed by the corresponding parent and child trees, completing the bottom-up hierarchical aggregation to obtain the raster image pyramid.

[0030] Compared with the prior art, this application has the following advantages and beneficial effects:

[0031] This application provides a distributed pyramid construction method and apparatus. The method includes: obtaining the target range and maximum level in the target reference coordinate system, generating a set of overlay tiles based on the target range and maximum level, and constructing a quadtree structure based on the overlay tile set; then determining multiple subtrees on the quadtree structure, executing each subtree in a depth-first scheduling order, treating each subtree as an independent distributed computing task, processing tiles in parallel using a distributed computing framework within the subtree, and executing the subtree containing the highest resolution layer first, followed by the corresponding parent and child trees, to complete bottom-up hierarchical aggregation. This method can reduce memory usage from exponential growth to a constant order of magnitude related to the subtree height while maintaining the correctness of pyramid hierarchical aggregation, thereby meeting the need for efficient construction of large-scale raster images in a limited memory environment. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the exemplary embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0033] Figure 1 A flowchart illustrating a distributed pyramid construction method provided in this application embodiment;

[0034] Figure 2 A schematic diagram of a distributed pyramid building device provided in an embodiment of this application;

[0035] The attached diagram shows the markings and corresponding component names:

[0036] 201 - Initial processing module, 202 - Quadtree construction module, 203 - Subtree splitting module, 204 - Pyramid construction module. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this application are only for explaining this application and are not intended to limit this application.

[0038] To enable those skilled in the art to more easily understand the technical solutions described in the embodiments of this application, the embodiments of this application first briefly describe the pyramid construction process, which is as follows: using remote sensing raster images (such as images with spatial reference information) as input data sources, by reading image metadata and pixel raster data, the pyramid construction process is completed, including projection unification, range and level inference, tile key planning, windowed reading and slicing, bottom-up aggregation, and output data, thereby generating a multi-level tile pyramid.

[0039] like Figure 1 As shown in the figure, this application provides a distributed pyramid construction method, including:

[0040] S101. Obtain the image association information corresponding to the raster image metadata, and obtain the target range and maximum level in the target reference coordinate system based on the image association information.

[0041] The target reference coordinate system refers to the coordinate system used in the pyramid construction process to unify the tile grid, tile index, and spatial resolution of each level. The target reference coordinate system is preferably the Web Mercator coordinate system (WebMercator, corresponding to EPSG:3857 / 900913) to be consistent with the mainstream WebGIS tile coordinate system, or set to other projected coordinate systems according to mapping requirements. When the source image coordinate reference system is inconsistent with the target coordinate reference system, coordinate transformation is performed on the image range and pixel resolution before range and level inference are performed.

[0042] S102. Generate a set of covering tiles according to the target range and the maximum level, and construct a quadtree structure according to the set of covering tiles.

[0043] S103. Mark the root node of the subtree on the quadtree structure according to the fixed subtree height, and determine multiple subtrees based on the root node of the subtree.

[0044] S104. The depth-first scheduling order is adopted to execute subtree by subtree. Each subtree is treated as an independent distributed computing task. Within the subtree, the tiles are processed in parallel using a distributed computing framework. The subtree containing the highest resolution layer is executed first, and then the corresponding parent and child trees are executed. This completes the bottom-up hierarchical aggregation and obtains the raster image pyramid.

[0045] In one possible implementation, image association information corresponding to raster image metadata is obtained, and the target range and maximum level in the target reference coordinate system are obtained based on the image association information, including:

[0046] Obtain the source image range, coordinate system, number of pixel columns, and number of pixel rows corresponding to the raster image metadata to obtain image association information;

[0047] Based on the image association information, the target range and maximum level in the target reference coordinate system are obtained.

[0048] Preferably, obtaining the target range and maximum level under the target projection based on the image association information includes: reprojecting the source image range from the source coordinate reference system to the target coordinate reference system to obtain the target range; calculating the pixel ground resolution based on the number of pixel columns and rows of the source image and the source image range, and inferring the maximum level that satisfies "the number of meters per pixel of the target level ≤ the number of pixels of the source image in meters" in combination with the tile size; and then determining the actual maximum level as the preset maximum level to avoid unnecessary oversampling.

[0049] In one possible implementation, a set of overlay tiles is generated based on the target range and the maximum level, and a quadtree structure is constructed based on the set of overlay tiles, including:

[0050] A quadtree is constructed from top to bottom using a queue method with L0 level as the root node; L0 level refers to the top level.

[0051] For each level, the range of tile row and column numbers intersecting with the target range is calculated, and the tiles located within the tile row and column number range are constructed as a set of covering tiles. Quadtree nodes are generated only for the tiles in the set of covering tiles, and the quadtree continues to expand downward according to the parent-child relationship of the quadtree until the maximum level is reached, resulting in a quadtree structure that covers the target range and contains only intersecting tiles.

[0052] In one possible implementation, the root nodes of subtrees are marked on the quadtree structure at fixed subtree heights, and multiple subtrees are determined based on the root nodes of the subtrees, including: based on the quadtree structure, a root node of a subtree is determined every d-1 layers, and a continuous d layers of tiles are regarded as a subtree, thus obtaining multiple subtrees; where d is a pre-set positive integer.

[0053] The principle of the subtree task splitting strategy is as follows: treat a continuous d-layer tile as a subtree, use the high-resolution leaf layer as the input cache of the subtree, and aggregate the upper-layer tile layer by layer along the parent-child dependency within the subtree; once the upper-layer tile is generated, it is immediately used for the next synthesis and the corresponding lower-layer cache is released, forming a memory pipeline that synthesizes immediately when the input is ready and discards after use.

[0054] In one possible implementation, execution is carried out subtree by subtree using a depth-first scheduling order, including:

[0055] The subtree is used as a distributed computing task. After one distributed computing task is completed, the next distributed computing task is executed to reduce the memory peak caused by parallel superposition. During the execution of the distributed computing task, the intermediate results are only cached in memory and not stored.

[0056] In one possible implementation, executing the next distributed computing task after the completion of one distributed computing task includes: caching the next distributed computing task in memory during the execution of one distributed computing task, and starting the next distributed computing task only after all computing nodes of the currently executed distributed computing task have completed and released memory.

[0057] The principle of depth-first scheduling is as follows: taking subtrees as the execution unit, the process of "high-resolution leaf layer generation - parent layer aggregation - result output" is organized into a single dependency chain. Once the downstream input of the current subtree is ready, aggregation is completed immediately and the corresponding cache is released before switching to the next subtree. The purpose is to avoid multiple dependency chains from concurrently residing, reducing the peak memory usage from the exponential growth of full-layer parallelism to a constant level related to d. The reason for this is that pyramid construction has a strict bottom-up dependency relationship. If horizontal parallelism or full-layer expansion is used, multiple layers of intermediate results will be retained and superimposed, resulting in uncontrollable memory consumption. Downstream input refers to the set of lower-level tiles (preferably the four sub-tiles corresponding to the upper-level tile, or the set of neighboring tiles related to the coverage of the resampling kernel) on which the upper-level tile to be aggregated depends. Ready means that the dependent lower-level tiles have been generated and can be read / accessed: for the bottom-level subtree, the leaf layer tiles have been read, projected and sliced ​​from the source image; for non-bottom-level subtrees, the subtree has been completed and the root tiles used for connection are cached in memory. The parent and child trees can obtain and assemble the input layer from the memory cache according to the tile key set as the aggregation input, thereby triggering the aggregation of the upper-level tiles.

[0058] Depth-first scheduling executes tasks in batches in a distributed environment, starting only one subtree task per batch. The next subtree task is started only after all computing nodes have completed and released their memory. The principle behind this is to avoid the memory accumulation of each node due to the concurrent execution of multiple subtree tasks in the cluster. Its purpose is to keep the total memory usage of the cluster at a constant level. The reason is that if multiple subtrees are executed concurrently in a distributed environment, the cache of each node will accumulate linearly, which may still exceed the total memory capacity of the cluster.

[0059] In one possible implementation, each subtree is treated as an independent distributed computing task. Within the subtree, tiles are processed in parallel using a distributed computing framework. This includes distributing the reading, projection, and slicing operations of each tile within the subtree to multiple computing nodes for parallel processing, and releasing the cache of the corresponding lower-level tile after any upper-level tile has been processed.

[0060] Subtree tasks are executed on a distributed computing framework, distributing the reading, projection, and slicing operations of each tile within the subtree to multiple computing nodes for parallel processing. The principle is that there are no dependencies between tiles within the subtree, allowing for full parallelism. The purpose is to improve the execution efficiency of a single subtree. The reason is that a single subtree of a TB-level image can contain thousands to tens of thousands of tiles, and distributed parallelism can significantly shorten the execution time.

[0061] In one possible implementation, during the process of reading source images and constructing tiles in the lowest-level subtree of all subtrees, the method further includes: determining a source data window based on the lowest-level subtree, expanding the source data window, intersecting the expanded source data window with the source image range to obtain a source image set, and using the source image set to perform reading, projection, and slicing operations to obtain the highest resolution tile. For the upper-level subtree, leaf layer data is read from the memory cache, upsampling is performed to generate low-resolution layer tiles, and these tiles are then output.

[0062] The lowest-level subtree performs range expansion and intersection processing when reading the source image: first, the source data window is calculated from the target tile range, and then the source window is appropriately expanded and intersected with the source image range; the principle is that projection and resampling will introduce boundary errors; the purpose is to ensure the integrity of the tile boundary; the reason is that directly cropping according to the precise window may lead to the loss of boundary after resampling.

[0063] Preferably, the association between the source data window and the source image is a target tile range inverse calculation reading window. For the leaf layer tile key of the lowest subtree, the corresponding target tile range is first determined in the target coordinate reference system. Then, the target tile range is expanded to cover the boundary errors introduced by projection distortion and resampling, and the expanded range is reprojected from the target coordinate reference system to the source image coordinate reference system to obtain the source data window. The intersection of the source data window and the source image range is obtained to obtain the effective reading window, and only the data blocks that intersect with the effective reading window are read. If the source image coordinate reference system and the target coordinate reference system are inconsistent after reading, reprojection is performed, and then the tiles are sliced ​​according to the maximum level layout and the leaf layer tiles are filtered out to obtain the highest resolution tiles.

[0064] The expansion employs an adaptive expansion method: using an expansion ratio (expandRatio=0) or pixel buffer (pixelBuffer=0) as an initial value, the expansion amount is increased sequentially at a preset step size; after each expansion, a round-trip projection check is performed on the "target range, source coordinate reference system, and target coordinate reference system". If the range after the round-trip projection still cannot contain the original target tile range (or its range after expansion according to the resampling kernel radius), the expansion amount continues to increase until the inclusion relationship is satisfied or the maximum expansion threshold is reached; this adaptive expansion enables the coverage of the projection deformation and the influence of the resampling boundary, avoiding missing tile edges.

[0065] In one possible implementation, the method further includes:

[0066] For subtrees, only leaf layer tiles are cached in memory, and intermediate layer tiles above the leaf layer are released immediately after generation. Output is only performed on the target level to be persisted, avoiding long-term residence of intermediate results across layers.

[0067] Optionally, caching only leaf layer tiles serves the following purpose: using leaf layer tiles as the sole input for bottom-up aggregation within the subtree, allowing parent layer tiles to be aggregated and output immediately after their child tile inputs are ready; releasing the corresponding child tile cache after outputting the parent layer tile, thereby avoiding long-term resident status of intermediate results across layers and controlling memory peaks to a constant level related to the subtree height d. To avoid ambiguity, the leaf layer tile refers to the leaf layer tile used as input to the subtree task, preferably a tile at the level of the subtree root node (denoted as z_leaf, equal to the scaling level of the subtree root node), whose tile key is the level (z_leaf), column number (col), and row number (row); the leaf layer tile is not limited to the tile of the globally highest resolution layer.

[0068] Optionally, the pre-cached leaf layer tiles are used for data handover between parent and child trees: under the depth-first scheduling order, the child tree completes before the parent and child trees and caches its child tree root tiles in memory; when the parent and child trees are executed, they retrieve the child tree root tiles from the memory cache according to the tile key set and assemble them into the input layer, and then perform upsampling / aggregation to generate the upper layer tiles, avoiding disk readback of the generated tiles.

[0069] Optionally, the memory cache adopts a memory key-value mapping with (zoom,col,row) as the key and tile pixel data as the value; after the parent and child trees complete the upper-level aggregation, the corresponding subtree root tile is evicted from the cache according to the tile key set to control the cache peak.

[0070] The method also includes: during the reading process of the lowest level subtree, a batch reading and spatial sequential reading strategy is adopted. First, adjacent tiles are aggregated at a higher level to form a batch reading unit. Then, the reading keys are sorted according to the spatial filling curve order (such as Z-order) and read and sliced ​​sequentially to improve the spatial locality of I / O access and reduce throughput jitter.

[0071] The principle of spatial sequential reading is to map two-dimensional tile coordinates to one-dimensional sequence numbers and execute readings in order of these sequence numbers, making the access order of adjacent spatial tiles more continuous and reducing random addressing of disks or remote storage. Its purpose is to stabilize the throughput of the lowest-level subtree in the source data-projection-slicing stage. The reason is that the number of read requests in this stage is large and the span is wide. If the access order is discrete, it will lead to a decrease in cache hit rate and fluctuation in read performance.

[0072] Optionally, the tile construction process performs global statistics and linear stretching on non-byte type images, calculates global min / max and performs linear stretching to uniformly output a pixel value range of 0-255.

[0073] In one specific embodiment, to verify the pyramid construction process and effect of the method described in this application on real raster images, three TIFF format raster images were used for testing. The test equipment was configured with an Intel Core Ultra 7 255H processor (2.00 GHz), 32 GB DDR5 memory (8533 MT / s), and an Intel Arc 140T graphics card (16 GB video memory), running a Windows operating system optimized based on Atlas Playbook v0.5.0. The basic information of the test images is shown in Table 1.

[0074] Table 1

[0075] This embodiment employs the method described in this application, SuperMap's image tile generation tool, ArcGIS Pro's map tile creation tool, and GeoServer's tile tool to construct the pyramid. Each method uses PNG tile format, a tile size of 256×256 pixels, 16 processing threads, and output layers from layer 6 to layer 17. A complete end-to-end construction is performed once on the same test device. Each test starts timing from the start of pyramid construction until all target tiles are output. The method described in this application uses the Web Mercator coordinate system as the target reference coordinate system and sets the fixed subtree height d to 9.

[0076] Taking data 1.tif as an example, the actual pyramid construction process of the method described in this application embodiment is as follows: First, execute step S101 to read the image metadata of data 1.tif, obtain the image range, source coordinate system, number of pixel columns 72989, number of pixel rows 39861 and number of bands 4, and convert the image range to the Web Mercator coordinate system to obtain the target range. The actual maximum level is determined to the preset 17th level, and the minimum output level is determined to the 6th level.

[0077] Next, step S102 is executed. For each level from the 6th to the 17th layer, the range of tile row and column numbers that intersect with the target range is calculated, a set of covering tiles is generated, and a quadtree structure containing only intersecting tiles is constructed from top to bottom according to the parent-child relationship between tiles.

[0078] Then, step S103 is executed, and the quadtree is split according to the fixed subtree height d=9. The 9th to 17th levels are divided into subtrees containing the highest resolution level, and the remaining 6th to 8th levels are divided into corresponding parent and child trees.

[0079] Finally, step S104 is executed, using a depth-first scheduling order. First, the subtree containing layers 9 to 17 is executed. Within this subtree, 16 processing threads are used in parallel to read, project, and slice the source image window, obtaining the highest-resolution tile for layer 17. Then, tiles for each upper layer are generated from bottom to top according to the quadtree parent-child relationship. After the upper-layer tiles are generated and output, their corresponding lower-layer tile cache is released. After the subtree containing the highest-resolution layer is executed, the corresponding parent-child trees from layers 6 to 8 are executed. The root tiles of the subtrees cached in memory are used to continue bottom-up hierarchical aggregation, finally outputting PNG format pyramid tiles from layers 6 to 17. Data 2.tif and Data 3.tif are constructed using the same process.

[0080] The time taken for a single complete construction for each method is shown in Table 2.

[0081] Table 2

[0082] The time reduction percentage was calculated as "(time taken by the comparison tool - time taken by the method in this application) / time taken by the comparison tool × 100%": Compared to SuperMap, the construction time for data 1.tif, data 2.tif, and data 3.tif by the method in this application was reduced by approximately 65.4%, 86.7%, and 79.4%, respectively; compared to ArcGIS Pro, the construction time was reduced by approximately 36.3%, 59.9%, and 69.1%, respectively; and for data 1.tif, the construction time was reduced by approximately 94.8% compared to GeoServer. Since GeoServer did not complete the pyramid construction for data 2.tif and data 3.tif, the time reduction percentage for these two sets of results was not calculated.

[0083] As can be seen from the test results above, under the hardware, data, and construction parameters specified in this embodiment, the method of this application successfully constructed pyramids from layer 6 to layer 17 for three real raster images, and the construction can still be completed as the image data capacity and pixel scale increase. This result demonstrates that the method of fixed-height subtree splitting, depth-first subtree scheduling, and parallel processing within subtrees, as used in this embodiment, can balance pyramid construction efficiency and execution stability under these conditions.

[0084] like Figure 2 As shown in the figure, this application provides a distributed pyramid construction device, including:

[0085] The initial processing module 201 is used to obtain image association information corresponding to raster image metadata, and to obtain the target range and maximum level in the target reference coordinate system based on the image association information.

[0086] The quadtree construction module 202 is used to generate a set of covering tiles according to the target range and the maximum level, and to construct a quadtree structure according to the set of covering tiles;

[0087] The subtree segmentation module 203 is used to mark the root node of the subtree on the quadtree structure according to a fixed subtree height, and to determine multiple subtrees based on the root node of the subtree;

[0088] The pyramid building module 204 is used to execute subtrees in a depth-first scheduling order, treating each subtree as an independent distributed computing task. Within each subtree, the distributed computing framework is used to process tiles in parallel. The subtree containing the highest resolution layer is executed first, followed by the corresponding parent and child trees, completing the bottom-up hierarchical aggregation to obtain the raster image pyramid.

[0089] The distributed pyramid construction device provided in this application embodiment can execute the above-described method and technical solution. Its principle and beneficial effects are similar, and will not be repeated here.

[0090] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0091] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0094] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0095] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A distributed pyramid construction method, characterized in that, include: Obtain the image association information corresponding to the raster image metadata, and obtain the target range and maximum level in the target reference coordinate system based on the image association information; A set of covering tiles is generated based on the target range and the maximum level, and a quadtree structure is constructed based on the set of covering tiles. On the quadtree structure, the root node of the subtree is marked with a fixed subtree height, and multiple subtrees are determined based on the root node of the subtree; The algorithm employs a depth-first scheduling order to execute subtrees one by one, treating each subtree as an independent distributed computing task. Within each subtree, the distributed computing framework is used to process tiles in parallel. The algorithm executes the subtree containing the highest resolution layer first, followed by the corresponding parent and child trees, thus completing the bottom-up hierarchical aggregation and obtaining the raster image pyramid.

2. The distributed pyramid construction method according to claim 1, characterized in that, Obtain image association information corresponding to raster image metadata, and obtain the target range and maximum level in the target reference coordinate system based on the image association information, including: Obtain the source image range, coordinate system, number of pixel columns, and number of pixel rows corresponding to the raster image metadata to obtain image association information; Based on the image association information, the target range and maximum level in the target reference coordinate system are obtained.

3. The distributed pyramid construction method according to claim 1, characterized in that, Generate a set of overlay tiles based on the target range and the maximum level, and construct a quadtree structure based on the set of overlay tiles, including: A quadtree is constructed from top to bottom using a queue method with L0 level as the root node; L0 level refers to the top level. For each level, the range of tile row and column numbers intersecting with the target range is calculated, and the tiles located within the tile row and column number range are constructed as a set of covering tiles. Quadtree nodes are generated only for the tiles in the set of covering tiles, and the quadtree continues to expand downward according to the parent-child relationship of the quadtree until the maximum level is reached, resulting in a quadtree structure that covers the target range and contains only intersecting tiles.

4. The distributed pyramid construction method according to claim 1, characterized in that, On the quadtree structure, the root node of the subtree is marked according to a fixed subtree height, and multiple subtrees are determined according to the root node of the subtree, including: based on the quadtree structure, a root node of the subtree is determined every d-1 layers, and a continuous d layers of tiles are regarded as a subtree to obtain multiple subtrees; where d is a pre-set positive integer.

5. The distributed pyramid construction method according to claim 1, characterized in that, The process is executed subtree by subtree using a depth-first scheduling order, including: The subtree is used as a distributed computing task. After one distributed computing task is completed, the next distributed computing task is executed to reduce the memory peak caused by parallel superposition. During the execution of the distributed computing task, the intermediate results are only cached in memory and not stored.

6. The distributed pyramid construction method according to claim 5, characterized in that, After a distributed computing task is completed, the next distributed computing task is executed. This includes: during the execution of a distributed computing task, caching the next distributed computing task in memory, and starting the next distributed computing task only after all computing nodes of the currently executed distributed computing task have completed and released memory.

7. The distributed pyramid construction method according to claim 1, characterized in that, Treat each subtree as an independent distributed computing task. Within each subtree, use a distributed computing framework to process tiles in parallel. This includes distributing the reading, projection, and slicing operations of each tile within the subtree to multiple computing nodes for parallel processing, and releasing the cache of the corresponding lower-level tile after any upper-level tile has been processed.

8. The distributed pyramid construction method according to claim 1, characterized in that, The process of reading the source image and constructing tiles in the lowest subtree of all subtrees also includes: determining the source data window based on the lowest subtree, expanding the source data window, finding the intersection of the expanded source data window with the source image range to obtain the source image set, and using the source image set to perform reading, projection, and slicing operations to obtain the highest resolution tile.

9. The distributed pyramid construction method according to claim 5, characterized in that, Also includes: For subtrees, only leaf layer tiles are cached in memory, and all intermediate layer tiles above the leaf layer are released immediately after generation. Output is only performed on the target level to be persisted. It also includes: employing batch reading and spatial sequential reading strategies during the reading of the lowest-level subtree.

10. A distributed pyramid construction device, characterized in that, include: The initial processing module is used to obtain image association information corresponding to raster image metadata, and to obtain the target range and maximum level in the target reference coordinate system based on the image association information. The quadtree construction module is used to generate a set of covering tiles based on the target range and the maximum level, and to construct a quadtree structure based on the set of covering tiles. A subtree segmentation module is used to mark the root node of a subtree at a fixed subtree height on the quadtree structure, and to determine multiple subtrees based on the root node of the subtree. The pyramid building module is used to execute subtrees in a depth-first scheduling order, treating each subtree as an independent distributed computing task. Within each subtree, the distributed computing framework is used to process tiles in parallel. The subtree containing the highest resolution layer is executed first, followed by the corresponding parent and child trees, completing the bottom-up hierarchical aggregation to obtain the raster image pyramid.

Images