Freehand sketching method and system

Abstract

This application relates to the field of geographic information data processing technology, providing a method and system for processing image data without tiling. This invention constructs a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, thereby adapting the physical storage layout of the data to the geographic density. Distributed data blocks are loaded into memory using the HDFS short-circuit read mechanism, bypassing the network protocol stack to eliminate network transmission overhead, thus improving the problem of limited I / O throughput caused by frequent kernel-mode data copying and bandwidth occupation. By performing distance-weighted feathering fusion on overlapping areas, smooth transition of pixel values ​​at dynamic stitching points of multi-source images is achieved. Through HDFS short-circuit read and pixel-level real-time dynamic rendering, instant publishing of massive images without preprocessing is realized, solving the problem of low data response timeliness in sudden scenarios.

Description

Technical Field

[0001] This application relates to the field of geographic information data processing technology, and in particular to a method and system for processing image data without tiling. Background Technology

[0002] With the rapid development of Earth observation technology, the means of acquiring remote sensing image data have become increasingly diverse, encompassing multiple sources such as high-resolution optical satellites, synthetic aperture radar, and UAV aerial surveys. This data exhibits characteristics of massive volume, multi-source heterogeneity, and high timeliness. In applications such as geographic information systems and emergency command, the core technological challenge lies in achieving the rapid dissemination and real-time visualization of massive amounts of raw image data.

[0003] However, most traditional technologies use offline pre-slicing models, which are time-consuming to preprocess, resulting in low data response timeliness in sudden scenarios. Summary of the Invention

[0004] Therefore, it is necessary to provide a method and system for processing image data without slicing, in order to address the aforementioned technical problems.

[0005] A method for processing slice-free image data includes the following steps:

[0006] S1. Construct a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, and cut the original image data into corresponding distributed data blocks, which are then stored in multiple computing nodes. S2. Traverse all distributed data blocks, extract spatial geographic ranges, and aggregate to generate a global multi-level spatial index; S3. Based on the input tile request, query the global multi-level spatial index that matches the tile request, obtain the corresponding distributed data block and computing node, and record the corresponding computing node as a candidate computing node; S4. Decompose the tile request into multiple sub-computation tasks, and distribute the sub-computation tasks to candidate computing nodes that store distributed data blocks based on the data locality principle. S5. Candidate computing nodes load distributed data blocks into memory through the HDFS short-circuit read mechanism, and perform pixel-level dynamic projection transformation in memory to generate local image data. S6. After performing coordinate alignment on all generated local image data, perform distance-weighted feathering fusion on the overlapping areas of the local image data to complete image stitching and generate target image tiles.

[0007] In one embodiment, S1 involves constructing a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, including: S11. Traverse the spatial center coordinates of all original image data in the current dataset, and calculate the first variance of the spatial center coordinates in the longitude dimension and the second variance in the latitude dimension respectively. S12. The dimension corresponding to the larger variance value in the first variance value and the second variance value is used as the spatial splitting dimension of the current level. S13. Obtain the median of the spatial center coordinates of the image data block in the spatial split dimension, and use it as the spatial split point; S14. Divide the current dataset into two subspace sets based on the spatial split point, and use the subspace sets as the dataset for the next iteration. Return to S11 until the number of image data blocks contained in the subspace sets does not exceed the preset threshold.

[0008] In one embodiment, all distributed data blocks are traversed, spatial geographic ranges are extracted, and aggregated to generate a global multi-level spatial index, including: Read and parse the file header metadata of all distributed data blocks to obtain the latitude and longitude range of the minimum bounding rectangle; The coordinates of the center point of the latitude and longitude range are mapped to a normalized interval, and the longitude binary code sequence and the latitude binary code sequence are recursively generated by the binary search method. Generate a one-dimensional binary string using the following formula:

[0009] in, The result is a one-dimensional binary value after interleaving and merging. The total number of bits in the encoded sequence. For the current bit index, The first dimensional binary encoded sequence in the latitude A binary value of 1 bit. For the longitude binary encoded sequence, the first... A binary value of 1 bit; The one-dimensional binary string is mapped to characters in the Base32 character set in groups of five, and a Geohash string is generated as the index key. The corresponding distributed data block identifier and data node IP address are stored as index values ​​in the key-value pair database.

[0010] In one embodiment, S3 involves querying the global multi-level spatial index matching the input tile request based on the input tile request, and obtaining the corresponding distributed data block and computing node, including: Based on the input standard tile request parameters, which include the level, column number, and row number, the minimum geospatial bounding rectangle range corresponding to the standard tile request parameters is calculated using Mercator projection. Calculate the Geohash encoding prefix corresponding to the minimum bounding rectangle range of the geospatial area, wherein the Geohash encoding prefix has the same length as the index key of the global multi-level spatial index; Based on the Geohash encoded prefix, a prefix matching query is performed in the global multi-level spatial index to obtain a list of matched distributed data block identifiers and their corresponding list of storage and computing node IP addresses.

[0011] In one embodiment, S4, distributing sub-computation tasks to candidate computing nodes storing distributed data blocks based on the data locality principle includes: Parse the target distributed data block ID corresponding to the sub-computation task, and obtain the list of physical node IPs where all replicas of the target distributed data block corresponding to the ID are located; Obtain real-time CPU utilization and memory usage data for each physical node in the physical node IP list, and select physical nodes with CPU utilization below a preset load threshold as target execution nodes; The task instruction containing the data block file path is sent to the target execution node, and the target execution node calls the HDFS short-circuit read interface to read the physical file block of the target distributed data block corresponding to the ID.

[0012] In one embodiment, S5 involves performing a pixel-level dynamic projection transformation in memory to generate local image data, including: The Mercator projection inverse calculation algorithm is used to convert the pixel coordinates in the target tile into the corresponding floating-point coordinates of the original image data. Using the integer part of the floating-point coordinates as the reference pixel coordinates, determine the pixel values ​​of four adjacent integer points in the original image data: top left, top right, bottom left, and bottom right. Using the fractional part of the floating-point coordinates as the distance weighting factor, a bilinear weighted summation operation is performed on the four integer pixel values ​​to obtain the pixel value of the target tile at the current pixel coordinates.

[0013] In one embodiment, after performing coordinate alignment on all generated local image data, distance-weighted feathering fusion is performed on the overlapping areas of the local image data to complete image stitching and generate target image tiles, including: Perform coordinate alignment on all generated local image data; The set of pixel coordinates that intersect in space between the first image layer and the second image layer requested by the tile is taken as the overlapping area; Calculate the first Euclidean distance from the current pixel to the boundary of the first image layer and the second Euclidean distance from the current pixel to the boundary of the second image layer within the overlapping region; Based on the first Euclidean distance and the second Euclidean distance, calculate the normalized weight coefficients for the first image layer:

[0014] in, These are the normalized weighting coefficients; This is the first Euclidean distance; This is the second Euclidean distance; Generate fused pixel values ​​based on normalized weight coefficients and complementary weights:

[0015] in, To merge pixel values; The complementary weights of the second image layer; This represents the original pixel value of the first image layer at the current coordinates; This represents the original pixel value of the second image layer at the current coordinates.

[0016] In one embodiment, the minimum bounding rectangle of the geospatial area corresponding to the standard tile request parameters is calculated according to the following formula: ;

[0017] in, The minimum longitude of the geographical range of the target tile; The maximum latitude of the geographical range of the target tile; For tile column numbering; For tile row numbers; For tile layers; Pi is a constant.

[0018] In one embodiment, the floating-point coordinates are converted according to the following formula: ;

[0019] in, The longitude value corresponding to the original image; The latitude value corresponding to the original image; and For the target tile at the level Normalized global pixel coordinates; Pi is a constant.

[0020] A slice-free image data processing system, comprising: The data segmentation module is used to construct a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, and to segment the original image data into corresponding distributed data blocks, which are then stored in multiple computing nodes. The spatial indexing module is used to traverse all distributed data blocks, extract spatial geographic ranges, and aggregate them to generate a global multi-level spatial index. The request matching module is used to query the global multi-level spatial index that matches the input tile request, obtain the corresponding distributed data block and computing node, and record the corresponding computing node as a candidate computing node. The task distribution module is used to decompose tile requests into multiple sub-computation tasks and distribute the sub-computation tasks to candidate computing nodes that store distributed data blocks based on the data locality principle. The local image generation module is used by candidate computing nodes to load distributed data blocks into memory through the HDFS short-circuit read mechanism, and perform pixel-level dynamic projection transformation in memory to generate local image data. The image tile generation module performs coordinate alignment on all generated local image data, then performs distance-weighted feathering fusion on the overlapping areas of the local image data to complete image stitching and generate target image tiles.

[0021] The above-mentioned non-slicing image data processing method and system constructs a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, thereby adapting the physical storage layout of the data to the geographical density. This solves the problem of unbalanced cluster storage and computing load caused by the traditional distributed storage method of using a fixed grid for uniform division and ignoring the differences in spatial distribution of data.

[0022] The HDFS short-circuit read mechanism loads distributed data blocks into memory, thereby bypassing the network protocol stack to eliminate network transmission overhead. This improves upon the traditional data reading method, which mostly uses standard network socket communication and suffers from limited I / O throughput due to frequent kernel-mode data copying and bandwidth consumption.

[0023] By performing distance-weighted feathering fusion on the overlapping areas, a smooth transition of pixel values ​​at the dynamic stitching point of multi-source images is achieved, avoiding the problems of physical seams and color banding in traditional image stitching due to the lack of dynamic weight processing for boundaries.

[0024] This invention enables the instant publishing of massive images without preprocessing by using HDFS short-circuit reading and pixel-level real-time dynamic rendering, thus solving the problem of low data response timeliness in sudden scenarios. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating a slice-free image data processing method in one embodiment; Figure 2 This is a block diagram of a slice-free image data processing system in one embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] The slice-free image data processing method provided by this invention is used in an image playback system, which may be a geographic information system, emergency command system, etc.

[0028] In one embodiment, such as Figure 1 As shown, a method for processing image data without slicing is provided, including the following steps: S1. Construct a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, and cut the original image data into corresponding distributed data blocks, which are then stored in multiple computing nodes. S2. Traverse all distributed data blocks, extract spatial geographic ranges, and aggregate to generate a global multi-level spatial index; S3. Based on the input tile request, query the global multi-level spatial index that matches the tile request, obtain the corresponding distributed data block and computing node, and record the corresponding computing node as a candidate computing node; S4. Decompose the tile request into multiple sub-computation tasks, and distribute the sub-computation tasks to candidate computing nodes that store distributed data blocks based on the data locality principle. S5. Candidate computing nodes load distributed data blocks into memory through the HDFS short-circuit read mechanism, and perform pixel-level dynamic projection transformation in memory to generate local image data. S6. After performing coordinate alignment on all generated local image data, perform distance-weighted feathering fusion on the overlapping areas of the local image data to complete image stitching and generate target image tiles.

[0029] The aforementioned non-tiled image data processing method constructs a non-uniform distributed storage structure based on the spatial distribution density characteristics of the original image data. This adapts the physical storage layout of the data to the geographical density, solving the problem of unbalanced cluster storage and computational load caused by the fixed grid uniform division and neglect of spatial distribution differences in traditional distributed storage. The HDFS short-circuit read mechanism loads distributed data blocks into memory, bypassing the network protocol stack to eliminate network transmission overhead. This improves upon the problem of limited I / O throughput caused by frequent kernel-mode data copying and bandwidth occupation, which is largely achieved through standard network socket communication in traditional data reading. By performing distance-weighted feathering fusion on overlapping areas, a smooth transition of pixel values ​​at dynamic stitching points in multi-source images is achieved, avoiding the physical seams and color banding problems caused by the lack of dynamic weight processing for boundaries in traditional image stitching.

[0030] In one embodiment, S1 involves constructing a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, including: S11. Traverse the spatial center coordinates of all original image data in the current dataset, and calculate the first variance of the spatial center coordinates in the longitude dimension and the second variance in the latitude dimension respectively. S12. The dimension corresponding to the larger variance value in the first variance value and the second variance value is used as the spatial splitting dimension of the current level. S13. Obtain the median of the spatial center coordinates of the image data block in the spatial split dimension, and use it as the spatial split point; S14. Divide the current dataset into two subspace sets based on the spatial split point, and use the subspace sets as the dataset for the next iteration. Return to S11 until the number of image data blocks contained in the subspace sets does not exceed the preset threshold.

[0031] Understandably, the input data for the image playback system is the set of spatial center coordinates of the original remote sensing image data blocks to be stored. Specifically, the first variance and the second variance are calculated according to the following formula:

[0032] in, This represents the variance value for the current dimension. This represents the total number of image data blocks in the current dataset. For the first The coordinate values ​​of each image data block in the current dimension; This represents the mean of the coordinates of the current dataset in the current dimension. After obtaining the first and second variance values, the video playback system compares the variance values ​​and selects the dimension with the larger variance value as the spatial splitting dimension for the current level.

[0033] A large variance indicates a high degree of dispersion in the spatial distribution of data along the corresponding dimension. Therefore, cutting along the spatial split dimension can more effectively reduce the geographical span of the subspace.

[0034] After obtaining the spatial split points, the current dataset is divided into two equally sized subspace sets. These subspace sets are then used as the dataset for the next iteration, and the variance calculation, spatial split dimension selection, and spatial split point acquisition are repeated until the number of image data blocks contained within each subspace set does not exceed a preset threshold. The final output is a distributed storage structure. The original image data is cut into corresponding distributed data blocks and mapped and stored to the corresponding distributed computing nodes.

[0035] In this embodiment, a density-based non-uniform partitioning strategy is adopted to solve the load imbalance problem caused by the uneven spatial distribution of remote sensing image data when traditional fixed grid partitioning is used. By selecting the dimension with the largest variance and the median for splitting, it is ensured that the amount of data contained in each generated leaf node is approximately equal, thus achieving physical load balancing of data in the distributed storage cluster. This storage structure adapts the physical layout of the data to the geographic spatial density, reduces the amount of irrelevant data scanning during subsequent spatial queries, and reduces cross-node I / O overhead.

[0036] In one embodiment, all distributed data blocks are traversed, spatial geographic ranges are extracted, and aggregated to generate a global multi-level spatial index, including: Read and parse the file header metadata of all distributed data blocks to obtain the latitude and longitude range of the minimum bounding rectangle; The coordinates of the center point of the latitude and longitude range are mapped to a normalized interval, and the longitude binary code sequence and the latitude binary code sequence are recursively generated by the binary search method. Generate a one-dimensional binary string using the following formula:

[0037] in, The result is a one-dimensional binary value after interleaving and merging. The total number of bits in the encoded sequence. For the current bit index, The first dimensional binary encoded sequence in the latitude A binary value of 1 bit. For the longitude binary encoded sequence, the first... A binary value of 1 bit; The one-dimensional binary string is mapped to characters in the Base32 character set in groups of five, and a Geohash string is generated as the index key. The corresponding distributed data block identifier and data node IP address are stored as index values ​​in the key-value pair database.

[0038] In this embodiment, by performing Z-order curve bit error correction operations on the longitude binary encoded sequence and the latitude binary encoded sequence, the spatial filling characteristics of the Z-order curve are utilized to realize the mapping from geographic spatial proximity to one-dimensional encoded proximity.

[0039] It should be noted that geographically adjacent data blocks have the same or similar Geohash prefixes. During the service retrieval phase, the system only needs to calculate the encoded prefix of the requested range to directly locate the target data block and its physical storage node in the key-value database through prefix matching. This eliminates the need to traverse the full data metadata, reducing the time complexity of spatial queries to constant levels and solving the retrieval latency problem in scenarios with massive distributed small files.

[0040] In one embodiment, S3 involves querying the global multi-level spatial index matching the input tile request based on the input tile request, and obtaining the corresponding distributed data block and computing node, including: Based on the input standard tile request parameters, which include the level, column number, and row number, the minimum geospatial bounding rectangle range corresponding to the standard tile request parameters is calculated using Mercator projection. Calculate the Geohash encoding prefix corresponding to the minimum bounding rectangle range of the geospatial area, wherein the Geohash encoding prefix has the same length as the index key of the global multi-level spatial index; Based on the Geohash encoded prefix, a prefix matching query is performed in the global multi-level spatial index to obtain a list of matched distributed data block identifiers and their corresponding list of storage and computing node IP addresses.

[0041] Specifically, standard tile requests are initiated by the client, and the standard tile request parameters include the tile level. Tile Column Number and tile row number The minimum bounding rectangle of the geospatial area corresponding to the standard tile request parameters is calculated using the following formula: ;

[0042] in, The minimum longitude of the geographical range of the target tile; The maximum latitude of the geographical range of the target tile; For tile column numbering; For tile row numbers; For tile layers; Pi is a constant. By substituting... and The system calculates the corresponding maximum longitude. and minimum latitude This allows us to determine the complete geographic bounding rectangle.

[0043] In this embodiment, a direct mapping channel is established between logical tile requests and physical data storage. Through Mercator inverse calculation and Geohash prefix matching, the system can quickly convert user access requests for specific map views into physical addresses of underlying HDFS distributed data blocks without traversing the entire file system's metadata. This mechanism reduces the time complexity of data retrieval to constant level (O(1)), providing accurate physical address basis for subsequent computational task scheduling and short-circuit reading.

[0044] In one embodiment, S4, distributing sub-computation tasks to candidate computing nodes storing distributed data blocks based on the data locality principle includes: Parse the target distributed data block ID corresponding to the sub-computation task, and obtain the list of physical node IPs where all replicas of the target distributed data block corresponding to the ID are located; Obtain real-time CPU utilization and memory usage data for each physical node in the physical node IP list, and select physical nodes with CPU utilization below a preset load threshold as target execution nodes; The task instruction containing the data block file path is sent to the target execution node, and the target execution node calls the HDFS short-circuit read interface to read the physical file block of the target distributed data block corresponding to the ID.

[0045] Specifically, the system resolves the unique identifier of the target distributed data block that each sub-computation task needs to process. By accessing the metadata service of the distributed file system, i.e., the NameNode, the system queries and obtains a list of physical IP addresses of the compute nodes that store all physical copies of that data block.

[0046] The system calls the cluster monitoring interface to collect real-time resource usage metrics for each physical node in the IP address list. Node selection follows the load constraint logic formula: ; in This indicates the selected target execution nodes. Represents the first in the cluster One physical node This represents the set of nodes that store replicas of the target data block. Indicates the first Real-time CPU utilization values ​​for each node. This indicates the preset load threshold. High-load nodes are filtered out, and nodes with idle resources and holding data are prioritized.

[0047] After identifying the target execution node, the scheduling module sends it a task instruction containing the local file path of the data blocks. The target execution node responds to the instruction and starts a read process configured with a short-circuit read mechanism. This process bypasses the TCP / IP network protocol stack of the DataNode service and directly obtains the file descriptor from the local operating system. Through this file descriptor, the data block files on the physical disk are directly mapped and loaded into the Java Virtual Machine heap memory of the computing process.

[0048] In this embodiment, by forcibly scheduling computing tasks to the location of data replicas and combining this with load filtering, the network bandwidth consumption caused by transmitting massive amounts of raw remote sensing data across nodes is eliminated. By utilizing short-circuit read interfaces, multiple copies of data between the DataNode network service kernel mode and user mode are avoided, reducing data loading latency from the network transmission level to the local disk I / O or memory copy level, thereby improving the throughput of the pixel processing pipeline. This implements a localized computing strategy that directs computing power towards the data.

[0049] In one embodiment, S5 involves performing a pixel-level dynamic projection transformation in memory to generate local image data, including: The Mercator projection inverse calculation algorithm is used to convert the pixel coordinates in the target tile into the corresponding floating-point coordinates of the original image data. Using the integer part of the floating-point coordinates as the reference pixel coordinates, determine the pixel values ​​of four adjacent integer points in the original image data: top left, top right, bottom left, and bottom right. Using the fractional part of the floating-point coordinates as the distance weighting factor, a bilinear weighted summation operation is performed on the four integer pixel values ​​to obtain the pixel value of the target tile at the current pixel coordinates.

[0050] It should be noted that the target tile is the tile corresponding to the tile request.

[0051] Specifically, for each pixel in the target tile, the processor calls the Mercator projection inverse algorithm to map the pixel position in the Web Mercator coordinate system back to the geographic coordinate system of the original image data, and then converts it into the corresponding floating-point coordinates in the original image matrix. The floating-point coordinates are converted according to the following formula: ;

[0052] in, The longitude value corresponding to the original image; The latitude value corresponding to the original image; and For the target tile at the level Normalized global pixel coordinates; Pi is a constant. After calculating the latitude and longitude, the system uses the geographic transformation parameters of the original imagery to... Convert to row and column numbers in the original image pixel matrix ,Should It is a floating-point value that includes a decimal part.

[0053] Obtain the corresponding floating-point coordinates Then, extract its integer part. As the reference pixel coordinates, extract their decimal part. and As a distance weighting factor. Locking in the original image data and Four adjacent integer points: top left Top right Bottom left and lower right Perform a bilinear weighted summation on the four integer pixel values ​​to obtain the pixel value of the target tile at the current pixel coordinates:

[0054] in, The calculated target pixel value; , , , These are the pixel values ​​of the four neighboring points in the original image: top left, top right, bottom left, and bottom right. and It represents the inverse weight of the distance between the floating-point coordinates and the corresponding integer points.

[0055] In this embodiment, a real-time pixel-level mapping from the original data block to the target tile is achieved through a full-memory dynamic inverse calculation and resampling mechanism. No intermediate temporary files are generated during the process; the data loaded into memory is directly read via short-circuit reading using HDFS (Hadoop Distributed File System) for streaming computation. The bilinear interpolation algorithm uses distance weights to estimate pixel values ​​at non-integer coordinates, eliminating image jaggedness and blockiness caused by resolution differences, ensuring that dynamically generated image tiles maintain visual smoothness and spatial geometric accuracy at any scaling level.

[0056] In one embodiment, after performing coordinate alignment on all generated local image data, distance-weighted feathering fusion is performed on the overlapping areas of the local image data to complete image stitching and generate target image tiles, including: Perform coordinate alignment on all generated local image data; The set of pixel coordinates that intersect in space between the first image layer and the second image layer requested by the tile is taken as the overlapping area; Calculate the first Euclidean distance from the current pixel to the boundary of the first image layer and the second Euclidean distance from the current pixel to the boundary of the second image layer within the overlapping region; Based on the first Euclidean distance and the second Euclidean distance, calculate the normalized weight coefficients for the first image layer:

[0057] in, These are the normalized weighting coefficients; This is the first Euclidean distance; This is the second Euclidean distance; Generate fused pixel values ​​based on normalized weight coefficients and complementary weights:

[0058] in, To merge pixel values; The complementary weights of the second image layer; This represents the original pixel value of the first image layer at the current coordinates; This represents the original pixel value of the second image layer at the current coordinates.

[0059] Specifically, the master node receives all generated local image data and performs panoramic alignment based on their respective spatial coordinate information. Within the geographical area covered by the tile service request, the processor identifies the set of pixel coordinates where the first image layer and the second image layer intersect in space, and uses this set of pixel coordinates as the overlapping area. For each current pixel within the overlapping pixel region, calculate the first Euclidean distance from the current pixel to the boundary of the first image layer and the second Euclidean distance from the current pixel to the boundary of the second image layer. The first Euclidean distance is defined as the straight-line distance from the pixel to the effective boundary of the first image layer, and the second Euclidean distance is defined as the straight-line distance from the pixel to the effective boundary of the second image layer.

[0060] The calculated normalized weight coefficients ensure that the farther a pixel is from the boundary of the first image layer, the greater the weight of the first image layer during fusion.

[0061] In this embodiment, a dynamic feathering algorithm based on boundary distance solves the problem of physical seams and color banding caused by differences in illumination or sensor parameters during dynamic stitching of multi-source remote sensing images. By constructing a weight field that smoothly varies with distance within the overlapping area, a seamless transition of pixel values ​​from one image layer to another is achieved. The processing eliminates visually hard cutting boundaries, ensuring the color continuity and spatial consistency of the output image when the slice-free system aggregates multiple scattered data in real time.

[0062] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0063] In one embodiment, such as Figure 2 As shown, a slice-free image data processing system is provided, comprising: The data segmentation module 901 is used to construct a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, and to segment the original image data into corresponding distributed data blocks, which are then stored in multiple computing nodes. Spatial index module 902 is used to traverse all distributed data blocks, extract spatial geographic ranges, and aggregate to generate a global multi-level spatial index; The request matching module 903 is used to query the global multi-level spatial index that matches the input tile request, obtain the corresponding distributed data block and computing node, and record the corresponding computing node as a candidate computing node. The task distribution module 904 is used to decompose the tile request into multiple sub-computation tasks and distribute the sub-computation tasks to candidate computing nodes that store distributed data blocks based on the data locality principle. The local image generation module 905 is used by candidate computing nodes to load distributed data blocks into memory through the HDFS short-circuit read mechanism, and perform pixel-level dynamic projection transformation in memory to generate local image data. The image tile generation module 906 is used to perform coordinate alignment on all generated local image data, and then perform distance-weighted feathering fusion on the overlapping areas of the local image data to complete image stitching and generate target image tiles.

[0064] Specific limitations regarding the non-tiling image data processing system can be found in the limitations of the non-tiling image data processing method described above, and will not be repeated here. Each module in the aforementioned non-tiling image data processing system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing image data without slicing, characterized in that, Includes the following steps: S1. Construct a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, and cut the original image data into corresponding distributed data blocks, which are then stored in multiple computing nodes. S2. Traverse all distributed data blocks, extract spatial geographic ranges, and aggregate to generate a global multi-level spatial index; S3. Based on the input tile request, query the global multi-level spatial index that matches the tile request, obtain the corresponding distributed data block and computing node, and record the corresponding computing node as a candidate computing node; S4. Decompose the tile request into multiple sub-computation tasks, and distribute the sub-computation tasks to candidate computing nodes that store distributed data blocks based on the data locality principle. S5. Candidate computing nodes load distributed data blocks into memory through the HDFS short-circuit read mechanism, and perform pixel-level dynamic projection transformation in memory to generate local image data. S6. After performing coordinate alignment on all generated local image data, perform distance-weighted feathering fusion on the overlapping areas of the local image data to complete image stitching and generate target image tiles.

2. The method for processing slice-free image data according to claim 1, characterized in that, In S1, a non-uniform distributed area storage structure is constructed based on the spatial distribution density characteristics of the original image data, including: S11. Traverse the spatial center coordinates of all original image data in the current dataset, and calculate the first variance of the spatial center coordinates in the longitude dimension and the second variance in the latitude dimension respectively. S12. The dimension corresponding to the larger variance value in the first variance value and the second variance value is used as the spatial splitting dimension of the current level. S13. Obtain the median of the spatial center coordinates of the image data block in the spatial split dimension, and use it as the spatial split point; S14. Divide the current dataset into two subspace sets based on the spatial split point, and use the subspace sets as the dataset for the next iteration. Return to S11 until the number of image data blocks contained in the subspace sets does not exceed the preset threshold.

3. The method for processing slice-free image data according to claim 1, characterized in that, Traverse all distributed data blocks, extract spatial geographic extents, and aggregate them to generate a global multi-level spatial index, including: Read and parse the file header metadata of all distributed data blocks to obtain the latitude and longitude range of the minimum bounding rectangle; The coordinates of the center point of the latitude and longitude range are mapped to a normalized interval, and the longitude binary code sequence and the latitude binary code sequence are recursively generated by the binary search method. Generate a one-dimensional binary string using the following formula: in, The result is a one-dimensional binary value after interleaving and merging. The total number of bits in the encoded sequence. For the current bit index, The first dimensional binary encoded sequence in the latitude A binary value of 1 bit. For the longitude binary encoded sequence, the first... A binary value of 1 bit; The one-dimensional binary string is mapped to characters in the Base32 character set in groups of five, and a Geohash string is generated as the index key. The corresponding distributed data block identifier and data node IP address are stored as index values ​​in the key-value pair database.

4. The method for processing slice-free image data according to claim 1, characterized in that, In S3, based on the input tile request, the global multi-level spatial index matching the tile request is queried to obtain the corresponding distributed data block and computing node, including: Based on the input standard tile request parameters, which include the level, column number, and row number, the minimum geospatial bounding rectangle range corresponding to the standard tile request parameters is calculated using Mercator projection. Calculate the Geohash encoding prefix corresponding to the minimum bounding rectangle range of the geospatial area, wherein the Geohash encoding prefix has the same length as the index key of the global multi-level spatial index; Based on the Geohash encoded prefix, a prefix matching query is performed in the global multi-level spatial index to obtain a list of matched distributed data block identifiers and their corresponding list of storage and computing node IP addresses.

5. The method for processing slice-free image data according to claim 1, characterized in that, In S4, based on the principle of data locality, sub-computation tasks are distributed to candidate computation nodes that store distributed data blocks, including: Parse the target distributed data block ID corresponding to the sub-computation task, and obtain the list of physical node IPs where all replicas of the target distributed data block corresponding to the ID are located; Obtain real-time CPU utilization and memory usage data for each physical node in the physical node IP list, and select physical nodes with CPU utilization below a preset load threshold as target execution nodes; The task instruction containing the data block file path is sent to the target execution node, and the target execution node calls the HDFS short-circuit read interface to read the physical file block of the target distributed data block corresponding to the ID.

6. The method for processing slice-free image data according to claim 1, characterized in that, In S5, pixel-level dynamic projection transformation is performed in memory to generate local image data, including: The Mercator projection inverse calculation algorithm is used to convert the pixel coordinates in the target tile into the corresponding floating-point coordinates of the original image data. Using the integer part of the floating-point coordinates as the reference pixel coordinates, determine the pixel values ​​of four adjacent integer points in the original image data: top left, top right, bottom left, and bottom right. Using the fractional part of the floating-point coordinates as the distance weighting factor, a bilinear weighted summation operation is performed on the four integer pixel values ​​to obtain the pixel value of the target tile at the current pixel coordinates.

7. The method for processing slice-free image data according to claim 1, characterized in that, After performing coordinate alignment on all generated local image data, distance-weighted feathering fusion is performed on the overlapping areas of the local image data to complete image stitching and generate target image tiles, including: Perform coordinate alignment on all generated local image data; The set of pixel coordinates that intersect in space between the first image layer and the second image layer requested by the tile is taken as the overlapping area; Calculate the first Euclidean distance from the current pixel to the boundary of the first image layer and the second Euclidean distance from the current pixel to the boundary of the second image layer within the overlapping region; Based on the first Euclidean distance and the second Euclidean distance, calculate the normalized weight coefficients for the first image layer: in, These are the normalized weighting coefficients; This is the first Euclidean distance; This is the second Euclidean distance; Generate fused pixel values ​​based on normalized weight coefficients and complementary weights: in, To merge pixel values; The complementary weights of the second image layer; This represents the original pixel value of the first image layer at the current coordinates; This represents the original pixel value of the second image layer at the current coordinates.

8. The method for processing slice-free image data according to claim 4, characterized in that, The minimum bounding rectangle of the geospatial area corresponding to the standard tile request parameters is calculated using the following formula: ； in, The minimum longitude of the geographical range of the target tile; The maximum latitude of the geographical range of the target tile; For tile column numbering; For tile row numbers; For tile layers; Pi is a constant.

9. The method for processing slice-free image data according to claim 6, characterized in that, The floating-point coordinates are converted according to the following formula: ； in, The longitude value corresponding to the original image; The latitude value corresponding to the original image; and For the target tile at the level Normalized global pixel coordinates; Pi is a constant.

10. A slice-free image data processing system, characterized in that, include: The data segmentation module is used to construct a non-uniform distributed area storage structure based on the spatial distribution density characteristics of the original image data, and to segment the original image data into corresponding distributed data blocks, which are then stored in multiple computing nodes. The spatial indexing module is used to traverse all distributed data blocks, extract spatial geographic ranges, and aggregate them to generate a global multi-level spatial index. The request matching module is used to query the global multi-level spatial index that matches the input tile request, obtain the corresponding distributed data block and computing node, and record the corresponding computing node as a candidate computing node. The task distribution module is used to decompose tile requests into multiple sub-computation tasks and distribute the sub-computation tasks to candidate computing nodes that store distributed data blocks based on the data locality principle. The local image generation module is used by candidate computing nodes to load distributed data blocks into memory through the HDFS short-circuit read mechanism, and perform pixel-level dynamic projection transformation in memory to generate local image data. The image tile generation module performs coordinate alignment on all generated local image data, then performs distance-weighted feathering fusion on the overlapping areas of the local image data to complete image stitching and generate target image tiles.

Images