A multi-threaded parallel processing and slicing generation method based on geographic image data

Abstract

The present application relates to a multi-thread parallel processing and tile generation method based on geographic image data, which includes the following steps: constructing a geographic image data set, obtaining the width, height, projection information and geographic transformation parameters of the images in the geographic image data set, accurately converting the source projection into the target projection coordinate system, and calculating the unit pixel value UPP; setting and configuring the environmental parameters for multi-thread processing, controlling the maximum number of threads in the thread pool, and creating a thread pool; processing the geographic images in the thread pool to generate tiles; storing the generated tiles according to a hierarchical directory structure for easy access and management. The present invention can achieve multi-thread parallel processing, perform geographic spatial data operations, and realize the rapid conversion from a single large geographic image to a multi-level pyramid tile map.

Description

Technical Field

[0001] This application relates to the technical field of geographic image processing, and particularly to a multi-threaded parallel processing and tile generation method based on geographic image data. Background Art

[0002] In the fields of geographic information systems and remote sensing image processing, the processing of large-scale geographic image data is a common and very important task. With the rapid development of remote sensing technology, the volume of image data has increased significantly, reaching dozens or even hundreds of GB. Therefore, existing image tile processing technologies are facing huge challenges, specifically including:

[0003] (1) Coordinate system conversion accuracy: Some existing tools have accuracy losses when performing coordinate conversion, especially when dealing with non-standard coordinate systems; this will cause the generated tiles to be misaligned or deformed when spliced, greatly affecting the overall quality of the map.

[0004] (2) Processing efficiency issue: Most existing tile software still uses single-threaded processing methods, with low processing efficiency.

[0005] (3) Insufficient adaptability: Existing software usually has difficulty automatically adjusting processing strategies according to different hardware configurations, such as the number of CPU cores and available memory, which will result in the inability to fully utilize system resources.

[0006] Facing these challenges, there is an urgent need for a solution that can efficiently process large-scale geographic image data, perform accurate coordinate conversion, flexibly manage system resources, and generate high-quality multi-level tile maps. Summary of the Invention

[0007] The purpose of the present invention is to provide a multi-threaded parallel processing and tile generation method based on geographic image data, which can achieve multi-threaded parallel processing, perform geographic spatial data operations, and realize the rapid conversion from a single large-scale geographic image to a multi-level pyramid tile map.

[0008] The technical solution adopted by the present invention is: A multi-threaded parallel processing and tile generation method based on geographic image data, including the following steps:

[0009] S1: Construct a geographic image data set, obtain the width, height, projection information, and geographic transformation parameters of the images in the geographic image data set, accurately convert the source projection to the target projection coordinate system, and calculate the value per pixel unit (UPP).

[0010] S2: Set the driver and configure the environmental parameters for multi-threaded processing, control the maximum number of threads in the thread pool, and create a thread pool.

[0011] S3: Process the geospatial image in the thread pool to generate tiles. When generating tiles, it is necessary to calculate the corresponding area of the current tile in the original image, read the data of the corresponding area from the original image, and perform resampling. The resampling process is based on the per-pixel unit value UPP of different standard map zoom levels to ensure the geographical accuracy of the generated tiles.

[0012] S4: Store the generated tiles according to the hierarchical directory structure for easy access and management.

[0013] Further, the specific steps of step S3 are as follows:

[0014] S301: Calculate the number of tiles for each layer and the geographical coordinate range corresponding to each tile based on the minimum zoom level, maximum zoom level, and pixel resolution of the geospatial image data input by the user. The specific calculation formulas are as follows:

[0015] hNo = (lon + 180.0) × scale / 360.0;

[0016] vNo = (90.0 + lat) × scale / 360.0;

[0017] where hNo represents the horizontal number of the tile, vNo represents the vertical number of the tile, lon represents the latitude of the upper left corner of the tile, lat represents the longitude of the upper left corner of the tile, and scale represents the zoom ratio.

[0018] S302: Generate tiles through the thread pool and use multi-threading to process tile slicing. When generating each tile, first obtain the semaphore for controlling multi-threading to control the concurrency number.

[0019] S303: In the specific slicing method, use the mutex QMutexLocker to protect shared resources. Within the scope of the mutex QMutexLocker, any thread attempting to obtain the same mutex will be blocked until the current thread releases the mutex.

[0020] S304: Read the geographical information data such as longitude and latitude of the corresponding area of the slice in the source dataset and write the read data into the raster data buffer of the target tile. The corresponding area is based on the per-pixel unit ratio UPP of each zoom level, and the reading area is dynamically adjusted through the zoom adjustment mechanism to ensure consistent geographical accuracy at each zoom level. After reading, close the source dataset and release the resources.

[0021] S305: Unlock the mutex and release the semaphore to allow other threads to continue executing the tile generation task. Use atomic operations to update the number of completed tiles and notify the main thread to update the processing progress.

[0022] S306: Set correct geotransform parameters for each generated tile, and adjust the geotransform parameters of the target geographic image dataset of the tile to be generated, so that the positions of the tiles in the target projection coordinate system are accurate. The calculation formula for the new geotransform parameters is as follows:

[0023] newGeoTransform[0] = lat;

[0024] newGeoTransform[1] = geoTransform[1]* tileWidth / 256.0;

[0025] newGeoTransform[3] = lon;

[0026] newGeoTransform[5] = geoTransform[5]* tileHeight / 256.0;

[0027] Among them, tileWidth represents the horizontal resolution of a single tile, tileHeight represents the vertical resolution of a single tile, newGeoTransform[0] represents the X coordinate of the upper-left pixel of the new tile, and newGeoTransform[1] represents the pixel width of the new tile; geoTransform[1] represents the pixel width of the old tile; newGeoTransform[3] represents the Y coordinate of the upper-left pixel of the new tile; newGeoTransform[5] represents the pixel height of the new tile, and geoTransform[5] represents the pixel height of the old tile;

[0028] S307: Set the projection information of the target geographic image dataset of the tile to be generated to ensure that all tiles are in the same projection coordinate system as the geographic image dataset; use the function for setting projection information to apply the projection information of the geographic image dataset to all target tiles.

[0029] The beneficial effects of the present invention are as follows:

[0030] (1) The present invention adopts a multi-threaded parallel processing technology, which makes full use of the computing power of multi-core processors. Through actual measurement, it is shown that in a 16-core CPU environment, the processing speed is about 3 to 4 times higher than that of the single-threaded method; through block reading, large image data can be effectively processed; and by using the high-precision coordinate conversion function of the GDAL library, the positioning accuracy is significantly improved compared with the traditional method; the present invention can automatically adjust the parallelism according to the current hardware configuration and achieve accurate progress calculation based on task granularity, and the accuracy of progress display reaches more than 99%;

[0031] (2) The present invention proposes a high-precision coordinate conversion system, which supports complex projection conversions, ensures accurate mapping between different coordinate systems, realizes the mutual conversion of custom longitude and latitude and tile coordinates, and improves the efficiency and accuracy of coordinate transformation. At the same time, the present invention adopts an adaptive zoom level calculation method based on map resolution to ensure consistent geographical accuracy at different zoom levels and realizes the correlation mapping between zoom levels and pixel resolutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0033] Figure 1 It is a flowchart of the method for the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be further described in detail below with reference to the drawings and specific embodiments. Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited by the limitations of the specific embodiments disclosed below.

[0035] As Figure 1 shown, a multi-threaded parallel processing and tile generation method based on geographic image data includes the following steps:

[0036] S1: Construct a geographic image data set, obtain the width, height, projection information and geographic transformation parameters of the images in the geographic image data set, accurately convert the source projection to the target projection coordinate system, and calculate the unit value per pixel UPP. In the embodiment of the present invention, the selected data set is a geographic image data set in GeoTIFF format, and the GDALOpen function of the GDAL library is used to open the geographic image data set in read-only mode to ensure the integrity of the format and content of the input file. The GetRasterXSize function and the GetRasterYSize function are used to obtain the width and height of the geographic image respectively, the GetGeoTransform function is used to obtain the geographic transformation parameters of the geographic image, and finally the GetProjectionRef function is used to obtain the projection information of the geographic image for maintaining the consistency of the projection in the subsequent tile generation process. The target projection coordinate system is the WGS84 coordinate system.

[0037] S2: Set up the environment parameters for driving and configuring multi-threaded processing, control the maximum number of threads in the thread pool, and create the thread pool. Specifically, in the embodiments of the present invention, all available GDAL drivers are first initialized and registered, and then the environment parameters for multi-threaded processing are configured. By setting the GDAL_NUM_THREADS parameter in the CPLSetConfigOption function, the maximum number of threads in the thread pool is controlled. At the same time, the default configuration is to use all available CPU cores to improve the efficiency of parallel processing. Finally, the QThreadPool class is used to construct the thread pool, and the setMaxThreadCount function is called to set the maximum number of concurrent threads. The setting of the setMaxThreadCount function is consistent with the GDAL_NUM_THREADS parameter.

[0038] S3: Process the geospatial image in the thread pool to generate tiles; when generating tiles, it is necessary to calculate the corresponding area of the current tile in the original image, read the data of the corresponding area from the original image, and perform resampling. The resampling process is based on the per-pixel unit value UPP of different standard map zoom levels to ensure the geographical accuracy of the generated tiles; the specific steps are as follows:

[0039] S301: Calculate the number of tiles for each layer and the geographical coordinate range corresponding to each tile according to the minimum zoom level, maximum zoom level input by the user, and the pixel resolution of the geospatial image data. The specific calculation formula is:

[0040] hNo=(lon + 180.0)×scale / 360.0;

[0041] vNo = (90.0 + lat)×scale / 360.0;

[0042] Among them, hNo represents the horizontal number of the tile, vNo represents the vertical number of the tile, lon represents the latitude of the upper left corner of the tile, lat represents the longitude of the upper left corner of the tile, and scale represents the zoom ratio.

[0043] S302: Generate tiles through the thread pool and adopt multi-threaded processing for tile slicing; when generating each tile, first obtain the semaphore for controlling multi-threading to control the concurrency. In the embodiments of the present invention, the semaphore is obtained by calling the acquire function.

[0044] S303: In the specific slicing method, a mutex QMutexLocker is used to protect shared resources. Within the scope of the mutex QMutexLocker, any thread attempting to acquire the same mutex will be blocked until the current thread releases the mutex. The mutex QMutexLocker effectively realizes exclusive access to shared resources and avoids data competition problems caused by multiple threads modifying shared resources simultaneously.

[0045] S304: Read geographic information data such as longitude and latitude of the corresponding area of the slice in the source dataset, and write the read data into the raster data buffer of the target tile; where the corresponding area is based on the per-pixel unit value UPP of each zoom level, and through a zoom adjustment mechanism, the read area is dynamically adjusted to ensure consistent geographic accuracy at each zoom level; after reading is completed, close the source dataset and release resources. In the embodiments of the present invention, the RasterIO function of the GDAL library is used to read the specified area of the source dataset and write the read data into the raster data buffer of the target tile. After reading is completed, the GDALClose function is used to close the target dataset and release resources.

[0046] S305: Unlock the mutex and release the semaphore to allow other threads to continue executing the tile generation task; use atomic operations to update the number of completed tiles and notify the main thread to update the processing progress. Atomic operations allow a thread to safely increment an integer value, ensuring that even in a multi-threaded environment, each increment is based on the currently known number of tiles, thus avoiding race conditions.

[0047] S306: Set the correct geographic transformation parameters for each generated tile and adjust the geographic transformation parameters of the target geographic image dataset of the tiles to be generated, so that the positions of the tiles in the target projection coordinate system are accurate. Specifically, the SetGeoTransform function can be used to adjust the geographic transformation parameters of the target dataset. The calculation formula for the new geographic transformation parameters is as follows:

[0048] newGeoTransform[0] = lat;

[0049] newGeoTransform[1] = geoTransform[1]* tileWidth / 256.0;

[0050] newGeoTransform[3] = lon;

[0051] newGeoTransform[5] = geoTransform[5]* tileHeight / 256.0;

[0052] Among them, tileWidth represents the horizontal resolution of a single tile, tileHeight represents the vertical resolution of a single tile, newGeoTransform[0] represents the X coordinate of the upper-left pixel of the new tile, also known as the minimum longitude or starting longitude, and newGeoTransform[1] represents the pixel width of the new tile; geoTransform[1] represents the pixel width of the old tile; newGeoTransform[3] represents the Y coordinate of the upper-left pixel of the new tile, also known as the maximum latitude or starting latitude; newGeoTransform[5] represents the pixel height of the new tile, and geoTransform[5] represents the pixel height of the old tile. The pixel heights of the old tile and the new tile are usually negative because the values of the Y-axis increase downward in the geographic coordinate system.

[0053] S307: Set the projection information of the target geographic image dataset of the tile to be generated to ensure that all tiles are in the same projection coordinate system as the geographic image dataset; use the function for setting projection information to apply the projection information of the geographic image dataset to all target tiles. In the embodiment of the present invention, the function for setting projection information is the SetProjection function.

[0054] S4: Store the generated tiles according to the hierarchical directory structure for easy access and management. In the embodiment of the present invention, use the Create function in the GDAL library to create an output file in the GeoTIFF format, write the tile data into the output directory, and use the file selection function provided by the QFileDialog class to allow the user to specify the output path. The generated tiles are stored in the specified directory according to the standard tile naming rule (z / x / y.tif), that is, using the hierarchical directory structure, and storing the tiles according to the triple structure of zoom level / column number / row number, which is convenient for subsequent quick access and management.

[0055] The experimental statistical results of the embodiment of the present invention are shown in Table 1:

[0056] Table 1 Experimental Statistical Results of the Embodiment of the Present Invention

[0057]

[0058] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multi-threaded parallel processing and slicing generation method based on geographic image data, characterized in that It includes the following steps: S1: Construct a geographic image dataset, obtain the width, height, projection information, and geographic transformation parameters of the images in the geographic image dataset, accurately transform the source projection into the target projection coordinate system, and calculate the unit value per pixel UPP; The selected dataset is a geographic image dataset in GeoTIFF format, and the GDALOpen function of the GDAL library is used to open the geographic image dataset in read-only mode to ensure the integrity of the format and content of the input file; The GetRasterXSize function and the GetRasterYSize function are used to obtain the width and height of the geographic image respectively, the GetGeoTransform function is used to obtain the geographic transformation parameters of the geographic image, and finally the GetProjectionRef function is used to obtain the projection information of the geographic image for maintaining the consistency of the projection during the subsequent tile generation process; The target projection coordinate system is the WGS84 coordinate system; S2: Set the driver and configure the environment parameters for multi-threaded processing, control the maximum number of threads in the thread pool, create a thread pool, control the maximum number of threads in the thread pool by setting the GDAL_NUM_THREADS parameter in the CPLSetConfigOption function, construct the thread pool using the QThreadPool class, and call the setMaxThreadCount function to set the maximum number of concurrent threads; S3: Process the geographic image in the thread pool to generate tiles; When generating tiles, it is necessary to calculate the corresponding area of the current tile in the original image, read the data of the corresponding area from the original image, and perform resampling. The resampling process is based on the unit value per pixel UPP of different standard map zoom levels to ensure the geographical accuracy of the generated tiles; S4: Store the generated tiles according to the hierarchical directory structure for easy access and management.

2. The multi-thread parallel processing and slice generation method based on geographic image data according to claim 1, wherein, The specific steps of step S3 are as follows: S301: According to the minimum zoom level, maximum zoom level input by the user, and the pixel resolution of the geographic image data, calculate the number of tiles for each layer and the geographic coordinate range corresponding to each tile. The specific calculation formula is: hNo=(lon + 180.0)×scale / 360.0; vNo = (90.0 + lat)×scale / 360.0; Among them, hNo represents the horizontal number of the tile, vNo represents the vertical number of the tile, lon represents the latitude of the upper left corner of the tile, lat represents the longitude of the upper left corner of the tile, and scale represents the zoom ratio; S302: Generate tiles through the thread pool, and use multi-threaded processing for tile slicing; When generating each tile, first obtain the semaphore for controlling multi-threading to control the concurrency; S303: In the specific slicing method, use the mutex QMutexLocker to protect shared resources. Within the scope of the mutex QMutexLocker, any thread attempting to obtain the same mutex will be blocked until the current thread releases the mutex; S304: Read the longitude and latitude geographic information data of the slice in the corresponding area of the source dataset, and write the read data into the raster data buffer of the target tile. The corresponding area is based on the value of each pixel unit (UPP) at each zoom level, and the read area is dynamically adjusted through the zoom adjustment mechanism to ensure consistent geographic accuracy at each zoom level. After the reading is completed, close the source dataset and release the resources. S305: Unlock the mutex and release the semaphore to allow other threads to continue executing the tile generation task. Use atomic operations to update the number of completed tiles and notify the main thread to update the processing progress. S306: Set the correct geographic transformation parameters for each generated tile, and adjust the geographic transformation parameters of the target geographic image dataset of the tiles to be generated, so that the positions of the tiles in the target projection coordinate system are accurate. The calculation formula for the new geographic transformation parameters is as follows: newGeoTransform[0] = lat; newGeoTransform[1] = geoTransform[1] * tileWidth / 256.0; newGeoTransform[3] = lon; newGeoTransform[5] = geoTransform[5] * tileHeight / 256.0; Among them, tileWidth represents the horizontal resolution of a single tile, tileHeight represents the vertical resolution of a single tile, newGeoTransform[0] represents the X coordinate of the upper left pixel of the new tile, newGeoTransform[1] represents the pixel width of the new tile; geoTransform[1] represents the pixel width of the old tile; newGeoTransform[3] represents the Y coordinate of the upper left pixel of the new tile; newGeoTransform[5] represents the pixel height of the new tile, and geoTransform[5] represents the pixel height of the old tile. S307: Set the projection information of the target geographic image dataset of the tiles to be generated to ensure that all tiles are in the same projection coordinate system as the geographic image dataset. Use the function for setting projection information to apply the projection information of the geographic image dataset to all target tiles.

Images