Efficient spatio-temporal penetration method and apparatus based on TIFF files

Abstract

This application provides an efficient spatiotemporal penetration method and apparatus based on TIFF files. The method includes: determining the number of threads; determining the number of TIFF files and a subset of file paths for each thread based on the number of TIFF files and the number of threads; for each thread's subset of file paths, determining the type of each file path in the current subset and converting the file paths according to their types; establishing a corresponding read-only dataset for each converted file path; extracting geographic transformation parameters and scaling factors and offsets from the read-only dataset; determining the row and column numbers of the data blocks in the TIFF files based on the geographic transformation parameters and target latitude and longitude; reading the original data according to the row and column numbers of the data blocks in the TIFF files; and calibrating the original data according to the scaling factors and offsets to obtain calibrated data, which can improve the efficiency of data penetration query.

Description

Technical Field

[0001] This application relates to the field of meteorological data processing technology, specifically to an efficient spatiotemporal penetration method and apparatus based on TIFF files. Background Technology

[0002] COGTIF (Cloud Optimized GeoTIFF) is a TIFF format optimized for geospatial data. Its core features include: tiled storage (dividing large imagery into fixed-size data blocks), built-in geospatial indexing (containing coordinate ranges and data block mappings), and support for HTTP range requests (allowing on-demand reading of specified data blocks over the network without requiring a full download). This format is widely used in meteorology and remote sensing for storing and sharing large gridded data (such as precipitation and temperature fields), serving as a fundamental format for achieving integrated "display and penetration query." In meteorological gridded data penetration query scenarios, for any point on the map, the operation of querying multi-time series, multi-height layer, or ensemble forecast multi-member data of multiple meteorological elements (precipitation, temperature, etc.) is often presented in the form of time series plots. This scenario requires efficient processing of large-scale gridded data, avoiding redundant calculations and storage.

[0003] Currently, there are three main methods for penetrating and querying meteorological grid data: First, the full-load memory query scheme, which loads the complete raster data file (such as a TIFF format meteorological grid file) into memory at once, forming a complete two-dimensional or three-dimensional array; when querying data for a specific point, the row and column indices (xy indices) of that point in the array are calculated based on its geographic coordinates, and the value at the corresponding position is read directly from the array; Second, the database storage query scheme, which decomposes the meteorological grid data into structured records of "coordinate-feature value" (such as each record containing fields such as longitude, latitude, time, altitude layer, and precipitation), stores them in relational databases (such as MySQL) or spatial databases (such as PostgreSQL and PostGIS), and queries are performed using SQL statements (such as "WHERE longitude=xxx AND latitude=xxx"). The first step is to filter the records corresponding to the target point using the "xxx" function and return the feature values. The second step is to pre-generate a tile query scheme. Based on the pyramid tile model, the meteorological grid data is pre-cut according to a fixed geographical range (such as 256×256 pixel tiles) and different scaling levels to generate a large number of tile files (such as each tile corresponding to a certain region and a certain scaling level of grid data). The mapping relationship between tiles and geographical range is established. When querying, the corresponding scaling level is determined according to the coordinates of the target point and the required accuracy. The tile to which the point belongs is calculated, the tile file is read directly, and then the feature values ​​of the target point are extracted from the tile data.

[0004] However, the full-load memory query scheme requires loading the entire raster file into memory, failing to utilize data chunking or on-demand reading mechanisms. For large-scale meteorological grid data (such as high-resolution hourly precipitation data covering the entire country), the file size can reach GB levels, and full loading will consume a large amount of memory resources, even leading to memory overflow. At the same time, when only data from a single point is needed during the query, redundant data from the entire file is loaded, resulting in wasted memory and I / O resources and low efficiency. The database storage query scheme relies on the structured storage and query capabilities of the database, failing to utilize the native chunking characteristics of raster data. On the one hand, splitting continuous grid data into a large number of structured records will generate additional storage overhead (such as database table structure, indexes, transaction logs, etc.), leading to increased storage costs. On the other hand, queries require traversing indexes or full table scans, which is time-consuming and inefficient for large-scale grid data containing hundreds of millions of records. In addition, databases have weak support for batch queries of multi-time series and multi-height layer data, making it difficult to meet the high-efficiency requirements of penetration queries. The pre-generated tile query scheme relies on pre-processed tile files and does not achieve dynamic on-demand processing or real-time block reading. On the one hand, pre-generating tiles requires a lot of computing resources and time (especially for multi-time series and multi-element meteorological data, a complete set of tiles needs to be generated for each time series and each element), resulting in high preprocessing costs. On the other hand, tile files have redundant storage issues—tile data at different scaling levels have overlapping areas (e.g., lower-level tiles are generalizations of higher-level tiles, containing duplicate data), and for areas not covered by the query, their tile files are invalid storage, wasting storage space. In addition, if the data is updated (e.g., adding grid data for a certain time period), all tiles for the corresponding time period need to be regenerated, resulting in high maintenance costs.

[0005] With the continuous growth of meteorological data, the above three methods are insufficient to meet the demand for efficient querying. First, they are inefficient when processing large amounts of meteorological data and cannot quickly respond to query requests. Second, excessive memory and storage requirements increase the system's operating costs. Finally, they lack flexibility and accuracy when dealing with dynamic weather changes. Summary of the Invention

[0006] This application provides an efficient spatiotemporal penetration method and apparatus based on TIFF files.

[0007] The first aspect of this application provides an efficient spatiotemporal penetration method based on TIFF files, including: Obtain multiple TIFF file path arrays and target latitude and longitude, determine the number of threads, and determine the number of TIFF files and file path subsets for each thread based on the number of TIFF files and the number of threads. The file path subsets are selected from the TIFF file path arrays, and the TIFF files are bitmap image format files. For each thread's subset of file paths, determine the type of each file path in the current subset and convert the file paths according to their types; For each transformed file path, a corresponding read-only dataset is created. Geographic transformation parameters, scaling factors, and offsets from the metadata are extracted from the read-only dataset. The row and column numbers of the data block in the TIFF file are determined based on the geographic transformation parameters and the target latitude and longitude. The original data is read according to the row and column numbers of the data blocks in the TIFF file, and the original data is calibrated according to the scaling factor and offset to obtain the calibrated data.

[0008] In an optional embodiment of this application, determining the number of threads includes: Set the minimum threshold for each thread to process TIFF files to Y, and the total number of TIFF files to M. The theoretical number of threads is determined by the following expression: T_candidate = min (M, N) Where T_candidate is the theoretical number of threads, and N is the number of available threads in the system. The minimum required number of threads is determined using the following expression: T_min = M % Y == 0 ? M / Y : M / Y + 1 Where T_min is the minimum required number of threads. The actual number of threads is determined by the following expression: T = max (1, min (T_candidate, T_min)) Where T is the actual number of threads.

[0009] In an optional embodiment of this application, the conversion of the file path according to the file path type includes: If the current file path is a network path, convert the current file path to a virtual local path; If the current file path is a local path, the current file path will remain unchanged.

[0010] In an optional embodiment of this application, establishing a corresponding read-only dataset for each converted file path includes: For local paths, the TIFF file is opened directly through the file system, a connection to the storage medium is established, and the dataset object is obtained. For virtual local paths, the path is opened through the VSI interface, a streaming connection is established with the remote server, and the dataset object is obtained.

[0011] In an optional embodiment of this application, the row and column numbers of the data block in the TIFF file are determined based on the geographic transformation parameters and the target latitude and longitude using the following expression: x = (int) Math.round ((lon - xmin) / xPixelSize) Where x is the line number of the data block in the TIFF file, lon is the target longitude, xmin is the longitude of the top left corner, and xPixelSize is the grid spacing in the X direction. y = (int) Math.round ((lat - ymax) / yPixelSize) Where y is the column number of the data block in the TIFF file, lat is the target latitude, ymin is the longitude of the top left corner, and yPixelSize is the grid spacing in the Y direction.

[0012] In an optional embodiment of this application, the step of calibrating the original data according to the scaling factor and offset to obtain calibrated data includes: The difference between the ratio of the original data to the scaling factor and the offset is used as the calibrated data.

[0013] In an optional embodiment of this application, the method further includes: Upon completion of each file processing, release the dataset object and the band object corresponding to the read raw data; for network files, additionally release the network connection of the VSI interface. Release all resources when the thread task ends.

[0014] A second aspect of this application provides a high-efficiency spatiotemporal penetration device based on TIFF files, comprising: The determination module is used to obtain multiple TIFF file path arrays and target latitude and longitude, determine the number of threads, and determine the number of TIFF files and file path subsets for each thread based on the number of TIFF files and the number of threads. The file path subset is selected from the TIFF file path array, and the TIFF files are bitmap image format files. The conversion module is used to determine the type of each file path in the current file path subset for each thread, and convert the file paths according to the file path type; The module is used to create a corresponding read-only dataset for each transformed file path, extract the geographic transformation parameters and scaling factors and offsets from the read-only dataset, and determine the row and column numbers of the data block in the TIFF file based on the geographic transformation parameters and target latitude and longitude. The calibration module is used to read the raw data according to the row and column numbers of the data blocks in the TIFF file, and calibrate the raw data according to the scaling factor and offset to obtain calibrated data.

[0015] A third aspect of this application provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above-mentioned efficient spatiotemporal traversal methods based on TIFF files.

[0016] A fourth aspect of the embodiments of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the efficient spatiotemporal traversal method based on TIFF files as described above.

[0017] Compared with the prior art, the technical solutions provided in this application have at least some or all of the following advantages: The efficient spatiotemporal penetration method based on TIFF files described in this application involves obtaining multiple TIFF file path arrays and target latitude and longitude, determining the number of threads, and determining the number of TIFF files and file path subsets for each thread based on the number of TIFF files and the number of threads. The file path subsets are selected from the TIFF file path array, and the TIFF files are bitmap image format files. For each thread's file path subset, the type of each file path in the current file path subset is determined, and the file paths are converted according to their types. A corresponding read-only dataset is established for each converted file path. Geographic transformation parameters and scaling factors and offsets from the metadata are extracted from the read-only dataset, and the row and column numbers of the data blocks in the TIFF files are determined based on the geographic transformation parameters and target latitude and longitude. The original data is read according to the row and column numbers of the data blocks in the TIFF files, and the original data is calibrated according to the scaling factors and offsets to obtain calibrated data. This method can improve the efficiency of data penetration query and reduce resource consumption. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating an efficient spatiotemporal penetration method based on TIFF files provided in one embodiment of this application; Figure 2 A flowchart of an efficient spatiotemporal penetration method based on TIFF files provided in another embodiment of this application; Figure 3 A schematic diagram of a high-efficiency spatiotemporal penetration device based on TIFF files is provided as an embodiment of this application; Figure 4 This is a schematic diagram of a computer device structure provided in one embodiment of this application. Detailed Implementation

[0019] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0020] Please see Figure 1 and Figure 2 The efficient spatiotemporal penetration method based on TIFF files provided in this application includes the following steps S100~S400: S100: Obtain multiple TIFF file path arrays and target latitude and longitude, determine the number of threads, and determine the number of TIFF files and file path subsets for each thread based on the number of TIFF files and the number of threads. The file path subset is selected from the TIFF file path array, and the TIFF files are bitmap image format files. S200: For each thread's file path subset, determine the type of each file path in the current file path subset, and convert the file paths according to their types; S300: For each transformed file path, a corresponding read-only dataset is created. Geographic transformation parameters, scaling factors, and offsets from the metadata are extracted from the read-only dataset. The row and column numbers of the data block in the TIFF file are determined based on the geographic transformation parameters and the target latitude and longitude. S400 reads the original data according to the row and column numbers of the data blocks in the TIFF file, and calibrates the original data according to the scaling factor and offset to obtain calibrated data.

[0021] In an optional embodiment of this application, GDAL (Geospatial Data Abstraction Library) is the basic library for implementing TIFF file reading in this application. Its VSI (Virtual FileSystem) interface documentation is used to understand the mechanism of the " / vsicurl / " prefix and the principle of network file access. In an optional embodiment of this application, the TIFF file can be a single-band TIFF file or a dual-band TIFF file. The elements in the single-band TIFF file include, but are not limited to, precipitation, temperature, and visibility. The elements in the dual-band TIFF file include, but are not limited to, UV wind. The U and V components are used to represent the decomposition of wind direction: U component: represents the wind speed in the east-west direction, with negative values ​​for westerly winds and positive values ​​for easterly winds; V component: represents the wind speed in the north-south direction, with negative values ​​for southerly winds and positive values ​​for northerly winds. The wind direction can be determined by calculating the composite angle of the U and V components (e.g., 0 degrees for northerly winds and 90 degrees for easterly winds). This application calculates the wind direction and wind speed based on the UV wind by penetrating both bands, and often only the wind speed is taken. This application presents an efficient spatiotemporal penetration method based on TIFF files, which includes path array initialization, multi-threaded task dynamic allocation, single-threaded file processing (path judgment, dataset creation, header file parsing, index calculation, data block extraction, and data calibration), and result summarization. Through path identification, multi-threaded dynamic allocation, header file parsing, data block extraction, and calibration processing, a unified processing framework is constructed. By identifying path types, differentiated processing of local TIFF files and network TIFF files is achieved in the same process, including path conversion, dataset creation, data reading, and resource release.

[0022] In an optional embodiment of this application, step S100, obtaining the array of multiple TIFF file paths and the target latitude and longitude, includes: The input contains an array of multiple TIFF file paths and the target latitude and longitude (lon, lat), where the elements of the path array are local paths or HTTP / HTTPS network paths; Iterate through the path array, perform format validation on each path, and determine whether the current path is a valid TIFF file path; Create a list of path type tags, and label each path as "local" or "network" (based on "http: / / " or "https: / / "). Initialize the result container (such as a List), with each element corresponding to a path, to store the final processed data value.

[0023] This application presents an efficient spatiotemporal traversal method based on TIFF files. By identifying the path type (whether it is prefixed with http / https), it directly processes local paths in the same process and performs VSI interface conversion on network paths. It shares the subsequent data extraction, processing, and resource release logic, thereby achieving unified processing of local and network TIFF files. This overcomes the shortcomings of the existing technology in separating the processing logic of local and network files, simplifies the system architecture, facilitates maintenance and expansion, and reduces development costs. In an optional embodiment of this application, step S100, determining the number of threads, includes: Set the minimum threshold for each thread to process TIFF files to Y, and the total number of TIFF files to M. The theoretical number of threads is determined by the following expression: T_candidate = min (M, N) Where T_candidate is the theoretical number of threads, and N is the number of available threads in the system. The minimum required number of threads is determined using the following expression: T_min = M % Y == 0 ? M / Y : M / Y + 1 Where T_min is the minimum required number of threads. The actual number of threads is determined by the following expression: T = max (1, min (T_candidate, T_min)) Where T is the actual number of threads.

[0024] In an optional embodiment of this application, the number of available threads in the system is equal to the number of CPU cores in the system, and is set to N.

[0025] This application's efficient spatiotemporal traversal method based on TIFF files rounds up when calculating the minimum required number of threads to ensure that each thread processes at least Y files. The actual number of threads T = max(1, min(T_candidate, T_min)): when M ≤ Y, T=1 (a single thread processes all files); when Y ≤ M, T=1 (a single thread processes all files); when M ≤ Y, T=1 (a single thread processes all files). <m ≤ y*n 时，t="T_min（如Y=30，M=70时，T_min=3，即3个线程分别处理30、30、10个文件，最后一个线程允许略少于" y个）；当 m>When Y*N, T=N (limited by the number of CPU cores, each thread processes ≥Y files). The fastest efficiency is achieved when Y is between 20 and 80.

[0026] In an optional embodiment of this application, step S100, determining the number of TIFF files and the subset of file paths for each thread based on the number of TIFF files and the number of threads, includes: The following expression calculates the number of files processed by each thread: Average number of fragments = M / T. The first (T-1) threads process an average number of fragments of files, and the Tth thread processes the remaining (M - (T-1) * average number of fragments) files. Generate a task list containing the fragmented path subsets, with each task bound to the corresponding target latitude and longitude (lon, lat). In an optional embodiment of this application, after determining the number of TIFF files per thread and the subset of file paths for each thread based on the number of TIFF files and the number of threads in step S100, the method further includes: T threads are started from a thread pool, and each thread executes a Callable task with a return value. The thread pool parameters are set as follows: core number of threads = T, maximum number of threads = T, and idle time = 60s to avoid frequent creation and destruction of threads.

[0027] In an optional embodiment of this application, step S200, converting the file path according to its type, includes: If the current file path is a network path, convert the current file path to a virtual local path; If the current file path is a local path, the current file path will remain unchanged.

[0028] In an optional embodiment of this application, if the path begins with "http: / / " or "https: / / " (a network path): add the prefix " / vsicurl / " to the beginning of the path to generate a virtual local path (e.g., "http: / / example.com / data.tif" → " / vsicurl / http: / / example.com / data.tif"). This operation maps the network URL to a locally operable path through the Virtual File System Interface (VSI), without requiring the download of the complete file. If it is a local path (e.g., " / data / local.tif" "D:\files\test.tif"): the original path is directly preserved.

[0029] This application presents an efficient spatiotemporal traversal method based on TIFF files. It employs a Virtual File System Interface (VSI) by adding the prefix " / vsicurl / " to the network TIFF file path. This prefix triggers the VSI interface, mapping the network URL to a virtual local path and establishing a direct connection with the network server. This enables direct construction and reading of datasets without prior downloading of network TIFF files. This solves the problems of high storage consumption, high processing latency, and high system complexity caused by the need to pre-download network TIFF files in existing technologies, significantly improving processing efficiency and reducing bandwidth consumption.

[0030] In an optional embodiment of this application, step S300, which involves establishing a corresponding read-only dataset for each converted file path, includes: For local paths, the TIFF file is opened directly through the file system, a connection to the storage medium is established, and the dataset object is obtained. For virtual local paths, the path is opened through the VSI interface, a streaming connection is established with the remote server, and the dataset object is obtained. Only file metadata is loaded, not the complete data.

[0031] The efficient spatiotemporal traversal method based on TIFF files in this application utilizes the streaming and random access capabilities of the VSI interface to extract only header information and target data blocks for network TIFF files, thus avoiding the loading of the complete file.

[0032] In an optional embodiment of this application, step S300, which involves extracting geographic transformation parameters and scaling factors and offsets from the read-only dataset and metadata, includes: Parse the header file from the dataset and extract key parameters: Metadata parsing: Read the "scale" (scaling factor) and "offset" (offset) fields from the metadata. If they do not exist, use the default values ​​(scale=1, offset=0). Geographic transformation parameter extraction: Obtain the geographic transformation array (6 elements in total) from the dataset, and extract: The top-left longitude xmin = array[0]; the X-direction grid spacing xPixelSize = array[1] (pixel width); the top-left latitude ymax = array[3]; the Y-direction grid spacing yPixelSize = array[5] (pixel height, usually negative).

[0033] In an optional embodiment of this application, step S300 involves determining the row and column numbers of the data block in the TIFF file based on the geographic transformation parameters and the target latitude and longitude using the following expression: x = (int) Math.round ((lon - xmin) / xPixelSize) Where x is the line number of the data block in the TIFF file, lon is the target longitude, xmin is the longitude of the top left corner, and xPixelSize is the grid spacing in the X direction. y = (int) Math.round ((lat - ymax) / yPixelSize) Where y is the column number of the data block in the TIFF file, lat is the target latitude, ymin is the longitude of the top left corner, and yPixelSize is the grid spacing in the Y direction.

[0034] In an optional embodiment of this application, lon=116.3°, xmin=116.0°, xPixelSize=0.1°, then x=(116.3-116.0) / 0.1=3→line number=3. In an optional embodiment of this application, lat=39.9°, ymax=40.0°, yPixelSize=-0.1° (negative for south direction), then y=(39.9-40.0) / (-0.1)=1→column number=1.

[0035] This application presents an efficient spatiotemporal penetration method based on TIFF files. For network TIFF files, it utilizes the random access capability of the VSI interface to directly extract 1x1 pixel data blocks based on the calculated row and column numbers. For local files, it directly locates and reads the target data blocks. Neither method requires loading the complete file, accurately extracting the target data blocks and reducing the loading of invalid data. Compared with the existing technology that downloads the complete file and then extracts the data, this method significantly reduces the amount of data processing and is especially suitable for large-scale batch processing of files, thus improving processing speed. In an optional embodiment of this application, step S400, which involves reading the original data according to the row and column numbers of the data block in the TIFF file, includes: Locate the 1x1 pixel data block corresponding to (x, y) and read the original data; Get (Band 1): Retrieves the band object from the dataset that stores the target data; For local files, read the raw data of the 1x1 pixel at the (x,y) position directly from the band object and store it in a float array (e.g., float [] dataArr = new float [1]). For network files, the random access capability of the VSI interface is used to read only the 1x1 pixel block at position (x,y) from the remote file, without loading data from other areas, and store it in dataArr. In an optional embodiment of this application, step S400, which involves calibrating the original data based on the scaling factor and offset to obtain calibrated data, includes: The difference between the ratio of the original data to the scaling factor and the offset is used as the calibrated data.

[0036] In an optional embodiment of this application, the original data = 3000, the scaling factor scale = 0.1, the offset offset = 273.15, and the calibrated data is 3000 / 0.1 - 273.15 = 30000 - 273.15 = 29726.85.

[0037] In an optional embodiment of this application, the method further includes: Upon completion of each file processing, release the dataset object and the band object corresponding to the read raw data. For network files, additionally release the network connection of the VSI interface, such as closing the HTTP connection. In the release of the band object, if the band is not empty, call the delete() method. In the release of the dataset object, if the dataset is not empty, call the delete() method. When a thread task finishes, the thread pool executes shutdown() to release all resources.

[0038] In an optional embodiment of this application, if the path is invalid, the file is corrupted, or the network is interrupted, after the exception is captured, the file size of the network file is obtained through the original URL (with the " / vsicurl / " prefix removed) (e.g., by reading the Content-Length via an HTTP HEAD request). Write the error information (path, exception details, file size) to the log and set the value field of the current file to -99999f (error flag).

[0039] The efficient spatiotemporal traversal method based on TIFF files in this application enables direct access to network files through the Virtual File System Interface (VSI) and ensures system stability through a dual resource release mechanism.

[0040] This application presents an efficient spatiotemporal penetration method based on TIFF files. By using the COGTIF data format, it reduces data loading redundancy, improves data query efficiency, and achieves efficient data penetration query. It optimizes existing meteorological data processing methods, reduces the consumption of memory and storage for data queries, achieves resource optimization, enables flexible and efficient meteorological data penetration query, and supports dynamic weather data changes.

[0041] This application presents an efficient spatiotemporal traversal method based on TIFF files. It releases storage media connection resources for local files and additionally releases network connection resources of the VSI interface for network files. It also ensures complete resource release through a dual release mechanism of inner and outer layers, optimizes the resource management mechanism, avoids resource leakage, and solves the problems of memory leakage and network connection residue caused by incomplete resource release in the prior art, thereby improving system stability and resource utilization efficiency. This application's efficient spatiotemporal penetration method based on TIFF files uniformly marks abnormal data with specific values ​​(such as -99999f), additionally obtains the file size corresponding to the original URL for network files to assist in anomaly analysis, records error information in detail, improves anomaly handling, and enhances data reliability. Compared with the simple anomaly handling in existing technologies, it facilitates subsequent data verification and problem localization, and improves the reliability and traceability of data processing.

[0042] One embodiment of this application provides a meteorological operational processing method, including: The Kubernetes-based end-to-end meteorological algorithm scheduling method is used as the driving force to coordinate the execution of tasks such as high and low pressure system partitioning, meteorological data visualization rendering, front-end component development and deployment, and efficient spatiotemporal penetration. High and low pressure system partitioning relies on multi-source meteorological data obtained by efficient spatiotemporal penetration, which is processed to provide information for meteorological data visualization rendering. Meteorological data visualization rendering transforms complex meteorological data into intuitive charts and images to assist meteorological personnel in analysis and decision-making.

[0043] In an optional embodiment of this application, the high- and low-voltage system partitioning method of this application includes: To obtain the target pressure field, for each grid point in the target pressure field, take the current grid point as the starting point and move to the minimum and maximum points respectively along the gradient descent direction and the gradient ascent direction; The target pressure field is divided into multiple first target low-pressure zones and first target high-pressure zones based on the locations of the minimum and maximum points, respectively. Each first target low-pressure zone and first target high-pressure zone is preprocessed to obtain the second target low-pressure zone and the second target high-pressure zone; The partitioning results of the second target high-pressure partition and the second target low-pressure partition are merged. The overlapping areas of the second target low-pressure partition and the second target high-pressure partition in the merged results are evenly distributed to the second target low-pressure partition and the second target high-pressure partition, resulting in the third target low-pressure partition and the third target high-pressure partition with no overlapping areas.

[0044] In an optional embodiment of this application, the step of moving, for each grid point in the target pressure field, from the current grid point as the starting point along the gradient descent direction and the gradient ascent direction to the minimum and maximum points respectively includes: In the target pressure field, for each grid point in the target pressure field, the current grid point is taken as the starting point, and the grid point is moved along the gradient descent direction to the minimum point, which is taken as the minimum point in the target pressure field. Determine the negative number of the target pressure field to obtain a new target pressure field. In the new target pressure field, for each grid point in the target pressure field, take the current grid point as the starting point and move along the gradient descent direction to the minimum point. This minimum point is taken as the maximum point in the target pressure field.

[0045] In an optional embodiment of this application, the target pressure field is divided into multiple first target low-pressure zones based on the location of the minimum point, including: All grid points that are moved along the gradient descent direction to the minimum point form the initial low-pressure partition; Determine whether the initial low-pressure partition size is less than a first preset threshold; If the size of the initial low-voltage partition is less than the first preset threshold, the initial low-voltage partition is merged with its adjacent low-voltage partitions to obtain the first target low-voltage partition. If the size of the initial low-voltage partition is greater than or equal to the first preset threshold, the initial low-voltage partition is used as the first target low-voltage partition.

[0046] In an optional embodiment of this application, the target pressure field is divided into multiple first target high-pressure zones based on the location of the maximum point, including: All grid points that are moved along the gradient ascent direction to the maximum point form the initial high-pressure partition; Determine whether the initial high-voltage partition size is less than the second preset threshold; If the size of the initial high-voltage zone is less than the second preset threshold, the initial high-voltage zone is merged with its adjacent high-voltage zones to obtain the first target high-voltage zone. If the scale of the initial high-voltage partition is greater than or equal to the second preset threshold, the initial high-voltage partition is used as the first target high-voltage partition.

[0047] In an optional embodiment of this application, each first target low-voltage partition is preprocessed to obtain a second target low-voltage partition, including: For each first target low-pressure zone, sort the air pressure values ​​of all grid points in the current first target low-pressure zone from smallest to largest to obtain the sorted first grid point group. Find the grid point whose sorting position is in the 1 / 3 of the first grid point group and use it as the first target grid point; The pressure value of the first target grid point is used as the first pressure threshold. Grid points in the current first target low-pressure partition that are less than or equal to the first pressure threshold are retained to generate the second target low-pressure partition.

[0048] In an optional embodiment of this application, each first target high-voltage zone is preprocessed to obtain a second target high-voltage zone, including: For each first target high-pressure zone, sort the air pressure values ​​of all grid points in the current first target high-pressure zone from largest to smallest to obtain the sorted second grid point group; Find the grid point whose sorting position is in the 1 / 3 of the second grid point group and use it as the second target grid point; The pressure value of the second target grid point is used as the second pressure threshold. Grid points that are greater than or equal to the second pressure threshold among all grid points of the current first target high-pressure partition are retained to generate the second target high-pressure partition.

[0049] In an optional embodiment of this application, the step of allocating the overlapping area of ​​the second target low-pressure zone and the second target high-pressure zone to the second target low-pressure zone and the second target high-pressure zone based on the center positions of the second target high-pressure zone and the second target low-pressure zone, to obtain a third target low-pressure zone and a third target high-pressure zone with no overlapping area, includes: For each grid point in the overlapping region of the second target low-pressure zone and the second target high-pressure zone, determine the first nearest distance between the current grid point and the edge of the non-overlapping low-pressure zone, and determine the second nearest distance between the current grid point and the edge of the non-overlapping high-pressure zone. Compare the first nearest distance with the second nearest distance; If the first nearest distance is less than the second nearest distance, the current grid point will be used as the grid point for the second target low-pressure partition. If the first nearest distance is greater than the second nearest distance, the current grid point will be used as the grid point of the second target high-pressure partition. If the first nearest distance equals the second nearest distance, the current grid point is evenly distributed to the second target high-pressure zone and the second target low-pressure zone.

[0050] In an optional embodiment of this application, the meteorological data visualization rendering method of this application includes: Create map objects and layer instances, and initialize the created map objects; Register multiple projections on the initialized map object and add the created layer instances to the map object; Initialize the rendering context for the layer instance added to the map object, initialize the data loader based on the Uniform Resource Locator parameter in the layer instance, and add the current layer instance to the layer collection of the loader object; The data loader is used to request and parse data, and the parsed data is pushed to the current layer instance. Based on various projections on the map object, the data is rendered on the layer instance and the rendering effect is presented on the map object.

[0051] In an optional embodiment of this application, creating a layer instance includes: Create color patch layer, contour extraction layer, grid value layer, UV color patch layer, wind vane layer, RGB color patch layer, wind particle layer, surge layer, arrow layer, ground filling layer, upper-level filling layer, station map layer, and contour layer.

[0052] In an optional embodiment of this application, registering multiple projections on the initialized map object includes: Register the Albers equal-area projection, Lambert conformal conic projection, Northern Hemisphere stereographic projection, Southern Hemisphere stereographic projection, and nominal projection on the initialized map object.

[0053] In an optional embodiment of this application, requesting data using a data loader includes: Use the data loader to initiate a request based on the Uniform Resource Locator (URL) parameter in the layer instance to obtain remote data.

[0054] In an optional embodiment of this application, data is parsed using a data loader, including: If the acquired remote data is raster data, the raster data is cropped according to the map extent; When the acquired remote data is vector data, the vector data is preprocessed, wherein the preprocessing includes at least transforming the data structure and filtering invalid data.

[0055] In an optional embodiment of this application, the data rendering based on multiple projections on the map object at the layer instance includes: The layer organizes the data according to the context type in the parsed data, performs WebGL or Canvas2D context rendering, and selects the corresponding projection from multiple projections on the map object during the rendering process. WebGL is the World Wide Web graphics library, and Canvas2D is a two-dimensional canvas.

[0056] In an optional embodiment of this application, selecting a corresponding projection from multiple projections on the map object during the rendering process includes: Obtain the projection information of the current map, and organize the relevant parameters for projection transformation based on the projection information; Determine the projection transformation method based on the current layer rendering context: With the current layer rendering context being Canvas2D, a transformation function is generated based on the projection transformation parameters to convert the latitude and longitude coordinates of the data into screen coordinates so that the vector data can be rendered on the canvas. With the current layer rendering context being WebGL, the projection transformation parameters are passed to the shader. In the shader, the raster data is transformed pixel by pixel according to the projection transformation formula, and the corresponding color is extracted from the legend texture according to the value and colored pixel by pixel to complete the rendering of the TIFF format meteorological data, where TIFF format is a bitmap image format.

[0057] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the diagram may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0058] Please see Figure 3 One embodiment of this application provides a high-efficiency spatiotemporal penetration device 300 based on TIFF files, comprising: The determination module 310 is used to obtain multiple TIFF file path arrays and target latitude and longitude, determine the number of threads, and determine the number of TIFF files and file path subsets for each thread based on the number of TIFF files and the number of threads. The file path subset is selected from the TIFF file path array, and the TIFF files are bitmap image format files. The conversion module 320 is used to determine the type of each file path in the current file path subset for each thread, and convert the file path according to the type of the file path; Module 330 is used to create a corresponding read-only dataset for each transformed file path, extract the geographic transformation parameters and scaling factors and offsets from the read-only dataset, and determine the row and column numbers of the data block in the TIFF file based on the geographic transformation parameters and target latitude and longitude. The calibration module 340 is used to read the raw data according to the row and column numbers of the data block in the TIFF file, and calibrate the raw data according to the scaling factor and offset to obtain calibrated data.

[0059] Specific limitations regarding the aforementioned device 300 can be found in the above description of the efficient spatiotemporal traversal method based on TIFF files, and will not be repeated here. Each module in the aforementioned device 300 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the operations corresponding to each module.

[0060] In one embodiment, a computer device is provided, the internal structure of which can be as follows: Figure 4 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described efficient spatiotemporal traversal method based on TIFF files. It includes: memory and a processor; the memory stores the computer program; and the processor executes the computer program to implement any step of the above-described efficient spatiotemporal traversal method based on TIFF files.

[0061] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can perform any of the steps in the above-described efficient spatiotemporal traversal method based on TIFF files.

[0062] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0064] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0065] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0066] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0067] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.< / m>

Claims

1. An efficient spatiotemporal penetration method based on TIFF files, characterized in that, include: Obtain multiple TIFF file path arrays and target latitude and longitude, determine the number of threads, and determine the number of TIFF files and file path subsets for each thread based on the number of TIFF files and the number of threads. The file path subsets are selected from the TIFF file path arrays, and the TIFF files are bitmap image format files. For each thread's subset of file paths, determine the type of each file path in the current subset and convert the file paths according to their types; For each transformed file path, a corresponding read-only dataset is created. Geographic transformation parameters, scaling factors, and offsets from the metadata are extracted from the read-only dataset. The row and column numbers of the data block in the TIFF file are determined based on the geographic transformation parameters and the target latitude and longitude. The original data is read according to the row and column numbers of the data blocks in the TIFF file, and the original data is calibrated according to the scaling factor and offset to obtain the calibrated data.

2. The method according to claim 1, characterized in that, Determining the number of threads includes: Set the minimum threshold for each thread to process TIFF files to Y, and the total number of TIFF files to M. The theoretical number of threads is determined by the following expression: T_candidate = min (M, N) Where T_candidate is the theoretical number of threads, and N is the number of available threads in the system. The minimum required number of threads is determined using the following expression: T_min = M % Y == 0 ? M / Y : M / Y + 1 Where T_min is the minimum required number of threads. The actual number of threads is determined by the following expression: T = max (1, min (T_candidate, T_min)) Where T is the actual number of threads.

3. The method according to claim 1, characterized in that, The process of converting file paths based on their type includes: If the current file path is a network path, convert the current file path to a virtual local path; If the current file path is a local path, the current file path will remain unchanged.

4. The method according to claim 3, characterized in that, The step of creating a corresponding read-only dataset for each converted file path includes: For local paths, the TIFF file is opened directly through the file system, a connection to the storage medium is established, and the dataset object is obtained. For virtual local paths, the path is opened through the VSI interface, a streaming connection is established with the remote server, and the dataset object is obtained. The VSI interface is the virtual file system interface.

5. The method according to claim 1, characterized in that, The row and column numbers of a data block in a TIFF file are determined using the following expressions based on geographic transformation parameters and target latitude and longitude: x = (int) Math.round ((lon - xmin) / xPixelSize) Where x is the line number of the data block in the TIFF file, lon is the target longitude, xmin is the longitude of the top left corner, and xPixelSize is the grid spacing in the X direction. y = (int) Math.round ((lat - ymax) / yPixelSize) Where y is the column number of the data block in the TIFF file, lat is the target latitude, ymin is the longitude of the top left corner, and yPixelSize is the grid spacing in the Y direction.

6. The method according to claim 1, characterized in that, The calibration of the original data based on the scaling factor and offset to obtain calibrated data includes: The difference between the ratio of the original data to the scaling factor and the offset is used as the calibrated data.

7. The method according to claim 1, characterized in that, The method further includes: Upon completion of each file processing, release the dataset object and the band object corresponding to the read raw data; for network files, additionally release the network connection of the VSI interface. Release all resources when the thread task ends.

8. A high-efficiency spatiotemporal penetration device based on TIFF files, characterized in that, include: The determination module is used to obtain multiple TIFF file path arrays and target latitude and longitude, determine the number of threads, and determine the number of TIFF files and file path subsets for each thread based on the number of TIFF files and the number of threads. The file path subset is selected from the TIFF file path array, and the TIFF files are bitmap image format files. The conversion module is used to determine the type of each file path in the current file path subset for each thread, and convert the file paths according to the file path type; The module is used to create a corresponding read-only dataset for each transformed file path, extract the geographic transformation parameters and scaling factors and offsets from the read-only dataset, and determine the row and column numbers of the data block in the TIFF file based on the geographic transformation parameters and target latitude and longitude. The calibration module is used to read the raw data according to the row and column numbers of the data blocks in the TIFF file, and calibrate the raw data according to the scaling factor and offset to obtain calibrated data.

9. A computer device, comprising: A memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the efficient spatiotemporal traversal method based on TIFF files as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the efficient spatiotemporal traversal method based on TIFF files as described in any one of claims 1 to 7.

Images