Multi-source heterogeneous mass remote sensing information real-time rapid browsing method

Abstract

The invention discloses a method for quickly browsing multi-source heterogeneous mass remote sensing information in real time. Belongs to the field of remote sensing information technology application and particularly relates to the technical field of remote sensing information real-time rapid browsing. The method comprises the following steps: processing a multi-source heterogeneous remote sensing image by adopting a multivariate heterogeneous remote sensing information standardized archiving processing method, and constructing a multi-dimensional cloud storage with a place-time as an index; for an image real-time browsing request, a self-adaptive heterogeneous image tile real-time rendering method is adopted to render tiles; a multi-dimensional multi-level cache method is adopted to provide data for each link returned by tile rendering; a multi-source heterogeneous remote sensing information browsing service is provided for a user through a self-adaptive heterogeneous image tile real-time rendering technology and a multi-dimensional multi-level caching technology; according to the use habit of a user, intelligent pre-loading and intelligent cache management of tiles are realized through an intelligent cache management and pre-loading method. The technical problems of high delay and long user waiting time of non-slicing release are solved.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing information technology applications, and specifically relates to the technical field of real-time rapid browsing of remote sensing information. Background Art

[0002] The images captured by satellite remote sensing technology carry a wealth of surface information and have become an indispensable data resource in many fields such as geographic information system research and digital earth research. They are widely used in many industries such as agriculture and environmental protection. Since 2014, with the vigorous development of my country's commercial aerospace, the amount of remote sensing data has increased rapidly, and the amount of data has reached the PB level. In addition, these images are usually accompanied by multiple attached files from many sources, and there are significant differences in pixel depth, correction information status, bands, projections, etc. between different images. Therefore, exploring how to effectively standardize remote sensing images, integrate the management of massive standardized remote sensing data, and further explore how to efficiently browse and use massive satellite remote sensing data from different sources, scales and resolutions are of great significance and value for scientific research and practical applications.

[0003] Limited by the size of remote sensing image data files, in order to browse remote sensing image data on any computing terminal, a different storage management and publishing method from the traditional image file must be adopted. In the early days, when people browsed online maps, the server generally queried all the geographic data within the geographic range to be displayed on the page map, and then rendered it into a map image according to the configuration style of the thematic map on the server, and then returned it to the client for browsing. The OGC standard born in this era is the WMS (Web Map Service) service, which has been used to this day. For a more efficient and instant remote sensing image browsing method, the current satellite remote sensing image data can be mainly divided into sliced ​​release and non-sliced ​​release according to the process difference from receiving the original image to the formal release of the image.

[0004] Slice publishing is mainly to solve the key problems of services like WMS, such as the time-consuming IO and network transmission when forming maps; data rendering map images occupy a lot of CPU resources. Google Maps pioneered the idea of ​​rendering remote sensing images based on the Web Mercator projection, pre-rendering them on the server, and then traversing and cutting the entire remote sensing image data into images indexed by x, y (grid number), and z (zoom level) according to different display levels of the map (image pyramid principle), and storing them on the server. Each time a user visits, the current geographic range is mapped to the serial number index of the xyz image. After receiving the returned image from the backend, the client can directly organize and display the image in sequence. This processing greatly improves the user's experience of browsing remote sensing images. This technology has become the WebGIS standard, and a series of services such as TMS services based on industrial standards and WMTS services based on OGC specifications have been born on this basis. For the client, the service backend publishes remote sensing image data as a service, and the client only needs to request the corresponding image tiles, which is the best technical solution at this stage. However, for the server side, the following problems still exist in slice publishing: full map pre-cutting takes a long time, high computing resource requirements, storage redundancy, untimely data updates, and inability to publish in real time. These problems make it impossible for many users to use the massive remote sensing data in an organized manner in a short period of time.

[0005] To solve the above problems, a method for publishing remote sensing images without slicing has emerged. This method essentially uses the remote sensing data service engine and algorithm on the server to render the corresponding image in real time after receiving a request for a certain xyz image from the client, and returns it to the client. In this way, remote sensing images can be published in real time, eliminating the pre-slicing time, and because they are rendered in real time based on access, storage redundancy is avoided. Some expired data can be flexibly replaced.

[0006] However, compared to slice publishing, which only requires indexing image tiles and returning them, non-slice publishing requires real-time rendering of tile images, which means that there is still a gap in the browsing performance of the client between non-slice publishing and slice publishing. In addition, the existing image publishing methods can work well when the data volume is small and the data source is single. In addition to simple image publishing, there is a lack of standardized processing procedures and systematic storage organization for images from different sources and structures. At the same time, in terms of fast browsing services for multi-source heterogeneous massive remote sensing information, there is a lack of services that are based on space-time, closely integrated with image publishing technology and back-end architecture, and efficiently utilize multi-source heterogeneous massive remote sensing information. Summary of the invention

[0007] In order to solve the technical problems of high latency in non-sliced ​​publishing and long user waiting time, the present invention provides a method for real-time fast browsing of multi-source heterogeneous massive remote sensing information, the method comprising the following steps:

[0008] S1. For multi-source heterogeneous remote sensing images, a standardized archiving and processing method for multi-dimensional heterogeneous remote sensing information is used to process them, and a multi-dimensional cloud storage with location-time as the index is constructed;

[0009] S2. For real-time image browsing requests, an adaptive heterogeneous image tile real-time rendering method is used to render tiles;

[0010] S3, adopts multi-dimensional and multi-level caching method to provide data for each link of tile rendering return;

[0011] S4, providing users with multi-source heterogeneous remote sensing information browsing services through adaptive heterogeneous image tile real-time rendering technology and multi-dimensional multi-level caching technology;

[0012] S5. According to the user's usage habits, intelligent preloading and intelligent cache management of tiles are realized through intelligent cache management and preloading methods.

[0013] The beneficial effects of the method are:

[0014] In terms of technology, we first use multi-dimensional multi-level caching combined with intelligent preloading to bring the speed of non-sliced ​​publishing close to the speed of sliced ​​publishing, and based on cloud storage, we effectively standardize and organize the storage of massive remote sensing data. Through the multi-dimensional multi-level caching strategy, the required remote sensing data fragments can be retrieved and loaded from the cloud faster, greatly shortening the user's waiting time. Regardless of the size of the data, users can be guaranteed to obtain a low-latency, high-reliability service experience. Combined with adjustable publishing format and quality and standardized URL service publishing, it can also achieve efficient adaptation to different remote sensing big data analysis systems.

[0015] In terms of services, through the effective organization of multi-source heterogeneous massive remote sensing information, remote sensing data can be quickly located and browsed with space-time index, and the target can be analyzed more deeply by combining the metadata of remote sensing data when it was taken by satellite. In addition, a front-end management module for point-time axis images is added to expand users' management capabilities for massive remote sensing information. Functions such as the release of large-area atlases without slicing are added to increase the breadth and depth of multi-source heterogeneous massive remote sensing information and real-time quick view services. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the method of the present invention;

[0017] Figure 2 A flow chart of the standardized archiving and processing method for multi-heterogeneous remote sensing information;

[0018] Figure 3 A flow chart of the method for real-time rendering of adaptive heterogeneous image tiles;

[0019] Figure 4 Schematic diagrams of four situations where the latitude and longitude ranges and images do not overlap at all;

[0020] Figure 5 Schematic diagram of bounding box calculation method;

[0021] Figure 6 Schematic diagram of the method for calculating the relative grid coordinates of the four corner points of the original window range and the four corner points of the bounding box;

[0022] Figure 7 Provides a structural diagram for the interaction between multi-dimensional and multi-level caching methods and real-time rendering of tiles;

[0023] Figure 8 A schematic diagram of the interactive process for image data caching;

[0024] Fig. 9 This is a schematic diagram of the initial interface of the three-dimensional earth;

[0025] Fig.10 This is a schematic diagram of the switching effect of multiple time axes;

[0026] Fig.11 This is the VRT technology rendering;

[0027] Fig.12 A schematic diagram of the elevation space quadtree tile preload;

[0028] Fig.13 This is a schematic diagram of an index list. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present invention will be more comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0031] This embodiment provides a method for real-time fast browsing of multi-source heterogeneous massive remote sensing information, such as Figure 1 As shown, the method comprises the following steps:

[0032] S1. For multi-source heterogeneous remote sensing images, a standardized archiving and processing method for multi-dimensional heterogeneous remote sensing information is used to process them, and a multi-dimensional cloud storage with location-time as the index is constructed;

[0033] S2. For real-time image browsing requests, an adaptive heterogeneous image tile real-time rendering method is used to render tiles;

[0034] S3, adopts multi-dimensional and multi-level caching method to provide data for each link of tile rendering return;

[0035] S4, providing users with multi-source heterogeneous remote sensing information browsing services through adaptive heterogeneous image tile real-time rendering technology and multi-dimensional multi-level caching technology;

[0036] S5. According to the user's usage habits, intelligent preloading and intelligent cache management of tiles are realized through intelligent cache management and preloading methods.

[0037] First, for massive multi-source heterogeneous remote sensing images, standardized processing is performed through the standardized archiving processing method of multi-heterogeneous remote sensing information, and a multi-dimensional cloud storage indexed by location and time is constructed. On this basis, for real-time image browsing requests, an adaptive heterogeneous image tile real-time rendering method is used to render tiles. In the service process, combined with the multi-dimensional multi-level cache method, data is quickly provided for each link of tile rendering return to speed up service response. On the basis of high-speed and efficient non-slicing publishing service, combined with our image organization context, a high-speed browsing and publishing mode of remote sensing images organized in a location-time manner is provided, providing users with a clear way to browse massive images. Finally, combined with the user's browsing and usage information, the entire multi-source heterogeneous remote sensing information browsing service is fed back, and based on the user's usage habits, intelligent cache management and preloading methods are proposed to further improve the user experience from the user's perspective while maximizing resource savings.

[0038] The following content provides a detailed introduction to the main technical features involved in the real-time rapid browsing method of multi-source heterogeneous massive remote sensing information.

[0039] The standardized archiving and processing method of multi-heterogeneous remote sensing information:

[0040] Satellite images come from various sources, and there are large differences in pixel depth, correction information and other structures of images from different sources. However, there is currently a lack of standardized processing procedures and clear and effective organizational storage structures for multi-source heterogeneous images. In response to this problem, the present invention proposes a real-time fast browsing method for multi-source heterogeneous massive remote sensing information, which standardizes multi-source heterogeneous images and archives them to multi-dimensional storage with a reasonable structure to support the upper layer non-slicing release.

[0041] First, for the standardization of heterogeneous images, the standard image must meet the following requirements: (1) the image pixel bit depth is 8 bits; (2) the image must be orthorectified; (3) the image must contain a pyramid file.

[0042] In order to meet the above principles, satellite remote sensing images from different sources and at different levels should be classified as follows: Figure 2 The specific process is as follows:

[0043] Step 1: First read the six parameters of the image. If the six parameters contain the image geographic information, it means that the image has been orthorectified and go directly to step 3; if the six parameters do not contain geographic information, it means that the image has not been orthorectified and needs to be orthorectified, go to step 2.

[0044] Step 2: Orthorectify the original image using a rational polynomial coefficient model method. The orthorectified image will contain geographic information.

[0045] Step 3: Read the image bit depth. If the image bit depth is equal to 8 bits, it means that the image has been downgraded and you can go directly to step 5. If the image bit depth is greater than 8 bits, it means that the image has not been downgraded and needs to be downgraded and you can go to step 4.

[0046] Step 4: Use the linear stretching method to down-sample the orthorectified image. After down-sampling, the image bit depth will be unified to 8 bits.

[0047] Step 5: Read the image pyramid. If the image pyramid exists, it means that the image is already a standard image and the process ends. If the image pyramid does not exist, go to step 6.

[0048] Step 6: Use the down sampling method to generate pyramid processing for the image.

[0049] Through the above process, it is possible to batch process multi-source heterogeneous images into standardized images that meet the requirements, solving the problem of lack of standardized image processing process.

[0050] The six image parameters include: translation parameter (T x ,T y ), rotation parameter θ, scaling parameter (S x ,S y ) and shear parameter S h .

[0051] Specifically:

[0052] Translation parameter (T x ,T y ), the translation parameter controls the displacement of the image on the X-axis and Y-axis, indicating the overall translation of the image;

[0053] Rotation parameter θ The rotation parameter controls the rotation angle of the image and describes the rotation of the image relative to the reference frame. This parameter is usually expressed as an angle. The image rotates around a certain center point (usually the origin or the center of mass of the image).

[0054] Scaling parameter (S x ,S y ), the scaling parameters represent the scaling ratios of the image in the X-axis and Y-axis directions respectively; when the two scaling factors are not equal, non-uniform scaling will occur (i.e., the image is scaled at different ratios in different directions);

[0055] Shear parameter S h , the shearing parameter represents the parallel stretching deformation of the image, that is, the stretching deformation along one direction; the shearing operation will cause the image shape to be tilted. For example, shearing in the X direction will cause the vertical lines in the image to become tilted straight lines.

[0056] These parameters work together to describe various geometric transformations of an image, including translation, rotation, scaling, and shearing.

[0057] Point-time indexing and cloud storage archiving: In order to efficiently and concisely utilize standardized massive remote sensing images, the time and geographic point of image shooting are used as the main index identifiers. Through this indexing method, image data of a specific time and place can be quickly and accurately located and retrieved, providing users with efficient data query and acquisition services. For the storage of massive remote sensing images, with the help of the elastic storage capabilities of disk arrays and cloud services, large-scale remote sensing image data can be carried. Through a reasonable storage architecture and data partitioning strategy, high reliability and high availability of data are achieved to ensure that users can obtain the required data anytime and anywhere. In addition, a database corresponding to image data and metadata is established to associate and manage image data with related information such as shooting time, point, sensor information, etc. Such a database not only facilitates data management and maintenance, but also provides users with rich data analysis and mining functions.

[0058] The adaptive heterogeneous image tile real-time rendering method:

[0059] Although some differences have been eliminated in the massive remote sensing images after standardization, there are still differences in different bands, projection methods, etc., and there is a lack of efficient image tile real-time rendering technology. To address this pain point, the present invention innovatively proposes an adaptive heterogeneous image tile real-time rendering method, which can intelligently and adaptively adjust the real-time rendering process according to the characteristics of multi-source heterogeneous images and output standard tiles, thereby greatly improving the efficiency and quality of image processing.

[0060] For real-time rendering of adaptive tiles with multiple projections, the processing flow is as follows Figure 3 As shown: First, use the image metadata and request data to calculate whether the requested visible range and the remote sensing image range intersect. If there is an intersection, it will further determine whether it is a Mercator projection. The standard tiles of the WMTS service are based on the Mercator projection, so depending on whether it is a Mercator projection, there will be two different core tile rendering methods, namely the longitude and latitude range direct rendering method and the four-corner point projection rendering method.

[0061] The processes of these two methods are as follows Figure 3 shown.

[0062] First, determine whether it is a Mercator projection. If the image projection is a web Mercator projection (spherical Mercator projection), it is the same as the WMTS default projection, and there is no need to perform projection conversion. The longitude and latitude range direct rendering method is directly used. This rendering method reduces the calculation process as much as possible and can achieve high-speed rendering. The process of the longitude and latitude range direct rendering method is as follows:

[0063] Step 1: First, read the metadata, and use the metadata to convert the xyz tile grid number into the longitude and latitude window range under the web Mercator projection. Calculate the longitude and latitude of the corresponding area according to the image level. The specific calculation method is as follows:

[0064]

[0065] lat_min=90-{[(y+1) / 2 z ]×360};

[0066]

[0067] Among them, lon_min and lat_min are the longitude and latitude of the lower left vertex of the region, lon_max and lat_max are the longitude and latitude of the lower right vertex of the region, and x, y, and z are the coordinates of the region.

[0068] Step 2: Calculate the pixel grid number of the requested tile in the image based on the calculated latitude and longitude window range and the six parameters of the remote sensing image. Calculate the row and column number range corresponding to the area in the image based on the latitude and longitude range of the area calculated in the second step. First, read the six parameters of the image, and further calculate the row and column number range corresponding to the area range based on the six parameters. The specific calculation method is as follows:

[0069] temp=lc[1]×lc[5]-lc[2]×lc[4];

[0070]

[0071] Among them, lc is a one-dimensional array composed of six parameters, temp is the intermediate quantity calculated by the six parameters, leftx and lefty are the row and column numbers corresponding to the upper left vertex of the longitude and latitude area range in the original image, and rightx and righty are the row and column numbers corresponding to the lower right vertex of the longitude and latitude area range in the original image.

[0072] Step 3: According to the grid number and visible window range, combined with the remote sensing image pixel information, the tile image is rendered in real time, multi-dimensionally cached and the tile image is returned.

[0073] Step 4: Crop the corresponding area in the original image according to the row and column number range. When the row and column number range does not completely overlap with the original image, such as Figure 4 As shown, the clipping area needs to be calculated accordingly.

[0074] Step 5: If the row and column number range does not completely overlap with the original image, it is necessary to supplement the pixels of the cropped rectangle and add transparent pixels to ensure that the final image is a square.

[0075] Step 6: Scale the square image to a required size.

[0076] Step 7: Return the cropped and scaled image.

[0077] For non-Mercator projection remote sensing images, although the information is the same at the pixel level, due to the difference in projection information, the latitude and longitude ranges calculated by the tile grid number will have differences in the actual positions of adjacent tiles after rendering, which will cause visual misalignment after publishing. To solve this problem, we use the four-corner bounding box rendering method. We need to take effective methods to perform real-time projection correction during the rendering process to ensure that tiles with different projections can achieve real-time conversion to the WMTS standard projection when publishing. The following is the processing flow:

[0078] Step 1: First read the metadata, and use the same method as step 1 in the longitude and latitude range direct rendering method to convert the tile grid number into the longitude and latitude window range under the web Mercator projection using the metadata.

[0079] Step 2: Convert the longitude and latitude range under the Web Mercator projection to the projection coordinates of the target image.

[0080] Step 3: Use the same method as step 2 in the longitude and latitude range direct rendering method to calculate the pixel grid number of the requested tile in the image based on the converted window range projection coordinates and the six parameters of the remote sensing image. The window range may be a parallelogram in the original pixel, so calculate the bounding box of the window range pixel grid number, take the longitude and latitude of the bounding box, and the calculation method is as follows Figure 5 For any situation, the four vertices of the outer bounding box can be determined based on the maximum (max) and minimum (min) longitude (lon) and latitude (lat). These four vertices are: (lon min ,lat min ), (lon min ,lat max ), (lon max ,lat max ) and (lon max ,lat min ), and obtain the pixel information within the bounding box in the original image in the same way as steps 3-6 in the longitude and latitude range direct rendering method.

[0081] Step 4: After obtaining the bounding box pixel information, calculate the relative grid coordinates of the four corner points of the original window range and the four corner points of the bounding box. The calculation method is as follows Figure 6 shown.

[0082] Step 5: Use the bounding box and the relative grid coordinates of the original window range to solve the affine transformation matrix, affine transform the pixel information of the original window range to the tile image range, and return the transformed image.

[0083] Through the above two different tile rendering methods, we can quickly and efficiently perform real-time rendering of different projection remote sensing image tiles.

[0084] The multi-dimensional multi-level cache method:

[0085] Non-slicing publishing uses real-time rendering to publish images. Compared with the slicing publishing mode in which tiles are pre-processed, although it greatly saves computing and storage resources, there is a certain gap in response speed. In view of the performance gap between non-slicing publishing and slicing publishing, the present invention proposes a multi-dimensional multi-level caching method to effectively improve the speed of image re-access and meet the following requirements: (1) Multi-dimensional caching of storage indexes, metadata and related information of remote sensing data images; (2) Memory caching and persistent caching of rendered tiles. (3) Maximizing the use of cache to optimize the response process and perform multi-dimensional cache management.

[0086] In the multi-dimensional multi-level cache method, multi-dimensional refers to the diversity of the underlying structure and principle dimensions of the cache, and multi-level refers to the cache having different levels, and different levels of cache have different functions. The multi-dimensional multi-level cache and adaptive heterogeneous image tile real-time rendering method have multi-dimensional and multi-level interactions, and the interaction relationship and process are as follows: Figure 7 As shown:

[0087] The first dimension cache is the image path and basic data memory cache. The first dimension mainly stores request information (including point, time, tile number) and basic data such as image path. It mainly uses memory dictionary for storage, and the key-value pair is "point-time": "image database index". Using the location-time image index, an image can be located by "point" plus "time". For the located image, the path and information of the image and related metadata will be read through the database connection for the first time. All subsequent basic image information and paths are directly obtained from the cache, saving a lot of time and computing resources to establish database connections, so as to ensure the rapid positioning of remote sensing images and enter the processing flow.

[0088] The second dimension cache is the memory cache of image metadata parameters. Its main form is a memory dictionary to store key-value pairs, with an additional jump list to accelerate indexing. The cache key-value pair is "point-time": "image metadata parameter data structure". The image metadata parameter data structure mainly stores the latitude and longitude range of the image, six parameters, image projection information, and band information. This cache is used to store the image metadata required for calculating the tile window range, determining the rendering mode, and determining the image intersection. After the first request is warmed up, in subsequent requests, all calculated data are quickly responded to by the cache, saving a lot of GDAL connection resources and time to ensure that the tile rendering process is fast and efficient.

[0089] For the third dimension cache, i.e. image data cache, there are two main forms: distributed memory database and tree-structured disk file persistent storage. In the memory database, the cache structure is "point-time-XYZ": "tile image binary data". For the persistent storage of disk files, the file path tree is organized as: "point-time-XYZ". When the non-sliced ​​publishing service receives a request, the distributed cache and file system cache of the image data will follow a fixed process to determine whether to render tiles in real time or return tiles from the multi-level cache. The process is as follows: Figure 8 As shown:

[0090] Step 1: For a request that needs to return a tile, first search the distributed memory cache for the corresponding cache key value using the sequence of the request information combination as the key. If it exists, the tile data is returned.

[0091] Step 2: If the memory cache does not exist, search in the file system cache. If it exists, return the tile data and load it into the distributed memory cache.

[0092] Step 3: If the file system cache does not exist, there is no tile data cache, and the tile real-time rendering module is called to perform tile real-time rendering.

[0093] Step 4: After rendering, load the tile data into the file system cache and memory cache.

[0094] Through this method, an intelligent and efficient caching mechanism can be implemented, which can save storage resources to the greatest extent and further increase the response time of non-sliced ​​publishing services.

[0095] The multi-heterogeneous remote sensing information browsing service:

[0096] Based on the efficient storage index mode of point-time remote sensing images organized in the location-time mode, and with the adaptive real-time rendering method of heterogeneous image tiles as the core, this service focuses on massive remote sensing data and combines with the client to realize the point-time rapid publishing and browsing service mode of massive remote sensing data. This mode expands the practice of publishing small-area loose images without slicing, solves the pain point of low efficiency in organizing massive images, and uses VRT technology to expand the rapid publishing of remote sensing image collections, with the ability to publish a regional map, thereby expanding the breadth and depth of non-slicing publishing.

[0097] In order to demonstrate the effect of the real-time quick overview service of multi-heterogeneous massive remote sensing information, the present invention selects a mountainous area as a test area. Fig. 9 The following is the initial interface of the 3D Earth. The area list will be displayed on the right side of the interface. Fig.10 As shown in the figure, when you select the corresponding area in the list, the 3D earth will jump to that area. At the same time, the image history timeline will be displayed at the bottom of the software. The timeline shows the shooting time of all historical images of the area. Click the time on the timeline to jump between multiple images of the area. Fig.10 (a), (b) and (c) are images under different time axes respectively. When the user selects a point and clicks a time on the time axis, the access frequency of the point and time will be recorded, providing a basis for subsequent intelligent preloading and cache management.

[0098] Through the point-time browsing technology introduced above, we can quickly switch and browse multiple images along a clear index. With the support of real-time rendering technology, we can browse remote sensing images smoothly and quickly. At the same time, for the management side, new remote sensing images can be published in real time, so as to achieve timely release and instant browsing.

[0099] Unsliced ​​publishing is mostly used for publishing and browsing remote sensing images of small areas. The publishing of images of large areas still requires pre-processing and slicing. In order to expand the breadth of unsliced ​​publishing applications, we have expanded the publishing function of remote sensing image collections by combining VRT (Virtual Raster Format) virtual raster dataset technology. Fig.11 As shown in the figure, VRT technology can combine multiple remote sensing images into a virtual data set. For remote sensing image real-time rendering technology, only the VRT set is read for rendering. However, from the bottom layer of the file system, the data is stored in multiple remote sensing images. Fig.11 In the figure, (1) represents the original remote sensing image and (2) represents the VRT virtual dataset effect. VRT technology can integrate the remote sensing images of different formats, projections and resolutions and publish them in a unified format.

[0100] This technology has broad application prospects in the fields of geographic information systems (GIS), remote sensing image processing, environmental monitoring, urban planning, etc. It provides users with an efficient and fast way to process and browse remote sensing data, and is expected to generate important technical and application value in related fields.

[0101] The intelligent cache management and preloading method:

[0102] According to the frequency statistics of user access location-time in the real-time fast browsing method of multi-source heterogeneous massive remote sensing information, the algorithm model can be used to intelligently perform tile preloading and cache management. Therefore, the present invention mainly focuses on the access frequency of images, constructs access popularity, and realizes intelligent preloading and intelligent cache management.

[0103] In terms of tile preloading, the real-time rendering process without slicing release takes a certain amount of time. Through the above-mentioned efficient rendering core and efficient access of multi-level cache, the average speed can be kept at the same level as slicing release. However, in high-concurrency scenarios such as image elevation up and down exploration and multi-period image browsing, a large number of concurrent real-time renderings will put pressure on the service cluster. Therefore, combined with the real-time quick overview service of multi-heterogeneous massive remote sensing information, this patent proposes an intelligent preloading technology, which mainly consists of two aspects: elevation space quadtree tile preloading and multi-period image time heat preloading. Images that users may access are preloaded to make image browsing smoother.

[0104] The intelligent cache management and preloading method mainly consists of an elevation space tile preloading method and a multi-period image time heat preloading method.

[0105] For the elevation space quadtree tile preloading, the process diagram is as follows Fig.11 As shown: The latitude and longitude tile grid calculation formula is:

[0106]

[0107] The calculation formula for preloaded tile grid is:

[0108]

[0109] Among them, x new and new Indicates the grid number of the preloaded tile, x old and old Indicates the tile grid number of the current zoom level, z old and z new Represents the current zoom level and the next zoom level respectively.

[0110] The specific method of preloading elevation space tiles is as follows:

[0111] Step 1: Starting from a certain zoom level Z, such as level 9, the zoom level at this time requests more tiles. In order to ensure smooth browsing for users, it is necessary to preload the images of the next zoom level. First, according to the tiles requested at this level, the quadtree index number formula is calculated for the upper one and the lower four tiles. Then a data structure is used to cache the quadtree structure.

[0112] Step 2: Based on the quadtree structure, the parent node and leaf node of a single tile are preloaded in the background, so that when the user zooms in or out, the tiles at adjacent elevations are preloaded and can respond quickly.

[0113] Step 3: For the indexed quadtree structure, when its leaf node or parent node is calculated, the subtree structure consisting of the tile and its adjacent elevation tiles can be directly merged into the complete quadtree index structure of the image according to its index number. In this way, the quadtree index construction of a single image can reduce the amount of repeated calculations.

[0114] Step 4: Finally, the quadtree tile index is stored in the basic information cache dimension of the image, and is closely combined with the elevation space preloading technology to speed up the smoothness of the elevation space zoom browsing of the remote sensing image.

[0115] The specific method for preloading the time and heat of multiple images is as follows:

[0116] Step 1: First, at the back end, the access frequency of the multi-phase images of each point is recorded. After a period of time, the initial data of the time access frequency of the multi-phase images of the point is formed.

[0117] Step 2: Calculate the access tile heat according to parameters such as access frequency and image quality, and then divide the current image into three types: high heat, medium heat, and low heat according to a certain heat threshold calculation formula. The heat is calculated by:

[0118]

[0119] Get, where F i is the access frequency of the i-th image, F avg is the average access frequency of multiple images, Q i is the quality score of the i-th image, α is the frequency contribution parameter, β is the quality score contribution parameter,

[0120] H i >μ+kσ, the current image is classified as medium heat.

[0121] H i When >2(μ+kσ), the current image is classified as high heat, and the rest are classified as low heat.

[0122] Step 3: Create index lists for the three heat levels of high, medium and low, respectively, and then create index lists for all images. The created index lists are as follows: Fig.13 shown.

[0123] Step 4: When browsing images, for an image, we first pre-cache adjacent time images with the same heat and adjacent images along the entire time axis.

[0124] Step 5: During pre-caching, hierarchical pre-caching is also performed. First, for images that need to be pre-cached, basic image information and metadata are cached to save time and connection resources required to obtain these data.

[0125] Step 6: Then, according to the heat threshold, choose whether to preload the tiles at other times corresponding to the tiles being browsed.

[0126] Combining the two preloading methods, we can manage the cache intelligently. The steps of cache management are as follows:

[0127] Step 1: Use the formula to comprehensively calculate the access heat of multiple images in all points to calculate the point heat;

[0128] Step 2: Classify the image points according to their heat and the impact of the visit heat under the points into three categories: low, medium and high. Different categories have different weights. According to the formula θ = ωH p +H i Calculate the cache score of the image, where H p is the deviation of the point visit frequency from the average value, H i is the multi-period image access heat, ω is the point heat contribution parameter, all images are standardized according to the calculated different cache scores, and mapped to the [1,100] interval, and different cache strategies are set for different images according to the scores.

[0129] Step 3: For images with cache scores higher than 80, all requests for these images are persisted in the file system for the multi-dimensional cache during the response process, and the memory cache is cleared every 15 days.

[0130] Step 4: If the cache score is higher than 60 points, the image tile data is persisted in the file system, and the memory cache is cleaned up for 7 days.

[0131] Step 5: For the remaining cache scores, local file system persistence is not performed, and the memory cache cleanup time is 24 hours.

[0132] The implementation plan of the main methods is further summarized:

[0133] Implementation plan for standardized archiving and processing methods for multi-source heterogeneous remote sensing information: Collect and build a multi-source heterogeneous massive remote sensing information database, select specific data contexts, such as image level, projection information and other metadata, and perform structured transformation and indexing on the image database. Then closely combine standardized processing and archiving technology with the database, design respective standardized processing processes for multi-source heterogeneous massive remote sensing information, and realize standardized archiving and processing of multi-source heterogeneous massive remote sensing information.

[0134] Implementation plan for adaptive real-time rendering of heterogeneous image tiles: For heterogeneous image tiles, the minimum difference set method is used in the processing flow. The shared models of the calculation imaging area and grid number, projection conversion, etc. are used to perform differential processing on the key parts of tile formation, grid data acquisition and rendering, and the heterogeneous projection information processing of the image is placed in the real-time rendering part, which greatly speeds up the real-time rendering of heterogeneous image tiles.

[0135] Implementation plan for multi-dimensional and multi-level cache method: First, determine the data type and access mode that need to be cached, analyze the popularity and usage frequency of the data, and identify hot data and long-tail data. On this basis, design a multi-level cache structure, each level of cache has different storage capacity and access speed to meet different levels of data access requirements. Secondly, combined with the multi-dimensional characteristics of the data, design a multi-dimensional cache strategy to ensure that the cache can be quickly hit when accessing data in a specific area. Combined with data pre-fetching technology, by predicting the data that the user may access next, load it into the cache in advance, and further reduce data access latency.

[0136] Implementation plan for multi-source heterogeneous remote sensing information browsing service: Based on the space-time context of images in multi-source heterogeneous massive remote sensing databases, image data is quickly indexed through spatial point-imaging time, and quickly displayed in the form of point-time axis in the front-end browsing software. On the basis of displaying and browsing images, it facilitates the subsequent interpretation and application of various images.

[0137] Implementation plan for intelligent cache management and preloading methods: First, establish a spatial quadtree structure for the tile cache and manage the geospatial data in layers. Secondly, for the preloading of multi-period images in time, analyze the historical data and behavior patterns of user access, and predict the time periods and image data that users may pay attention to. At the same time, through the cache replacement algorithm, regularly clean up infrequently accessed historical image data to free up cache space. In order to further improve the preloading efficiency, implement intelligent prefetching algorithms for both spatial and temporal tile cache preloading. Predict the spatial area and time period that may be visited next based on the user's browsing behavior, and load the relevant tile and image data into the cache in advance.

[0138] Through intelligent cache management and preloading methods, it is possible to publish massive image data without slicing, while providing a user experience that is equal to or even better than that of remote sensing image slicing, while occupying less memory and file system resources than remote sensing image slicing.

[0139] The method described in the present invention uses a remote sensing image slice real-time rendering technology that combines multi-dimensional caching and intelligent preloading, combined with point-time image storage organization technology, to achieve fast and efficient non-slicing publishing of remote sensing images. In the experiment, we selected the same type of non-slicing publishing service (OriWMTS) and different types of slice publishing services and the core service based on two rendering methods proposed in this patent to compare the speed and storage resource occupancy. In terms of service running speed, from sending a request to request return, the request analysis tool is used to calculate in milliseconds (ms). In terms of storage and connection resource occupancy, after the same operation, each storage resource is queried in the service cluster, and the connection resource is to analyze and count the calls to the connection in the code structure. In the specific experimental implementation, multiple remote sensing images with different projections were used for publishing. The average image size was 3GB. After the same browsing and use, the performance statistics and analysis were performed through the request analysis tool to ensure that the browsing operation was the same. The results are as follows:

[0140] In terms of response speed, the direct rendering method achieved the best results compared with the four-corner rendering method and similar methods. Although the performance of the four-corner method has declined, it is adapted to a variety of projection methods. For an image with an average size of 3GB, the average time for reprojection using QGIS is about 1 minute. Therefore, the four-corner real-time rendering method can publish images in real time, and the actual experience seems to be no different. In the comparison process with the slice publishing process, it can be seen that our method has also narrowed the performance gap with the slice publishing method.

[0141] Table 1 shows a comparison of the response speed of various publishing methods, and Table 2 shows a comparison of storage performance without slicing:

[0142] Table 1

[0143] method Real-time rendering speed Cache return speed Average return speed Ours-RasterIO 53ms 16ms 35ms Ours-FourConer 75ms 16ms 40ms OriWMTS 61ms 25ms 37ms Slice publishing method - 21ms 21ms

[0144] Table 2

[0145]

[0146] In the comparison of cache and storage performance, it can be seen that although the method of the present invention adopts multi-dimensional multi-level cache, it is better than the YTZ method in average running cache and persistent storage projects thanks to intelligent cache management. At the same time, because the multi-dimensional multi-level cache caches all levels of image data in an all-round way, the average number of data connection resources occupied in each request response process is 1.5 times, which is 2.5 times less than that of the YTZ method.

[0147] In terms of VRT, an average of 6 tif images can be processed per minute, with a total of 1,800 square kilometers of remote sensing images. After the VRT service is built, it can be released instantly. The traditional image fusion and slicing method can process and release an average of 1,300 kilometers of remote sensing images per minute.

[0148] In summary, the method proposed in the present invention can meet the demand for real-time publishing of multi-dimensional heterogeneous massive remote sensing images in various existing scenarios.

Claims

1. A real-time fast browsing method for multi-source heterogeneous massive remote sensing information, characterized in that: The method comprises the following steps: S1. For multi-source heterogeneous remote sensing images, a standardized archiving and processing method for multi-dimensional heterogeneous remote sensing information is used to process them, and a multi-dimensional cloud storage with location-time as the index is constructed; S2. For real-time image browsing requests, an adaptive heterogeneous image tile real-time rendering method is used to render tiles; S3, adopts multi-dimensional and multi-level caching method to provide data for each link of tile rendering return; S4, providing users with multi-source heterogeneous remote sensing information browsing services through adaptive heterogeneous image tile real-time rendering technology and multi-dimensional multi-level caching technology; S5. According to the user's usage habits, intelligent preloading and intelligent cache management of tiles are realized through intelligent cache management and preloading methods.

2. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 1 is characterized in that: The method for standardized archiving of multivariate heterogeneous remote sensing information is specifically as follows: S21, read the six parameters of the image. If the six parameters contain the image geographic information, it means that the image has been orthorectified, and directly enter step S23; if the six parameters do not contain the geographic information, it means that the image has not been orthorectified and needs to be orthorectified, and enter step S22; S22, orthorectifying the original image using a rational polynomial coefficient model method, and the orthorectified image will contain geographic information; S23, read the image bit depth, if the image bit depth is equal to 8 bits, it means that the image has been de-bited, and directly go to step S25; if the image bit depth is greater than 8 bits, it means that the image has not been de-bited, and it needs to be de-bited, and go to step S24; S24, using a linear stretching method to perform bit reduction processing on the orthorectified image, after the bit reduction processing, the image bit depth will be unified to 8 bits; S25, read the image pyramid. If the image pyramid exists, it means that the image is already a standard image and the process ends; if the image pyramid does not exist, go to step S26; S26. Use a down sampling method to generate a pyramid for the image.

3. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 2 is characterized in that: The six image parameters include: translation parameter (T x ,T y ), rotation parameter θ, scaling parameter (S x ,S y ) and shear parameter S h .

4. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 1 is characterized in that: The method for real-time rendering of adaptive heterogeneous image tiles is specifically as follows: S41, using the image metadata and the request data, calculating and determining whether the requested visible range and the remote sensing image range have an intersection; S42. If there is an intersection, further determine whether the projection of the remote sensing image is a Mercator projection S43, if it is a Mercator projection, use the longitude and latitude range direct rendering method to render the tiles in the requested visible range; S44. If the projection is not Mercator projection, four-corner point projection rendering method is used for rendering.

5. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 4 is characterized in that: The latitude and longitude range direct rendering method is specifically: S51, reading metadata, and using the metadata to convert the xyz tile grid number into a longitude and latitude window range under the web Mercator projection; S52, calculating the pixel grid number of the requested tile in the image according to the calculated latitude and longitude window range and six parameters of the remote sensing image; S53, according to the grid number and the visible window range, combined with the remote sensing image pixel information, real-time rendering tile images, multi-dimensional caching and returning tile images; S54, cropping a corresponding area in the original image according to the range of row and column numbers, and when the range of row and column numbers does not completely overlap with the original image, performing corresponding calculations on the cropped area; S55, if the row and column number range does not completely overlap with the original image, pixels are supplemented for the cropped rectangle, and transparent pixels are supplemented to ensure that the final image is a square; S56, scaling the square image to a size that meets the requirements; S57. Return the cropped and scaled image.

6. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 4 is characterized in that: The four-corner point projection rendering method is specifically as follows: S61, reading metadata, and using the metadata to convert the xyz tile grid number into a longitude and latitude window range under the web Mercator projection; S62, converting the latitude and longitude range under the web Mercator projection into the projection coordinates of the target image; S63, calculating the pixel grid number of the requested tile in the image according to the calculated latitude and longitude window range and the six parameters of the remote sensing image; calculating the coordinates of the four corner points of the bounding box of the pixel grid number of the window range; S64, after obtaining the pixel information of the bounding box, the relative grid coordinates of the four corner points of the original window range and the four corner points of the bounding box are calculated; S65. Use the bounding box and the relative grid coordinates of the original window range to solve the affine transformation matrix, affine transform the pixel information of the original window range to the tile image range, and return the transformed image.

7. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 1 is characterized in that: The multi-dimensional multi-level cache method includes three-dimensional cache, namely, first-dimensional cache, second-dimensional cache and third-dimensional cache; First dimension cache: image path and basic data memory cache. The first dimension cache stores basic data, including request information and image path. Second Dimension Cache: This is the memory cache of image metadata parameters. The second dimension cache stores the image metadata required for calculating tile window range, determining rendering mode, and determining image intersection. The third dimension cache: that is, the image data cache. The third dimension cache stores distributed memory data and tree-structured disk files.

8. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 1 is characterized in that: The multi-source heterogeneous remote sensing information browsing service is specifically as follows: based on the space-time context of images in the multi-source heterogeneous massive remote sensing database, the image data is quickly indexed by spatial point-imaging time, and quickly displayed in the form of point-time axis in the front-end browsing software, and on the basis of displaying the browsed images, various subsequent image interpretation and judgment applications are performed.

9. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 1, characterized in that: The intelligent cache management and preloading method is specifically as follows: S91, establishing a spatial quadtree structure for the tile cache and performing hierarchical management of the geographic spatial data; S92, analyzing the historical data and behavior patterns of user visits for preloading of multiple-period images, and predicting the time periods and image data that the user may be interested in; S93, regularly clearing infrequently accessed historical image data through a cache replacement algorithm to release cache space; S94. Implement intelligent pre-fetching algorithms in both spatial tile and temporal tile cache preloading, predict the spatial area and time period that may be visited next based on the user's browsing behavior, and load relevant tile and image data into the cache in advance.

10. The method for real-time fast browsing of multi-source heterogeneous massive remote sensing information according to claim 9, characterized in that: The space is obtained through the elevation space tile preloading technology, and the time tile is obtained through the multi-period image time heat preloading technology.

Images