Remote sensing image real-time display method and device, electronic equipment and storage medium

Abstract

The application provides a remote sensing image real-time display method and device, electronic equipment and storage medium, and relates to the technical field of remote sensing image visualization. The method comprises the following steps: receiving a plurality of original projection tiles of a remote sensing image to be displayed; calculating the geographical range of each original projection tile in a general coordinate system, wherein the geographical range comprises original coordinates of the original projection tile in an original projection coordinate system of the remote sensing image and general coordinates in the general coordinate system; constructing a triangular mesh of each original projection tile according to the geographical range, wherein the triangular mesh is composed of a plurality of triangles, and each triangle comprises a plurality of pixels; calculating the texture sampling coordinates corresponding to each pixel in the plurality of triangles according to the original coordinates and the general coordinates of the vertices of the plurality of triangles; performing texture sampling in the plurality of original projection tiles according to the texture sampling coordinates to obtain the color values corresponding to each pixel; and outputting the color values corresponding to the plurality of pixels to obtain a display result of the remote sensing image.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image visualization technology, and in particular to a method, apparatus, electronic device, and storage medium for real-time display of remote sensing images. Background Technology

[0002] With the rapid development of remote sensing Earth observation technology and the widespread adoption of browser-based graphics rendering technology, remote sensing image visualization based on 3D digital earth has become an important application in the field of geographic information systems. Open-source web-based 3D earth engines can build world-class 3D earth scenes within a browser and natively support image services in the 1984 World Geodetic Coordinate System and the Network Mercator Projection Coordinate System, and are widely used for the online display of remote sensing imagery.

[0003] In practical applications, remote sensing image data is typically stored and distributed using various projection coordinate systems, such as the Universal Transverse Mercator projection and the Albers equal-area projection, due to factors such as satellite orbits, imaging mechanisms, and surveying standards. To display remote sensing images using these projection coordinate systems on a 3D digital earth, the following approach is commonly used: On the server side, a geographic information service middleware is used to perform projection transformation on the original projection remote sensing image data, converting pixels from the original projection coordinate system to a coordinate system supported by the 3D earth engine. These pixels are then organized into standard tiles according to a global tile pyramid model and published externally through a standard image service interface. The client requests the corresponding standard tiles from the server based on the view extent and zoom level, and the obtained tiles are directly attached to the 3D earth surface for rendering and display.

[0004] In the implementation of the aforementioned technologies, the server is responsible for the projection transformation of remote sensing images from the original projection coordinate system to the target coordinate system and for generating tiles. This process is computationally intensive, which consumes a lot of computing resources on the server and results in a large server load. Summary of the Invention

[0005] In view of the above problems, the present invention provides a method, apparatus, electronic device and storage medium for real-time display of remote sensing images.

[0006] The first aspect of this invention provides a method for real-time display of remote sensing images, comprising: receiving multiple original projection tiles of a remote sensing image to be displayed; calculating the geographical extent of each original projection tile in a universal coordinate system, the geographical extent including the original coordinates of the original projection tile in the original projection coordinate system of the remote sensing image and the universal coordinates in the universal coordinate system; constructing a triangular mesh of each original projection tile based on the geographical extent, the triangular mesh consisting of multiple triangles, each triangle containing multiple pixels; calculating the texture sampling coordinates corresponding to each pixel in the multiple triangles based on the original coordinates and universal coordinates of each vertex of the multiple triangles; performing texture sampling on the multiple original projection tiles based on the texture sampling coordinates to obtain the color value corresponding to each pixel; and outputting the multiple color values ​​corresponding to the multiple pixels to obtain the display result of the remote sensing image.

[0007] Further, the geographic extent of each original projection tile in the universal coordinate system is calculated, including: obtaining the coordinates of multiple corner points of each original projection tile in the original projection coordinate system; converting the multiple corner point coordinates into universal coordinates in the universal coordinate system using a coordinate transformation library to obtain the initial geographic extent of each original projection tile; and determining the geographic extent of the original projection tile in the universal coordinate system based on the boundary of the initial geographic extent.

[0008] Furthermore, the universal coordinates include latitude and longitude coordinates. Determining the geographical extent of the original projected tiles in the universal coordinate system based on the boundaries of the initial geographical extent includes: sampling each boundary of the initial geographical extent to obtain multiple sampling points on each boundary; converting the coordinates of each sampling point in the original projected coordinate system to universal coordinates in the universal coordinate system through a coordinate transformation library; and determining the geographical extent of the original projected tiles in the universal coordinate system based on the extreme longitude and latitude values ​​in the universal coordinates.

[0009] Furthermore, based on the original coordinates and general coordinates of each vertex of the multiple triangles, the texture sampling coordinates corresponding to each pixel in the multiple triangles are calculated, including: determining the centroid coordinates corresponding to each pixel in the multiple triangles based on the general coordinates of each vertex of the multiple triangles, wherein the centroid coordinates include multiple weight values ​​corresponding to the triangle vertices respectively, and the sum of the multiple weight values ​​is equal to 1; performing centroid coordinate interpolation on the original coordinates based on the centroid coordinates and the original coordinates corresponding to the triangle vertices to obtain the sampling point coordinates of the pixel in the original projected coordinate system; and determining the texture sampling coordinates corresponding to the pixel based on the sampling point coordinates.

[0010] Further, determining the texture sampling coordinates corresponding to the pixel based on the sampling point coordinates includes: obtaining the range parameters of the original projection tile in the original projection coordinate system, the range parameters including the reference coordinates and size parameters; and normalizing the sampling point coordinates based on the range parameters to obtain the texture sampling coordinates corresponding to the pixel.

[0011] Furthermore, multiple color values ​​corresponding to multiple pixels are output to obtain the display results of multiple original projection tiles, including: rendering the triangular mesh to the three-dimensional Earth surface corresponding to the universal coordinate system based on the universal coordinates of each vertex in the triangular mesh; outputting the color value corresponding to each pixel to the position of the triangular mesh on the three-dimensional Earth surface; and seamlessly processing the splicing area between adjacent original projection tiles to obtain the display results of multiple original projection tiles.

[0012] Furthermore, seamless processing is performed on the splicing area between adjacent original projection tiles, including at least one of the following: keeping the original coordinates and general coordinates of the vertices of the triangular mesh at the boundary of adjacent original projection tiles consistent, so that the boundary triangles of adjacent original projection tiles fit seamlessly on the three-dimensional Earth surface; extending the original projection tiles outward by a predetermined number of pixels; and performing smooth transition processing on the color values ​​of the pixels of the boundary triangles of adjacent original projection tiles through centroid coordinate interpolation.

[0013] A second aspect of the present invention provides a real-time display device for remote sensing images, comprising: a tile receiving module for receiving multiple original projection tiles of a remote sensing image to be displayed;

[0014] The scope confirmation module calculates the geographic scope of each original projection tile in a universal coordinate system. The geographic scope includes the original coordinates of the original projection tile in the original projection coordinate system of the remote sensing image and its universal coordinates in the universal coordinate system. The triangulation construction module constructs a triangulation for each original projection tile based on the geographic scope. The triangulation consists of multiple triangles, each containing multiple pixels. The texture sampling coordinate calculation module calculates the texture sampling coordinates corresponding to each pixel in the multiple triangles based on the original coordinates and universal coordinates of each vertex. The rendering output module performs texture sampling on the multiple original projection tiles based on the texture sampling coordinates to obtain the color value corresponding to each pixel, and outputs the multiple color values ​​corresponding to multiple pixels to obtain the display result of the remote sensing image.

[0015] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the above-described method.

[0017] The remote sensing image real-time display method, apparatus, electronic device, and storage medium provided by the present invention can achieve the following technical effects:

[0018] (1) By transferring the projection transformation processing of remote sensing images from the server to the client, the server no longer needs to undertake computationally intensive tasks such as batch conversion of the original projection tiles to the target coordinate system tiles and tile recombination. It only needs to provide the original projection tiles to complete data distribution, which can significantly reduce the computational burden and storage pressure of the server and effectively alleviate the problem of large server load in related technologies.

[0019] (2) By constructing a triangular network on the client based on the geographic range of the original projection tiles in the universal coordinate system, and using triangles as the basic geometric units to carry the local mapping relationship from the original projection coordinate system to the universal coordinate system, the process of reprojecting each pixel on the server can be transformed into a lightweight geometric transformation of each triangle on the client, so that remote sensing images under different original projection coordinate systems can be attached to the three-dimensional Earth surface in real time without pre-reprojection.

[0020] (3) By interpolating the centroid coordinates of the universal coordinates of the triangle vertices with the original coordinates, the sampling position of each pixel in the original projection coordinate system can be accurately restored at the pixel level. The texture sampling coordinates are obtained by combining the normalization of the original projection tile range parameters, so that the texture sampling process can maintain geometric accuracy while having high execution efficiency, and effectively avoid geometric distortion caused by projection transformation approximation.

[0021] (4) By sampling each boundary of the initial geographic range of the original projection tile and taking the extreme values ​​of latitude and longitude, the true coverage range of the original projection tile in the general coordinate system can be accurately enclosed in the case of nonlinear transformation relationship between the original projection coordinate system and the general coordinate system. This avoids the range estimation deviation caused by only using the four corner point transformation and improves the reliability of triangulation construction and tile scheduling.

[0022] (5) By sharing consistent vertex coordinates at the boundaries of adjacent original projection tiles, expanding outward by a preset number of pixels, and performing smooth transition processing on the boundary triangles, display defects such as cracks, misalignments, and color abrupt changes that may occur at the splicing points of adjacent tiles on the three-dimensional Earth surface can be effectively eliminated, so that remote sensing images can maintain a continuous and natural visual effect during large-scale roaming and multi-level scaling. Attached Figure Description

[0023] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0024] Figure 1 This illustration schematically depicts an application scenario of a method, apparatus, and electronic device for real-time display of remote sensing images according to embodiments of this application.

[0025] Figure 2A flowchart illustrating a method for real-time display of remote sensing images according to an embodiment of the present invention is shown schematically.

[0026] Figure 3 A block diagram of a remote sensing image real-time display device according to an embodiment of the present invention is shown schematically;

[0027] Figure 4 A block diagram of an electronic device suitable for implementing a real-time display method for remote sensing images according to an embodiment of the present invention is shown schematically. Detailed Implementation

[0028] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0029] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0030] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0031] In view of the problems existing in the real-time display of remote sensing images in related technologies, based on the principles and application effects of the pre-tiling publishing method and the server-side real-time dynamic projection method mentioned in the background, the shortcomings of the existing technologies are summarized as follows:

[0032] (1) Low timeliness. Preprocessing massive amounts of remote sensing images with global pyramid tiles is an extremely time-consuming process, which cannot meet the application needs of emergency response, real-time monitoring and other applications that require rapid visualization of the latest data.

[0033] (2) High server load and high response latency. Real-time dynamic projection on the server side is a computationally intensive operation. When there are a large number of concurrent requests, the server is likely to become a performance bottleneck, resulting in excessively long tile response time, screen lag on the client side, and poor user experience.

[0034] (3) Poor flexibility. Both existing solutions tightly couple the projection conversion logic with the data service. Once the service is published, the projection relationship is fixed, and the client cannot dynamically select or switch to other projection methods for display.

[0035] (4) Waste of resources. The preprocessing tiling scheme processes a copy of all image data in order to display image data of a certain area, resulting in a waste of storage and computing resources.

[0036] In view of this, embodiments of the present invention provide a method, apparatus, electronic device and storage medium for real-time display of remote sensing images. The system can be designed from the reception of original projection tiles, calculation of geographical range, construction of triangulation, calculation of texture sampling coordinates to rendering output, so as to efficiently realize the real-time bonding and seamless display of non-standard projection remote sensing images on the client side.

[0037] Figure 1 The illustration shows an application scenario of a method, apparatus, and electronic device for real-time display of remote sensing images according to embodiments of this application.

[0038] like Figure 1 As shown, the application scenario according to this embodiment may include a client 101, a network 102, and a server 103. The network 102 is used as a medium to provide a communication link between the client 101 and the server 103. The network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0039] Users can use client 101 to interact with server 103 via network 102 to receive or send remote sensing image tile data, etc. Client 101 can be equipped with client applications that support 3D earth rendering, such as 3D geographic information visualization applications based on the Cesium framework, and can be integrated with a graphics processing unit to support shader-based real-time graphics rendering.

[0040] Client 101 can be any electronic device with a display screen, graphics processing capabilities, and the ability to browse 3D scenes, including but not limited to smartphones, tablets, laptops, and desktop computers. Client 101 is used to send a request for original projection tiles to server 103 in the 3D globe scene based on the user's viewing angle and zoom level, and to perform real-time reprojection calculations and rendering of the received original projection tiles.

[0041] Server 103 can be any server providing remote sensing image tile services, such as a backend data server providing remote sensing image tiles that retain their original projection (e.g., universal transverse Mercator projection, Lambert projection, etc.) in the form of Hypertext Transfer Protocol file service or standard tile map service (this is just an example). The backend data server can parse and process the received client tile requests, and feed back the remote sensing image tiles under the original projection and the corresponding metadata (including projection identifier, projection zone, central meridian, ellipsoid parameters, tile range parameters in the original projection coordinate system, etc.) to the client according to the pyramid hierarchy and request row and column number, without performing any projection transformation calculations on the server side.

[0042] It should be noted that the real-time remote sensing image display method provided in this application embodiment can generally be executed by the client 101. Correspondingly, the real-time remote sensing image display device provided in this application embodiment can generally be located in the client 101. The real-time remote sensing image display method provided in this application embodiment can also be executed by a client or client cluster that is different from the client 101 but can communicate with the client 101 and / or the server 103. Correspondingly, the real-time remote sensing image display device provided in this application embodiment can also be located in a client or client cluster that is different from the client 101 but can communicate with the client 101 and / or the server 103.

[0043] It should be understood that Figure 1 The number of clients, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of clients, networks, and servers.

[0044] Figure 2 A flowchart illustrating a method for real-time display of remote sensing images according to an embodiment of the present invention is shown.

[0045] like Figure 2 As shown, the real-time display method for remote sensing images in this embodiment includes operations S201 to S205.

[0046] In operation S201, multiple raw projection tiles of the remote sensing image to be displayed are received.

[0047] Specifically, when the client navigates within a 3D Earth scene, it determines the row and column numbers of the tiles to be loaded based on the current viewpoint and zoom level, and then sends a request for the original projected tiles to the data service layer. The data service layer provides data in the simplest form of a Hypertext Transfer Protocol file service or a standard tile map service. The tiles it provides always maintain the original projected coordinate system of the remote sensing image, requiring no projection transformation logic to be executed on the server side. For example, for an image using the Universal Transverse Mercator projection 50 North zone, the tile request path can carry parameters such as the original projection identifier, zoom level, row number, and column number, allowing the client to obtain the required tile data in an orderly, pyramid-like manner.

[0048] Optionally, this operation may also include a process of parsing the metadata of the original projected tiles. The parsed content includes at least one or more of the following: projection identifier, projection zone, central meridian, ellipsoid parameters, and the tile's range parameters in the original projection coordinate system. The metadata can be obtained through the tile service capability description document or returned as an auxiliary field in the tile request response by the data service layer. Based on the projection parameters carried by the metadata, the client can automatically identify the coordinate system to which the original projected tile belongs without any manual intervention or pre-configuration, thereby achieving plug-and-play adaptation to image services for different projection methods.

[0049] In operation S202, the geographic extent of each original projection tile in the universal coordinate system is calculated. The geographic extent includes the original coordinates of the original projection tile in the original projection coordinate system of the remote sensing image and the universal coordinates in the universal coordinate system.

[0050] In some embodiments, the above operation S202 may further include: obtaining the coordinates of multiple corner points of each original projection tile in the original projection coordinate system; converting the multiple corner point coordinates into general coordinates in the general coordinate system using a coordinate transformation library to obtain the initial geographical range of each original projection tile; and determining the geographical range of the original projection tile in the general coordinate system based on the boundary of the initial geographical range.

[0051] Specifically, for each original projected tile, its range parameters in the original projected coordinate system are first extracted from the tile's metadata, and the coordinates of the four corner points of the tile are obtained from this, namely the lower left corner... bottom right corner Top right corner and the top left corner By using a coordinate transformation library and calling the standard projection inverse calculation formula, the coordinates of the four corner points are converted from the original projected coordinate system to latitude and longitude coordinates in the universal coordinate system, thus obtaining the initial geographical extent of the tile. .

[0052] In some embodiments, the universal coordinates include latitude and longitude coordinates. Determining the geographical extent of the original projected tile in the universal coordinate system based on the boundaries of the initial geographical extent may further include: sampling each boundary of the initial geographical extent to obtain multiple sampling points on each boundary; converting the coordinates of each sampling point in the original projected coordinate system to universal coordinates in the universal coordinate system through a coordinate transformation library; and determining the geographical extent of the original projected tile in the universal coordinate system based on the extreme longitude and extreme latitude values ​​in the universal coordinates.

[0053] Specifically, due to the characteristics of nonlinear projections such as the Universal Transverse Mercator projection and the Lambert projection, straight lines along the rectangular boundary in the original projected coordinate system are not straight lines in the universal coordinate system. The bounding box obtained solely based on the transformation results of the four corner points may not completely cover the true geographical extent of the tile in the universal coordinate system. Therefore, N points are sampled more densely along the four boundaries of the original projected tile, with N preferably ranging from 8 to 16. For example, for the left boundary... Along the y-direction from arrive N sampling points are generated uniformly; for the remaining three boundaries, sampling is performed uniformly in the same way. The coordinates of all sampling points in the original projected coordinate system are converted to latitude and longitude in the universal coordinate system one by one using a coordinate transformation library. The minimum and maximum values ​​of longitude and latitude of all sampling points are then taken as the final precise bounding box.

[0054]

[0055]

[0056]

[0057]

[0058] in , They represent the first The longitude and latitude of each sampling point in a universal coordinate system , , , These represent the minimum and maximum longitude and latitude values ​​for all sampling points, respectively. The resulting geographic area completely covers the actual coverage of the original projected tiles in the universal coordinate system, preventing tiles from being incorrectly removed during view clipping and ensuring the integrity of subsequent rendering.

[0059] In operation S203, a triangular mesh of each original projection tile is constructed according to the geographical range. The triangular mesh consists of multiple triangles, and each triangle contains multiple pixels.

[0060] Specifically, for each original projected tile, a regular grid is created within its rectangular coverage area in the original projected coordinate system. The grid density can be 16×16, 32×32, or other adaptive densities. Higher grid density results in higher geometric accuracy after reprojection, but also increases computational overhead. In practical applications, the grid density can be dynamically adjusted based on the view zoom level, tile geographic coverage size, and the degree of non-linearity of the target projection.

[0061] For each vertex in the regular mesh, calculate its coordinates in the original projected coordinate system. And convert it to latitude and longitude coordinates in a universal coordinate system using a coordinate transformation library. Therefore, each grid vertex has two sets of coordinate values: one set is the original projected coordinates, which are used for subsequent sampling on the original projected tile texture; the other set is the latitude and longitude coordinates in a universal coordinate system (which can eventually be converted into three-dimensional Cartesian coordinates by the client), which are used for positioning on the three-dimensional Earth surface.

[0062] Subsequently, each rectangular cell in the regular grid is divided into two triangles, resulting in a triangular mesh composed of multiple triangles. Each triangle's three vertices carry both original and universal coordinates, serving as the geometric basis for establishing the subsequent mapping relationship from the universal coordinate system to the original projected coordinate system.

[0063] In operation S204, based on the original coordinates and general coordinates of each vertex of multiple triangles, the texture sampling coordinates corresponding to each pixel in the multiple triangles are calculated.

[0064] In some embodiments, the above operation S204 may further include: determining the centroid coordinates corresponding to each pixel in the plurality of triangles based on the general coordinates of each vertex of the plurality of triangles, wherein the centroid coordinates include multiple weight values ​​corresponding to the vertices of the triangles respectively, and the sum of the multiple weight values ​​is equal to 1; performing centroid coordinate interpolation on the original coordinates based on the centroid coordinates and the original coordinates corresponding to the vertices of the triangles to obtain the sampling point coordinates of the pixel in the original projected coordinate system; and determining the texture sampling coordinates corresponding to the pixel based on the sampling point coordinates.

[0065] Specifically, the mathematical principle of barycentric coordinate interpolation is based on a local linear approximation of affine transformation. For a triangle, the position of any point inside it can be uniquely expressed by the positions of the three vertices using barycentric coordinates. Assume the positions of the three vertices of a triangle in a universal coordinate system (or their screen coordinates after rendering pipeline processing) are respectively... Their corresponding coordinates in the original projected coordinate system are as follows: .

[0066] For any pixel point inside the triangle Its centroid coordinates satisfy:

[0067]

[0068]

[0069] in, Each pixel The weight values ​​corresponding to the three vertices of the triangle.

[0070] Since the centroid coordinates remain unchanged under affine transformation, this pixel point The corresponding sampling point coordinates in the original projected coordinate system The original coordinates can be obtained by interpolating the same centroid coordinates:

[0071]

[0072] In actual rendering, barycentric coordinate interpolation can be automatically completed by the graphics pipeline during the rasterization stage. Specifically, the original projected coordinates of each triangle vertex are output from the vertex shading stage as variables that can be passed between different rendering stages. The rasterization stage performs barycentric coordinate interpolation on this variable inside the triangle, so that each pixel can directly obtain the interpolated original projected coordinates without having to repeatedly perform projection inverse calculations at the pixel level.

[0073] In some embodiments, determining the texture sampling coordinates corresponding to a pixel based on the sampling point coordinates may further include: obtaining the range parameters of the original projection tile in the original projection coordinate system, the range parameters including the reference coordinates and the size parameters; and normalizing the sampling point coordinates based on the range parameters to obtain the texture sampling coordinates corresponding to the pixel.

[0074] Specifically, the range parameters of the original projected tile in the original projected coordinate system can be directly extracted from the tile metadata, where the reference coordinates are taken as the coordinates of the lower left corner of the tile in the original projected coordinate system. This is used to map a local position within a tile to a global position in the original projected coordinate system; the size parameters characterize the width and height of the tile in the original projected coordinate system, and can be expressed as:

[0075]

[0076]

[0077] After obtaining the above parameters, for the pixels Coordinates of sampling points in the original projected coordinate system Normalization is performed to obtain the texture sampling coordinates corresponding to the pixel. :

[0078]

[0079]

[0080] The resulting texture sampling coordinates range from 0 to 1, consistent with the normalized coordinate space required for texture mapping. The reference coordinates and size parameters can be pre-calculated by the client on the central processing unit and passed to the graphics processor as global variables, allowing all pixels to share this set of parameters. This avoids repetitive parameter extraction and calculation at the pixel level, improving the efficiency of texture sampling coordinate calculation.

[0081] In other implementations, for multi-channel remote sensing images or multi-band fusion display scenarios, in addition to normalization processing, further processing logic such as texture coordinate offset and texture coordinate wrapping can be introduced to adapt to different data organization methods and display requirements.

[0082] In operation S205, texture sampling is performed on multiple original projection tiles according to the texture sampling coordinates to obtain the color value corresponding to each pixel. The multiple color values ​​corresponding to multiple pixels are output to obtain the display result of the remote sensing image.

[0083] In some embodiments, the above operation S205 may further include: rendering the triangular mesh to the three-dimensional Earth surface corresponding to the universal coordinate system according to the universal coordinates of each vertex in the triangular mesh; outputting the color value corresponding to each pixel to the position of the triangular mesh on the three-dimensional Earth surface; and performing seamless processing on the splicing area between adjacent original projection tiles to obtain the display result of multiple original projection tiles.

[0084] Specifically, the universal coordinates of each vertex in the triangular mesh are transformed into Cartesian coordinates and then fed into the graphics pipeline. The graphics pipeline correctly fits the triangular mesh onto the 3D Earth surface based on the model view projection matrix of the current view. At the pixel level, based on the calculated texture sampling coordinates, color values ​​are sampled from the corresponding original projected tile texture and output to the current pixel position, thus completing the rendering of a single pixel. The parallel rendering of all pixels together correctly renders an original projected tile onto the 3D Earth surface.

[0085] In some embodiments, seamless processing of the splicing area between adjacent original projection tiles may include at least one of the following: keeping the original coordinates and general coordinates of the vertices of the triangular mesh at the boundary of adjacent original projection tiles consistent, so that the boundary triangles of adjacent original projection tiles fit seamlessly on the three-dimensional Earth surface; extending the original projection tiles outward by a predetermined number of pixels; and performing smooth transition processing on the color values ​​of the pixels of the boundary triangles of adjacent original projection tiles through centroid coordinate interpolation.

[0086] Specifically, for the first processing method, the original projected coordinates of the vertices of the triangulation used by adjacent original projected tiles at the shared boundary are completely consistent with the universal coordinates. For example, the right boundary vertex of a tile has the same value as the left boundary vertex of the adjacent tile. In terms of vertex buffer organization, adjacent tiles can share boundary vertex data, so that the boundary triangles of adjacent tiles fit together geometrically without gaps on the three-dimensional Earth surface.

[0087] For the second processing method, when the original projected tile returns from the data service layer, its texture edge can be extended outward by a preset number of pixels. The extended pixel value can be obtained by copying the pixel value of the tile edge, and the preset number is preferably 1 pixel. This extension mechanism ensures that at the pixel level, pixels at the tile boundary can still sample valid texture pixels, avoiding black edge gaps caused by floating-point precision errors.

[0088] For the third processing method, since the pixel color value inside the boundary triangle is obtained by smoothing the texture sampling result at the vertex through centroid coordinate interpolation, adjacent original projected tiles share consistent original vertex coordinates at the boundary, which can make the color value of pixels near the boundary present a smooth transition when crossing the tile boundary, avoiding obvious color abrupt changes.

[0089] Furthermore, the tiles are strictly divided into rows and columns according to the pyramid hierarchy in the data service layer, with no overlapping areas between adjacent tiles, thereby avoiding flickering caused by repeated rendering at the splicing points of adjacent tiles.

[0090] By combining one or more of the above three processing methods, a seamless, continuous, and natural splicing display effect between adjacent original projection tiles can be achieved on the three-dimensional Earth surface, thereby ensuring the visual consistency of remote sensing images during large-scale roaming and multi-level scaling.

[0091] In other embodiments, to further seamlessly integrate with the 3D Earth client framework, the above-mentioned real-time remote sensing image display method can be encapsulated into a standard image provision interface that conforms to the client framework specifications. This allows the reprojection layer of the present invention to interface with the client's native image layer management, caching, and scheduling mechanisms, eliminating the need for additional pre-slicing caching of the original projection tiles and further reducing the consumption of storage and computing resources.

[0092] As can be seen from the above description, the embodiments of the present invention achieve at least the following technical effects:

[0093] (1) By transferring the projection transformation processing of remote sensing images from the server to the client, the server no longer needs to undertake computationally intensive tasks such as batch conversion of the original projection tiles to the target coordinate system tiles and tile recombination. It only needs to provide the original projection tiles to complete data distribution, which can significantly reduce the computational burden and storage pressure of the server and effectively alleviate the problem of large server load in related technologies.

[0094] (2) By constructing a triangular network on the client side based on the geographic range of the original projection tiles in the universal coordinate system, and using triangles as the basic geometric units to carry the local mapping relationship from the original projection coordinate system to the universal coordinate system, the process of reprojecting each pixel on the server side can be transformed into a lightweight geometric transformation of each triangle on the client side, so that remote sensing images under different original projection coordinate systems can be attached to the three-dimensional Earth surface in real time without pre-reprojection.

[0095] (3) By interpolating the centroid coordinates of the universal coordinates of the triangle vertices with the original coordinates, the sampling position of each pixel in the original projection coordinate system can be accurately restored at the pixel level. The texture sampling coordinates are obtained by combining the normalization of the original projection tile range parameters, so that the texture sampling process can maintain geometric accuracy while having high execution efficiency, and effectively avoid geometric distortion caused by projection transformation approximation.

[0096] (4) By sampling each boundary of the initial geographic range of the original projection tile and taking the extreme values ​​of latitude and longitude, the true coverage range of the original projection tile in the general coordinate system can be accurately enclosed in the case of nonlinear transformation relationship between the original projection coordinate system and the general coordinate system. This avoids the range estimation deviation caused by only using the four corner point transformation and improves the reliability of triangulation construction and tile scheduling.

[0097] (5) By sharing consistent vertex coordinates at the boundaries of adjacent original projection tiles, expanding outward by a preset number of pixels, and performing smooth transition processing on the boundary triangles, display defects such as cracks, misalignments, and color abrupt changes that may occur at the splicing points of adjacent tiles on the three-dimensional Earth surface can be effectively eliminated, so that remote sensing images can maintain a continuous and natural visual effect during large-scale roaming and multi-level scaling.

[0098] Based on the methods disclosed in the above embodiments, the present invention also provides a real-time display device for remote sensing images, which will be described below in conjunction with... Figure 3 The device is described in detail.

[0099] Figure 3 A block diagram of a remote sensing image real-time display device according to an embodiment of the present invention is shown schematically.

[0100] like Figure 3As shown, the remote sensing image real-time display device 300 according to this embodiment includes a tile receiving module 310, a range confirmation module 320, a triangulation construction module 330, a texture sampling coordinate calculation module 340, and a rendering output module 350.

[0101] Tile receiving module 310 is used to receive multiple original projection tiles of the remote sensing image to be displayed;

[0102] The range confirmation module 320 is used to calculate the geographical range of each of the original projection tiles in a general coordinate system. The geographical range includes the original coordinates of the original projection tiles in the original projection coordinate system of the remote sensing image and the general coordinates in the general coordinate system.

[0103] The triangular mesh construction module 330 is used to construct a triangular mesh for each of the original projected tiles according to the geographical range. The triangular mesh is composed of multiple triangles, and each triangle contains multiple pixels.

[0104] The texture sampling coordinate calculation module 340 is used to calculate the texture sampling coordinates corresponding to each pixel in the plurality of triangles based on the original coordinates and general coordinates of each vertex of the plurality of triangles;

[0105] The rendering output module 350 is used to perform texture sampling on the multiple original projection tiles according to the texture sampling coordinates to obtain the color value corresponding to each pixel, and output the multiple color values ​​corresponding to the multiple pixels to obtain the display result of the remote sensing image.

[0106] It should be noted that the embodiments of the device section are similar to those of the method section, and the technical effects achieved are also similar. For specific details, please refer to the above-mentioned method embodiment section, which will not be repeated here.

[0107] According to embodiments of the present invention, any plurality of the tile receiving module 310, the range confirmation module 320, the triangulation construction module 330, the texture sampling coordinate calculation module 340, and the rendering output module 350 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the tile receiving module 310, the range confirmation module 320, the triangulation construction module 330, the texture sampling coordinate calculation module 340, and the rendering output module 350 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these. Alternatively, at least one of the tile receiving module 310, the range confirmation module 320, the triangulation construction module 330, the texture sampling coordinate calculation module 340, and the rendering output module 350 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0108] Figure 4 A block diagram of an electronic device suitable for implementing a real-time display method for remote sensing images according to an embodiment of the present invention is shown schematically. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0109] like Figure 4 As shown, an electronic device 400 according to an embodiment of the present invention includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage portion 408 into a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include onboard memory configured for caching purposes. The processor 401 may include a single processing unit or multiple processing units configured to perform different actions of the method flow according to an embodiment of the present invention.

[0110] RAM 403 stores various programs and data required for the operation of electronic device 400. Processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Processor 401 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 402 and / or RAM 403. It should be noted that the programs may also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0111] According to an embodiment of the present invention, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to a bus 404. The electronic device 400 may also include one or more of the following components connected to the input / output (I / O) interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0112] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code configured to perform the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by processor 401, it performs the functions defined in the system of the embodiments of the present invention. According to embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0113] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0114] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0115] For example, according to embodiments of the present invention, a computer-readable storage medium may include the ROM 402 and / or RAM 403 described above and / or one or more memories other than ROM 402 and RAM 403.

[0116] Embodiments of the present invention also include a computer program product comprising a computer program containing program code configured to perform the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is configured to enable the electronic device to implement the real-time display method based on remote sensing images provided in the embodiments of the present invention.

[0117] When the computer program is executed by the processor 401, it performs the functions defined in the system / apparatus of this embodiment of the invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0118] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via communication section 409, and / or installed from removable medium 411. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0119] According to embodiments of the present invention, program code configured to execute the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0121] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0122] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A method for real-time display of remote sensing images, characterized in that, include: Receives multiple raw projection tiles of remote sensing images to be displayed; Calculate the geographic extent of each of the original projection tiles in a universal coordinate system, wherein the geographic extent includes the original coordinates of the original projection tiles in the original projection coordinate system of the remote sensing image and the universal coordinates in the universal coordinate system; Based on the geographical range, a triangular mesh is constructed for each of the original projected tiles. The triangular mesh is composed of multiple triangles, and each triangle contains multiple pixels. Based on the original coordinates and general coordinates of each vertex of the plurality of triangles, calculate the texture sampling coordinates corresponding to each pixel in the plurality of triangles; Based on the texture sampling coordinates, texture sampling is performed on the multiple original projection tiles to obtain the color value corresponding to each pixel. The multiple color values ​​corresponding to the multiple pixels are output to obtain the display result of the remote sensing image.

2. The method according to claim 1, characterized in that, The calculation of the geographic extent of each of the original projected tiles in a universal coordinate system includes: Obtain the coordinates of multiple corner points of each original projection tile in the original projection coordinate system; The coordinates of the multiple corner points are converted into general coordinates under the general coordinate system using a coordinate transformation library to obtain the initial geographical range of each original projection tile; The geographic range of the original projected tile in the universal coordinate system is determined based on the boundary of the initial geographic range.

3. The method according to claim 2, characterized in that, The universal coordinate system includes latitude and longitude coordinates. Determining the geographical extent of the original projected tile in the universal coordinate system based on the boundary of the initial geographical extent includes: Sample each boundary of the initial geographical area to obtain multiple sampling points on each boundary; The coordinates of each sampling point in the original projected coordinate system are converted into general coordinates in the general coordinate system using the coordinate transformation library. The geographical range of the original projected tile in the universal coordinate system is determined based on the extreme longitude and latitude values ​​in the universal coordinate system.

4. The method according to claim 1, characterized in that, The step of calculating the texture sampling coordinates corresponding to each pixel in the plurality of triangles based on the original coordinates and general coordinates of each vertex of the plurality of triangles includes: Based on the common coordinates of each vertex of the plurality of triangles, the centroid coordinates corresponding to each pixel in the plurality of triangles are determined. The centroid coordinates include multiple weight values ​​corresponding to the vertices of the triangles, and the sum of the multiple weight values ​​is equal to 1. Based on the centroid coordinates and the original coordinates corresponding to the vertices of the triangle, centroid coordinate interpolation is performed on the original coordinates to obtain the sampling point coordinates of the pixel in the original projected coordinate system; The texture sampling coordinates corresponding to the pixel are determined based on the sampling point coordinates.

5. The method according to claim 4, characterized in that, Determining the texture sampling coordinates corresponding to the pixel based on the sampling point coordinates includes: Obtain the range parameters of the original projection tile in the original projection coordinate system, the range parameters including the reference coordinates and size parameters; The sampling point coordinates are normalized according to the range parameter to obtain the texture sampling coordinates corresponding to the pixel.

6. The method according to claim 1, characterized in that, The step of outputting the multiple color values ​​corresponding to multiple pixels to obtain the display result of the multiple original projection tiles includes: Based on the universal coordinates of each vertex in the triangulation, the triangulation is rendered onto the three-dimensional Earth surface corresponding to the universal coordinate system. The color value corresponding to each pixel is output to the position of the triangular mesh on the three-dimensional Earth surface; Seamless processing is performed on the splicing area between adjacent original projection tiles to obtain the display result of the multiple original projection tiles.

7. The method according to claim 6, characterized in that, The seamless processing of the splicing area between adjacent original projection tiles includes at least one of the following: The original coordinates and universal coordinates of the vertices of the triangulation at the boundary of adjacent original projection tiles are kept consistent, so that the boundary triangles of the adjacent original projection tiles fit seamlessly on the three-dimensional Earth surface. The original projection tile is expanded outward by a predetermined number of pixels; The color values ​​of pixels in the boundary triangles of adjacent original projected tiles are smoothly transitioned by using barycentric coordinate interpolation.

8. A real-time display device for remote sensing images, characterized in that, include: A tile receiving module is used to receive multiple raw projection tiles of the remote sensing image to be displayed; The range confirmation module is used to calculate the geographical range of each of the original projection tiles in a general coordinate system. The geographical range includes the original coordinates of the original projection tiles in the original projection coordinate system of the remote sensing image and the general coordinates in the general coordinate system. A triangular mesh construction module is used to construct a triangular mesh for each of the original projected tiles according to the geographical range. The triangular mesh is composed of multiple triangles, and each triangle contains multiple pixels. The texture sampling coordinate calculation module is used to calculate the texture sampling coordinates corresponding to each pixel in the plurality of triangles based on the original coordinates and general coordinates of each vertex of the plurality of triangles; The rendering output module is used to perform texture sampling on the multiple original projection tiles according to the texture sampling coordinates to obtain the color value corresponding to each pixel, and output the multiple color values ​​corresponding to the multiple pixels to obtain the display result of the remote sensing image.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program or instructions thereon, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images