Geographic entity information display method based on live-action three-dimensional modeling

Abstract

The invention relates to the technical field of geographic information display, in particular to a geographic entity information display method based on live-action three-dimensional modeling, which comprises the following steps: acquiring a multi-angle remote sensing image of a geographic entity area, and performing target detection on the multi-angle remote sensing image to obtain each geographic entity type in the multi-angle remote sensing image; and matching the geographic entity type with a preset geographic knowledge graph to obtain geographic information corresponding to the geographic entity type. By processing the multi-angle remote sensing image, the geographic entity can be effectively identified, more accurate geographic information is extracted according to different visual angles, the classification and identification precision of the geographic entity can be improved through the process, and the geographic entity type is matched with the geographic knowledge graph, so that the accuracy of classification and identification of the geographic entity is improved. Therefore, each geographic entity not only has description of a physical space position, but also can be combined with rich semantic information.

Description

Technical Field

[0001] The present invention relates to the technical field of geographic information display, and in particular to a geographic entity information display method based on real-scene three-dimensional modeling. Background Art

[0002] A geographic entity refers to various things or objects that exist in geographic space. They usually have specific spatial positions, forms, and attributes and can be located by geographic coordinates. Geographic entities include not only natural geographical features, but also various cultural facilities or phenomena generated by human activities.

[0003] Traditional methods usually rely on relatively simple image processing or manual measurement methods. The recognition accuracy is limited by the quality, resolution and analysis algorithm capabilities of remote sensing images. It is often difficult to accurately identify geographic entities (such as buildings, roads, bridges, etc.) under complex terrain or different lighting conditions. Traditional methods may lack consideration of different perspectives, resulting in insufficient accuracy in the classification and recognition of geographic entities. Traditional methods usually only focus on the physical location and basic attributes (such as coordinates, size, etc.) of geographic entities, and lack semantic understanding of geographic entities (such as functional use, historical background, environmental impact, etc.). The lack of such information makes it difficult for traditional geographic data presentation to meet the needs of in-depth analysis and practical applications, especially in the fields of urban planning and environmental monitoring. It is impossible to provide comprehensive decision support. Traditional methods often only provide planar data or simple three-dimensional models of geographic entities, but lack in-depth spatial analysis capabilities. This makes it difficult for traditional methods to provide sufficient data support when conducting complex spatial analysis, trend forecasting and decision support, affecting the scientificity and effectiveness of decision-making. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method for displaying geographic entity information based on real-scene three-dimensional modeling.

[0005] The technical solution adopted to solve the above technical problems is: a method for displaying geographic entity information based on real-scene three-dimensional modeling, including:

[0006] Acquire a multi-angle remote sensing image of the geographic entity area, and perform target detection on the multi-angle remote sensing image to obtain each geographic entity type in the multi-angle remote sensing image;

[0007] Matching the geographic entity type with a preset geographic knowledge graph to obtain geographic information corresponding to the geographic entity type;

[0008] Performing three-dimensional reconstruction of the geographic entity area based on the multi-angle remote sensing image to obtain a real-scene three-dimensional model corresponding to the geographic entity area;

[0009] Establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image, and mapping each geographic entity type in the multi-angle remote sensing image to the real-scene three-dimensional model based on the position mapping table to obtain a dynamic real-scene three-dimensional model;

[0010] The dynamic real-scene three-dimensional model is integrated based on the geographic information corresponding to the geographic entity type to obtain a dynamic information real-scene three-dimensional model, and the dynamic information real-scene three-dimensional model is displayed based on a three-dimensional rendering engine.

[0011] Preferably, performing target detection on the multi-angle remote sensing image to obtain each geographic entity type in the multi-angle remote sensing image includes:

[0012] Performing image enhancement on the multi-angle remote sensing image based on a pre-trained image enhancement model to obtain a multi-angle standard remote sensing image;

[0013] Target detection is performed on the multi-angle remote sensing image based on a pre-trained target detection model to obtain each geographic entity type in the multi-angle remote sensing image.

[0014] Preferably, the target detection model adopts an improved YOLOv7-tiny model, and introduces a visual channel attention mechanism into the backbone feature extraction network of the YOLOv7-tiny model, wherein the visual channel attention mechanism is used to perform a global average pooling operation on the input feature map, and the size of the feature map is C*H*W, wherein C represents the number of channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map. The expression of the visual channel attention mechanism is as follows:

[0015]

[0016] Among them, z c represents the output of the visual channel attention mechanism, u c (i,j) represents the output of feature map pixel (i,j).

[0017] Preferably, the image enhancement model includes a shared encoding module, an edge decoding module, a color decoding module and a feature fusion module, the shared encoding module is used to perform feature extraction on the multi-angle remote sensing image to obtain edge features and color features corresponding to the multi-angle remote sensing image, the edge decoding module is used to decode the edge features corresponding to the multi-angle remote sensing image based on the edge block and the image block to obtain the image edge result of the multi-angle remote sensing image, the color decoding module is used to decode the color features of the multi-angle remote sensing image based on the HSV color map to obtain the image color result of the multi-angle remote sensing image, and the feature fusion module is used to splice the image edge result and the image color result of the multi-angle remote sensing image to obtain a spliced ​​feature map, and perform feature fusion on the spliced ​​feature map to obtain a standard multi-angle remote sensing image.

[0018] Preferably, three-dimensional reconstruction of the geographic entity area is performed based on the multi-angle remote sensing image to obtain a real-scene three-dimensional model corresponding to the geographic entity area, including:

[0019] Performing distortion correction on the multi-angle remote sensing image to obtain a multi-angle corrected remote sensing image;

[0020] Analyzing the aerial triangulation within the geographic entity area based on photogrammetry to obtain the exterior orientation elements corresponding to the multi-angle remote sensing image;

[0021] Extracting geometric information of each geographic entity in the multi-angle remote sensing image according to the exterior orientation elements based on the adjustment solution matrix;

[0022] The geometric information of each geographic entity is converted into three-dimensional point cloud data based on a pre-trained three-dimensional reconstruction model, and a real-scene three-dimensional model corresponding to the geographic entity area is obtained based on the three-dimensional point cloud data.

[0023] Preferably, performing distortion correction on the multi-angle remote sensing image to obtain a multi-angle corrected remote sensing image includes:

[0024] The multi-angle remote sensing image is subjected to camera distortion correction based on the image point coordinate transformation model to obtain a multi-angle camera distortion corrected remote sensing image, wherein the camera distortion correction calculation formula is as follows:

[0025]

[0026] Among them, (X, Y) represents the real pixel coordinates of the multi-angle remote sensing image, (F X ,F Y ) represents the coordinate transformation function in camera distortion correction, (α, β) represents the remote sensing image corrected by multi-angle camera distortion;

[0027] The multi-angle camera distortion-corrected remote sensing image is subjected to geometric correction processing to obtain a multi-angle corrected remote sensing image, wherein the calculation formula of the geometric correction processing is as follows:

[0028] G Q (x Q ,y Q )=G N (x N ,y N )(X,Y);

[0029] Among them, G Q (x Q ,y Q ) represents the gray value of the resampled point Q in the multi-angle camera distortion correction remote sensing image, G N (x N ,y N ) represents the corresponding gray value of the pixel closest to the resampling point Q, (x Q ,y Q ) represents G Q (x Q ,y Q )’s grayscale coordinate value, (x N ,y N ) represents G N (x N ,y N )’s grayscale coordinate value.

[0030] Preferably, the three-dimensional reconstruction model includes a feature extraction module, a feature aggregation module, a feature decomposition module and a feature regression module, the feature extraction module is used to extract the first local feature of the geometric information of the preset multi-angle reference image and the second local feature of the geometric information image of the multi-angle remote sensing image, and convert the second local feature into the viewing angle of the first local feature to obtain a third local feature, and match the first local feature and the third local feature in pairs to obtain multiple cost volumes.

[0031] Preferably, the feature aggregation module adopts a feature pyramid network, and aggregates the multiple cost bodies based on the feature pyramid network to obtain an aggregated cost body. The feature decomposition module is used to downsample the aggregated cost body to obtain high-dimensional features, and then perform capsule operation according to the feature channel dimension to decouple the aggregated cost body into multiple capsule blocks. The feature regression module is used to encode the high-dimensional features in the multiple capsule blocks through a preset capsule convolution layer to obtain multiple high-dimensional feature semantics, and splice the multiple high-dimensional feature semantics with the multiple capsule blocks to obtain multiple spliced ​​features, and regress the multiple spliced ​​features through a normalization operation to obtain multiple depth maps, and convert the multiple depth maps into three-dimensional point cloud data.

[0032] Preferably, establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image includes:

[0033] Acquire three-dimensional point cloud data of the real-scene three-dimensional model, and extract feature points from the three-dimensional point cloud data to obtain a first key feature point set;

[0034] Extracting feature points from the multi-angle remote sensing image to obtain a second key feature point set;

[0035] The first key feature point set and the second key feature point set are matched based on a scale-invariant feature transformation method to obtain matching feature point pairs.

[0036] Preferably, establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image also includes:

[0037] Calculating the pose parameters of the multi-angle remote sensing image relative to the real-scene three-dimensional model based on the matching feature point pairs based on visual positioning technology, wherein the pose parameters include a rotation matrix and a translation vector;

[0038] A position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image is established based on the posture parameters, wherein the position mapping table includes a correspondence between image coordinates and three-dimensional coordinates.

[0039] The beneficial effects of the present invention are as follows: (1) The present invention can effectively identify geographic entities (such as buildings, roads, bridges, vegetation, etc.) by processing multi-angle remote sensing images, and extract more accurate geographic information according to different perspectives. Through this process, the accuracy of geographic entity classification and recognition can be improved, and the geographic entity type can be matched with the geographic knowledge graph, so that each geographic entity not only has a description of the physical spatial location, but also can be combined with rich semantic information (such as functional use, historical background, environmental impact, etc.). The integration of such information helps to enhance the understanding of geographic entities and make geographic data more valuable for application; (2) The present invention realizes high-precision three-dimensional modeling of geographic entity areas by integrating multi-angle remote sensing images, thereby providing a realistic and accurate spatial display. The real-life three-dimensional model can It can present the real spatial structure in different scenes, so that users can more intuitively understand and analyze the distribution and layout of geographic entities. Moreover, through the established location mapping table, each geographic entity type in the multi-angle remote sensing image can be accurately mapped to the real-life three-dimensional model, thereby forming a dynamic real-life three-dimensional model. This means that in the model display, the location, form and actual changes of the geographic entity can be reflected synchronously, ensuring that the display content is synchronized with the reality and is updated with the passage of time and environmental changes. (3) By integrating the geographic information corresponding to the geographic entity type, the present invention can form a dynamic information real-life three-dimensional model that combines physical space and semantic information. This model not only has spatial visual effects, but also carries real-time updated geographic data, so that users can obtain more comprehensive and in-depth information when browsing. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic flow chart of the steps of an overall method in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] Embodiment 1, as Figure 1 As shown, the method for displaying geographic entity information based on real-scene three-dimensional modeling proposed by the present invention includes:

[0042] S1. Acquire a multi-angle remote sensing image of a geographic entity area, and perform target detection on the multi-angle remote sensing image to obtain each geographic entity type in the multi-angle remote sensing image;

[0043] S2. Matching the geographic entity type with the preset geographic knowledge graph to obtain the geographic information corresponding to the geographic entity type;

[0044] S3, reconstructing the geographic entity area in three dimensions based on the multi-angle remote sensing image to obtain a real-scene three-dimensional model corresponding to the geographic entity area;

[0045] S4, establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image, and mapping each geographic entity type in the multi-angle remote sensing image to the real-scene three-dimensional model based on the position mapping table to obtain a dynamic real-scene three-dimensional model;

[0046] S5. Integrate the dynamic real-scene three-dimensional model based on the geographic information corresponding to the geographic entity type to obtain a dynamic information real-scene three-dimensional model, and display the dynamic information real-scene three-dimensional model based on a three-dimensional rendering engine.

[0047] In the present invention, remote sensing images refer to surface image data acquired from a distance by sensors (such as sensors on platforms such as satellites, drones, and airplanes); multi-angle remote sensing images refer to remote sensing image data acquired from different angles or viewing angles. These images provide observations of the same geographical area from different perspectives, which can usually help to more comprehensively understand the morphology and structure of geographical entities. The use of multi-angle images for analysis helps to improve the accuracy of three-dimensional reconstruction; target detection is a task in computer vision, which aims to automatically identify and locate objects or areas of interest from images. In the display of geographical entity information, target detection is used to identify different geographical entities in images (such as buildings, roads, trees, etc.); geographical entity types refer to different categories of geographical entities identified in remote sensing images, and each type usually corresponds to different geographical elements or features, such as buildings, roads, rivers, mountains, forests, etc.; a geographical knowledge graph is a structured database that stores and represents geographical knowledge in the form of a graph. The nodes in the graph usually represent geographical entities (such as cities, roads, buildings, etc.), and the edges represent the relationships between these entities (such as adjacency, belonging to, passing through, etc.). In this method, the geographic knowledge graph is used to match the geographic entity types obtained by target detection with specific geographic information (such as coordinates, functions, attributes, etc.) to provide detailed background information about the geographic entities; the three-dimensional rendering engine is a computer graphics tool used to convert a three-dimensional model into a two-dimensional image that can be presented on a display. The rendering engine processes the geometric data, materials, lighting, shadows, etc. in the three-dimensional model to generate the final visual effect. In the dynamic information real-scene three-dimensional model display, the rendering engine generates a realistic scene display based on real-time data (such as lighting changes, perspective changes, etc.).

[0048] Embodiment 2, the method for displaying geographic entity information based on real-scene three-dimensional modeling proposed by the present invention, compared with embodiment 1, this embodiment also includes: performing target detection on multi-angle remote sensing images to obtain each geographic entity type in the multi-angle remote sensing images, including:

[0049] Perform image enhancement on multi-angle remote sensing images based on a pre-trained image enhancement model to obtain multi-angle standard remote sensing images;

[0050] Based on the pre-trained target detection model, target detection is performed on multi-angle remote sensing images to obtain each geographic entity type in the multi-angle remote sensing images.

[0051] In an optional embodiment, the target detection model adopts an improved YOLOv7-tiny model, and introduces a visual channel attention mechanism in the backbone feature extraction network of the YOLOv7-tiny model, wherein the visual channel attention mechanism is used to perform a global average pooling operation on the input feature map, and the size of the feature map is C*H*W, wherein C represents the number of channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map. The expression of the visual channel attention mechanism is as follows:

[0052]

[0053] Among them, z c represents the output of the visual channel attention mechanism, u c (i,j) represents the output of feature map pixel (i,j).

[0054] It should be noted that YOLOv7-tiny is a lightweight version of YOLOv7. In order to improve the detection speed and reduce the model size, YOLOv7-tiny simplifies the network structure of YOLOv7, reducing the number of layers and parameters. In target detection, the backbone network is responsible for extracting features from the input image. The channel attention mechanism is an attention mechanism that focuses on different channels of the input feature map to weight the importance of the channel. In target detection, each channel of the feature map represents a specific feature (such as color, texture, edge, etc.). The channel attention mechanism automatically adjusts the weight of each channel, thereby improving the response of key features, suppressing irrelevant or unimportant features, and improving the representation ability of the model.

[0055] In an optional embodiment, the image enhancement model includes a shared encoding module, an edge decoding module, a color decoding module and a feature fusion module. The shared encoding module is used to extract features of multi-angle remote sensing images to obtain edge features and color features corresponding to the multi-angle remote sensing images. The edge decoding module is used to decode the edge features corresponding to the multi-angle remote sensing images based on edge blocks and image blocks to obtain image edge results of the multi-angle remote sensing images. The color decoding module is used to decode the color features of the multi-angle remote sensing images based on the HSV color map to obtain the image color results of the multi-angle remote sensing images. The feature fusion module is used to splice the image edge results and image color results of the multi-angle remote sensing images to obtain a spliced ​​feature map, and perform feature fusion on the spliced ​​feature map to obtain a standard multi-angle remote sensing image.

[0056] It should be noted that the encoding module is usually used to extract features from the input image, mapping the original image to a latent space (usually a low-dimensional space) to extract high-level features of the image. The shared encoding module means that this module uses a shared structure to process images from different angles when processing multi-angle remote sensing images, which means that the module uses the same weights and network structure when extracting multi-angle image features, thereby improving the efficiency of the model and reducing the amount of calculation; the task of the edge decoding module is to recover or generate the edge information of the image from the features of the image. The edge is the embodiment of the outline or structure of the object in the image, and is often used in tasks such as image segmentation and object recognition. Here, the edge decoding module uses a specific decoding mechanism (such as deconvolution, inverse The color decoding module is used to recover the color information of the image from the color features of the image. In image processing, colors are usually represented by different color spaces, such as RGB (red, green, blue) or HSV (hue, saturation, brightness). Here, the color decoding module decodes the color features of the image based on the HSV color map to reconstruct the color information of the image. The function of the feature fusion module is to combine the edge results and color results of the image to form a new feature map for further processing and enhancement. This process usually requires fusing different types of features (such as edge information and color information) into a new feature map to provide more comprehensive image information.

[0057] In an optional embodiment, three-dimensional reconstruction of a geographic entity region is performed based on multi-angle remote sensing images to obtain a real-scene three-dimensional model corresponding to the geographic entity region, including:

[0058] Perform distortion correction on multi-angle remote sensing images to obtain multi-angle corrected remote sensing images;

[0059] Aerial triangulation within geographic entity areas is analyzed based on photogrammetry to obtain exterior orientation elements corresponding to multi-angle remote sensing images;

[0060] Extract the geometric information of each geographic entity in the multi-angle remote sensing image according to the exterior orientation elements based on the adjustment matrix;

[0061] Based on the pre-trained 3D reconstruction model, the geometric information of each geographic entity is converted into 3D point cloud data, and the real-scene 3D model corresponding to the geographic entity area is obtained based on the 3D point cloud data.

[0062] It should be noted that distortion correction refers to correcting the geometric distortion caused by the optical error of the imaging system or other factors in the remote sensing image, so that the geometric shape of the image is more consistent with the actual ground conditions. The distortion of the remote sensing image may include lens distortion, projection distortion, perspective distortion, etc. After distortion correction, more accurate geographic information can be obtained; exterior orientation elements refer to parameters that describe the external position and attitude of the remote sensing image. In photogrammetry, exterior orientation elements include the geographic coordinates (position) of the image and the orientation (attitude) when shooting; aerial triangulation is a technology used to determine the spatial position and attitude of an image, which is usually used in aerial or satellite images. In this process, the exterior orientation elements of the camera are calculated by measuring the relative position relationship under different perspectives using known ground control points (such as GPS positions or ground landmarks). Simply put, aerial triangulation is through the use of triangles. The spatial coordinates and attitude information of the image are inferred based on the geometric relationship (viewing angle and distance); adjustment solution is the process used to optimize the measurement data in photogrammetry, aiming to minimize the error and obtain the optimal solution through mathematical models. The adjustment solution matrix is ​​the mathematical tool used in this process, usually a set of linear equations or a least squares solution matrix; geometric information refers to the data such as the spatial shape, position and size of the object in the image. In remote sensing image processing, geometric information usually includes the spatial coordinates, shape outline, size, etc. of the geographic entity. By extracting the geometric information in the image, the characteristics and attributes of the actual geographic area represented by the image can be understood; 3D point cloud data is a set of points distributed in three-dimensional space obtained by measurement. Each point usually includes three-dimensional coordinates (X, Y, Z), which represent the specific position of the surface of the object in space. Through a large number of three-dimensional point clouds, the shape of the object or terrain can be reconstructed.

[0063] In an optional embodiment, performing distortion correction on the multi-angle remote sensing image to obtain the multi-angle corrected remote sensing image includes:

[0064] The camera distortion correction of multi-angle remote sensing images is performed based on the image point coordinate transformation model to obtain multi-angle camera distortion corrected remote sensing images. The calculation formula of camera distortion correction is as follows:

[0065]

[0066] Among them, (X, Y) represents the real pixel coordinates of the multi-angle remote sensing image, (F X ,F Y ) represents the coordinate transformation function in camera distortion correction, (α, β) represents the remote sensing image corrected by multi-angle camera distortion;

[0067] The multi-angle camera distortion-corrected remote sensing image is subjected to geometric correction processing to obtain a multi-angle corrected remote sensing image, wherein the calculation formula of the geometric correction processing is as follows:

[0068] G Q (x Q ,y Q )=G N (x N ,y N )(X,Y);

[0069] Among them, G Q (x Q ,y Q ) represents the gray value of the resampled point Q in the multi-angle camera distortion correction remote sensing image, G N (x N ,y N ) represents the corresponding gray value of the pixel closest to the resampling point Q, (x Q ,y Q ) represents G Q (x Q ,y Q )’s grayscale coordinate value, (x N ,y N ) represents G N (x N ,y N )’s grayscale coordinate value.

[0070] In an optional embodiment, the three-dimensional reconstruction model includes a feature extraction module, a feature aggregation module, a feature decomposition module and a feature regression module. The feature extraction module is used to extract the first local feature of the geometric information of a preset multi-angle reference image and the second local feature of the geometric information image of the multi-angle remote sensing image, and convert the second local feature into the viewing angle of the first local feature to obtain a third local feature, and match the first local feature and the third local feature in pairs to obtain multiple cost volumes.

[0071] It should be noted that feature extraction is a basic step in computer vision and image processing, which aims to extract representative key points or local features from the image. In 3D reconstruction, the feature extraction module is responsible for extracting geometric information from multi-angle remote sensing images; the first local feature refers to the preliminary geometric features extracted from the preset reference image. These features are usually local areas in the image with high recognition, stability and repeatability (such as corners, edges, etc.). These features can be accurately identified in the image and used as matching benchmarks in subsequent processing steps; the second local feature is the geometric feature extracted from the multi-angle remote sensing image, which usually refers to the features obtained from the target remote sensing image (that is, the image different from the reference image). These features represent the local geometric information in the image, but due to the different shooting angles, the second local feature usually has differences from the first local feature. Geometric changes such as spatial displacement or rotation; Perspective refers to the specific position and orientation of the camera or remote sensing sensor when taking an image. Different perspectives will affect the shape, proportion and display of geometric features of objects or features in the image. In 3D reconstruction, it is necessary to consider the geometric relationship between images at different perspectives; The third local feature is a new feature obtained by converting the second local feature to the same perspective as the first local feature. This process is usually achieved through perspective conversion. The goal is to standardize the features of different perspectives for subsequent matching; Pairwise matching refers to comparing and matching the features in two images in image processing. Here, the first local feature and the third local feature are matched pairwise, that is, through some distance metric (such as Euclidean distance, angular distance, etc.), find out whether these feature points correspond in space; Cost volume (Cost volume) Volume) is a data structure used to measure the matching error in the matching process. It represents the similarity or cost between each pair of matching feature points in the image, and is usually used to measure the point cloud matching quality under different perspectives in 3D reconstruction; feature aggregation refers to combining feature information in different images, and is usually used to fuse local features in multi-angle or multi-frame images into global features; feature decomposition refers to decomposing the aggregated features, usually with the purpose of extracting more detailed local features or separating the contributions of different feature dimensions. The decomposed features help understand the geometric structure of each part of the image; feature regression refers to predicting or optimizing the target value based on the extracted feature information. In 3D reconstruction, the feature regression module can be used to optimize the position, posture and other parameters of feature points to ensure the accuracy of the model.

[0072] In an optional embodiment, the feature aggregation module adopts a feature pyramid network, and aggregates multiple cost bodies based on the feature pyramid network to obtain an aggregated cost body. The feature decomposition module is used to downsample the aggregated cost body to obtain high-dimensional features, and then perform capsule operation according to the feature channel dimension to decouple the aggregated cost body into multiple capsule blocks. The feature regression module is used to encode the high-dimensional features in the multiple capsule blocks through a preset capsule convolution layer to obtain multiple high-dimensional feature semantics, and splice the multiple high-dimensional feature semantics with the multiple capsule blocks to obtain multiple spliced ​​features, and regress the multiple spliced ​​features through a normalization operation to obtain multiple depth maps, and convert the multiple depth maps into three-dimensional point cloud data.

[0073] It should be noted that the Feature Pyramid Network is a neural network architecture commonly used for multi-scale target detection and feature extraction. Its core idea is to effectively capture information of different scales by constructing feature maps of different resolutions (i.e., feature pyramids). FPN extracts and fuses features from different levels of the network, so that high-level features can contain rich semantic information, while low-level features retain detailed information; the encapsulation operation originates from Capsule Networks, a deep learning method designed to overcome the problem of spatial information loss in traditional convolutional neural networks (CNNs). Capsule Networks retain spatial information by encapsulating the outputs of multiple neurons into a "capsule", and each capsule represents a specific attribute of an object (such as position, posture, rotation, etc.); encapsulation is to organize the high-dimensional features after feature decomposition according to the "feature channel dimension" to form multiple capsule blocks, each of which captures a specific category or local area. feature information, and can effectively process the transformation information (such as rotation, translation, etc.) in the image; the capsule convolution layer is a convolution layer specially designed to process capsules. Unlike the traditional convolution layer, the capsule convolution layer transfers information between multiple capsule blocks to capture spatial transformations and richer contextual relationships; feature splicing refers to connecting features from multiple different sources (such as high-dimensional features of multiple capsule blocks) in the feature dimension to form a larger feature vector or feature map; a depth map is a two-dimensional image, each pixel value represents the distance from the camera or observation point to the surface of the object. In three-dimensional reconstruction, the depth map is used to extract the spatial position information of the object or scene from the two-dimensional image; a three-dimensional point cloud is a three-dimensional spatial data set composed of spatial coordinate (X, Y, Z) data obtained from multiple perspectives or depth maps. In three-dimensional reconstruction, a point cloud is an important data structure that describes the geometric shape of an object or scene, and is usually used for further three-dimensional modeling, visualization or object recognition.

[0074] In an optional embodiment, establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image includes:

[0075] Acquire three-dimensional point cloud data of the real-scene three-dimensional model, and extract feature points from the three-dimensional point cloud data to obtain a first key feature point set;

[0076] Extracting feature points from multi-angle remote sensing images to obtain a second key feature point set;

[0077] The first key feature point set and the second key feature point set are matched based on a scale-invariant feature transformation method to obtain matching feature point pairs.

[0078] It should be noted that feature point extraction is to extract points with recognition, stability and significance from images or point clouds. These points are usually used for subsequent matching, registration and modeling tasks. Feature points have certain spatial uniqueness in images or three-dimensional space and can be used to describe local information of objects; SIFT (Scale Invariant Feature Transform) is a feature extraction and description algorithm commonly used in image matching, object recognition, three-dimensional reconstruction and other tasks. The SIFT algorithm can extract scale-invariant key points from images. These key points are highly robust to image rotation, scale changes and illumination changes; matching feature point pairs refer to corresponding feature point pairs found in different images or point cloud data based on certain similarity metrics (such as matching of SIFT descriptors). These feature point pairs represent the same points in the same actual object or scene observed from different perspectives or data sources (such as three-dimensional point clouds and remote sensing images).

[0079] In an optional embodiment, establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image also includes:

[0080] Based on the visual positioning technology, the pose parameters of the multi-angle remote sensing image relative to the real-scene three-dimensional model are calculated according to the matching feature point pairs, wherein the pose parameters include the rotation matrix and the translation vector;

[0081] A position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image is established based on the posture parameters, wherein the position mapping table includes the corresponding relationship between the image coordinates and the three-dimensional coordinates.

[0082] It should be noted that visual positioning technology refers to estimating the position and posture of a camera through image or video information. Usually, these technologies rely on feature points extracted from images (such as feature points extracted by algorithms such as SIFT, SURF, and ORB) to infer the position. The positioning process usually includes two parts: camera pose estimation and feature matching. Camera pose estimation refers to estimating the position (coordinates in space) and posture (i.e., the direction of rotation of the camera). Feature matching and registration refers to determining the specific position of the camera by matching the feature points in the image with the feature points in a known three-dimensional model or map. The rotation matrix (Rotation Matrix) refers to describing the direction of rotation of an object, indicating the rotation transformation from the world coordinate system to the object coordinate system. The translation vector (Translation Vector) refers to describing the position of an object, indicating the translation from the origin of the world coordinate system to the origin of the object coordinate system.

[0083] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto, and various changes can be made within the knowledge scope of technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A method for displaying geographic entity information based on real-scene three-dimensional modeling, characterized in that: include: Acquire a multi-angle remote sensing image of the geographic entity area, and perform target detection on the multi-angle remote sensing image to obtain each geographic entity type in the multi-angle remote sensing image; Matching the geographic entity type with a preset geographic knowledge graph to obtain geographic information corresponding to the geographic entity type; Performing three-dimensional reconstruction of the geographic entity area based on the multi-angle remote sensing image to obtain a real-scene three-dimensional model corresponding to the geographic entity area; Establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image, and mapping each geographic entity type in the multi-angle remote sensing image to the real-scene three-dimensional model based on the position mapping table to obtain a dynamic real-scene three-dimensional model; The dynamic real-scene three-dimensional model is integrated based on the geographic information corresponding to the geographic entity type to obtain a dynamic information real-scene three-dimensional model, and the dynamic information real-scene three-dimensional model is displayed based on a three-dimensional rendering engine.

2. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 1, characterized in that: Performing target detection on the multi-angle remote sensing image to obtain each geographic entity type in the multi-angle remote sensing image includes: Performing image enhancement on the multi-angle remote sensing image based on a pre-trained image enhancement model to obtain a multi-angle standard remote sensing image; Target detection is performed on the multi-angle remote sensing image based on a pre-trained target detection model to obtain each geographic entity type in the multi-angle remote sensing image.

3. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 2 is characterized in that: The target detection model adopts an improved YOLOv7-tiny model, and introduces a visual channel attention mechanism into the backbone feature extraction network of the YOLOv7-tiny model, wherein the visual channel attention mechanism is used to perform a global average pooling operation on the input feature map, and the size of the feature map is C*H*W, wherein C represents the number of channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map. The expression of the visual channel attention mechanism is as follows: Among them, z c represents the output of the visual channel attention mechanism, u c (i,j) represents the output of feature map pixel (i,j).

4. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 3 is characterized in that: The image enhancement model includes a shared encoding module, an edge decoding module, a color decoding module and a feature fusion module. The shared encoding module is used to extract features from the multi-angle remote sensing image to obtain edge features and color features corresponding to the multi-angle remote sensing image. The edge decoding module is used to decode the edge features corresponding to the multi-angle remote sensing image based on the edge block and the image block to obtain the image edge result of the multi-angle remote sensing image. The color decoding module is used to decode the color features of the multi-angle remote sensing image based on the HSV color map to obtain the image color result of the multi-angle remote sensing image. The feature fusion module is used to splice the image edge result and the image color result of the multi-angle remote sensing image to obtain a spliced ​​feature map, and perform feature fusion on the spliced ​​feature map to obtain a standard multi-angle remote sensing image.

5. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 1, characterized in that: The geographical entity area is three-dimensionally reconstructed based on the multi-angle remote sensing image to obtain a real-scene three-dimensional model corresponding to the geographical entity area, including: Performing distortion correction on the multi-angle remote sensing image to obtain a multi-angle corrected remote sensing image; Analyzing the aerial triangulation within the geographic entity area based on photogrammetry to obtain the exterior orientation elements corresponding to the multi-angle remote sensing image; Extracting geometric information of each geographic entity in the multi-angle remote sensing image according to the exterior orientation elements based on the adjustment solution matrix; The geometric information of each geographic entity is converted into three-dimensional point cloud data based on a pre-trained three-dimensional reconstruction model, and a real-scene three-dimensional model corresponding to the geographic entity area is obtained based on the three-dimensional point cloud data.

6. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 5, characterized in that: Performing distortion correction on the multi-angle remote sensing image to obtain a multi-angle corrected remote sensing image includes: The multi-angle remote sensing image is subjected to camera distortion correction based on the image point coordinate transformation model to obtain a multi-angle camera distortion corrected remote sensing image, wherein the calculation formula of the camera distortion correction is as follows: Among them, (X, Y) represents the real pixel coordinates of the multi-angle remote sensing image, (F X ,F Y ) represents the coordinate transformation function in camera distortion correction, (α, β) represents the remote sensing image corrected by multi-angle camera distortion; The multi-angle camera distortion-corrected remote sensing image is subjected to geometric correction processing to obtain a multi-angle corrected remote sensing image, wherein the calculation formula of the geometric correction processing is as follows: G Q (x Q ,y Q )=G N (x N ,y N )(X,Y); Among them, G Q (x Q ,y Q ) represents the gray value of the resampled point Q in the multi-angle camera distortion correction remote sensing image, G N (x N ,y N ) represents the corresponding gray value of the pixel closest to the resampling point Q, (x Q ,y Q ) represents G Q (x Q ,y Q )’s grayscale coordinate value, (x N ,y N ) represents G N (x N ,y N )’s grayscale coordinate value.

7. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 6, characterized in that: The three-dimensional reconstruction model includes a feature extraction module, a feature aggregation module, a feature decomposition module and a feature regression module. The feature extraction module is used to extract the first local feature of the geometric information of a preset multi-angle reference image and the second local feature of the geometric information image of the multi-angle remote sensing image, and convert the second local feature into the viewing angle of the first local feature to obtain a third local feature, and match the first local feature and the third local feature in pairs to obtain multiple cost volumes.

8. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 7 is characterized in that: The feature aggregation module adopts a feature pyramid network, and aggregates the multiple cost bodies based on the feature pyramid network to obtain an aggregated cost body. The feature decomposition module is used to downsample the aggregated cost body to obtain high-dimensional features, and then perform capsule operation according to the feature channel dimension to decouple the aggregated cost body into multiple capsule blocks. The feature regression module is used to encode the high-dimensional features in the multiple capsule blocks through a preset capsule convolution layer to obtain multiple high-dimensional feature semantics, and splice the multiple high-dimensional feature semantics with the multiple capsule blocks to obtain multiple spliced ​​features, and regress the multiple spliced ​​features through a normalization operation to obtain multiple depth maps, and convert the multiple depth maps into three-dimensional point cloud data.

9. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 1, characterized in that: Establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image includes: Acquire three-dimensional point cloud data of the real-scene three-dimensional model, and extract feature points from the three-dimensional point cloud data to obtain a first key feature point set; Extracting feature points from the multi-angle remote sensing image to obtain a second key feature point set; The first key feature point set and the second key feature point set are matched based on a scale-invariant feature transformation method to obtain matching feature point pairs.

10. The method for displaying geographic entity information based on real-scene three-dimensional modeling according to claim 9, characterized in that: Establishing a position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image also includes: Calculating the pose parameters of the multi-angle remote sensing image relative to the real-scene three-dimensional model based on the matching feature point pairs based on visual positioning technology, wherein the pose parameters include a rotation matrix and a translation vector; A position mapping table between the real-scene three-dimensional model and the multi-angle remote sensing image is established based on the posture parameters, wherein the position mapping table includes a correspondence between image coordinates and three-dimensional coordinates.

Images