A method, device and computer-readable storage medium for generating an electronic map

Abstract

The present application discloses a method, device and computer-readable storage medium for generating an electronic map, the method comprising: obtaining an image data set, the image data set comprising a plurality of remote sensing stereo image pairs, the remote sensing stereo image pairs comprising remote sensing images; segmenting the remote sensing images to obtain a first segmentation prediction result; creating a first map based on the first segmentation prediction result; processing the remote sensing images to generate a second map, the dimension of the second map being greater than the dimension of the first map; generating an electronic map based on the first map and the second map. In the above manner, the present application can generate maps of multiple dimensions by processing remote sensing stereo image pairs.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for generating an electronic map, and a computer-readable storage medium. Background Art

[0002] With the widespread application of computers and other electronic devices in mapping, maps are no longer limited to expression on paper media, but can be displayed on the screen in digital form; satellite remote sensing images provide a new and diverse data source for map making, greatly enriching the production form and display effect of maps; currently, different thematic analysis and visualization requirements are proposed for maps, but the map products currently provided by map service providers are highly homogeneous and not very flexible, and cannot be customized according to research topics or project requirements. Even if customization is possible, expensive customization fees are required. Summary of the invention

[0003] The present application provides a method, device and computer-readable storage medium for generating an electronic map, which can generate maps of multiple dimensions by processing remote sensing stereo image pairs.

[0004] To solve the above technical problems, the technical solution adopted in the present application is: to provide a method for generating an electronic map, the method comprising: obtaining an image data set, the image data set comprising a plurality of remote sensing stereo image pairs, the remote sensing stereo image pairs comprising remote sensing images; segmenting the remote sensing images to obtain a first segmentation prediction result; creating a first map based on the first segmentation prediction result; processing the remote sensing images to generate a second map, the dimension of the second map being greater than the dimension of the first map; and generating an electronic map based on the first map and the second map.

[0005] To solve the above technical problems, another technical solution adopted in the present application is: to provide a map generating device, which includes a memory and a processor connected to each other, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, it is used to implement the method for generating an electronic map in the above technical solution.

[0006] To solve the above technical problem, another technical solution adopted by the present application is: providing a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, it is used to implement the method for generating an electronic map in the above technical solution.

[0007] Through the above scheme, the beneficial effects of the present application are: first, an image data set including multiple remote sensing stereo image pairs is obtained, and the remote sensing stereo image pair includes at least two remote sensing images; then the remote sensing image is segmented to generate a first segmentation prediction result, and the first segmentation prediction result is used to create a first map; at the same time, the remote sensing image is also processed to obtain a second map with a dimension different from that of the first map; by superimposing the first map and the second map to generate a final electronic map, it is possible to generate maps of multiple dimensions by processing the remote sensing stereo image pairs, which is convenient for users to understand more terrain information, and there is no need to use other auxiliary tools, so it is simple to implement and can meet the customization needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a flowchart of an embodiment of a method for generating an electronic map provided by the present application;

[0010] Figure 2 It is a flowchart of another embodiment of the method for generating an electronic map provided by the present application;

[0011] Figure 3 It is a structural schematic diagram of the segmentation model provided by this application;

[0012] Figure 4 It is a schematic diagram of the process of segmenting remote sensing images using a segmentation model provided in this application;

[0013] Figure 5 is a schematic diagram of the structure of the second extraction network provided by the present application;

[0014] Figure 6 is the remote sensing image provided by this application;

[0015] Figure 7 Yes Figure 6 The 2D map generated by processing the remote sensing image shown;

[0016] Figure 8 Yes Figure 6 A digital elevation model generated by processing the remote sensing image shown;

[0017] Fig. 9 Yes Figure 6 The remote sensing images shown are processed to generate 2D-2.5D-3D electronic maps;

[0018] Fig.10 is a structural schematic diagram of an embodiment of a map generating device provided by the present application;

[0019] Fig.11 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0020] The present application is further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present application, but are not intended to limit the scope of the present application. Similarly, the following examples are only some embodiments of the present application rather than all embodiments, and all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0021] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] It should be noted that the terms "first", "second" and "third" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second" and "third" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0023] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a method for generating an electronic map provided by the present application, the method comprising:

[0024] S11: Obtain an image dataset.

[0025] The image data set includes multiple remote sensing stereo image pairs, each remote sensing stereo image pair includes at least two remote sensing images; or, the image data set includes multi-view remote sensing images, that is, remote sensing images of different viewpoints generated by shooting in different shooting directions; the input remote sensing stereo image pairs / multi-view remote sensing images can be preprocessed, such as: geometric correction, registration or band fusion operations, to obtain multispectral remote sensing images with higher positioning accuracy.

[0026] In a specific embodiment, taking a remote sensing satellite as an example, the three-line array camera carried by the satellite can respectively obtain the front view image, the front view image and the rear view image of the ground, and provide front view full-color images with a resolution of 2.1m, front view images with a resolution of 22° with a resolution of 2.5m, rear view full-color images and front view multi-spectral images with a resolution of 5.8m (including four bands of near infrared, red, green and blue). After the images are acquired, they can be geometrically precisely corrected, registered and fused to generate multi-spectral images with a spatial resolution of 2.1m for subsequent module processing and analysis, such as using the pre-processed remote sensing images for remote sensing intelligent analysis, building outline and height extraction or three-dimensional reconstruction.

[0027] S12: Segment the remote sensing image to obtain a first segmentation prediction result.

[0028] After acquiring the remote sensing image, the remote sensing image semantic segmentation algorithm based on deep learning can be used to segment the remote sensing image to extract information such as buildings, roads, water bodies, vegetation or bare land in the remote sensing image, and generate a first segmentation prediction result. The first segmentation prediction result includes the confidence (i.e., probability value) that each pixel in the remote sensing image belongs to each category; for example, taking the segmentation of all pixels in the remote sensing image into three categories as an example, assuming that the categories are roads, water bodies and vegetation, the probabilities of the pixels belonging to the three categories are P1-P3, P1+P2+P3=1.

[0029] Understandably, a pre-trained segmentation model can be used to segment remote sensing images, such as a UNet network.

[0030] S13: Creating a first map based on the first segmentation prediction result.

[0031] After obtaining the first segmentation prediction result, the first segmentation prediction result can be vectorized to generate a vector map; then the vector map is anti-aliased to obtain the first map. Specifically, the dimension of the first map is 2D (Dimension, D), and the first segmentation prediction result can be regarded as a remote sensing image intelligent analysis result generated by intelligent analysis of the remote sensing image. The remote sensing image intelligent analysis result can be automatically vectorized to obtain surface elements such as roads, water bodies, vegetation or bare land, among which the road object needs to extract the boundary line, center line and intersection point in order to construct the vector map of the road; the vector map is anti-aliased to retain obvious inflection points and remove redundant vector points, and finally a 2D vector map (i.e., the first map) is generated.

[0032] Furthermore, the vectorization operation is to traverse each corner point of the remote sensing image in turn to extract the connected block contour, and then add the inner contour to the external connected block with the containment relationship. The anti-aliasing operation is: for any two points A and B on the vector contour, if there is a point C on the contour path of AB that satisfies "the distance from point C to line AB is less than the preset threshold", then point C is a redundant point and is removed.

[0033] S14: Process the remote sensing image to generate a second map.

[0034] The second map may be a single map or a combination of at least two maps. For example, an extraction network may be used to extract and process remote sensing images to obtain extraction results, which include the outline and height of buildings. The extraction results are then vectorized to obtain building models, and a second map is constructed based on the building model. The dimension of the second map is greater than that of the first map.

[0035] In another specific embodiment, a three-dimensional reconstruction algorithm in related technology may be used to reconstruct the remote sensing image to obtain a reconstruction result; then a digital elevation model (DEM) is constructed based on the reconstruction result to obtain a second map.

[0036] In other embodiments, the second map is composed of at least two maps. The remote sensing image can be extracted by using an extraction network to obtain an extraction result; the extraction result is then vectorized to obtain a building model, and a third map is constructed based on the building model, and the dimension of the third map can be 2.5D; at the same time, the remote sensing image is reconstructed to obtain a reconstruction result; then a DEM is constructed based on the reconstruction result to obtain a fourth map, and the dimension of the fourth map can be 3D; then the second map is generated based on the third map and the fourth map.

[0037] S15: Generate an electronic map based on the first map and the second map.

[0038] After the first map and the second map are acquired, the first map and the second map may be superimposed to obtain an electronic map for display on an electronic device.

[0039] This embodiment provides a method for producing an electronic map based on remote sensing images, which first pre-processes the acquired remote sensing images; then performs remote sensing intelligent analysis, building outline and height extraction, and three-dimensional reconstruction on the pre-processed remote sensing images; and finally outputs 2D, 2.5D, and 3D electronic maps by post-processing the remote sensing image intelligent analysis results (such as vectorization), constructing building models, and constructing DEM.

[0040] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of a method for generating an electronic map provided by the present application, the method comprising:

[0041] S21: Obtain an image dataset.

[0042] S21 is the same as S11 in the above embodiment and will not be described again.

[0043] S22: Use the segmentation model to perform segmentation processing on the remote sensing image to obtain a first segmentation prediction result.

[0044] The segmentation model can use the Cascade encoder-decoder framework, such as Figure 3 As shown, the segmentation model includes an Encoder module, a Neck module, a Decoder module and a Refine module; specifically, Figure 4 As shown, the following steps are used to segment the remote sensing image:

[0045] S41: Input the remote sensing image into the segmentation model, and use the Encoder module in the segmentation model to encode the remote sensing image to generate a first feature.

[0046] The Encoder module can use Resnet101 (Deep residual network), such as Figure 3As shown in the figure, the Encoder module includes 4 stages. For a remote sensing image with an input size of H*W, the remote sensing image is first processed into a 64-channel feature layer with a size of 1 / 2H*1 / 2W using an extraction layer. The extraction layer includes a convolution layer with a convolution kernel of 7*7, an activation layer, and a maximum pooling (MaxPool) layer. The activation layer can use a linear rectified unit (ReLU); then feature extraction is performed through Stage1, Stage2, Stage3, and Stage4 respectively.

[0047] Furthermore, Stage 1 consists of 3 bottleneck layers, and the output is a feature layer with 256 channels and a size of 1 / 4H*1 / 4W; Stage 2 consists of 4 bottleneck layers, and the output is a feature layer with 512 channels and a size of 1 / 8H*1 / 8W; Stage 3 consists of 23 bottleneck layers, and the dilation factor of 2 is used to replace the downsampling operation, and the output is a feature layer with 1024 channels and a size of 1 / 8H*1 / 8W; Stage 4 consists of 3 bottleneck layers, and the dilation factor of 4 is used to replace the downsampling operation, and the output is a feature layer with 2048 channels and a size of 1 / 8H*1 / 8W.

[0048] S42: Use the Neck module in the segmentation model to perform feature fusion processing on the first feature to obtain the second feature.

[0049] The Neck module uses the Feature Pyramid Networks (FPN) structure for feature fusion, such as Figure 3 As shown in the figure, the Neck module consists of 4 stages, and the input of each stage is the output of the previous stage and the output of the corresponding stage in the Encoder module. Specifically, first, the output of the previous stage and the output of the corresponding stage in the Encoder module are processed using a convolution layer with a convolution kernel of 3*3 to extract the feature layer as a 256-channel feature layer; secondly, after the output of the previous stage is upsampled to the same size as the feature layer output by the corresponding stage in the Encoder module, it is concatenated with the feature layer output by the corresponding stage in the converted Encoder module, and finally the fused feature layer is obtained through a convolution layer with a convolution kernel of 3*3.

[0050] Furthermore, the feature layer of Stage4 in the Neck module is the result of the feature layer output by Stage4 in the Encoder module after the convolution layer; the feature layer of Stage3 in the Neck module is the result of the fusion of the feature layer output by Stage4 in the Neck module and the feature layer output by Stage3 in the Encoder module; the feature layer of Stage2 in the Neck module is the result of the fusion of the feature layer output by Stage3 in the Neck module and the feature layer output by Stage2 in the Encoder module; the feature layer of Stage1 in the Neck module is the result of the fusion of the feature layer output by Stage2 in the Neck module and the feature layer output by Stage1 in the Encoder module.

[0051] S43: Using the Decoder module in the segmentation model to decode the second feature to obtain a second segmentation prediction result.

[0052] The Decoder module uses the FPN structure to predict the preliminary segmentation results. This structure is similar to the Neck module. First, the feature layer output by each Stage is extracted as a feature layer with 128 channels. Then all feature layers are bilinearly interpolated to a size of 1 / 8H*1 / 8W and added together. Finally, a convolution layer with a convolution kernel of 1*1 and the number of channels equal to the number of segmentation categories is used for coarse-grained segmentation prediction to generate the second segmentation prediction result.

[0053] S44: using the Refine module in the segmentation model to revise the second segmentation prediction result to obtain the first segmentation prediction result.

[0054] The Refine module uses an iterative upsampling method to gradually optimize the segmentation result. The input of this module is the output of the Decoder module, that is, the second segmentation prediction result of size 1 / 8H*1 / 8W, and the output is the first segmentation prediction result of H*W.

[0055] In a specific embodiment, the second segmentation prediction result is recorded as the current segmentation prediction result. When the size of the current segmentation prediction result is smaller than the size of the original image (i.e., the input remote sensing image), iterative optimization is performed; specifically, it is determined whether the size of the current segmentation prediction result is the same as the size of the remote sensing image; if the size of the current segmentation prediction result is different from the size of the remote sensing image, the current segmentation prediction result is interpolated to obtain a third segmentation prediction result; a preset number of first candidate points are selected from the second segmentation prediction result; based on the first candidate points, the second segmentation prediction result is corrected to obtain the first segmentation prediction result. For example, the preset number is recorded as N. In each optimization, the current segmentation prediction result is upsampled by 2 times bilinear interpolation, so that the size of the interpolated current segmentation prediction result (i.e., the third segmentation prediction result) is doubled; then N first candidate points are selected from the second segmentation prediction result, and the point selection strategy is to randomly select N first candidate points from the second segmentation prediction result in a uniformly distributed manner.

[0056] Furthermore, the second segmentation prediction result includes the first sub-segmentation result, and the third segmentation prediction result includes the second sub-segmentation result. The second candidate point is first screened out from a preset number of first candidate points; specifically, βN (β∈[0,1]) most uncertain points are selected from the N first candidate points, and the uncertainty measurement standard is the distance between the preset threshold and the predicted probability in the first sub-segmentation result. The smaller the distance, the greater the uncertainty, and the point corresponding to the distance is more likely to be the second candidate point; for example, the preset threshold is 0.5, and it is determined whether the difference between the predicted probability corresponding to the first candidate point and 0.5 is less than the preset difference. If the difference is less than the preset difference, it indicates that it may be an uncertain point, and βN points with the smallest difference can be selected from all first candidate points whose differences with 0.5 are less than the preset difference as the second candidate points.

[0057] Then, a characterization vector of the second candidate point is extracted, and the characterization vector includes two parts: ① the third segmentation prediction result, and ② the feature layer output by Stage 1 in the Neck module; specifically, the second sub-segmentation result corresponding to the second candidate point in the third segmentation prediction result is obtained to obtain the first feature to be spliced; the feature corresponding to the second candidate point in the second feature is obtained to obtain the second feature to be spliced; the first feature to be spliced ​​and the second feature to be spliced ​​are spliced ​​to generate a spliced ​​segmentation result; a multi-layer perceptron (MLP) is used to predict the spliced ​​segmentation result to obtain a revised segmentation result; the first sub-segmentation result corresponding to the second candidate point in the second segmentation prediction result is replaced by the revised segmentation result to obtain the first segmentation prediction result, thereby updating the first segmentation prediction result.

[0058] S23: Creating a first map based on the first segmentation prediction result.

[0059] S23 is the same as S13 in the above embodiment and will not be described again.

[0060] S24: Use the extraction network to extract the remote sensing image to obtain the extraction result.

[0061] In order to extract the outline and height of the building, the remote sensing image can be processed by an extraction network to obtain an extraction result, which includes the outline and height of the building. Specifically, the building outline and height extraction can use a building roof segmentation and height prediction algorithm based on deep learning to segment the roof of the building and predict the height value and shooting direction of each pixel. The specific algorithm used can be the same as the related technology and will not be repeated here.

[0062] It can be understood that this embodiment can use one network to obtain the extraction results, or it can also use two networks to obtain the extraction results, that is, independently constructing an extraction network for buildings and a prediction network for the height and shooting direction of buildings. The following uses the use of two networks to generate extraction results as an example for explanation.

[0063] The extraction network includes a first extraction network and a second extraction network. The first extraction network is used to perform feature extraction processing on the remote sensing image to obtain the buildings in the remote sensing image. For example, a remote sensing semantic segmentation algorithm is used to extract buildings, and the output is changed to two channels, one channel outputs the information of the building, and the other channel outputs the information of the background. Finally, a mask map of the building can be generated. The value corresponding to the position where the building exists in the mask map is 1, and the value corresponding to the position where the building does not exist in the mask map is 0.

[0064] Furthermore, since semantic segmentation cannot distinguish individuals, when post-processing the building mask map, the mask map is first opened to disconnect adjacent buildings; then the connected domain analysis is used to extract the instance information of each building; different identifications (IDs) can be used to identify different buildings, and the instance information of the building is filled into the mask map, thereby obtaining the information of all buildings in the remote sensing image.

[0065] The second extraction network is used to perform feature extraction processing on the remote sensing image to obtain attribute information of the building in the remote sensing image, and the attribute information includes the height of the building and the shooting direction of the building.

[0066] In a specific embodiment, Figure 5As shown, the second extraction network includes a backbone network, a first branch network and a second branch network. The backbone network is used to process the remote sensing image to generate the first extraction feature; the first branch network is used to regress the first extraction feature to obtain the height of the building; the second branch network is used to regress the first extraction feature to obtain the shooting direction of the building.

[0067] Furthermore, the second extraction network adopts Resnet34 as the main feature extraction network, and combines the FPN network to construct a lightweight height prediction network. The output of the second extraction network includes two parts, one part is the shooting direction of the whole picture, and the other part is the height information of each pixel, and the height is in pixels.

[0068] like Figure 5 As shown in the figure, the feature extraction network includes four stages. For remote sensing images with a resolution of 1024×1024, a 7×7 convolution kernel is first used for convolution. After convolution, the resolution of the data is reduced to 512×512, and the number of channels is increased to 64; then the maximum pooling layer is used to reduce the resolution of the data to 256×256; after processing by the first Stage (i.e., Stage 1), the resolution of the data is reduced to 128×128, and the number of channels is increased to 128. Stage 1 consists of 3 blocks, each block consists of two 3×3 convolutional layers; after processing by the second Stage (consisting of 4 blocks), the resolution is reduced to 64×64, and the number of channels is increased to 256; after processing by the third Stage (consisting of 6 blocks), the resolution of the data is reduced to 32×32, and the number of channels is increased to 512; after processing by the fourth Stage (consisting of 3 blocks), the resolution and number of channels of the data remain unchanged.

[0069] Moreover, the attention mechanism in the Dual Attention Network (DANet) is introduced, which consists of two parts: the spatial attention module and the channel attention module. The data output by the spatial attention module and the channel attention module are fused and input into the two branches (i.e., the first branch network and the second branch network). The first branch network simulates FPN for upsampling. After the first upsampling, the data resolution is restored to 64×64, and the number of channels is reduced to 256. It is merged with the output of Stage4 in the backbone network and then subjected to two 3×3 convolutions. After the second upsampling, the data resolution is restored to 128×128, and the number of channels is reduced to 128. It is merged with the output of Stage3 in the backbone network and then subjected to two 3×3 convolutions. After the third upsampling, the data resolution is restored to 256×256, and the number of channels is reduced to 64. It is merged with the output of Stage2 in the backbone network and then subjected to two 3×3 convolutions. After the fourth upsampling, the data resolution is restored to 512×512, and the number of channels is reduced to 32. It is merged with the output of Stage1 in the backbone network and then subjected to two 3×3 convolutions. Finally, the upsampling layer and two 3×3 convolution layers are used to restore the data resolution to 1024×1024, with a channel number of 1, and output a height map, which includes the height of each pixel in the remote sensing image. The second branch network uses an adaptive pooling layer and a fully connected layer to output a vector data representing the direction (i.e., the shooting direction) and is supervised by a loss function during training.

[0070] S25: Vectorizing the extraction results to obtain a building model, and constructing a third map based on the building model.

[0071] Obtain the first segmentation prediction result output by S22, the first segmentation prediction result including the segmentation prediction result of the top surface of the building, translate the top surface of the building to the position of the bottom surface of the building according to the height of the building and the shooting direction of the building, and obtain the outline of the bottom surface of the building, the top surface of the building after translation can be equivalent to the base of the building; vectorize the outline of the bottom of the building, and calculate the average height of each building; construct a building model based on the average height and the bottom surface of the building, specifically, assign the calculated height to the outline of the bottom of the building as its height attribute, and the effect of the building model can be displayed through visualization.

[0072] It can be understood that the method of generating a map using a building model is the same as in the related art and will not be described in detail here.

[0073] S26: Reconstruct the remote sensing image to obtain a reconstruction result, and construct a DEM based on the reconstruction result to obtain a fourth map.

[0074] In order to achieve 3D reconstruction, binocular stereo matching method can be used to process multiple independent remote sensing stereo image pairs, and then the reconstruction results are fused. Specifically, the process of 3D reconstruction of remote sensing images includes: image cropping → rational polynomial coefficients (RPC) update → calculation of corresponding points and basic matrix → epipolar correction → binocular stereo matching → triangulation to obtain point cloud → point cloud denoising, alignment and fusion → digital surface model (DSM) → removal of buildings and vegetation → DEM, which is described in detail below.

[0075] 1) Image cropping and RPC parameter update

[0076] Since satellite imaging belongs to push-broom imaging, which is different from the pinhole imaging model, epipolar correction cannot be performed through simple affine transformation. Therefore, the remote sensing image is cropped and divided into small image blocks (for example: 2048*2048), and the small image blocks are imaged using affine camera pinhole imaging to approximate push-broom imaging. This process requires input of remote sensing stereo image pairs / multi-view remote sensing images, RPC parameters corresponding to the images, and coarse-resolution elevation data, such as DEM data produced by the Shuttle Radar Topography Mission (SRTM); RPC parameters are an approximation of the remote sensing satellite imaging model, which associates the geodetic coordinates (latitude, longitude and altitude) of the ground point with the image coordinates of its object using a polynomial.

[0077] In a specific embodiment, the remote sensing image in the remote sensing stereo image pair includes a left-view remote sensing image and a right-view remote sensing image, the left-view remote sensing image is cropped to obtain a plurality of left-view image blocks; the coordinates of the corner points of the left-view image block are calculated based on the RPC parameters of the left-view remote sensing image to obtain the first corner point coordinates, which may include the pixel points at the upper left corner and the pixel points at the lower right corner of the image block; the first corner point coordinates are subjected to coordinate conversion processing to obtain the second corner point coordinates (i.e., geodetic coordinates); since the geodetic coordinates corresponding to the same object in the left-view remote sensing image and the right-view remote sensing image are the same, the corner point coordinates corresponding to the left-view image block in the right-view remote sensing image can be calculated based on the RPC parameters of the right view and the second corner point coordinates to obtain the third corner point coordinates; the image block corresponding to the third corner point coordinates is cut out from the right-view remote sensing image to obtain the right-view image block matching the left-view image block; the corresponding RPC parameters are updated based on the sizes of the left-view image block and the right-view image block to obtain the updated RPC parameters.

[0078] 2) Calculate corresponding points and basic matrix

[0079] Since remote sensing images cannot be calibrated like binocular images acquired by ordinary cameras, it is necessary to obtain matching points of remote sensing images. RPC parameters can be used to obtain virtual corresponding points between two remote sensing images. For example, according to the longitude, latitude and elevation range of the left and right views of the remote sensing image, random sampling is performed within the range. According to the longitude, latitude and altitude, the RPC parameters are used to obtain the pixel coordinates of the two corresponding remote sensing images. These points are used as virtual corresponding points to calculate the basic matrix (including intrinsic parameters and rotation and translation information) between the left view image block and the right view image block, and the basic matrix is ​​used to perform corresponding epipolar correction on the left view image block and the right view image block.

[0080] 3) Stereo matching

[0081] The semi-global block matching (SGBM) stereo matching algorithm is used to calculate the disparity of two rectified image blocks (including the left view image block and the right view image block). The SRTM data can be used to estimate the initial disparity range; the disparity is converted into the correspondence relationship of the original image; based on the correspondence relationship, the 3D position of the point is triangulated using the improved RPC parameters to obtain the elevation.

[0082] 4) Triangulation

[0083] For triangulation, two images u and v and their projection functions RPC and inverse RPC -1 The epipolar curve is defined. Assuming x is a point in image u, its epipolar equation in image v can be defined as:

[0084]

[0085] Among them, epi is the epipolar plane image (epipolar plane image), and h is the height.

[0086] This curve defines the possible correspondence of x at different heights h. When the matching point pair (x, x') is known, that is, the projections of the same point in the geographic space in images u and v are known, the epipolar curve of x must pass through x', and x'=epi u x v The height h of (h) is the elevation of the point; therefore, the elevation data of a point in geographic space can be measured through the improved RPC parameters.

[0087] 5) Generate DSM and DEM

[0088] The image coordinates of the image block are transformed into Universal Transverse Mercator Grid System (UTM) coordinates, and the corresponding elevation data are obtained to form three-dimensional point cloud data; the point cloud data are resampled according to the actual resolution of the remote sensing image, and the point cloud data corresponding to the cropped image block are spliced ​​to generate the DSM corresponding to the remote sensing image.

[0089] Furthermore, in order to generate DEM data using DSM data, point cloud data of the exposed surface can be obtained, while point cloud data of other non-bare land areas can be removed, and the missing surface areas can be completed by interpolation methods. Specifically, a mask is constructed based on the contours of buildings, roads or water bodies extracted by the remote sensing image semantic segmentation algorithm, and the point cloud data of the corresponding area is removed using the mask. The lowest point is obtained in the remaining data by filtering, and the data of the lowest point is used to complete the hole to generate DEM data; then the DEM data is converted into a corresponding map (i.e., a fourth map) using a solution in the related art. The dimension of the fourth map is greater than the dimension of the third map, and the dimension of the fourth map can be 3D.

[0090] S27: Superimpose the third map and the fourth map to obtain a second map.

[0091] After obtaining the 2.5D third map and the 3D fourth map, the third map and the fourth map may be superimposed to generate a second map.

[0092] S28: Overlaying the first map and the second map to generate an electronic map.

[0093] Based on the acquired 2D vector map, the outline and height of the building, and the DEM of the area, 2D, 2.5D, and 3D electronic maps can be output. Figure 6-Figure 9 As shown, Figure 6 For remote sensing images, Figure 7 For 2D maps, Figure 8 For 2.5D maps, Fig. 9 It is a 2D-2.5D-3D electronic map, which includes 2D map, 2.5 map and 3D map.

[0094] This embodiment extracts information on surface objects such as roads, water bodies, vegetation or bare land in remote sensing images, extracts the outlines and heights of buildings in remote sensing images, constructs DSM and DEM, and finally generates 2D, 2.5D, and 3D electronic maps through surface object segmentation and three-dimensional reconstruction methods of buildings and terrain. The height of buildings can be extracted by only remote sensing stereo image pairs without the need for other external information or modeling software. The implementation is simple and can meet the needs of different users.

[0095] See also Fig.10 , Fig.10 It is a structural diagram of an embodiment of a map generating device provided in the present application. The map generating device 100 includes a memory 101 and a processor 102 connected to each other. The memory 101 is used to store a computer program. When the computer program is executed by the processor 102, it is used to implement the method for generating an electronic map in the above embodiment.

[0096] This embodiment uses remote sensing image intelligent analysis and three-dimensional reconstruction technology to obtain surface objects in wide areas or city-level areas (such as extraction results of elements such as buildings, roads or water bodies), and combines the three-dimensional reconstruction of buildings and terrain to automatically and efficiently obtain 2D, 2.5D and 3D electronic maps of wide areas. Electronic maps can be output using only remote sensing stereo image pairs as input, without the need for other auxiliary tools and additional information, and are simple to implement.

[0097] See also Fig.11 , Fig.11 It is a structural diagram of an embodiment of a computer-readable storage medium provided in the present application. The computer-readable storage medium 110 is used to store a computer program 111. When the computer program 111 is executed by a processor, it is used to implement the method for generating an electronic map in the above embodiment.

[0098] The computer-readable storage medium 110 may be a server, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program codes.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0101] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0102] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for generating an electronic map, characterized in that: include: Acquire an image data set, wherein the image data set includes a plurality of remote sensing stereo image pairs, and the remote sensing stereo image pairs include remote sensing images; Segmenting the remote sensing image to obtain a first segmentation prediction result; Creating a first map based on the first segmentation prediction result, where the dimension of the first map is 2 dimensions; Processing the remote sensing image to generate a second map, wherein the dimension of the second map is greater than the dimension of the first map, and the dimension is Dimension; Overlaying the first map with the second map to generate an electronic map; The step of processing the remote sensing image to generate a second map includes: Using an extraction network to extract the remote sensing image to obtain an extraction result, wherein the extraction result includes the outline and height of the building; performing vectorization processing on the extraction result to obtain a building model, and constructing the second map based on the building model; or Reconstructing the remote sensing image to obtain a reconstruction result; constructing a digital elevation model based on the reconstruction result to obtain a second map; or, The remote sensing image is extracted by using an extraction network to obtain an extraction result, wherein the extraction result includes the outline and height of the building; the extraction result is vectorized to obtain a building model, and a third map is constructed based on the building model; the remote sensing image is reconstructed to obtain a reconstruction result; a digital elevation model is constructed based on the reconstruction result to obtain a fourth map, wherein the dimension of the fourth map is greater than that of the third map; the third map is superimposed on the fourth map to obtain the second map.

2. The method for generating an electronic map according to claim 1, characterized in that: The step of segmenting the remote sensing image to obtain a first segmentation prediction result includes: Inputting the remote sensing image into a segmentation model; Using the encoding module in the segmentation model to encode the remote sensing image to generate a first feature; Using the neck module in the segmentation model to perform feature fusion processing on the first feature to obtain a second feature; Using a decoding module in the segmentation model to decode the second feature to obtain a second segmentation prediction result; The second segmentation prediction result is corrected by using the improved module in the segmentation model to obtain the first segmentation prediction result.

3. The method for generating an electronic map according to claim 2, characterized in that: The step of using the improved module in the segmentation model to correct the second segmentation prediction result to obtain the first segmentation prediction result includes: Recording the second segmentation prediction result as the current segmentation prediction result; Determining whether the size of the current segmentation prediction result is the same as the size of the remote sensing image; If not, interpolating the current segmentation prediction result to obtain a third segmentation prediction result; Filtering a preset number of first candidate points from the second segmentation prediction results; Based on the first candidate point, the second segmentation prediction result is corrected to obtain the first segmentation prediction result.

4. The method for generating an electronic map according to claim 3, characterized in that: The second segmentation prediction result includes the first sub-segmentation result, the third segmentation prediction result includes the second sub-segmentation result, and the step of correcting the second segmentation prediction result based on the first candidate point to obtain the first segmentation prediction result includes: Selecting a second candidate point from the preset number of first candidate points; Obtaining a second sub-segmentation result corresponding to the second candidate point in the third segmentation prediction result to obtain a first feature to be spliced; Acquire a feature corresponding to the second candidate point in the second feature to obtain a second feature to be spliced; Splicing the first feature to be spliced ​​and the second feature to be spliced ​​to generate a splicing segmentation result; A multi-layer perceptron is used to predict the splicing segmentation result to obtain a modified segmentation result; The first segmentation prediction result is obtained by replacing a first sub-segmentation result corresponding to the second candidate point in the second segmentation prediction result with the modified segmentation result.

5. The method for generating an electronic map according to claim 1, characterized in that: The step of generating an electronic map based on the first map and the second map includes: The first map and the second map are superimposed to generate the electronic map.

6. The method for generating an electronic map according to claim 1, characterized in that: The extraction network includes a first extraction network and a second extraction network. The step of using the extraction network to extract the remote sensing image to obtain an extraction result includes: Using the first extraction network to perform feature extraction processing on the remote sensing image to obtain buildings in the remote sensing image; The second extraction network is used to perform feature extraction processing on the remote sensing image to obtain attribute information of the building in the remote sensing image, wherein the attribute information includes the height of the building and the shooting direction of the building.

7. The method for generating an electronic map according to claim 6, characterized in that: The second extraction network includes a trunk network, a first branch network and a second branch network, and the method further includes: Processing the remote sensing image using the backbone network to generate a first extracted feature; Using the first branch network to perform regression processing on the first extracted feature to obtain the height of the building; The second branch network is used to perform regression processing on the first extracted features to obtain the shooting direction of the building.

8. The method for generating an electronic map according to claim 6, characterized in that: The first segmentation prediction result includes a segmentation prediction result of a top surface of a building, and the step of vectorizing the extraction result to obtain a building model includes: The top surface of the building is translated to the position where the bottom surface of the building is located according to the height of the building and the shooting direction of the building, so as to obtain the outline of the bottom surface of the building; Vectorizing the outline of the base of the buildings and calculating the average height of each of the buildings; The building model is constructed based on the average height and the bottom surface of the building.

9. The method for generating an electronic map according to claim 4, characterized in that: The step of creating a first map based on the first segmentation prediction result comprises: Performing vectorization processing on the first segmentation prediction result to generate a vector graph; Anti-aliasing is performed on the vector image to obtain the first map.

10. A map generating device, characterized in that: The invention comprises a memory and a processor connected to each other, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, it is used to implement the method for generating an electronic map according to any one of claims 1 to 9.

11. A computer-readable storage medium for storing a computer program, characterized in that: When the computer program is executed by a processor, it is used to implement the method for generating an electronic map according to any one of claims 1 to 9.

Images