Map generation method, model training method, readable medium, and electronic device

Abstract

The application discloses a map generation method, a model training method, a readable medium and an electronic device. In the process of generating a vector map by using a neural network model, the contour mask of a map element is converted into a vector map by learning the geometric features of the map element in a sample image of a target region, instead of being realized by setting a vectorization rule by a technician, so that the accuracy of the obtained vector map can be improved. In addition, by the method provided in the application, for different target regions, after retraining a neural network model by using sample images of different target regions, the vector map of the different regions can be obtained by using the retrained neural network, without the need of complex parameter adjustment and vectorization rule setting, so that the efficiency of generating the vector map is improved while the accuracy is ensured, and the method is more suitable for the scene of generating a large-scale map.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a map generation method, a model training method, a readable medium, and an electronic device. Background Technology

[0002] With the development of artificial intelligence (AI) technology, the application of neural network models is becoming increasingly widespread. For example, neural network models can be used to obtain vector maps of a region based on remote sensing images of that region. Currently, the process typically involves first using a neural network model to infer the initial outlines of map elements (such as houses, lakes, roads, and rivers) in the remote sensing image. Then, using vectorization rules set by developers, the initial outlines of the map elements are adjusted, such as adjusting the angles between lines, to convert the outlines of the map elements into a vector map.

[0003] However, due to the diversity of geographical environments and map elements—for example, different regions have different geographical environments and significant differences in the geometric features of buildings / roads—the vectorization rules set by developers are difficult to match the geographical environments and map elements of different regions. Vectorizing images of different regions based on the same vectorization rules results in vector maps with low accuracy. Furthermore, setting different contour acquisition methods and vectorization rules for different regions is complex and unsuitable for large-scale vector map modeling scenarios. Summary of the Invention

[0004] In view of this, embodiments of this application provide a map generation method, a model training method, a readable medium, and an electronic device. By learning the geometric features of map elements in a certain area through a neural network model, the outlines of the map elements are converted into corresponding vector maps, which helps to improve the accuracy of the obtained vector maps and is more suitable for scenarios involving large-scale vector map modeling.

[0005] In a first aspect, embodiments of this application provide a map generation method applied to an electronic device. The method includes: acquiring an image of a certain area, the image including map elements, wherein the map elements are elements in the image to be converted into a vector map; inferring from the image using a first model to obtain a first geometric figure corresponding to the map elements, the first geometric figure including geometric primitives; inputting the first geometric figure into a second model to obtain the orientation of each geometric primitive, and obtaining a second geometric figure corresponding to the map elements based on the first geometric figure, the second geometric figure including the same geometric primitives as the first geometric figure, and the arrangement of the geometric primitives in the second geometric figure being different from the arrangement of the geometric primitives in the first geometric figure; using a third model to obtain the topological relationship between each geometric primitive based on the orientation of the geometric primitives and the second geometric figure; and obtaining a vector map corresponding to the image based on the topological relationship between each geometric primitive, the orientation of each geometric primitive, and the second geometric figure.

[0006] In this embodiment, the electronic device first uses a first model (e.g., the shape initialization network hereinafter referred to as the shape initialization network) to infer the contours of map elements in an image of a certain region, obtaining a first geometric shape (e.g., the initial shape hereinafter referred to as the initial shape) corresponding to the map elements. Then, a second model (e.g., the shape regression network hereinafter referred to as the shape initialization network) is used to adjust the first geometric shape, obtaining a second geometric shape with higher accuracy and more regular shape (e.g., the regression shape hereinafter referred to as the regression shape). Then, a third model (e.g., the topology reconstruction network hereinafter referred to as the topology reconstruction network) is used to infer the topological relationships between geometric primitives in the second geometric shape (e.g., in the case that the second geometric shape is a polyline, inferring the topological relationships between the points that make up the polyline). Finally, based on the topological relationships between geometric primitives, a vector map corresponding to the image is obtained. In this way, the electronic device converts map elements into a vector map based on pre-trained first, second, and third models, rather than obtaining a vector map based on vectorization rules set by technicians, which helps to improve the accuracy of the obtained vector map. Furthermore, in the scenario of large-scale vector map modeling, retraining at least one of the first, second, and third models for different regions can effectively adapt to the geometric features of map elements in different regions without the need for setting vectorization rules and complex parameter adjustments, which is beneficial to improving the efficiency of vector map modeling.

[0007] It can be understood that geometric primitives are the basic building blocks of geometric figures. For example, when the first geometric figure is a polygon, the geometric primitives can be the line segments that make up the polygon; when the first geometric figure is a broken line, the geometric primitives can be the points of each line segment in the broken line.

[0008] In one possible implementation of the first aspect above, at least one of the first model, the second model, and the third model is trained based on the geometric features of map elements in a certain region.

[0009] In this embodiment, at least one of the first model, the second model, and the third model can be trained based on the geometric features of map elements in a certain region. In other words, the electronic device uses the geometric features of map elements in that region to vectorize the map elements, which helps to improve the accuracy of the obtained vector map.

[0010] In one possible implementation of the first aspect above, when the geometric primitives are line segments, the second geometric figure also includes the connection order of each geometric primitive; and, based on the topological relationship between each geometric primitive, the orientation of each geometric primitive, and the second geometric figure, a vector map corresponding to the image is obtained, including: adjusting the orientation of the first geometric primitive in the second geometric figure to be the same as the orientation corresponding to the first geometric primitive, wherein the orientation of the first geometric primitive in the second geometric figure is different from the orientation corresponding to the first geometric primitive; connecting the first geometric primitive and the second geometric primitive to obtain a polygon corresponding to the second geometric figure, wherein the connection order of the second geometric primitive is adjacent to the first geometric primitive.

[0011] In one possible implementation of the first aspect above, the polygon corresponding to the second geometric figure includes a first line segment, a second line segment, and a third line segment connected in sequence; and, based on the topological relationship between each geometric primitive, the orientation of each geometric primitive, and the second geometric figure, a vector map corresponding to the image is obtained, further comprising: deleting the second line segment if the length of the second line segment is less than a preset side length threshold; merging the first line segment and the second line segment into one line segment if the topological relationship between the first line segment and the third line segment is collinear or parallel; and extending the first line segment and / or the third line segment so that the first line segment and the third line segment intersect if the topological relationship between the first line segment and the third line segment is not collinear or parallel.

[0012] In one possible implementation of the first aspect above, when the geometric primitives are points, obtaining the vector map corresponding to the image based on the topological relationship between each geometric primitive, the orientation of each geometric primitive, and the second geometric figure includes: connecting the points whose topological relationship is connected to obtain the corresponding vectorized polyline.

[0013] In one possible implementation of the first aspect above, the above-mentioned reasoning on the image using the first model to obtain the first geometric figure corresponding to the map element includes: performing semantic segmentation on the image to obtain the contour mask of the map element, the contour mask being used to indicate the region where the map element is located in the image; extracting the mask edge of the contour mask; and simplifying the mask edge to obtain the first geometric figure.

[0014] For example, an electronic device can use the semantic segmentation network described below to obtain the contour mask of the region where map elements are located in the image, and use the edge extraction network described below to extract the edges of the contour mask to obtain the mask edges. Then, the DP algorithm is used to simplify multiple edges or the NMS algorithm is used to simplify polylines to obtain a first geometric figure with fewer geometric primitives. This can reduce the number of geometric primitives in the first geometric figure and help improve the speed of reasoning based on the first geometric figure of the electronic device.

[0015] In one possible implementation of the first aspect above, the map elements include at least one of houses, roads, lakes, oceans, rivers, forests, and deserts; and the first geometric figure corresponding to the houses, lakes, oceans, forests, and deserts is a polygon; the first geometric figure corresponding to the roads and rivers is a polyline.

[0016] In the embodiments of this application, an image may include one map element or multiple map elements. The electronic device can represent map elements that need to be represented by specific shapes, such as houses, lakes, oceans, forests, and deserts, as polygons, and roads, rivers, etc., as polylines.

[0017] In one possible implementation of the first aspect above, the method further includes training the first model in the following manner:

[0018] Acquire sample data, which includes a set of sample images of a certain region and reference contours corresponding to map elements in each sample image. Utilize a first model to analyze the image features of each sample image and obtain contour masks for map elements in each sample image based on these features. The contour masks indicate the regions of map elements in the corresponding sample images. Based on the contour masks, obtain the first predicted geometry corresponding to the map elements in each sample image. Train the first model based on the values ​​of a first loss function and a second loss function, where the first loss function indicates the accuracy of the contour mask, and the second loss function indicates the similarity between the first predicted geometry and the reference contours.

[0019] In other words, in this embodiment of the application, the first model is trained based on a set of sample images of a certain region. As a result, the contour mask and the first predicted geometry extracted by the first model from the map elements in the sample images have a high similarity to the reference contour corresponding to the map elements. The model has learned the geometric features of the map elements in the sample images. As a result, the first model infers that the first geometric shape of the map elements in the images of a certain region is more consistent with the geometric features of the map elements in that region, which is beneficial to improving the accuracy of the first geometric shape and thus improving the accuracy of the obtained vector map.

[0020] For example, in some embodiments, the first loss function may be the cross-entropy loss L described below.11-12-CEL The second loss function can be the L2 loss described below. 13-L2 .

[0021] In one possible implementation of the first aspect above, the method further includes: training a second model by: acquiring sample data, the sample data including reference contours corresponding to map elements in each sample image of a sample image set of a certain region, reference directions corresponding to each geometric primitive in the reference contours, and third geometric figures corresponding to map elements in each sample image obtained using the first model; using the second model to obtain second predicted geometric figures corresponding to each map element in each sample image, and predicted directions of geometric primitives in the third geometric figures, wherein the second predicted geometric figures include the same geometric primitives as the third geometric figures, and the arrangement of geometric primitives in the second predicted geometric figures is different from that in the third geometric figures; training the second model based on a third loss function and a fourth loss function, wherein the third loss function is used to indicate the similarity between the predicted directions of geometric primitives in the third geometric figures and the corresponding reference directions, and the fourth loss function is used to indicate the similarity between the second predicted geometric figures and the corresponding reference contours.

[0022] For example, in some embodiments, the third loss function may be the L2 loss L mentioned below. 23-L2 The fourth loss function can be the relative shape loss described below.

[0023] In one possible implementation of the first aspect described above, the method further includes: acquiring sample data, which includes a set of sample images of a certain region, reference topological relationships between geometric primitives in the reference contours corresponding to map elements of each sample image, and fourth geometric figures corresponding to map elements in each sample image obtained using the first model, and the orientation of geometric primitives in the fourth geometric figures; using the third model to determine the latent space features of each geometric primitive in the fourth geometric figure, and based on the latent space features, determining the predicted topological relationships between geometric primitives in the fourth geometric figure; training the third model based on the fifth loss function and the sixth loss function, wherein the fifth loss function is used to indicate the matching degree between the predicted topological relationships between geometric primitives in the fourth geometric figure and the corresponding reference topological relationships, and the sixth loss function is used to indicate the similarity of latent space features between geometric primitives whose predicted topological relationships are parallel, collinear, or connected.

[0024] In the embodiments of this application, when training the third model, the sixth loss function (e.g., the supervised contrastive loss hereinafter) indicates the similarity of latent space features between geometric primitives whose predicted topological relationships are parallel, collinear, or connected. This allows for a higher similarity of latent space features between geometric primitives with parallel, collinear, or connected relationships when using the third model to infer the second geometric figure. This improves the accuracy of the topological relationships between geometric primitives in the obtained second geometric figure, and consequently, improves the accuracy of the vector map obtained based on these topological relationships. For example, in some embodiments, the fifth loss function may be the cross-entropy loss L hereinafter. CEL The sixth loss function can be the supervised contrastive loss L mentioned below. SCL .

[0025] Secondly, embodiments of this application provide a model training method applied to electronic devices, the method comprising:

[0026] Acquire sample data, which includes the reference contours corresponding to map elements in each sample image of a certain region, the fifth or sixth geometric figure corresponding to each map element, the orientation of the geometric primitives in the fifth geometric figure, and the image features of the geometric primitives in the fifth geometric figure. The image features of the geometric primitives in the fifth geometric figure are generated when the fifth geometric figure of each map element is obtained by reasoning using the fourth model. The similarity between the fifth geometric figure and the corresponding reference contour is lower than that between the sixth geometric figure and the corresponding reference contour, and the fifth and sixth geometric figures have the same geometric primitives.

[0027] By inputting the image features of the fifth or sixth geometric figure, the geometric primitives in the fifth geometric figure, and the orientation of the geometric primitives in the fifth geometric figure into the fifth model with the first network parameters, the latent space features corresponding to each geometric primitive are obtained, and the predicted topological relationship between each geometric primitive is inferred based on the latent space features corresponding to each geometric primitive.

[0028] Based on the predicted topological relationships and corresponding reference topological relationships between geometric primitives in the fifth geometric figure, a seventh loss function and an eighth loss function are determined. The reference topological relationship can be determined based on the reference contours corresponding to map elements in each sample image. The seventh loss function is used to indicate the matching degree between the predicted topological relationships and corresponding reference topological relationships between geometric primitives in the fifth geometric figure, and the eighth loss function is used to indicate the similarity of latent space features between geometric primitives whose predicted topological relationships are parallel, collinear, or connected. If the seventh and eighth loss functions meet the termination conditions, the fifth model with the first network parameters is saved. If the seventh and eighth loss functions do not meet the termination conditions, the network parameters of the fifth model are adjusted to the second network parameters, and the next round of training is performed.

[0029] In this embodiment, the fifth model can be used to obtain the features of the geometric shape corresponding to the map element and the orientation of the geometric primitives in the geometric shape, such as the third model in the first aspect above, the topology reconstruction network below, etc. During the training process of this model, a fifth geometric shape with lower precision can be used as input to train the fifth model, so that the predicted topological relationship between the geometric primitives in the fifth geometric shape obtained by the fifth model has a higher degree of regionality with the corresponding reference topological relationship. That is, the fifth model can obtain high-precision output data with low-precision input data, which is beneficial to improve the noise resistance of the fifth model. Thus, even when the second geometric shape obtained by the second model has low precision, a more accurate topological relationship between geometric primitives can be obtained, thereby improving the precision of the vector map obtained based on the topological relationship between the geometric primitives.

[0030] For example, in some embodiments, the seventh loss function may be the cross-entropy loss LC described below. EL The eighth loss function can be the supervised contrastive loss Ls mentioned below. CL .

[0031] In one possible implementation of the second aspect above, when the geometric primitives in the fifth geometric figure are line segments, the seventh loss function and the eighth loss function are determined to satisfy the termination condition in the following way: based on the direction of the geometric primitives in the fifth geometric figure, the directional relationship between the geometric primitives and the reference directional relationship corresponding to the topological relationship are determined, and the ninth loss function is determined. The ninth loss function is used to indicate the consistency of the predicted topological relationship and direction of each geometric primitive.

[0032] If the seventh, eighth, and ninth loss functions all converge, or the seventh, eighth, and ninth loss functions are all less than their corresponding preset loss function values, or the total loss function converges, or the total loss function is less than its corresponding preset total loss function value, then the termination condition is determined to be met. The total loss function includes the weighted sum of the seventh, eighth, and ninth loss functions.

[0033] For example, in some embodiments, the ninth loss function may be the consistency loss L of geometric attributes and relationships, as discussed below. C .

[0034] In one possible implementation of the second aspect above, obtaining the latent space features corresponding to each geometric primitive based on the image features of the fifth or sixth geometric figure, the geometric primitives in the fifth geometric figure, and the orientation of the geometric primitives in the fifth geometric figure includes: when the geometric primitive of the fifth geometric figure is a point, obtaining the latent space features corresponding to each geometric primitive based on the fifth geometric figure, the image features of the geometric primitives in the fifth geometric figure, and the orientation of the geometric primitives in the fifth geometric figure; when the geometric primitive of the fifth geometric figure is a line segment, obtaining the latent space features corresponding to each geometric primitive based on the image features of the sixth geometric figure, the geometric primitives in the fifth geometric figure, and the orientation of the geometric primitives in the fifth geometric figure.

[0035] In this embodiment, since the complexity of a polyline is lower than that of a polygon, when training the fifth model, the fifth geometric figure is used as the input of the fifth model when the fifth geometric figure is a polyline, and the sixth geometric figure with higher accuracy is used as the input of the fifth model when the fifth geometric figure is a polygon. This can improve the noise resistance of the fifth model to simpler polyline input data while ensuring the accuracy of the fifth model in reasoning about the topological relationships of geometric primitives in more complex polygons.

[0036] Thirdly, embodiments of this application provide a map generation apparatus, comprising: a data acquisition unit for acquiring an image of a certain area, the image including map elements, wherein the map elements are elements in the image to be converted into a vector map; an initial shape generation unit for inferring from the image using a first model to obtain a first geometric figure corresponding to the map elements, the first geometric figure including geometric primitives; a shape regression unit for inputting a second model based on the first geometric figure to obtain the orientation of each geometric primitive, and obtaining a second geometric figure corresponding to the map elements based on the first geometric figure, the second geometric figure including the same geometric primitives as the first geometric figure, and the arrangement of the geometric primitives in the second geometric figure being different from the arrangement of the geometric primitives in the first geometric figure; a topology reconstruction unit for using a third model to obtain the topological relationship between each geometric primitive based on the orientation of the geometric primitives and the second geometric figure; and a post-processing unit for obtaining a vector map corresponding to the image based on the topological relationship between each geometric primitive, the orientation of each geometric primitive, and the second geometric figure.

[0037] In this embodiment, the electronic device first uses a first model (e.g., the shape initialization network hereinafter referred to as the shape initialization network) to infer the contours of map elements in an image of a certain region, obtaining a first geometric shape (e.g., the initial shape hereinafter referred to as the initial shape) corresponding to the map elements. Then, a second model (e.g., the shape regression network hereinafter referred to as the shape initialization network) is used to adjust the first geometric shape, obtaining a second geometric shape with higher accuracy and more regular shape (e.g., the regression shape hereinafter referred to as the regression shape). Then, a third model (e.g., the topology reconstruction network hereinafter referred to as the topology reconstruction network) is used to infer the topological relationships between geometric primitives in the second geometric shape (e.g., in the case that the second geometric shape is a polyline, inferring the topological relationships between the points that make up the polyline). Finally, based on the topological relationships between geometric primitives, a vector map corresponding to the image is obtained. In this way, the electronic device converts map elements into a vector map based on pre-trained first, second, and third models, rather than obtaining a vector map based on vectorization rules set by technicians, which helps to improve the accuracy of the obtained vector map. Furthermore, in the scenario of large-scale vector map modeling, retraining at least one of the first, second, and third models for different regions can effectively adapt to the geometric features of map elements in different regions without the need for setting vectorization rules and complex parameter adjustments, which is beneficial to improving the efficiency of vector map modeling.

[0038] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed by an electronic device, cause the electronic device to implement any one of the methods provided in the first aspect, various possible implementations of the first aspect, the second aspect, and various possible implementations of the second aspect.

[0039] Fifthly, embodiments of this application provide an electronic device comprising: a memory for storing instructions executable by one or more processors of the electronic device; and a processor, one of the processors of the electronic device, for executing the instructions stored in the memory to implement any one of the methods provided in the first aspect, various possible implementations of the first aspect, the second aspect, and various possible implementations of the second aspect.

[0040] Sixthly, embodiments of this application provide a computer program product, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, it implements any one of the methods provided in the first aspect, various possible implementations of the first aspect, the second aspect, and various possible implementations of the second aspect described above. Attached Figure Description

[0041] Figure 1A According to some embodiments of this application, a schematic diagram of a process for obtaining a vector map from an image is shown;

[0042] Figure 1B According to some embodiments of this application, an image schematic diagram including only one map element is shown;

[0043] Figure 1C According to some embodiments of this application, an image schematic diagram including multiple map elements is shown;

[0044] Figure 2 According to some embodiments of this application, a schematic diagram of a process for generating maps using a neural network model is shown;

[0045] Figure 3 According to some embodiments of this application, a schematic diagram of the structure of a shape initialization network 1 is shown;

[0046] Figure 4 According to some embodiments of this application, a schematic diagram of the training process of a shape initialization network 1 is shown.

[0047] Figure 5 According to some embodiments of this application, a schematic diagram of a house and its corresponding contour mask in image IM2 is shown;

[0048] Figure 6A According to some embodiments of this application, a schematic diagram of the coordinate distance between a point in the edge region of a contour mask and a reference contour is shown.

[0049] Figure 6B According to some embodiments of this application, a schematic diagram is shown of the coordinate distance between a point in the edge region of a contour mask and the outermost pixel of the contour mask;

[0050] Figure 7 According to some embodiments of this application, a schematic diagram of the structure of a shape regression network 2 is shown;

[0051] Figure 8 According to some embodiments of this application, a schematic diagram of the training process of a shape regression network 2 is shown;

[0052] Figure 9 According to some embodiments of this application, a schematic diagram of the structure of a topology reconstruction network 3 is shown;

[0053] Figure 10 According to some embodiments of this application, a schematic diagram of the training process of a topology reconstruction network 3 is shown;

[0054] Figure 11 According to some embodiments of this application, a schematic diagram of the calculation process of topological relationship and cross-entropy loss is shown;

[0055] Figure 12According to some embodiments of this application, a schematic diagram of a training process and an inference process is shown;

[0056] Figure 13 According to some embodiments of this application, a flowchart of a map generation method is shown;

[0057] Figure 14 According to some embodiments of this application, a schematic diagram of post-processing a polygon is shown;

[0058] Figure 15 According to some embodiments of this application, a schematic diagram of the result of vectorizing houses in a partial remote sensing image using neural network model 0 is shown.

[0059] Figure 16 According to some embodiments of this application, a schematic diagram of the result of vectorizing a road in a remote sensing image using a neural network model 0 is shown.

[0060] Figure 17A According to some embodiments of this application, a schematic diagram of the reconstruction effect of a road in a relatively complex remote sensing image using a neural network model 0 is shown.

[0061] Figure 17B According to some embodiments of this application, a schematic diagram of the reconstruction effect of a road in another complex remote sensing image using a neural network model 0 is shown.

[0062] Figure 18 According to some embodiments of this application, a schematic diagram of the structure of a map generation device is shown;

[0063] Figure 19 According to some embodiments of this application, a schematic diagram of the structure of an electronic device 100 for performing embodiments of this application is shown. Detailed Implementation

[0064] The illustrative embodiments of this application include, but are not limited to, map generation methods, model training methods, readable media, program products, devices, and electronic devices.

[0065] To facilitate understanding, the terminology used in this application will first be explained.

[0066] (1) Loss Function

[0067] In the training process of a neural network model, since the goal is to make the output of the neural network model as close as possible to the actual predicted value, we can compare the current predicted value of the network with the actual target value, and then update the weight vector of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring the parameters of each layer in the neural network model). For example, if the network's predicted value is too high, the weight vector is adjusted to lower the predicted value, and this adjustment is continued until the neural network model can predict the actual target value or a value very close to the actual target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value", which is the loss function or objective function. These are important equations used to measure the difference between the predicted value and the target value. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the neural network model becomes the process of minimizing this loss as much as possible. Therefore, whether the loss function is set reasonably directly affects the quality of the neural network model training method.

[0068] The technical solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0069] As mentioned earlier, the method of converting the contours of map elements inferred by a neural network model into vector maps using vectorization rules may result in lower accuracy if the same vectorization rules are applied to images of different regions. For example, in different cities or regions, the contour features of buildings vary significantly due to differences in architectural style. If the vectorization rules optimize the corners of building geometric contours to right angles, incorrect building contours will be obtained for regions where building geometric contours are circular or irregular polygonal. Setting different contour acquisition methods and vectorization rules for different regions is complex, leading to low efficiency in generating vector maps and making it unsuitable for large-scale vector map modeling scenarios.

[0070] Vectorization rules, as understood, refer to rules set by technicians to adjust the relationships between lines or points in the outline of map elements to obtain a more reasonable vectorized map. For example, setting two lines with an angle greater than a preset value as parallel or collinear, setting two lines with an angle within a certain range as perpendicular, and moving points less than a preset value from a line to that line. It's easy to understand that the effectiveness of setting vectorization rules depends on the experience of the technicians and the outline of the referenced map elements, making the adjustment process complex. When the geometric features of the map elements referenced for setting vectorization rules differ significantly from the features of the map elements in the area to be vectorized, a vector map with lower accuracy will be obtained. For example, if the angles of the sides of houses in the referenced map elements are mostly right angles, while the houses in a certain area are mainly circular, then the vectorization rules may result in a vector map with a shape significantly different from the actual map elements.

[0071] To address the aforementioned problems, this application provides a map generation method based on a neural network model. This model represents map elements using geometric shapes (e.g., polylines for roads, polygons for buildings). By learning the geometric features of map elements from sample images of the target region, the method ensures that the geometric shapes inferred from the contours of map elements by the neural network model match the geometric features of the map elements in the target region. Then, based on the trained neural network model, the method infers from the predicted image of the target region to obtain the geometric shapes corresponding to the map elements in that region, thus generating a vector map. In other words, this application vectorizes map elements in an image by learning their geometric features, rather than by developers setting vectorization rules, thereby improving the accuracy of the vector map. Furthermore, when performing vector map inference for different regions, the neural network model can be retrained using sample images from each region to adapt well to the geometric features of map elements in that region, without the need for complex vectorization rule settings and adjustments. In large-scale map construction scenarios, such as vectorizing maps of regions including multiple areas, cities, or countries, the efficiency of vector map generation can be improved while ensuring the accuracy of the vector map.

[0072] For example, refer to Figure 1AIn some embodiments, a semantic segmentation network can be used to obtain contour masks of map elements (such as houses, roads, etc.) in an image. Then, based on heuristic rules (i.e., vectorization rules) set by the developer, the contours of the map elements can be converted into a vector map. In some embodiments of this application, geometric feature learning and topology reconstruction can be used instead of heuristic rules. A neural network model is used to learn the geometric features of map elements in sample images of the target region, so that the geometric figures of map elements obtained by the neural network model can more accurately reflect the geometric features of map elements in the target region. Then, the neural network model is used to determine the topological relationships between geometric primitives in the geometric figures, and then the geometric primitives of the geometric figures are topologically connected to obtain a vector map. For example, connecting points into polylines to represent roads or rivers, and connecting lines into polygons to represent houses or lakes.

[0073] It can be understood that geometric primitives refer to the basic constituent elements of various geometric figures. For example, the geometric primitive of a broken line can be a point, and the geometric primitive of a polygon can be an ordered line segment.

[0074] It is understood that map elements can include, but are not limited to, houses, lakes, oceans, roads, rivers, forests, deserts, etc. For map elements that need to be described with specific shapes on the map, such as houses, lakes, oceans, forests, deserts, etc., polygons can be used to represent them. For map elements that do not need to be described with specific shapes on the map, such as roads, rivers, etc., polylines can be used to represent them. For ease of description, in the following embodiments, houses are used as map elements represented by polygons and roads are used as map elements represented by polylines.

[0075] It is understood that an image can include at least one map element. For example, see reference. Figure 1B Image IM11 only includes road RD1, which can be represented by a polyline in a vector map; for example, refer to Figure 1C Image IM12 includes house HE1, road RD2, and river RR1. House HE1 can be represented by a polygon, while road RD2 and river RR1 can be represented by polylines.

[0076] To facilitate understanding, we will first introduce the process of converting remote sensing images into vector maps using a neural network model.

[0077] Figure 2 According to some embodiments of this application, a schematic diagram of a process for generating maps using a neural network model is shown. For example... Figure 2 As shown, converting remote sensing images into vector maps using neural network model 0 typically involves the following steps:

[0078] S21: Labeling Features. Map elements, such as houses, roads, and lakes, in a portion of the remote sensing image of the target area are labeled to obtain reference contours of the map elements in that portion of the image (e.g., vectorized contours of houses, vectorized centerlines of roads, etc.). This portion of the image and the corresponding reference contours of the map elements can be used as a sample image set.

[0079] S22: Model Training. The neural network model 0 is trained using a set of sample images, enabling it to vectorize map elements in each sample image from the set, resulting in predicted shapes with high similarity to the reference contours of each map element.

[0080] S23: Map Reasoning. Using the trained neural network model 0, reasoning is performed on the predicted image set (i.e., images other than the sample image set) in the remote sensing images of the target area to obtain the predicted shape of the map elements in each predicted image, wherein the predicted shape includes the geometric primitives of the predicted shape and the topological relationships between the geometric primitives;

[0081] S24: Post-processing. Post-processing is performed on the predicted shapes output by neural network model 0, such as connecting the geometric primitives in each predicted shape to obtain the vectorized shapes of map elements, stitching together the vectorized shapes of map elements in different remote sensing images, etc., to obtain a predicted vector map;

[0082] S25: Correction. The predicted vector map is corrected by surveyors to obtain a vector map of the target area, ensuring the accuracy of the vector map.

[0083] Continue to refer to Figure 2 In some embodiments, the neural network model 0 described above may include a shape initialization network 1, a shape regression network 2, and a topology reconstruction network 3.

[0084] The shape initialization network 1 is used to extract the initial shape of each map element in the remote sensing image. For example, the shape initialization network 1 can extract houses as polygons and roads as polylines in the remote sensing image IM2. In some embodiments, the shape initialization network 1 is also used to determine key points in roads and rivers, such as intersections (road junctions) and branching and confluence points in rivers.

[0085] Shape Regression Network 2 is used to optimize the initial shape obtained by Shape Initialization Network 1, to obtain the regression shape of each map element and the orientation data of each geometric primitive in the regression shape, so as to improve the accuracy of the geometry of each map element.

[0086] The topology reconstruction network 3 is used to infer the topological relationships between geometric primitives in the regression shape of each map element. For example, the geometric primitives of a map element described by a polygon can be line segments, and the topological relationships between line segments can include, but are not limited to, collinearity and parallelism. For another example, a map element described by a polyline can be a point, and the topological relationships between points can include connection and disconnection.

[0087] After obtaining the topological relationships between the geometric primitives of each map element, the post-processing module 4 can be used to perform post-processing operations such as connecting the geometric primitives and stitching the map elements according to the topological relationships between them, to obtain a vector map. It is understood that in some embodiments, the post-processing module 4 can be implemented as a neural network or other processing logic, which is not limited here.

[0088] It is understood that each network in neural network model 0 may include one or more neural network layers, including but not limited to semantic segmentation networks, convolutional networks, pooling networks, classification networks, activation networks, attention mechanism networks, fully connected networks, recurrent neural networks, batch normalization (BN) networks, etc.

[0089] Understandable. Figure 2 The structure of the neural network model 0 shown is merely an example. In other embodiments, the neural network model 0 may include more or fewer networks, and may combine or split parts of the networks; this is not limited here. For example, in some embodiments, the post-processing module 4, implemented in the form of a neural network, may be included in the neural network model 0.

[0090] The training process of each network in neural network model 0 is described below.

[0091] First, we will introduce the training process of Shape Initialization Network 1.

[0092] Figure 3 According to some embodiments of this application, a schematic diagram of the structure of a shape initialization network 1 is shown.

[0093] like Figure 3 As shown, the shape initialization network 1 includes a semantic segmentation network 11, a mask generation network 12, an edge extraction network 13, and a shape generation network 14.

[0094] The semantic segmentation network 11 is used to extract image features (Image Embedding) from each sample image in the sample image set. In some embodiments, the semantic segmentation network 11 may include a Feature Pyramid Network (FPN).

[0095] The mask generation network 12 is used to obtain the contour mask of each map element based on the image features of each sample image in the sample image set. For example, in some embodiments, the mask generation network 12 may include a concatenated convolutional network, a batch normalization (BN) network and an activation network (e.g., a rectified linear unit, ReLU).

[0096] The edge extraction network 13 is used to extract the edges of the contour mask based on the contour mask and image features of each map element, thereby obtaining the mask edges of each map element, wherein the mask edges are used to describe the contour of the map element. In some embodiments, the edge extraction network 13 may include a concatenated convolutional network, a batch normalization (BN) network, and an activation network (e.g., a linear rectified function).

[0097] The shape generation network 14 is used to simplify the mask edges of each map element to obtain the initial shape of each map element, thereby reducing the number of geometric primitives included in the initial shape. For example, it reduces the number of line segments included in polygons and the number of points included in polylines, thereby improving the speed of inference on the input image using the neural network model 0. In some embodiments, algorithms such as Douglas Peucker (DP) can be used to simplify polygons, and algorithms such as non-maximum suppression (NMS) can be used to simplify polylines.

[0098] Understandable. Figure 3 The structure of the shape initialization network 1 shown is only an example. In other embodiments, the shape initialization network 1 may also adopt other structures, and each network may also be implemented using other types of neural networks. This is not limited here.

[0099] The following is combined Figure 3 The structure of Shape Initialization Network 1 is shown, and the training process of Shape Initialization Network 1 is introduced.

[0100] Specifically, Figure 4 According to some embodiments of this application, a schematic diagram of the training process for a shape initialization network 1 is shown. The execution entity of this process is an electronic device, such as... Figure 4 As shown, the process includes the following steps:

[0101] S401: Obtain the sample image set.

[0102] The electronic device acquires a set of sample images of the target area, which includes reference contours of map elements in each sample image.

[0103] In some embodiments, the sample image set may include N sample images, each sample image having a size of H×W (i.e., H pixels high and W pixels wide). Each pixel in the sample image may include n channels (n is the number of color channels in the sample image; for example, if the sample image is an RGB image, then n = 3). Thus, the sample image set can be represented as a 4-dimensional matrix P, with a size of N×n×H×W. The element P(i, j, k, m) of matrix P represents the value of the j-th color channel of the element in the k-th row and m-th column of the i-th sample image.

[0104] It is understood that, in some embodiments, the reference contours of map elements in the sample image set may include vector data of each map element determined manually, such as roads represented by polylines, houses represented by polygons, etc., so that the quality of the results inferred by the neural network model 0 can be evaluated through the reference contours during the training of the neural network model 0, and the network parameters of the neural network model 0 can be adjusted according to the evaluation results.

[0105] S402: Use semantic segmentation network 11 to obtain the image features of map elements in the sample image set.

[0106] Electronic devices utilize semantic segmentation networks 11, such as FPN networks, to extract features from a set of sample images, thereby obtaining the image features of the set of sample images.

[0107] In some embodiments, the electronic device can input matrix P into the semantic segmentation network 11 to obtain an image feature matrix F of the sample image set. The size of matrix F is N×C×H×W, where C is the number of image features extracted by the semantic segmentation network 11 for each sample image. The element P(i, j, k, m) in matrix F represents the value of the j-th feature of the element in the k-th row and m-th column of the i-th sample image, which is predetermined by the type of the semantic segmentation network 11 or preset by the developer.

[0108] S403: Based on the image features of the sample image set, the contour mask of each map element is obtained by using the mask generation network 12, and the mask edge of each contour mask is obtained by using the edge extraction network 13.

[0109] In some embodiments, the electronic device can input the image feature matrix F of the sample image into the mask generation network 12 to obtain the contour mask of each map element. For example, in some embodiments, the contour mask of each map element can be represented as a matrix M, the size of which is N×p×H×W, where p represents the number of categories of the map element (hereinafter, we will take two categories as an example). For example, map elements can be divided into map elements represented by polygons (p=1) and map elements represented by polylines (p=2). Thus, for a given p and N, a submatrix of size 1×1×H×W in matrix M represents the contour mask of the p-th class of map elements in the N-th sample image. For example, in this 1×1×H×W submatrix, the values ​​of elements belonging to the same class of map elements can be the same, as shown in the reference. Figure 5 For the sample image IM2, in a submatrix of size 1×1×H×W, the value of the pixel containing the house can be 1, and the value of other pixels can be 0.

[0110] After obtaining the contour mask of the map elements, in some embodiments, the electronic device can input the aforementioned contour mask and the corresponding reference contour into the edge extraction network 13 to obtain the mask edges of the contour mask. For example, the reference... Figure 6A The edge extraction network 13 can infer the coordinate distance DT between each pixel in the edge region of the contour mask and the reference contour. The size of DT is N×2×H×W. The element DT(i,j,k,m) in DT represents the coordinate distance between the pixel in the k-th row and m-th column of the j-th class map element in the i-th sample image and the reference contour. DT(i,j,k,m) can include two elements dx and dy, representing the coordinate distance in the H and v directions, respectively. For example, B1 is... Figure 6A Given a point within the edge region of the contour mask with coordinates (x, y), and a distance DT from B1 to the reference contour with coordinates (dx, dy), then the coordinates of the point on the mask edge corresponding to B1(x, y) are (x + dx, y + dy). Thus, by adding the coordinates of all points within the edge region to their corresponding coordinate distances, the coordinates of the points on the mask edge of the contour mask can be obtained. Assuming the edge region of the contour mask has L pixels, the mask contour obtained from these L pixels can be represented as a point set (x...y ... i +dx i ,y i +dy i (i = 1, 2, ..., L).

[0111] It is understandable that the size of the edge region of the contour mask can be preset. For example, the edge region can be a region composed of pixels whose distance from the outermost pixel of the contour mask is less than a preset edge distance threshold.

[0112] It is understood that, in some embodiments, the coordinate distance from a pixel within the edge region to the reference contour can be the difference between the coordinates of the nearest point on the reference contour to the pixel and the coordinates of the pixel itself.

[0113] It is understandable that during the process of extracting the mask edges corresponding to the contour masks of map elements using the trained shape initialization network 1, since there is no reference contour, the aforementioned DT can be the coordinate distance from a point within the edge region of the contour mask to the contour formed by the outermost pixels of the contour mask. For example, reference... Figure 6B If the coordinate distance from point E1(x,y) in the edge region of the contour mask to the contour formed by the outermost pixels of the contour mask is (dx,dy), then the coordinates of point E1 corresponding to point E1′ on the edge of the contour mask are (x+dx,y+dy).

[0114] It is understood that in other embodiments, the mask edge of the contour mask can also be obtained in other ways, such as directly using the outermost point of the contour mask as the mask edge of the contour mask, which is not limited here.

[0115] S404: Use shape generation network 14 to simplify the mask edges and obtain the initial shape.

[0116] Electronic devices simplify the mask edges of the contour masks of each map element to obtain the initial shape of each map element. This reduces the number of geometric primitives included in the initial shape and improves the speed at which electronic devices can perform inference on remote sensing images using neural network models. For example, polygon simplification algorithms, such as the DP algorithm, can be used to simplify polygons to obtain an initial polygon shape with fewer line segments; similarly, line simplification algorithms, such as the NMS algorithm, can be used to thin out the points on a polyline to obtain an initial polyline shape with fewer points.

[0117] It is understood that the above simplification of the mask edge using the DP algorithm or NMS algorithm is just an example. In other embodiments, other algorithms can also be used for simplification, which will not be elaborated here.

[0118] It is understood that, in some embodiments, the initial shape obtained by the electronic device may also include image features of each geometric primitive in the initial shape.

[0119] S405: Calculate the loss function and determine whether the termination condition is met based on the loss function.

[0120] The electronic device calculates the loss function based on the coordinates of each predicted point in the initial shape and the coordinates of the corresponding reference points on the reference contour, and determines whether the termination condition is met based on the loss function. If it is met, it means that the initial shape obtained by the shape initialization network 1 meets the requirements, and proceeds to step S406; otherwise, it means that the shape initialization network 1 cannot obtain an initial shape that meets the requirements based on the current network parameters, and proceeds to step S407.

[0121] It is understood that in some embodiments, different loss functions may be used for each network in the shape initialization network 1.

[0122] In some embodiments, for the semantic segmentation network 11 and the mask generation network 12, the loss function can be the cross-entropy loss function, the focal loss function, the 0-1 loss, the entropy and cross-entropy loss, the softmax loss, etc.

[0123] For example, assuming an image contains N1 pixels, the semantic segmentation network 11 and the mask generation network 12 classify the N1 pixels into M1 classes (i.e., into M1 class map elements). Then, the cross-entropy loss L of the semantic segmentation network 11 and the mask generation network 12... 11-12-CEL It can be expressed as the following formula (1).

[0124]

[0125] In formula (1), yi j For a 0-1 variable, when the i-th pixel is within the contour mask area of ​​the j-th type of map element, yi j =1, otherwise yij=0; p ij The probability of the i-th pixel within the contour mask region of the j-th map element determined by the mask generation network 12. From formula (1), it can be seen that the cross-entropy loss L... 11-12-CEL L is used to indicate the accuracy of the contour mask obtained by the mask generation network 12. 11-12-CEL The smaller the value, the higher the accuracy of the contour mask obtained by the mask generation network 12.

[0126] It is understandable that the cross-entropy loss L between semantic segmentation network 11 and mask generation network 12... 11-12-CEL This reflects the accuracy of the contour mask obtained using the semantic segmentation network 11 and the mask generation network 12, L 11-12-CEL The smaller the value, the higher the precision.

[0127] In some embodiments, the loss function corresponding to the edge extraction network 13 may include mean squared error (MSE, also known as L2 loss).

[0128] Assume an initial shape includes N² prediction points, and the coordinates of the i-th prediction point are (x... i y i The coordinates of the reference point corresponding to the i-th predicted point in the reference profile are (X... si y si Then the L2 loss L of edge extraction network 13 is... 13-L2 It can be expressed as the following formula (2):

[0129]

[0130] It is understandable that the L2 loss L of edge extraction network 13 13-L2 This reflects the similarity between the mask edges of the contour mask obtained by the edge extraction network 13 and the corresponding reference contour, L 13-L2 The smaller the value, the higher the similarity, and the higher the accuracy of the edge extraction network 13.

[0131] It is understood that in other embodiments, other types of loss functions may also be used to determine whether the termination condition is met.

[0132] It is understood that the termination condition may include at least one of the following conditions: the loss function corresponding to each network converges, or the loss function value corresponding to each network is less than the corresponding preset loss function value. For example, the termination condition is determined to be met when both the cross-entropy loss function and the L2 loss converge; or, for example, the termination condition is determined to be met when the cross-entropy loss function is less than the corresponding first preset loss function value and the L2 loss value is less than the corresponding second preset loss function value.

[0133] It is understood that in other embodiments, the termination condition may also include other conditions, which are not limited here. For example, in some embodiments, the loss functions of each network may be weighted and summed (i.e., the loss functions of each network are multiplied by their respective weight values ​​and then summed) to obtain the total loss function. When the total loss function converges or is less than a preset total loss function value, the termination condition is determined to be met. For example, when the loss function includes the aforementioned L... 11-12-CEL and L 13-L2 In this case, the total loss function can be expressed as λ1L 11-12-CEL +λ2L 13-L2 λ1 represents the weight of cross-entropy loss and λ2 represents the weight of L2 loss. λ1 and λ2 can be preset by the developers.

[0134] S406: Store network parameters to obtain shape initialization network 1.

[0135] The electronic device stores the network parameters currently used by shape initialization network 1, thus obtaining shape initialization network 1.

[0136] S407: Adjust network parameters and proceed with the next round of training.

[0137] When the electronic device determines that the termination condition is not met, it adjusts the network parameters of shape initialization network 1 and proceeds to the next round of training. For example, if the loss functions of all networks fail to meet the corresponding termination condition, the network parameters of all networks are adjusted, and the next round of training is conducted. Another example is if the loss functions of some networks meet the corresponding termination condition, while the loss functions of others do not; in this case, the network parameters of the networks whose loss functions do not meet the termination condition are adjusted, and the next round of training is conducted. Yet another example is if the total loss function fails to meet the corresponding termination condition; in this case, the network parameters of at least some networks are adjusted, and the next round of training is conducted.

[0138] As can be seen from the training process of the shape initialization network 1 described above, the shape initialization network 1 is based on learning the reference contours of each map element in the sample image of the target region, that is, learning the geometric features of the map elements in the target region, to perform semantic segmentation on the image and obtain the initial shape of each map element. It can better adapt to the geometric features of the map elements in the target region and improve the accuracy of the initial shape of each map element.

[0139] For the trained shape initialization network 1, the electronic device can input the remote sensing image into the network to obtain the initial shape and image features of the map elements in the remote sensing image.

[0140] The training process of Shape Regression Network 2 is described below.

[0141] Figure 7 According to some embodiments of this application, a schematic diagram of the structure of a shape regression network 2 is shown.

[0142] like Figure 7 As shown, the shape regression network 2 includes a pooling network 21, a feature encoding network 22, an orientation generation network 23, and a shape adjustment network 24.

[0143] Pooling network 21 is used to pool and interpolate the feature parameters of each geometric primitive in the initial shape to obtain the pooled features of each geometric primitive. For example, the image features of the sample image extracted by the aforementioned semantic segmentation network 11 are in pixels. However, after passing through the aforementioned shape generation network 13, there is no one-to-one correspondence between the coordinates of each geometric primitive in the initial shape and the image features obtained by the semantic segmentation network 11. In this case, the pooling network 21 can obtain the features of each geometric primitive in the initial shape by interpolation based on the image features of the adjacent pixels of each geometric primitive in the initial shape. For example, for an initial shape represented by a polygon, the image features of the geometric primitive can be interpolated using the Line of Interest (LOI) method. For an initial shape represented by a polyline, the image features can be interpolated using the Point of Interest (POI) method. Specific examples will be given below and will not be elaborated here.

[0144] The feature encoding network 22 is used to re-encode the pooling features of each geometric primitive in the initial shape to obtain the regression encoding features of each geometric primitive. These regression encoding features can be used to infer the orientation data of each geometric primitive, adjust the initial shape, etc. In some embodiments, the feature encoding network 22 may include a multi-head-attention network.

[0145] The direction generation network 23 is used to obtain the direction data of each geometric primitive based on the regression encoding features of each geometric primitive. Specifically, when the geometric primitive is a point, the direction of the geometric primitive can be the tangent direction of that point; when the geometric primitive is a line segment, the direction of the geometric primitive can be the direction of the line segment. In some embodiments, the direction generation network 23 may include a concatenated convolutional network, a batch normalization (BN) network, and an activation network.

[0146] It is understood that in some embodiments, during the training of the shape regression network 2, the orientation data obtained by the orientation generation network 23 can be used to calculate the angle constraint loss and L2 loss of the orientation data, and the network parameters of the shape regression network 2 can be adjusted according to the obtained angle constraint loss and L2 loss to improve the accuracy of the orientation data of the geometric primitives obtained by the orientation generation network 23. The specific calculation method will be introduced below and will not be elaborated here.

[0147] The shape adjustment network 24 is used to adjust the position of each geometric primitive in the initial shape according to the regression encoding features of each geometric primitive, so as to obtain a more accurate regression shape, and to calculate the coordinate residual of the predicted point in each geometric primitive in the regression shape relative to the coordinate residual of the corresponding point on the reference contour, so as to calculate the loss function. In some embodiments, the orientation generation network 23 may include a concatenated convolutional network, a batch normalization network, and an activation network.

[0148] It is understood that in some embodiments, the regression shape generated by the shape adjustment network 24 can be used to calculate the relative shape loss, and the network parameters of the shape regression network 2 can be adjusted according to the relative shape loss to improve the accuracy of the regression shape obtained by the shape adjustment network 24. The specific calculation method will be introduced below and will not be elaborated here.

[0149] Specifically, Figure 8 According to some embodiments of this application, a schematic diagram of the training process of a shape regression network 2 is shown. The execution entity of this process is an electronic device, such as... Figure 8 As shown, the process includes the following steps.

[0150] S801: Perform feature pooling on the image features and classification features of the geometric primitives in the initial shape to obtain the pooled features of the initial shape.

[0151] As mentioned earlier, the image features and classification features of the sample images obtained by the aforementioned shape initialization network 1 are based on pixels. However, there is no one-to-one correspondence between the points and pixels included in the initial shape. Therefore, pooling, interpolation, etc., can be performed on the image features and classification features of the geometric primitives in the initial shape to obtain the pooling features of each geometric primitive.

[0152] For example, for an initial shape represented by a polyline, whose geometric primitives are points, the pooling features of each geometric primitive in the initial shape can be obtained using the POI algorithm. Specifically, assuming that the aforementioned semantic segmentation network 11 obtains the vector c0 corresponding to the image features of point A and the vector c1 corresponding to the image features of point B, then in the initial shape, the vector corresponding to the image features of point C located on line segment AB is c0 + (c1 - c0)1. AC / l AB , where l AC Let AC be the length of line segment AC, 1 AB Let AB be the length of line segment AB.

[0153] For example, for an initial shape represented by a polygon, whose geometric primitives are line segments, the pooling features of each geometric primitive in the initial shape can be obtained using the LOI algorithm. Specifically, multiple points on the line segment (e.g., 32 points) can be obtained, and these multiple points can be divided into several groups (e.g., 32 points can be divided into 4 groups). The pooling features of these multiple points can be obtained using the POI algorithm. Then, the average of the pooling features of the points in each group can be calculated to obtain the pooling features of that group (e.g., the vectors corresponding to the pooling features of the 4 groups are n1, n2, n3, n4). The pooling features of each group can be connected to obtain the pooling features of the line segment (e.g., connecting the aforementioned vectors n1, n2, n3, n4 yields the vector n5 = [n1 n2 n3 n4] corresponding to the pooling features of the line segment).

[0154] It is understood that in other embodiments, the pooling characteristics of each geometric primitive in the initial shape can also be determined in other ways, which are not limited here.

[0155] S802: The pooling features of the initial shape are encoded using the feature encoding network 22 to obtain the regression encoded features.

[0156] The electronic device uses the feature coding network 22 to re-encode the pooling features of the initial shape. For example, it discards features in the pooling features that have little impact on shape adjustment and orientation data, and re-extracts features that have a greater impact on shape adjustment and orientation data to obtain regression coding features.

[0157] In some embodiments, the feature encoding network 22 may be an encoding network based on a global attention mechanism, such as the aforementioned multi-head attention network.

[0158] S803: Based on the regression coding features of the initial shape, the direction generation network 23 is used to obtain the predicted direction data of the initial shape, and the shape adjustment network 24 is used to adjust the initial shape to obtain the regression shape.

[0159] The electronic device inputs the encoded features of each geometric primitive of the initial shape into the orientation generation network 23 and the shape adjustment network 24 to obtain the predicted orientation data and the regressed shape of each geometric primitive in the initial shape, respectively.

[0160] It is understood that in some embodiments, when the initial shape is a polygon and the geometric primitive is a line segment, the direction of the geometric primitive is the direction of the line segment; when the initial shape is a polyline and the geometric primitive is a point, the direction of the geometric primitive is the tangent direction of the point.

[0161] It is understood that in some embodiments, the obtained regression shape includes the orientation data of each geometric primitive, the coordinate data of the points in each geometric primitive, and the order of each geometric primitive.

[0162] S804: Based on the orientation data of the regression shape and geometric primitives and the calculation of the loss function.

[0163] The electronic device calculates the loss function based on the orientation data of each geometric primitive in the initial shape and the regression shape.

[0164] For example, in some embodiments, the loss function may include an L2 loss between the predicted orientation data of each geometric primitive and the orientation data of the corresponding point or line in the reference contour, used to indicate the accuracy of the orientation of each geometric primitive obtained by the orientation generation network 23. The smaller the L2 loss, the more accurate the orientation data of the geometric primitive obtained by the orientation generation network 23. Specifically, assuming an initial shape includes N3 primitives, and the orientation data of the i-th primitive is dr i The direction data of the point or line corresponding to the i-th primitive in the reference contour is dr. si Then the L2 loss L of the direction generation network 23 23-L2 It can be expressed as the following formula (3):

[0165]

[0166] For example, in some embodiments, the loss function may include a relative shape loss based on the coordinate residuals of the geometric primitives in the regressed shape and the corresponding points or lines in the reference contour, used to evaluate the accuracy of the regressed shape. The relative shape loss can be represented by the average shape loss of all points in the initial shape, such as the sum of the average and sum of the shape losses. Specifically, the shape loss of a non-intersection point in a polyline can be the projected distance from that point to the reference contour; the shape loss of an intersection point in a polyline can be the distance from that point to the corresponding reference intersection point in the reference contour; and the shape loss of a point in a polygon can be the projected distance from that point to the reference contour. The relative shape loss indicates the similarity between the regressed shape of a map element and the reference contour. A lower relative shape loss indicates a higher similarity between the regressed shape obtained using the shape regression network 2 and the corresponding reference contour, and a higher accuracy of the regressed shape.

[0167] For example, in some embodiments, the loss function may include an angle constraint loss to improve the regularity of the regressed shape. In some embodiments, the angle constraint loss L Tv It can be expressed as the following formula (4).

[0168]

[0169] In formula (4), N4 is the number of angles in the initial shape. α is the average angle of all angles in the initial shape. k Let L be the angle of the k-th angle. This can be understood as... TvThe smaller the value, the more regular the regression shape.

[0170] It is understood that in other embodiments, the loss function may also include other losses, such as smoothing constraint loss for improving the smoothness of the regression shape, etc., which are not limited here.

[0171] S805: Determine whether the termination condition is met based on the loss function.

[0172] The electronic device determines whether the termination condition is met based on the loss function. If it is met, it means that the regression shape meets the requirements, and proceeds to step S806; otherwise, it means that the regression shape does not meet the requirements, and proceeds to step S807.

[0173] It is understood that the termination condition may include at least one of the following conditions: the loss function of each network converges, or the loss function value of each network is less than the corresponding preset loss function value.

[0174] It is understood that the termination condition may also include the total loss function being less than the total loss function threshold or the total loss function converging, wherein the total loss function can be obtained by weighted summation of the loss functions in the aforementioned step S805.

[0175] It is understood that in other embodiments, the termination condition may also include other conditions, which are not limited here.

[0176] S806: Store network parameters to obtain shape regression network 2.

[0177] The electronic device stores the network parameters of shape regression network 2, thus obtaining shape regression network 2.

[0178] S807: Adjust network parameters and proceed with the next round of training.

[0179] If the electronic device determines that the termination condition is not met, it adjusts the network parameters of Shape Regression Network 2 and proceeds to the next round of training. For example, if the loss functions of each network in Shape Regression Network 2 do not meet the corresponding termination condition, the network parameters of each network can be adjusted; if only the loss functions of some networks do not meet the corresponding termination condition, only the network parameters of those networks can be adjusted and proceed to the next round of training; and if the total loss function does not meet the corresponding termination condition, at least some network parameters of each network can be adjusted and proceed to the next round of training.

[0180] The training process of Topology Reconstruction Network 3 is described below.

[0181] Figure 9 According to some embodiments of this application, a schematic diagram of the structure of a topology reconstruction network 3 is shown. For example... Figure 9As shown, the topology reconstruction network 3 includes a pooling network 31, a feature encoding network 32, and a relational reasoning network 33.

[0182] Pooling network 31 is used to interpolate the image features and orientation data of each geometric primitive in the initial shape to obtain the pooling features of each geometric primitive. For details, please refer to the relevant description of pooling network 21 mentioned above, which will not be repeated here.

[0183] The feature encoding network 32 is used to re-encode the pooled features of each geometric primitive in the initial shape, such as discarding features that have little impact on topological relationship reasoning and extracting features that have a greater impact on topological relationship reasoning, to obtain the reasoning encoding features of each geometric primitive. In some embodiments, the feature encoding network 32 may include a multi-head attention network.

[0184] The relational reasoning network 33 is used to obtain the topological relationships between each pair of geometric primitives in the initial shape based on the reasoning encoding features of each geometric primitive. Specifically, when the initial shape is a polygon, the geometric primitives are line segments, and the topological relationships between each pair of geometric primitives include collinearity, parallelism, etc.; when the initial shape is a polyline, the geometric primitives are points, and the relationships between each geometric primitive include connection / disconnection. In some embodiments, the relational reasoning network 33 may include convolutional networks, batch normalization (BN) networks, activation networks, etc.

[0185] It is understood that, in some embodiments, during the training of the topology reconstruction network 3, the predicted topological relationships between geometric primitives obtained by the relational reasoning network 33 can be used to calculate cross-entropy loss and supervised contrast loss, and can be combined with the directional data of geometric primitives to calculate loss functions such as geometric attribute and relational consistency loss. The network parameters of the topology reconstruction network 3 can be adjusted based on the loss functions to improve the accuracy of the predicted topological relationships between geometric primitives obtained by the topology reconstruction network 3. The specific calculation method will be introduced below and will not be elaborated here.

[0186] Specifically, Figure 10 According to some embodiments of this application, a schematic diagram of a training process for a topology reconstruction network 3 is shown. The execution entity of this process is an electronic device, such as... Figure 10 As shown, the process includes the following steps.

[0187] S1001: Perform feature pooling on the image features and orientation data of the geometric primitives to obtain the pooled features of the geometric primitives.

[0188] The electronic device performs feature pooling and other operations on the image features and orientation data of the geometric primitives in the initial shape to obtain the pooling features of each geometric primitive. For details, please refer to the relevant description of step S801, which will not be repeated here.

[0189] It is understandable that for map elements represented by polygons, electronic devices can perform feature pooling on the image features of the regressed shape and the orientation data to obtain the pooled features of the geometric primitives.

[0190] S1002: The pooling features of geometric primitives are encoded using the feature coding network 32 to obtain the inference coding features.

[0191] The electronic device uses the feature coding network 32 to encode the pooling features of the initial shape to obtain the inference coding features of each geometric primitive. For details, please refer to the aforementioned step S802, which will not be repeated here.

[0192] S1003: Based on the inference coding features, the predicted topological relationship between geometric primitives is obtained using the relational inference network 33.

[0193] The electronic device inputs the encoded features of each geometric primitive into the relational reasoning network 33 to obtain the predicted topological relationships between each geometric primitive in the initial shape.

[0194] In some embodiments, the predicted topological relationship between the geometric primitives in the initial shape can be represented by a matrix R, which has a size of K×K, where K is the number of geometric primitives in the initial shape. The element R(i,j) in the matrix R is used to indicate the topological relationship between the i-th geometric primitive and the j-th geometric primitive, such as connection, collinearity, parallelism, etc.

[0195] For example, refer to Figure 11 The predicted topological relationships between the geometric primitives of the initial shape composed of 8 points P1, P2, P3, P4, P5, P6, P7, and P8 can be an 8×8 matrix, where the element R(i,j) in the i-th row and j-th column represents the topological relationship between points Pi and Pj. For example, R(3,5) = 0 indicates that points P3 and P5 are not connected, R(3,4) = 1 indicates that points P3 and P4 are connected, and R(1,1) = -1 indicates that point P1 has no topological relationship with itself.

[0196] It is understood that in some embodiments, for the initial shape represented by a broken line, the relational reasoning network 33 can perform feature extraction again based on the reasoning encoding features of each geometric primitive to obtain the features of each geometric primitive in the hidden space (also known as the feature space) (hereinafter referred to as hidden space features). By calculating the distance between each geometric primitive and the hidden space features of a certain primitive, and setting the topological relationship between the geometric primitive and the geometric primitive with the geometric primitive whose hidden space distance is the smallest preset number of geometric primitives with the geometric primitive, the network can be connected.

[0197] S1004: Calculate the loss function based on the predicted topological relationships between geometric primitives in the initial shape.

[0198] The electronic device calculates the loss function based on the topological relationships between geometric primitives obtained from the relational reasoning network 33.

[0199] For example, refer to Figure 11 In some embodiments, the loss function may include the cross-entropy loss L between the predicted and reference topological relationships of each geometric primitive. CEL This loss is determined based on the predicted topological relationships between the geometric primitives in the initial shape and the reference topological relationships between the geometric primitives. For example, in some embodiments, L CEL It can be calculated using the following formula (5).

[0200]

[0201] In formula (5), N5 is the number of geometric primitives in the initial shape, and R ij For the predicted topological relation values ​​of the i-th and j-th geometric primitives (e.g., the element value in the i-th row and j-th column of the aforementioned matrix R), R0 ij This represents the reference topological relationship between the i-th and j-th geometric primitives. For example, based on formula (5), Figure 11 The loss of consistency between geometric attributes and relations in the polyline pair L CEL =-18.

[0202] For example, in some embodiments, when the initial shape is a polygon, the loss function may also include a consistency loss L of geometric attributes and relationships. C This is used to characterize the properties of geometric primitives and the consistency of topological relationships between them. During training, this is achieved by reducing L... C This can improve the accuracy of the predicted topological relationships determined by the relational reasoning network 33. In some embodiments, L C It can be calculated using the following formula (6).

[0203]

[0204] In formula (6), N6 is the number of geometric primitives in the initial shape; ci is the attribute of the i-th geometric primitive; c j tr represents the attribute of the j-th geometric primitive; tr represents the ideal distance between the attributes of two geometric primitives with a topological relationship of r. For example, when r represents the topological relationship of two geometric primitives as parallel or collinear, the attribute of the geometric primitive can include the orientation data of the geometric primitive, such as the tangent direction. When two line segments are parallel or collinear, the tangent directions of the two line segments should be the same, so tr should be 0.

[0205] For example, in some embodiments, the loss function may also include a supervised contrastive loss to improve the consistency between the latent space features of each geometric primitive extracted by the relational inference network 33 and the inferred topological relationships between geometric primitives during the process of determining the topological relationships between geometric primitives based on the inference coding features of each geometric primitive. That is, by ensuring that the supervised contrastive loss meets the termination condition, such as being less than a preset supervised contrastive loss or the supervised contrastive loss function converging, the latent space features of geometric primitives with topological relationships such as connectivity and collinearity can be made similar. This results in better consistency between the latent space features of each geometric primitive extracted by the relational inference network 33 and the inference results when the inference coding features corresponding to the geometric primitives of map elements in the prediction image set are input into the relational inference network 33, thereby improving the accuracy of the predicted topological relationships. Specifically, in some embodiments, the supervised contrastive loss L... SCL The calculation can be performed using the following formula (7).

[0206]

[0207] In formula (7), I represents the set of geometric primitives; P(i) represents the set of geometric primitives that are connected or collinear with the i-th geometric primitive, |P(i)| represents the cardinality of the set P(i) (i.e., the number of elements included in the set P(i); A(i) represents the set of geometric primitives that are not connected or collinear with the i-th geometric primitive; z i The vector representing the latent space features of geometric primitive i; z p The vector representing the latent space features of the geometric primitive p; z α τ represents the vector corresponding to the latent space features of the geometric primitive α; τ is the scalar temperature parameter, which is a positive real-domain hyperparameter (i.e., τ∈R). + ), which can be preset by the developers; · represents the vector dot product. From formula (7), it can be seen that the smaller the value of the supervised contrast loss, the greater the similarity of the vectors corresponding to the latent space features of geometric primitives with connected or collinear relationships, and the smaller the similarity of the vectors corresponding to the latent space features of geometric primitives without connected or collinear relationships.

[0208] It is understood that in other embodiments, the supervised comparison loss can also be calculated in other ways, which are not limited here.

[0209] It is understood that in other embodiments, the loss function may include more loss functions, which is not limited here.

[0210] It is understood that in some embodiments, the reference topological relationship of each geometric primitive can be dynamically calculated, that is, the reference points of each geometric primitive in the reference contour are first determined, and the topological relationship between the reference points is used as the topological relationship of each geometric primitive.

[0211] S1005: Determine whether the termination condition is met based on the loss function.

[0212] The electronic device determines whether the termination condition is met based on the loss function. If it is, it means that the predicted topological relationship obtained by the topology reconstruction network 3 meets the requirements, and proceeds to step S1006; otherwise, it means that the predicted topological relationship obtained by the topology reconstruction network 3 does not meet the requirements, and proceeds to step S1007.

[0213] It is understood that in some embodiments, the electronic device can determine that the termination condition is met when all loss functions converge or when all loss function values ​​are less than the corresponding preset loss function values.

[0214] In other embodiments, if the electronic device determines multiple loss functions in step S1005, it can determine that the termination condition is met if the total loss function obtained by the weighted sum of the multiple loss functions converges or is less than a preset total loss function value. For example, if the loss function includes cross-entropy loss L... CEL Consistency loss of geometric attributes and relations L C Compared with supervision, the loss L SCL In this case, the total loss function can be expressed as λ³L CEL +λ4L c +λ5L SCL Where λ3 represents the cross-entropy loss L CEL The weights and λ4 represent the consistency loss between geometric attributes and relations. C The weights and λ5 represent the supervised contrast loss L. SCL The weights λ3, λ4, and λ5 can be preset by the developers.

[0215] S1006: Store network parameters to obtain topology reconstruction network 3.

[0216] Once the electronic device determines that the termination condition is met, it stores the network parameters of the topology reconstruction network 3 to obtain the topology reconstruction network.

[0217] S1007: Adjust network parameters and proceed with the next round of training.

[0218] If the electronic device determines that the termination condition is not met, it adjusts the network parameters of the topology reconstruction network 3 and proceeds to the next round of training.

[0219] Through the aforementioned Figures 3 to 11The training process shown in the embodiment involves training the network parameters of neural network model 0 and, based on these parameters, inferring from the predicted images in the predicted image set of the target region to obtain vectorized maps of each predicted image. Specifically, the initial shape of the map elements in the predicted image can be obtained first using shape initialization network 1, then the initial shape can be adjusted using shape regression network 2 to obtain a more accurate regression shape, then the topological relationship between the geometric primitives in the regression shape can be obtained using topology reconstruction network 3, and finally the geometric primitives in the regression shape can be connected through post-processing module 4 to obtain a vectorized map of the map elements.

[0220] Therefore, reference Figure 12 The training process of neural network model 0 and the process of using the trained neural network model 0 to infer images can be asymmetrical. For map elements represented by polygons, since polygons are more complex than polylines, the regression shape is used as the input to train topology reconstruction network 3 during training. For map elements represented by polylines, since polylines are relatively simple, the initial shape of the polyline is used as the input to train topology reconstruction network 3. Because the initial shape is less accurate than the regression shape, topology reconstruction network 3 can still obtain correct prediction results even when the input data accuracy is low. This improves the noise resistance and stability of topology reconstruction network 3.

[0221] The following describes the process of generating a vector map using the aforementioned trained neural network model 0.

[0222] Figure 13 According to some embodiments of this application, a schematic flowchart of a map generation method is shown. The execution entity of this process is an electronic device, such as... Figure 13 As shown, the process includes the following steps.

[0223] S1301: Use shape initialization network 1 to obtain the initial shape of map elements in the predicted image.

[0224] The electronic device inputs the predicted image into the shape initialization network 1, extracts image features from the predicted image using the semantic segmentation network 11, then uses the mask generation network 12 to obtain the contour masks of map elements in the predicted image, then uses the edge extraction network 13 to extract the mask edges of the contour masks, and finally uses the shape generation network 14 to simplify the mask edges to obtain the initial shape of the map primitives. For example, refer to the aforementioned... Figure 2 After inputting image IM2 into shape initialization network 1, the initial shapes of houses and roads in image IM2 can be obtained.

[0225] S1302: Use shape regression network 2 to infer the initial shape and obtain the regression shape of the initial shape and the orientation data of the geometric primitives.

[0226] The electronic device inputs the initial shape of the map elements into the shape regression network 2, uses the pooling network 21 and the feature encoding network 22 to obtain the regression encoding features of the geometric primitives in the initial shape, then uses the direction generation network 23 to obtain the direction data of the geometric primitives, and uses the shape adjustment network 24 to adjust the initial shape to obtain a more accurate and more regular regression shape.

[0227] S1303: Use topology reconstruction network 3 to obtain the topological relationships between geometric primitives in the regressed shape.

[0228] The electronic device inputs the regressed shape into the topology reconstruction network 3, uses the pooling network 31 and the feature encoding network 32 to obtain the inference encoding features of each geometric primitive in the regressed shape, and then uses the relation inference network 33 to obtain the topological relationships between the geometric primitives in the regressed shape. For example, after inputting the regressed shape of the road in the aforementioned image IM2 into the topology reconstruction network 3, the following can be obtained: Figure 11 The topological relationships are shown.

[0229] S1304: Based on the regression shape, the topological relationship between geometric primitives, and the orientation data of geometric primitives, a vector map is obtained using post-processing module 4.

[0230] The electronic device uses post-processing module 4 to obtain a vector map based on the regression shape, the topological relationship between geometric primitives, and the orientation data of geometric primitives.

[0231] In some embodiments, when the regression shape is a polygon, the post-processing module 4 can first rotate each line segment in the regression shape to be the same as the direction data of the line segment, for example, referring to... Figure 14 The directions of line segments S1S2 and S2S3 in the regression shape are inconsistent with the direction (horizontal) generated by the aforementioned shape regression network 2. The post-processing module can rotate line segment S1S2 clockwise to horizontal and line segment S2S3 counterclockwise to horizontal.

[0232] Then, post-processing module 4 connects the line segments that are no longer connected due to the adjustment of the line segment direction (that is, connects the endpoint of each line segment to the point of the line segment adjacent to its endpoint that is closest to the endpoint), resulting in a closed polygon. For example, refer to... Figure 14 After rotating line segments S1S2 and S2S3 to the horizontal, line segments S1S2', S2S3' and S3S4 are no longer connected. The post-processing module 4 connects the endpoint S2' of line segment S1S2' to the endpoint S2 of the adjacent line segment S2S3', and connects the endpoint S3' of line segment S2S3' to the endpoint S3 of the adjacent line segment S3S4.

[0233] Finally, post-processing module 4 can delete line segments in the obtained closed polygon whose length is less than a preset side length threshold. When deleting a line segment, it can determine whether the two lines connecting the two ends are parallel or collinear. If they are parallel or collinear, the two line segments are merged into one line segment; otherwise, the two line segments are extended until they intersect. In this way, the simplicity of the output polygon can be adjusted by setting different preset side length thresholds. For example, refer to... Figure 14 If the lengths of line segments S2'S2, S3S3', S6S7, and S9S10 are less than the preset side length threshold, then line segments S2'S2, S3S3', S6S7, and S9S10 can be deleted. Since line segments S1S2', S2S3', and S3S4 have the same direction, they can be merged into a single line segment S1S4. Since line segments S5S6 and S7S8 are neither parallel nor collinear, and line segments S8S9 and S10S11 are not parallel, then line segment S7S8 is extended to intersect with line segment S5S6 to obtain line segment S6S8, and line segment S10S11 is extended to intersect with line segment S8S9 to obtain line segment S9S11, resulting in a vector map with a regular shape.

[0234] In some embodiments, when the regression shape is a polyline, the post-processing module 4 can connect the points with connections based on the points in the regression shape and the topological relationships between the points to obtain a vector polyline.

[0235] The method provided in this application, since the neural network model 0 is obtained based on the geometric features of map elements in the target area, can generate a more accurate vector map based on the remote sensing image of the target area. Furthermore, for different target areas, by marking the reference contours of map elements in the remote sensing images of those different target areas and retraining the neural network model 0, the trained neural network model 0 can be used to obtain vector maps of those different target areas based on the remote sensing images, without the need for complex heuristic rule settings and parameter adjustments. In large-scale map construction scenarios, such as vectorizing regions including multiple areas, cities, or countries, this method can improve the efficiency of vector map generation while ensuring the accuracy of the vector map.

[0236] To further verify the accuracy of the map generation method provided in this application, it was validated using remote sensing images from a publicly available dataset.

[0237] First, based on the dataset in the open-source dataset CrowdAI, the vectorization results of the map generation method provided in this application are compared with the performance of the current state-of-the-art (SOTA) algorithm with higher accuracy. The results are shown in Table 1.

[0238] Table 1. Test results on the CrowdAI dataset

[0239] method Average tangent angle error SOTA algorithm 31.9° This application 26.7°

[0240] As shown in Table 1, the SOTA algorithm has an average maximum tangent angle error of 31.9° for the vectorized house data, while the map generation method provided in this application has an average maximum tangent angle error of 26.7°, representing a 16.3% improvement. The average maximum tangent angle error refers to the average error of the tangent angle between a line in the vector map obtained from different remote sensing images and its corresponding reference line. The lower this error value, the higher the accuracy of the model's house vectorization. For example, assuming a model is used to vectorize N6 remote sensing images, and the maximum direction difference between each line segment in each remote sensing image and its corresponding reference line segment is dtan(i), then the average maximum tangent angle error of the model for these N6 remote sensing images can be denoted as:

[0241] further, Figure 15 According to some embodiments of this application, a schematic diagram of the result of vectorizing houses in a partial remote sensing image using a neural network model 0 is shown. Figure 15 It can be seen that the contour mask of the map elements in the image obtained by the semantic segmentation network 11 differs greatly from the actual contour of the house. However, the shape regression network 2 obtains a regression shape with a high similarity to the actual contour of the house, and thus the house in the vector map obtained also has a high similarity to the actual shape of the house.

[0242] In addition, the vectorization results of the road generated by the map generation method provided in this application embodiment were compared with the current high-precision Sat2Graph algorithm based on the open-source dataset SpaceNet3_Road. The results are shown in Table 2.

[0243] Table 2 shows the test results on the SpaceNet3_Road dataset.

[0244] method Model size Topological similarity Average path length similarity Sat2Graph algorithm 200M 80.97 64.43 This application 100M 86.63 67.67

[0245] As shown in Table 2, the neural network model of this application occupies less space than the Sat2Graph algorithm, and the resulting vector map has higher topology similarity and average path length similarity (APLS) compared to the reference vector map. Topology similarity refers to the similarity between the topology of the vector road network obtained by the model and the reference vector road network; a higher score indicates higher accuracy of the vector road network obtained by the model. APLS indicates the similarity between the lines in the vector road network obtained by the model and the lines in the reference vector road network; a higher score indicates higher accuracy of the vector road network obtained by the model.

[0246] further, Figure 16 According to some embodiments of this application, a schematic diagram of the result of vectorizing roads in remote sensing images using a neural network model 0 is shown. Figure 16 It can be seen that the direction of the points in the polyline obtained by neural network model 0 is consistent with the direction of the reference road.

[0247] Figure 17A and Figure 17B According to some embodiments of this application, schematic diagrams illustrating the reconstruction effect of roads in partially complex remote sensing images using neural network model 0 are shown. Figure 17A and Figure 17B As can be seen, the vector road network obtained by the map generation method provided in this application has a high degree of overlap with the road center in the remote sensing map, indicating that the obtained vector map has high accuracy.

[0248] It is understood that the use of remote sensing images to describe the technical solutions of this application in the foregoing embodiments is only an example. The technical solutions of the embodiments of this application can also be applied to vectorizing map elements in any other image (such as photos, aerial images, etc.) that includes map elements.

[0249] Furthermore, this application also provides a map generation apparatus for implementing the map generation methods provided in the foregoing embodiments.

[0250] Specifically, Figure 18 According to some embodiments of this application, a schematic diagram of the result of a map generation apparatus 200 is shown. For example... Figure 18 As shown, the map generation device 200 includes: a data acquisition unit 201, an initial shape generation unit 202, a shape regression unit 203, a topology reconstruction unit 204, and a post-processing unit 205.

[0251] The data acquisition unit 201 is used to acquire an image of a certain area, the image including map elements, wherein the map elements are elements in the image to be converted into a vector map.

[0252] The initial shape generation unit 202 is used to infer the image using a first model (such as the aforementioned shape initialization network 1) to obtain a first geometric figure corresponding to the map element. The first geometric figure includes geometric primitives. For details, please refer to the relevant description of step S1301 above, which will not be repeated here.

[0253] The shape regression unit 203 is used to input a second model (e.g., the aforementioned shape regression network 2) based on a first geometric shape to obtain the orientation of each geometric primitive, and to obtain a second geometric shape corresponding to a map element based on the first geometric shape. The second geometric shape includes the same geometric primitives as the first geometric shape, but the arrangement of the geometric primitives in the second geometric shape is different from that in the first geometric shape. For details, please refer to the relevant description of step S1302 above, which will not be repeated here.

[0254] Topology reconstruction unit 204 is used to obtain the topological relationships between geometric primitives based on the orientation of the geometric primitives and the second geometric figure using a third model (such as the aforementioned topology reconstruction network 3). For details, please refer to the relevant description of step S1303 above, which will not be repeated here.

[0255] The post-processing unit 205 obtains a vector map corresponding to the image based on the topological relationships between the geometric primitives, the orientation of each geometric primitive, and the second geometric figure. For example, in some embodiments, the post-processing unit 205 can be used to perform the relevant operations of the aforementioned post-processing module 4. For details, please refer to the relevant description of the aforementioned step S1304, which will not be repeated here.

[0256] Understandable. Figure 18 The structure of the map generation device 200 shown is only schematic. In other embodiments, the map generation device 200 may include more or fewer units, and some units may be merged or split. This is not limited here.

[0257] It is understood that in the above embodiments, the electronic device used for training the neural network model 0 or the electronic device used for inference using the neural network model 0 can be any electronic device capable of training or inference using the neural network model, including but not limited to laptop computers, desktop computers, tablet computers, servers, etc., and is not limited here. The following uses electronic device 100 as an example to illustrate the structure of the electronic device used for training the neural network model 0 or the electronic device used for inference using the neural network model 0. Specifically, Figure 19According to some embodiments of this application, a schematic diagram of an electronic device 100 for executing embodiments of this application is shown. The electronic device 100 may include one or more processors 101, system memory 102, non-volatile memory (NVM) 103, input / output (I / O) devices 104, communication interfaces 105, and system control logic 106 for coupling the processor 101, system memory 102, non-volatile memory 103, input / output (I / O) devices 104, and communication interfaces 105. Wherein:

[0258] Processor 101 may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. In some embodiments, processor 101 may be used to execute relevant instructions for training the aforementioned neural network model O or for performing inference on remote sensing images using the trained neural network model O.

[0259] Specifically, in some embodiments, the NPU can be used to run relevant instructions of the neural network model 0 to perform semantic segmentation of images, generate contour masks of map elements, generate mask contours of contour masks, generate initial / regressed shapes of map elements, generate directional data / topological relationships of geometric primitives, etc.

[0260] System memory 102 is volatile memory, such as random-access memory (RAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc. System memory is used for temporary storage of data and / or instructions. For example, in some embodiments, system memory 102 can be used to temporarily store network parameters of neural network model 0, sample image sets, intermediate data during the training or inference process of neural network model 0, and to store vector maps, etc.

[0261] The non-volatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), a solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 103 may also be a removable storage medium, such as a secure digital (SD) memory card. In other embodiments, the non-volatile memory 103 may be used to permanently store network parameters of the neural network model 0, sample image sets, intermediate data during the training or inference process of the neural network model 0, and vector maps, etc.

[0262] Specifically, system memory 102 and / or non-volatile memory 103 may include a copy of instruction 107. When executed by at least one of processors 101, instruction 107 causes electronic device 100 to train all or at least a portion of neural network model 0, or to perform inference using neural network model 0, by means of the methods provided in embodiments of this application.

[0263] Input / output (I / O) device 104 may include a user interface that enables a user to interact with electronic device 100, such as selecting or inputting a set of sample images, marking map elements in the set of sample images, etc.

[0264] Network interface 105 may include a transceiver for providing a wired or wireless communication interface for electronic device 100, thereby enabling it to communicate with any other suitable device via one or more networks. In some embodiments, electronic device 100 may establish communication connections with other electronic devices through network interface 105 to obtain sample image sets, prediction image sets, etc., from other electronic devices.

[0265] System control logic 106 may include any suitable interface controller to provide any suitable interface for other modules of electronic device 100. For example, in some embodiments, system control logic 106 may include one or more memory controllers to provide an interface for processor 101 to connect to system memory 102 and non-volatile memory 103. In other embodiments, system control logic 106 may include at least one Peripheral Component Interconnect (PCI) controller to provide an interface for processor 101 to connect via the PCI bus to devices / devices / modules (e.g., graphics cards, sound cards, etc.) connected to electronic device 100 via PCI interfaces.

[0266] In some embodiments, at least one of the processors 101 may be packaged together with the logic of one or more controllers for system control logic 106 to form a system-in-package (SiP). In other embodiments, at least one of the processors 101 may also be integrated on the same chip with the logic of one or more controllers for system control logic 106 to form a system-on-chip (SoC).

[0267] It is understood that electronic device 100 can be any electronic device capable of training deep learning models, including but not limited to laptops, desktop computers, tablets, servers, etc., without any limitation.

[0268] It is understood that the structure of the electronic device 100 shown in the embodiments of this application does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown, or combine some components, or split some components, or have different component arrangements. The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0269] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0270] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as a digital signal processor (DSP), microcontroller, application-specific integrated circuit (ASIC), or microprocessor.

[0271] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0272] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other propagation signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0273] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0274] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0275] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0276] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A map generation method, applied to electronic devices, characterized in that, The method comprises: acquiring an image of a certain area, the image comprising map elements, wherein the map elements are elements in the image to be converted into a vector map; using a first model to infer the image to obtain a first geometric figure corresponding to the map elements, the first geometric figure comprising geometric primitives; inputting a second model based on the first geometric figure to obtain the directions of the geometric primitives, and obtaining a second geometric figure corresponding to the map elements based on the first geometric figure, the second geometric figure comprising the same geometric primitives as the first geometric figure, and the arrangement of the geometric primitives in the second geometric figure being different from the arrangement of the geometric primitives in the first geometric figure; using a third model, obtaining the topological relationships between the geometric primitives based on the directions of the geometric primitives and the second geometric figure; obtaining a vector map corresponding to the image based on the topological relationships between the geometric primitives, the directions of the geometric primitives, and the second geometric figure.

2. The method of claim 1, wherein, At least one of the first model, the second model, and the third model is trained based on the geometric features of the map elements of the certain area.

3. The method of claim 1, wherein, In the case where the geometric primitives are line segments, the second geometric figure further comprises the connection order of the geometric primitives; and The obtaining of the vector map corresponding to the image based on the topological relationships between the geometric primitives, the directions of the geometric primitives, and the second geometric figure comprises: adjusting the direction of a first geometric primitive in the second geometric figure to be the same as the direction corresponding to the first geometric primitive, wherein the direction of the first geometric primitive in the second geometric figure is different from the direction corresponding to the first geometric primitive; connecting the first geometric primitive and a second geometric primitive to obtain a polygon corresponding to the second geometric figure, wherein the connection order of the second geometric primitive is adjacent to the first geometric primitive.

4. The method of claim 3, wherein, The polygon corresponding to the second geometric figure comprises sequentially connected first, second, and third line segments; and the obtaining of the vector map corresponding to the image based on the topological relationships between the geometric primitives, the directions of the geometric primitives, and the second geometric figure further comprises: in the case where the length of the second line segment is less than a preset side length threshold, deleting the second line segment; and in the case where the topological relationship between the first line segment and the third line segment is collinear or parallel, merging the first line segment and the second line segment into one line segment; in the case where the topological relationship between the first line segment and the third line segment is not collinear or parallel, extending the first line segment and / or the third line segment so that the first line segment and the third line segment intersect.

5. The method of claim 1, wherein, In the case where the geometric primitives are points, the obtaining of the vector map corresponding to the image based on the topological relationships between the geometric primitives, the directions of the geometric primitives, and the second geometric figure comprises: connecting points with a topological relationship of connection to obtain a corresponding vectorized polyline.

6. The method of claim 1, wherein, The first model is used to infer the image to obtain the first geometric figure corresponding to the map element, comprising: performing semantic segmentation on the image to obtain a contour mask of the map element, the contour mask being used to indicate a region in the image where the map element is located; extracting a mask edge of the contour mask; simplifying the mask edge to obtain the first geometric figure.

7. The method of claim 1, wherein, The map element comprises at least one of a house, a road, a lake, an ocean, a river, a forest, and a desert; and The first geometric figure corresponding to the house, the lake, the ocean, the forest, and the desert is a polygon; The first geometric figure corresponding to the road and the river is a polyline.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: training the first model by: obtaining sample data, the sample data comprising a sample image set of the certain region, and a reference contour corresponding to a map element in each sample image in the sample image set; using the first model to obtain image features of each sample image, and based on the image features, obtaining a contour mask of the map element in each sample image, the contour mask indicating a region of the map element in the corresponding sample image; based on the contour mask, obtaining a first predicted geometric figure corresponding to the map element in each sample image; training the first model based on a first loss function value and a second loss function value, wherein the first loss function is used to indicate the accuracy of the contour mask, and the second loss function is used to indicate the similarity between the first predicted geometric figure and the reference contour.

9. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: training the second model by: obtaining sample data, the sample data comprising a reference contour corresponding to a map element in each sample image in a sample image set of the certain region, a reference direction corresponding to each geometric primitive in the reference contour, and a third geometric figure corresponding to the map element in each sample image obtained by using the first model; using the second model to obtain a second predicted geometric figure corresponding to each map element in each sample image, a predicted direction of a geometric primitive in the third geometric figure, wherein the second predicted geometric figure comprises the same geometric primitives as the third geometric figure, and the arrangement of the geometric primitives in the second predicted geometric figure is different from that in the third geometric figure; training the second model based on a third loss function and a fourth loss function, wherein the third loss function is used to indicate the similarity between the predicted direction of the geometric primitive in the third geometric figure and the corresponding reference direction, and the fourth loss function is used to indicate the similarity between the second predicted geometric figure and the corresponding reference contour.

10. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: obtaining sample data, the sample data comprising a reference topological relationship between geometric primitives in a reference contour corresponding to a map element in each sample image in a sample image set of the certain region, and a fourth geometric figure corresponding to the map element in each sample image obtained by using the first model, and a direction of a geometric primitive in the fourth geometric figure; The third model is used to determine hidden space features of each geometric primitive in the fourth geometric graph, and a predicted topological relationship between the geometric primitives in the fourth geometric graph is determined based on the hidden space features; The third model is trained based on a fifth loss function and a sixth loss function, where the fifth loss function is used to indicate a matching degree between the predicted topological relationship and a corresponding reference topological relationship, and the sixth loss function is used to indicate a similarity of the hidden space features between geometric primitives in parallel, collinear or connected states.

11. A model training method applied to an electronic device, comprising: The method comprises: obtaining sample data, wherein the sample data comprises reference contours of map elements in each sample image in a sample image set of a certain region, fifth geometric graphs and sixth geometric graphs of the map elements, directions of geometric primitives in the fifth geometric graphs, and image features of the geometric primitives in the fifth geometric graphs, wherein the image features of the geometric primitives in the fifth geometric graphs are generated when the fifth geometric graphs of the map elements are inferred by using a fourth model, the similarity between the fifth geometric graphs and the corresponding reference contours is lower than the similarity between the sixth geometric graphs and the corresponding reference contours, and the fifth geometric graphs and the sixth geometric graphs have the same geometric primitives; inputting the fifth geometric graphs or the sixth geometric graphs, the image features of the geometric primitives in the fifth geometric graphs, and the directions of the geometric primitives in the fifth geometric graphs into a fifth model with first network parameters to obtain hidden space features corresponding to each geometric primitive, and inferring a predicted topological relationship between each geometric primitive based on the hidden space features corresponding to each geometric primitive; determining a seventh loss function and an eighth loss function based on the predicted topological relationship between the geometric primitives in the fifth geometric graphs and a corresponding reference topological relationship, wherein the reference topological relationship can be determined based on the reference contours of the map elements in each sample image, the seventh loss function is used to indicate a matching degree between the predicted topological relationship and the corresponding reference topological relationship, and the eighth loss function is used to indicate a similarity of the hidden space features between geometric primitives in parallel, collinear or connected states; saving the fifth model with the first network parameters when the seventh loss function and the eighth loss function satisfy a termination condition; adjusting the network parameters of the fifth model to second network parameters when the seventh loss function and the eighth loss function do not satisfy the termination condition, and performing a next round of training.

12. The method of claim 11, wherein, when the geometric primitives in the fifth geometric graphs are line segments, the seventh loss function and the eighth loss function satisfy the termination condition in the following manner: determining a direction relationship between the geometric primitives based on the directions of the geometric primitives in the fifth geometric graphs, and determining a reference direction relationship corresponding to the topological relationship, and determining a ninth loss function, wherein the ninth loss function is used to indicate a consistency between the predicted topological relationship and the direction. In a case where the seventh loss function, the eighth loss function, and the ninth loss function all converge, or the seventh loss function, the eighth loss function, and the ninth loss function are all less than corresponding preset loss function values, or a total loss function converges, or the total loss function is less than a corresponding preset total loss function value, it is determined that a termination condition is met, where the total loss function includes a weighted sum of the seventh loss function, the eighth loss function, and the ninth loss function.

13. The method of claim 11, wherein, The obtaining of the hidden space feature corresponding to each geometric primitive based on the fifth geometric figure or the sixth geometric figure, the image feature of the geometric primitive in the fifth geometric figure, or the direction of the geometric primitive in the fifth geometric figure includes: In a case where the geometric primitive of the fifth geometric figure is a point, the obtaining of the hidden space feature corresponding to each geometric primitive based on the fifth geometric figure, the image feature of the geometric primitive in the fifth geometric figure, or the direction of the geometric primitive in the fifth geometric figure includes: In a case where the geometric primitive of the fifth geometric figure is a line segment, the obtaining of the hidden space feature corresponding to each geometric primitive based on the sixth geometric figure, the image feature of the geometric primitive in the fifth geometric figure, or the direction of the geometric primitive in the fifth geometric figure includes.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed by an electronic device, cause the electronic device to implement the method of any one of claims 1-13.

15. An electronic device, comprising: The computer-readable storage medium includes instructions that, when executed by an electronic device, cause the electronic device to implement the method of any one of claims 1-13. The memory is configured to store instructions for execution by one or more processors of the electronic device. The processor is one of the processors of the electronic device and is configured to execute the instructions stored in the memory to implement the method of any one of claims 1-13. The computer program / instructions, when executed by a processor, implement the method of any one of claims 1-13.

16. A computer program product, characterised in that, ​

Images