A road layer height estimation method, device and system based on trajectory and satellite image

Abstract

The application provides a road layer height estimation method, device and system based on trajectories and satellite images, and the content comprises the following steps: S1, road cover area extraction, generating road topology data according to trajectories, extracting the area where the road cover exists, and downloading the corresponding satellite image of the area; S2, performing semantic segmentation on the downloaded satellite image by using a semantic segmentation model, and outputting semantic information and direction information; S3, judging whether the corresponding roads at the cover are parallel based on the semantic information and the direction information; S4, correcting and determining the layer height of the parallel cover road and the non-parallel cover road respectively; and S5, outputting the smoothed current road information after smoothing the layer height of other road points and attaching the road information with the relative layer height. The complexity of the layer height estimation method is greatly simplified, the data which is low in cost and easy to obtain is used, the problem of road layer height estimation is solved, and the whole process is automated.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of map generation, and particularly relates to a road layer height estimation method, device and system based on trajectories and satellite images. BACKGROUND

[0002] In the field of map production, there are often multiple layers of roads in the same area. Although the map does not need to give the accurate road height, it needs to express the upper and lower layer relationship of different roads. However, for the roads generated by low-precision trajectories due to cost reasons, it is difficult to determine the upper and lower relationship of the two intersecting roads from the source data information. The following method is generally used in the prior art to solve the technical problem:

[0003] RTK (Real-time kinematic, real-time dynamic) carrier phase difference technology is used for GPS positioning, so that the positioning accuracy is higher and the height error is smaller. The trajectory obtained by RTK GPS positioning can obtain a more accurate upper and lower layer relationship. However, this method needs to purchase services, and each vehicle needs to pay extra, which causes the cost of map production to rise.

[0004] Through a visual method, the slope of the road is estimated through the image of the front-view camera of the vehicle. This method has certain limitations. First, it needs to be obvious, and it is difficult to distinguish a gentle slope. Second, it can only be identified when it is just uphill, and the time window is small. Overall, the recall rate is not high, and it is difficult to be applied on a large scale.

[0005] The image taken by the unmanned aerial vehicle is used for multi-view geometric three-dimensional reconstruction, also known as oblique photography. This method can obtain relatively rich and accurate three-dimensional road data, and is a relatively new mapping technology in recent years. The defect of this scheme is that additional data acquisition cost is needed.

[0006] The above technical solutions are either high in cost or not good enough in effect. Especially in the field of automated mapping with rapid iteration of technology and rapid decline in cost, a road layer height estimation method with low cost and high efficiency and reliability needs to be considered. SUMMARY

[0007] The technical problem to be solved by the present application is to provide a road layer height estimation method, device and system based on trajectories and satellite images to solve at least one of the above problems existing in the prior art.

[0008] To achieve the above purpose, one or more embodiments of the present application provide a road layer height estimation method based on trajectories and satellite images, which includes the following steps:

[0009] S1, road cover area extraction, generating road topology data according to trajectories, extracting areas where road cover exists, and downloading satellite images corresponding to the areas;

[0010] S2, performing semantic segmentation on the downloaded satellite image by using a semantic segmentation model to output semantic information and direction information;

[0011] S3, determining whether the roads corresponding to the cover points are parallel based on the semantic information and the direction information;

[0012] S4, correcting and determining the layer height of the parallel cover point road and the non-parallel cover point road respectively;

[0013] S5, smoothing the layer height of other road points and outputting the current road information after the road information with relative layer height is attached.

[0014] Based on the above technical solutions of the application, the following improvements can be made:

[0015] Optionally, the semantic segmentation model used in the step S2 includes an FPN feature pyramid model framework and an EfficientNetb1 backbone network.

[0016] Optionally, when determining whether the roads corresponding to the cover points are parallel in the step S3, the layer height of all current roads is set to zero.

[0017] Optionally, the step S4 includes:

[0018] If the cover point roads are parallel, the topological relationship of the roads is introduced, the points where the two roads parallel to each other and covering each other change in different topological relationships are defined as topological change points, the topological relationship outside the topological change points is determined to locate the road at a high layer, the road layer height from the cover point to the topological change point is improved, and then the step S5 is jumped to.

[0019] If the cover point roads are not parallel, the digital direction of the uppermost road is inferred based on the semantic segmentation result of the satellite image, the digital direction of the uppermost road is compared with the road direction generated by the trajectory, the uppermost road is determined, the road layer height of the cover point is improved, and then the step S5 is jumped to.

[0020] According to the second aspect of the application, a road layer height estimation device based on a trajectory and a satellite image is provided, the device includes a memory, a processor and a communication circuit, the memory and the communication circuit are coupled with the processor, the communication circuit is connected with the processor, and the communication circuit interacts with an external terminal device under the control of the processor; the memory includes a local storage and stores a computer program; the processor is used to run the computer program to execute the road layer height estimation method based on the trajectory and the satellite image.

[0021] According to a third aspect of the present application, there is provided a road layer height estimation system based on trajectories and satellite images, which adopts the road layer height estimation method based on trajectories and satellite images according to any one of the above.

[0022] The present application has the advantage that it provides a road layer height estimation method, device and system based on trajectories and satellite images, which can estimate road layer height by means of low-cost low-precision trajectories and public satellite image data, thereby saving the cost of data collection, generating road topology data without layer height by using low-precision trajectories, extracting a road cover, estimating the direction of the uppermost road layer by using satellite images, and thereby layering the upper and lower roads. The low-precision trajectories have the disadvantage of inaccurate height, and the satellite images have the disadvantage of only one perspective and cannot estimate accurate depth information. In the present application, the height of the low-precision trajectories is not used, and the satellite images are not used to estimate depth and slope, but only the direction of the road cover estimated by the satellite images is needed. In this way, the disadvantages of the two types of data are avoided, and the advantages are complementary, which greatly simplifies the complexity of the layer height estimation method, uses easily available low-cost data, solves the problem of road layer height estimation, and automates the entire process. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 FIG. 1 is a flowchart of a road layer height estimation method based on trajectories and satellite images according to an embodiment of the present application.

[0024] Figure 2 FIG. 2 is a schematic diagram of road cover points and satellite images for a road layer height estimation method based on trajectories and satellite images according to an embodiment of the present application.

[0025] Figure 3 FIG. 3 is a direction diagram of a satellite image cover area for a road layer height estimation method based on trajectories and satellite images according to an embodiment of the present application.

[0026] Figure 4 FIG. 4 is a schematic diagram of a topology change point of upper and lower parallel roads for a road layer height estimation method based on trajectories and satellite images according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to specific embodiments and the accompanying drawings.

[0028] It should be noted that the technical terms or scientific terms used in one or more embodiments of the present application should be understood as the general meaning understood by those skilled in the art to which the present disclosure belongs, unless otherwise defined. The terms "first", "second", and the like used in one or more embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. The terms "include" or "contain" and the like mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connected" or "connected" and the like are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "up", "down", "left", "right" and the like are only used to represent relative positional relationships, and when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0029] One or more embodiments of the present application provide a road layer height estimation method based on trajectories and satellite images, as shown in Figure 1 and Figure 2 The method comprises the following steps:

[0030] S1, road cover area extraction, generating road topology data according to trajectories, extracting areas where road cover exists, and downloading satellite images corresponding to the areas;

[0031] Specifically, road topology data can be generated according to low-precision trajectories, but the elevation data of low-precision trajectories is unstable, and cannot distinguish the upper and lower layer relationship of cross-cover roads. In this step, areas where road cover exists are extracted, and corresponding satellite images are downloaded according to these areas. Figure 2 In the figure, the line with an arrow is the road generated by the trajectory, and the arrow represents the road direction; the rectangular frame represents the road cover area.

[0032] S2, performing semantic segmentation on the downloaded satellite images using a semantic segmentation model to output semantic information and direction information; the semantic segmentation model used in step S2 includes an FPN feature pyramid model framework and an EfficientNetb1 backbone network.

[0033] Specifically, for the semantic segmentation model of satellite images, FPN (Feature Pyramid Networks) is used as the model framework to obtain high-resolution and strong semantic features. The backbone network uses EfficientNet b1 version, which well balances the three dimensions of depth, width and resolution, and uniformly scales the three dimensions through a set of fixed scaling coefficients. The input of the semantic segmentation model is a satellite image, and the output is semantic information and direction information. The semantic information is divided into three categories: road surface, road edge, and invalid information. The direction information represents the digitized direction of the road, which is divided into six types at an interval of 30 degrees from 0 to 180 degrees. The digitized direction of the road is different from the driving direction of the road. The digitized direction represents 0 to 180 degrees, and if the driving direction of the road is completely opposite, the digitized direction is completely the same. The satellite image does not need to extract the driving direction of the road, but only needs to extract the rough digitized direction, because the trajectory direction can supplement the real road form direction. It can be understood that semantic_map is a semantic map with three channels representing the probabilities of road surface, road edge, and invalid information; direction_map is a direction map with six channels representing the probabilities of six digitized directions; aerial_image is the original satellite image with three channels of rgb; FPN_EfficientNetb1 is a semantic segmentation model based on the feature pyramid architecture with the backbone network of EfficientNet. The loss function used in the training of the model is cross entropy (CrossEntropy); please refer to Figure 3 , Figure 3 is the direction map (direction_map) of the satellite image overlay area. The direction of the arrow is the estimated digitized direction of the road.

[0034] S3, determining whether the roads corresponding to the overlay are parallel based on the semantic information and the direction information; when determining whether the roads corresponding to the overlay are parallel in step S3, the height of all roads is set to zero.

[0035] S4, correcting and determining the height of the roads at the parallel overlay and the roads at the non-parallel overlay, respectively; step S4 includes:

[0036] If the roads at the overlay are parallel, introduce the road topological relationship, define the point where the two roads parallel to each other and overlaid and having different topological relationship changes as a topological change point, determine the road at the high layer based on the topological change point and the topological relationship outside the topological change point, and then jump to step S5.

[0037] It can be understood that, due to the too long area of the road covered by the cover, it is impossible to determine which road is on the top by the covered area of the satellite image alone, so the road topological relationship is introduced, and the point where the two roads parallel to each other and covered by each other and having different topological relationship changes becomes a topological change point. As shown in FIG. 8, the bottom map is a satellite image, the lines represent the roads generated by the trajectory, the two parallel white lines on the left and downward represent two roads parallel to each other, and the position of the circle is a topological change point of one of the roads. Figure 4 The topological change point of the two roads covered by each other and parallel to each other is that the road on the right has a higher layer height, which can be determined by the road edge of the satellite image, because the edge of the road on the higher layer is closed and has no topological connection to the outside. After the road on the higher layer is determined at the topological change point, the layer height of the entire parallel road connected by the road on the higher layer is increased by 1.

[0038] If the roads at the covered position are not parallel, the digital direction of the road on the top layer is determined based on the semantic segmentation result of the satellite image, the digital direction is compared with the direction of the road generated by the trajectory, then the road on the top layer is determined, the layer height of the covered point is increased, and then the step S5 is jumped to.

[0039] It can be understood that, the digital direction of the road on the top layer can be determined from the segmentation result of the satellite image, and the direction of the road generated by the trajectory is compared, so that it can be determined which road is on the top layer, and the layer height of the road on the top layer at the covered point is increased by 1.

[0040] S5, after smoothing the layer height of the other road points, the road information with relative layer height is output, and the current road information after smoothing is output.

[0041] It can be understood that, in the embodiment, after the first two steps, the layer height of some roads has been reset, and the road points between different layer heights need to be smoothed to change the relative height gently. Thus, the road with relative layer height has been completely generated.

[0042] In another embodiment, a device for estimating road layer height based on a trajectory and a satellite image is provided, the device comprising a memory, a processor and a communication circuit, the memory and the communication circuit being coupled with the processor, wherein the communication circuit is connected with the processor, and the communication circuit interacts with an external terminal device under the control of the processor; the memory comprises a local storage and stores a computer program; the processor is configured to run the computer program to execute the road layer height estimation method based on the trajectory and the satellite image.

[0043] In another embodiment, a trajectory and satellite image based road layer height estimation system is also provided, which employs any of the trajectory and satellite image based road layer height estimation methods described above.

[0044] Those skilled in the art will appreciate that embodiments of the present application can be supplied as methods, systems, or computer program products. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can be embodied in the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) having computer usable program code embodied therein.

[0045] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the present application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0046] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0047] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0048] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims include all such modifications and variations as fall within the scope of the present application.

[0049] It is apparent that those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. It is therefore intended that the present application cover all such changes and modifications that are within its scope.

Claims

1. A method for estimating road level based on trajectory and satellite imagery, characterized in that, It comprises the following steps: S1, road cover area extraction, generating road topology data according to the track, extracting the area where the road cover exists, and downloading the satellite image corresponding to the area; S2, using a semantic segmentation model to perform semantic segmentation on the downloaded satellite image, and outputting semantic information and direction information; S3, judging whether the roads corresponding to the cover are parallel based on the semantic information and the direction information; S4, correcting and determining the layer height of the parallel cover road and the non-parallel cover road respectively; S5, smoothing the layer height of other road points and outputting the smoothed current road information with the relative layer height of the road information; The step S4 comprises: If the cover road is parallel, the road topology relationship is introduced, the points where the two parallel and cover roads change in different topology relationships are defined as topology change points, the topology change points are used to determine the road located in the upper layer, the layer height of the road from the cover point to the topology change point is improved, and then the step S5 is jumped to; If the cover road is not parallel, the digital direction of the uppermost road is inferred based on the semantic segmentation result of the satellite image, the digital direction is compared with the road direction generated by the track, the uppermost road is determined, the layer height of the cover point is improved, and then the step S5 is jumped to.

2. The method of claim 1, wherein the road layer height is estimated based on the trajectory and satellite imagery. The semantic segmentation model used in the step S2 comprises an FPN feature pyramid model framework and an EfficientNetb1 backbone network.

3. The method of claim 1, wherein the road layer height is estimated based on the trajectory and satellite imagery. When the step S3 judges whether the roads corresponding to the cover are parallel, it is set that the layer height of all current roads is zero.

4. A device for estimating a road level based on a trajectory and satellite imagery, characterized by, The device comprises a memory, a processor and a communication circuit, the memory and the communication circuit are coupled with the processor, the communication circuit is connected with the processor, and the communication circuit interacts with the external terminal equipment under the control of the processor; the memory comprises a local storage and stores a computer program; the processor is used to run the computer program to execute the road layer height estimation method based on the track and the satellite image according to any one of claims 1-3.

5. A trajectory and satellite imagery based road layer height estimation system, characterized in that, The system adopts the road layer height estimation method based on the track and the satellite image according to any one of claims 1-3.

Images