A stop line extraction method and system integrating remote sensing image and trajectory data

Abstract

The embodiment of the present invention provides a stop line extraction method and system that integrates remote sensing images and trajectory data. Based on the rule that stop lines and zebra crossings appear in pairs at intersections, and the characteristics of zebra crossings are relatively obvious, it is first necessary to use machine learning to extract the geographical location of zebra crossings from remote sensing images; in order to more accurately obtain the length coverage of the stop line, the extracted stop line and its extension line are intersected with the surrounding driving trajectories, so that the length of the stop line is adjusted to the coverage of the surrounding same-direction trajectories, and the trajectories are used for further screening to filter out stop lines that are not covered by the same-direction trajectories. The positional relationship between the stop lines is calculated, and the stop lines that can form an intersection relationship with other stop lines around them are retained, which is the final stop line extraction result.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of high-precision map production, and in particular to a stop line extraction method and system that integrates remote sensing images and trajectory data. Background Art

[0002] With the development of society and science and technology, cars play an increasingly important role in people's daily lives. While cars bring convenience to people's lives, they also bring huge hidden dangers to people's safety, especially at road traffic intersections. The development of autonomous driving technology and unmanned driving has brought good news to solve this hidden danger.

[0003] At present, stop line detection technology based on Hough transform is often used in autonomous driving and unmanned driving. This technology uses preprocessing technology, image edge technology, and Hough transform technology in the collected grayscale image to detect the "straight line" in the image, and uses prior knowledge such as the angle of the straight line to determine whether there is a stop line. Because the Hough transform algorithm is very time-consuming, the overall algorithm time consumption is relatively high, which cannot meet the needs of current vehicle-mounted real-time systems.

[0004] Stop lines are one of the elements of crowdsourced high-precision maps and are also an indispensable element for intelligent driving. For the original trajectory data collected by crowdsourcing, in the absence of other elements such as lane lines and traffic lights, obtaining stop lines first is a crucial step to obtain the correct topology. Currently, most stop lines are derived from perception data, that is, the vehicle-side camera scans in real time and performs image recognition to obtain stop line data, which is costly and has a high misrecognition rate, and requires more post-processing before it can be put into use. Summary of the invention

[0005] The embodiments of the present invention provide a stop line extraction method and system that integrate remote sensing images and trajectory data. The method utilizes existing open source remote sensing image data and combines crowdsourced trajectory data to extract high-quality stop lines from remote sensing images, which can effectively reduce the acquisition cost of stop lines while having high extraction accuracy.

[0006] In a first aspect, an embodiment of the present invention provides a stop line extraction method for fusing remote sensing images and trajectory data, comprising:

[0007] Step S1, acquiring road image data, extracting zebra crossing pixel coordinate points based on the road image data, and clustering to form a zebra crossing outline;

[0008] Step S2, extracting the endpoints of the two long sides of the zebra crossing outline as the starting position and the ending position of the stop line, wherein the normal direction of the stop line is perpendicular to the centroid of the zebra crossing outline;

[0009] The starting position is moved to the midpoint of the corresponding long side, and the stop line to be screened is determined based on the starting position and the end position after the movement;

[0010] Step S3, performing an intersection operation on the stop line to be screened and the extension line of the stop line to be screened with the trajectory data of the same direction of travel, extending the stop line to cover all intersection points, and obtaining the stop line.

[0011] Preferably, the step S1 specifically includes:

[0012] Step S11, obtaining original trajectory data and remote sensing image data;

[0013] Step S12: constructing a track surface based on the track data, performing an intersection operation on the track surface and the remote sensing image data to obtain road image data of the area covered by the track surface;

[0014] Step S13: input the road image data into a pre-trained zebra crossing detection model, extract zebra crossing pixel points, and obtain the zebra crossing outline.

[0015] Preferably, the method further comprises:

[0016] Mark the zebra crossings on the road image data, obtain the true value, and generate the corresponding label mask image;

[0017] The road image data and the label mask are cut based on a sliding window to obtain a data set, the data set is rotated, flipped, Gaussian noise is added, and blur processing is performed to expand the data in the data set, and the data set is divided into a training set and a test set according to a preset ratio;

[0018] The u-net network is used as the training skeleton for neural network training. The training set data is classified into two categories to determine whether each pixel is background or zebra crossing. The model parameters with the smallest loss in the test set are saved to obtain the zebra crossing detection model.

[0019] As a preferred method, obtaining the zebra crossing outline specifically includes:

[0020] The coordinate information of zebra crossing pixel points is extracted and clustered to form multiple different zebra crossing areas; the minimum enclosing rectangle is calculated based on each zebra crossing area to obtain the zebra crossing outline.

[0021] As a preferred embodiment, it also includes:

[0022] Step S4, filtering the stop lines that are not covered by the same-direction trajectory data, calculating the positional relationship between the stop lines, and retaining the stop lines that exist in the front area and can form an intersection relationship with it.

[0023] Preferably, in step S2, the normal direction of each stop line to be screened is perpendicular to the centroid of the zebra crossing contour.

[0024] In a second aspect, an embodiment of the present invention provides a stop line extraction system integrating remote sensing images and trajectory data, comprising:

[0025] A zebra crossing extraction module, used to obtain road image data, extract zebra crossing pixel coordinate points based on the road image data, and cluster to form a zebra crossing outline;

[0026] A preliminary screening module, used to extract the endpoints of the two long sides of the zebra crossing outline as the starting position and the ending position of the stop line, wherein the normal direction of the stop line is perpendicular to the centroid of the zebra crossing outline;

[0027] The starting position is moved to the midpoint of the corresponding long side, and the stop line to be screened is determined based on the starting position and the end position after the movement;

[0028] The fine screening module is used to perform intersection operation on the stop line to be screened and the extension line of the stop line to be screened with the trajectory data of the same direction of travel, and extend the stop line to cover all intersection points to obtain the stop line.

[0029] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the stop line extraction method for fusing remote sensing images and trajectory data as described in the embodiment of the first aspect of the present invention are implemented.

[0030] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a stop line extraction method for fusing remote sensing images and trajectory data as described in an embodiment of the first aspect of the present invention.

[0031] The stop line extraction method and system provided by the embodiment of the present invention is based on the rule that the stop line and the zebra crossing appear in pairs at the intersection, and the characteristics of the zebra crossing are relatively obvious. Therefore, it is necessary to first use machine learning to extract the geographical location of the zebra crossing from the remote sensing image; using the existing open source remote sensing image data, combined with crowdsourcing trajectory data, to extract high-quality stop lines from the remote sensing image, it can effectively reduce the acquisition cost of the stop line, and at the same time have a high extraction accuracy. In order to more accurately obtain the length coverage of the stop line, the extracted stop line and its extension line are intersected with the surrounding driving trajectory, so that the length of the stop line is adjusted to the coverage of the surrounding same-direction trajectory. At the same time, the trajectory is used for further screening to filter out the stop lines that are not covered by the same-direction trajectory. Finally, the positional relationship between the stop lines is calculated, and the stop lines that can form an intersection relationship with other stop lines around them are retained, which is the final stop line extraction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A flowchart of a stop line extraction method for fusing remote sensing images and trajectory data provided by an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of a specific process of a stop line extraction method that integrates remote sensing images and trajectory data provided by an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Stop lines are one of the elements of crowdsourced high-precision maps and are also an indispensable element for intelligent driving. For the original trajectory data collected by crowdsourcing, in the absence of other elements such as lane lines and traffic lights, obtaining stop lines first is a crucial step to obtain the correct topology. Currently, most stop lines are derived from perception data, that is, the vehicle-side camera scans in real time and performs image recognition to obtain stop line data, which is costly and has a high misrecognition rate, and requires more post-processing before it can be put into use.

[0038] Therefore, the embodiments of the present invention provide a stop line extraction method and system that integrates remote sensing images and trajectory data, which can utilize existing open source remote sensing image data and combine crowdsourced trajectory data to extract high-quality stop lines from remote sensing images, effectively reducing the acquisition cost of stop lines while having high extraction accuracy. The following will be described and introduced through multiple embodiments.

[0039] Figure 1 A stop line extraction method for fusing remote sensing images and trajectory data is provided for an embodiment of the present invention, comprising:

[0040] Step S1, acquiring road image data, extracting zebra crossing pixel coordinate points based on the road image data, and clustering to form a zebra crossing outline;

[0041] Since the stop line at the intersection usually appears in pairs with the zebra crossing, and the features of the zebra crossing are relatively obvious, it is first necessary to extract the geographical location of the zebra crossing from the remote sensing image. In this embodiment, the step S1 specifically includes:

[0042] Step S11, obtaining original trajectory data and remote sensing image data; in order to improve efficiency, it is necessary to use the trajectory data to pre-process the image data, and only retain the remote sensing image (ie, the road surface) of the trajectory coverage area to filter out interference from other ground objects.

[0043] Step S12: construct a trajectory surface based on the trajectory data, perform an intersection operation on the trajectory surface and the remote sensing image data, and obtain road image data of the area covered by the trajectory surface; perform a 10m buffer analysis on the trajectory point data to form a trajectory surface composed of trajectory points; perform an intersection operation on the generated trajectory surface and the image data to obtain an image of the area covered by the trajectory surface, and obtain road image data.

[0044] Step S13, input the road image data into a pre-trained zebra crossing detection model, extract zebra crossing pixel points, and obtain the zebra crossing outline. After completing the zebra crossing recognition, extract the coordinate information of the zebra crossing pixel points, cluster to form a plurality of different zebra crossing instances, and calculate the minimum enclosing rectangle according to each instance to obtain the outline information of the zebra crossing.

[0045] Based on the above embodiment, as a preferred implementation, the method further includes:

[0046] Mark the zebra crossings on the road image data, obtain the true value, and generate the corresponding label mask image;

[0047] The road image data and label mask are cut based on a sliding window (256×256) to obtain a data set, and the data set is rotated, flipped, Gaussian noise is added, and blurred to expand the data in the data set, and divided into a training set and a test set according to a preset ratio (3:1); in order to reduce the cost of labeling, Gaussian filtering, image rotation, blurring, adding noise points, etc. are used to expand the training data, while enhancing the robustness of the training model.

[0048] The u-net network is used as the training skeleton for neural network training. The training set data is classified into two categories to determine whether each pixel is background or zebra crossing. The model parameters with the smallest loss in the test set are saved to obtain the zebra crossing detection model. During training, in order to reduce the impact of extremely unbalanced data distribution (small target and large background), dice_loss is used as the model loss function to optimize the model training effect. Through this model, zebra crossings can be identified from any remote sensing image.

[0049] Step S2, extracting the endpoints of the two long sides of the zebra crossing outline as the starting position and the ending position of the stop line, wherein the normal direction of the stop line is perpendicular to the centroid of the zebra crossing outline;

[0050] The starting position is moved to the midpoint of the corresponding long side, and the stop line to be screened is determined based on the starting position and the end position after the movement;

[0051] For each zebra crossing contour information, extract its long side as the stop line position to be screened, and at the same time, its normal direction is perpendicular to the centroid of the zebra crossing contour. Since the stop lines are distributed on one side of the zebra crossing, the length of the stop line is roughly adjusted according to this rule, that is, the starting point is moved along the direction of the stop line to the midpoint to obtain the stop line to be screened.

[0052] Step S3, intersecting the stop line to be screened and the extension of the stop line to be screened with the trajectory data of the same direction, extending the stop line to cover all intersections, and obtaining the stop line. In order to more accurately obtain the length coverage of the stop line, the extracted stop line and its extension are intersected with the surrounding driving trajectory, and the length of the stop line is adjusted to just cover the intersection, so that the length of the stop line is adjusted to the coverage of the surrounding same-direction trajectory.

[0053] Step S4, filter the stop lines without the same direction track data coverage, calculate the positional relationship between the stop lines, and retain the stop lines that can form an intersection relationship with them in the front area. Use the track for further screening, filter the stop lines without the same direction track coverage. Finally, calculate the positional relationship between the stop lines, retain the stop lines that can form an intersection relationship with other stop lines around them (angle difference is 90, 180, 270), which is the final stop line extraction result.

[0054] The embodiment of the present invention further provides a stop line extraction system for fusing remote sensing images and trajectory data, based on the stop line extraction method for fusing remote sensing images and trajectory data in the above embodiments, including:

[0055] A zebra crossing extraction module, used to obtain road image data, extract zebra crossing pixel coordinate points based on the road image data, and cluster to form a zebra crossing outline;

[0056] A preliminary screening module, used to extract the endpoints of the two long sides of the zebra crossing outline as the starting position and the ending position of the stop line, wherein the normal direction of the stop line is perpendicular to the centroid of the zebra crossing outline;

[0057] The starting position is moved to the midpoint of the corresponding long side, and the stop line to be screened is determined based on the starting position and the end position after the movement;

[0058] The fine screening module is used to perform intersection operation on the stop line to be screened and the extension line of the stop line to be screened with the trajectory data of the same direction of travel, and extend the stop line to cover all intersection points to obtain the stop line.

[0059] In one embodiment, the present invention also provides an electronic device, such as Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the steps of the stop line extraction method for fusing remote sensing images and trajectory data as described in the above embodiments. For example, it includes:

[0060] Step S1, acquiring road image data, extracting zebra crossing pixel coordinate points based on the road image data, and clustering to form a zebra crossing outline;

[0061] Step S2, extracting the endpoints of the two long sides of the zebra crossing outline as the starting position and the ending position of the stop line, wherein the normal direction of the stop line is perpendicular to the centroid of the zebra crossing outline;

[0062] The starting position is moved to the midpoint of the corresponding long side, and the stop line to be screened is determined based on the starting position and the end position after the movement;

[0063] Step S3, performing an intersection operation on the stop line to be screened and the extension line of the stop line to be screened with the trajectory data of the same direction of travel, extending the stop line to cover all intersection points, and obtaining the stop line.

[0064] In one embodiment, the present invention also provides a non-transitory computer-readable storage medium, which stores a computer program, which includes at least one code segment, and the at least one code segment can be executed by a main control device to control the main control device to implement the steps of the stop line extraction method for fusing remote sensing images and trajectory data as described in the above embodiments. For example, it includes:

[0065] Step S1, acquiring road image data, extracting zebra crossing pixel coordinate points based on the road image data, and clustering to form a zebra crossing outline;

[0066] Step S2, extracting the endpoints of the two long sides of the zebra crossing outline as the starting position and the ending position of the stop line, wherein the normal direction of the stop line is perpendicular to the centroid of the zebra crossing outline;

[0067] The starting position is moved to the midpoint of the corresponding long side, and the stop line to be screened is determined based on the starting position and the end position after the movement;

[0068] Step S3, performing an intersection operation on the stop line to be screened and the extension line of the stop line to be screened with the trajectory data of the same direction of travel, extending the stop line to cover all intersection points, and obtaining the stop line.

[0069] In one embodiment, the present application also provides a computer program, which, when executed by a main control device, is used to implement the above method embodiment.

[0070] The program may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor.

[0071] In one embodiment, the present application also provides a processor, which is used to implement the above method embodiment. The above processor can be a chip.

[0072] In summary, the stop line extraction method and system provided by the embodiment of the present invention, which integrates remote sensing images and trajectory data, is based on the rule that the stop line at the intersection appears in pairs with the zebra crossing, and the characteristics of the zebra crossing are relatively obvious. Therefore, it is necessary to first use machine learning to extract the geographical location of the zebra crossing from the remote sensing image; using the existing open source remote sensing image data, combined with crowdsourcing trajectory data, to extract high-quality stop lines from the remote sensing image, it can effectively reduce the acquisition cost of the stop line, and at the same time have a high extraction accuracy. In order to more accurately obtain the length coverage of the stop line, the extracted stop line and its extension line are intersected with the surrounding driving trajectory, so that the length of the stop line is adjusted to the coverage of the surrounding same-direction trajectory. At the same time, the trajectory is used for further screening to filter out the stop lines that are not covered by the same-direction trajectory. Finally, the positional relationship between the stop lines is calculated, and the stop lines that can form an intersection relationship with other stop lines around them are retained, which is the final stop line extraction result.

[0073] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0074] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0078] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0079] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A stop line extraction method integrating remote sensing images and trajectory data, characterized in that: include: Step S1, acquiring road image data, extracting zebra crossing pixel coordinate points based on the road image data, and clustering to form a zebra crossing outline; Step S2, extracting the endpoints of the long side of the zebra crossing outline as the starting position and the ending position of the stop line, wherein the normal direction of the stop line is perpendicular to the centroid of the zebra crossing outline; The starting position is moved to the midpoint of the corresponding long side, and the stop line to be screened is determined based on the starting position and the end position after the movement; Step S3, performing an intersection operation on the stop line to be screened and the extension line of the stop line to be screened with the trajectory data of the same direction of travel, extending the stop line to cover all intersection points, and obtaining the stop line.

2. The stop line extraction method of fusion remote sensing image and trajectory data according to claim 1, characterized in that: The step S1 specifically includes: Step S11, obtaining original trajectory data and remote sensing image data; Step S12: constructing a track surface based on the track data, performing an intersection operation on the track surface and the remote sensing image data to obtain road image data of the area covered by the track surface; Step S13: input the road image data into a pre-trained zebra crossing detection model, extract zebra crossing pixel points, and obtain the zebra crossing outline.

3. The stop line extraction method of fusion remote sensing image and trajectory data according to claim 2 is characterized in that: The method further comprises: Mark the zebra crossings on the road image data, obtain the true value, and generate the corresponding label mask image; The road image data and the label mask are cut based on a sliding window to obtain a data set, the data set is rotated, flipped, Gaussian noise is added, and blur processing is performed to expand the data in the data set, and the data set is divided into a training set and a test set according to a preset ratio; The u-net network is used as the training skeleton for neural network training. The training set data is classified into two categories to determine whether each pixel is background or zebra crossing. The model parameters with the smallest loss in the test set are saved to obtain the zebra crossing detection model.

4. The stop line extraction method of fusion remote sensing image and trajectory data according to claim 2, characterized in that: To obtain the zebra crossing outline, specifically including: The coordinate information of zebra crossing pixel points is extracted and clustered to form multiple different zebra crossing areas; the minimum enclosing rectangle is calculated based on each zebra crossing area to obtain the zebra crossing outline.

5. The stop line extraction method of fusion remote sensing image and trajectory data according to claim 1, characterized in that: Also includes: Step S4, filtering the stop lines that are not covered by the same-direction trajectory data, calculating the positional relationship between the stop lines, and retaining the stop lines that exist in the front area and can form an intersection relationship with it.

6. The stop line extraction method of fusion remote sensing image and trajectory data according to claim 1, characterized in that: In the step S2, the normal direction of each of the stop lines to be screened is perpendicular to the centroid of the zebra crossing contour.

7. A stop line extraction system integrating remote sensing images and trajectory data, characterized in that: include: A zebra crossing extraction module is used to obtain road image data, extract zebra crossing pixel coordinate points based on the road image data, and cluster to form a zebra crossing outline; A preliminary screening module, used to extract the endpoints of the long side of the zebra crossing outline as the starting position and the ending position of the stop line, wherein the normal direction of the stop line is perpendicular to the centroid of the zebra crossing outline; The starting position is moved to the midpoint of the corresponding long side, and the stop line to be screened is determined based on the starting position and the end position after the movement; The fine screening module is used to perform intersection operation on the stop line to be screened and the extension line of the stop line to be screened with the trajectory data of the same direction of travel, and extend the stop line to cover all intersection points to obtain the stop line.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the stop line extraction method for fusing remote sensing images and trajectory data as described in any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the stop line extraction method of fusing remote sensing images and trajectory data as described in any one of claims 1 to 6 are implemented.

Images