A method and system for estimating divergence and merging points on expressways based on trajectory images
Through the deep learning object detection method, traditional map data is used to generate training samples and image encoding to identify high-speed road divergence points, solving the problem of difficulty in identifying high-speed road divergence points, and achieving automated recognition and efficient recognition effects.
Patent Information
- Application Number
- CN202111410440.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-24
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2041-11-24
AI Technical Summary
In the field of high-precision map construction, it is difficult to identify divergent confluence points on high-speed roads. Traditional methods rely on the perception results that are not reliable enough, and long-distance trajectory observation is required to judge divergent confluence points, and there is a lack of effective automated identification methods.
Using deep learning object detection method, by extracting high-speed roads and divergence points from traditional map data, generating training samples in image format, using CenterNet model and ResNet50 backbone network, automatic labeling and data enhancement, trajectory image encoding is realized, and then recognition is achieved. Key points of divergence and convergence.
It realizes the automatic identification of divergence points on highway roads, reduces the efficiency of manual labeling, improves the accuracy and efficiency of identification, and solves the problem of long-distance observation.
Smart Images
Figure CN114140551B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of high-precision map production, and in particular to a method and system for estimating divergence and merging points of expressways based on trajectory images. Background Art
[0002] In the field of high-precision map construction, it is very difficult to identify highway intersections (i.e. divergence and confluence points). Generally, they are identified with the help of clear perception results (such as road boundaries and guide strips), but perception results are not always reliable. Although a large amount of trajectory data does not clearly mark divergence and confluence points, it is easy to determine whether the trajectory passes through the divergence and confluence points under human visual observation. How to transfer human experience to a computer model that can be used is an issue that we urgently need to solve.
[0003] In traditional trajectory processing methods, the angle change distribution of trajectory points is often used to judge intersections. However, at the divergence and merging points on highways, the angle change is small, and it is necessary to observe the trajectory over a longer distance to judge the divergence and merging points. Therefore, new trajectory encoding methods and new recognition methods are needed to solve the problem. Summary of the invention
[0004] In view of the technical problems existing in the prior art, the present invention provides a method and system for estimating diverging and merging points of highways based on trajectory images, uses a deep learning target detection method to identify diverging and merging key points, models human experience, and solves the problem of automatically identifying diverging and merging points of highways through trajectories.
[0005] According to a first aspect of the present invention, a method for estimating a highway divergence and merging point based on a trajectory image is provided, comprising: step 1, extracting highways and highway divergence and merging points from traditional map data, automatically annotating key points and key areas, and generating training samples in image format;
[0006] The key point is the selected diverging and merging point of the expressway, and the key area is a road area determined according to a preset rule where it can be determined that the diverging and merging point of the expressway exists;
[0007] Step 2, obtaining a target detection model based on the training sample training, wherein the output of the target detection model is the position of the key point;
[0008] Step 3: Convert the trajectory data into an image and input it into the trained target detection model. The position output by the target detection model is the inferred position of the divergence and confluence point of the highway.
[0009] Based on the above technical solution, the present invention can also make the following improvements.
[0010] Optionally, in step 1, all highway nodes with 1 in and 2 out or 2 in and 1 out are extracted from the traditional map data and selected as the highway divergence and merging points to be selected.
[0011] Optionally, the process of determining the key areas corresponding to the key points in step 1 includes:
[0012] The key point is extended in the direction of the three roads connected to it. After the distance of the road shape point in the direction of 2 entering or 2 exiting roads reaches the set distance, the road shape point is taken as a convex polygon and expanded to the outside of the road edge line by the set distance to obtain the key area corresponding to the key point.
[0013] Optionally, after the key points and key areas are automatically labeled in step 1, data enhancement is performed on the labeled original data; the data enhancement includes:
[0014] After performing random rotation and translation on each of the key points, multiple enhanced data of an original data are generated respectively, and the original data and the enhanced data are rasterized to generate training samples in image format.
[0015] Optionally, the rasterization process includes: using an HSV model to encode the image, the color tone of the HSV model is represented by the direction of the road shape point, the saturation takes a uniform value, and the brightness is represented by the elevation relative to the road shape point or takes a uniform value.
[0016] Optionally, the target detection model in step 2 is a CenterNet model, and its backbone network uses ResNet50;
[0017] During the training process of step 2, the training samples are divided into a training set, a validation set and a test set, and k-fold cross validation is adopted to take the best AP performance on the test set as the training result.
[0018] Optionally, in step 3, the trajectory data is encoded into an image using HSV;
[0019] After the target detection model outputs the position of the key point, the image coordinates are converted to vector map coordinates to obtain the position output by the target detection model.
[0020] According to a second aspect of the present invention, there is provided a highway divergence and merging point estimation system based on trajectory images, comprising: a training sample generation module, a training module and an estimation module;
[0021] The training sample generation module is used to extract expressways and expressway divergence and confluence points from traditional map data, automatically annotate key points and key areas, and then generate training samples in image format;
[0022] The key point is the selected diverging and merging point of the expressway, and the key area is a road area determined according to a preset rule where it can be determined that the diverging and merging point of the expressway exists;
[0023] The training module is used to obtain a target detection model based on the training samples, and the output of the target detection model is the position of the key point;
[0024] The inference module is used to convert the trajectory data into an image and input it into the trained target detection model. The position output by the target detection model is the inferred position of the divergence and confluence point of the highway.
[0025] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the processor is configured to implement the steps of a method for estimating a diverging and merging point of a highway based on a trajectory image when executing a computer management program stored in the memory.
[0026] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer management program stored thereon, which, when executed by a processor, implements the steps of a method for estimating a diverging and merging point of a highway based on a trajectory image.
[0027] The present invention provides a method, system, electronic device and storage medium for estimating diverging and merging points of a highway based on trajectory images. The method uses images to encode trajectories to solve the problem of long-distance observation; uses a deep learning target detection method to identify diverging and merging key points to solve the problem of modeling human experience; introduces the use of traditional maps to automatically generate annotated data sets, so that the entire annotation training process and inference process are automated, solving the problem of manual annotation efficiency; and solves the problem of automatically identifying diverging and merging points of highways through trajectories. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 A flowchart of an embodiment of an automatic labeling and training process in a method for estimating diverging and merging points on a highway based on a trajectory image provided by the present invention;
[0029] Figure 2 A flowchart of an embodiment of a prediction process in a method for predicting a diverging and merging point on a highway based on a trajectory image provided by the present invention;
[0030] Figure 3 A schematic diagram of automatically generated annotation data provided by an embodiment of the present invention;
[0031] Figure 4 A schematic diagram of key points and key areas provided for an embodiment of the present invention;
[0032] Figure 5 A schematic diagram of a key point estimation result provided by an embodiment of the present invention;
[0033] Figure 6 A structural block diagram of a highway divergence and merging point estimation based on trajectory images provided by the present invention;
[0034] Figure 7 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;
[0035] Figure 8 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION
[0036] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.
[0037] The present invention provides a method for estimating a divergence and merging point of a highway based on a trajectory image, comprising:
[0038] Step 1: Extract highways and highway divergence and confluence points from traditional map data, automatically annotate key points and key areas, and generate training samples in image format.
[0039] The key point is a selected expressway divergence and merging point, and the key area is a road area determined according to preset rules where it can be determined that there is an expressway divergence and merging point.
[0040] Step 2: A target detection model is obtained based on training samples, and the output of the target detection model is the position of the key point.
[0041] Step 3: Convert the trajectory data into images and input them into the trained target detection model. The output position of the target detection model is the inferred position of the divergence and confluence point of the highway.
[0042] The present invention provides a method for estimating diverging and merging points of highways based on trajectory images, uses a deep learning target detection method to identify diverging and merging key points, models human experience, and solves the problem of automatically identifying diverging and merging points of highways through trajectories.
[0043] Example 1
[0044] Embodiment 1 of the present invention is an embodiment of a method for estimating a diverging and merging point of a highway based on a trajectory image provided by the present invention, such as Figure 1 FIG. 1 is a flow chart of an embodiment of an automatic labeling and training process in a method for estimating diverging and merging points on a highway based on a trajectory image provided by the present invention, as shown in FIG. Figure 2FIG. 1 is a flow chart of an embodiment of a method for estimating a diverging and merging point on a highway based on a trajectory image provided by the present invention. Figure 1 and Figure 2 The rectangular boxes on the left are the various processes. The labeling and training processes have been framed separately. The cylindrical boxes on the right are the input and output data of the corresponding processes.
[0045] Combination Figure 1 and Figure 2 It can be seen that the embodiment of the method for estimating the divergence and merging point of a highway includes:
[0046] Step 1: Extract highways and highway divergence and confluence points from traditional map data, automatically annotate key points and key areas, and generate training samples in image format.
[0047] The key point is a selected expressway divergence and merging point, and the key area is a road area determined according to preset rules where it can be determined that there is an expressway divergence and merging point.
[0048] In a possible implementation manner, in step 1, all highway nodes with 1-in and 2-out or 2-in and 1-out roads are extracted from traditional map data and selected as the highway divergence and merging points to be selected.
[0049] Traditional maps refer to maps that only contain road data. The traditional map data itself can distinguish road levels such as highways, and the roads are connected by nodes. Nodes and roads (edges) form a graph structure. All highway nodes that meet the requirements of 1-in 2-out and 2-in 1-out are candidates for highway divergence and confluence.
[0050] In a possible embodiment, the marking process in step 1 is automatic marking, such as Figure 3 FIG. 1 is a schematic diagram of automatically generated annotation data provided by an embodiment of the present invention. Figure 3 Roads of different colors represent different directions, small squares represent the marked key point targets, and the upper left and lower right corners of the small squares are the border corners of the box.
[0051] The process of determining the key areas corresponding to each key point includes:
[0052] The key point is extended in the direction of the three roads connected to it. When the distance between the road shape points in the direction of the two entrances or two exits reaches the set distance (for example, 30 meters), it can be used as a road in the key area. The road shape points are taken as convex polygons, and the set distance (for example, 10 meters) is extended to the outside of the road edge to generate a buffer zone to obtain the key area corresponding to the key point. Figure 4The figure shows a schematic diagram of key points and key areas provided in an embodiment of the present invention. The solid dots in the figure are key points, and the outer polygonal border of the solid dots is the key area. The position of the key points can be observed based on the trajectory within the key area.
[0053] The length of roads in key areas is often more than 100 meters. The purpose of setting key areas is to ensure that key points are recognized when the key areas are observed (that is, when they are completely within the image range) during data enhancement; when the key areas are not completely within the image range, key points should not be recognized. This ensures that the final automatically generated label is consistent with the results observed by the human eye.
[0054] In a possible implementation manner, after the key points and key areas are automatically labeled in step 1, data enhancement is performed on the labeled original data; the data enhancement includes:
[0055] After randomly rotating and translating each key point, multiple enhanced data of the original data are generated respectively, and the original data and the enhanced data are rasterized to generate training samples in image format.
[0056] Data augmentation is to enhance the generalization of data. Since the original data is vector data, we can directly perform rotation and translation based on the vector data, and finally rasterize it into an image.
[0057] Timing of real-time enhancement: The reason why real-time enhancement is not performed during training is that the process of obtaining the vector data required for rotation is complicated. The training process often runs for a long time on a separate GPU server. The program and data should not be too complicated. Therefore, the data enhancement process is brought forward for practical operation considerations.
[0058] In the specific implementation, the generation of enhanced data includes: taking the surrounding roads of each key point, the range is 1280x1280 meters, the scale is 2.5 meters / pixel, and the image size is 512x512 pixels. Each rotation and translation amount is a random value, and 200 enhanced images can be generated for each key point.
[0059] In a possible implementation manner, the process of rasterization includes: using the HSV model for image encoding, the HSV (Hue, Saturation, Value) model is a hexagonal pyramid model that uses hue (hue), saturation (saturation), and brightness (value) to represent color.
[0060] The hue of the HSV model is represented by an angle value, which is consistent with the direction of the road shape point. The road color in the same direction is consistent. The saturation is uniformly set to 1. The brightness is represented by the elevation relative to the road shape point. If there is no elevation, the value is uniform.
[0061] At this point, the HSV-encoded road is drawn, converted into an image and saved, and a label border is generated for the key points, with a border size of 8x8 pixels.
[0062] Step 2: A target detection model is obtained based on training samples, and the output of the target detection model is the position of the key point.
[0063] In a possible implementation manner, the target detection model in step 2 is a CenterNet model, and its backbone network uses ResNet50.
[0064] In the specific implementation, there are 70,000 highway key points generated according to step 1. Each key point can generate 200 enhanced images. Based on these data, deep model training for target detection can be performed.
[0065] The present invention adopts the CenterNet model for training, and its backbone network uses ResNet50. CenterNet is a single-stage key point detection model with relatively balanced performance and effect.
[0066] During the training process in step 2, the training samples are divided into a training set, a validation set, and a test set. K-fold cross validation is used, and the AP (average precision) with the best performance on the test set is taken as the training result.
[0067] During training, the 70,000 data sets are divided into three parts: training set, validation set and test set. K-fold cross validation is used, and k can be 11. The effects of the 11 model parameters in the validation set and the test set are compared, and the better result is selected after comprehensive comparison. Generally, the AP (average precision) with better performance on the test set is taken as the final training result.
[0068] Step 3: Convert the trajectory data into images and input them into the trained target detection model. The output position of the target detection model is the inferred position of the divergence and confluence point of the highway.
[0069] In a possible implementation manner, in step 3, the trajectory data is encoded into an image using HSV; and according to the image style during training, the vector trajectory is encoded into an image using the same encoding method.
[0070] Use the trained CenterNet model parameters to infer the locations of key points in the image.
[0071] After the target detection model outputs the position of the key point, the image coordinates are converted to vector map coordinates to obtain the position output by the target detection model.
[0072] In a possible implementation manner, the training data is roads, and the inference data is trajectories. The two are consistent most of the time. There are two types of inconsistencies, which can also be handled:
[0073] 1) For images with a large number of tracks, this problem can be solved by generating multiple parallel tracks when augmenting the data.
[0074] 2) For data enhancement that is not possible, the trajectory images can be directly added to the training set without changing the labels.
[0075] like Figure 5 FIG. 1 is a schematic diagram of a key point inference result provided by an embodiment of the present invention. Figure 5 The box in the middle is the inferred key point area. The text ms on the box indicates that the type is merge_split, and the number indicates the confidence value.
[0076] Example 2
[0077] Embodiment 2 provided by the present invention is an embodiment of a highway divergence and merging point estimation system based on trajectory images provided by the present invention. Figure 6 A structure diagram of a highway divergence and merging point estimation system based on trajectory images provided by an embodiment of the present invention, combined with Figure 6 It can be seen that the embodiment of the inference system includes: a training sample generation module, a training module and an inference module.
[0078] The training sample generation module is used to extract expressways and expressway divergence and confluence points from traditional map data, automatically annotate key points and key areas, and then generate training samples in image format.
[0079] The key point is a selected expressway divergence and merging point, and the key area is a road area determined according to preset rules where it can be determined that there is an expressway divergence and merging point.
[0080] The training module is used to obtain a target detection model based on training samples, and the output of the target detection model is the position of the key point.
[0081] The inference module is used to convert the trajectory data into images and input them into the trained target detection model. The position output by the target detection model is the inferred position of the divergence and confluence point of the highway.
[0082] It can be understood that the highway divergence and merging point estimation system based on trajectory images provided by the present invention corresponds to the highway divergence and merging point estimation method based on trajectory images provided by the aforementioned embodiments. The relevant technical features of the highway divergence and merging point estimation system based on trajectory images can refer to the relevant technical features of the highway divergence and merging point estimation method based on trajectory images, which will not be repeated here.
[0083] See also Figure 7 , Figure 7 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 7 As shown, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, the following steps are implemented: extracting highways and highway divergence and merging points from traditional map data, automatically annotating key points and key areas, and generating training samples in image format; the key points are selected highway divergence and merging points, and the key areas are road areas determined according to preset rules that can be used to determine the existence of highway divergence and merging points; obtaining a target detection model based on the training samples, and the output of the target detection model is the position of the key points; converting the trajectory data into images and inputting them into the trained target detection model, and the position output by the target detection model is the inferred position of the highway divergence and merging point.
[0084] See also Figure 8 , Figure 8 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 8 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, the following steps are implemented: extracting highways and highway divergence and merging points from traditional map data, automatically annotating key points and key areas, and generating training samples in image format; the key points are selected highway divergence and merging points, and the key areas are road areas determined according to preset rules that can be used to determine the existence of highway divergence and merging points; a target detection model is obtained based on training samples, and the output of the target detection model is the position of the key points; the trajectory data is converted into an image and input into the trained target detection model, and the position output by the target detection model is the inferred position of the highway divergence and merging point.
[0085] The embodiments of the present invention provide a method, system, electronic device and storage medium for estimating diverging and merging points of a highway based on trajectory images. The method encodes trajectories with images to solve the problem of long-distance observation; uses a deep learning target detection method to identify diverging and merging key points to solve the problem of modeling human experience; introduces the use of traditional maps to automatically generate annotated data sets, so that the entire annotation training process and inference process are automated, solving the problem of manual annotation efficiency; and solves the problem of automatically identifying diverging and merging points of highways through trajectories.
[0086] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0087] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0088] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0089] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0091] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0092] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for estimating divergence and merging points on a highway based on trajectory images, characterized in that: The inference method includes: Step 1: extract highways and highway divergence and confluence points from traditional map data, automatically annotate key points and key areas, and generate training samples in image format; The key point is the selected diverging and merging point of the expressway, and the key area is a road area determined according to a preset rule where it can be determined that the diverging and merging point of the expressway exists; Step 2, obtaining a target detection model based on the training sample training, wherein the output of the target detection model is the position of the key point; Step 3, converting the trajectory data into an image and inputting it into the trained target detection model, wherein the position output by the target detection model is the inferred position of the divergence and confluence point of the highway; In the step 1, all highway nodes with 1-in and 2-out or 2-in and 1-out roads are extracted from the traditional map data and selected as the highway divergence and confluence points to be selected; The process of determining the key areas corresponding to the key points in step 1 includes: The key point is extended in the direction of the three roads connected to it. When the distance to the road shape point in the direction of 2 entering or 2 exiting roads reaches the set distance, the road shape point is taken as a convex polygon and expanded to the outside of the road edge by the set distance to obtain the key area corresponding to the key point.
2. The inference method according to claim 1, characterized in that: After the key points and key areas are automatically marked in step 1, data enhancement is performed on the marked original data; The data enhancement includes: After randomly rotating and translating each of the key points, multiple enhanced data of an original data are generated respectively, and the original data and the enhanced data are rasterized to generate training samples in image format.
3. The inference method according to claim 2, characterized in that: The process of rasterization includes: using the HSV model to encode the image, the color tone of the HSV model is represented by the direction of the road shape point, the saturation takes a uniform value, and the brightness is represented by the elevation relative to the road shape point or takes a uniform value.
4. The inference method according to claim 1, characterized in that: The target detection model in step 2 is a CenterNet model, and its backbone network uses ResNet50; During the training process of step 2, the training samples are divided into a training set, a validation set and a test set, and k-fold cross validation is adopted to take the best AP performance on the test set as the training result.
5. The inference method according to claim 1, characterized in that: In the step 3, the trajectory data is encoded into an image using HSV; After the target detection model outputs the position of the key point, the image coordinates are converted to vector map coordinates to obtain the position output by the target detection model.
6. A highway divergence and merging point estimation system based on trajectory images, characterized in that: include: Training sample generation module, training module and inference module; The training sample generation module is used to extract expressways and expressway divergence and confluence points from traditional map data, automatically annotate key points and key areas, and then generate training samples in image format; The key point is the selected diverging and merging point of the expressway, and the key area is a road area determined according to a preset rule where it can be determined that the diverging and merging point of the expressway exists; The training module is used to obtain a target detection model based on the training samples, and the output of the target detection model is the position of the key point; The inference module is used to convert the trajectory data into an image and input it into the trained target detection model, and the position output by the target detection model is the inferred position of the divergence and confluence point of the highway; The training sample generation module extracts all highway nodes with 1-in and 2-out or 2-in and 1-out roads from the traditional map data and selects them as the highway divergence and confluence points to be selected; The process of determining the key areas corresponding to the key points in the training sample generation module includes: The key point is extended in the direction of the three roads connected to it. When the distance to the road shape point in the direction of 2 entering or 2 exiting roads reaches the set distance, the road shape point is taken as a convex polygon and expanded to the outside of the road edge by the set distance to obtain the key area corresponding to the key point.
7. An electronic device, characterized in that: It includes a memory and a processor, and the processor is used to implement the steps of the highway divergence and merging point estimation method based on trajectory images as described in any one of claims 1 to 5 when executing the computer management program stored in the memory.
8. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by the processor, the steps of the method for estimating a diverging and merging point of a highway based on a trajectory image as described in any one of claims 1 to 5 are implemented.