Road information recognition method and system based on vehicle-mounted point cloud and aerial photographs

Data obtained by vehicle-mounted lidar and drones, combined with edge recognition and density detection, solve the problem of inaccurate identification of street lights and signs in dense street trees, realize accurate identification of tree trunks, street lights and signs, and improve roads Applicability and efficiency of information identification.

CN118781507BActive Publication Date: 2024-11-22CHENGDU JINGKAI GEOGRAPHIC INFORMATION SURVEY & DESIGN INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411276326.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-11-22
Estimated Expiration
2044-09-12

AI Technical Summary

Technical Problem

In the area of ​​dense tree canopy in the streets, it is difficult to accurately identify street lights and signs, especially when they are mixed with the canopy, resulting in inaccurate identification of point cloud data.

Method used

Three-dimensional point cloud data is obtained through vehicle-mounted lidar and drone obtaining aerial video images, combining edge recognition and density detection algorithms to divide roads, canopy and building areas, project and classify point cloud data, and identify tree trunks, street lights and signs. Card.

Benefits of technology

It improves the accuracy and applicability of road information identification, and can quickly identify tree trunks, street lights and signs in various roads, enhancing the recognition efficiency of point cloud data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118781507B_ABST
    Figure CN118781507B_ABST
Patent Text Reader

Abstract

The present invention discloses a road information recognition method and system based on vehicle-mounted point cloud and aerial photographs, which is applied to the field of intelligent point cloud recognition technology. The method includes: obtaining three-dimensional point cloud data and aerial photograph images; obtaining edge data; identifying road point cloud data through road areas; dividing other point cloud data into tree crown point cloud data, rod point cloud data and building point cloud data; forming tree crown projection data; using road point cloud data, building point cloud data, tree point cloud data, street lamp point cloud data and sign point cloud data as recognized road information data. Through the above technical solution, the present invention expands the application of existing road point cloud recognition technology to various roads by means of data fusion, effectively improving the applicability, and at the same time identifying tree trunks, street lamps and signs by means of tree crown projection fusion, thereby facilitating the rapid and accurate recognition of road-related information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to intelligent point cloud recognition technology, and in particular to a road information recognition method and system based on vehicle-mounted point clouds and aerial photographs. Background Art

[0002] The vehicle point cloud is a data format generated by the vehicle laser radar system, which represents the three-dimensional structural information of the vehicle's surrounding environment. The vehicle point cloud is generated by the vehicle laser radar (LiDAR) emitting laser beams and receiving the signals reflected by these laser beams after encountering the target object, and then analyzing the return time, intensity and other parameters of these signals to calculate the relative distance between the target object and the vehicle, three-dimensional coordinates (x, y, z) and reflectivity information. This information is collected and integrated into a data set consisting of a large number of discrete three-dimensional coordinate points, which is then used to identify various objects.

[0003] In the prior art, a Chinese patent application numbered CN201911406215.2 discloses a method and device for extracting street lamp elements based on vehicle-mounted laser scanning data, which makes full use of the two-dimensional arc features and three-dimensional rod-shaped features of lamp posts, and improves the extraction accuracy of the skew and bending of lamp posts. In lamp head detection, according to the spatial morphological differences between the street lamp head point cloud and the traffic sign surface and the tree crown point cloud, the interference of other rod-shaped targets is effectively removed by using the coverage analysis and projection area analysis, and finally a complete and accurate street lamp point cloud is obtained; in view of the problem of uneven density distribution of the data obtained by the vehicle-mounted laser scanning system, an adaptive filtering algorithm based on scanning lines is designed, which adaptively determines the number of points in the moving window according to the point density of the center point of the window, overcomes the influence of a single threshold on the point cloud filtering effect, and improves the extraction efficiency. However, in actual recognition operations, in areas where the crowns of street trees are dense, street lamps and signboards may be mixed in the tree crowns, making it difficult to be accurately identified by point cloud data. Summary of the invention

[0004] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide a road information recognition method and system based on vehicle-mounted point clouds and aerial photographs.

[0005] In a first aspect, the present application provides a road information recognition method based on vehicle-mounted point clouds and aerial photographs, including:

[0006] Acquire three-dimensional point cloud data of a target road section by using a vehicle-mounted laser radar, and acquire aerial photographs of the target road section by using a drone;

[0007] Performing edge recognition on the aerial photograph image to obtain edge data, and dividing the internal area between the edge data into a road area, a tree crown area, and a building area;

[0008] Projecting the coordinates of the road area to the three-dimensional point cloud data, and identifying the road in the three-dimensional point cloud data through the road area to form road point cloud data;

[0009] Eliminating the road point cloud data from the three-dimensional point cloud data to form other point cloud data, and dividing the other point cloud data into tree crown point cloud data, rod point cloud data and building point cloud data by a density detection algorithm;

[0010] Verify the building point cloud data through the building area, project the tree crown point cloud data onto a preset reference plane and fuse the tree crown area to form tree crown projection data;

[0011] Dividing the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data and sign point cloud data according to the tree crown projection data, and integrating the tree trunk point cloud data and the corresponding tree crown point cloud data into a whole to form tree point cloud data;

[0012] The road point cloud data, building point cloud data, tree point cloud data, street lamp point cloud data and sign point cloud data are used as the recognized road information data.

[0013] Furthermore, the internal area between the edge data is divided into road area, tree crown area and building area including:

[0014] Calculate the slopes between multiple consecutive pixel points in the same non-closed edge data, and when the slope difference between different pixel points is less than a preset value, determine that the edge data is road edge data;

[0015] Acquire the inner area between two adjacent road edge data as the road area;

[0016] Get the pixel value of the inner area of ​​the closed edge data;

[0017] When the average value of the pixel values ​​matches the crown pixel value, the internal area is judged to be a crown area; when the average value of the pixel values ​​matches the building pixel value and the crown area is greater than a threshold, the internal area is judged to be a building area.

[0018] Furthermore, projecting the coordinates of the road area to the three-dimensional point cloud data, and identifying the road in the three-dimensional point cloud data through the road area to form the road point cloud data includes:

[0019] Taking the plane where the point with the lowest vertical coordinate in the three-dimensional point cloud data is located as the projection plane, and projecting the coordinates of the road area onto the projection plane to form a projection area;

[0020] The three-dimensional point cloud data is searched for a point having the same horizontal coordinate as the projection area and the closest vertical coordinate as the road point cloud data.

[0021] Further, verifying the building point cloud data through the building area includes:

[0022] Projecting the building point cloud data and the building area onto a preset reference plane;

[0023] If the degree of overlap between the projected building point cloud data and the building area is higher than a preset value, the building point cloud data is determined to be building point cloud data that correctly corresponds to the building.

[0024] Further, projecting the tree crown point cloud data onto a preset reference plane and fusing the tree crown area to form tree crown projection data includes:

[0025] Projecting the tree crown point cloud data onto a preset reference plane to form projection point cloud data, and projecting the tree crown area onto the reference plane to form projection area data;

[0026] The projection point cloud data is divided according to the edges in the projection area data to form the tree crown projection data.

[0027] Furthermore, dividing the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data and sign point cloud data according to the tree crown projection data includes:

[0028] Acquire the included angle between the axis of the rod-shaped point cloud data and the vertical angle, and mark the rod-shaped point cloud data as vertical rod-shaped point cloud data when the included angle is less than a preset angle, and mark the rod-shaped point cloud data as inclined tree trunk point cloud data when the included angle is greater than or equal to the preset angle;

[0029] Projecting the vertical rod-shaped point cloud data onto the reference plane to form rod-shaped projection data;

[0030] If the rod-shaped projection data is enveloped by the tree crown projection data, the rod-shaped projection data is used as suspected tree trunk projection data;

[0031] If the rod-shaped projection data is not enveloped by the tree crown projection data, the rod-shaped projection data is used as street lamp identification projection data, and the rod-shaped point cloud data corresponding to the street lamp identification projection data is divided into reference street lamp point cloud data and reference identification point cloud data;

[0032] The rod-shaped point cloud data corresponding to the suspected tree trunk projection data is divided into vertical tree trunk point cloud data, blocked street lamp point cloud data and blocked sign point cloud data by using the corresponding tree crown projection data, reference street lamp point cloud data and reference sign point cloud data;

[0033] The vertical tree trunk point cloud data and the inclined tree trunk point cloud data are used as the tree trunk point cloud data, the reference street lamp point cloud data and the blocked street lamp point cloud data are used as the street lamp point cloud data, and the reference marker point cloud data and the blocked marker point cloud data are used as the marker point cloud data.

[0034] Furthermore, the rod-shaped point cloud data corresponding to the suspected tree trunk projection data is divided into vertical tree trunk point cloud data, occluded street lamp point cloud data and occluded sign point cloud data through the corresponding tree crown projection data, reference street lamp point cloud data and reference sign point cloud data:

[0035] Calculating the center point of the tree crown projection data;

[0036] If the number of the suspected tree trunk projection data corresponding to the tree crown projection data is one, the rod-shaped point cloud data corresponding to the suspected tree trunk projection data is used as the vertical tree trunk point cloud data;

[0037] If the number of the suspected trunk projection data corresponding to the tree crown projection data is greater than one, the rod-shaped point cloud data corresponding to the suspected trunk projection data falling within the preset range of the center point is used as the vertical trunk point cloud data, and the rod-shaped point cloud data corresponding to the suspected trunk projection data not falling within the preset range of the center point is used as the suspected street lamp identification data;

[0038] The suspected street lamp sign data is identified as vertical tree trunk point cloud data, obstructed street lamp point cloud data and obstructed sign point cloud data through the point cloud data at the top of the reference street lamp point cloud data and the reference sign point cloud data.

[0039] Further, identifying the suspected street lamp sign data as vertical tree trunk point cloud data, obstructing street lamp point cloud data and obstructing sign point cloud data through the point cloud data at the top of the reference street lamp point cloud data and the reference sign point cloud data includes:

[0040] Acquire the point cloud data within the preset range at the top of the reference street lamp point cloud data as the street lamp top point cloud data, and acquire the point cloud data within the preset range at the top of the reference marker point cloud data as the marker top point cloud data;

[0041] The street lamp top point cloud data is projected as a whole to the top of the suspected street lamp identification data, and when the degree of similarity between the street lamp top point cloud data after projection and the tree crown point cloud data on the top of the suspected street lamp identification data reaches a preset value, the suspected street lamp identification data is used as the obstructed street lamp point cloud data;

[0042] Projecting the top point cloud data of the logo as a whole to the top of the suspected street light logo data, and when the degree of similarity between the top point cloud data of the logo after projection and the tree crown point cloud data at the top of the suspected street light logo data reaches a preset value, using the suspected street light logo data as the obstructed logo point cloud data;

[0043] The portion of the suspected street lamp identification data that is not identified as obstructed street lamp point cloud data and obstructed identification point cloud data is used as vertical tree trunk point cloud data.

[0044] In the second aspect, the present application also provides a road information recognition system based on vehicle-mounted point clouds and aerial photographs, including:

[0045] An acquisition unit is configured to acquire three-dimensional point cloud data of a target road section through a vehicle-mounted laser radar, and acquire an aerial photograph image of the target road section through a drone;

[0046] An edge detection unit is configured to perform edge recognition on the aerial photograph image to obtain edge data, and divide the internal area between the edge data into a road area, a tree crown area and a building area;

[0047] a projection unit configured to project the coordinates of the road area to the three-dimensional point cloud data, and identify the road in the three-dimensional point cloud data through the road area to form road point cloud data;

[0048] A density detection unit is configured to remove the road point cloud data from the three-dimensional point cloud data to form other point cloud data, and divide the other point cloud data into tree crown point cloud data, rod point cloud data and building point cloud data through a density detection algorithm;

[0049] A checking unit is configured to check the building point cloud data through the building area, project the tree crown point cloud data to a preset reference plane and fuse the tree crown area to form tree crown projection data;

[0050] a dividing unit configured to divide the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data and sign point cloud data according to the tree crown projection data, and integrate the tree trunk point cloud data and the corresponding tree crown point cloud data into a whole to form tree point cloud data;

[0051] The recognition unit is configured to use the road point cloud data, the building point cloud data, the tree point cloud data, the street lamp point cloud data and the sign point cloud data as the recognized road information data.

[0052] Furthermore, the verification unit is further configured to:

[0053] Projecting the tree crown point cloud data onto a preset reference plane to form projection point cloud data, and projecting the tree crown area onto the reference plane to form projection area data;

[0054] The projection point cloud data is divided according to the edges in the projection area data to form the tree crown projection data.

[0055] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0056] Through the above technical scheme, the present invention expands the application of existing road point cloud recognition technology to various roads by means of data fusion, effectively improving the applicability. At the same time, tree trunks, street lights and signs are identified by means of crown projection fusion, thereby facilitating rapid and accurate identification of road-related information. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0058] Figure 1 This is a schematic diagram of the steps of the method of the embodiment of the present application;

[0059] Figure 2 A schematic diagram of road recognition in an embodiment of the present application;

[0060] Figure 3 This is a schematic diagram of the difference in pixel values ​​between the tree crown area and the building area in the embodiment of the present application;

[0061] Figure 4 This is a schematic diagram of the projection filtering solution of an embodiment of the present application;

[0062] Figure 5 This is a schematic diagram of the tree crown point cloud projection of the embodiment of the present application;

[0063] Figure 6 This is a schematic diagram of the specific positions of the crown projection data and the suspected trunk projection data on the reference plane in the embodiment of the present application;

[0064] Figure 7 This is a schematic diagram of the street light point cloud identified in an embodiment of the present application. DETAILED DESCRIPTION

[0065] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the embodiment of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0066] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0067] Please refer to Figure 1 , which is a flow chart of a road information recognition method based on vehicle-mounted point clouds and aerial photographs provided in an embodiment of the present invention. Furthermore, the road information recognition method based on vehicle-mounted laser point clouds and aerial photographs may specifically include the contents described in the following steps S1 to S7.

[0068] S1: Acquire three-dimensional point cloud data of a target road section through a vehicle-mounted laser radar, and acquire an aerial photograph image of the target road section through a drone;

[0069] S2: performing edge recognition on the aerial photograph image to obtain edge data, and dividing the internal area between the edge data into a road area, a tree crown area and a building area;

[0070] S3: projecting the coordinates of the road area to the three-dimensional point cloud data, and identifying the road in the three-dimensional point cloud data through the road area to form road point cloud data;

[0071] S4: removing the road point cloud data from the three-dimensional point cloud data to form other point cloud data, and dividing the other point cloud data into tree crown point cloud data, rod point cloud data and building point cloud data by a density detection algorithm;

[0072] S5: Verifying the building point cloud data through the building area, projecting the tree crown point cloud data onto a preset reference plane and fusing the tree crown area to form tree crown projection data;

[0073] S6: dividing the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data and sign point cloud data according to the tree crown projection data, and integrating the tree trunk point cloud data and the corresponding tree crown point cloud data into a whole to form tree point cloud data;

[0074] S7: Use the road point cloud data, building point cloud data, tree point cloud data, street lamp point cloud data and sign point cloud data as the recognized road information data.

[0075] When implementing the embodiment of the present application, it is necessary to first obtain three-dimensional point cloud data and aerial photograph images. In order to improve the accuracy of subsequent detection, a simultaneous acquisition method is generally adopted, that is, aerial photograph images are taken by drone while obtaining three-dimensional point cloud data. When edge recognition is performed on aerial photograph images, edge recognition methods in the prior art can be used, such as comparing the color difference between adjacent pixels, etc. The identified edge data generally includes vehicle edges, tree crown edges, road edges, building edges, etc. In order to perform auxiliary recognition of point cloud data through these edge information, the areas corresponding to these edge information need to be divided into road areas, tree crown areas and building areas. The division method can be performed by color recognition or other means.

[0076] In the embodiment of the present application, since aerial photograph images are used for auxiliary identification, the road point cloud identification can be assisted by the road area during road point cloud identification. The advantage is that there is no need to use curbstones and other auxiliary identification methods to effectively improve the applicability of road identification. For some old roads, the road and the sidewalk are on the same horizontal plane, so the auxiliary identification through aerial photographs can effectively identify such roads. After the road identification is completed, other point cloud data can be identified with the identified road as the horizontal reference. During the identification process, the existing density detection algorithm can be used for identification, that is, the DBSCAN algorithm can be used for segmentation and identification, which is a relatively mature technology in the prior art and is not limited in the embodiment of the present application. Based on the DBSCAN algorithm, other point cloud data can be divided into crown point cloud data, rod point cloud data, and building point cloud data.

[0077] In an embodiment of the present application, the identified building point cloud data can be checked by the building area that has been obtained from the aerial photograph image. The specific checking method can be performed by projecting the building area. In order to divide the tree crown point cloud data more accurately, the tree crown point cloud data can be projected and then fused with the tree crown area. The specific fusion can be performed by dividing the trees by the projection of the tree crown point cloud data by the tree crown area. The main reason is that some denser trees are intertwined in the tree crown point cloud data, while from the aerial image, their tops have more obvious dividing lines. When the segmented tree crown projection data has been obtained, the rod point cloud data can be further divided into trunk point cloud data, street light point cloud data and sign point cloud data based on the positional relationship between the rod point cloud data and the tree crown projection data, wherein the street light point cloud data and the sign point cloud data can be divided by the difference in the top position point cloud. After completing these content divisions, the trunk point cloud data and the corresponding crown point cloud data can be merged to form tree point cloud data, and the final road point cloud data, building point cloud data, tree point cloud data, street lamp point cloud data and sign point cloud data can be used as the identified specific features to form the final identification information. The embodiment of the present application uses the above technical solution to expand the application of existing road point cloud recognition technology to various roads through data fusion, effectively improving the applicability. At the same time, the trunks, street lamps and signs are identified through the crown projection fusion method, which can facilitate the rapid and accurate identification of road-related information.

[0078] In a possible implementation, dividing the inner area between edge data into a road area, a tree crown area, and a building area includes:

[0079] Calculate the slopes between multiple consecutive pixel points in the same non-closed edge data, and when the slope difference between different pixel points is less than a preset value, determine that the edge data is road edge data;

[0080] Acquire the inner area between two adjacent road edge data as the road area;

[0081] Get the pixel value of the inner area of ​​the closed edge data;

[0082] When the average value of the pixel values ​​matches the crown pixel value, the internal area is judged to be a crown area; when the average value of the pixel values ​​matches the building pixel value and the crown area is greater than a threshold, the internal area is judged to be a building area.

[0083] When the embodiment of the present application is implemented, there is a clear distinction between different edge data. Road edge data is generally non-closed and has a gently changing slope, so the road edges can be screened out through this process, and the internal area between adjacent road edge data is used as the road area; for tree crowns and buildings, the color pixel value of the tree crown is generally higher than that of the building when grayscaled, so the building area and the tree crown area can be distinguished by the pixel value.

[0084] In a possible implementation, projecting the coordinates of the road area to the three-dimensional point cloud data, and identifying the road in the three-dimensional point cloud data through the road area to form the road point cloud data includes:

[0085] Taking the plane where the point with the lowest vertical coordinate in the three-dimensional point cloud data is located as the projection plane, and projecting the coordinates of the road area onto the projection plane to form a projection area;

[0086] The three-dimensional point cloud data is searched for a point having the same horizontal coordinate as the projection area and the closest vertical coordinate as the road point cloud data.

[0087] When the embodiment of the present application is implemented, a specific scheme is provided for identifying roads in three-dimensional point cloud data through road areas. In the prior art, the technical scheme for road point cloud identification mainly adopts the cloth filtering method. The filtering method adopted in the embodiment of the present application is actually an improved filtering scheme. Its essence is that the road area is already known, so it is only necessary to search for points with the same horizontal coordinates and the closest vertical coordinates in the projection area to accurately identify the road point cloud data. The filtering process is simpler, consumes less computing power, and can effectively reduce the noise of non-road point clouds.

[0088] In a possible implementation, verifying the building point cloud data through the building area includes:

[0089] Projecting the building point cloud data and the building area onto a preset reference plane;

[0090] If the degree of overlap between the projected building point cloud data and the building area is higher than a preset value, the building point cloud data is determined to be building point cloud data that correctly corresponds to the building.

[0091] When the embodiment of the present application is implemented, when checking the building point cloud, it can be judged by projection, wherein the building point cloud data and the building area need to be projected onto the same reference plane, and judged based on the degree of overlap. The degree of overlap can be judged by the proportion of the overlapping area to the building area. If it is greater than 95%, it is judged to be the building point cloud data that correctly corresponds to the building.

[0092] In a possible implementation, projecting the tree crown point cloud data onto a preset reference plane and fusing the tree crown area to form tree crown projection data includes:

[0093] Projecting the tree crown point cloud data onto a preset reference plane to form projection point cloud data, and projecting the tree crown area onto the reference plane to form projection area data;

[0094] The projection point cloud data is divided according to the edges in the projection area data to form the tree crown projection data.

[0095] When the embodiment of the present application is implemented, the purpose of the fusion is to split the crowns of adjacent trees, wherein the basis for the splitting is the projection area data formed by the projection of the crown area. The projection area data will divide the projection point cloud data into areas corresponding to different trees, thereby facilitating the subsequent identification of tree trunks.

[0096] In a possible implementation, dividing the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data, and sign point cloud data according to the tree crown projection data includes:

[0097] Acquire the included angle between the axis of the rod-shaped point cloud data and the vertical angle, and mark the rod-shaped point cloud data as vertical rod-shaped point cloud data when the included angle is less than a preset angle, and mark the rod-shaped point cloud data as inclined tree trunk point cloud data when the included angle is greater than or equal to the preset angle;

[0098] Projecting the vertical rod-shaped point cloud data onto the reference plane to form rod-shaped projection data;

[0099] If the rod-shaped projection data is enveloped by the tree crown projection data, the rod-shaped projection data is used as suspected tree trunk projection data;

[0100] If the rod-shaped projection data is not enveloped by the tree crown projection data, the rod-shaped projection data is used as street lamp identification projection data, and the rod-shaped point cloud data corresponding to the street lamp identification projection data is divided into reference street lamp point cloud data and reference identification point cloud data;

[0101] The rod-shaped point cloud data corresponding to the suspected tree trunk projection data is divided into vertical tree trunk point cloud data, blocked street lamp point cloud data and blocked sign point cloud data by using the corresponding tree crown projection data, reference street lamp point cloud data and reference sign point cloud data;

[0102] The vertical tree trunk point cloud data and the inclined tree trunk point cloud data are used as the tree trunk point cloud data, the reference street lamp point cloud data and the blocked street lamp point cloud data are used as the street lamp point cloud data, and the reference marker point cloud data and the blocked marker point cloud data are used as the marker point cloud data.

[0103] When the embodiment of the present application is implemented, when analyzing the rod-shaped point cloud data, the verticality of the axis of the rod-shaped point cloud data is first analyzed to reduce the data. Point cloud data with an inclination greater than a certain degree are generally tree trunks, while vertical rod-shaped point cloud data may exist in three situations: tree trunks, street lights, and signs. In order to classify and divide the vertical rod-shaped point cloud data, it is necessary to first project the vertical rod-shaped point cloud data into a reference plane, and obtain the direct positional relationship between the rod-shaped projection data and the crown projection data; in one case, the rod-shaped projection data is not within the range of the crown projection data, which means that it is definitely not a tree trunk, because the inclined trunks have been removed in advance, then it can be determined that these rod-shaped point cloud data are street lights or signs. At this time, the solution provided in the prior art such as CN201911406215.2 can be used to divide these rod-shaped point cloud data into reference street light point cloud data and reference sign point cloud data. The reason why they can be used as a reference is that these data are not blocked by the crown, so they can be used for further identification of other subsequent data. In another case, the rod projection data is within the range of the tree crown projection data, which may be a tree, a street lamp or a sign. In this case, further identification can be performed based on previously acquired benchmark street lamp point cloud data and benchmark sign point cloud data.

[0104] In a possible implementation, the rod-shaped point cloud data corresponding to the suspected tree trunk projection data is divided into vertical tree trunk point cloud data, occluding street lamp point cloud data and occluding marker point cloud data by using the corresponding tree crown projection data, reference street lamp point cloud data and reference marker point cloud data:

[0105] Calculating the center point of the tree crown projection data;

[0106] If the number of the suspected tree trunk projection data corresponding to the tree crown projection data is one, the rod-shaped point cloud data corresponding to the suspected tree trunk projection data is used as the vertical tree trunk point cloud data;

[0107] If the number of the suspected trunk projection data corresponding to the tree crown projection data is greater than one, the rod-shaped point cloud data corresponding to the suspected trunk projection data falling within the preset range of the center point is used as the vertical trunk point cloud data, and the rod-shaped point cloud data corresponding to the suspected trunk projection data not falling within the preset range of the center point is used as the suspected street lamp identification data;

[0108] The suspected street lamp sign data is identified as vertical tree trunk point cloud data, obstructed street lamp point cloud data and obstructed sign point cloud data through the point cloud data at the top of the reference street lamp point cloud data and the reference sign point cloud data.

[0109] When implementing the embodiment of the present application, in order to identify the tree trunk, it is necessary to calculate the center point of the crown projection data. The projection of the trunk corresponding to the crown will be not far from the center point. Based on this principle, if the number of suspected trunk projection data is one, then the suspected trunk projection data is the corresponding trunk; if the number of suspected trunk projection data is more than one, it means that there are street lights or signs. At this time, it is necessary to distinguish them by the position of the center point. The suspected trunk projection data within a certain range of the center point corresponds to the trunk, and the others need to be further identified. The identification process can be further identified by the benchmark street light point cloud data and benchmark sign point cloud data that have been acquired previously.

[0110] In a possible implementation, identifying the suspected street lamp sign data as vertical tree trunk point cloud data, obstructing street lamp point cloud data, and obstructing sign point cloud data by using the point cloud data at the top of the reference street lamp point cloud data and the reference sign point cloud data includes:

[0111] Acquire the point cloud data within the preset range at the top of the reference street lamp point cloud data as the street lamp top point cloud data, and acquire the point cloud data within the preset range at the top of the reference marker point cloud data as the marker top point cloud data;

[0112] The street lamp top point cloud data is projected as a whole to the top of the suspected street lamp identification data, and when the degree of similarity between the street lamp top point cloud data after projection and the tree crown point cloud data on the top of the suspected street lamp identification data reaches a preset value, the suspected street lamp identification data is used as the obstructed street lamp point cloud data;

[0113] Projecting the top point cloud data of the logo as a whole to the top of the suspected street light logo data, and when the degree of similarity between the top point cloud data of the logo after projection and the tree crown point cloud data at the top of the suspected street light logo data reaches a preset value, using the suspected street light logo data as the obstructed logo point cloud data;

[0114] The portion of the suspected street lamp identification data that is not identified as obstructed street lamp point cloud data and obstructed identification point cloud data is used as vertical tree trunk point cloud data.

[0115] When implementing the embodiments of the present application, the inventors found that the tops of street lamps and signs have obvious features in the point cloud data, and street lamps and signs can be effectively identified based on this feature. Specifically, the point cloud data of the top of the street lamp can be projected as a whole to the top of the suspected street lamp sign data that needs to be identified, and a similarity match is performed. When the similarity reaches a certain level, it means that the suspected street lamp sign data is a street lamp; similarly, the sign can be identified in this way. After the street lamp and sign are identified, the remaining suspected street lamp sign data is generally a dead tree trunk, that is, a tree trunk without a crown. It should be understood that in the projection process of the street lamp top point cloud data and the sign top point cloud data, the projection direction needs to be considered, that is, the street lamp top point cloud data needs to face the road, that is, the street lamp top point cloud data needs to be perpendicular to the extension direction of the road, and similarly, the sign top point cloud data also needs to be perpendicular to the extension direction of the road.

[0116] In a second aspect, the embodiment of the present application further provides a road information recognition system based on vehicle-mounted point clouds and aerial photographs, including:

[0117] An acquisition unit is configured to acquire three-dimensional point cloud data of a target road section through a vehicle-mounted laser radar, and acquire an aerial photograph image of the target road section through a drone;

[0118] An edge detection unit is configured to perform edge recognition on the aerial photograph image to obtain edge data, and divide the internal area between the edge data into a road area, a tree crown area and a building area;

[0119] a projection unit configured to project the coordinates of the road area to the three-dimensional point cloud data, and identify the road in the three-dimensional point cloud data through the road area to form road point cloud data;

[0120] A density detection unit is configured to remove the road point cloud data from the three-dimensional point cloud data to form other point cloud data, and divide the other point cloud data into tree crown point cloud data, rod point cloud data and building point cloud data through a density detection algorithm;

[0121] A checking unit is configured to check the building point cloud data through the building area, project the tree crown point cloud data to a preset reference plane and fuse the tree crown area to form tree crown projection data;

[0122] a dividing unit configured to divide the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data and sign point cloud data according to the tree crown projection data, and integrate the tree trunk point cloud data and the corresponding tree crown point cloud data into a whole to form tree point cloud data;

[0123] The recognition unit is configured to use the road point cloud data, the building point cloud data, the tree point cloud data, the street lamp point cloud data and the sign point cloud data as the recognized road information data.

[0124] In a possible implementation, the checking unit is further configured to:

[0125] Projecting the tree crown point cloud data onto a preset reference plane to form projection point cloud data, and projecting the tree crown area onto the reference plane to form projection area data;

[0126] The projection point cloud data is divided according to the edges in the projection area data to form the tree crown projection data.

[0127] In a possible implementation, the present application embodiment provides a specific technical implementation solution, please refer to Figure 2 , shows the implementation method of road recognition through aerial photo images in the embodiment of the present application, wherein the yellow line represents the edge data corresponding to the road area obtained by edge recognition of aerial photo images. Figure 4 The projective filtering scheme shown in is used to identify road point clouds. Figure 3 , shows the difference in pixel values ​​between the tree crown area and the building area in the aerial photo image, which can be used to identify the tree crown area and the building area. In the process of identifying the pole point cloud data, when distinguishing the tree trunk from the street lamp and the signboard, it is necessary to project the tree crown point cloud and the suspected tree trunk projection data onto the reference plane. For the projection process, please refer to Figure 5 , where the blue part is the crown point cloud and the suspected trunk projection data, the projected crown projection data is green, and the suspected trunk projection data is red and yellow projection data. The specific positions of the crown projection data and the suspected trunk projection data on the reference plane can be found in Figure 6 , the crown projection data will be enveloped by the edge data representing the crown. By calculating the center point of the crown projection data, it can be found that the red projection point cloud data will be at the center point, and the yellow projection point cloud data will be at a certain distance from the center point. At this time, the rod point cloud data corresponding to the red color is used as the vertical trunk point cloud data, and the rod point cloud data corresponding to the yellow color is used as the suspected street light sign data. Since the corresponding benchmark street light point cloud data and benchmark sign point cloud data have been obtained in the previous article, the suspected street light sign data is identified through these data at this time. Figure 7It can be clearly seen that the black point cloud at the lower end is the lamp pole part of the street lamp, and the black point cloud at the bottom is the lamp head part of the street lamp. In summary, the embodiments of the present application can effectively distinguish and calculate street lamps, tree trunks and signs. Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0128] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0129] The units described as separate components may or may not be physically separated. As units, it is obvious that a person of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0130] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0132] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A road information recognition method based on vehicle-mounted point cloud and aerial photographs, characterized in that: include: Acquire three-dimensional point cloud data of a target road section by using a vehicle-mounted laser radar, and acquire aerial photographs of the target road section by using a drone; Performing edge recognition on the aerial photograph image to obtain edge data, and dividing the internal area between the edge data into a road area, a tree crown area, and a building area; Projecting the coordinates of the road area to the three-dimensional point cloud data, and identifying the road in the three-dimensional point cloud data through the road area to form road point cloud data; Eliminating the road point cloud data from the three-dimensional point cloud data to form other point cloud data, and dividing the other point cloud data into tree crown point cloud data, rod point cloud data and building point cloud data by a density detection algorithm; Verify the building point cloud data through the building area, project the tree crown point cloud data onto a preset reference plane and fuse the tree crown area to form tree crown projection data; Dividing the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data and sign point cloud data according to the tree crown projection data, and integrating the tree trunk point cloud data and the corresponding tree crown point cloud data into a whole to form tree point cloud data; The road point cloud data, building point cloud data, tree point cloud data, street lamp point cloud data and sign point cloud data are used as the recognized road information data; Verifying the building point cloud data through the building area includes: Projecting the building point cloud data and the building area onto a preset reference plane; If the degree of overlap between the projected building point cloud data and the building area is higher than a preset value, the building point cloud data is determined to be building point cloud data that correctly corresponds to the building.

2. The road information recognition method based on vehicle-mounted point cloud and aerial photographs according to claim 1 is characterized in that: Dividing the inner area between edge data into road area, tree crown area and building area includes: Calculate the slopes between multiple consecutive pixel points in the same non-closed edge data, and when the slope difference between different pixel points is less than a preset value, determine that the edge data is road edge data; Acquire the inner area between two adjacent road edge data as the road area; Get the pixel value of the inner area of ​​the closed edge data; When the average value of the pixel values ​​matches the crown pixel value, the internal area is judged to be a crown area; when the average value of the pixel values ​​matches the building pixel value and the crown area is greater than a threshold, the internal area is judged to be a building area.

3. The road information recognition method based on vehicle-mounted point cloud and aerial photographs according to claim 1 is characterized in that: Projecting the coordinates of the road area to the three-dimensional point cloud data, and identifying the road in the three-dimensional point cloud data through the road area to form the road point cloud data comprises: Taking the plane where the point with the lowest vertical coordinate in the three-dimensional point cloud data is located as the projection plane, and projecting the coordinates of the road area onto the projection plane to form a projection area; The three-dimensional point cloud data is searched for a point having the same horizontal coordinate as the projection area and the closest vertical coordinate as the road point cloud data.

4. The road information recognition method based on vehicle-mounted point cloud and aerial photographs according to claim 1 is characterized in that: Projecting the tree crown point cloud data onto a preset reference plane and fusing the tree crown area to form tree crown projection data includes: Projecting the tree crown point cloud data onto a preset reference plane to form projection point cloud data, and projecting the tree crown area onto the reference plane to form projection area data; The projection point cloud data is divided according to the edges in the projection area data to form the tree crown projection data.

5. The road information recognition method based on vehicle-mounted point cloud and aerial photographs according to claim 4 is characterized in that: Dividing the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data and sign point cloud data according to the tree crown projection data includes: Acquire the included angle between the axis of the rod-shaped point cloud data and the vertical angle, and mark the rod-shaped point cloud data as vertical rod-shaped point cloud data when the included angle is less than a preset angle, and mark the rod-shaped point cloud data as inclined tree trunk point cloud data when the included angle is greater than or equal to the preset angle; Projecting the vertical rod-shaped point cloud data onto the reference plane to form rod-shaped projection data; If the rod-shaped projection data is enveloped by the tree crown projection data, the rod-shaped projection data is used as suspected tree trunk projection data; If the rod-shaped projection data is not enveloped by the tree crown projection data, the rod-shaped projection data is used as street lamp identification projection data, and the rod-shaped point cloud data corresponding to the street lamp identification projection data is divided into reference street lamp point cloud data and reference identification point cloud data; The rod-shaped point cloud data corresponding to the suspected tree trunk projection data is divided into vertical tree trunk point cloud data, blocked street lamp point cloud data and blocked sign point cloud data by using the corresponding tree crown projection data, reference street lamp point cloud data and reference sign point cloud data; The vertical tree trunk point cloud data and the inclined tree trunk point cloud data are used as the tree trunk point cloud data, the reference street lamp point cloud data and the blocked street lamp point cloud data are used as the street lamp point cloud data, and the reference marker point cloud data and the blocked marker point cloud data are used as the marker point cloud data.

6. The road information recognition method based on vehicle-mounted point cloud and aerial photographs according to claim 5 is characterized in that: The rod-shaped point cloud data corresponding to the suspected tree trunk projection data is divided into vertical tree trunk point cloud data, occluding street lamp point cloud data and occluding sign point cloud data by using the corresponding tree crown projection data, reference street lamp point cloud data and reference sign point cloud data: Calculating the center point of the tree crown projection data; If the number of the suspected tree trunk projection data corresponding to the tree crown projection data is one, the rod-shaped point cloud data corresponding to the suspected tree trunk projection data is used as the vertical tree trunk point cloud data; If the number of the suspected trunk projection data corresponding to the tree crown projection data is greater than one, the rod-shaped point cloud data corresponding to the suspected trunk projection data falling within the preset range of the center point is used as the vertical trunk point cloud data, and the rod-shaped point cloud data corresponding to the suspected trunk projection data not falling within the preset range of the center point is used as the suspected street lamp identification data; The suspected street lamp sign data is identified as vertical tree trunk point cloud data, obstructed street lamp point cloud data and obstructed sign point cloud data through the point cloud data at the top of the reference street lamp point cloud data and the reference sign point cloud data.

7. The road information recognition method based on vehicle-mounted point cloud and aerial photographs according to claim 6 is characterized in that: Identifying the suspected street lamp sign data as vertical tree trunk point cloud data, obstructing street lamp point cloud data and obstructing sign point cloud data by using the point cloud data at the top of the reference street lamp point cloud data and the reference sign point cloud data includes: Acquire the point cloud data within the preset range at the top of the reference street lamp point cloud data as the street lamp top point cloud data, and acquire the point cloud data within the preset range at the top of the reference marker point cloud data as the marker top point cloud data; Projecting the street lamp top point cloud data as a whole to the top of the suspected street lamp identification data, and when the degree of similarity between the street lamp top point cloud data after projection and the tree crown point cloud data on the top of the suspected street lamp identification data reaches a preset value, using the suspected street lamp identification data as the obstructed street lamp point cloud data; Projecting the top point cloud data of the logo as a whole to the top of the suspected street light logo data, and when the degree of similarity between the top point cloud data of the logo after projection and the tree crown point cloud data at the top of the suspected street light logo data reaches a preset value, using the suspected street light logo data as the obstructed logo point cloud data; The portion of the suspected street lamp identification data that is not identified as obstructed street lamp point cloud data and obstructed identification point cloud data is used as vertical tree trunk point cloud data.

8. A road information recognition system based on vehicle-mounted point cloud and aerial photographs using the method according to any one of claims 1 to 7, characterized in that: include: An acquisition unit is configured to acquire three-dimensional point cloud data of a target road section through a vehicle-mounted laser radar, and acquire an aerial photograph image of the target road section through a drone; An edge detection unit is configured to perform edge recognition on the aerial photograph image to obtain edge data, and divide the internal area between the edge data into a road area, a tree crown area and a building area; a projection unit configured to project the coordinates of the road area to the three-dimensional point cloud data, and identify the road in the three-dimensional point cloud data through the road area to form road point cloud data; A density detection unit is configured to remove the road point cloud data from the three-dimensional point cloud data to form other point cloud data, and divide the other point cloud data into tree crown point cloud data, rod point cloud data and building point cloud data through a density detection algorithm; A checking unit is configured to check the building point cloud data through the building area, project the tree crown point cloud data to a preset reference plane and fuse the tree crown area to form tree crown projection data; a dividing unit configured to divide the rod-shaped point cloud data into tree trunk point cloud data, street lamp point cloud data and sign point cloud data according to the tree crown projection data, and integrate the tree trunk point cloud data and the corresponding tree crown point cloud data into a whole to form tree point cloud data; The recognition unit is configured to use the road point cloud data, the building point cloud data, the tree point cloud data, the street lamp point cloud data and the sign point cloud data as the recognized road information data.

9. The road information recognition system based on vehicle-mounted point cloud and aerial photographs according to claim 8, characterized in that: The verification unit is further configured to: Projecting the tree crown point cloud data onto a preset reference plane to form projection point cloud data, and projecting the tree crown area onto the reference plane to form projection area data; The projection point cloud data is divided according to the edges in the projection area data to form the tree crown projection data.