Road segmentation method and device based on airborne laser radar, equipment and medium

Abstract

The invention provides a road segmentation method and device based on an airborne laser radar, equipment and a medium. Three-dimensional point cloud data acquired by the airborne laser radar and a color image with color information acquired by a camera are used for jointly performing model training and road segmentation. Wherein the point cloud data is a set composed of a plurality of vectors in a three-dimensional coordinate system, the position of each vector in the space is represented by using X, Y and Z coordinates, and model training of coding and deep learning is carried out in combination with attributes such as RGB color, point cloud intensity value, echo frequency and acquisition time; the method achieves the understanding and classification of a complex point cloud structure through rich data features, and achieves the precise and automatic segmentation of a road.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, in particular to the field of laser data segmentation technology, and discloses a road segmentation method and device, equipment, and medium based on airborne laser radar. Background Art

[0002] With the improvement of hardware performance and the advancement of data processing algorithms, airborne LiDAR systems can provide higher density and more accurate point cloud data, which provides a better basis for road extraction; however, since the point cloud intensity is affected by factors such as transmission power, angle, distance, and the reflection characteristics of road surfaces of different materials are inconsistent, there are consistency and accuracy problems in the road extraction process. Although scholars have done a lot of research on road extraction, accuracy and automation are still key issues. Summary of the invention

[0003] The present disclosure at least provides a road segmentation method, device, equipment, and medium based on airborne laser radar to improve the accuracy and automation of road segmentation.

[0004] According to one aspect of the present disclosure, a road segmentation method based on airborne laser radar data is provided, comprising:

[0005] Obtain a sample point cloud data set obtained by scanning a ground area with an airborne laser radar, and obtain a sample color image set obtained by photographing the ground area with a camera;

[0006] Obtaining sample label information of the sample point cloud data set;

[0007] Acquire point cloud feature information of each point in the sample point cloud data set, and acquire color feature information of each point from the color image set; the point cloud feature information includes point cloud coordinates, point cloud intensity, and echo number;

[0008] Set the feature validity flag corresponding to each point; wherein the feature validity flag is used to identify whether the color feature information and point cloud intensity of the corresponding point are valid;

[0009] The feature coding operation is performed using the point cloud feature information, color feature information and feature validity flag of each point to obtain the feature code of each point;

[0010] Inputting sample point cloud data, sample color images, and feature codes of each point into a road segmentation model to be trained, determining a loss function between the output of the road segmentation model and the sample label information, and adjusting the parameters of the road segmentation model with the goal of minimizing the value of the loss function until the value of the loss function is less than a preset threshold, thereby obtaining a trained road segmentation model;

[0011] Acquire a test point cloud data set and test label information, and input the test point cloud data set into the road segmentation model, determine the accuracy information of the road segmentation model according to the output of the road segmentation model and the test label information, and determine the road segmentation model as a target road segmentation model when the accuracy information meets a preset condition;

[0012] Obtain laser point cloud data and a color image of a target area to be segmented, process the laser point cloud data and the color image using a target road segmentation model to obtain a road point cloud within the target area; and determine the road position within the target area based on the road point cloud.

[0013] In a possible implementation, the road segmentation model is a PointNet model or a variant model of PointNet.

[0014] In a possible implementation, the loss function is a cross entropy loss function.

[0015] In a possible implementation manner, adjusting the parameters of the road segmentation model with the goal of minimizing the value of the loss function includes:

[0016] Using stochastic gradient descent (SGD) or Adam, the parameters of the road segmentation model are adjusted with the goal of minimizing the value of the loss function.

[0017] In a possible implementation manner, the process of training the road segmentation model further includes:

[0018] Early stopping and regularization methods are used to prevent the road segmentation model from overfitting.

[0019] In a possible implementation manner, the point cloud feature information further includes at least one of the following:

[0020] Normal direction; local point cloud density.

[0021] In a possible implementation, when the value of the feature validity flag is 1, it indicates that the color feature information is valid; when the value of the feature validity flag is 2, it indicates that the point cloud intensity is valid; when the value of the feature validity flag is 3, it indicates that both the color feature information and the point cloud intensity are valid; when the value of the feature validity flag is 0, it indicates that the point cloud coordinates are valid.

[0022] According to another aspect of the present disclosure, a road segmentation device based on airborne laser radar data is provided, comprising:

[0023] A data acquisition module is used to acquire a sample point cloud data set obtained by scanning a ground area with an airborne laser radar, and to acquire a sample color image set obtained by photographing the ground area with a camera;

[0024] A labeling information acquisition module, used to obtain sample label information of the sample point cloud data set;

[0025] A feature extraction module, used to obtain point cloud feature information of each point in the sample point cloud data set, and to obtain color feature information of each point from the color image set; the point cloud feature information includes point cloud coordinates, point cloud intensity and echo number;

[0026] A flag setting module is used to set the feature validity flag corresponding to each point; wherein the feature validity flag is used to identify whether the color feature information and point cloud intensity of the corresponding point are valid;

[0027] The encoding module is used to perform feature encoding operation using the point cloud feature information, color feature information and feature validity flag of each point to obtain the feature code of each point;

[0028] A training module, used for inputting sample point cloud data, sample color images, and feature codes of each point into a road segmentation model to be trained, determining a loss function between the output of the road segmentation model and the sample label information, and adjusting the parameters of the road segmentation model with the goal of minimizing the value of the loss function until the value of the loss function is less than a preset threshold, thereby obtaining a trained road segmentation model;

[0029] A testing module, used to obtain a test point cloud data set and test label information, and input the test point cloud data set into the road segmentation model, determine the accuracy information of the road segmentation model according to the output of the road segmentation model and the test label information, and determine that the road segmentation model is a target road segmentation model when the accuracy information meets a preset condition;

[0030] The application module is used to obtain laser point cloud data and color images of a target area to be segmented, process the laser point cloud data and color images using a target road segmentation model to obtain a road point cloud within the target area; and determine the road position within the target area based on the road point cloud.

[0031] According to another aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0032] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned methods is implemented.

[0033] The disclosed airborne laser radar-based road segmentation method, device, equipment, and medium use the three-dimensional point cloud data collected by the airborne laser radar and the color image with color information collected by the camera to jointly perform model training and road segmentation. Among them, the point cloud data is a set of multiple vectors in a three-dimensional coordinate system, each vector uses X, Y, and Z coordinates to represent its position in space, and is combined with RGB color, point cloud intensity value, echo number, and acquisition time. The model training is encoded and deep learning, which realizes the use of rich data features to understand and classify complex point cloud structures, and realizes accurate and automated segmentation of roads.

[0034] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0036] Figure 1 is a flow chart of a road segmentation method based on airborne laser radar according to the present disclosure;

[0037] Figure 2A is one of the schematic diagrams of the sample point cloud data set annotated according to the embodiment of the present disclosure;

[0038] Figure 2B This is a second schematic diagram of a sample point cloud data set labeled according to an embodiment of the present disclosure;

[0039] Figure 3A It is a true color RGB diagram of the road;

[0040] Figure 3BIt is a schematic diagram of road point cloud intensity;

[0041] Figure 3C is one of the schematic diagrams of road segmentation results according to an embodiment of the present disclosure;

[0042] Figure 3D This is a second schematic diagram of a road segmentation result according to an embodiment of the present disclosure;

[0043] Figure 4 is a schematic diagram of the structure of a road segmentation device based on an airborne laser radar according to the present disclosure;

[0044] Figure 5 is a schematic structural diagram of an electronic device according to the present disclosure. DETAILED DESCRIPTION

[0045] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0046] In order to solve the defects of poor accuracy and low automation in current road segmentation, the present invention provides a road segmentation method, device, equipment, and medium based on airborne laser radar. The present invention uses the three-dimensional point cloud data collected by the airborne laser radar and the color image with color information collected by the camera to jointly perform model training and road segmentation. Among them, the point cloud data is a set composed of multiple vectors in a three-dimensional coordinate system, each vector uses X, Y, and Z coordinates to represent its position in space, and is combined with RGB color, point cloud intensity value, echo number, and acquisition time. The model training is performed for encoding and deep learning, which realizes the use of rich data features to understand and classify complex point cloud structures, and realizes accurate and automated segmentation of roads.

[0047] The technical solution of the present disclosure is described below through specific embodiments.

[0048] like Figure 1 As shown, it is a flow chart of the road segmentation method based on airborne laser radar in this embodiment. The execution subject of this embodiment is a computing device or component with data processing capability. Specifically, the method of this embodiment may include the following steps:

[0049] S110, obtaining a sample point cloud data set obtained by scanning a ground area with an airborne laser radar, and obtaining a sample color image set obtained by photographing the ground area with a camera.

[0050] Airborne laser radar (LiDAR) is capable of acquiring high-precision three-dimensional point cloud data over a large area. It determines the distance of the target object by emitting laser pulses to the ground and measuring the time difference of the reflected pulses. This non-contact measurement method can quickly cover large areas and generate dense and accurate point cloud data sets. In addition to the XYZ coordinates, the LiDAR system can also record the echo intensity (Intensity) of each point, that is, the point cloud intensity, which reflects the degree to which the laser pulse is reflected by the surface of the object, which is very useful for distinguishing between objects of different materials. In the present disclosure, the LiDAR device works synchronously with a digital camera to add true color information (RGB) to the point cloud, which not only enhances the visualization effect, but also provides additional spectral information for subsequent analysis, which helps to improve classification accuracy.

[0051] S120: Obtain sample label information of the sample point cloud data set.

[0052] In order to train a deep learning model, i.e. a road segmentation model, labeled training samples are required, i.e., the sample point cloud dataset is labeled. This step is usually completed by experts using professional point cloud editing software, who manually or semi-automatically label the road parts in the point cloud. This process is time-consuming but crucial because it directly affects the learning effect of the model. Figure 2A , 2B Shown is a labeled sample point cloud dataset.

[0053] S130, obtaining point cloud feature information of each point in the sample point cloud data set, and obtaining color feature information of each point from the color image set; the point cloud feature information includes point cloud coordinates, point cloud intensity, and echo times.

[0054] The point cloud feature information also includes at least one of the following: normal direction; local point cloud density.

[0055] The selection of features is directly related to the accuracy of the road segmentation model. In addition to the basic spatial coordinates, color feature information (RGB), point cloud intensity (Intensity), echo count, etc. are all valuable features that can be used for subsequent feature encoding. Depending on the application scenario and data quality, other features may need to be considered, such as normal direction, local point cloud density and other geometric features. These features are encoded together to improve the accuracy of model classification.

[0056] S140. Set the feature validity flag corresponding to each point; wherein the feature validity flag is used to identify whether the color feature information and point cloud intensity of the corresponding point are valid.

[0057] In some cases, color feature information and point cloud intensity may be missing. For features that may be missing (for example, some points have no color information), it is necessary to introduce a feature validity flag (Sign) to indicate the existence of color feature information and point cloud intensity, so as to ensure that all points can be correctly encoded. Specifically, when the value of the feature validity flag is 1, it indicates that the color feature information is valid; when the value of the feature validity flag is 2, it indicates that the point cloud intensity is valid; when the value of the feature validity flag is 3, it indicates that both the color feature information and the point cloud intensity are valid; when the value of the feature validity flag is 0, it indicates that the point cloud coordinates are valid. Figure 3A The following is a true color schematic diagram of the road RGB; Figure 3B Shown is a schematic diagram of road point cloud intensity.

[0058] S150, performing feature coding operation using the point cloud feature information, color feature information and feature validity flag of each point to obtain feature coding of each point.

[0059] There are N points in the sample point cloud data set, and each point will at least input the point cloud coordinates, that is, the XYZ three-dimensional coordinates for feature encoding, but if only relying on spatial features, the effect of road classification cannot be guaranteed. In this embodiment, the color feature information of the road can be obtained through the sample color image set, such as RGB color. In addition, the point cloud intensity, that is, the Intensity intensity information, can also be determined. The Intensity intensity information is often quite different from other surrounding objects. For example, the RGB color of the road is generally more uniform, and the point cloud intensity is also different from other objects. Due to the large difference in the numerical range of different features, direct input into the model may cause certain features to dominate the learning process. Therefore, before the features are sent into the model, they are normalized so that each feature varies on a similar scale. Afterwards, all selected features are integrated into a multidimensional matrix to form an N*8 format (XYZSRGBI) encoding, where N is the number of points and S is the feature validity flag, which indicates the available status of RGBI (i.e., color feature information and point cloud intensity). If it is unavailable, the corresponding value is set to -1, and if it is available, it is the specific value after normalization, ensuring that even if some features are missing, they can be effectively processed.

[0060] S160, inputting the sample point cloud data, the sample color image, and the feature code of each point into the road segmentation model to be trained, determining the loss function between the output of the road segmentation model and the sample label information, and adjusting the parameters of the road segmentation model with the goal of minimizing the value of the loss function until the value of the loss function is less than a preset threshold, thereby obtaining a trained road segmentation model.

[0061] In this example, due to the computationally intensive nature of deep learning algorithms, a computer equipped with a high-performance GPU is used for model training.

[0062] In this embodiment, the PyTorch deep learning framework is used to optimize the parameters of the road segmentation model by minimizing the loss function between the output of the road segmentation model and the sample label information until satisfactory performance indicators are achieved. In view of the characteristics of point cloud data, PointNet and its variants can be selected as the basic model of the road segmentation model. These models are specially designed to process unordered point set data and can effectively capture the spatial relationship between points. During the training process, the cross entropy loss function is used to measure the gap between the output of the road segmentation model and the sample label information, and the parameters of the road segmentation model are adjusted using optimizers such as stochastic gradient descent (SGD) or Adam. At the same time, overfitting can be prevented by early stopping, regularization, etc. to ensure the generalization ability of the model. When the loss drops to a certain ratio and is relatively stable, the training ends and a parameter file of the road segmentation model is generated. This file records a series of parameters used to identify roads in the point cloud. The parameter file of the road segmentation model can be used to classify and predict point cloud data.

[0063] S170, obtaining a test point cloud dataset and test label information, and inputting the test point cloud dataset into the road segmentation model, determining the accuracy information of the road segmentation model according to the output of the road segmentation model and the test label information, and determining that the road segmentation model is a target road segmentation model when the accuracy information meets preset conditions.

[0064] In this embodiment, a test point cloud dataset independent of the sample point cloud dataset is used to evaluate the performance of the road segmentation model and check its accuracy and generalization ability. It is very necessary to use a test point cloud dataset independent of the training set to evaluate the performance of the road segmentation model. By calculating indicators such as precision and recall, the advantages and disadvantages of the road segmentation model can be fully understood. If conditions permit, field verification can also be carried out, that is, the actual performance of the road segmentation model can be verified using newly collected point cloud data.

[0065] S180, obtaining laser point cloud data and a color image of a target area to be segmented, processing the laser point cloud data and the color image using a target road segmentation model to obtain a road point cloud within the target area; and determining the road position within the target area based on the road point cloud.

[0066] Once the road segmentation model has passed rigorous testing and verification, it can be deployed in the actual environment for the automatic classification of laser point cloud data in the target area to be segmented. With the accumulation of more high-quality data and the advancement of technology, the performance of the road segmentation model will continue to improve. Figure 3C , 3D As shown, automatic classification of the newly collected point cloud data can better identify the roads in the point cloud.

[0067] The airborne laser radar (LiDAR) point cloud system of the above embodiment can start from multiple aspects such as geometric features, intensity information, spectral information, etc. when classifying roads. The geometric features include height information, smoothness and flatness, connectivity and continuity, and shape regularity.

[0068] Height information: Roads are usually located on or close to the ground, so elevation data can be used to preliminarily screen out possible road areas. Non-ground objects such as buildings and trees can be identified.

[0069] Smoothness and flatness: The road surface is relatively flat, so the local curvature or plane fitting residual of the point cloud is small. Using the point cloud data in the local area to calculate its flatness is an effective way to distinguish roads from other terrains.

[0070] Connectivity and continuity: Roads tend to have a ribbon-like structure and good connectivity. This feature can be detected and enhanced by analyzing the connectivity between point clouds or using morphological operations such as dilation and erosion.

[0071] Shape regularity: Artificial roads generally have relatively regular boundaries and shapes, such as straight lines or arcs. Based on this, algorithms can be designed to detect these geometric patterns to assist in road extraction.

[0072] Intensity information includes reflectivity differences and texture analysis.

[0073] Reflectivity difference: Different materials have different reflective properties to lasers. The reflectivity of road materials (such as asphalt and concrete) is usually different from that of vegetation and buildings. The intensity value reflects the energy returned by the LiDAR pulse and can be used as one of the classification bases.

[0074] Texture analysis: Texture analysis of intensity images can capture surface details such as roughness, cracks, etc., which helps to further distinguish different types of land features.

[0075] Spectral information includes visible light images and multi-source image fusion.

[0076] Visible light images: Orthophotos obtained using high-resolution satellite images or aerial photogrammetry can provide rich visual information. The color, texture, and edge clarity of the road can be intuitively observed in these images, providing an important reference for road classification.

[0077] Multi-source image fusion: Combining LiDAR point cloud data with visible light images (RGB), near infrared (NIR) images, and other types of remote sensing images can fully utilize the advantages of each band. For example, visible light images are suitable for depicting natural colors, while near infrared images are particularly effective in distinguishing vegetation; short wave infrared (SWIR) images are good at detecting minerals and water content.

[0078] In this example, multi-source data fusion was performed. With the advancement of technology, especially the improvement of sensor resolution and the development of processing algorithms, the comprehensive use of this information from multiple angles has become a key trend to improve the effect of road classification. Combining the above-mentioned multiple information sources, the key to using deep learning technology to classify point clouds is to construct accurate data features. Appropriate feature extraction can make the classification algorithm more powerful, the classification more accurate, and the speed faster.

[0079] Based on the same inventive concept, the present disclosure provides a road segmentation device based on airborne laser radar, and the steps performed by the components of the device are the same or similar to those of the above method, so similar parts are not repeated. Figure 4 As shown, the road segmentation device based on airborne laser radar in this embodiment includes:

[0080] The data acquisition module 410 is used to acquire a sample point cloud data set obtained by scanning a ground area with an airborne laser radar, and to acquire a sample color image set obtained by photographing the ground area with a camera.

[0081] The label information acquisition module 420 is used to obtain sample label information of the sample point cloud data set.

[0082] The feature extraction module 430 is used to obtain point cloud feature information of each point in the sample point cloud data set, and to obtain color feature information of each point from the color image set; the point cloud feature information includes point cloud coordinates, point cloud intensity and echo number.

[0083] The flag setting module 440 is used to set the feature validity flag corresponding to each point; wherein the feature validity flag is used to identify whether the color feature information and point cloud intensity of the corresponding point are valid.

[0084] The encoding module 450 is used to perform feature encoding operations using the point cloud feature information, color feature information and feature validity flag of each point to obtain the feature code of each point.

[0085] The training module 460 is used to input the sample point cloud data, the sample color image, and the feature code of each point into the road segmentation model to be trained, determine the loss function between the output of the road segmentation model and the sample label information, and adjust the parameters of the road segmentation model with the goal of minimizing the value of the loss function until the value of the loss function is less than a preset threshold, thereby obtaining a trained road segmentation model.

[0086] The testing module 470 is used to obtain a test point cloud data set and test label information, and input the test point cloud data set into the road segmentation model, determine the accuracy information of the road segmentation model according to the output of the road segmentation model and the test label information, and determine that the road segmentation model is the target road segmentation model when the accuracy information meets the preset conditions.

[0087] Application module 480 is used to obtain laser point cloud data and color images of the target area to be segmented, process the laser point cloud data and color images using the target road segmentation model to obtain the road point cloud within the target area; and determine the road position within the target area based on the road point cloud.

[0088] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a computer-readable storage medium.

[0089] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0090] like Figure 5 As shown, the device 500 includes a computing unit 510, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 520 or a computer program loaded from a storage unit 580 into a random access memory (RAM) 530. In the RAM 530, various programs and data required for the operation of the device 500 can also be stored. The computing unit 510, the ROM 520, and the RAM 530 are connected to each other via a bus 540. An input / output (I / O) interface 550 is also connected to the bus 540.

[0091] A number of components in the device 500 are connected to the I / O interface 550, including: an input unit 560, such as a keyboard, a mouse, etc.; an output unit 570, such as various types of displays, speakers, etc.; a storage unit 580, such as a disk, an optical disk, etc.; and a communication unit 590, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 590 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0092] The computing unit 510 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 510 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 510 performs the various methods and processes described above. For example, in some embodiments, any of the above methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 580. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via ROM520 and / or communication unit 590. When the computer program is loaded into RAM530 and executed by the computing unit 510, one or more steps of any of the methods described above may be performed. Alternatively, in other embodiments, the computing unit 510 may be configured to perform any of the methods described above in any other appropriate manner (e.g., by means of firmware).

[0093] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0095] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0096] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0097] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0098] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0099] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0100] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A road segmentation method based on airborne laser radar, characterized in that: include: Obtain a sample point cloud data set obtained by scanning a ground area with an airborne laser radar, and obtain a sample color image set obtained by photographing the ground area with a camera; Obtaining sample label information of the sample point cloud data set; Acquire point cloud feature information of each point in the sample point cloud data set, and acquire color feature information of each point from the color image set; the point cloud feature information includes point cloud coordinates, point cloud intensity, and echo number; Set the feature validity flag corresponding to each point; wherein the feature validity flag is used to identify whether the color feature information and point cloud intensity of the corresponding point are valid; The feature coding operation is performed using the point cloud feature information, color feature information and feature validity flag of each point to obtain the feature code of each point; Inputting sample point cloud data, sample color images, and feature codes of each point into a road segmentation model to be trained, determining a loss function between the output of the road segmentation model and the sample label information, and adjusting the parameters of the road segmentation model with the goal of minimizing the value of the loss function until the value of the loss function is less than a preset threshold, thereby obtaining a trained road segmentation model; Acquire a test point cloud data set and test label information, and input the test point cloud data set into the road segmentation model, determine the accuracy information of the road segmentation model according to the output of the road segmentation model and the test label information, and determine the road segmentation model as a target road segmentation model when the accuracy information meets a preset condition; Obtain laser point cloud data and a color image of a target area to be segmented, process the laser point cloud data and the color image using a target road segmentation model to obtain a road point cloud within the target area; and determine the road position within the target area based on the road point cloud.

2. The method according to claim 1, characterized in that The road segmentation model is a PointNet model or a variant model of PointNet.

3. The method according to claim 1, characterized in that The loss function is a cross entropy loss function.

4. The method according to claim 1, characterized in that The step of adjusting the parameters of the road segmentation model with the goal of minimizing the value of the loss function includes: Using stochastic gradient descent (SGD) or Adam, the parameters of the road segmentation model are adjusted with the goal of minimizing the value of the loss function.

5. The method according to claim 1, characterized in that The process of training the road segmentation model also includes: Early stopping and regularization methods are used to prevent the road segmentation model from overfitting.

6. The method according to claim 1, characterized in that The point cloud feature information also includes at least one of the following: Normal direction; local point cloud density.

7. The method according to claim 1, characterized in that When the value of the feature validity flag is 1, it indicates that the color feature information is valid; When the value of the feature validity flag is 2, it indicates that the point cloud intensity is valid. When the value of the feature validity flag is 3, it indicates that both the color feature information and the point cloud intensity are valid. When the value of the feature validity flag is 0, it indicates that the point cloud coordinates are valid.

8. A road segmentation device based on airborne laser radar, characterized in that: include: A data acquisition module is used to acquire a sample point cloud data set obtained by scanning a ground area with an airborne laser radar, and to acquire a sample color image set obtained by photographing the ground area with a camera; A labeling information acquisition module, used to obtain sample label information of the sample point cloud data set; A feature extraction module, used to obtain point cloud feature information of each point in the sample point cloud data set, and to obtain color feature information of each point from the color image set; the point cloud feature information includes point cloud coordinates, point cloud intensity and echo number; A flag setting module is used to set the feature validity flag corresponding to each point; wherein the feature validity flag is used to identify whether the color feature information and point cloud intensity of the corresponding point are valid; The encoding module is used to perform feature encoding operation using the point cloud feature information, color feature information and feature validity flag of each point to obtain the feature code of each point; A training module, used for inputting sample point cloud data, sample color images, and feature codes of each point into a road segmentation model to be trained, determining a loss function between the output of the road segmentation model and the sample label information, and adjusting the parameters of the road segmentation model with the goal of minimizing the value of the loss function until the value of the loss function is less than a preset threshold, thereby obtaining a trained road segmentation model; A testing module, used to obtain a test point cloud data set and test label information, and input the test point cloud data set into the road segmentation model, determine the accuracy information of the road segmentation model according to the output of the road segmentation model and the test label information, and determine that the road segmentation model is a target road segmentation model when the accuracy information meets a preset condition; The application module is used to obtain laser point cloud data and color images of a target area to be segmented, process the laser point cloud data and color images using a target road segmentation model to obtain a road point cloud within the target area; and determine the road position within the target area based on the road point cloud.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images