Dynamic object automatic filtering system and method based on vehicle-mounted dual-band laser radar

Abstract

The invention provides a dynamic object automatic filtering system and method based on a vehicle-mounted dual-band laser radar, and the system comprises a rigid connection platform, a data processing and control system, a dual-band laser radar module, an IMU and GNSS integrated navigation module, and a multispectral camera. The method comprises the steps of collecting dual-band laser point cloud data and multispectral data based on the dynamic object automatic filtering system, performing dynamic object automatic filtering operation based on the dual-band laser point cloud data, and performing dynamic object automatic filtering operation based on the multispectral data. According to the invention, synchronous denoising of the dual-band laser point cloud and the multispectral image and effective filtering of the dynamic object can be realized, and the method is convenient to use.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a dynamic object automatic filtering system and method based on a vehicle-mounted dual-band laser radar. Background Art

[0002] The rapid development of intelligent driving technology has greatly promoted the research of various environmental perception sensors. Vehicle-mounted laser radar (LiDAR) has an important position in the intelligent driving environment perception system because of its advantages such as accurate acquisition of three-dimensional information of the target, high resolution, strong anti-interference ability, wide detection range, and near all-weather operation. The specific scenario of intelligent driving places high demands on the performance of both the hardware technology and the application algorithms of vehicle-mounted LiDAR. For example, the laser safety level of vehicle-mounted LiDAR used in intelligent driving systems above Level 3 (the intelligent driving automation level proposed by SAE (Society of Automotive Engineers), with a total of 6 levels from Level 0 to 5) should be Level 1 (power less than 0.4mW, low-output laser) to ensure the safety of pedestrians during the driving of intelligent driving cars; its detection distance should be at least 100m, the measurement accuracy should be at the centimeter level, and the scanning frequency should be at least 5Hz to 20Hz to meet the requirements of high-speed driving of intelligent driving cars; and the transmission rate of LiDAR point cloud data, the running speed, accuracy and processing effect of the point cloud processing algorithm must also meet the requirements of high-speed driving of cars; at the same time, the size, weight, cost and driving voltage of the LiDAR should all meet the installation level of the vehicle-mounted system.

[0003] In order to meet the high requirements of intelligent driving for vehicle-mounted laser radar, the relevant research on key technologies and application algorithms of vehicle-mounted laser radar is constantly deepening and expanding. Among the many laser radar hardware technologies such as laser emission, reception, and scanning, it is generally believed that scanning technology is the key to vehicle-mounted laser radar. It directly determines the important parameters of the radar, such as scanning frequency, scanning range, and point cloud data volume. It is the key to reducing the overall volume and cost of vehicle-mounted laser radar, and it also directly affects the realization of vehicle-grade mass production of vehicle-mounted laser radar. In terms of application algorithm development, there are many point cloud processing algorithms for the millions of point cloud data generated per second by high-scanning frequency and resolution vehicle-mounted laser radars. They are all based on different application scenarios selected by application developers, and their characteristics have their own advantages and disadvantages. There is no perfect unified algorithm yet, but the goal pursued by application algorithm research is the same, that is, to continuously pursue real-time and high precision on the basis of effective data processing, so as to provide effective technical support for the control and decision-making of intelligent driving systems. At present, the denoising algorithm for vehicle-mounted laser radar mainly processes single-modal data, and the single-modal data denoising algorithm can no longer meet the current denoising needs of massive multimodal data. At present, the research on filtering dynamic objects relies heavily on the pulse information of laser point clouds and intelligent recognition algorithms. The algorithm performance is unstable in complex urban scene environments and cannot meet the current high-precision map production needs for high-quality data. Therefore, it is very necessary to design an automatic dynamic object filtering system and method based on vehicle-mounted dual-band laser radar. Summary of the invention

[0004] In order to overcome the deficiencies of the prior art, the object of the present invention is to provide a dynamic object automatic filtering system and method based on a vehicle-mounted dual-band laser radar.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] The present invention provides a dynamic object automatic filtering system method based on a vehicle-mounted dual-band laser radar, comprising: a rigid connection platform, a data processing and control system, a dual-band laser radar module, an IMU and GNSS combined navigation module and a multi-spectral camera, wherein the rigid connection platform is arranged on a vehicle, the data processing and control system, the dual-band laser radar module, the IMU and GNSS combined navigation module and the multi-spectral camera are arranged on the rigid connection platform, and the dual-band laser radar module, the IMU and GNSS combined navigation module and the multi-spectral camera are connected to the data processing and control system.

[0007] Preferably, the dual-band laser radar module includes a dual-wavelength continuous laser generating module, a receiving module, a dual-wavelength scanning module, a scanning oscillating mirror and a servo scanning system. The dual-wavelength continuous laser generating module includes a 905nm narrow linewidth continuous light semiconductor laser, a 1550nm narrow linewidth continuous light semiconductor laser, a phase modulator, a linear frequency modulation microwave signal generator and a power amplifier. The 905nm narrow linewidth continuous light semiconductor laser and the 1550nm narrow linewidth continuous light semiconductor laser are respectively connected to a phase modulator through an optical fiber, each of the phase modulators is connected to a power amplifier through an optical fiber, and the phase modulator is electrically connected to the linear frequency modulation microwave signal generator, each of the power amplifiers is connected to the dual-wavelength scanning module and a fiber optic attenuator through an optical fiber, and the fiber optic attenuator is connected to the receiving module through an optical fiber;

[0008] The receiving module includes a Si balanced detector and an InGaAs balanced detector, and the optical fiber attenuator and dual-wavelength scanning module corresponding to the 905nm narrow linewidth continuous light semiconductor laser are connected to the Si balanced detector through optical fiber, and the optical fiber attenuator and dual-wavelength scanning module corresponding to the 1550nm narrow linewidth continuous light semiconductor laser are connected to the InGaAs balanced detector through optical fiber, and the Si balanced detector and the InGaAs balanced detector are electrically connected to an AD digital acquisition card, and the AD digital acquisition card is connected to the data processing and control system;

[0009] The dual-wavelength scanning module includes a 905nm fiber circulator and a 1550nm fiber circulator. The power amplifier and Si balanced detector corresponding to the 905nm narrow linewidth continuous light semiconductor laser are connected to the 905nm fiber circulator through optical fibers. The power amplifier and InGaAs balanced detector corresponding to the 1550nm narrow linewidth continuous light semiconductor laser are connected to the 1550nm fiber circulator through optical fibers. The 905nm fiber circulator and the 1550nm fiber circulator scan adjacent space through a scanning oscillating mirror and a servo scanning system.

[0010] The present invention also provides a method for automatically filtering dynamic objects based on a vehicle-mounted dual-band laser radar, comprising:

[0011] Collect dual-band laser point cloud data and multi-spectral data based on dynamic object automatic filtering system;

[0012] Automatic filtering of dynamic objects based on dual-band laser point cloud data;

[0013] Automatic filtering of dynamic objects based on multispectral data.

[0014] Preferably, the automatic filtering operation of dynamic objects is performed based on the dual-band laser point cloud data, specifically:

[0015] Based on dual-band laser point cloud data, the surface growth algorithm is used to obtain different surface segmentation bodies. The features of each surface segmentation body are calculated according to the common features. The common features include surface feature p, height h, area a and normal vector n. The surface segmentation body is identified by the constraint function g(·). Those that meet the conditions are classified as road surface G:

[0016] G=g(p,h,a,n) (1)

[0017] The road points are removed and the point cloud is connected and analyzed to obtain different segmented bodies on the road. Dynamic objects are automatically filtered based on the band information. The filtering function is expressed as d(·). The band information includes the proportion of single-band points in the segmented body rt and the dual-band attributes bi of dynamic objects such as vehicles or bicycles. The segmented bodies that meet the conditions are classified as D:

[0018] D=d(rt,bi) (2)

[0019] Preferably, the automatic filtering operation of dynamic objects is performed based on multispectral data, specifically:

[0020] Build a 3D-CNN classification network;

[0021] Train the 3D-CNN classification network based on an end-to-end supervised training method;

[0022] Classify dynamic objects based on the trained 3D-CNN classification network;

[0023] The classified points are removed based on the mapping algorithm.

[0024] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0025] The present invention provides a dynamic object automatic filtering system and method based on a vehicle-mounted dual-band laser radar, the system comprising a rigid connection platform, a data processing and control system, a dual-band laser radar module, an IMU and GNSS combined navigation module and a multispectral camera, the method comprising collecting dual-band laser point cloud data and multispectral data based on the dynamic object automatic filtering system, performing dynamic object automatic filtering operations based on the dual-band laser point cloud data, and performing dynamic object automatic filtering operations based on the multispectral data. The present invention can achieve synchronous denoising of dual-band laser point clouds and multispectral images, as well as effective filtering of dynamic objects, and is easy to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0027] Figure 1 A schematic diagram of the system structure provided by an embodiment of the present invention;

[0028] Figure 2 It is a schematic diagram of the dual-band laser radar module structure;

[0029] Figure 3 A schematic diagram of a method flow chart provided by an embodiment of the present invention;

[0030] Figure 4 Schematic diagram of the 3D-CNN classification network structure. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The purpose of the present invention is to provide a dynamic object automatic filtering system and method based on a vehicle-mounted dual-band laser radar, which can realize the synchronous denoising of dual-band laser point cloud and multispectral image, as well as the effective filtering of dynamic objects, and is easy to use.

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Figure 1 A schematic diagram of the system structure provided by an embodiment of the present invention, such as Figure 1 As shown, the present invention provides a dynamic object automatic filtering system based on a vehicle-mounted dual-band laser radar, comprising:

[0035] A rigid connection platform, a data processing and control system, a dual-band laser radar module, an IMU and GNSS combined navigation module and a multi-spectral camera. The rigid connection platform is arranged on a vehicle. The data processing and control system, the dual-band laser radar module, the IMU and GNSS combined navigation module and the multi-spectral camera are arranged on the rigid connection platform. The dual-band laser radar module, the IMU and GNSS combined navigation module and the multi-spectral camera are connected to the data processing and control system.

[0036] like Figure 2 As shown, the dual-band laser radar module includes a dual-wavelength continuous laser generating module, a receiving module, a dual-wavelength scanning module, a scanning swing mirror and a servo scanning system. The dual-wavelength continuous laser generating module includes a 905nm narrow linewidth continuous light semiconductor laser, a 1550nm narrow linewidth continuous light semiconductor laser, a phase modulator, a linear frequency modulation microwave signal generator and a power amplifier. The 905nm narrow linewidth continuous light semiconductor laser and the 1550nm narrow linewidth continuous light semiconductor laser are respectively connected to a phase modulator through an optical fiber, each of the phase modulators is connected to a power amplifier through an optical fiber, and the phase modulator is electrically connected to the linear frequency modulation microwave signal generator, each of the power amplifiers is connected to an optical fiber splitting optical path module through an optical fiber, one of which is connected to the dual-wavelength scanning module, and the other is connected to a fiber attenuator, and the fiber attenuator is connected to the receiving module through an optical fiber;

[0037] The receiving module includes a Si balanced detector and an InGaAs balanced detector, and the optical fiber attenuator and dual-wavelength scanning module corresponding to the 905nm narrow linewidth continuous light semiconductor laser are connected to the Si balanced detector through an optical fiber and a beam expansion-combination-scanning module, and the optical fiber attenuator and dual-wavelength scanning module corresponding to the 1550nm narrow linewidth continuous light semiconductor laser are connected to the InGaAs balanced detector through an optical fiber and a beam expansion-combination-scanning module, and the Si balanced detector and the InGaAs balanced detector are electrically connected to an AD digital acquisition card, and the AD digital acquisition card is connected to the data processing and control system;

[0038] The dual-wavelength scanning module includes a 905nm fiber circulator and a 1550nm fiber circulator. The power amplifier and Si balanced detector corresponding to the 905nm narrow linewidth continuous light semiconductor laser are connected to the 905nm fiber circulator through optical fibers. The power amplifier and InGaAs balanced detector corresponding to the 1550nm narrow linewidth continuous light semiconductor laser are connected to the 1550nm fiber circulator through optical fibers. The 905nm fiber circulator and the 1550nm fiber circulator scan adjacent space through a scanning oscillating mirror and a servo scanning system.

[0039] The present invention adopts CW system and realizes the detection of target space three-dimensional information and spectral information through coherent detection method, such as Figure 1As shown: 905nm and 1550nm single-frequency narrow-linewidth laser modules output continuous lasers; the modulated output 905nm and 1550nm beams enter their respective amplifiers to amplify the output power; the continuous lasers are modulated by a radio frequency modulation signal. The above components constitute the frequency-modulated continuous laser module of the dual-band laser radar. The dual-band laser performs a 360° scan of the adjacent space through a scanning mirror. Due to the translation of the scanning platform, the system performs a helical scan. The reflected laser echo is converged to the Si detector (for 905nm wavelength CW echo signal detection) and the InGaAs detector (for 1550nm wavelength CW echo signal detection) on the focal plane through the receiving optical system.

[0040] The main functions of the data processing and control system are real-time high-speed acquisition (analog / digital conversion) and processing (high-resolution phase detection) of two-way laser echo signals, and precise control of the pointing direction of the two-dimensional high-precision servo scanning device and hardware synchronization technology, so that the high-precision posture measurement value obtained after the deep combination of GNSS / IMU can be synchronized with other sensors, and the absolute time of each distance measurement value can be calculated. Ultimately, all point clouds can be accurately projected from the user coordinate system to the absolute coordinate system.

[0041] The present invention also provides a method for automatically filtering dynamic objects based on a vehicle-mounted dual-band laser radar, comprising:

[0042] Step 100: Collecting dual-band laser point cloud data and multi-spectral data based on the dynamic object automatic filtering system;

[0043] Step 200: performing automatic filtering of dynamic objects based on dual-band laser point cloud data;

[0044] Step 300: Perform an automatic dynamic object filtering operation based on multispectral data.

[0045] In step 200, an automatic filtering operation of dynamic objects is performed based on the dual-band laser point cloud data, specifically:

[0046] First, the ground points are removed based on the feature information and the point cloud on the ground is connected to analyze to obtain different segmented bodies on the road surface. Then, the segmented bodies are analyzed based on the two band information (905nm and 1550nm) and the dynamic objects are filtered. The specific process is as follows:

[0047] To detect ground points, we first perform a surface growth algorithm on the point cloud based on dual-band laser point cloud data to obtain different surface segmentation bodies. Then, we design a series of universal features and calculate the features of each surface segmentation body based on the universal features. The universal features include surface features p, height h, area a and normal vector n. The surface segmentation body is identified through the constraint function g(·), and those that meet the conditions are classified as road surface G:

[0048] G=g(p,h,a,n) (1)

[0049] The road points are removed and the point cloud is connected and analyzed to obtain different segmented bodies on the road. Dynamic objects are automatically filtered based on the band information. The filtering function is expressed as d(·). The band information includes the proportion of single-band points in the segmented body rt and the dual-band attributes bi of dynamic objects such as vehicles or bicycles. The segmented bodies that meet the conditions are classified as D:

[0050] D=d(rt,bi) (2)

[0051] In step 300, an automatic filtering operation of dynamic objects is performed based on multispectral data, specifically:

[0052] When building maps in a dynamic environment represented by open roads in cities, it is inevitable to encounter interference from dynamic objects, such as pedestrians and vehicles. In scenes with dense populations and vehicles, point cloud positioning often has poor results. The semantic segmentation method based on deep learning in this invention performs pixel-level classification on hyperspectral images and directly classifies dynamic objects. The mapping algorithm only needs to directly remove these points.

[0053] In order to make full use of the joint information of space and spectrum, the present invention adopts the 3D-CNN framework for classification. 3D-CNN uses a three-dimensional convolution kernel to perform convolution operations on a three-dimensional data cube, which can simultaneously utilize the spatial and spectral information of the target and effectively improve the accuracy of classification.

[0054] The present invention constructs a 3D-CNN classification network containing seven network layers (such as Figure 4 As shown in Figure 2, pixel-level classification of hyperspectral images is performed. An end-to-end supervised training method is used. This method optimizes the classification network parameters based on a portion of hyperspectral image sample data with known classification labels. The above steps are summarized as follows:

[0055] Build a 3D-CNN classification network;

[0056] Train the 3D-CNN classification network based on an end-to-end supervised training method;

[0057] Classify dynamic objects based on the trained 3D-CNN classification network;

[0058] The classified points are removed based on the mapping algorithm.

[0059] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0060] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A dynamic object automatic filtering system based on vehicle-mounted dual-band laser radar, characterized in that: include: A rigid connection platform, a data processing and control system, a dual-band laser radar module, an IMU and GNSS combined navigation module and a multi-spectral camera. The rigid connection platform is arranged on a vehicle. The data processing and control system, the dual-band laser radar module, the IMU and GNSS combined navigation module and the multi-spectral camera are arranged on the rigid connection platform. The dual-band laser radar module, the IMU and GNSS combined navigation module and the multi-spectral camera are connected to the data processing and control system.

2. The system according to claim 1, characterized in that The dual-band laser radar module includes a dual-wavelength continuous laser generating module, a receiving module, a dual-wavelength scanning module, a scanning swing mirror and a servo scanning system. The dual-wavelength continuous laser generating module includes a 905nm narrow linewidth continuous light semiconductor laser, a 1550nm narrow linewidth continuous light semiconductor laser, a phase modulator, a linear frequency modulation microwave signal generator and a power amplifier. The 905nm narrow linewidth continuous light semiconductor laser and the 1550nm narrow linewidth continuous light semiconductor laser are respectively connected to a phase modulator through an optical fiber, each of the phase modulators is connected to a power amplifier through an optical fiber, and the phase modulator is electrically connected to the linear frequency modulation microwave signal generator, each of the power amplifiers is connected to the dual-wavelength scanning module and a fiber attenuator through an optical fiber, and the fiber attenuator is connected to the receiving module through an optical fiber; The receiving module includes a Si balanced detector and an InGaAs balanced detector, and the optical fiber attenuator and dual-wavelength scanning module corresponding to the 905nm narrow linewidth continuous light semiconductor laser are connected to the Si balanced detector through optical fiber, and the optical fiber attenuator and dual-wavelength scanning module corresponding to the 1550nm narrow linewidth continuous light semiconductor laser are connected to the InGaAs balanced detector through optical fiber, and the Si balanced detector and the InGaAs balanced detector are electrically connected to an AD digital acquisition card, and the AD digital acquisition card is connected to the data processing and control system; The dual-wavelength scanning module includes a 905nm fiber circulator and a 1550nm fiber circulator. The power amplifier and Si balanced detector corresponding to the 905nm narrow linewidth continuous light semiconductor laser are connected to the 905nm fiber circulator through optical fibers. The power amplifier and InGaAs balanced detector corresponding to the 1550nm narrow linewidth continuous light semiconductor laser are connected to the 1550nm fiber circulator through optical fibers. The 905nm fiber circulator and the 1550nm fiber circulator scan adjacent space through a scanning oscillating mirror and a servo scanning system.

3. A method for automatic filtering of dynamic objects based on vehicle-mounted dual-band laser radar, characterized in that: include: Collect dual-band laser point cloud data and multi-spectral data based on dynamic object automatic filtering system; Automatic filtering of dynamic objects based on dual-band laser point cloud data; Automatic filtering of dynamic objects based on multispectral data.

4. The method according to claim 3, characterized in that Automatic filtering of dynamic objects based on dual-band laser point cloud data, specifically: Based on dual-band laser point cloud data, the surface growth algorithm is used to obtain different surface segmentation bodies. The features of each surface segmentation body are calculated according to the common features. The common features include surface feature p, height h, area a and normal vector n. The surface segmentation body is identified by the constraint function g(·). Those that meet the conditions are classified as road surface G: G=g(p,h,a,n) (1) The road points are removed and the point cloud is connected and analyzed to obtain different segmented bodies on the road. Dynamic objects are automatically filtered based on the band information. The filtering function is expressed as d(·). The band information includes the proportion of single-band points in the segmented body rt and the dual-band attributes bi of dynamic objects such as vehicles or bicycles. The segmented bodies that meet the conditions are classified as D: D=d(rt,bi) (2) 5. The method according to claim 3, characterized in that: Automatic filtering of dynamic objects based on multispectral data, specifically: Build a 3D-CNN classification network; Train the 3D-CNN classification network based on an end-to-end supervised training method; Classify dynamic objects based on the trained 3D-CNN classification network; The classified points are removed based on the mapping algorithm.

Images