Hyperspectral lidar-based environmental modeling method and apparatus

Abstract

The application discloses an environment modeling method and device based on a hyperspectral laser radar, and the method comprises the following steps: acquiring point cloud information of a to-be-detected scene and original spectrum of a detection target in the to-be-detected scene through the hyperspectral laser radar; positioning and mapping are performed by using the point cloud information of the to-be-detected scene, and an initial environment three-dimensional model is constructed; surface attribute parameters of the detection target are determined; the original spectrum of the detection target is inversely processed according to the surface attribute parameters of the detection target and an output response model of the hyperspectral laser radar, so that reflectivity spectrum of the detection target is obtained; and the initial environment three-dimensional model is mapped again by using the reflectivity spectrum of the detection target, so that a target environment multi-dimensional model is obtained. According to the scheme, the original spectrum output by the hyperspectral laser radar is finely inversely processed, so that the real spectrum of the detection target is obtained, the accuracy of the spectrum is effectively improved, and the spatial geometric structure information and the spectrum information are fully matched, so that accurate environment modeling is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of laser radar, in particular to an environment modeling method and device based on hyperspectral laser radar, a computing device and a computer storage medium. BACKGROUND

[0002] Environment perception sensors are generally divided into passive detection type and active detection type. The passive detection type environment perception sensors include optical cameras, optical hyperspectral cameras, etc., which detect targets by passively receiving reflected light of the environment. The active detection type environment perception sensors include laser radars, millimeter wave radars, etc., which detect targets by actively emitting pulse signals to the environment and then receiving signals of the environment. However, in a sheltered space environment, the environmental lighting condition is limited or even completely without environmental lighting. The passive detection type optical cameras and optical hyperspectral cameras will fail to obtain texture or spectral information of the environment. The scheme of integrating detection of ordinary monochromatic laser radars and hyperspectral cameras on the same platform is also greatly affected by the lighting environment, and it is difficult to conveniently and accurately obtain real spectral information, thereby making it difficult to achieve accurate environment modeling. SUMMARY

[0003] In view of the above problems, the present application is proposed to provide an environment modeling method and device based on hyperspectral laser radar, a computing device and a computer storage medium, which overcome the above problems or at least partially solve the above problems.

[0004] According to one aspect of the present application, an environment modeling method based on hyperspectral laser radar is provided, which comprises:

[0005] acquiring point cloud information of a to-be-detected scene and original spectrum of a detected target in the to-be-detected scene by a hyperspectral laser radar;

[0006] performing positioning and mapping by using the point cloud information of the to-be-detected scene to construct an initial environment three-dimensional model;

[0007] determining surface attribute parameters of the detected target according to the initial environment three-dimensional model;

[0008] performing inversion processing on the original spectrum of the detected target according to the surface attribute parameters of the detected target and an output response model of the hyperspectral laser radar to obtain reflectivity spectrum of the detected target;

[0009] performing secondary mapping on the initial environment three-dimensional model by using the reflectivity spectrum of the detected target to obtain a target environment multi-dimensional model, wherein the target environment multi-dimensional model contains point cloud information and reflectivity spectrum.

[0010] Further, the point cloud information of the to-be-detected scene includes local point cloud information corresponding to each scanning frame.

[0011] The positioning and mapping are performed by using the point cloud information of the to-be-detected scene, and the initial environment three-dimensional model is further constructed by comprising:

[0012] For each scanning frame in the plurality of scanning frames, a geometric residual corresponding to the scanning frame is determined according to the local point cloud information corresponding to the scanning frame and the state variable to be calculated;

[0013] The state variable is iteratively optimized by using a first preset optimization algorithm to meet a spatial geometric structure optimization target, and a first target value of the state variable is obtained.

[0014] According to the first target value of the state variable, the local point cloud information corresponding to each scanning frame is converted to a global coordinate system to obtain the initial environment three-dimensional model.

[0015] Further, the state variable at least includes: position information of the hyperspectral laser radar, attitude information of the hyperspectral laser radar, and noise of the inertial measurement unit.

[0016] Further, the surface attribute parameters of the detection target include: surface smoothness of the detection target.

[0017] According to the initial environment three-dimensional model, the surface attribute parameters of the detection target are determined, and the method further comprises:

[0018] For the to-be-calculated point in the detection target, the point cloud information within a preset range of the to-be-calculated point is extracted from the initial environment three-dimensional model.

[0019] According to the point cloud information within the preset range of the to-be-calculated point, the surface smoothness of the surface of the detection target is determined.

[0020] Further, the output response model of the hyperspectral laser radar includes: a response model of a laser detection system and a laser active detection radiation transmission model.

[0021] The response model of the laser detection system records the corresponding relationship between the laser receiving energy and the original spectrum output by the hyperspectral laser radar;

[0022] The laser active detection radiation transmission model records the corresponding relationship between the laser emission energy, the detection distance, the laser angle, the surface attribute parameters of the detection target, and the laser receiving energy.

[0023] Among them, the reflectivity spectrum of the detection target is determined according to the surface smoothness in the surface attribute parameters of the detection target and the output response model of the hyperspectral laser radar.

[0024] Further, the initial environment three-dimensional model is further mapped by using the reflectivity spectrum of the detection target to obtain a target environment multi-dimensional model, and the method further comprises:

[0025] determine a spectral residual according to the reflectivity spectrum of the detection target;

[0026] iteratively optimize the state variable by using a second preset optimization algorithm to meet an optimization objective of joint matching of the spatial geometry and the spectrum, to obtain a second target value of the state variable;

[0027] convert the point cloud information of the scene to be measured and the reflectivity spectrum of the detection target to a global coordinate system according to the second target value of the state variable, and construct a target environment multidimensional model.

[0028] Further, the determining the spectral residual according to the reflectivity spectrum of the detection target further comprises:

[0029] extracting a spectrum of a laser point from the reflectivity spectrum of the detection target, to obtain an average spectrum of a neighboring point set corresponding to the laser point;

[0030] establishing the spectral residual according to the spectrum of the laser point and the average spectrum of the neighboring point set.

[0031] According to another aspect of the present application, there is provided an environment modeling device based on a hyperspectral laser radar, which comprises:

[0032] an acquisition module adapted to acquire point cloud information of a scene to be measured and an original spectrum of a detection target in the scene to be measured by using the hyperspectral laser radar;

[0033] a first modeling module adapted to perform positioning and mapping by using the point cloud information of the scene to be measured, to construct an initial environment three-dimensional model;

[0034] a determination module adapted to determine surface attribute parameters of the detection target according to the initial environment three-dimensional model;

[0035] a spectrum inversion module adapted to perform inversion processing on the original spectrum of the detection target according to the surface attribute parameters of the detection target and an output response model of the hyperspectral laser radar, to obtain a reflectivity spectrum of the detection target;

[0036] a second modeling module adapted to perform secondary mapping on the initial environment three-dimensional model by using the reflectivity spectrum of the detection target, to obtain a target environment multidimensional model, wherein the target environment multidimensional model comprises the point cloud information and the reflectivity spectrum.

[0037] According to still another aspect of the present application, there is provided a computing device comprising a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface being in communication with each other through the communication bus;

[0038] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the above-mentioned environment modeling method based on the hyperspectral laser radar.

[0039] According to still another aspect of the present application, there is provided a computer storage medium, having stored therein at least one executable instruction, which causes a processor to perform operations corresponding to the hyperspectral lidar-based environment modeling method as described above.

[0040] According to the technical solution provided by the present application, the point cloud information of the scene to be measured and the original spectrum of the detection target in the scene to be measured can be simultaneously acquired by the hyperspectral lidar, effectively overcoming the shortcoming of the traditional optical hyperspectral camera that is easily affected by natural light changes, and the hyperspectral lidar can work normally in the environment without light or with weak light; the surface attribute parameters of the detection target can be accurately determined by using the initial three-dimensional environment model established based on the point cloud information; and the relationship between the laser emission energy, the laser receiving energy, the detection distance, the laser angle, the surface smoothness of the surface of the detection target, etc. of different wavelengths is established, the original spectrum of the detection target output by the hyperspectral lidar is finely inversed, the reflectivity of the detection target is corresponded, and the real spectrum of the detection target is obtained, effectively improving the accuracy of the spectrum; in addition, the initial three-dimensional environment model is re-mapped by using the real spectrum of the detection target, the spatial geometric structure information and the spectrum information are fully matched, fine mapping is realized, a fine target environment multi-dimensional model containing the three-dimensional point cloud and the real spectrum is obtained, the accuracy and reliability of the registration are effectively improved, and the simultaneous localization and mapping technology is realized, which is suitable for scenes with single texture and scenes with single spatial geometric structure, and accurate environment modeling is realized.

[0041] The above description is only a summary of the technical solution of the present application, in order to more clearly understand the technical means of the present application, the specific embodiments of the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiments. The accompanying drawings are intended to depict only preferred embodiments of the application, and therefore should not be considered to limit the scope of the application in any way. Similarly, like reference numerals have been used in the drawings to depict like parts of the application. In the drawings:

[0043] Figure 1 A flowchart of a hyperspectral lidar-based environment modeling method according to an embodiment of the present application is shown;

[0044] Figure 2A structural block diagram of the hyperspectral lidar-based environment modeling device according to Embodiment Two of the present application is shown.

[0045] Figure 3 A structural schematic diagram of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0046] Exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0047] Figure 1 A flowchart of a hyperspectral lidar-based environment modeling method according to Embodiment One of the present application is shown, as shown in Figure 1 The method includes the following steps:

[0048] In step S101, point cloud information of a scene to be measured and original spectra of a detection target in the scene to be measured are acquired by a hyperspectral lidar.

[0049] In this embodiment, fine environment modeling is performed using the point cloud information and the original spectra acquired by the hyperspectral lidar, thereby obtaining a target environment multi-dimensional model containing both three-dimensional point cloud information and real spectra of the detection target. The hyperspectral lidar is also known as a multi-wavelength lidar. Based on the design of a single-wavelength lidar, the hyperspectral lidar uses optical splitting and an avalanche photodiode array for detection, so that full waveform information can be acquired while rich return intensity signals are also acquired. The hyperspectral lidar relies on active laser emission to simultaneously acquire ranging information and spectral information, thereby overcoming the shortcoming of traditional optical hyperspectral cameras that are easily affected by changes in natural light.

[0050] The point cloud information of the to-be-measured scene and the original spectrum of a detection target in the to-be-measured scene can be obtained simultaneously by detecting the to-be-measured scene by the hyperspectral lidar, where the detection target can be specifically an object existing in the to-be-measured scene. In this embodiment, the spectrum of the detection target directly obtained by the hyperspectral lidar is referred to as the original spectrum of the detection target, and the spectrum obtained by inverting the original spectrum is referred to as the reflectivity spectrum of the detection target, that is, the true spectrum of the detection target. The hyperspectral lidar emits laser to the environmental space of the to-be-measured scene according to a preset scanning period, receives the returned echo signal, and can output the echo waveform to reflect the spectral information characteristics of the detection target by the voltage change of the echo signal. The obtained original spectrum of the detection target is related to the laser emission energy, the detection distance, the surface attribute parameters (for example, the surface smoothness of the detection target) of the detection target, and the like. Therefore, the original spectrum of the detection target obtained by the hyperspectral lidar cannot be directly used for matching mapping.

[0051] In step S102, positioning and mapping are performed by using the point cloud information of the to-be-measured scene, and an initial environmental three-dimensional model is constructed.

[0052] With the progress and popularity of sensor technologies such as lidar, and the demand for augmented reality, autonomous driving, and intelligent robots, simultaneous localization and mapping technology has attracted more and more attention from the academic and industrial communities. Simultaneous localization and mapping is a technology for self-localization and simultaneously constructing and updating an environmental map in an unknown environment. Simultaneous localization and mapping based on a hyperspectral lidar can not only effectively improve the accuracy of localization and mapping by using spectral information, but also simultaneously obtain spatial geometric structure information and spectral information, which is an effective means for realizing fine environmental modeling. Spatial geometric structure information and spectral information can reflect the environmental space information of the to-be-measured scene from different dimensions.

[0053] The spatial geometric structure information describes the geometric structure attributes such as the position, size, and shape of all objects in the environmental space; and the spectral information describes the surface material properties of all objects in the environmental space. The spatial geometric structure information can be used to establish an environmental three-dimensional model, support collapse risk identification and analysis, personnel positioning and rescue decision-making in disaster sites, be used for detailed parameter extraction of forestry resources, and be used for three-dimensional terrain construction and other different applications; and the spectral information can be used for specific target detection, identification, deep-sea exploration, and mineral and life sign detection analysis in extraterrestrial exploration. Simultaneously obtaining and constructing the spatial structure and spectral information of the environment not only can improve the inversion accuracy of the spatial structure and spectral information, but also can expand the range and ability of spatial information application.

[0054] In the embodiment, the point cloud information of the to-be-measured scene acquired by the hyperspectral laser radar is used to perform coarse mapping, and the spatial structure information of the environment is established, so as to obtain an initial environment three-dimensional model. The inertial measurement unit (IMU) can acquire acceleration and angular velocity measurement values of itself at a high frequency, and is used to measure three-axis attitude angle (or angular velocity) and acceleration. The IMU pre-integration technology is used to add the measurement values and bias noise of the inertial measurement unit to the position and attitude update. The relative motion of the next laser radar scanning period is obtained by IMU integration, so as to eliminate the motion distortion of the laser radar scanning period. The hyperspectral laser radar can acquire ranging information and three-dimensional point cloud information of the surrounding scene at a low frequency. The relative position and attitude between two point cloud frames are added to the position and attitude update through point cloud inter-frame matching and point cloud and map matching. For an unstructured scene, the point cloud sparsity is calculated and evaluated, and the weight of the point cloud in registration is adaptively adjusted, so as to improve the reliability and accuracy of the point cloud registration.

[0055] The hyperspectral laser radar can acquire point cloud information and original spectrum simultaneously, and the acquired point cloud information and original spectrum are denoted as (p, s raw , where p is the three-dimensional coordinates of the laser point (that is, the point cloud information), and s raw is the original spectrum of the laser point. Based on the point cloud matching, the geometric residual of the local point cloud information corresponding to the current scanning frame needs to be established, where the geometric residual is defined as the distance from the current laser point p to the nearest plane in the point cloud map in the global coordinate system.

[0056] Specifically, the point cloud information of the to-be-measured scene includes local point cloud information corresponding to each scanning frame. In step S102, for each scanning frame, the geometric residual corresponding to the scanning frame is determined according to the local point cloud information corresponding to the scanning frame and the to-be-calculated state variable; the state variable is iteratively optimized by using a first preset optimization algorithm to meet a spatial geometric structure optimization target, so as to obtain a first target value of the state variable; and the local point cloud information corresponding to each scanning frame is converted to the global coordinate system according to the first target value of the state variable, so as to construct an initial environment three-dimensional model.

[0057] Specifically, the state variable at least includes position information of the hyperspectral laser radar, attitude information of the hyperspectral laser radar, and noise of the inertial measurement unit, and the state variable can also include other contents according to actual needs, which is not limited herein.

[0058] Suppose that the to-be-calculated state variable is x, the transformation of the laser radar to the global coordinate system is T(x), the normal vector of the nearest plane corresponding to the current laser point p is n, and q is a point on the nearest plane. The geometric residual r g (x, p) is

[0059] r g (x,p)=n T (T(x)pq) Formula 1;

[0060] The specific objectives of spatial geometry optimization can be as follows:

[0061]

[0062] Where, r g (x,p i ) represents geometric residual; p i Represents different laser points within the same scan frame; g represents geometry, ∑ g The covariance represents the geometric residual.

[0063] The first preset optimization algorithm may include the Levenberg-Marquardt (LM) algorithm, etc. Taking the LM algorithm as the first preset optimization algorithm as an example, the LM algorithm is used to iteratively optimize the state variables to meet the spatial geometric structure optimization objective, thereby obtaining the first target value of the state variables. The local point cloud information corresponding to each scanning frame acquired by the hyperspectral lidar is transformed to the global coordinate system to construct an initial 3D environmental model. The initial 3D environmental model contains 3D point cloud information, specifically including the 3D coordinate values ​​of each measurement point.

[0064] Step S103: Determine the surface attribute parameters of the target based on the initial three-dimensional environmental model.

[0065] The surface attribute parameters of the target object may include parameters such as surface smoothness. In the process of calculating reflectance, the surface smoothness of the target object is an influencing factor. The surface smoothness of the target object needs to be calculated from the initial 3D model of the environment. Specifically, for the point to be calculated within the target object, point cloud information within a preset range of the point to be calculated is extracted from the initial 3D model of the environment; based on the point cloud information within the preset range of the point to be calculated, the surface smoothness of the target object is determined. Those skilled in the art can set the size of the preset range according to actual needs; no limitation is made here.

[0066] Assume the point to be calculated in the target is X. i Extracting the point X to be calculated from the initial 3D environmental model i The point cloud information S within the preset range is equivalent to extracting the point X to be calculated. i The surface smoothness c of the surrounding 3D point cloud is calculated according to Formula 3:

[0067]

[0068] The smaller the surface smoothness c, the more likely the calculation point is on a smooth surface.

[0069] Step S104: Based on the surface property parameters of the target and the output response model of the hyperspectral lidar, the original spectrum of the target is inverted to obtain the reflectance spectrum of the target.

[0070] Based on the laser incident angle and the surface property parameters of the target, a fine spectral inversion is performed to obtain the accurate true reflectance spectrum of the target. Furthermore, in this embodiment, the acquisition of parameters such as the laser incident angle and the surface property parameters of the target does not rely on any external equipment; rapid and fine spectral inversion can be performed solely using a hyperspectral lidar.

[0071] To facilitate spectral inversion, it is necessary to first establish the output response model of the hyperspectral lidar. This model includes the response model of the laser detection system and the laser active detection radiative transfer model. Establishing the output response model involves two parts: first, establishing the response model of the laser detection system, which records the correspondence between the laser received energy (also known as the entrance pupil energy of the laser detector) and the original spectrum output by the hyperspectral lidar; then, establishing the laser active detection radiative transfer model, which records the correspondence between the laser emitted energy, detection distance, laser angle, surface property parameters of the target, and the laser received energy. The laser angle includes either the laser incident angle or the laser exit angle. In this embodiment, the detection distance, laser incident angle or laser exit angle, and surface smoothness of the target surface are accurately obtained. The relationship between laser emission energy, laser reception energy, detection distance, laser incident angle / laser exit angle, and surface smoothness of the target surface at different wavelengths is established. The original spectrum of the target output by the hyperspectral lidar is finely inverted and mapped to the reflectivity of the target to obtain the true spectrum of the target, i.e., the reflectivity spectrum, which effectively improves the accuracy of the spectrum.

[0072] The relationship between the hyperspectral lidar echo signal and the bidirectional reflectivity distribution function (BRDF) is established as follows:

[0073]

[0074] Among them, P, which affects the echo signal of each band of hyperspectral lidar s The factors (λ) include: the transmitted signal power P of each spectral band of the hyperspectral lidar. i (λ), lidar receiver aperture D r System parameter η sysAtmospheric Influence Factor η atm (λ), the spectral curve of the target's dihedral reflectance distribution. <f(θ i ,φ i ,θ s ,φ s ,c,λ)>,Target distance R,Incident angle θ i , emission angle θ s Specifically, in the bidirectional reflectance distribution spectral curve <f(θ i ,φ i ,θ s ,φ s In the context of ,c,λ), θ i φ is the angle of incidence, specifically the zenith angle; i θ is the incident azimuth angle; s The exit angle, specifically the exit zenith angle; φ s λ is the emission azimuth angle; c is the surface smoothness; λ is the wavelength.

[0075] The dichroic reflectance distribution spectral curve describes the reflectance distribution across the entire space and is related to the laser incident or exit direction and the laser wavelength. In this embodiment, it is necessary to obtain the reflectance perpendicular to the incident or exit direction, which is equivalent to the normalized reflectance. In this embodiment, a hyperspectral lidar is used to measure an object with a known spectrum, and the measurement results are used to fit the relationship between the normalized reflectance and the laser emission energy, laser reception energy, detection distance, laser incident / exit angle, and surface smoothness.

[0076] In this embodiment, the reflectance spectrum of the target is determined based on the surface smoothness of the target and the output response model of the hyperspectral lidar.

[0077] Step S105: Use the reflectance spectrum of the detected target to perform secondary mapping on the initial three-dimensional environmental model to obtain a multi-dimensional model of the target environment.

[0078] In this embodiment, by using spatial geometric structure information and spectral information for joint matching, a fine mapping is achieved, resulting in a fine three-dimensional environmental model that simultaneously contains three-dimensional point clouds and real spectra, i.e., a multi-dimensional model of the target environment.

[0079] For the original spectrum s rawAfter fine inversion, the reflectance spectrum s of the detected target is obtained. The point cloud information and reflectance spectrum are denoted as (p,s), where p is the three-dimensional coordinates of the laser point (i.e., point cloud information), and s is the reflectance spectrum of that laser point. The reflectance spectrum of the detected target obtained by hyperspectral lidar is not affected by illumination and reflects the true attributes of the detected target. It can not only serve as an additional constraint for point cloud and point cloud registration, improving the accuracy and reliability of registration, but also realize loop closure detection for simultaneous localization and mapping based on multi-dimensional information such as point cloud and spectrum. This makes it suitable for both scenes with simple textures and scenes with simple spatial geometry.

[0080] In step S105, the spectral residual is determined based on the reflectance spectrum of the detected target; the state variables are iteratively optimized using a second preset optimization algorithm to satisfy the optimization objective of joint matching of spatial geometry and spectrum, resulting in a second target value for the state variables; based on the second target value of the state variables, the point cloud information of the scene under test and the reflectance spectrum of the detected target are transformed into a global coordinate system to construct a multi-dimensional model of the target environment. The state variables include at least: the position information of the hyperspectral lidar, the attitude information of the hyperspectral lidar, and the noise of the inertial measurement unit. The spectral residual based on spectral differences is defined as the Euclidean distance between the spectrum of the current laser point in the global coordinate system and the average spectrum of the nearest point set in the point cloud map.

[0081] Specifically, for the laser point corresponding to the detection target, the spectrum of the laser point is extracted from the reflectance spectrum of the detection target, and the average spectrum of the set of neighboring points corresponding to the laser point is obtained. The set of neighboring points can specifically be the set of the nearest neighbors. Based on the spectrum of the laser point and the average spectrum of the set of neighboring points, the spectral residual is established.

[0082] Assuming the state variable to be calculated is x, and the average spectrum of the nearest neighbor set corresponding to the current laser point p is... Then the spectral residual r s (x,s) is

[0083]

[0084] The optimization objective of joint matching of spatial geometry and spectrum can be specifically as follows:

[0085]

[0086] Where, r g (x,p i ) represents geometric residual; p i Represents different laser points in the same scan frame; g represents geometry, ∑ g The covariance of the geometric residuals; r s (x,s i ) represents the spectral residual; si Representing different laser points p i The reflectance spectrum; s represents the spectrum, ∑ s The covariance represents the spectral residual.

[0087] The second preset optimization algorithm may include the LM algorithm, etc. Taking the LM algorithm as an example, the LM algorithm is used to iteratively optimize the state variables to meet the optimization objective of joint matching of spatial geometry and spectrum, thereby obtaining the second target value of the state variables. The local point cloud information corresponding to each scanning frame acquired by the hyperspectral lidar and the reflectance spectrum of the detected target are transformed to the global coordinate system to construct a refined environment model, namely the multidimensional target environment model. The multidimensional target environment model contains point cloud information and reflectance spectrum. That is to say, the multidimensional target environment model not only contains three-dimensional point cloud information, but also contains the true spectrum of each measurement point of the detected target.

[0088] According to the environment modeling method based on hyperspectral lidar provided in this embodiment, hyperspectral lidar can simultaneously acquire point cloud information of the scene under test and the original spectrum of the target in the scene under test, effectively overcoming the shortcomings of traditional optical hyperspectral cameras that are easily affected by changes in natural lighting. It can operate normally in environments with no light or low light, unaffected by ambient light. Using the initial 3D environmental model established based on point cloud information, the surface attribute parameters of the target can be accurately determined. Furthermore, the relationship between laser emission energy, laser reception energy, detection distance, laser angle, and surface smoothness of the target surface at different wavelengths is established, and the detection output of the hyperspectral lidar is considered as the detection... The original spectrum of the target is finely inverted and mapped to the reflectance of the target to obtain the true spectrum of the target, effectively improving the accuracy of the spectrum. In addition, the true spectrum of the target is used to perform secondary mapping of the initial 3D environmental model, and the spatial geometric structure information and spectral information are fully matched together to achieve fine mapping. This results in a fine multi-dimensional model of the target environment that contains both 3D point cloud and true spectrum. This not only effectively improves the accuracy and reliability of registration, but also enables loop closure detection of simultaneous localization and mapping technology based on multi-dimensional information such as point cloud and spectrum. This makes it suitable for both scenes with simple textures and scenes with simple spatial geometry, achieving accurate environmental modeling.

[0089] Figure 2 A structural block diagram of an environment modeling device based on hyperspectral lidar according to Embodiment 2 of the present invention is shown, as follows: Figure 2 As shown, the device includes: an acquisition module 210, a first modeling module 220, a determination module 230, a spectral inversion module 240, and a second modeling module 250.

[0090] The acquisition module 210 is adapted to acquire point cloud information of the scene under test and the original spectrum of the target in the scene under test through hyperspectral lidar.

[0091] The first modeling module 220 is suitable for: using the point cloud information of the scene to be tested for localization and mapping, and constructing an initial three-dimensional model of the environment.

[0092] The determination module 230 is suitable for: determining the surface attribute parameters of the target based on the initial three-dimensional environmental model.

[0093] The spectral inversion module 240 is adapted to: perform inversion processing on the original spectrum of the target based on the surface property parameters of the target and the output response model of the hyperspectral lidar to obtain the reflectance spectrum of the target.

[0094] The second modeling module 250 is adapted to: perform secondary mapping on the initial three-dimensional environmental model using the reflectance spectrum of the detected target to obtain a multi-dimensional model of the target environment, wherein the multi-dimensional model of the target environment includes point cloud information and reflectance spectrum.

[0095] Optionally, the point cloud information of the scene to be tested includes: local point cloud information corresponding to each scan frame. The first modeling module 220 is further adapted to: for each scan frame, determine the geometric residual corresponding to the scan frame based on the local point cloud information and the state variables to be calculated; iteratively optimize the state variables using a first preset optimization algorithm to meet the spatial geometric structure optimization objective, and obtain the first target value of the state variables; based on the first target value of the state variables, transform the local point cloud information corresponding to each scan frame to the global coordinate system to construct an initial three-dimensional model of the environment.

[0096] The state variables include at least the position information of the hyperspectral lidar, the attitude information of the hyperspectral lidar, and the noise of the inertial measurement unit.

[0097] Optionally, the surface attribute parameters of the probe target include: the surface smoothness of the probe target. The determination module 230 is further adapted to: extract point cloud information within a preset range of the point to be calculated from the initial environment 3D model for the point to be calculated in the probe target; and determine the surface smoothness of the probe target surface based on the point cloud information within the preset range of the point to be calculated.

[0098] Optionally, the output response model of the hyperspectral lidar includes: a response model of the laser detection system and a laser active detection radiative transfer model; the response model of the laser detection system records the correspondence between the laser received energy and the original spectrum output by the hyperspectral lidar; the laser active detection radiative transfer model records the correspondence between the laser emitted energy, detection distance, laser angle, surface attribute parameters of the target, and the laser received energy; wherein, the reflectivity spectrum of the target is determined based on the surface smoothness in the surface attribute parameters of the target and the output response model of the hyperspectral lidar.

[0099] Optionally, the second modeling module 250 is further adapted to: determine the spectral residual based on the reflectance spectrum of the detected target; iteratively optimize the state variables using a second preset optimization algorithm to meet the optimization objective of joint matching of spatial geometry and spectrum, and obtain the second target value of the state variables; based on the second target value of the state variables, transform the point cloud information of the scene to be tested and the reflectance spectrum of the detected target to the global coordinate system, and construct a multi-dimensional model of the target environment.

[0100] Optionally, the second modeling module 250 is further adapted to: extract the spectrum of the laser point from the reflectivity spectrum of the target for the laser point corresponding to the target, obtain the average spectrum of the neighboring point set corresponding to the laser point; and establish the spectral residual based on the spectrum of the laser point and the average spectrum of the neighboring point set.

[0101] According to the environment modeling device based on hyperspectral lidar provided in this embodiment, the hyperspectral lidar can simultaneously acquire point cloud information of the scene under test and the original spectrum of the target in the scene under test, effectively overcoming the shortcomings of traditional optical hyperspectral cameras that are easily affected by changes in natural lighting. It can operate normally in environments with no light or low light, unaffected by ambient light. Using the initial three-dimensional environmental model established based on point cloud information, the surface attribute parameters of the target can be accurately determined. Furthermore, the relationship between laser emission energy, laser reception energy, detection distance, laser angle, and surface smoothness of the target surface at different wavelengths is established, and the detection output of the hyperspectral lidar is used to model the target's surface properties. The original spectrum of the target is finely inverted and mapped to the reflectance of the target to obtain the true spectrum of the target, effectively improving the accuracy of the spectrum. In addition, the true spectrum of the target is used to perform secondary mapping of the initial 3D environmental model, and the spatial geometric structure information and spectral information are fully matched together to achieve fine mapping. This results in a fine multi-dimensional model of the target environment that contains both 3D point cloud and true spectrum. This not only effectively improves the accuracy and reliability of registration, but also enables loop closure detection of simultaneous localization and mapping technology based on multi-dimensional information such as point cloud and spectrum. This makes it suitable for both scenes with simple textures and scenes with simple spatial geometry, achieving accurate environmental modeling.

[0102] The present invention also provides a non-volatile computer storage medium storing at least one executable instruction that can execute the environment modeling method based on hyperspectral lidar in any of the above method embodiments.

[0103] Figure 3 The diagram illustrates the structure of a computing device according to an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.

[0104] like Figure 3 As shown, the computing device may include: a processor 302, a communications interface 304, a memory 306, and a communications bus 308.

[0105] in:

[0106] The processor 302, communication interface 304, and memory 306 communicate with each other via communication bus 308.

[0107] Communication interface 304 is used to communicate with other network elements such as clients or other servers.

[0108] The processor 302 is used to execute program 310, which can specifically execute the relevant steps in the above-described embodiment of the environment modeling method based on hyperspectral lidar.

[0109] Specifically, program 310 may include program code that includes computer operation instructions.

[0110] Processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computing device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0111] Memory 306 is used to store program 310. Memory 306 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0112] Specifically, program 310 can be used to cause processor 302 to execute the environment modeling method based on hyperspectral lidar in any of the above method embodiments. The specific implementation of each step in program 310 can be found in the corresponding descriptions of the steps and units in the above-described environment modeling embodiments based on hyperspectral lidar, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0113] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0114] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0115] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0116] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0117] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0118] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0119] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A method for environment modeling based on hyperspectral lidar, characterized in that, The method comprises: acquiring point cloud information of a to-be-measured scene and original spectrum of a detection target in the to-be-measured scene by a hyperspectral laser radar; performing positioning and mapping by using the point cloud information of the to-be-measured scene to construct an initial environment three-dimensional model; determining surface attribute parameters of the detection target according to the initial environment three-dimensional model; performing inversion processing on the original spectrum of the detection target according to the surface attribute parameters of the detection target and an output response model of the hyperspectral laser radar to obtain reflectivity spectrum of the detection target; performing secondary mapping on the initial environment three-dimensional model by using the reflectivity spectrum of the detection target to obtain a target environment multi-dimensional model, wherein the target environment multi-dimensional model comprises point cloud information and the reflectivity spectrum; wherein the point cloud information of the to-be-measured scene comprises local point cloud information corresponding to each scanning frame; the positioning and mapping by using the point cloud information of the to-be-measured scene to construct the initial environment three-dimensional model further comprises: for each scanning frame in each scanning frame, determining a geometric residual corresponding to the scanning frame according to the local point cloud information corresponding to the scanning frame and a to-be-calculated state variable; iteratively optimizing the state variable by using a first preset optimization algorithm to meet a spatial geometric structure optimization target to obtain a first target value of the state variable; according to the first target value of the state variable, converting the local point cloud information corresponding to each scanning frame to a global coordinate system to construct the initial environment three-dimensional model.

2. The method of claim 1, wherein, The state variable at least comprises position information of the hyperspectral laser radar, attitude information of the hyperspectral laser radar and noise of an inertial measurement unit.

3. The method of claim 1, wherein, The surface attribute parameters of the detection target comprise surface smoothness of the detection target. The determination of the surface attribute parameters of the detection target according to the initial environment three-dimensional model further comprises: for a to-be-calculated point in the detection target, extracting point cloud information within a preset range of the to-be-calculated point from the initial environment three-dimensional model; determining surface smoothness of the surface of the detection target according to the point cloud information within the preset range of the to-be-calculated point.

4. The method of claim 1, wherein, The output response model of the hyperspectral laser radar comprises a response model of a laser detection system and a laser active detection radiation transmission model; the response model of the laser detection system records a corresponding relationship between laser receiving energy and the original spectrum output by the hyperspectral laser radar; the laser active detection radiation transmission model records a corresponding relationship between laser emitting energy, detection distance, laser angle, surface attribute parameters of the detection target and laser receiving energy; wherein the reflectivity spectrum of the detection target is determined according to the surface smoothness in the surface attribute parameters of the detection target and the output response model of the hyperspectral laser radar.

5. The method according to any one of claims 1 to 4, characterized in that, the secondary mapping on the initial environment three-dimensional model by using the reflectivity spectrum of the detection target to obtain the target environment multi-dimensional model further comprises: determining a spectral residual according to the reflectivity spectrum of the detection target; iteratively optimize the state variable by using a second preset optimization algorithm to meet an optimization target of joint matching of spatial geometry structure and spectrum, to obtain a second target value of the state variable; convert the point cloud information of the scene to be measured and the reflectivity spectrum of the detection target to a global coordinate system according to the second target value of the state variable, and construct a target environment multi-dimensional model.

6. The method of claim 5, wherein, The determining the spectral residual according to the reflectivity spectrum of the detection target further includes: extracting a spectrum of a laser point corresponding to the detection target from the reflectivity spectrum of the detection target, to obtain an average spectrum of a set of adjacent points corresponding to the laser point; establishing a spectral residual according to the spectrum of the laser point and the average spectrum of the set of adjacent points.

7. A hyperspectral lidar-based environment modeling apparatus, characterized by, The device includes: an acquisition module adapted to acquire point cloud information of a scene to be measured and an original spectrum of a detection target in the scene to be measured by a hyperspectral laser radar; a first modeling module adapted to perform positioning and mapping by using the point cloud information of the scene to be measured, to construct an initial environment three-dimensional model; a determination module adapted to determine a surface attribute parameter of the detection target according to the initial environment three-dimensional model; a spectrum inversion module adapted to perform inversion processing on the original spectrum of the detection target according to the surface attribute parameter of the detection target and an output response model of the hyperspectral laser radar, to obtain a reflectivity spectrum of the detection target; a second modeling module adapted to perform secondary mapping on the initial environment three-dimensional model by using the reflectivity spectrum of the detection target, to obtain a target environment multi-dimensional model, wherein the target environment multi-dimensional model includes point cloud information and the reflectivity spectrum; The point cloud information of the scene to be measured includes local point cloud information corresponding to each scanning frame. The first modeling module is further adapted to: for each scanning frame in each scanning frame, determine a geometric residual corresponding to the scanning frame according to local point cloud information corresponding to the scanning frame and a state variable to be calculated; iteratively optimize the state variable by using a first preset optimization algorithm to meet a spatial geometry structure optimization target, to obtain a first target value of the state variable; convert the local point cloud information corresponding to each scanning frame to a global coordinate system according to the first target value of the state variable, to construct the initial environment three-dimensional model.

8. A computing device comprising: The processor, the memory, the communication interface and the communication bus complete communication among each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the environment modeling method based on the hyperspectral laser radar in any one of claims 1-6.

9. A computer storage medium, the storage medium storing at least one executable instruction, the executable instruction causing a processor to perform operations corresponding to the environment modeling method based on the hyperspectral laser radar in any one of claims 1-6.

Images