A four-dimensional hyperspectral depth imaging system

Abstract

This invention discloses a four-dimensional hyperspectral depth imaging system, comprising a calibration camera, a hyperspectral spectrometer module, and a three-dimensional topography scanning module. The calibration camera collects surface information of an object, the hyperspectral spectrometer module acquires hyperspectral information of the surface, and the three-dimensional topography scanning module acquires three-dimensional topography information and distance information. The three-dimensional topography scanning module and the calibration camera are calibrated, and the hyperspectral spectrometer module and the calibration camera are calibrated so that each pixel of the calibration camera corresponds to a piece of three-dimensional topography information and a piece of hyperspectral information. This invention can simultaneously acquire four-dimensional information of an object's spectrum and 3D topography based on a single pixel, with a compact structure, high degree of integration, and simple operation.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional topography mapping and spectral imaging. The calibration of an auxiliary camera, a hyperspectral spectrometer module and a three-dimensional topography scanning module realizes a four-dimensional hyperspectral depth imaging system. Background Art

[0002] Light sheet profilers made based on the Sham principle can effectively obtain the three-dimensional topography of an object's surface. At the same time, hyperspectral imagers can detect fluorescence spectrum data on the object's surface. However, there is currently no equipment system that combines these two technologies to scan and obtain four-dimensional information of the object's topography and hyperspectral information.

[0003] In previous related technologies, three-dimensional topography scanning and hyperspectral imaging were separated from each other. If the spectral information of a certain point in space was to be obtained, it was often necessary to estimate it with the naked eye or perform tedious subsequent inversion processing to make the spectral information and spatial position correspond one to one. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, an object of the present invention is to provide a four-dimensional hyperspectral depth imaging system.

[0005] A four-dimensional hyperspectral depth imaging system includes a calibration camera, a hyperspectral spectrometer module, and a three-dimensional topography scanning module; the calibration camera collects surface information of an object, the hyperspectral spectrometer module obtains hyperspectral information of the surface, and the three-dimensional topography scanning module obtains three-dimensional topography information and distance information; the three-dimensional topography scanning module and the calibration camera are calibrated, and the hyperspectral spectrometer module and the calibration camera are calibrated so that each pixel of the calibration camera corresponds to a piece of three-dimensional topography information and a piece of hyperspectral information.

[0006] The hyperspectral spectrometer module includes: a laser, a beam splitter, a long-pass filter, an imaging lens, a slit, a first aspheric lens, a prism grating group, a second aspheric lens, and an area array camera connected in sequence on an optical path; the excitation light emitted by the laser is horizontally incident on the beam splitter set at a 45-degree angle, and is reflected and irradiated onto the surface of the object to be detected. The object to be detected is irradiated by the laser and generates a fluorescence signal in the linear area irradiated by the excitation light. After passing through the beam splitter and the long-pass filter, the fluorescence signal is imaged on the slit by the imaging lens, and the diffusely reflected excitation light is filtered out by the beam splitter and the long-pass filter; when the fluorescence signal passes through the slit, the first aspheric lens collimates it into parallel light, and the prism grating group performs dispersion and spectral separation. The split fluorescence is focused by the second aspheric lens on the area array camera to obtain fluorescence spectrum data corresponding to each pixel.

[0007] The prism grating group consists of two wedges and a blazed grating.

[0008] The three-dimensional profile scanning module is any three-dimensional imaging system including structured light, light sheet profiler, and binocular stereo vision system.

[0009] The three-dimensional profile scanning module adopts a light sheet profiler, which includes an imaging lens, a filter and an array detector.

[0010] When the extended surfaces of the photographed plane, the image plane, and the lens plane intersect in a straight line, the light sheet profiler obtains a fully clear image of the affected plane, and targets at different distances from the photographed plane correspond one-to-one to points on the image plane. Wherein, the focal length of the lens is f, the distance from the lens center to the photographed plane is L, the angle between the lens plane and the photographed plane is θ, and the photographed plane corresponds to the distance direction z. Assuming the calibration distance is z0, the specific distance calculation method is as follows:

[0011]

[0012] Where P is the target pixel to be measured, P 0 is the pixel point corresponding to the calibration distance z0, Pixptich is the pixel spacing, and the parameter P 0 is determined by the following formula 2:

[0013] .

[0014] Beneficial effects of the present invention:

[0015] 1. It can simultaneously obtain the four-dimensional information of the object's spectrum and 3D shape based on a single pixel point;

[0016] 2. Compact structure, high degree of integration and easy operation;

[0017] 3. It can accurately capture the spectral information in space to the pixel level, achieving a one-to-one correspondence between spectral information and spatial position information. It can be widely used in various surveying and mapping scenarios, such as hyperspectral-based detection of pests and diseases in farmland. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a structural schematic diagram of the present invention;

[0019] Figure 2 This is a schematic diagram of the Sham principle;

[0020] In the figure, the object to be detected 1, beam splitter 2, long-pass filter 3, imaging lens 4, slit 5, first aspheric lens 6, prism grating group 7, second aspheric lens 8, area array camera 9, filter and area array detector 10, calibration camera 11, and laser 12. DETAILED DESCRIPTION

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] like Figure 1 As shown, laser 12 first emits laser light. The excitation light is horizontally incident on beam splitter 2, which is set at a 45-degree angle. After being reflected, it strikes object 1. Object 1 is illuminated by the laser and produces a fluorescence signal within the linear area illuminated by the excitation light. After passing through beam splitter 2 and long-pass filter 3, the fluorescence signal is imaged by imaging lens 4 onto a 50-μm-wide slit 5. During this process, a small amount of excitation light diffusely reflected from the sample surface is filtered out by beam splitter 2 and long-pass filter 3 and is not received by the lens. After passing through slit 5, the fluorescence signal is collimated into parallel light by the first aspheric lens 6, which is then dispersed and split by the prism grating assembly 7. This assembly consists of two wedges with a vertex angle of 9.72 degrees and a blazed grating with 300 grooves and a blaze angle of 17.5 degrees. The split fluorescence light is focused by a second aspheric lens 8 onto an area array camera 9 (CMOS), which obtains the final fluorescence spectrum data.

[0023] On the other hand, the light sheet profiler composed of the imaging lens 4, the filter and the area array detector 10 can scan and obtain the three-dimensional shape of the object according to the Sham principle. The schematic diagram of the Sham principle is shown in FIG. Figure 2 shown.

[0024] The linear light source emits a light beam, and the extended surfaces of the photographed plane, wide-angle imaging lens, and area array detector intersect in a straight line. Targets at different distances from the photographed plane correspond one by one to the area array detector to form a comprehensive and clear image.

[0025] When the extended surfaces of the three planes (the subject plane, the image plane, and the lens plane) intersect in a straight line, a fully clear image can be obtained on the affected plane. Targets at different distances from the subject plane are mapped one-to-one to points on the image plane. Here, the focal length of the lens is f, the distance from the lens center to the subject plane is L, the angle between the lens plane and the subject plane is θ, and the subject plane corresponds to the distance direction z. Assume that the calibration distance is z0. The specific distance calculation method is as follows:

[0026]

[0027] Where P is the target pixel to be measured, P 0 is the pixel point corresponding to the calibration distance z0, and Pixptich is the pixel spacing. P 0 is determined by the following formula 2:

[0028]

[0029] The calibration camera 11 and the area array camera 9, the area array camera 9 and the filter and the area array detector 10 are calibrated respectively to achieve pixel-by-pixel mapping of the calibration camera image pixel information, hyperspectral information and three-dimensional morphology information, thereby realizing scanning of the object morphology and spectral thinking information.

[0030] The array camera 9 can also be corrected by using the video stream captured by the calibration camera 11: the calibration camera captures the video in the hyperspectral data to extract the pixel offset between video frames caused by jitter, which is used to compensate for the offset value, thereby obtaining a scene after jitter offset correction.

[0031] Assume that the starting frame of the video is set to f s , the end frame of the video is set to f e , the current frame is set to f i , the subsequent frames are set to f i+j , then the offset is:

[0032]

[0033] .

[0034] Where N is the number of feature points that have been filtered and matched, x 1n is the horizontal coordinate of the feature point in the current frame, x 2n is the horizontal coordinate of the feature point in the (i+j) frame, y 1n is the ordinate of the feature point in the current frame, y 2n is the ordinate of the feature point in the (i+j) frame, Hor is the horizontal pixel offset between the (i+j)th frame and the starting frame, Ver It is the vertical pixel offset of the (i+j)th frame relative to the starting frame.

[0035] Determine whether to compensate for the correction: If the vertical offset A_Ver (f i+j ) is less than the threshold T_sh , j=j+1, if it is greater than the threshold, then the current frame f i Move to the (i+j)th frame of the video and iterate until the current frame fi moves to the end frame.

[0036] Unified coordinate system: Through calibration, find the rotation and translation matrix of the relevant transformation (the image plane of the hyperspectral imager is parallel to the image plane of the calibration camera). The rotation matrix is ​​the unit matrix, and the relationship is as follows:

[0037]

[0038] where x s ,y sis the coordinate of the hyperspectral imager stitching image, x c ,y c is the coordinate of the calibrated camera stitching image, R is the rotation vector, S is the scaling vector, and T is the translation vector.

Claims

1. A four-dimensional hyperspectral depth imaging system, characterized by: The method comprises a calibration camera (11), a hyperspectral spectrometer module, and a three-dimensional shape scanning module; the calibration camera (11) collects surface information of an object, the hyperspectral spectrometer module obtains hyperspectral information of the surface, and the three-dimensional shape scanning module obtains three-dimensional shape information and distance information; the three-dimensional shape scanning module and the calibration camera (11) are calibrated, and the hyperspectral spectrometer module and the calibration camera (11) are calibrated, so that each pixel of the calibration camera (11) corresponds to a three-dimensional shape information and a hyperspectral information; The hyperspectral spectrometer module comprises: a laser (12), a beam splitter (2), a long-pass filter (3), an imaging lens (4), a slit (5), a first aspheric lens (6), a prism grating group (7), a second aspheric lens (8), and a surface array camera (9) connected in sequence on an optical path; the excitation light emitted by the laser (12) is horizontally incident on the beam splitter (2) set at a 45-degree angle, and is reflected and irradiated onto the surface of the object to be detected (1); the object to be detected is irradiated by the laser and generates a fluorescence signal in the linear area irradiated by the excitation light; the fluorescence signal passes through the beam splitter (2) and the long-pass filter (3), and is imaged on the slit (5) by the imaging lens (4); the diffusely reflected excitation light is filtered out by the beam splitter (2) and the long-pass filter (3); When the fluorescence signal passes through the slit (5), the first aspheric lens (6) collimates it into parallel light, and the prism grating group (7) disperses and splits it. The fluorescence after splitting is focused by the second aspheric lens (8) on the array camera (9) to obtain the fluorescence spectrum data corresponding to each pixel; The prism grating group (7) is composed of two wedges and a blazed grating; The three-dimensional profile scanning module adopts a light sheet profiler, comprising: an imaging lens (4), a filter and an array detector (10); When the extended surfaces of the photographed plane, the image plane, and the lens plane intersect in a straight line, the light sheet profiler obtains a fully clear image of the affected plane, and targets at different distances from the photographed plane correspond one-to-one to points on the image plane. Wherein, the focal length of the lens is f, the distance from the lens center to the photographed plane is L, the angle between the lens plane and the photographed plane is θ, the photographed plane corresponds to the distance direction z, and the calibration distance is z0. The specific distance calculation method is as follows: （1） Where P is the target pixel to be measured, P 0 is the pixel point corresponding to the calibration distance z0, Pixptich is the pixel spacing, and the parameter P 0 is determined by the following formula 2: （2）。

Images