Multi-view hyperspectral acquisition device

Abstract

The present application belongs to the field of spectral imaging technology, and particularly relates to a spectral three-dimensional reconstruction system based on a multi-view hyperspectral acquisition device. The acquisition device comprises a host computer, a stepper motor controller, a motorized turntable, a halogen light source and a hyperspectral imager. The hyperspectral imager is connected to the host computer through a network data interface for data transmission. The motorized turntable is connected to the stepper motor controller, and the stepper motor controller is in communication with the host computer. The present application integrates an automated multi-view acquisition device and a data processing system based on the principle of stereo vision, solves the problems of incomplete single-view information, redundant multi-view data and non-uniform reference, realizes compact and complete hyperspectral point cloud model reconstruction of an object, and meets the technical requirements of geometric and spectral information fusion measurement in the fields of medical imaging, optical sorting, plant health monitoring and cultural heritage protection.

Description

Technical Field

[0001] This invention belongs to the field of spectral imaging technology, and in particular relates to a multi-view hyperspectral acquisition device. Background Technology

[0002] The fusion system of hyperspectral imaging and multi-view stereo vision measurement is a measurement device that simultaneously characterizes the chemical and physical properties of an object. This system acquires hyperspectral images of an object from different perspectives and then generates a 3D point cloud with accompanying hyperspectral information using multi-view stereo geometry technology. This four-dimensional data format possesses both a three-dimensional spatial dimension and a single-point spectral attribute dimension. It allows for the selection of single-band grayscale stereo images at specific wavelengths to observe the surface morphology of the target, or the selection of spectral information from a single point to understand the material properties of the target. Thanks to the "point-spectrum integration" advantage of the generated data format, this fusion system enables simultaneous morphological and spectral analysis of the object under test.

[0003] To address the challenges in 3D hyperspectral modeling, current solutions are often discrete and non-integrated. One approach involves using a spectral imaging device and a laser profilometer to separately acquire the spectral and spatial information of an object, obtaining a one-to-one mapping of point cloud spectra through the positional relationship between the two sensors. Another approach uses a structured light projector and a spectrometer system to acquire modulated images from a single viewpoint, restoring the object's spectrum and shape by demodulating the deformed patterns. While these acquisition methods can generate 3D hyperspectral data, the fusion measurement of spectral imaging and laser profilometers at close range is susceptible to calibration parameter dependence, easily leading to spectral and geometric misalignment. Furthermore, the structured light projector and spectrometer system is prone to interference from active light sources, resulting in distortion of the acquired spectral data.

[0004] Traditional single-view hyperspectral imaging equipment typically lacks stereo imaging capabilities, making it impossible to measure the morphology and spectral information of an object from all angles; it can only acquire two-dimensional spectral information from one surface. While rotating the object can acquire multi-view hyperspectral image data, simply piling up this data results in significant data redundancy, placing immense pressure on storage, computation, and transmission. In the field of plant health monitoring, there is an urgent need for refined material identification and three-dimensional morphological analysis of objects. Constructing a complete, redundancy-free hyperspectral point cloud model of an object has significant application value. Developing a measurement device and data processing system capable of acquiring complete morphological and spectral information of an object can address the need for fusion measurement of spectral and geometric information. Summary of the Invention

[0005] In view of this, the present invention aims to provide a multi-view hyperspectral acquisition device to overcome the two major problems of incomplete information from a single view and data redundancy and inconsistent benchmarks from multiple views. The present invention integrates an automated multi-view hyperspectral data acquisition device and couples it with a data processing system based on the principle of stereo vision to ultimately realize the reconstruction of a compact and complete hyperspectral point cloud model of an object, so as to meet the technical needs of geometric and spectral information fusion measurement in fields such as medical imaging, optical sorting, plant health monitoring and cultural heritage protection.

[0006] To achieve the above objectives, the technical solution created by this invention is implemented as follows: A multi-view hyperspectral acquisition device includes: a host computer, a stepper motor controller, an electric turntable, a halogen light source, and a hyperspectral imager, wherein: The hyperspectral imager establishes a data transmission connection with the host computer via a network cable data interface, the electric turntable is connected to the stepper motor controller, and the stepper motor controller is connected to the host computer via a communication connection. The object to be tested is placed on an electric turntable. A halogen light source is used to provide uniform illumination for the object on the turntable. The host computer controls the electric turntable to rotate by a predetermined angle through a stepper motor controller. The hyperspectral imager collects the current reflected light signal of the object to be tested and generates a single-view hyperspectral image. The host computer controls the rotation of the electric turntable so that the hyperspectral imager can complete multi-view acquisition and obtain a multi-view hyperspectral image sequence with overlapping areas that cover the preset viewing range of the object to be tested. The control module of the host computer receives and processes the multi-view hyperspectral image sequence to obtain a three-dimensional hyperspectral point cloud.

[0007] Furthermore, the electric turntable is connected to the stepper motor controller via an RS232 serial port, and the stepper motor controller communicates with the host computer via an RS232 serial port.

[0008] Furthermore, the control module includes a preprocessing module, a camera pose parameter calibration module, a dense matching module, a multi-view point cloud stitching module, and a point cloud post-processing module connected in sequence, wherein: The preprocessing module receives multi-view hyperspectral image sequences and converts all hyperspectral images contained in the sequences into spectral reflectance images. It then segments and matches feature points in these spectral reflectance images to obtain cross-view feature point matching pairs. The camera pose parameter calibration module solves for the camera optimization intrinsic and extrinsic parameters of the hyperspectral imager across all views based on the cross-view feature point matching pairs. The dense matching module generates depth maps for each view based on the camera optimization intrinsic and extrinsic parameters of the hyperspectral imager across all views using a multi-view stereo vision algorithm. The multi-view point cloud stitching module constructs hyperspectral point cloud data based on the depth maps and spectral reflectance images from each view. The point cloud post-processing module denoises the hyperspectral point cloud data to obtain a three-dimensional hyperspectral point cloud.

[0009] Furthermore, the specific operation steps of the preprocessing module are as follows: The preprocessing module receives a multi-view hyperspectral image sequence, acquires a standard white board image and a dark current image, and performs spectral reflectance conversion on the hyperspectral images contained in the multi-view hyperspectral image sequence based on the standard white board image and the dark current image to obtain the spectral reflectance image of each view. The spectral reflectance images from each viewpoint are segmented sequentially to obtain the background region and target region of each spectral reflectance image. The target region of each spectral reflectance image is taken as the effective observation region of each spectral reflectance image. Scale-invariant feature points in the effective observation region of each spectral reflectance image are extracted and matched with corresponding pixels to obtain cross-view feature point matching pairs.

[0010] Furthermore, the specific operation steps of the camera pose parameter calibration module are as follows: Based on cross-view feature point matching pairs, the motion reconstruction structure technique is used to solve the camera intrinsic and extrinsic parameters of the hyperspectral imager at all views. The bundle adjustment method optimizes the camera intrinsic and extrinsic parameters of the hyperspectral imager at all viewing angles with the goal of minimizing the reprojection error of sparse point clouds. This yields the optimized camera intrinsic and extrinsic parameters of the hyperspectral imager at all viewing angles. Based on cross-view feature point matching pairs, incremental reconstruction technology calculates the camera pose of the hyperspectral imager at each viewpoint, and establishes a unified world coordinate system with the camera pose of the hyperspectral imager at the first acquisition viewpoint as the reference.

[0011] Furthermore, the specific operation steps of the dense matching module are as follows: A multi-view stereo vision algorithm is used to perform pixel-level matching of each spectral reflectance image, and the depth value of each pixel in each spectral reflectance image is estimated using the triangulation principle, outputting a depth map corresponding to each viewpoint.

[0012] Furthermore, the specific operation of the multi-view point cloud stitching module is as follows: Based on the camera extrinsic parameters and depth maps of each viewpoint, the spectral reflectance images of each viewpoint are back-projected onto a unified world coordinate system to generate an initial multi-view 3D hyperspectral point cloud. The initial multi-view 3D hyperspectral point cloud is identified and aggregated based on a graph clustering algorithm. For each aggregated point cluster, a feature point cloud is generated by average weighted fusion to obtain hyperspectral point cloud data.

[0013] Compared with the prior art, the present invention can achieve the following beneficial effects: (1) The multi-view hyperspectral acquisition device described in this invention addresses the problems of incomplete single-view hyperspectral imaging information, large data redundancy in multi-view systems, and the fusion measurement of spectral and geometric information. This invention integrates an automated multi-view acquisition device with a stereo vision-based data processing system to automatically fuse multi-view two-dimensional hyperspectral image sequences into a unified three-dimensional hyperspectral point cloud model, achieving complete and accurate measurement of object geometry and spectral attributes. The back-end data processing system automatically solves high-precision camera parameters and reconstructs dense three-dimensional geometry using motion reconstruction structures and multi-view stereo vision algorithms, providing a unified spatial reference for spectral data. Through breadth-first search-based spatial clustering and average weighted fusion algorithms, redundant observation points from multiple views are identified and aggregated, significantly reducing the total data volume.

[0014] (2) The multi-view hyperspectral acquisition device created by the present invention adopts a three-dimensional hyperspectral modeling method based on multi-view stereo vision and intelligent fusion. By developing a data acquisition and processing system, the three-dimensional shape and hyperspectral information of the object are acquired simultaneously, which solves the problems of limited viewpoint, high model redundancy, and difficulty in accurately fusing spectral and geometric information in traditional hyperspectral imaging technology. Attached Figure Description

[0015] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A schematic diagram of the structure of the multi-view hyperspectral acquisition device described in the embodiment of the present invention; Figure 2 A schematic diagram of the control module described in an embodiment of the present invention.

[0016] Explanation of reference numerals in the attached figures: 1. Host computer; 2. Stepper motor controller; 3. Electric turntable; 4. Halogen light source; 5. Hyperspectral imager; 6. Preprocessing module; 7. Camera pose parameter calibration module; 8. Dense matching module; 9. Multi-view point cloud stitching module; 10. Point cloud post-processing module. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0018] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0019] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] like Figure 1 As shown, this invention proposes a multi-view hyperspectral acquisition device, comprising: a host computer 1, a stepper motor controller 2, an electric turntable 3, a halogen light source 4, and a hyperspectral imager 5, wherein: The hyperspectral imager 5 establishes a data transmission connection with the host computer 1 through a network cable data interface, the electric turntable 3 is connected to the stepper motor controller 2, and the stepper motor controller 2 is connected to the host computer 1 through a communication connection. The object to be tested is placed on the electric turntable 3. The halogen light source 4 is used to provide uniform illumination for the object to be tested placed on the electric turntable 3. The host computer 1 controls the electric turntable 3 to rotate by a predetermined angle through the stepper motor controller 2. The hyperspectral imager 5 collects the current reflected light signal of the object to be tested and generates a single-view hyperspectral image. The host computer 1 controls the rotation of the electric turntable 3 so that the hyperspectral imager 5 can complete multi-view acquisition and obtain a multi-view hyperspectral image sequence with overlapping areas and covering the preset viewing range of the object to be tested. The control module of the host computer 1 receives and processes the multi-view hyperspectral image sequence to obtain a three-dimensional hyperspectral point cloud.

[0023] It should be noted that the hyperspectral imager 5 transmits data to the host computer 1 via a network cable data interface, and the motorized turntable 3 is connected to the stepper motor controller 2 via an RS232 serial port. The stepper motor controller 2 communicates with the host computer 1 via the RS232 serial port. The halogen light source 4 is independently powered and provides uniform illumination for the object under test. Under the illumination of the halogen light source 4, the reflected light signal of the object under test, placed on the motorized turntable 3, is collected by the front objective lens of the hyperspectral imager 5. The hyperspectral imager 5 adopts a line scan working mode, and uses a built-in dispersive element to spatially broaden the spectrum of each column of pixels in the scene. In single-view hyperspectral image acquisition, the hyperspectral imager 5 is located on the translation stage and horizontally scans the imaging surface of the front objective lens to obtain the broadened spectrum of each column of pixels. Finally, the field of view is expanded by image-side scanning, and all column data are merged into a hyperspectral image with a planar field of view. After each viewpoint is acquired, the host computer 1 controls the turntable to rotate by a predetermined angle through the stepper motor controller 2. This process is repeated until 360° or the preset viewpoint range is covered, thereby acquiring a series of multi-view hyperspectral image sequences with overlapping areas and known viewpoint relationships.

[0024] In some embodiments, the electric turntable 3 is connected to the stepper motor controller 2 via an RS232 serial port, and the stepper motor controller 2 is connected to the host computer 1 via an RS232 serial port.

[0025] In some embodiments, such as Figure 2 As shown, the control module includes a preprocessing module 6, a camera pose parameter calibration module 7, a dense matching module 8, a multi-view point cloud stitching module 9, and a point cloud post-processing module 10 connected in sequence, wherein: The preprocessing module 6 receives a multi-view hyperspectral image sequence and converts all hyperspectral images contained in the sequence into spectral reflectance images. It then segments and matches feature points in the spectral reflectance images to obtain cross-view feature point matching pairs. The camera pose parameter calibration module 7 solves for the camera optimization intrinsic and extrinsic parameters of the hyperspectral imager 5 at all views based on the cross-view feature point matching pairs. The dense matching module 8 generates a depth map for each view based on the camera optimization intrinsic and extrinsic parameters of the hyperspectral imager 5 at all views using a multi-view stereo vision algorithm. The multi-view point cloud stitching module 9 constructs hyperspectral point cloud data based on the depth maps and spectral reflectance images from each view. The point cloud post-processing module 10 denoises the hyperspectral point cloud data to obtain a three-dimensional hyperspectral point cloud.

[0026] It should be noted that the data acquisition and processing flow of this invention is as follows: The light signal reflected from the surface of the object under test is collected by the front-end objective lens. The built-in line-scanning imaging spectrometer (i.e., hyperspectral imager 5) broadens the spectrum of each pixel column by column, and obtains a single-view hyperspectral data cube through orthophoto stitching technology. The host computer 1 synchronously controls the electric turntable 3 to cooperate with the hyperspectral imager 5 to acquire data and obtain a multi-view hyperspectral image sequence covering the four sides of the object under test. The multi-view hyperspectral acquisition device converts the original data cube into spectral reflectance through the preprocessing module 6, and simultaneously segments and matches feature points in the spectral reflectance image. Then, based on the cross-view feature point matching, it solves the internal imaging parameters (i.e., camera intrinsic parameters) and external motion parameters (i.e., camera extrinsic parameters) of the hyperspectral imager 5 in all views. Based on the accurate camera parameters, the dense matching module 8 generates the depth map and normal vector of each view through a multi-view stereo vision algorithm. The multi-view point cloud stitching module 9 uses the solved camera pose and depth maps to back-project two-dimensional hyperspectral data (i.e., spectral reflectance images) from multiple perspectives into three-dimensional space. It then uses a breadth-first search graph clustering algorithm to aggregate overlapping redundant points, generating low-redundancy hyperspectral point cloud data under the same reference. The point cloud post-processing module 10 uses radius filtering to traverse all point cloud coordinates, checks local point cloud density, and removes isolated noise points from the fused point cloud, ultimately obtaining complete and continuous topography and hyperspectral texture information of the object surface.

[0027] Furthermore, the specific operation steps of preprocessing module 6 are as follows: The preprocessing module 6 receives a multi-view hyperspectral image sequence, acquires a standard white board image and a dark current image, and performs spectral reflectance conversion on the hyperspectral images contained in the multi-view hyperspectral image sequence based on the standard white board image and the dark current image to obtain spectral reflectance images from each view. The spectral reflectance images from each viewpoint are segmented sequentially to obtain the background region and target region of each spectral reflectance image. The target region of each spectral reflectance image is taken as the effective observation region of each spectral reflectance image. Scale-invariant feature points in the effective observation region of each spectral reflectance image are extracted and matched with corresponding pixels to obtain cross-view feature point matching pairs.

[0028] It should be noted that the preprocessing module 6 first performs spectral reflectance conversion on the original hyperspectral data cube. By acquiring image data from a standard whiteboard, the spectral digital values ​​of the original hyperspectral data cube (i.e., the hyperspectral image) are corrected to physically meaningful reflectance to eliminate the influence of sensor response and ambient light. Subsequently, the preprocessing module 6 segments the corrected spectral reflectance image to define the effective observation area and extracts scale-invariant feature points. A random consistency sampling algorithm is used to robustly match feature points across different viewpoints, establishing the correspondence between corresponding pixels in spectral reflectance images from different viewpoints.

[0029] Furthermore, the specific operation steps of the camera pose parameter calibration module 7 are as follows: Based on cross-view feature point matching pairs, the motion reconstruction structure technique is used to solve the camera intrinsic and extrinsic parameters of the hyperspectral imager 5 from all views. The bundle adjustment method optimizes the camera intrinsic and extrinsic parameters of hyperspectral imager 5 from all viewing angles with the goal of minimizing the reprojection error of sparse point clouds. This yields the optimized camera intrinsic and extrinsic parameters of hyperspectral imager 5 from all viewing angles. Based on cross-view feature point matching pairs, incremental reconstruction technology is used to calculate the camera pose of hyperspectral imager 5 at each viewpoint, and a unified world coordinate system is established with the camera pose of hyperspectral imager 5 at the first acquisition viewpoint as the reference.

[0030] It should be noted that the camera pose parameter calibration module 7 automatically solves the camera intrinsic parameters (image distance, principal point) and camera extrinsic parameters (rotation matrix and translation matrix) of the hyperspectral imager 5 at all viewpoints based on cross-view feature point matching pairs and using motion reconstruction structure technology. The camera intrinsic and extrinsic parameters are optimized by the bundle adjustment method, so that the reprojection error of the sparse point cloud is better than 1 pixel, reaching the sub-pixel level. At the same time, a unified world coordinate system is established for the hyperspectral image sequence through incremental reconstruction technology.

[0031] The camera pose parameter calibration module 7 solves for the essential matrix based on epipolar geometry constraints. By performing singular value decomposition on the essential matrix, it obtains the relative rotation matrix and relative translation vector of the initial cross-view feature point matching pairs. Based on the initial cross-view feature point matching pairs, new hyperspectral images from different viewpoints are added sequentially. For each new hyperspectral image, the camera extrinsic parameters are estimated by solving the perspective n-point projection problem, based on the 2D-3D correspondence between the image and the existing 3D point cloud. After adding each new hyperspectral image, local bundle adjustment is performed, simultaneously optimizing the intrinsic and extrinsic parameters of all cameras and the coordinates of all 3D point clouds to minimize the reprojection error of the sparse point cloud. The process of repeatedly registering new hyperspectral images from different viewpoints continues until all hyperspectral images from all viewpoints have been processed, and the camera extrinsic parameters for all viewpoints are obtained.

[0032] In some embodiments, the specific operation steps of the dense matching module 8 are as follows: A multi-view stereo vision algorithm is used to perform pixel-level matching of each spectral reflectance image, and the depth value of each pixel in each spectral reflectance image is estimated using the triangulation principle, outputting a depth map corresponding to each viewpoint.

[0033] It should be noted that this invention specifically employs the tilted window block matching algorithm in multi-view stereo vision algorithms for pixel-level matching, and uses the triangulation principle to estimate the depth and normal vector of each pixel in order to describe the shape information of the object under test from a single viewpoint.

[0034] In some embodiments, the multi-view point cloud stitching module 9 operates as follows: Based on the camera extrinsic parameters and depth maps of each viewpoint, the spectral reflectance images of each viewpoint are back-projected onto a unified world coordinate system to generate an initial multi-view 3D hyperspectral point cloud. The initial multi-view 3D hyperspectral point cloud is identified and aggregated based on a graph clustering algorithm. For each aggregated point cluster, a feature point cloud is generated by average weighted fusion to obtain hyperspectral point cloud data.

[0035] It should be noted that the multi-view point cloud stitching module 9 uses the camera pose obtained by the camera pose parameter calibration module 7 and the depth map estimated by the dense matching module 8 to back-project the two-dimensional hyperspectral data (i.e., spectral reflectance image) of multiple effective points from multiple viewpoints into three-dimensional space. A breadth-first search graph clustering algorithm is used to identify and aggregate redundant points in three-dimensional space that are adjacent to each other and represent the same physical surface. For each aggregated point cluster, its geometric coordinates are fused into a more representative three-dimensional coordinate system. Simultaneously, the multiple spectra within each cluster are averaged and weighted to generate a more representative spectrum with a higher signal-to-noise ratio. Ultimately, the amount of hyperspectral point cloud data is reduced to less than 10% of the original data.

[0036] Furthermore, the motion reconstruction technology includes feature point extraction and matching, initial camera pair selection, incremental camera pose estimation, and triangulation to generate sparse point clouds. Specifically, the normalized eight-point algorithm is used to calculate the extrinsic parameters of the initial camera pair, the PnP algorithm is used to realize incremental camera pose estimation, and the three-dimensional point coordinates of sparse feature points are recovered based on the triangulation principle.

[0037] Bundle adjustment improves the geometric accuracy of the reconstructed model by minimizing the sum of reprojection errors of all 3D points across all images, while simultaneously adjusting camera intrinsic and extrinsic parameters and 3D point coordinates.

[0038] The tilted window block matching algorithm based on the COLMAP library estimates the dense depth map and surface normal vector of the target image at the pixel level through random initialization and iterative propagation.

[0039] The multi-view stereo vision algorithm is based on camera parameters obtained from the structure of motion reconstruction (SfM) and sparse 3D point cloud, and further generates dense 3D point cloud or 3D mesh model; the present invention adopts the tilted window matching algorithm based on PatchMatch to realize the estimation of dense geometric information.

[0040] The principle of triangulation is based on the principle of binocular parallax, which calculates the three-dimensional coordinates of the target point using the known baseline length and parallax.

[0041] Graph clustering algorithms are based on the geometric similarity of nodes in a multi-view 3D point cloud, aggregating nodes with similar geometric features to form several point cloud clusters.

[0042] These algorithms are all existing algorithms. Among them, the motion reconstruction structure technique and the multi-view stereo vision algorithm are generalized methods. The motion reconstruction structure technique is specifically expressed as an incremental motion reconstruction structure algorithm, and the multi-view stereo vision algorithm is specifically expressed as a multi-view stereo vision algorithm based on depth map fusion.

[0043] The sparse point cloud adopts the incremental motion recovery structure method, extracts sparse feature points from multi-view 2D images and performs cross-view matching, then performs 3D calculation on the successfully matched feature point pairs based on the triangulation principle, and iteratively optimizes the camera parameters and 3D point coordinates through the bundle adjustment method, finally generating the sparse point cloud.

[0044] The point cloud post-processing module 10 performs radius filtering on the initially fused point cloud to remove isolated noise points that do not conform to the local point density, and finally generates hyperspectral point cloud data with high integrity and low redundancy.

[0045] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0046] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multi-view hyperspectral acquisition device, characterized in that: It includes a host computer, a stepper motor controller, an electric turntable, a halogen light source, and a hyperspectral imager, among which: The hyperspectral imager establishes a data transmission connection with the host computer via a network cable data interface, the electric turntable is connected to the stepper motor controller, and the stepper motor controller is connected to the host computer via a communication connection. The object to be tested is placed on an electric turntable. The halogen light source is used to provide uniform illumination for the object on the electric turntable. The host computer controls the electric turntable to rotate by a predetermined angle through a stepper motor controller. The hyperspectral imager collects the current reflected light signal of the object to be tested and generates a single-view hyperspectral image. The host computer controls the rotation of the electric turntable so that the hyperspectral imager can complete multi-view acquisition and obtain a multi-view hyperspectral image sequence with overlapping areas and covering the preset viewing range of the object to be tested. The control module of the host computer receives and processes the multi-view hyperspectral image sequence to obtain a three-dimensional hyperspectral point cloud.

2. The multi-view hyperspectral acquisition device according to claim 1, characterized in that: The electric turntable is connected to the stepper motor controller via an RS232 serial port, and the stepper motor controller is connected to the host computer via an RS232 serial port.

3. The multi-view hyperspectral acquisition device according to claim 1, characterized in that: The control module includes a preprocessing module, a camera pose parameter calibration module, a dense matching module, a multi-view point cloud stitching module, and a point cloud post-processing module connected in sequence, wherein: The preprocessing module receives multi-view hyperspectral image sequences and converts all hyperspectral images contained in the sequences into spectral reflectance images. It then segments and matches feature points in these spectral reflectance images to obtain cross-view feature point matching pairs. The camera pose parameter calibration module solves for the camera optimization intrinsic and extrinsic parameters of the hyperspectral imager across all views based on the cross-view feature point matching pairs. The dense matching module generates depth maps for each view based on the camera optimization intrinsic and extrinsic parameters of the hyperspectral imager across all views using a multi-view stereo vision algorithm. The multi-view point cloud stitching module constructs hyperspectral point cloud data based on the depth maps and spectral reflectance images from each view. The point cloud post-processing module denoises the hyperspectral point cloud data to obtain a three-dimensional hyperspectral point cloud.

4. The multi-view hyperspectral acquisition device according to claim 3, characterized in that: The specific operation steps of the preprocessing module are as follows: The preprocessing module receives a multi-view hyperspectral image sequence, acquires a standard white board image and a dark current image, and performs spectral reflectance conversion on the hyperspectral images contained in the multi-view hyperspectral image sequence based on the standard white board image and the dark current image to obtain the spectral reflectance image of each view. The spectral reflectance images from each viewpoint are segmented sequentially to obtain the background region and target region of each spectral reflectance image. The target region of each spectral reflectance image is taken as the effective observation region of each spectral reflectance image. Scale-invariant feature points in the effective observation region of each spectral reflectance image are extracted and matched with corresponding pixels to obtain cross-view feature point matching pairs.

5. The multi-view hyperspectral acquisition device according to claim 3, characterized in that: The specific operation steps of the camera pose parameter calibration module are as follows: Based on cross-view feature point matching pairs, the motion reconstruction structure technique is used to solve the camera intrinsic and extrinsic parameters of the hyperspectral imager at all views. The bundle adjustment method optimizes the camera intrinsic and extrinsic parameters of the hyperspectral imager at all viewing angles with the goal of minimizing the reprojection error of sparse point clouds. This yields the optimized camera intrinsic and extrinsic parameters of the hyperspectral imager at all viewing angles. Based on cross-view feature point matching pairs, incremental reconstruction technology calculates the camera pose of the hyperspectral imager at each viewpoint, and establishes a unified world coordinate system with the camera pose of the hyperspectral imager at the first acquisition viewpoint as the reference.

6. The multi-view hyperspectral acquisition device according to claim 3, characterized in that: The specific operation steps of the dense matching module are as follows: A multi-view stereo vision algorithm is used to perform pixel-level matching of each spectral reflectance image, and the depth value of each pixel in each spectral reflectance image is estimated using the triangulation principle, outputting a depth map corresponding to each viewpoint.

7. The multi-view hyperspectral acquisition device according to claim 3, characterized in that: The specific operation of the multi-view point cloud stitching module is as follows: Based on the camera extrinsic parameters and depth maps of each viewpoint, the spectral reflectance images of each viewpoint are back-projected onto a unified world coordinate system to generate an initial multi-view 3D hyperspectral point cloud. The initial multi-view 3D hyperspectral point cloud is identified and aggregated based on a graph clustering algorithm. For each aggregated point cluster, a feature point cloud is generated by average weighted fusion to obtain hyperspectral point cloud data.

Images