A portable auto-focus hyperspectral imaging system and method

Abstract

This application discloses a portable autofocus hyperspectral imaging system and method. The system includes a spectral data acquisition module, a spectral analysis module, a displacement platform, an inertial measurement module, and a control module. The spectral data acquisition module includes an adjustable focus lens, a beam splitter, a first collimating lens group, and an image sensor. The spectral analysis module is equipped with an electronic shutter, a slit, a second collimating lens group, a grating, a third collimating lens group, and an imaging element along the optical path. The displacement platform drives the spectral analysis module to perform internal scanning motion. The control module executes the autofocus algorithm, automatic exposure control, and multi-source data fusion and preprocessing. This application can improve focusing accuracy and spatial registration accuracy of spectral data in complex depth-of-field scenes, intelligently and adaptively adjust exposure parameters, compensate for geometric distortion, support unattended continuous operation, reduce the cost of ineffective operations, and improve the reliability and usability of hyperspectral data.

Description

Technical Field

[0001] This application belongs to the field of optical imaging technology, specifically relating to a portable autofocus hyperspectral imaging system and method. Background Technology

[0002] Portable hyperspectral imaging systems have wide applications in fields such as agricultural remote sensing and environmental monitoring. Their core function is to rapidly acquire the spectral characteristics and spatial image information of targets. However, existing technologies still have many shortcomings in areas such as autofocus, horizontal calibration, dark-field data acquisition, and exposure parameter adjustment, which directly affect the accuracy and efficiency of data acquisition and restrict the reliability of subsequent quantitative analysis.

[0003] A search revealed that CN114967093A discloses an autofocus method and system based on a microscopic hyperspectral imaging platform. This patent achieves autofocus by acquiring downsampled images and combining forward and reverse interpolation sequences to calculate the position of the clearest image. However, this technical solution is primarily designed for microscopic scenes and does not adequately consider focusing requirements in non-planar or complex depth-of-field environments. This may lead to data blurring in practical applications (such as monitoring vine crops) due to deviations between the focused area and the actual acquisition area. Furthermore, the system lacks a horizontal calibration mechanism; tilting the equipment can easily cause geometric distortion, which can only be detected during data post-processing, increasing the cost of ineffective operations.

[0004] CN115598075A discloses a deep-sea hyperspectral imaging detection system and method based on a dual-channel coaxial optical path. This patent employs a coaxial optical path dual-sensor channel design, achieving simultaneous generation of RGB color images and hyperspectral images, and possesses high integration and portability. However, this technical solution still relies on a manual lens-masking operation mode for dark-field data acquisition, which cannot adapt to the automation requirements of unattended scenarios. Furthermore, its exposure parameter adjustment requires empirical trial-and-error to determine the optimal value, significantly reducing operational efficiency, especially in dynamic environments (such as mobile detection operations). Summary of the Invention

[0005] This application provides a portable autofocus hyperspectral imaging system and method, which solves the problems of low data accuracy, poor work efficiency, and insufficient automation caused by issues such as manual focusing being too subjective and lacking a unified standard; deviation between the autofocus area and the target acquisition area; lagging horizontal calibration leading to geometric distortion that can only be detected in post-acquisition processing; the need for manual occlusion in dark field data acquisition, making it unsuitable for unattended operation; and the need for empirical exposure parameters, which significantly reduces work efficiency.

[0006] In a first aspect, this application provides a portable autofocus hyperspectral imaging system, including a spectral data acquisition module, a spectral analysis module, an inertial measurement module, a control module, and a displacement platform; the spectral data acquisition module includes an adjustable focus lens, a beam splitter, a first collimating lens group, and an image sensor; the adjustable focus lens is disposed at the incident end of the system's optical path, and the beam splitter is located at the exit end of the adjustable focus lens, used to divide the incident optical path into a first optical path and a second optical path, the first optical path being collimated by the first collimating lens group and then incident on the image sensor, and the second optical path being incident on the spectral analysis module; the spectral analysis module... The block is arranged sequentially along the optical path, comprising an electronic shutter, a slit, a second collimating lens group, a grating, a third collimating lens group, and an imaging element, with the electronic shutter, slit, second collimating lens group, grating, third collimating lens group, and imaging element aligned in position. The displacement platform supports the spectral analysis module and drives its internal scanning motion. The control module is electrically connected to the adjustable focus lens, image sensor, electronic shutter, imaging element, and inertial measurement module, and is used to execute autofocus algorithms, automatic exposure control, data acquisition synchronization, and multi-source data fusion and preprocessing. Using the above technical solution, through the design of beam splitting and parallel acquisition and intelligent fusion of multi-source data, combined with adaptive optics adjustment and built-in active scanning, a highly integrated, automated, high-precision, and stable optical analysis and detection system is constructed. This significantly improves the performance of a single function and, through functional expansion, realizes the advantages of a portable autofocus hyperspectral imaging system.

[0007] Furthermore, the spectral data acquisition module also includes a depth camera, which is parallel to the optical axis of the adjustable focus lens and has a field of view that covers the field of view of the adjustable focus lens, and is electrically connected to the control module.

[0008] Furthermore, the inertial measurement module integrates a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. The X-axis of the inertial measurement module is parallel to the long axis of the slit, and the Y-axis and Z-axis are perpendicular to the X-axis.

[0009] Secondly, this application provides a portable autofocus hyperspectral imaging method for performing a portable autofocus hyperspectral imaging system, the method comprising:

[0010] After the system attitude is stabilized based on the data from the inertial measurement module, the electronic shutter is closed, and the imaging element collects and stores the dark current data.

[0011] Based on the depth image acquired by the depth camera and the image information collected by the image sensor, the focal length of the adjustable lens is adjusted in two stages: coarse adjustment and fine adjustment, and the sharpness algorithm is called to determine the optimal focus focal length.

[0012] After autofocus is completed, the electronic shutter is opened, and the optimal exposure time is dynamically adjusted and determined based on the histogram of the focused image.

[0013] The displacement platform is activated to perform internal scanning, simultaneously acquiring hyperspectral data, depth data, and inertial measurement data;

[0014] The acquired hyperspectral data is subjected to geometric correction, dark background subtraction, uniformity correction, noise reduction and reflectance calculation. The processed hyperspectral data is then fused with depth data to generate multidimensional spectral data.

[0015] Furthermore, adjusting the focal length of the adjustable lens through two stages—coarse adjustment and fine adjustment—includes:

[0016] Based on the depth data collected in real time by the depth camera, the range of depth values ​​above a preset ratio is statistically analyzed. The target distance interval is determined by the center value of the depth values. A preset object distance-focal length correspondence table is consulted to obtain the first coarse adjustment focal length F1. And / or, the adjustable lens is controlled to traverse the entire focal length range with a step length. After each adjustment, an image is collected by the image sensor and the sharpness value is calculated. The second coarse adjustment focal length F corresponding to the maximum sharpness value is recorded. The first coarse adjustment focal length F1 and the second coarse adjustment focal length F are determined as the final coarse adjustment focal length.

[0017] Centered on the final coarse adjustment focal length, a fine adjustment range containing N focal length points is determined. The fine adjustment range is traversed with a second step size. An image is acquired at each focal length point, and the sharpness value is calculated using a sharpness algorithm. The focal length with the largest sharpness value is selected as the optimal focusing focal length Fx.

[0018] Furthermore, the step of determining a final coarse focal length from the first coarse focal length F1 and the second coarse focal length F includes:

[0019] Calculate the absolute value of the difference between the first coarse focus distance F1 and the second coarse focus distance F;

[0020] If the absolute value is less than or equal to the preset threshold, the first coarse adjustment focal length F1 will be used as the final coarse adjustment focal length.

[0021] If the absolute value is greater than the preset threshold, the second coarse adjustment focal length F will be used as the final coarse adjustment focal length.

[0022] Furthermore, the steps for using the sharpness algorithm to determine the optimal focus focal length include:

[0023] Perform grayscale normalization on the acquired images;

[0024] Multi-directional feature extraction is performed on the standardized image to enhance edge and detail features;

[0025] Calculate the feature statistics of the enhanced image as a sharpness evaluation index.

[0026] Furthermore, dynamically adjusting to determine the optimal exposure time includes:

[0027] Obtain the image histogram at the current exposure time;

[0028] If the pixel values ​​above the first preset ratio are distributed to the left of the preset target value DN, then increase the exposure time;

[0029] If the pixel values ​​above the first preset ratio are distributed to the right of the preset target value DN, then reduce the exposure time;

[0030] Repeat the above steps until pixel values ​​above the first preset ratio are distributed in the interval near the preset target value DN and form a near-normal distribution;

[0031] The final repeated exposure time is compared with the exposure time recommended based on the focus results, and the smaller of the two is taken as the optimal exposure time for hyperspectral data acquisition.

[0032] Furthermore, the geometric correction is based on the displacement platform's swing-scan angle data and the inertial measurement module's attitude data, using an affine transformation model to correct image distortion; the dark background subtraction is achieved by subtracting pre-stored dark current data from the acquired hyperspectral data; the uniformity correction uses a standard whiteboard reference method, achieving radiometric calibration by calculating the reciprocal of the response coefficient of each pixel; the noise reduction process uses the Savitzky-Golay smoothing algorithm; and the reflectance calculation is achieved by calculating the ratio of the target spectral data to the reference whiteboard spectral data.

[0033] The beneficial effects of the technical solution provided by the portable autofocus hyperspectral imaging system and method of the present invention include at least the following:

[0034] By splitting the incident light path into an image acquisition path and a spectral analysis path using a beam splitter, and combining this with depth camera-assisted focusing, spatial consistency between the focus area and the spectral acquisition area is achieved, effectively eliminating image blur and spectral distortion in non-uniform depth-of-field scenes. An integrated inertial measurement module and strict coordinate system alignment enable real-time perception of the system's attitude, providing a data foundation for active correction of geometric distortion and significantly improving data spatial registration accuracy. Utilizing an electronic shutter for automatic control of dark-field data acquisition achieves full automation in unattended scenarios, overcoming the technical bottleneck of traditional methods relying on manual occlusion. Furthermore, employing an autofocus strategy combining coarse and fine adjustments, along with a closed-loop exposure adjustment mechanism based on image histograms, reduces focusing time from several minutes in traditional methods. Through multi-source data fusion and preprocessing, multi-dimensional spectral data containing spatial coordinates, spectral features, and depth information is generated, providing richer and more reliable data support for accurate identification and quantitative analysis.

[0035] Other features and advantages of various embodiments of this specification will be further revealed in the following detailed description and accompanying drawings. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of a portable autofocus hyperspectral imaging system provided as an embodiment of this specification.

[0038] Figure 2 This is a schematic flowchart of a portable autofocus hyperspectral imaging method provided in the embodiments of this specification.

[0039] Figure 3 This is a schematic diagram illustrating the process of coarsely adjusting the focal length of an adjustable lens as provided in the embodiments of this specification.

[0040] Figure 4 This is a schematic diagram illustrating the process of finely adjusting the focal length of an adjustable lens as provided in the embodiments of this specification.

[0041] Figure 5 This is a schematic diagram illustrating the process of dynamically adjusting and determining the optimal exposure time provided in the embodiments of this specification.

[0042] Explanation of reference numerals in the attached figures: 1. Adjustable focus lens; 2. Beam splitter; 3. First collimating lens group; 4. Image sensor; 5. Electronic shutter; 6. Slit; 7. Second collimating lens group; 8. Grating; 9. Third collimating lens group; 10. Imaging element; 11. Inertial measurement module; 12. Control module; 13. Depth camera. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0044] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to facilitate the description of the embodiments and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.

[0045] Please see Figure 1 This application provides a portable autofocus hyperspectral imaging system that acquires multimodal data in parallel through optical path splitting, and combines adaptive optics adjustment with a built-in active scanning mechanism to construct a highly integrated, automated, high-precision, and stable optical analysis system. It includes a spectral data acquisition module, a spectral analysis module, an inertial measurement module 11, a control module 12, and a displacement platform.

[0046] The spectral data acquisition module includes an adjustable focus lens 1, a beam splitter 2, a first collimating lens group 3, an image sensor 4, and a depth camera 13. The adjustable focus lens 1 is positioned at the incident end of the system's optical path to acquire light signals from the target scene. Furthermore, the adjustable focus lens 1 employs a high-speed piezoelectric ceramic-driven zoom mechanism with a focal length adjustment range of 0-255 mm and a minimum adjustment step of 1 mm. It also integrates a high-precision absolute position encoder to ensure the accuracy and repeatability of focal length settings. The lens's optical design ensures low-distortion, high-resolution imaging quality throughout the entire focal length range. The adjustable focus lens 1 is connected to the control module 12 via a cable, adjusting its focal length according to commands issued by the control module 12 to adapt to different imaging distances, providing clear incident light for subsequent beam splitting and spectral analysis.

[0047] Beam splitter 2, located at the exit end of the adjustable focus lens 1, splits the incident light path into a first light path and a second light path. The first light path, after being collimated by the first collimating lens group 3, is incident on the image sensor 4 for visible light imaging or real-time monitoring. The first collimating lens group 3 is composed of multiple low-dispersion optical glasses, optimized to correct axial chromatic aberration and spherical aberration, ensuring the parallelism of the collimated beam is within ±0.1 milliradians. Specifically, beam splitter 2 employs a prism-type beam splitter 2 with a specific splitting ratio. For example, in the visible to near-infrared band, it can precisely split the incident light path according to a ratio of 70% reflected to the first light path and 30% transmitted to the second light path, ensuring that both the image sensor 4 and the spectral analysis module receive sufficient and balanced light. The second light path is incident on the spectral analysis module for spectral imaging of the target object. Furthermore, the optical axis of the beam splitter 2 is aligned horizontally with the adjustable focus lens 1 and the spectral analysis module, and vertically with the first collimating lens group 3, ensuring the stability of the beam splitting optical path. The first collimating lens group 3 collimates the split light into parallel light, providing stable optical path conditions for visible light imaging.

[0048] Image sensor 4 is mounted on the vertical output optical path of beam splitter 2, with its optical center coinciding with the optical axis of that path. It is also co-focal with the spectral analysis unit on the optical path. It is connected to control module 12 via a cable, receiving commands and uploading image data for real-time observation or to assist system focusing. Preferably, image sensor 4 is a high-sensitivity CMOS image sensor. Compared to the previously mainstream CCD image sensors, CMOS image sensors feature low power consumption, small size, high speed, and high integration. Specifically, the CMOS image sensor has a 1920x1080 pixel resolution, a pixel size of 3.75 micrometers, a frame rate of up to 128 frames per second, and a dynamic range of 70dB. Its output wideband two-dimensional image data is used for sharpness evaluation in autofocus algorithms and brightness histogram analysis in automatic exposure algorithms. Depth camera 13 is parallel to the adjustable focus lens 1 and maintains a consistent distance from the object plane. Its field of view is precisely matched to ensure that its field of view completely covers the observation area of ​​the adjustable focus lens 1 at its widest field of view. The depth camera 13 employs active stereo vision, calculating depth information by emitting structured light and analyzing its deformation within the scene. Its depth measurement range is 0.3 meters to 10 meters, with a depth accuracy better than 1% at a distance of 1 meter. It can output a 640x480 pixel resolution depth map at 30 frames per second. This is used to acquire depth information of the target scene, assisting the system in achieving rapid autofocus, and can be fused with spectral data to achieve multi-dimensional spectral imaging.

[0049] The spectral analysis module is sequentially arranged along the optical path, comprising an electronic shutter 5, a slit 6, a second collimating lens group 7, a grating 8, a third collimating lens group 9, and an imaging element 10. These components are aligned and integrated onto the same displacement stage, forming an internally scannable spectral analysis unit. All mechanical mounting structures are made of high-rigidity materials with low thermal expansion coefficients and are fixed by a precision optical platform, ensuring the system maintains an optical alignment accuracy of 0.01 degrees even under wide temperature ranges and vibration environments. The electronic shutter 5 is connected to the control module 12 via a cable and employs a high-response LCD shutter with a switching time of less than 5 milliseconds and an optical density greater than 6.0 in the closed state. It is used to precisely control the light entering the spectral analysis module, enabling automatic acquisition of dark backgrounds without human intervention. Slit 6 restricts the spatial range of the incident light. Slit 6 is perpendicular to the operating direction of the displacement stage, providing a basis for subsequent beam splitting. Grating 8 is a reflective holographic grating used to disperse the collimated parallel beam according to wavelength, achieving precise spectral separation with a spectral resolution of 5 nanometers. The second collimating lens group 7 further collimates the light passing through slit 6, improving the optical signal quality. The third collimating lens group 9 focuses the dispersed light onto the imaging element 10, improving spectral resolution. Specifically, both the second and third collimating lens groups 7 and 9 are composed of multiple aspherical lenses and achromatic lenses, employing advanced optical coating technology, and can collimate or converge the light passing through slit 6, resulting in a converged spot diameter smaller than one pixel. The imaging element 10 is located on the horizontal output optical path of the third collimating lens group 9, transmitting data to the control module 12 via a cable for subsequent spectral data analysis.

[0050] The displacement platform supports the spectral analysis module and drives its internal sweeping motion. The platform employs a precision electric slide rail and a high-precision servo motor-driven mechanism, with a range of ±15 degrees and an angular velocity precisely controlled between 0.1 degrees / second and 10 degrees / second, achieving a repeatability accuracy of 0.005 degrees. Through its internal sweeping motion, the system can acquire continuous line spectral data along the scanning dimension, thereby constructing a hyperspectral data cube with spatial-spectral dimensions. The inertial measurement module 11 integrates a triaxial accelerometer, a triaxial gyroscope, and a triaxial magnetometer. Specifically, the triaxial accelerometer uses MEMS technology to measure the linear acceleration of the system in the X, Y, and Z directions. The triaxial gyroscope is a MEMS vibrating gyroscope used to measure the angular velocity of the system around the X, Y, and Z axes. The triaxial magnetometer uses an anisotropic magnetoresistive sensor to measure the Earth's magnetic field strength at the system's location. During installation, the X-axis of the inertial measurement module 11 is parallel to the major axis of the slit 6, and the Y-axis and Z-axis are perpendicular to each other on both sides of the X-axis. The inertial measurement module 11 maintains a stable relative position with the other modules and monitors the system attitude in real time, such as vibration and tilt, providing raw data for attitude correction of the spectral data.

[0051] The control module 12, as the core of the system, integrates a computing chip and is electrically connected to the adjustable focus lens 1, image sensor 4, electronic shutter 5, imaging element 10, inertial measurement module 11, and depth camera 13. It is used to execute autofocus algorithms, automatic exposure control, data acquisition synchronization, and multi-source data fusion and preprocessing, and coordinate the timing control, data acquisition, autofocus, attitude compensation calculation and other functions of the entire system.

[0052] In this embodiment, when the system enters the autofocus stage, the control module 12 issues a command to drive the optical lens to move along the optical axis in preset steps. With each step, the depth camera 13 acquires a visible light image of the current foreground depth position. Multi-region sharpness evaluation processes this image and divides it into nine non-overlapping rectangular regions, located at the four corners, the midpoints of the four sides, and the center of the image, respectively. Then, the Laplacian gradient energy value is calculated independently for each region using the following formula: .

[0053] in, Let I(x,y) be the sharpness evaluation value of the i-th region, and let I(x,y) be the grayscale value of the pixel in that region. For the Laplace operator.

[0054] The regional weighting coefficients are distributed inversely proportional to the Euclidean distance between the region's center point and the image center. A weighting coefficient of 1 is applied when the distance is 0, and a weighting coefficient of 0.5 is applied when the distance is half the length of the image diagonal. The weighted sum is then used to obtain the global sharpness evaluation value, calculated using the following formula: ,in, Let be the weight coefficient for the i-th region.

[0055] The control module 12 records the global sharpness evaluation value at each step, performs gradient climbing search along the optical axis, and stops moving when the global sharpness evaluation value reaches a local maximum and the rate of change of the evaluation values ​​of the three adjacent steps is less than a preset convergence threshold, thus completing autofocus. The multi-region weighted evaluation strategy effectively addresses the focusing requirements of non-planar targets or complex depth-of-field scenes, avoiding the problem of local sharpness and global blur, and ensuring the spatial consistency of spectral data throughout the entire field of view.

[0056] After focusing is complete, the system enters the formal acquisition phase. Control module 12 locks the focus position, applies optimal exposure parameters, and simultaneously starts depth camera 13 and adjustable lens 1. The depth camera is adjusted according to the object distance-focal length correspondence table to acquire the formal spectral image and spatial reference image. After acquisition, control module 12 executes the data post-processing procedure. First, the acquired raw spectral image is subtracted from the dark field image to perform noise subtraction, eliminating the influence of dark current and readout noise. Second, based on the pitch and roll angle data recorded by the spatial pose sensing unit, a three-dimensional rotation matrix is ​​constructed.

[0057] in, and These are the rotation matrices about the roll axis and the pitch axis, respectively. and This corresponds to the angle value.

[0058] The matrix is ​​projected onto a two-dimensional image plane to generate affine transformation parameters. An affine transformation is then performed independently on each band of the hyperspectral data cube to eliminate trapezoidal distortion or perspective distortion introduced by device tilt. The final output is a corrected hyperspectral data cube, ensuring the accuracy of spatial geometric information and the physical comparability of the spectral data.

[0059] Furthermore, this embodiment provides a portable autofocus hyperspectral imaging method for performing portable autofocus hyperspectral imaging systems. Please refer to [link to relevant documentation]. Figure 2 The method includes the following steps:

[0060] S101: System Initialization and Dark Background Acquisition. After the system is powered on, based on the data from the inertial measurement module 11, once the system attitude is determined to be stable, the host computer software clicks on dark background acquisition, controlling the electronic shutter 5 to close. The spectral analysis module returns to its mechanical zero point along with the displacement stage, and there is no light signal in the optical path of the spectral analysis module. The imaging element 10 acquires dark current data under no-light conditions, transmits it to the control module 12, and stores it as dark background data. This step aims to automate dark field data acquisition and eliminate manual intervention. The control module 12 continuously monitors the attitude data from the inertial measurement module 11, including the linear acceleration of the triaxial accelerometer, the angular velocity of the triaxial gyroscope, and the magnetic field strength of the triaxial magnetometer. In this embodiment, the attitude stability judgment criteria are as follows: within a preset continuous time period, such as 2 seconds, the angular velocity of the three-axis gyroscope is less than a preset threshold of 0.1 degrees / second in all X, Y, and Z axes, indicating that the system has not undergone significant rotation; simultaneously, the acceleration readings of each axis of the three-axis accelerometer fluctuate within a preset error range of ±0.05g, indicating that the system is in a relatively static state with no obvious linear motion; in addition, the overall tilt angle of the system is calculated by fusing accelerometer and gyroscope data, such as by using a Kalman filter or complementary filter. Within a preset allowable range of ±1 degree, the system is ensured to be basically horizontal or maintain a preset tilt attitude. Once the system attitude meets all the above stability criteria, the control module 12 sends a closing command to the electronic shutter 5, completely blocking the optical path of the spectral analysis module. During the period when the electronic shutter 5 is closed, the imaging element 10 performs multiple (e.g., 10) image acquisitions at a preset integration time (e.g., 100 milliseconds). Subsequently, the control module 12 performs pixel-by-pixel averaging on the acquired multiple dark current images to effectively reduce the inherent random noise and fixed-pattern noise of the sensor. The averaged dark current data is then stored in the system's internal non-volatile memory as a reference for dark background subtraction in subsequent hyperspectral data processing. Step S101 achieves complete automation of dark field data acquisition, eliminating the reliance on manual lens obstruction, thereby significantly improving the system's automation level and its suitability for unattended scenarios.

[0061] S102: Autofocus. The adjustable lens 1 initiates light signal acquisition. The target light signal is split by the beam splitter 2. One beam is collimated vertically by the first collimating lens group 3 and incident on the image sensor 4, which outputs a real-time preview image. The reference plate is fixed near the center height of the target object to confirm that the target object occupies a high proportion of the field of view. At the same time, the depth camera 13 works synchronously (the optical axis is parallel to the lens, and the field of view covers the lens's field of view), outputting a real-time depth image. Coarse focusing is performed based on the depth camera 13 or the focusing range. Then, the fine focusing range is determined based on the coarse focusing result. The sharpness algorithm is called to determine the focal length Fx corresponding to the highest sharpness value in the records. This Fx is the sharpest focal length position. The adjustable lens 1 is adjusted to the focal length Fx.

[0062] Adjusting the focal length of the adjustable lens 1 through two stages, coarse adjustment and fine adjustment, includes:

[0063] Please see Figure 3 The control module 12, based on the depth data collected in real time by the depth camera 13, statistically analyzes the range of depth values ​​above a preset ratio, determines the target distance range using its center value, queries a preset object distance-focal length correspondence table, and obtains the first coarse adjustment focal length F1; it controls the adjustable lens 1 to traverse the entire focal length range with a step length, and after each adjustment, it collects an image through the image sensor 4 and calculates the sharpness value, recording the second coarse adjustment focal length F corresponding to the maximum sharpness value; it then determines one of the first coarse adjustment focal length F1 and the second coarse adjustment focal length F as the final coarse adjustment focal length.

[0064] The step of determining a final coarse focal length from the first coarse focal length F1 and the second coarse focal length F includes:

[0065] Calculate the absolute value of the difference between the first coarse focus distance F1 and the second coarse focus distance F;

[0066] If the absolute value is less than or equal to the preset threshold, the first coarse adjustment focal length F1 will be used as the final coarse adjustment focal length.

[0067] If the absolute value is greater than a preset threshold, the second coarse adjustment focal length F is used as the final coarse adjustment focal length. For example, based on the depth data collected by the depth camera 13, the range containing more than 60% of the depth values ​​is statistically analyzed, and the center value of this range is defined as X. X ± 50mm is taken as the target distance interval X1. The preset "Distance to be Measured Object S - Focal Length F Correspondence Table M" is queried. It should be noted that in the table, S records the sharp focal length F obtained by manual focusing every 10cm interval, such as S=150mm corresponding to F=15, S=160mm corresponding to F=22, etc. The closest F value within the X1 interval is found and recorded as F1. If there is a deviation in depth-assisted positioning, the lens focal length is traversed in a step of 10 within the 0-255 range. After each adjustment, an image is acquired through a 4-CMOS sensor, and the sharpness algorithm is used to calculate the sharpness value. The correspondence between focal length and sharpness is recorded in array E. After the traversal is complete, the focal length with the highest sharpness in E is selected and recorded as F, which is used as the final coarse adjustment focal length. If F1 is close to F, F is used as the standard.

[0068] Please see Figure 4The coarse-adjustment + fine-adjustment focusing strategy uses the final focal length obtained from the coarse adjustment as the center, and defines a fine-adjustment range containing N focal length points. It iterates through this range with a second step size, acquiring an image at each focal length point and calculating the sharpness value using a sharpness algorithm. The focal length with the highest sharpness value is selected as the optimal focusing focal length Fx. This strategy combines the ability to quickly locate the target object using depth information with the ability to precisely fine-tune details using image sharpness assessment, effectively solving focusing challenges in complex non-planar scenes. Examples include focusing on undulating terrain with vegetation canopies or under complex industrial equipment. For instance, using the focal length F or F1 obtained from the coarse adjustment as the center, a fine-adjustment range is defined: if it's F, F-5 to F+4 (10 focal length values) are selected; if it's F1, F1-10 to F1+9 (20 focal length values) are selected, and this range is iterated with a step size of 1. After each adjustment, an image is acquired using a 4-CMOS sensor, and the sharpness algorithm is called to calculate and record the correspondence between focal length and sharpness in an array H. After traversing the data, select the focal length with the highest sharpness from H and denot it as Fx. This is the position of the sharpest focal length.

[0069] In this embodiment, the step of calling the sharpness algorithm to determine the optimal focus focal length includes:

[0070] The controller performs grayscale normalization on the images acquired by image sensor 4; it maps pixel information to a uniform dynamic range, eliminates interference from environmental factors such as lighting on feature extraction, and ensures consistency in subsequent analysis.

[0071] Multi-directional feature extraction is performed on the standardized image to enhance edge and detail features. Specifically, multi-directional feature extraction techniques can be used to enhance edge and detail information in the image, such as convolution kernel operations, to highlight grayscale variation features in different dimensions, such as horizontal, vertical and diagonal directions.

[0072] Statistical analysis is performed on the enhanced feature information, quantifying the dispersion of its distribution, such as the variance in the fluctuation range of the feature data, as a sharpness evaluation index. A higher index value indicates more significant image details and corresponding higher image sharpness. The corresponding focal length and sharpness are recorded, and this is used as a basis for objectively determining focus accuracy. For example, first, an image in img format is acquired; the image data is read in binary format, obtaining its width w and height h, and the data is normalized to 0-255. Specifically, the entire image is iterated through and stored in the q array. Total traversal Next, find the maximum value maxVal, then normalize and store it. In the middle, p[0] = q[0] × 255 / maxVal, and calls are made sequentially. Furthermore, we focus on the clarity in four directions, while also paying attention to the diagonal value. This is achieved through a convolution operation. First, we obtain a convolution kernel[3][3] and iterate through it. The convolutional values ​​are calculated sequentially with the kernel to obtain the convolutional values. Then to The average value M is calculated for all values ​​in the mean. Specifically, t[1][0] + t[2][0] + ... + t[w-2][0] = mean, and this is repeated h-2 times. The sum is stored in mean, and the number of data points is counted as count = (w-2) × (h-2). Finally, M = mean / count. The variance N is calculated using the following method:

[0073]

[0074] Finally, N = var / count, where N is the basis for judging sharpness; the larger N is, the sharper the image.

[0075] S103: Auto exposure, hyperspectral camera starts, after autofocus is complete, electronic shutter 5 is opened to allow light signals to enter the spectral analysis module. Based on the autofocus result, the current exposure time R at the optimal focal length is obtained, and the exposure time U is automatically adjusted using the histogram.

[0076] Please refer to Figure 5 Dynamically adjusting and determining the optimal exposure time R includes:

[0077] Obtain the image histogram at the current exposure time;

[0078] If the pixel values ​​above the first preset ratio are distributed to the left of the preset target value DN, then increase the exposure time;

[0079] If the pixel values ​​above the first preset ratio are distributed to the right of the preset target value DN, then reduce the exposure time;

[0080] Repeat the above steps until pixel values ​​above the first preset ratio are distributed in the interval near the preset target value DN and form a near-normal distribution;

[0081] The final repeated exposure time is compared with the exposure time recommended based on the focus results, and the smaller of the two is taken as the optimal exposure time for hyperspectral data acquisition.

[0082] Exemplarily, the X-axis of the histogram ranges from 0 to 4095, and the Y-axis corresponds to the number of occurrences of each data on the X-axis. At the same time, we have a target value DN for different scenarios. Data is continuously collected, and each time a frame of data is collected, it is counted. If more than 60% of the data is to the left of the DN value, the exposure time is low, and it is increased by 200 us. If more than 60% of the data is to the right of the DN value, the exposure time is high, and it is decreased by 200 us. It should be noted that if the result value is negative, subtract 100; if it is still negative, subtract 50 until it becomes positive. If 60% of the data is near the DN value, forming a quasi-normal distribution, then the current exposure time is appropriate. By comparing R and U, if R > U, take U; if R < U, take R. That is, the minimum value of the two is taken as the final exposure time. This selection aims to preferentially avoid image saturation because saturation will cause irreversible loss of spectral information, while slight underexposure can be compensated to a certain extent through subsequent processing, such as gain adjustment. This dynamic adjustment process enables real-time adaptation to rapidly changing environmental lighting conditions, such as cloud changes and the movement of target object shadows, ensuring the best signal-to-noise ratio and dynamic range utilization of hyperspectral data in various scenarios.

[0083] S104: Data acquisition. After the autofocus and automatic exposure parameters are determined, the control module 12 sends a start command to the displacement platform, driving the displacement platform to perform a preset internal swing scan motion. The swing scan motion is performed at a constant angular velocity, for example, 3 degrees per second, ensuring uniform coverage of the target scene throughout the scan stroke, thus avoiding scan gaps or overlaps. The optical signal on the surface of the target object passes through the horizontal optical path split by the adjustable focus lens 1 and the beam splitter 2, and sequentially passes through the spectral analysis module: cooperating with the built-in displacement stage to achieve three-dimensional coverage of the two-dimensional space and spectrum of the target area, generating a hyperspectral data cube. At the same time, the control module 12 controls the depth camera 13 and the inertial measurement module 11 to collect data at the same frequency as the hyperspectral image and transmits the data to the control module 12.

[0084] S105: Multi-source data fusion and preprocessing. Perform geometric correction, dark background subtraction, uniformity correction, noise reduction processing, and reflectance calculation on the collected hyperspectral data, and fuse the processed hyperspectral data with the depth data to generate multi-dimensional spectral data.

[0085] After the data acquisition is completed, the control module 12 performs a series of preprocessing operations on the stored hyperspectral data to improve the data quality and analysis accuracy.

[0086] First, the control module 12 performs geometric correction. This geometric correction is based on the precise angle data recorded by the displacement platform during the scanning process and the attitude data provided in real time by the inertial measurement module 11. The control module 12 comprehensively utilizes this spatial and attitude information to establish an affine transformation model or a more complex perspective transformation model. This model integrates the tilt, translation, and rotation that may occur during the acquisition process, remapping the pixel positions in the original hyperspectral image to correct geometric distortions introduced by changes in system attitude or uneven scanning motion. For example, if the inertial measurement module 11 detects a 3-degree pitch angle at a certain scanning moment, the affine transformation matrix will include the corresponding rotation component, making the output image a corrected front view, ensuring the accurate spatial position of ground features in the spectral image. The geometric correction algorithm uses bilinear interpolation or cubic convolution interpolation for pixel resampling to preserve image details to the greatest extent possible.

[0087] Next, the control module 12 performs dark background subtraction. Dark background subtraction is achieved by subtracting the dark current data pre-stored in step S101 from each acquired hyperspectral data frame pixel by pixel. This operation effectively removes the inherent dark current noise and some fixed-mode noise of the sensor, significantly improving the signal-to-noise ratio of the spectral data.

[0088] Next, control module 12 performs uniformity correction. Uniformity correction uses a standard white board reference method. Before or during data acquisition, for example, each time the system is powered on or at regular intervals, the system scans a standard white board with known reflectivity to obtain its hyperspectral image data as reference white board spectral data. A radiometric calibration coefficient matrix is ​​generated by calculating the response coefficient of each pixel to the standard white board at different wavelengths—that is, the reciprocal of the ratio of the true reflectivity of the standard white board to its actual measured value. Then, the acquired hyperspectral data (after dark background subtraction) is multiplied pixel-by-pixel and wavelength-by-wavelength by the radiometric calibration coefficient matrix to correct for non-uniformity errors caused by inconsistent responses at different pixels and wavelengths, thus achieving radiometric calibration and converting sensor readings into radiance values.

[0089] Next, control module 12 performs noise reduction processing. This noise reduction process employs the Savitzky-Golay smoothing algorithm. This algorithm effectively filters out random noise in the hyperspectral data by performing local polynomial fitting and smoothing filtering on the spectral curve of each wavelength channel, while preserving the shape and detail of the spectral features to the maximum extent, avoiding the spectral feature broadening that may be caused by traditional moving average filtering. The Savitzky-Golay smoothing algorithm is typically configured with a window size of 15 wavelength points, for example, 7 wavelength points before and after, and a polynomial order of 3.

[0090] Next, the control module 12 performs reflectance calculation. This reflectance calculation is achieved by performing a wavelength-by-wavelength, pixel-by-pixel ratio operation between the target spectral data (after the aforementioned geometric correction, dark background subtraction, uniformity correction, and noise reduction processes) and the pre-acquired reference white board spectral data (also after dark background subtraction and uniformity correction). Finally, the control module 12 fuses the processed hyperspectral data with the depth data to generate multidimensional spectral data.

[0091] The above description is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined in the claims, they should all fall within the protection scope of the present invention.

Claims

1. A portable autofocus hyperspectral imaging system, characterized in that, It includes a spectral data acquisition module, a spectral analysis module, an inertial measurement module (11), a control module (12), and a displacement platform; The spectral data acquisition module includes a focusable lens (1), a beam splitter (2), a first collimating lens group (3), and an image sensor (4). The focusable lens (1) is located at the incident end of the system's optical path, and the beam splitter (2) is located at the exit end of the focusable lens (1). It is used to divide the incident optical path into a first optical path and a second optical path. The first optical path is collimated by the first collimating lens group (3) and then incident on the image sensor (4). The second optical path is incident on the spectral analysis module. The spectral data acquisition module also includes a depth camera (13). The depth camera (13) is parallel to the optical axis of the focusable lens (1), and its field of view covers the field of view of the focusable lens (1). It is also electrically connected to the control module (12). The spectral analysis module is provided with an electronic shutter (5), a slit (6), a second collimating lens group (7), a grating (8), a third collimating lens group (9), and an imaging element (10) in sequence along the optical path, and the electronic shutter (5), slit (6), second collimating lens group (7), grating (8), third collimating lens group (9), and imaging element (10) are aligned in position; The displacement platform is used to support the spectral analysis module and drive the spectral analysis module to perform internal oscillating sweep motion; The inertial measurement module (11) integrates a triaxial accelerometer, a triaxial gyroscope and a triaxial magnetometer. The X-axis of the inertial measurement module (11) is parallel to the major axis of the slit (6), and the Y-axis and Z-axis are perpendicular to the X-axis. The control module (12) is electrically connected to the adjustable focus lens (1), image sensor (4), electronic shutter (5), imaging element (10) and inertial measurement module (11) to perform autofocus algorithm, automatic exposure control, data acquisition synchronization and multi-source data fusion and preprocessing. The autofocus algorithm includes a step of determining the coarse adjustment focal length through two detections to ensure the basis for subsequent fine adjustment. After determining the coarse adjustment focal length through two detections, the most suitable coarse adjustment focal length is selected, including: obtaining the first coarse adjustment focal length F1 based on the depth camera (13), obtaining the second coarse adjustment focal length F with the highest sharpness by traversing the focal length range based on the image sensor (4); calculating the absolute value of the difference between F1 and F. If the absolute value is less than or equal to the preset threshold, F1 is taken as the final coarse adjustment focal length; if the absolute value is greater than the preset threshold, F is taken as the final coarse adjustment focal length; and fine adjustment is performed with the final coarse adjustment focal length as the center to determine the optimal focus focal length Fx.

2. A portable autofocus hyperspectral imaging method, characterized in that, The method using the system of claim 1 includes the following steps: After the system attitude is stabilized based on the data from the inertial measurement module (11), the electronic shutter (5) is controlled to close, and the dark current data is collected and stored by the imaging element (10). Based on the depth image obtained by the depth camera (13) and the image information collected by the image sensor (4), the focal length of the adjustable lens (1) is adjusted through two stages: coarse adjustment and fine adjustment. The sharpness algorithm is called to determine the optimal focus focal length. After autofocus is completed, the electronic shutter (5) is opened. Based on the histogram of the focused image, the optimal exposure time is dynamically adjusted and determined. Among them, the autofocus algorithm includes a step of determining the coarse adjustment focal length through two detections to ensure the basis for subsequent fine adjustment. After determining the coarse adjustment focal length through two detections, the most suitable coarse adjustment focal length is selected, including: obtaining the first coarse adjustment focal length F1 based on the depth camera (13), obtaining the second coarse adjustment focal length F with the highest sharpness by traversing the focal length range based on the image sensor (4); calculating the absolute value of the difference between F1 and F. If the absolute value is less than or equal to the preset threshold, F1 is taken as the final coarse adjustment focal length; if the absolute value is greater than the preset threshold, F is taken as the final coarse adjustment focal length; fine adjustment is performed with the final coarse adjustment focal length as the center to determine the optimal focus focal length Fx. The displacement platform is activated to perform an internal oscillation scan, simultaneously acquiring hyperspectral data, depth data, and inertial measurement data; The acquired hyperspectral data is subjected to geometric correction, dark background subtraction, uniformity correction, noise reduction and reflectance calculation. The processed hyperspectral data is then fused with depth data to generate multidimensional spectral data.

3. The portable autofocus hyperspectral imaging method according to claim 2, characterized in that, The steps involved in using the sharpness algorithm to determine the optimal focus focal length include: Perform grayscale normalization on the acquired images; Multi-directional feature extraction is performed on the standardized image to enhance edge and detail features; Calculate the feature statistics of the enhanced image as a sharpness evaluation index.

4. The portable autofocus hyperspectral imaging method according to claim 2, characterized in that, Dynamically adjusting to determine the optimal exposure time includes: Obtain the image histogram at the current exposure time. If the pixel values ​​above the first preset ratio are distributed to the left of the preset target value DN, increase the exposure time. If the pixel values ​​above the first preset ratio are distributed to the right of the preset target value DN, decrease the exposure time. Repeat the previous step until the pixel values ​​above the first preset ratio are distributed in the interval near the preset target value DN and form a near-normal distribution. The final repeated exposure time is compared with the exposure time recommended based on the focus results, and the smaller of the two is taken as the optimal exposure time for hyperspectral data acquisition.

5. The portable autofocus hyperspectral imaging method according to claim 2, characterized in that, The geometric correction is based on the displacement platform swing angle data and the attitude data of the inertial measurement module (11), and the affine transformation model is used to correct image distortion; the dark background subtraction is achieved by subtracting the pre-stored dark current data from the acquired hyperspectral data; the uniformity correction adopts the standard white board reference method, and radiometric calibration is achieved by calculating the reciprocal of the response coefficient of each pixel; the noise reduction process adopts the Savitzky-Golay smoothing algorithm.

Images