Hyperspectral acquisition method and device with real-time automatic focusing

Abstract

The application discloses a hyperspectral acquisition method and device with real-time automatic focusing, comprising the following steps: step one, collecting dark background pixel signals; step two, automatic focusing; step three, repeating step two until the requirements are met, recording automatic focusing data and performing spectral acquisition to obtain original spectral data; step four, calculating a full-range object distance matrix of the original spectral data based on the phase difference obtained in step three; step five, correcting the original spectral data based on the full-range object distance matrix to obtain corrected spectral data; and step six, calculating the reflectivity of a target object to be measured based on the corrected spectral data and the dark background pixel signals, so that the hyperspectral acquisition with real-time automatic focusing is realized. The method innovatively analyzes the phase difference, efficiently processes light changes and target shifts in a dynamic scene, and improves the accuracy and stability of focusing by combining the two, effectively solves the problem that a scanning target is easy to be out of focus in a dynamic scene, and realizes efficient adaptation to the dynamic scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral acquisition, in particular to a hyperspectral acquisition method and device with real-time automatic focusing. BACKGROUND

[0002] The focusing system of a hyperspectral camera is the core support for guaranteeing its spectral resolution and spatial resolution, and its core components include hardware structure and algorithm modules. By precisely regulating the relative positions of the lens and the photosensitive element, the target light can be clearly imaged, and in dynamic monitoring and video hyperspectral scenarios, it is also the key link to achieve high-quality imaging.

[0003] If the focusing system has problems such as insufficient precision and slow response, it is easy to cause distortion of spectral data and blurring of spatial details, and thus cause misjudgment in applications such as agricultural pest identification and environmental monitoring, affecting the accuracy of decision-making. Therefore, it is crucial to guarantee the efficiency and accuracy of the focusing system.

[0004] To meet the focusing needs of hyperspectral cameras, current mainstream technical solutions are divided into two categories: manual focusing relies on the experience of operators to adjust the lens, and the focus position is determined by observing the image sharpness; automatic focusing is mainly contrast-based, which determines the focus point by repeatedly adjusting the lens and calculating the contrast extremum of the image. However, both solutions have obvious limitations: manual focusing lacks a unified standard, and in complex environments such as sudden changes in light and target shaking, it is easy to cause data deviation due to human error; contrast-based automatic focusing is extremely slow, and it often takes more than a minute to refocus after the object moves, making it difficult to meet the real-time needs of video hyperspectral. These factors have seriously restricted the application of hyperspectral cameras in dynamic scenarios, and have become a bottleneck that needs to be broken through in the focusing technology of current hyperspectral cameras. SUMMARY

[0005] The present application provides a hyperspectral acquisition method and device with real-time automatic focusing, which can respond to the dynamic changes of the target in the line scanning process in real time, ensure that the imaging of each line scanning can accurately correspond to the current position of the target object, effectively solve the problem of easy defocusing of the scanning target in dynamic scenarios, realize efficient adaptation to dynamic scenarios, and solve the problems mentioned in the background technology.

[0006] The application provides the following technical scheme: a hyperspectral acquisition method of real-time automatic focusing, comprising the following steps: step 1, collecting a dark background pixel signal; step 2, performing automatic focusing after ensuring that an imaging area of a lens covers a target object to be measured and a reference plate, the automatic focusing comprising the following steps: acquiring an image and adjusting brightness to reach a preset threshold; selecting a region from the center of the image and extracting a right view sequence and a left view sequence of the selected region image, and calculating a phase difference; calculating an image plane offset and a lens movement amount based on the phase difference, determining a focusing direction, and performing automatic focusing of the lens; step 3, repeating the automatic focusing process of step 2 until the phase difference error is less than a set threshold, then ending the automatic focusing, recording the automatic focusing data, and performing spectral acquisition to obtain original spectral data; step 4, the pixel size of the selected region image is inconsistent with the pixel size of the image acquired during the automatic focusing, the coordinates of the selected region image are mapped to the coordinates of the original spectral data, the object distance matrix of an image with the same size as the selected region image in the original spectral data is calculated based on the final phase difference obtained in step 3, and the full-frame object distance matrix of the original spectral data is obtained through interpolation; step 5, correcting the original spectral data based on the full-frame object distance matrix to obtain corrected spectral data; and step 6, calculating the corrected reference plate spectral data and the corrected target object spectral data based on the corrected spectral data and the dark background pixel signal, and calculating the reflectivity of the target object to realize the real-time automatic focusing hyperspectral acquisition.

[0007] As an optional solution of the real-time automatic focusing hyperspectral acquisition method, when the phase difference is calculated, the center position coordinates of the selected region image are extracted, the right view sequence and the left view sequence are defined based on the center position coordinates, the correlation of the right view sequence and the left view sequence is calculated, the phase difference is obtained by taking the pixel deviation number corresponding to the maximum correlation as the phase difference, the positive and negative of the pixel deviation number determine the lens adjustment direction, the pixel size of the selected region image and the pixel size of the original spectral data are obtained when the coordinates of the selected region image are mapped to the coordinates of the original spectral data, the coordinates of the selected region image in the original spectral data are calculated through any point coordinates of the selected region image, the selected region image is sampled to the pixel grid of the original spectral data through the bilinear interpolation algorithm based on the resolution of the original spectral data, and the full-frame object distance matrix of the original spectral data is obtained by calculating the object distance matrix of the selected region image and obtaining the full-frame object distance matrix of the original spectral data through the bilinear interpolation algorithm based on the mapping relationship between the selected region image and the original spectral data.

[0008] As an optional solution of the real-time automatic focusing hyperspectral acquisition method, the pixel values and distances of the center pixels of the original spectral data are used to calibrate the pixels of other regions in the image to obtain the corrected spectral data, the reference plate region is selected from the corrected spectral data, the corrected reference plate spectral data is calculated, and the reflectivity of the target object is calculated through the corrected reference plate spectral data and the corrected spectral data.

[0009] The application discloses a hyperspectral acquisition device with real-time automatic focusing, and applies the real-time automatic focusing hyperspectral acquisition method to the device.

[0010] The application has the following advantages:

[0011] 1. The real-time automatic focusing hyperspectral acquisition method relies on innovative phase difference analysis logic to efficiently process light changes and target shifts in a dynamic scene, thereby greatly improving the accuracy and stability of focusing and providing strong support for clear imaging in dynamic monitoring.

[0012] 2. The real-time automatic focusing hyperspectral acquisition method is specially adapted to the hyperspectral line scanning imaging mode, can accurately match the motion state of a scanning target object, and enables the focusing system to respond to the dynamic changes of the target in the line scanning process in real time, thereby ensuring that the imaging of each line scanning can accurately correspond to the current position of the target object, effectively solving the problem that the scanning target in a dynamic scene is easy to be out of focus, and realizing efficient adaptation to the dynamic scene.

[0013] 3. The real-time automatic focusing hyperspectral acquisition device is combined with an adaptive base and shading material on the hardware, and a structure design for reducing light interference is adopted, so as to provide a stable basis for light reception and signal acquisition.

[0014] 4. The real-time automatic focusing hyperspectral acquisition device is combined with the real-time automatic focusing hyperspectral acquisition method of the application, can be repeatedly adjusted and verified, and is installed through a precise positioning and flexible fixing mode, so that the accurate matching of the system and the sensor can be ensured, damage to core components can be avoided, and repeated adjustment in the later period is supported; after installation, the system performance can be confirmed and potential problems can be excluded through steps such as shading effect testing and phase consistency verification, so that stable performance can be maintained in long-term use, and the application scenarios and service life of the technical solution are further widened. Attached Figure Description

[0015] Fig. 1 This is a schematic diagram of the overall real-time autofocus hyperspectral acquisition device in an embodiment of the present invention.

[0016] Fig. 2 This is a schematic diagram of the pixel design in the autofocus module of an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figs. 1-2One type of real-time autofocus hyperspectral acquisition device includes: an adjustable lens 1, a beam splitter 2, a collimating lens group 3, an autofocus module 4, a hyperspectral module 5, and a main control module 6. The adjustable lens 1 is installed at the front end of the system's optical path and is responsible for acquiring incident light signals from dynamic scenes. It is connected to the main control module 6 via a cable, receives focus adjustment commands from the main control module 6, and adjusts the focus in real time based on the phase difference data fed back by the autofocus module 4, providing clear and stable incident light for subsequent beam splitting and spectral analysis, adapting to the needs of dynamic scenes such as target movement and changes in illumination. The beam splitter 2 is deployed on the output optical path of the adjustable lens 1, splitting the incident light into two paths: one path is guided along the vertical optical path to the collimating lens group 3 and the autofocus module 4 for focus status detection; the other path is transmitted along the horizontal optical path to the hyperspectral module 5 for spectral imaging analysis. Through optical axis alignment design, the module is coaxial with the hyperspectral module 5 in the horizontal direction and with the collimating lens group 3 in the vertical direction, ensuring that the autofocus module 4 and the hyperspectral module 5 are confocal. This guarantees the stability of the beam-splitting optical path and supports the parallel operation of focus detection and spectral acquisition. The collimating lens group 3 is installed between the beam splitter and the autofocus module 4, collimating the scattered light after beam splitting into parallel light. This optimizes the incident angle of light in the autofocus module 4, improves the pixel group's ability to distinguish the direction of light, and ensures that the right-viewing pixel receives light from the right side and the left-viewing pixel receives light from the left side, providing high-quality raw data for phase difference calculation. The autofocus module 4 is integrated at the end of the imaging link, adjacent to the sensor area. It includes a substrate, a light-shielding layer, and a special pixel group. The substrate is made of a 10-50μm transparent PET film with a light transmittance of >95%. It has the advantages of high temperature resistance and easy bonding. The light-shielding layer is evenly spaced on the substrate. The light-shielding layer can be a 0.8-1μm vacuum-deposited aluminum layer with a light-shielding rate of >99%, suitable for batch production scenarios, or black photoresist with an accuracy of ±0.1μm, which can be selected for small-batch laboratory scenarios. The edges of the light-shielding layer are rounded with 0.1μm to suppress stray light diffraction. The four corners are equipped with 1μm line width cross marks to assist in installation alignment and ensure an error of ≤±1μm. The autofocus module 4 uses a pixel group design where the right-view pixel receives light only from the right side of the lens, and the left-view pixel receives light only from the left side of the lens. This generates independent grayscale sequences, such as the right-view sequence [50, 200, 50, 200, 50, 200] and the left-view sequence [200, 50, 200, 50, 200, 50]. These sequences are uploaded to the main control module 6 via cable. The main control module 6 runs a correlation algorithm, calculates the phase difference using the corr(d) formula, derives the physical offset distance and lens adjustment amount, and adjusts the adjustable focus lens 1 in real time. The hyperspectral module 5 receives the beam splitter signal from the beam splitter 2. It incorporates a slit, dispersion, and focusing element to decompose the incident light into spectral information and collect it. It is connected to the main control module 6 via cable and performs an internal push-scan under the control of the main control module 6. While ensuring clear focus with the autofocus module 4, it outputs high-resolution spectral data to support spectral analysis in dynamic scenes.The main control module 6, as the core of the system, integrates a computing chip and is connected to the adjustable focus lens 1, autofocus module 4, and hyperspectral module 5 via cables. On one hand, it runs a phase difference algorithm to analyze the data from the autofocus module 4 and derive focus adjustment commands. On the other hand, it coordinates the timing control and data synchronization of the entire system, linking the focus status with spectral data in real time. This enables synergistic optimization of accurate focus and effective spectral analysis in dynamic scenarios, ensuring system reliability and operational efficiency.

[0019] The real-time autofocus hyperspectral acquisition process of the hyperspectral acquisition device includes:

[0020] Step 1: Return the adjustable focus lens 1 to zero. The autofocus module 4 and hyperspectral module 5 will complete initialization. Replace the lens cap. Start the hyperspectral module 5 to acquire the dark background pixel signal D. k .

[0021] Step 2: Turn on the light source and place a standard white board with a known reflectivity r(λ) near the target object as a reference board. Ensure that the lens imaging area covers both the target object and the reference board, and that the plane of the reference board is as consistent as possible with the target object. Perform autofocus. It should be noted that the autofocus module 4 and the hyperspectral module 5 share the same adjustable lens 1 for imaging, and their field of view is completely identical. The specific autofocus process is as follows:

[0022] The autofocus module 4 is used to acquire images and calculate the brightness of the acquired images. If the brightness does not reach the preset threshold, the exposure time is increased until the brightness of the image acquired by the autofocus module 4 reaches the requirement.

[0023] The center selection area of ​​the image acquired by the autofocus module 4 is used to obtain the selection image, which can be 64×64 in size. The autofocus module 4 outputs the right view sequence A and the left view sequence B. The definition rules for the right view sequence and the left view sequence are as follows: the left view / left view center column is selected according to the odd and even column rules. If the width of the phase sensor is w columns, from w1 to wn, the odd columns are the left view sequence and the even columns are the right view sequence. At this time, the position of the focus frame is (x0, y0, 64, 64), and the center position can be calculated as (x0+32, y0+32). If x0+32 is odd, it means that the current sequence is the left view sequence. Only x0+32 and x0+32+1 need to be selected as the right view sequence. If x0+32 is even, it means that the current sequence is the right view sequence. Only x0+32 and x0+32-1 need to be selected as the left view sequence. Phase difference estimation is achieved by maximizing correlation through correlation calculation. Integer offsets are used, and the correlation is calculated as corr(d) = [A[i]*B[i+d]], 0≤i+d≤N, where N represents the length of the right-view sequence A or left-view sequence B minus 1 (i.e., the last valid index), d is the pixel deviation, and N is the number of pixels in both right-view sequence A and left-view sequence B. The pixel deviation d corresponding to the maximum correlation between all current right-view sequences A and left-view sequences B is obtained. Based on the pixel deviation d, the image plane offset and lens movement are calculated: the image plane offset is Δx = d*s, where s is the pixel size, and the corresponding lens movement ΔL = Δx*f / v, where f is the focal length and v is the image distance. Focus direction mapping: when d > 0, the focus is behind the subject, and the lens is adjusted forward; when d < 0, the opposite is true. At this time, the main control module 6 drives the adjustable lens 1 to map and fine-tune according to the focusing direction, periodically recalculate the phase difference until the phase difference error is less than the set threshold, and then locks and records the current exposure time and focal length to complete the automatic focusing.

[0024] After autofocusing is completed, hyperspectral module 5 performs hyperspectral internal push-scan acquisition to obtain raw spectral data Dn(λ).

[0025] Since the autofocus module 4 and the hyperspectral module 5 share the same adjustable lens 1 for imaging, their fields of view are completely identical. This means that the original spectral data Dn(λ) is inconsistent with the pixel size of the image acquired during autofocus. Therefore, the only factor affecting image registration is the difference in pixel size, and registration cannot be directly performed using pixel coordinates. Thus, the pixel size S of the autofocus module 4 sensor is obtained from the sensor manual. A The pixel size S of the hyperspectral module 5 B The pixel center coordinates of both are (C AX C AY ) and (C BX C BYBecause the lens and focal length are the same, and the image plane is completely identical, the image from the autofocus module 4 can be mapped to the pixel coordinates of the hyperspectral module 5 by calculating the image plane offset distance using the pixel center coordinates. Specifically, (X... A Y A Let (x, y) be one of the coordinate points in the image captured by the autofocus module 4. The coordinates of this point, mapped to the pixel coordinates of the hyperspectral module 5, are (X, Y). The calculation method is as follows:

[0026]

[0027] Since (X, Y) is not an integer, integer coordinates can be calculated by bilinear interpolation of the surrounding non-integer coordinates. Therefore, based on the sensor resolution of the hyperspectral module 5, the image of the autofocus module 4 is sampled to the same pixel grid through bilinear interpolation and other algorithms.

[0028] The object distance can be calculated using the lens imaging formula. Therefore, the object distance matrix of the selected area image acquired by the autofocus module 4 can be obtained. After the image from the autofocus module 4 is mapped to the pixel coordinates of the hyperspectral module 5, the object distance matrices of the two images are identical in size. Therefore, using the same correspondence and interpolation, the object distance matrix of the selected area image mapped onto the original spectral data D under the reference of the hyperspectral module 5 can be obtained. n The object distance matrix of the image (λ) is then used, and the signal value and distance of the center pixel of the image are used to calibrate pixels in other areas. Specifically:

[0029] In the original spectral data D n The reference plate region is selected from (λ) to obtain the reference plate signal value D. nr (λ), and the distance u from the reference plate r Raw spectral data D n (λ) and original spectral distance u n Based on the object distance and the dark background pixel signal D in sequence k To correct the signal, specifically...

[0030] Reference board signal calibration includes:

[0031] First, based on the dark background pixel signal D k To correct the reference board signal and obtain the reference board dark background corrected data DN rf (λ): DN rf (λ) = (D) nr (λ)-D k )*(u r / u xr )^2. u xr The distance to the center signal of the reference board.

[0032] Next, perform distance correction on the reference board to obtain the center signal value D of the reference board. xr (λ), Pixel signal value D outside the center of the reference board yr (λ) and the pixel distance u outside the center of the reference board yr Reference board distance correction signal value DN rw (λ)=DN rf (λ)*(u yr / u xr )^2.

[0033] Original spectral signal correction includes:

[0034] First, based on the dark background pixel signal D k To correct the original spectral signal, we obtain the original spectral data after dark background correction (DN). nf (λ)=(D n (λ)-D k )*(u n / u xf )^2. u xf The distance between the center of the original spectral signal and the object.

[0035] Then perform distance correction on the original spectrum to obtain the original spectral data center signal value D. x (λ), pixel signal value D outside the original spectral data center y (λ) and pixel distance u outside the original spectral data center y The original spectral data is far from the corrected signal value DN y (λ)=DN nf (λ)*(u y / u x )^2.

[0036] Finally, the reflectance of the target object is calculated as: r m (λ)=DN y (λ) / DN rw (λ)*r(λ), where r(λ) is the known reflectivity of the reference plate, to complete the real-time autofocus hyperspectral acquisition.

[0037] The special pixel group in this application is designed to receive light from only the right side of the lens and only the left side of the lens, generating independent grayscale sequences. Therefore, it achieves synergistic optimization at the hardware and algorithm levels, significantly improving focusing performance. It solves the problems caused by the highly dynamic focusing scenarios of hyperspectral cameras, such as target movement, sudden changes in illumination, and subtle or indistinct target details in some scenarios. Existing autofocus algorithms are slow to respond, manual focusing is unstable, and current focusing solutions are out of step with the real-time requirements of dynamic monitoring scenarios such as video hyperspectral imaging. These problems result in insufficient focusing accuracy, degraded image quality, and affect the validity of spectral data and the accuracy of application decisions.

[0038] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0039] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A real-time autofocus hyperspectral acquisition method, characterized in that, The apparatus for applying the aforementioned real-time autofocus hyperspectral acquisition method includes: an adjustable focus lens, a beam splitter, a collimating lens group, an autofocus module, a hyperspectral module, and a main control module. The adjustable lens is installed at the front end of the optical path and is responsible for collecting the incident light signal of the dynamic scene. It is electrically connected to the main control module, receives the focus adjustment command issued by the main control module, and adjusts the focus in real time. The beam splitter is located on the output optical path of the adjustable focus lens. It splits the incident light into two paths. One path is guided along the vertical optical path to the collimating lens group and the autofocus module for focus status detection. The other path is transmitted along the horizontal optical path to the hyperspectral module for spectral imaging analysis. The beam splitter is coaxial with the hyperspectral module in the horizontal direction and with the collimating lens group in the vertical direction. The autofocus module and the hyperspectral module are confocal and electrically connected to the main control module respectively. The autofocus module is integrated at the end of the imaging link and provides the main control module with the raw data for autofocus calculation. The hyperspectral module receives the light signal split by the beam splitter, decomposes the incident light into spectral information and collects it. The main control module integrates algorithms from the spectral acquisition process and controls the operation of the adjustable focus lens, autofocus module, and hyperspectral module. The aforementioned real-time autofocus hyperspectral acquisition method includes: Step 1: Acquire pixel signals from the dark background; Step 2: After ensuring that the lens imaging area covers the target object and the reference board, perform autofocus. Autofocus includes: Acquire the image and adjust the brightness to reach the preset threshold; Select a region from the center of the image and extract the right and left view sequences of the selected image, then calculate the phase difference; Based on the phase difference, the image plane offset and lens movement are calculated to determine the focusing direction for automatic lens focusing. Step 3: Repeat the autofocus process in Step 2 until the phase difference error is less than the set threshold, then end the autofocus, record the autofocus data and perform spectral acquisition to obtain the raw spectral data; Step 4: The pixel size of the original spectral data is inconsistent with that of the image acquired during autofocus. The coordinates of the selected area image are mapped to the coordinates of the original spectral data. Based on the phase difference finally obtained in Step 3, the object distance matrix of the image with the same size as the selected area image in the original spectral data is calculated. The full-frame object distance matrix of the original spectral data is obtained by interpolation. Step 5: Correct the original spectral data based on the full-frame object distance matrix to obtain corrected spectral data; Step 6: Calculate the corrected reference plate spectral data and the corrected target spectral data based on the corrected spectral data and the dark background pixel signal, and calculate the reflectance of the target spectral data to achieve real-time autofocus hyperspectral acquisition.

2. The real-time autofocus hyperspectral acquisition method according to claim 1, characterized in that: When calculating the phase difference, the center position coordinates of the selected area image are extracted. Based on the center position coordinates, the right view sequence and the left view sequence are defined. The correlation between the right view sequence and the left view sequence is calculated. The pixel deviation number corresponding to the maximum correlation is obtained to obtain the phase difference. The sign of the pixel deviation number determines the lens adjustment direction.

3. The real-time autofocus hyperspectral acquisition method according to claim 1 or 2, characterized in that: When mapping the coordinates of the selected image to the coordinates of the original spectral data, the pixel size of the selected image and the pixel size of the original spectral data are obtained, and the coordinates of any point in the selected image are calculated to obtain the corresponding coordinates of that point in the original spectral data.

4. The real-time autofocus hyperspectral acquisition method according to claim 3, characterized in that: Based on the resolution of the original spectral data, the selected area image is sampled to the pixel raster of the original spectral data using a bilinear interpolation algorithm.

5. The real-time autofocus hyperspectral acquisition method according to claim 4, characterized in that: Obtaining the full-frame object distance matrix of the original spectral data includes: calculating the object distance matrix of the selected area image, and obtaining the full-frame object distance matrix of the original spectral data through a bilinear interpolation algorithm based on the mapping relationship between the selected area image and the original spectral data.

6. The real-time autofocus hyperspectral acquisition method according to claim 1, characterized in that: Corrected spectral data is obtained by calibrating pixels in other areas of the image based on the signal values ​​and distances of pixels in the original spectral data center.

7. The real-time autofocus hyperspectral acquisition method according to claim 1 or 6, characterized in that: Select a reference plate region from the calibrated spectral data, calculate the calibrated reference plate spectral data, and calculate the reflectance of the target analyte using the calibrated reference plate spectral data and the calibrated spectral data.

8. The real-time autofocus hyperspectral acquisition method according to claim 1, characterized in that: The autofocus module includes a substrate, a light-shielding layer, and a special pixel group. The substrate is made of a 10-50μm transparent PET film, and the light-shielding layer is evenly spaced on the substrate. The special pixel group is designed to receive light from the right side of the lens and light from the left side of the lens, generating an independent grayscale sequence.

Images