Parameter closed loop adaptive wide band high spectrum on-line acquisition fusion method and system

Abstract

The application discloses a parameter closed loop adaptive wide spectrum hyperspectral online acquisition fusion method and system, at least two hyperspectral cameras with partially overlapped spectral response ranges are used for synchronous acquisition, the closed loop adaptive adjustment of geometric and radiation parameters is completed by using a reference plate before acquisition, data preprocessing, spatial registration and spectral fusion are synchronously executed during acquisition, and dynamic re-calibration is automatically triggered between batches; through the implementation of the application, the availability and consistency of wide spectrum hyperspectral data are guaranteed from the source, motion distortion and overlapping band spectrum jump are eliminated, approximate real-time wide spectrum data output is realized, the automation level and long-term operation stability of the system are significantly improved, and the application is suitable for online nondestructive testing and quantitative analysis in an industrial field.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral imaging technology, and in particular to a broadband hyperspectral online acquisition and fusion method and system with parameter closed-loop adaptive acquisition. Background Technology

[0002] Broadband hyperspectral imaging technology can simultaneously acquire spatial and continuous spectral information of a target, enabling non-contact, non-destructive quantitative analysis of material composition. It has been widely used in fields such as agricultural product quality grading, defect detection of industrial parts, and food safety production. In industrial applications, hyperspectral systems are typically required to cover the full spectrum of 400-2500nm to simultaneously acquire color information in the visible light band and composition information in the near-infrared band.

[0003] Limited by the spectral response characteristics of semiconductor sensor materials, a single sensor cannot simultaneously achieve high-sensitivity detection across the entire 400-2500nm spectrum. Currently, the industry commonly employs a scheme of co-acquiring data using two hyperspectral cameras with a shared field of view: one visible hyperspectral camera covering the 400-1000nm band, and one near-infrared hyperspectral camera covering the 900-2500nm band. Complete broadband data is obtained through post-processing band stitching and spectral fusion.

[0004] However, existing technical solutions have the following core flaws:

[0005] Offline working mode leads to uncontrollable data quality: The common process of "collect first, judge later" means that data quality verification can only be carried out through offline processing after all collection is completed. Once problems such as local overexposure / underexposure, low signal-to-noise ratio in edge bands, or inconsistency between dual camera data occur, the collection must be reorganized. This not only makes the process cumbersome and relies on multiple people to collaborate, but also seriously delays the progress of industrial production line operations and increases operating costs.

[0006] The inherent differences between the two cameras lead to insufficient accuracy of the fused data: The two cameras have inherent differences in spectral response curves, quantum efficiency, noise levels and edge band stability. Moreover, the existing technology lacks a real-time consistency calibration mechanism during the acquisition stage, which causes obvious spectral jumps in the overlapping region of 900-1000nm after fusion, directly affecting the accuracy of quantitative analysis.

[0007] Manual calibration is inefficient and has poor adaptability: System parameters (frame rate, exposure time) are usually calibrated manually. The calibration process requires repeated shutdowns for adjustment, and each calibration takes a relatively long time. Furthermore, it cannot automatically adapt to environmental fluctuations common in industrial settings, such as light source attenuation, temperature drift, and changes in the reflectivity of the test object, resulting in poor long-term operational stability.

[0008] Motion distortion is difficult to eliminate: The imaging quality of a line-scan hyperspectral camera is closely related to the matching degree of conveyor belt speed and frame rate. Existing technologies lack real-time geometric distortion correction mechanisms, which can easily lead to stretching or compression deformation of images in the direction of motion, affecting spatial resolution and subsequent analysis accuracy.

[0009] In summary, there is an urgent need for a parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method and system to address the shortcomings of existing technologies. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides a broadband hyperspectral online acquisition and fusion method and system with parameter closed-loop adaptive design. It aims to solve the problem that the availability of data cannot be guaranteed during the acquisition stage, resulting in poor stability, band distortion, or even inability to process the fused data.

[0011] On the one hand, this application provides a parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method, which uses at least two hyperspectral cameras with partially overlapping spectral response ranges for synchronous acquisition, including the following steps:

[0012] Step 1: System initialization, configuring at least two hyperspectral cameras, a full-spectrum light source, a speed-controllable conveyor belt, and a material feeding / discharging triggering device;

[0013] Step 2: Acquire and store the average dark background data from each hyperspectral camera;

[0014] Step 3: Perform adaptive closed-loop parameter adjustment using a reference board, including iteratively adjusting the frame rate of each hyperspectral camera based on the geometric features of the reference board, iteratively adjusting the exposure time of each hyperspectral camera based on the radiation features of the reference board, and constraining the consistency of the spectral response of the hyperspectral cameras in overlapping bands, until all parameters converge and are locked.

[0015] Step 4: When the feeding trigger device detects the test object, it controls the hyperspectral camera through the external trigger board to lock the parameters and synchronously collect the raw hyperspectral data of the test object. When the discharging trigger device detects that the test object has left, it stops collecting data.

[0016] Step 5: The raw hyperspectral data acquired by the hyperspectral camera is preprocessed, spatially registered, and spectrally fused sequentially to generate continuous broadband hyperspectral data;

[0017] Step 6: Calculate the relative reflectance of the analyte based on the fused broadband hyperspectral data and the calibrated reflectance of the reference plate;

[0018] Step 7: At different sampling intervals for different batches or different types of test objects, automatically detect the reference plate on the conveyor belt and trigger the parameter recalibration process, repeating step 3 to complete the system parameter update.

[0019] By implementing full-process control including pre-acquisition parameter closed-loop self-calibration, synchronous fusion during acquisition, and dynamic recalibration between batches, the system solves the problems of uncontrollable data quality, cumbersome calibration, and low fusion accuracy in existing offline modes. This ensures the availability of wide-spectrum data from the source, improves the system's automation level and operational stability, and adapts to the needs of industrial online testing.

[0020] Furthermore, in step 1, there are two hyperspectral cameras, including a first hyperspectral camera covering the visible light band from 400nm to 1000nm and a second hyperspectral camera covering the near-infrared band from 900nm to 2500nm, which are installed side by side directly above the conveyor belt.

[0021] Further, step 2 specifically involves: automatically closing the shutters of each hyperspectral camera, continuously acquiring multiple frames of dark field images, calculating and storing the average dark background data band by band.

[0022] Furthermore, in step 3, the iterative adjustment of the frame rate of each hyperspectral camera based on the geometric features of the reference plate specifically involves:

[0023] The conveyor belt moves the reference plate, and two hyperspectral cameras scan simultaneously to acquire images of the reference plate.

[0024] The reference plate image is binarized, and the edges of the circular calibration region are extracted. The diameter D of the circular region in the direction of conveyor belt movement is calculated. x The number of pixels D perpendicular to the direction of motion y ;

[0025] According to the formula The frame rates of the first and second hyperspectral cameras were adjusted respectively.

[0026] in:

[0027] F A0 F is the current frame rate before adjustment for the first hyperspectral camera. A1 The frame rate adjusted for the first hyperspectral camera;

[0028] F B0 For the current frame rate before adjustment of the second hyperspectral camera, F B1 The frame rate was adjusted for the second hyperspectral camera;

[0029] D xA The number of pixels representing the diameter of the circular calibration area in the direction of conveyor belt movement in the reference plate image acquired by the first hyperspectral camera;

[0030] D yA The number of pixels in diameter of the circular calibration region perpendicular to the direction of motion in the reference plate image acquired by the first hyperspectral camera;

[0031] D xB The number of pixels representing the diameter of the circular calibration area in the direction of conveyor belt movement in the reference plate image acquired by the second hyperspectral camera;

[0032] D yB The number of pixels in diameter of the circular calibration region perpendicular to the direction of motion in the reference plate image acquired by the second hyperspectral camera;

[0033] Iterate the above process until D. x With D y The ratio approaches 1.

[0034] Furthermore, in step 3, the iterative adjustment of the exposure time of each hyperspectral camera based on the radiation characteristics of the reference plate specifically involves:

[0035] The DN values ​​of all spatial pixels within the circular calibration area of ​​the statistical reference plate are calculated in all bands. The exposure time is adjusted so that the DN values ​​are within the range of 30% to 90% of the sensor saturation value.

[0036] Simultaneously adjust the exposure time of the two hyperspectral cameras so that the relative deviation of their DN values ​​after dark background subtraction, bit depth normalization, and exposure time normalization within the overlapping band does not exceed 50%.

[0037] Furthermore, in step 3, until all parameters converge, specifically: the conveyor belt reciprocates and scans the reference board, repeating the adjustment steps of frame rate and exposure time until the relative deviation of the frame rate, exposure time, and overlapping band DN value obtained from two consecutive scans is less than 1%, then the final parameters are locked and solidified into the system configuration.

[0038] Furthermore, in step 5, data preprocessing specifically includes:

[0039] Dark background subtraction: Subtract the corresponding average dark background data from the raw data of the first hyperspectral camera and the second hyperspectral camera respectively;

[0040] Bit depth normalization: If the analog-to-digital conversion bit depth of two hyperspectral cameras is different, the data from the higher bit depth camera is linearly mapped to the numerical range of the lower bit depth camera.

[0041] Exposure time normalization: Normalizes camera data with longer exposure times to the exposure time reference of cameras with shorter exposure times.

[0042] Furthermore, in step 5, the spectral fusion specifically involves:

[0043] Calculate the average band spacing of the first and second hyperspectral cameras, take the smaller value as the uniform wavelength step size, and generate a new wavelength sequence covering 400nm to 2500nm.

[0044] For non-overlapping regions with wavelengths less than 900 nm, linear interpolation is performed using data from the first hyperspectral camera;

[0045] For non-overlapping regions with wavelengths greater than 1000 nm, linear interpolation is performed using data from a second hyperspectral camera;

[0046] For the overlapping region of 900nm to 1000nm, firstly remove bad pixels where the difference in DN value between the corresponding bands of the two cameras exceeds 2 to 10 times the adjustable abnormal threshold. Then, based on the principle of minimizing the wavelength distance from the target wavelength to the adjacent band nodes of the two cameras, select the data of the corresponding camera for linear interpolation.

[0047] On the other hand, this application provides a parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion system, including:

[0048] At least two hyperspectral cameras with partially overlapping spectral response ranges, each with a built-in automatic shutter module, are mounted side by side directly above the conveyor belt;

[0049] The full-spectrum light source module covers the 400nm to 2500nm wavelength band and provides uniform illumination to the acquisition area after light homogenization processing;

[0050] The conveyor belt module is used to continuously carry and transport the test object and the reference board, and has a programmable speed closed-loop control function.

[0051] Reference plate, used for calibration of system geometry and radiation parameters;

[0052] The triggering module, including feed triggering hardware and discharge triggering hardware, is installed at the feed inlet and discharge outlet of the conveyor belt, respectively, and is used to detect the position of the test object and the reference plate;

[0053] The external trigger board module is configured to output a synchronization pulse during the valid period of the feed trigger signal. It is electrically connected to the trigger module and each hyperspectral camera, and is used to generate a synchronization pulse according to the trigger signal to control all hyperspectral cameras to achieve frame-level synchronous acquisition.

[0054] The processing module is electrically connected to each hyperspectral camera, conveyor belt module, and external trigger board module, and is used to perform parameter adaptive closed-loop adjustment, data preprocessing, image registration, spectral fusion, and relative reflectance calculation.

[0055] Furthermore, the hyperspectral camera consists of two cameras, including a first hyperspectral camera covering the 400nm to 1000nm band and a second hyperspectral camera covering the 900nm to 2500nm band. The background of the reference plate is black, with a white circular calibration area in the center. The reflectance of the white circular area is calibrated by spectral radiometry and is 5% to 10% higher than the estimated maximum reflectance of the analyte.

[0056] The substantial effects of this invention:

[0057] 1. In this invention, by adopting a fully automatic closed-loop self-calibration technology before acquisition, the geometric and radiation parameters are automatically tuned simultaneously using a single specially designed reference plate. System calibration can be completed quickly without manual intervention. Optimal matching of all parameters is achieved before formal acquisition, avoiding problems such as local overexposure, underexposure, and motion distortion from the source, and completely eliminating the drawbacks of repeated acquisition caused by substandard data quality in traditional offline mode.

[0058] 2. In this invention, by constraining the consistency of the spectral response of the overlapping bands of the two cameras during the acquisition stage, and combining outlier removal and nearest-neighbor interpolation fusion algorithms, the spectral jumps caused by the inherent differences in the hardware of the two cameras are effectively eliminated; bit depth normalization and exposure time normalization are achieved simultaneously to solve the problem of inconsistent data between the two cameras, significantly improve the continuity and accuracy of the fused broadband data, and provide a reliable data foundation for subsequent quantitative analysis.

[0059] 3. In this invention, a parallel pipeline architecture of acquisition, preprocessing, registration, and fusion is adopted to achieve near real-time output of wide-spectrum hyperspectral data; analysis results can be obtained without waiting for all acquisitions to be completed, which greatly shortens the detection cycle and can be directly connected to the online sorting and control system of industrial production lines.

[0060] 4. In this invention, the frame rate adaptive adjustment technology based on the geometric invariance of the reference plate corrects motion stretching or compression distortion caused by the mismatch between the conveyor belt speed and the camera frame rate in real time, solves the problem of geometric deformation that easily occurs in line scan imaging, ensures the spatial imaging accuracy of hyperspectral images, and meets the requirements of high-precision spatial positioning and defect detection. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart of Example 1.

[0063] Figure 2 This is a system schematic diagram of Example 3.

[0064] Figure 3 This is a schematic diagram of the system data acquisition in Example 3. Detailed Implementation

[0065] To more clearly illustrate the technical solutions in the embodiments of the invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Example 1:

[0067] Reference Figure 1 As shown, a parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method is proposed, which uses two hyperspectral cameras with partially overlapping spectral response ranges for synchronous acquisition, and includes the following steps:

[0068] System initialization, configuration of two hyperspectral cameras, full-spectrum light source, speed-controllable conveyor belt and material feeding / discharging trigger device;

[0069] Collect and store the average dark background data from each hyperspectral camera;

[0070] The reference board is used to perform adaptive closed-loop adjustment of parameters. The background of the reference board is black, with a white circular calibration area in the center. The reflectance of the white circular area is calibrated by spectral radiometry and is 5% to 10% higher than the estimated maximum reflectance of the analyte. This includes iteratively adjusting the frame rate of each hyperspectral camera based on the geometric features of the reference board, iteratively adjusting the exposure time of each hyperspectral camera based on the radiometric features of the reference board, and constraining the consistency of the spectral response of the hyperspectral cameras in overlapping bands until all parameters converge and are locked.

[0071] When the feeding triggering device detects the test object, it controls the hyperspectral camera through the external triggering board to lock the parameters and synchronously collect the raw hyperspectral data of the test object. When the discharging triggering device detects that the test object has left, it stops collecting data.

[0072] The raw hyperspectral data acquired by the hyperspectral camera are preprocessed, spatially registered, and spectrally fused sequentially to generate continuous broadband hyperspectral data.

[0073] The relative reflectance of the analyte is calculated based on the fused broadband hyperspectral data and the calibrated reflectance of the reference plate.

[0074] During the sampling intervals of different batches or different types of test objects, the reference plate on the conveyor belt is automatically detected and the parameter recalibration process is triggered, and the system parameter update is repeated. It supports automatic triggering of dynamic recalibration process between batches, and can automatically adapt to industrial environmental fluctuations such as light source attenuation, temperature drift and changes in the reflectivity of test objects without manual intervention. It solves the problems of poor adaptability and frequent shutdown for adjustment of traditional manual calibration, and significantly improves the stability and versatility of the system in long-term operation. It is especially suitable for multi-variety mixed production line scenarios.

[0075] In one implementation, two hyperspectral cameras are used, including a first hyperspectral camera covering the visible light band from 400nm to 1000nm and a second hyperspectral camera covering the near-infrared band from 900nm to 2500nm, which are installed side by side directly above the conveyor belt.

[0076] As one implementation method, the acquisition and storage of average dark background data from each hyperspectral camera specifically involves: automatically closing the shutter of each hyperspectral camera, continuously acquiring multiple frames of dark field images, calculating and storing the average dark background data band by band.

[0077] As one implementation method, the frame rate of each hyperspectral camera is iteratively adjusted based on the geometric features of the reference plate as follows:

[0078] The conveyor belt moves the reference plate, and two hyperspectral cameras scan simultaneously to acquire images of the reference plate.

[0079] The reference plate image is binarized, and the edges of the circular calibration region are extracted. The diameter D of the circular region in the direction of conveyor belt movement is calculated. x The number of pixels D perpendicular to the direction of motion y ;

[0080] According to the formula The frame rates of the first and second hyperspectral cameras were adjusted respectively.

[0081] in:

[0082] F A0 F is the current frame rate before adjustment for the first hyperspectral camera. A1 The frame rate adjusted for the first hyperspectral camera;

[0083] F B0 For the current frame rate before adjustment of the second hyperspectral camera, F B1 The frame rate was adjusted for the second hyperspectral camera;

[0084] D xA The number of pixels representing the diameter of the circular calibration area in the direction of conveyor belt movement in the reference plate image acquired by the first hyperspectral camera;

[0085] D yA The number of pixels in diameter of the circular calibration region perpendicular to the direction of motion in the reference plate image acquired by the first hyperspectral camera;

[0086] D xB The number of pixels representing the diameter of the circular calibration area in the direction of conveyor belt movement in the reference plate image acquired by the second hyperspectral camera;

[0087] D yBThe number of pixels in diameter of the circular calibration region perpendicular to the direction of motion in the reference plate image acquired by the second hyperspectral camera;

[0088] Iterate the above process until D. x With D y The ratio approaches 1.

[0089] As one implementation method, the exposure time of each hyperspectral camera is iteratively adjusted based on the radiation characteristics of the reference plate as follows:

[0090] The DN values ​​of all spatial pixels within the circular calibration area of ​​the statistical reference plate are calculated in all bands. The exposure time is adjusted so that the DN values ​​are within the range of 30% to 90% of the sensor saturation value.

[0091] Simultaneously adjust the exposure time of the two hyperspectral cameras so that the relative deviation of their DN values ​​after dark background subtraction, bit depth normalization, and exposure time normalization does not exceed 50% in the overlapping band of 900nm to 1000nm.

[0092] As one implementation method, the process of converging all parameters specifically involves: reciprocating scanning of the reference board with a conveyor belt, repeating the adjustment steps for frame rate and exposure time until the relative deviation of the frame rate, exposure time, and DN value of the overlapping band obtained from two consecutive scans is less than 1%, locking the final parameters and fixing them to the system configuration.

[0093] As one implementation method, data preprocessing specifically includes:

[0094] Dark background subtraction: Subtract the corresponding average dark background data from the raw data of the first hyperspectral camera and the second hyperspectral camera respectively;

[0095] Bit depth normalization: If the analog-to-digital conversion bit depth of two hyperspectral cameras is different, the data from the higher bit depth camera is linearly mapped to the numerical range of the lower bit depth camera.

[0096] Exposure time normalization: Normalizes camera data with longer exposure times to the exposure time reference of cameras with shorter exposure times.

[0097] As one implementation method, spectral fusion specifically involves:

[0098] Calculate the average band spacing of the first and second hyperspectral cameras, take the smaller value as the uniform wavelength step size, and generate a new wavelength sequence covering 400nm to 2500nm.

[0099] For non-overlapping regions with wavelengths less than 900 nm, linear interpolation is performed using data from the first hyperspectral camera;

[0100] For non-overlapping regions with wavelengths greater than 1000 nm, linear interpolation is performed using data from a second hyperspectral camera;

[0101] For the overlapping region of 900nm to 1000nm, firstly remove bad pixels where the difference in DN value between the corresponding bands of the two cameras exceeds 2 to 10 times the adjustable abnormal threshold. Then, based on the principle of minimizing the wavelength distance from the target wavelength to the adjacent band nodes of the two cameras, select the data of the corresponding camera for linear interpolation.

[0102] As one implementation method, an outlier removal step is also included before spectral fusion:

[0103] If the difference in DN values ​​between two hyperspectral cameras in the overlapping band exceeds a preset multiple threshold, it is determined to be a bad pixel and removed.

[0104] As one implementation method, the parameter recalibration process is as follows: the system identifies the reference board on the conveyor belt in real time. When the reference board is detected to enter the camera's field of view, the system automatically pauses the acquisition of the object under test and starts the parameter adaptive closed-loop adjustment process. After the recalibration is completed, the acquisition of the object under test is automatically resumed.

[0105] Example 2:

[0106] This embodiment is basically the same as embodiment 1, except that it provides a quantitative detection of sugar content on the surface of apples as the application scenario. The specific data acquisition and fusion steps are as follows:

[0107] Step 1, System Initialization:

[0108] Secure hyperspectral camera A (covering the visible light band, 400–1000 nm) and hyperspectral camera B (covering the near-infrared band, 900–2500 nm) directly above the conveyor belt. Adjust their installation height to 300 mm and fine-tune the camera angles so that the centers of the two cameras' fields of view are completely aligned. Connect the power and signal cables of all hardware devices and turn on the main power of the system.

[0109] Close the darkroom door, turn on the full-spectrum light source and preheat for 30 minutes to stabilize the light source output and suppress ambient stray light interference;

[0110] The two cameras are controlled by an external trigger board C. Trigger hardware D is set at the feed inlet, and trigger hardware E is set at the discharge outlet. When trigger hardware D detects the object to be tested, the external trigger board C starts sending trigger signals to cameras A and B. When trigger hardware E detects that the object has left, the external trigger board C stops sending trigger signals. After the system starts, it sequentially completes self-tests of the hyperspectral camera, conveyor motor, and external trigger module.

[0111] The reference plate was confirmed to be fixed on the conveyor belt without relative displacement; the background of the reference plate was black, and the center was a white inscribed circular area. Its reflectivity was calibrated and the value was slightly higher than the estimated maximum reflectivity of the object under test (8% higher).

[0112] Calculate the initial parameters:

[0113] Ground sampling interval of hyperspectral camera A: ;

[0114] Initial frame rate of hyperspectral camera A: (less than the hardware's maximum frame rate of 2000Hz);

[0115] Initial exposure time of hyperspectral camera A: ;

[0116] Ground sampling interval of hyperspectral camera B: ;

[0117] Initial frame rate of hyperspectral camera B: (less than the hardware's maximum frame rate of 500Hz);

[0118] Initial exposure time of hyperspectral camera B: ;

[0119] Write the above initial parameters into the registers of the two hyperspectral cameras to complete system initialization;

[0120] Among them, P A P B For the size of two camera pixels; H A H B For camera installation height, F A F B This refers to the camera's focal length.

[0121] Step 2, capturing against a dark background:

[0122] The system software automatically sends commands to close the built-in shutters of both hyperspectral cameras, and then continuously acquires 10 frames of dark-field images for each camera. The average pixel value of the 10 dark-field images is calculated band-by-band to obtain the average dark background data (Dark) of hyperspectral camera A. A (λ) and the average dark background data of hyperspectral camera B B (λ), stored on the local hard disk of the processing unit.

[0123] Step 3, adaptive parameter adjustment:

[0124] Frame rate adaptive adjustment: Start the conveyor belt to move the reference plate at a speed of 0.1 m / s, the external trigger plate sends a synchronization pulse, and the two hyperspectral cameras scan the reference plate synchronously at the initial frame rate to obtain a complete image of the reference plate.

[0125] The reference board image is binarized, and the edges of the white circular calibration region are extracted using the Canny edge detection algorithm. The diameter D of the circular region in the conveyor belt movement direction (X-axis) is calculated. x The number of pixels D in diameter perpendicular to the direction of motion (Y-axis, parallel to the camera slit) y .

[0126] Adjust the frame rate according to the formula:

[0127] Hyperspectral camera A first adjustment: D xA =92, D yA =100, ;

[0128] First adjustment of hyperspectral camera B: D xB =95, D yB =100, ;

[0129] The conveyor belt reciprocates, repeating the scanning and adjustment process described above: After the second scan, D xA =98,

[0130] ;D xA =98, ,

[0131] The ratios all approached 1, indicating that the frame rate adjustment converged.

[0132] Adaptive exposure time: A white circular calibration area is cropped from the reference board image, and the DN values ​​of all spatial pixels within this area are calculated across all wavelengths. Both cameras are 12-bit, with a sensor saturation value of 4095. The upper safety threshold is 4095 × 90% = 3685, and the lower threshold is 4095 × 30% = 1229.

[0133] Adjust the exposure time of hyperspectral camera A to 520 μs. At this time, 99.2% of the pixel DN values ​​in the circular area are in the range of 1229-3685. Adjust the exposure time of hyperspectral camera B to 2800 μs. At this time, 98.7% of the pixel DN values ​​in the circular area are in the range of 1229-3685.

[0134] Data from the 900nm-1000nm overlapping bands were extracted. After dark background subtraction, bit depth normalization (two bits are the same in this embodiment, so we skipped them), and exposure time normalization, the relative deviation of the DN values ​​of the corresponding bands of the two cameras was calculated. The maximum deviation was 42%, which is within the constraint requirement of not exceeding 50%.

[0135] Parameter convergence and locking: The conveyor belt reciprocates and scans the reference board three times. In two consecutive scans, the frame rate change is less than 0.5%, the exposure time change is less than 0.8%, the relative deviation of the overlapping band DN value is less than 0.3%, and all parameter changes are less than 1%. Parameter convergence is then determined. The system automatically locks the final frame rate (F). A =1616Hz, F B =316Hz) and exposure time (T A =520μs, T B =2800μs), and then fixed to the system configuration file.

[0136] Step 4, Data Acquisition of the Test Object:

[0137] The apples to be tested are placed sequentially at the feed inlet of the conveyor belt. When the feed photoelectric sensor detects that an apple has entered the field of view, it sends a start signal to the trigger plate. The external trigger plate immediately sends a synchronization pulse to the two hyperspectral cameras. The two cameras begin synchronous acquisition at a locked frame rate and exposure time to obtain raw hyperspectral data. and .

[0138] When the photoelectric sensor detects that an apple has left the field of view, it sends a stop signal to the trigger board; the external trigger board immediately stops outputting synchronization pulses, and the acquisition ends.

[0139] Step 5, Data preprocessing and spectral integration:

[0140] Data preprocessing:

[0141] Dark background subtraction is performed by subtracting the average dark background of the corresponding camera from the original data, resulting in:

[0142] ,

[0143] ;

[0144] Exposure time normalization: Normalize the hyperspectral camera B data to the exposure time reference of hyperspectral camera A, and obtain:

[0145] .

[0146] Image registration:

[0147] Based on the pre-calibrated spatial mapping relationship between the two cameras, the image of the hyperspectral camera A, which has a larger field of view, is cropped to the same field of view as the hyperspectral camera B; the line scan image is split into groups of 10 frames, and the mapping relationship is used to perform point-to-point registration on each group of images so that the same pixel in the two groups of images corresponds to the same spatial position.

[0148] Spectral fusion:

[0149] The average spacing of the hyperspectral camera A band was calculated to be 1.5 nm, and the average spacing of the hyperspectral camera B band was 5 nm. Taking 1.5 nm as the uniform wavelength step, a new wavelength sequence covering 400 nm-2500 nm was generated.

[0150] Non-overlapping regions with wavelengths less than 900 nm: linear interpolation was performed using data from hyperspectral camera A;

[0151] Non-overlapping regions with wavelengths greater than 1000 nm: linear interpolation was performed using data from a hyperspectral camera B.

[0152] Overlapping region of 900nm-1000nm: First, remove bad points where the DN value difference between the corresponding bands of the two cameras exceeds 5 times (12 bad points were detected in this embodiment). Then, calculate the wavelength distance from the target wavelength to the adjacent band nodes of the two cameras, select the camera data with the smaller distance for linear interpolation, and finally generate continuous broadband hyperspectral data.

[0153] Step 6, Reflectance Calculation:

[0154] The average response value of the reference plate region in the fused image is extracted. Combined with the known 95% calibrated reflectance of the reference plate, the relative reflectance of the apple surface is calculated pixel by pixel to obtain the hyperspectral reflectance image of the apple. Based on this, a sugar content prediction model is established using partial least squares method to achieve quantitative detection of sugar content on the apple surface.

[0155] Step 7, Dynamic Recalibration and Automated Adaptation:

[0156] After collecting 100 apples, a reference board is placed at intervals on the conveyor belt. The system software identifies the reference board in real time. When the reference board is detected to enter the camera's field of view, apple acquisition is automatically paused, and the parameter adaptive closed-loop adjustment process in step 3 is initiated. After recalibration is completed, apple acquisition is automatically resumed to adapt to environmental changes such as light source attenuation and temperature drift.

[0157] In this embodiment, the system's automatic calibration time is approximately 2 minutes, far shorter than the 30 minutes required for traditional manual calibration; the data acquisition pass rate reaches over 99%, avoiding repeated data acquisition due to substandard data quality; the spectral jump in the overlapping band is less than 2%, and the coefficient of determination R of the sugar content prediction model is low. 2 The accuracy reached 0.92, and the root mean square error of prediction (RMSE) was 0.32°Brix, which fully meets the requirements for quantitative detection of agricultural product quality in industrial settings.

[0158] The substantial technical effects of this embodiment:

[0159] This embodiment utilizes a composite reference plate to simultaneously achieve geometric and radiation closed-loop self-tuning; it directly feeds back the frame rate matching error through the diameter ratio of the circular image in two directions, automatically iterating until distortion-free imaging is achieved; it automatically locks the exposure time through full-pixel full-band DN statistics and overlapping band consistency constraints; the entire process requires no manual intervention and converges in just a few rounds, solving the problem of repeated machine stop calibration required by traditional methods.

[0160] This embodiment adopts a "acquisition-registration-fusion" pipeline architecture, with fusion delay controlled within an acceptable range for industrial environments, meeting the near-real-time requirements of the production line. Overlapping bands first eliminate bad pixels and outliers, and then perform nearest-neighbor interpolation based on the principle of minimum wavelength distance, avoiding spectral jumps caused by direct splicing. At the same time, exposure time normalization and bit depth normalization eliminate data inconsistencies caused by differences in dual-camera hardware.

[0161] This embodiment supports placing reference boards at intervals between different material batches. The system automatically triggers recalibration to adapt to temperature drift, light source attenuation, or reflectivity changes without manual intervention. The entire calibration is completed using the same reference board, which is low-cost and particularly suitable for mixed production lines with multiple product varieties and large reflectivity differences, significantly improving the system's automation level and long-term stability.

[0162] Example 3:

[0163] Reference Figure 2 , Figure 3 As shown, this embodiment is basically the same as Embodiment 1, except that it provides a parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion system, including:

[0164] Two hyperspectral cameras 10 with partially overlapping spectral response ranges are mounted side by side directly above the conveyor belt, each with a built-in automatic shutter module.

[0165] The full-spectrum light source module 20 covers the 400nm to 2500nm wavelength band and provides uniform illumination to the acquisition area after light homogenization processing;

[0166] Conveyor belt module 30 is used to continuously carry and transport the test object and reference board, and has a programmable speed closed-loop control function;

[0167] Reference plate 40 is used for calibration of system geometric and radiation parameters;

[0168] The trigger module 50 includes feed trigger hardware 51 and discharge trigger hardware 52, which are installed at the feed inlet and discharge outlet of the conveyor belt, respectively, and are used to detect the position of the test object and the reference plate.

[0169] The external trigger board module 60 is configured to output a synchronization pulse during the valid period of the feed trigger signal. It is electrically connected to the trigger module and each hyperspectral camera respectively. It is used to generate a synchronization pulse according to the trigger signal and control all hyperspectral cameras to achieve frame-level synchronous acquisition.

[0170] The processing module 70 is electrically connected to each hyperspectral camera 10, conveyor belt module 30 and external trigger board module 60 respectively, and is used to perform parameter adaptive closed-loop adjustment, data preprocessing, image registration, spectral fusion and relative reflectance calculation.

[0171] In one implementation, the hyperspectral camera 10 includes a hyperspectral camera A covering the 400nm to 1000nm band and a hyperspectral camera B covering the 900nm to 2500nm band. The background of the reference plate 40 is black, and a white circular calibration area is provided in the center. The reflectance of the white circular area is calibrated by spectral radiometry and is 5% to 10% higher than the estimated maximum reflectance of the analyte.

[0172] It should be noted that while the preferred embodiments of the present invention are provided in the specification and accompanying drawings, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are not intended to impose additional limitations on the content of the present invention; their purpose is to provide a more thorough and comprehensive understanding of the disclosure of the present invention. Furthermore, the above-described technical features can be combined with each other to form various embodiments not listed above, all of which are considered to be within the scope of the present invention specification. Moreover, those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method, characterized in that, The method involves synchronous acquisition using at least two hyperspectral cameras with partially overlapping spectral response ranges, including the following steps: Step 1: System initialization, configuring at least two hyperspectral cameras with partially overlapping spectral response ranges, a full-spectrum light source, a speed-controllable conveyor belt, and a material feeding / discharging triggering device; Step 2: Acquire and store the average dark background data from each hyperspectral camera; Step 3: Perform adaptive closed-loop parameter adjustment using a reference board, including iteratively adjusting the frame rate of each hyperspectral camera based on the geometric features of the reference board, iteratively adjusting the exposure time of each hyperspectral camera based on the radiation features of the reference board, and constraining the consistency of the spectral response of the hyperspectral cameras in overlapping bands, until all parameters converge and are locked. Step 4: When the feeding trigger device detects the test object, it controls the hyperspectral camera through the external trigger board to lock the parameters and synchronously collect the raw hyperspectral data of the test object. When the discharging trigger device detects that the test object has left, it stops collecting data. Step 5: The raw hyperspectral data acquired by the hyperspectral camera is preprocessed, spatially registered, and spectrally fused sequentially to generate continuous broadband hyperspectral data; Step 6: Calculate the relative reflectance of the analyte based on the fused broadband hyperspectral data and the calibrated reflectance of the reference plate; Step 7: At different sampling intervals for different batches or different types of test objects, automatically detect the reference plate on the conveyor belt and trigger the parameter recalibration process, repeating step 3 to complete the system parameter update.

2. The parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method according to claim 1, characterized in that, In step 1, there are two hyperspectral cameras: a first hyperspectral camera covering the visible light band from 400nm to 1000nm and a second hyperspectral camera covering the near-infrared band from 900nm to 2500nm. The two cameras are installed side by side directly above the conveyor belt.

3. The parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method according to claim 2, characterized in that, Step 2 specifically involves: automatically closing the shutters of each hyperspectral camera, continuously acquiring multiple frames of dark field images, calculating and storing the average dark background data band by band.

4. The parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method according to claim 3, characterized in that, In step 3, the iterative adjustment of the frame rate of each hyperspectral camera based on the geometric features of the reference plate specifically involves: The conveyor belt moves the reference plate, and two hyperspectral cameras scan simultaneously to acquire images of the reference plate. The reference plate image is binarized, and the edges of the circular calibration region are extracted. The diameter D of the circular region in the direction of conveyor belt movement is calculated. x The number of pixels D perpendicular to the direction of motion y ; According to the formula The frame rates of the first and second hyperspectral cameras were adjusted respectively. in: F A0 F is the current frame rate before adjustment for the first hyperspectral camera. A1 The frame rate adjusted for the first hyperspectral camera; F B0 For the current frame rate before adjustment of the second hyperspectral camera, F B1 The frame rate was adjusted for the second hyperspectral camera; D xA The number of pixels representing the diameter of the circular calibration area in the direction of conveyor belt movement in the reference plate image acquired by the first hyperspectral camera; D yA The number of pixels in diameter of the circular calibration region perpendicular to the direction of motion in the reference plate image acquired by the first hyperspectral camera; D xB The number of pixels representing the diameter of the circular calibration area in the direction of conveyor belt movement in the reference plate image acquired by the second hyperspectral camera; D yB The number of pixels in diameter of the circular calibration region perpendicular to the direction of motion in the reference plate image acquired by the second hyperspectral camera; Iterate the above process until D. x With D y The ratio approaches 1.

5. The parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method according to claim 4, characterized in that, In step 3, the exposure time of each hyperspectral camera is iteratively adjusted based on the radiation characteristics of the reference plate as follows: The DN values ​​of all spatial pixels within the circular calibration area of ​​the statistical reference plate are calculated in all bands. The exposure time is adjusted so that the DN values ​​are within the range of 30% to 90% of the sensor saturation value. Simultaneously adjust the exposure time of the two hyperspectral cameras so that the relative deviation of their DN values ​​after dark background subtraction, bit depth normalization, and exposure time normalization within the overlapping band does not exceed 50%.

6. The parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method according to claim 1, characterized in that, In step 3, the process of converging all parameters specifically involves: reciprocating scanning of the reference board with a conveyor belt, repeating the adjustment steps for frame rate and exposure time until the relative deviation of the frame rate, exposure time, and DN value of the overlapping band obtained from two consecutive scans is less than 1%, locking the final parameters and fixing them to the system configuration.

7. The parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method according to claim 2, characterized in that, In step 5, data preprocessing specifically includes: Dark background subtraction: Subtract the corresponding average dark background data from the raw data of the first hyperspectral camera and the second hyperspectral camera respectively; Bit depth normalization: If the analog-to-digital conversion bit depth of two hyperspectral cameras is different, the data from the higher bit depth camera is linearly mapped to the numerical range of the lower bit depth camera. Exposure time normalization: Normalizes camera data with longer exposure times to the exposure time reference of cameras with shorter exposure times.

8. The parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion method according to claim 2, characterized in that, In step 5, spectral fusion specifically involves: Calculate the average band spacing of the first and second hyperspectral cameras, take the smaller value as the uniform wavelength step size, and generate a new wavelength sequence covering 400nm to 2500nm. For non-overlapping regions with wavelengths less than 900 nm, linear interpolation is performed using data from the first hyperspectral camera; For non-overlapping regions with wavelengths greater than 1000 nm, linear interpolation is performed using data from a second hyperspectral camera; For the overlapping region of 900nm to 1000nm, if the difference in DN values ​​between the two hyperspectral cameras in the overlapping band exceeds a preset multiple threshold, it is judged as a bad point and removed. Then, based on the principle of minimizing the wavelength distance from the target wavelength to the adjacent band nodes of the two cameras, the data of the corresponding camera is selected for linear interpolation.

9. A parameter-closed-loop adaptive broadband hyperspectral online acquisition and fusion system, used to implement the online acquisition and fusion method as described in any one of claims 1-8, characterized in that, include: At least two hyperspectral cameras with partially overlapping spectral response ranges, each with a built-in automatic shutter module, are mounted side by side directly above the conveyor belt; The full-spectrum light source module covers the 400nm to 2500nm wavelength band and provides uniform illumination to the acquisition area after light homogenization processing; The conveyor belt module is used to continuously carry and transport the test object and the reference board, and has a programmable speed closed-loop control function. Reference plate, used for calibration of system geometry and radiation parameters; The triggering module, including feed triggering hardware and discharge triggering hardware, is installed at the feed inlet and discharge outlet of the conveyor belt, respectively, and is used to detect the position of the test object and the reference plate; The external trigger board module is configured to output a synchronization pulse during the valid period of the feed trigger signal. It is electrically connected to the trigger module and each hyperspectral camera, and is used to generate a synchronization pulse according to the trigger signal to control all hyperspectral cameras to achieve frame-level synchronous acquisition. The processing module is electrically connected to each hyperspectral camera, conveyor belt module, and external trigger board module, and is used to perform parameter adaptive closed-loop adjustment, data preprocessing, image registration, spectral fusion, and relative reflectance calculation.

10. The parameter closed-loop adaptive broadband hyperspectral online acquisition and fusion system according to claim 9, characterized in that, The hyperspectral camera consists of two cameras: a first hyperspectral camera covering the 400nm to 1000nm band and a second hyperspectral camera covering the 900nm to 2500nm band. The background of the reference plate is black, with a white circular calibration area in the center. The reflectance of the white circular calibration area is calibrated by spectral radiometry and is 5% to 10% higher than the estimated maximum reflectance of the analyte.

Images