Hyperspectral imaging system based on prism-grating light splitting

Abstract

The invention discloses a hyperspectral imaging system based on prism-grating light splitting, which adopts a slit, a collimator objective, a prism, a grating, an imaging objective and a detector which are arranged in a matched manner, the detector performs push-broom imaging by sequentially collecting relatively moving target object images, the resolution of the imaging in the push-broom direction is in direct proportion to the image collection speed, and the resolution of the imaging in the push-broom direction is in direct proportion to the image collection speed. The image acquisition speed is inversely proportional to the push-scan speed, the detector selects part of the target surface through the region of interest to acquire data so as to increase the image acquisition speed, and the push-scan speed is increased on the premise that the resolution in the push-scan direction is determined; the spectral line bending caused by prism dispersion is opposite to the spectral line bending caused by grating dispersion in direction, and a spectral line bending eliminating light splitting structure is designed to correct the spectral line bending; on the basis of a prism-grating spectral line light splitting eliminating structure, an optical system is designed in combination with the ROI function of the high-speed near-infrared industrial camera, and the acquisition speed of the camera is greatly improved on the premise of moderately reducing the spectral resolution.

Description

Technical Field

[0001] The invention belongs to the technical field of spectral imaging, and in particular relates to a hyperspectral imaging system based on prism-grating spectrometry. Background Art

[0002] Hyperspectral camera is an integrated application device developed based on hyperspectral imaging technology. Compared with the acquisition capability of traditional RGB cameras for 3-4 discrete bands, hyperspectral cameras can simultaneously obtain spatial information of the photographed target and spectral information of more than 100 continuous bands in a wide spectral range. Combined with machine learning algorithms, the processing efficiency of hyperspectral data can be significantly improved, providing a technical basis for real-time processing for high-throughput applications in industrial sorting.

[0003] The acquisition speed of hyperspectral cameras has always been an important factor limiting their application in high-throughput industrial sorting. High-speed acquisition can effectively prevent dynamic blur and ensure the integrity of the materials photographed on the industrial conveyor belt in the spatial and spectral dimensions. Industrial sorting covers industries such as food processing, ore sorting, biopharmaceuticals, and plastic recycling. Combined with hyperspectral imaging technology, it can achieve high-speed, non-destructive, and efficient classification of materials on the assembly line. When setting the parameters of hyperspectral cameras for industrial sorting, it is also necessary to comprehensively consider the balance between spatial resolution, spectral resolution, and speed based on the materials to be sorted and the throughput.

[0004] On the other hand, for hyperspectral cameras based on prism-grating spectrometry, the grating will cause spectral line bending (Smile distortion) when it disperses, and the dispersion of the prism will also cause spectral line bending. Summary of the invention

[0005] In order to solve the shortcomings of the prior art, improve the acquisition speed of the hyperspectral camera, and thus improve the material sorting throughput, the present invention adopts the following technical solutions:

[0006] A hyperspectral imaging system comprises a spectroscopic mechanism and a detector. The system sequentially collects images of relatively moving target objects to perform push-scanning imaging. The resolution of the imaging in the push-scanning direction is proportional to the image acquisition speed and inversely proportional to the push-scanning speed. The image collected by the detector contains spatial dimension information and spectral dimension information. The required target frame rate is determined by testing the region of interest, and the required partial spectral dimension information is determined according to the target frame rate. The spectroscopic mechanism is arranged in cooperation with the detector to focus light on a partial target surface of the detector so that the detector collects images of spatial dimension information and partial spectral dimension information to improve the image acquisition speed, and the push-scanning speed is improved on the premise of determining the resolution in the push-scanning direction. The region of interest ROI is mainly used to focus on key information, optimize calculation efficiency, and improve processing accuracy. It is usually used in the field of image segmentation and feature extraction, which is mainly for improving the accuracy and efficiency of image classification. It is also used for image compression, which is mainly for facilitating the storage and transmission of images. The present invention is based on the detection of the region of interest, so that the detector selects a part of the target surface to collect data, thereby compressing the imaging area of ​​the optical system, thereby greatly improving the acquisition speed. The ultimate goal is to improve the throughput of materials per unit time by achieving ultra-high-speed acquisition.

[0007] Furthermore, the detector uses full pixels W in the spatial dimension and partial pixels N in the spectral dimension. After the cooperation of the light splitting mechanism, the area focused on the target surface is W×N, and the calculation formula of N is as follows:

[0008]

[0009] Where W represents the spatial dimension, H represents the spectral dimension, and f 0 Indicates the maximum frame rate under full pixel acquisition, f t It represents the target frame rate to be achieved after the region of interest test, and b represents the data bit depth.

[0010] Due to the limitation of transmission bandwidth and the need to ensure the full spatial dimension, part of the pixels N are used in the spectral dimension to achieve an increase in frame rate under bandwidth limitation. Based on the increase in frame rate, the image acquisition speed is improved, and then the push-scan speed is improved under the premise of determining the resolution in the push-scan direction.

[0011] Furthermore, an imaging objective lens is provided between the spectroscopic mechanism and the detector. Due to the existence of axial chromatic aberration, the perfect focusing position is not on the same plane. The photosensitive surface of the detector needs to be tilted at a certain angle so that the 900-1700nm light can achieve a good focusing effect on the photosensitive surface of the sensor as much as possible to weaken the influence of axial chromatic aberration on spectral imaging. The tilt angle is determined by acquiring monochromatic light with a certain step length within the wavelength range to be collected. When the radius of the focusing spot of different monochromatic lights is the smallest, the projection position in the optical axis direction and the projection position in the direction perpendicular to the optical axis are fitted, and the photosensitivity tilt angle is obtained through the fitting line, thereby achieving better imaging effects for all wavelengths.

[0012] Furthermore, the system also includes a collimating objective lens and an imaging objective lens that are arranged in coordination with each other. The light passes through the collimating objective lens, the spectroscopic mechanism, and the imaging objective lens in sequence to the detector. The collimating objective lens includes a front group composed of multiple lenses and a rear group composed of a double-cemented lens. The imaging objective lens includes a front group composed of a double-cemented lens and a rear group composed of multiple lenses.

[0013] In the collimating objective, the three-piece objective serves as the front group, providing basic aberration correction based on the collimated divergent light, and the doublet lens serves as the rear group, which is used to compensate for the residual axial chromatic aberration in the collimated light. In the imaging objective, the doublet lens serves as the front group, which is used to pre-correct the dispersed light after grating splitting, and the three-piece objective serves as the rear group, which offsets the field curvature of the front group and balances the astigmatism while focusing.

[0014] Furthermore, the system also includes a slit arranged in cooperation with the detector, and the image collected by the detector contains spatial dimension information and spectral dimension information. In the spectral dimension, each spectral channel contains all spatial information of the slit field of view at the same wavelength, and in the spatial dimension, each spatial channel contains all spectral information in the same slit field of view point. The spatial resolution of the imaging of the system includes the slit direction and the push-scan direction. The resolution in the slit direction is proportional to the number of pixels in the spatial dimension and inversely proportional to the width of the push-scan surface covered by the field of view. The slit collection direction is perpendicular to the push-scan direction of the system relative to the target object.

[0015] A hyperspectral imaging system based on prism-grating spectrometry includes a slit, a prism, a grating and a detector. The light of the target object passes through the slit, is split by the prism and the grating, and is finally imaged on the target surface of the detector. The hyperspectral imaging system described above is used, wherein the spectrometry mechanism is a prism and a grating that are arranged in a coordinated manner.

[0016] The grating is the main element that determines the spectral resolution. The grating will cause spectral line bending (Smile distortion) when it is dispersed. The light emitted from the slit is incident on the main cross section of the grating after collimation. The incident light and the diffracted light corresponding to different field points have different projections on the main cross section, which leads to spectral line bending during subsequent diffracted light imaging. The dispersion of the prism will also cause spectral line bending, but the direction of the spectral line bending is opposite to that of the grating. Therefore, the spectral line bending of the grating can be corrected by the prism. The light emitted from the center of the slit is incident on the prism along the optical axis, and is refracted by the prism and incident on the diffraction grating. The grating is parallel to the exit surface. The law of refraction gives:

[0017]

[0018]

[0019] Where β represents the inclination angle of the prism incident surface, Represents the refractive index of the prism at the central wavelength, where n represents the refractive index of the prism and the subscript λ 0 represents the central wavelength, i′ 1 represents the refraction angle on the main section of the prism incident surface, γ represents the inclination angle of the prism exit surface, and i′ represents the refraction angle on the main section of the prism exit surface 2 Equal to the grating incident angle φ, at this time the grating diffraction angle φ′ of the central wavelength 0 for:

[0020]

[0021] In the formula, k represents the diffraction order, λ represents the wavelength of the incident light, and d represents the grating constant;

[0022] When the focal lengths of the collimating objective and the imaging objective are the same, the spectral lines of the diffraction grating are bent by Δy according to the spectral line bending characteristics of the prism and grating. g ′ and the spectral line bending Δy after prism dispersion p ′ add to get the spectral bending Δy of the central wavelength:

[0023] Δy=Δy p ′+Δy g '

[0024]

[0025] In the formula, x represents the slit length, f′ represents the focal length of the objective lens, and α=γ+β represents the dispersion vertex angle of the main cross section of the prism;

[0026] Let Δy = 0 and simplify the prism parameters:

[0027]

[0028] Furthermore, the inclination angle of the prism exit surface γ=0°, that is, the prism exit surface is perpendicular to the optical axis, which facilitates calculation and adjustment.

[0029] Furthermore, the spectral resolution of the grating is:

[0030]

[0031] In the formula, represents the grating constant, N represents the number of grating lines, p represents the pixel size, m represents the diffraction order, and f 2 ′ represents the focal length of the imaging objective lens;

[0032] Based on the pixel size used, the diffraction order, and the required spectral resolution, the grating line bundle is determined by the focal length of the imaging objective.

[0033] Furthermore, the system further comprises a collimating objective lens arranged between the slit and the prism, and an imaging objective lens arranged between the grating and the detector; since the prism-grating composite light splitting structure will introduce lateral chromatic aberration, the light of different wavelengths emitted from the same field of view point on the slit is imaged at different positions on the image plane after being dispersed by the optical system, and since the angular component of the light of the same field of view point relative to the slit direction during the collimation and imaging process is unchanged, so the half-width of the spectral line at different wavelengths is expressed as:

[0034]

[0035] In the formula, ε′ represents the angular component of the slit direction, x represents the slit length, and f 1 ′ represents the focal length of the collimating lens, f 2 ′ represents the focal length of the imaging objective lens, φ′ λ It represents the angle of the diffracted light with wavelength λ relative to the optical axis, φ′ 0 Indicates the angle of the diffracted light of the central wavelength relative to the optical axis. The larger the angle between the system's outgoing light and the central wavelength light, the larger the half-width of the spectrum line, and the spectral bending (Keystone) distortion occurs in the spatial dimension. Then the spectral bending amount in the same field of view on the image plane is the ratio of the non-central wavelength λ to the central wavelength λ 0 The difference in the spatial coordinates of

[0036] Δx′=x λ ′-x′ 0 =f′ 2 tanε′[sec(φ′ λ -φ′ 0 )-1]

[0037] In the formula, x′ 0 Indicates the half-width of the spectral line at the minimum wavelength, φ′ 0 The grating diffraction angle representing the central wavelength;

[0038] When sec(φ′ λ -φ′ 0 ) hours, expressed approximately in radians Tanε′ is approximately expressed in radians as get:

[0039]

[0040] In the formula, Representing the magnification of the imaging system, the spectral curvature is proportional to the focal length of the objective, the square of the difference in the exit angle between the non-central wavelength and the central wavelength, and the slit length.

[0041] Furthermore, when the focal lengths of the collimating objective lens and the imaging objective lens are the same, m=1, and the spectral curvature has nothing to do with the focal length of the objective lens, but is only proportional to the square of the difference between the exit angles of the non-central wavelength and the central wavelength and the slit length.

[0042] Furthermore, the system further comprises a collimating objective lens arranged between the slit and the prism, and an imaging objective lens arranged between the grating and the detector; the incident surface of the grating is parallel to the exit surface of the prism, and the refraction angle i′ on the main cross section of the incident surface of the prism is 2 Equal to the grating incident angle φ, according to the refraction angle on the main cross section of the prism incident surface, the light dispersion angle range (0.3523°~2.6925°) corresponding to a certain wavelength range (900-1700nm) at the center field of view of the slit is obtained. In order to reduce the influence of the objective lens magnification on the spectrum bending, the focal length of the collimating objective lens is consistent with that of the imaging objective lens, and the central wavelength principal light is used as the optical axis of the imaging objective lens, that is, the imaging objective lens is tilted and the optical axis forms a certain angle with the collimating objective lens. This angle is the dispersion angle of the central wavelength (1.5224°). According to the dispersion angle and the imaging area of ​​the detector, the actual focal length of the objective lens is obtained:

[0043]

[0044] In the formula, θ max represents the maximum value of the light dispersion angle, θ min represents the minimum value of the light dispersion angle, θ 0 represents the dispersion angle of the central wavelength.

[0045] The advantages and beneficial effects of the present invention are:

[0046] The hyperspectral imaging system based on prism-grating spectrometry proposed in the present invention utilizes the ROI function of the image sensor to compress the imaging area of ​​the optical system, and compresses the imaging area to 9.6mm×1.5mm through optical design, thereby improving the acquisition speed of spectral images, and greatly improving the imaging frame rate on the basis of appropriately reducing the spectral resolution, so that the frame rate reaches 3448fps in the continuous band range of 900-1700nm, which is nearly 5 times higher than that of the existing commercial line-scanning hyperspectral camera, and maintains a spatial resolution of 640 pixels and an average spectral resolution of 13nm, providing key technical support for the large-scale application of hyperspectral imaging technology in industrial sorting scenarios. At the same time, the present invention combines prism-grating (PG) to design a spectroscopic structure that eliminates spectral line bending to suppress Keystone distortion and Smile distortion. The simulation shows that the root mean square (RMS) radius of the optical system point diagram is less than 7um, the MTF is greater than 0.6, and the imaging quality is good. The actual measurement shows that the Keystone distortion and Smile distortion are both less than one pixel. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the line scanning hyperspectral imaging principle in an embodiment of the present invention.

[0048] Figure 2 It is a comparison diagram of the area array detector in the embodiment of the present invention in the spectral dimension and the spatial dimension.

[0049] Figure 3 Schematic diagram of generating a push-scan image in an embodiment of the present invention.

[0050] Figure 4 Schematic diagram of the effect of frame rate variation on image horizontal resolution at a constant along-track scanning speed in an embodiment of the present invention.

[0051] Figure 5 Schematic diagram of ROI selection based on region of interest in an embodiment of the present invention.

[0052] Figure 6 Schematic diagram of the bending of the spectral lines of the transmission plane ruled grating in an embodiment of the present invention.

[0053] Figure 7 Schematic diagram of the refraction of light through a prism in an embodiment of the present invention.

[0054] Figure 8 Schematic diagram of the optical path of the central wavelength through the main cross section of the prism-grating element in an embodiment of the present invention.

[0055] Fig. 9 Schematic diagram of the optical system output light in an embodiment of the present invention.

[0056] Fig.10Schematic diagram of the optical path structure of the hyperspectral imaging system in an embodiment of the present invention.

[0057] Fig.11 3 is a fitting diagram of the image plane tilt angle in an embodiment of the present invention.

[0058] Fig.12 3 is a point array comparison diagram of all viewing fields of the optical system in the embodiment of the present invention at wavelengths of 950nm, 1300nm, and 1650nm.

[0059] Fig.13 It is a comparison diagram of the modulation transfer function (MTF) curves of all field points of the hyperspectral camera in an embodiment of the present invention at wavelengths of 950nm, 1300nm, and 1650nm.

[0060] Fig.14 It is a flow chart of actual measurement of a hyperspectral camera in an embodiment of the present invention.

[0061] Fig.15a This is a mercury lamp image of Smile distortion observed in an embodiment of the present invention.

[0062] Fig.15b 1 is a spectrum line diagram of a mercury lamp image observed by Smile distortion in an embodiment of the present invention.

[0063] Fig.16 It is a trapezoidal distortion observation diagram in an embodiment of the present invention.

[0064] Fig.17 Schematic diagram of spectral resolution measured by a monochromator in an embodiment of the present invention.

[0065] Fig.18a is a diagram showing the relationship between discrete wavelengths and pixel positions in wavelength calibration in an embodiment of the present invention.

[0066] Fig.18b It is a wavelength pixel fitting curve diagram in wavelength calibration in an embodiment of the present invention.

[0067] Fig.19 It is a comparison chart of spectral resolution at wavelengths of 950nm, 1300nm, and 1650nm in an embodiment of the present invention.

[0068] Fig. 20 Schematic diagram of verifying the application effect of the hyperspectral camera in plastic classification in an embodiment of the present invention.

[0069] Fig.21 It is the original hyperspectral image of the mixed plastic in the embodiment of the present invention.

[0070] Fig.22a Schematic diagram of the extraction of labeled spectral curves during the classification of the hyperspectral image of plastics in an embodiment of the present invention.

[0071] Figure 22b It is a graph of the original spectral reflectance during the hyperspectral image classification process of plastics in an embodiment of the present invention.

[0072] Fig.22c 4 is a diagram showing the classification result of a hyperspectral image of plastics in an embodiment of the present invention.

[0073] Fig.22d It is a spectrum curve diagram after processing in the hyperspectral image classification process of plastics in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0075] like Figure 1 As shown, the line scanning hyperspectral camera used in the present invention is composed of a front telescope system and a hyperspectral imaging system. The front telescope system uses a finished industrial lens of F / 1.4. The hyperspectral imaging system can be divided into a slit, a collimating lens, a prism, a grating, an imaging lens and a detector according to the imaging sequence. The light from the target is imaged on the slit through the lens, which serves as the field of view aperture of the system. The light passing through the slit enters the collimating lens and is collimated into parallel light, and then enters the spectroscopic element. The spectroscopic element disperses the light into parallel light of different wavelengths and emits it. After being focused by the imaging lens, it is finally imaged on the target surface of the array detector.

[0076] In a hyperspectral camera, the array detector is divided into two dimensions, such as Figure 2 As shown, X is the spatial dimension and Y is the spectral dimension. From the spectral dimension, each spectral channel contains all the spatial information of the slit field of view at the same wavelength. From the spatial dimension, each spatial channel contains all the spectral information at the same field of view point. Therefore, each frame of the image captured by the array detector contains one-dimensional spatial information and one-dimensional spectral information, such as Figure 3 As shown, a push-scan image is formed by the relative movement of the camera and the target object.

[0077] In the application of online scanning hyperspectral camera for industrial sorting, the camera and the target to be measured are pushed and scanned through a conveyor belt, such as Figure 4 As shown in Figure 1, the spatial resolution of imaging includes the slit direction and the push-scan direction. Taking the push-scan direction as the reference, the spatial resolution is defined as the vertical resolution and the horizontal resolution, respectively. The vertical resolution is proportional to the number of pixels in the spatial dimension of the hyperspectral camera and inversely proportional to the width of the conveyor belt covered by its field of view. Its calculation formula is shown in formula (1). The horizontal resolution is proportional to the acquisition speed of the hyperspectral camera and inversely proportional to the running speed of the conveyor belt. Its calculation formula is shown in formula (2):

[0078] R V=w / p (1)

[0079] R H =v / f (2)

[0080] In formula (1), w represents the actual conveyor belt width covered by the camera field of view, p represents the number of spatial pixels of the camera, v represents the running speed of the conveyor belt, and f represents the acquisition speed of the camera. Under the premise of ensuring that the spatial resolution and spectral resolution can meet the requirements of accurate identification and classification of materials, the higher the acquisition speed, the higher the matching conveyor belt speed, and the higher the material throughput per unit time.

[0081] Taking the plastic recycling industry as an example, when the conveyor belt width is 1.5m and the speed is 1.5m / s, the spectral resolution of the hyperspectral camera must reach at least 20nm, the vertical spatial resolution must be higher than 2mm / pixel, and the acquisition speed must be higher than 1500fps for accurate identification and classification. If the acquisition speed is increased to 3000fps on this basis, it can match the conveyor belt speed of 3m / s, and the material throughput per unit time can reach twice the original.

[0082] However, the maximum acquisition speed of the entire target surface of existing high-speed image sensors is only 724fps. The present invention selects a part of the target surface to collect data through the region of interest (ROI) function, thereby compressing the imaging area of ​​the optical system, greatly improving the acquisition speed, and improving the material throughput per unit time by achieving ultra-high-speed acquisition.

[0083] In the present invention, the camera needs to work in the wavelength range of 900-1700nm, so an indium gallium arsenide area array image sensor with high response characteristics to short-wave near infrared (SWIR) is selected; and because the luminous flux of the line scan camera is usually low, selecting a sensor with a larger pixel area can ensure a higher signal-to-noise ratio; in addition, the image sensor also needs to have an ROI function. In summary, the present invention uses a finished near-infrared industrial camera (FigSpec InSensor 220) with ROI function. The sensor target surface material is indium gallium arsenide, the pixel size is 15um×15um, the number of pixels is 640×512, and the available area of ​​the target surface is 9.6mm×7.68mm. Data is transmitted through the USB3.0 interface. When full pixel acquisition is performed, the frame rate can reach up to 724fps, and the data bit depth is 12bit. At this time, the maximum data transmission bandwidth is 640*512*724*12=2.65G bit / s.

[0084] Due to the bandwidth limitation of USB3.0, it is necessary to ensure that 640 is used as the spatial dimension and 512 is used as the spectral dimension to achieve the 900-1700nm line field spectrum acquisition. Therefore, ensuring that the spatial dimension uses full pixels and the spectral dimension uses partial pixels N can achieve an increase in frame rate under bandwidth limitations. By designing the optical path structure, the 900-1700nm light in the line field of view is split by the prism-grating and focused on the 640×N area on the sensor target surface. Therefore, N can be calculated by the following formula:

[0085]

[0086] Where W represents the spatial dimension 640, H represents the spectral dimension 512, and f 0 Indicates the maximum frame rate of 724fps under full pixel acquisition, f t represents the target frame rate to be achieved after ROI test, and b represents the data bit depth of 12 bits. According to the target frame rate of 3500fps, N=105 can be calculated, so the actual imaging area of ​​the sensor target surface should be 640×105 in theory.

[0087] Taking the above imaging area as the target, the sensor is tested on ROI, such as Figure 5 As shown, the results show that when the actual imaging is 640×100, it can reach more than 3500fps.

[0088] Therefore, it is necessary to design an optical system so that the 900-1700nm light is imaged on the 640×100 area of ​​the image sensor target surface after being split by the prism-grating (PG) element to achieve an acquisition speed of 3500fps. Since the pixel size is 15×15um, the actual imaging area on the sensor target surface can be calculated to be 9.6mm×1.5mm. At this time, 640 is used as the spatial dimension and 100 is used as the spectral dimension to achieve the acquisition of hyperspectral image data with a spatial channel of 640 and a spectral channel of 100.

[0089] The design indicators of the hyperspectral camera are shown in Table 1:

[0090] Table 1 Design parameters of high-speed hyperspectral camera

[0091]

[0092] The grating is the main component that determines the spectral resolution of a hyperspectral camera. Its spectral resolution is:

[0093]

[0094] In the formula, represents the grating constant (N represents the number of grating lines), p represents the pixel size, m represents the diffraction order, and f represents the 2' represents the focal length of the imaging objective lens. In the embodiment of the present invention, first-order diffraction light is used, and the design index requires that the spectral resolution is better than 15nm. Let Δλ = 10nm, then the calculation is:

[0095]

[0096] From formula (2), we can see that the number of grating lines is inversely proportional to the focal length of the objective lens. Considering the overall size of the optical system, let f 2 If ′=30mm, the number of grating lines is 50lp / mm, so a 50-line planar ruled grating can be selected.

[0097] When the grating is dispersed, it will cause spectral line bending (Smile distortion), such as Figure 6 As shown in the figure, the light emitted from the slit is incident on the main cross section of the grating after being collimated. The projections of the incident light and the diffracted light corresponding to different field points on the main cross section are different, resulting in spectral bending when the subsequent diffracted light is imaged. When the focal lengths of the collimating objective lens and the imaging objective lens are the same, the spectral bending formula of the diffraction grating is:

[0098]

[0099] In the formula, k represents the diffraction order, λ represents the wavelength of the incident light, x represents the slit length, d represents the grating constant, f′ represents the focal length of the objective lens, and φ′ 0 It represents the diffraction angle of the slit center field light after passing through the grating. Combined with the angular dispersion rate in the main section of the grating It can be seen that the spectral bending of the grating is related to the slit length, the wavelength of the incident light, the angular dispersion rate and the focal length of the objective lens. And from Δy g ′≥0, it can be seen that the spectrum lines of the grating are bent toward the long-wave direction, and the degree of bending is proportional to the slit length and the angular dispersion rate.

[0100] The dispersion of the prism will also cause spectral line bending, but the direction of the spectral line bending is opposite to that of the grating, so the prism can be used to correct the spectral line bending of the grating. Figure 7 The light emitted from different viewing points on the slit shown in the figure enters the prism and is refracted. The light of the central field of view of the slit is on the main section of the prism, and the light of the edge field of view is on the section of the prism. Since the vertical angle α′ of the cross-sectional dispersion is greater than the vertical angle α of the main cross-sectional dispersion, the spectrum line is bent. When the focal lengths of the collimating objective and the imaging objective are the same, the formula for the bending of the spectrum line after the light is dispersed by the prism is:

[0101]

[0102] Where n is the refractive index of the prism, f′ is the focal length of the collimating lens and the imaging lens, and i′ is 1 represents the refraction angle on the main section of the prism incident surface, i′ 2is the refraction angle on the main section of the prism exit surface, and x is the slit length. Combined with the prism angular dispersion formula It can be seen that the spectral line bending is related to the material and vertex angle of the prism, the length of the slit and the focal length of the objective lens. δ represents the prism deflection angle, which is the angle between the prism outgoing light and the incident light. And because Δy p ′≤0, it can be seen that the spectral lines of the prism are bent toward the short-wave direction and the degree of bending is proportional to the angular dispersion rate and the slit length.

[0103] The prism-grating combination can correct the central wavelength spectral line bending, such as Figure 8 As shown in the figure, the light emitted from the center of the slit is incident on the prism along the optical axis, and then incident on the diffraction grating after being refracted by the prism. The grating is parallel to the exit surface, the prism incident surface is inclined at an angle of β, the exit surface is inclined at an angle of γ, and the refraction angle of the prism exit surface is i′ 2 is equal to the grating incident angle φ. Then we can get from the law of refraction:

[0104]

[0105] In the formula, Represents the refractive index of the prism at the central wavelength, where n represents the refractive index and the subscript λ 0 Indicates the central wavelength. At this time, the grating diffraction angle is:

[0106]

[0107] According to the spectral bending characteristics of the prism and grating, the spectral bending of the central wavelength is obtained by adding equation (5) and equation (6):

[0108] Δy=Δy p ′+Δy g ′ (10)

[0109] Let Δy = 0, simplifying to get:

[0110]

[0111] Therefore, the prism parameters can be calculated by formula (11).

[0112] In the embodiment of the present invention, the prism design material is fused quartz (F_SILICA) with a central wavelength of 1300nm. To facilitate calculation and adjustment, the prism exit surface is made perpendicular to the optical axis, that is, the inclination angle γ=0°. By combining formulas (7), (8) and (11), the first-order diffraction light is taken to calculate the prism's vertex angle α=4.92051°.

[0113] Objective lens design: Due to the prism-grating composite beam splitting structure, lateral chromatic aberration will be introduced, such as Fig. 9As shown in the figure, the light emitted from the same field point on the slit is imaged at different positions on the image plane after being dispersed by the optical system, and because the light from the same field point has an angular component relative to the slit direction during the collimation and imaging process unchanged, so the half-width of the spectral line at different wavelengths can be expressed as:

[0114]

[0115] In the formula, ε′ represents the angular component of the slit direction, x represents the slit length, and f 1 ′ represents the focal length of the collimating lens, f 2 ′ represents the focal length of the imaging objective lens, φ′ λ It represents the angle of the diffracted light with wavelength λ relative to the optical axis, φ′ 0 Indicates the angle of the diffracted light of the central wavelength relative to the optical axis. The larger the angle between the system's outgoing light and the central wavelength light, the larger the half-width of the spectrum line, and the spectral bending (Keystone) distortion occurs in the spatial dimension. Then the spectral bending amount in the same field of view on the image plane is the ratio of the non-central wavelength λ to the central wavelength λ 0 The difference in the spatial coordinates of

[0116] Δx′=x′ λ -x′ 0 =f′ 2 tanε′[sec(φ′ λ -φ′ 0 )-1] (13)

[0117] When sec(φ′ λ -φ′ 0 ) hours, expressed approximately in radians Tanε′ is approximately expressed in radians as Substituting it into formula (13) we can get:

[0118]

[0119] In the formula, It represents the magnification of the imaging system. When the focal lengths of the collimating objective and the imaging objective are the same, m=1. The spectral curvature is proportional to the focal length of the objective, the square of the difference between the exit angles of the non-central wavelength and the central wavelength, and the slit length.

[0120] Since the grating incident surface is parallel to the prism exit surface, we have i′ 2=φ, according to formula (7), the dispersion angle of the light with a wavelength of 900-1700nm at the center of the slit field of view is 0.3523°~2.6925°. In order to reduce the influence of the objective lens magnification on the spectrum bending, the focal length of the collimating objective lens and the imaging objective lens are consistent, and the central wavelength principal light is used as the optical axis of the imaging objective lens, that is, the imaging objective lens is tilted and the optical axis forms a certain angle with the collimating objective lens. This angle is the dispersion angle of the central wavelength (1.5224°). According to the dispersion angle and the imaging area of ​​the detector, the actual focal length of the objective lens is obtained:

[0121]

[0122] Taking into account the adjustment errors between the mechanical structures and the image plane tilt, the focal length of the objective lens is taken as 36mm.

[0123] As the field stop of the system, the size of the slit not only determines the field of view of the system, but also affects the spectral resolution and signal-to-noise ratio of the system. If the slit width is too narrow, the light flux of the system will be reduced, resulting in a poor signal-to-noise ratio, while if it is too wide, the spatial resolution and spectral resolution will be reduced. Ideally, when the magnification of the optical system is 1, the slit width should be consistent with the pixel size, but considering the balance between the signal-to-noise ratio and the spectral resolution in actual situations, the slit width should be appropriately larger than the pixel size. Therefore, the slit width is set to 25um and the slit length is set to 15mm.

[0124] like Fig.10 As shown in the figure, the design of the objective lens adopts a three-piece objective lens as the main lens group, and on this basis, a double-cemented lens is added to optimize the design to achieve the purpose of collimation and imaging. In the collimating objective lens, the three-piece objective lens is used as the front group to provide basic aberration correction on the basis of collimating the divergent light, and the double-cemented lens is used as the rear group to compensate for the residual axial chromatic aberration in the collimated light; in the imaging objective lens, the double-cemented lens is used as the front group to pre-correct the dispersed light after the grating splits, and the three-piece objective lens is used as the rear group to offset the field curvature of the front group and balance the astigmatism while focusing.

[0125] After the light with wavelength of 900-1700nm is dispersed and focused by the optical system, the light with different wavelengths is focused at different positions in space. Due to the existence of axial chromatic aberration, the perfectly focused position is not on the same plane. It is necessary to tilt the photosensitive surface of the sensor at a certain angle so that the 900-1700nm light can achieve a good focusing effect on the photosensitive surface of the sensor as much as possible to reduce the impact of axial chromatic aberration on spectral imaging.

[0126] The tilt angle of the sensor photosensitive surface is determined by the following method: set the monochromatic light starting from 900nm with a step length of 50nm in the simulation software, record the projection position z in the optical axis direction and the projection position y in the direction perpendicular to the optical axis when the focus spot radius of different monochromatic lights is the smallest, and fit z and y once, as shown in Fig.11 The specific parameters are shown in the following table:

[0127] equation y=a+b*x intercept -46.2657±1.72835 Slope 2.78639±0.10407 Residual sum of squares 0.06914 Pearson's R 0.9897 R-squared (COD) 0.9795 Adjusted R-squared 0.97814

[0128] The fitted straight line is the optimal tilt angle of the sensor's photosensitive surface, which can achieve better imaging effects for all wavelengths.

[0129] In the embodiment of the present invention, the effect of the present invention is verified through simulation.

[0130] The system sets three viewing points of 0mm, 2mm, and 4mm and three wavelengths of 950nm, 1300nm, and 1650nm for simulation experiments. Fig.12 As shown, the results show that the root mean square (RMS) radius of the spot diagram of all field points at different wavelengths is less than one pixel.

[0131] like Fig.13 As shown in the figure, the MTF curve results show that the MTF values ​​of the optical system in all fields of view at different wavelengths are greater than 0.6 at the Nyquist frequency, and the imaging quality is good.

[0132] like Fig.14 As shown in the figure, during the installation process, the Keystone distortion and Smile distortion are tested and observed; then the spectrum is calibrated, the relationship between wavelength and pixel position is fitted, and the spectral range and spectral resolution of the system are obtained; finally, the actual application is verified to test the maximum frame rate and application effect.

[0133] During the adjustment process, a mercury lamp is used to observe the smile distortion of the imaging system. Fig.15a As shown in Figure 2, the spectrum line at the reference line does not bend significantly, and according to Fig.15b It can be seen that the peak values ​​of the mercury lamp spectrum at different spatial positions are all located within one pixel; a detector with a 7.5um pixel is used to test the grid paper and observe the Keystone distortion of the imaging system, such as Fig.16 As shown in the figure, the spectrum at the reference line does not bend significantly. Therefore, it can be considered that the Smile distortion and Keystone distortion of the camera meet the design requirements.

[0134] like Fig.17 As shown, a tunable laser is used to calibrate the optical-mechanical system, starting from 950nm, and testing in increments of about 50nm to 1650nm to obtain the positional relationship between each wavelength and the pixel, as shown in Fig.18aAs shown, a cubic polynomial is used to fit the corresponding relationship between wavelength and pixel, as shown in Fig.18b As shown in the figure, the camera spectral range is 899.03-1700.42nm, with a total of 103 spectral channels. Take the curves at 951nm, 1300nm, and 1649nm to calculate the half-height width, as shown in Fig.19 As shown, the spectral resolution at 951nm is better than 15nm, and the resolutions at the other two locations are better than 13nm. It can be considered that the average spectral resolution is about 13nm, which meets the design requirements.

[0135] Finally, the camera is verified for system application, such as Fig. 20 As shown in the figure, the hyperspectral camera is installed above the conveyor belt, and a halogen lamp is used as the light source to illuminate the conveyor belt at a 45° angle to eliminate specular reflection light. PS, PE, and PMMA mixed plastic samples are placed on the conveyor belt. The conveyor belt speed is adjusted to 3m / s for acquisition. The acquired images are shown in the figure below. Fig.21 As shown, the software shows that the maximum frame rate reaches 3448fps.

[0136] Select the spectral curves of the plastic sample and the background on the hyperspectral image and mark them, such as Fig.22a , Figure 22b is the original spectral reflectance curve, which is smoothed using a sliding average and baseline corrected using the first-order derivative method. Fig.22d The spectral curve after preprocessing is shown in the figure. Finally, the PLSDA algorithm is used to classify the remaining unlabeled plastics. The classification results are shown in the figure. Fig.22c shown.

[0137] The measured results show that the Smile distortion and Keystone distortion of the camera are both less than one pixel, which meets the design requirements; the spectral range covers 900-1700nm and the spectral resolution is generally better than 15nm, with an average spectral resolution of 13nm, which meets the design requirements; the camera acquisition speed does not reach 3500fps because there are errors in the camera adjustment and wavelength calibration process, resulting in the spectral range covering 103 spectral channels, and the increase in ROI area reduces the acquisition speed. However, the final results of classifying mixed plastics on a high-speed conveyor belt show that the camera can accurately identify and classify mixed plastic fragments on a high-speed conveyor belt, achieving our design goals.

[0138] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hyperspectral imaging system, comprising a spectroscopic mechanism and a detector, characterized in that: The system performs push-scan imaging by sequentially acquiring images of relatively moving target objects. The resolution of the imaging in the push-scan direction is proportional to the image acquisition speed and inversely proportional to the push-scan speed. The image acquired by the detector contains spatial dimension information and spectral dimension information. The required target frame rate is determined by testing the region of interest, and the required partial spectral dimension information is determined based on the target frame rate. The spectroscopic mechanism is arranged in coordination with the detector to focus light on a partial target surface of the detector so that the detector can acquire images with spatial dimension information and partial spectral dimension information.

2. A hyperspectral imaging system according to claim 1, characterized in that: The detector uses full pixels W in the spatial dimension and partial pixels N in the spectral dimension. After the light splitting mechanism cooperates, the area focused on the target surface is W×N. The calculation formula of N is as follows: Where W represents the spatial dimension, H represents the spectral dimension, f0 represents the maximum frame rate under full pixel acquisition, and f t It represents the target frame rate to be achieved after the region of interest test, and b represents the data bit depth.

3. A hyperspectral imaging system according to claim 1, characterized in that: An imaging objective lens is provided between the spectroscopic mechanism and the detector, and the photosensitive surface of the detector is tilted at a certain angle. The tilt angle is determined by acquiring monochromatic light with a certain step length within the wavelength range to be collected. When the radius of the focused spot of different monochromatic lights is the smallest, the projection position in the optical axis direction and the projection position in the direction perpendicular to the optical axis are fitted, and the photosensitivity tilt angle is obtained through the fitting line.

4. A hyperspectral imaging system according to claim 1, characterized in that: The system also includes a collimating objective lens and an imaging objective lens that are arranged in coordination. Light passes through the collimating objective lens, the light splitting mechanism, and the imaging objective lens in sequence to the detector. The collimating objective lens includes a front group consisting of multiple lenses and a rear group consisting of a double-cemented lens. The imaging objective lens includes a front group consisting of a double-cemented lens and a rear group consisting of multiple lenses.

5. A hyperspectral imaging system according to claim 1, characterized in that: The system also includes a slit arranged in cooperation with the detector. The image collected by the detector contains spatial dimension information and spectral dimension information. In the spectral dimension, each spectral channel contains all spatial information of the slit field of view at the same wavelength. In the spatial dimension, each spatial channel contains all spectral information in the same slit field of view point. The spatial resolution of the imaging of the system includes the slit direction and the push-scan direction. The resolution in the slit direction is proportional to the number of pixels in the spatial dimension and inversely proportional to the width of the push-scan surface covered by the field of view.

6. A hyperspectral imaging system based on prism-grating spectrometry, comprising a slit, a prism, a grating and a detector. The light of the target object passes through the slit, is split by the prism and the grating, and is finally imaged on the target surface of the detector. The system is characterized by: A hyperspectral imaging system according to any one of claims 1 to 4, wherein the light splitting mechanism is a prism and a grating arranged in coordination; The spectral line bending of the grating is corrected by the prism. The light emitted from the center of the slit is incident on the prism along the optical axis, and is incident on the diffraction grating after being refracted by the prism. The grating is parallel to the exit surface, and the refraction law is obtained: Where β represents the inclination angle of the prism incident surface, represents the refractive index of the prism at the center wavelength, where n represents the refractive index of the prism, the subscript λ0 represents the center wavelength, i′1 represents the refractive angle of the prism incident surface, γ represents the inclination angle of the prism output surface, and the refractive angle i′2 of the prism output surface is equal to the grating incident angle φ. At this time, the grating diffraction angle φ′0 of the center wavelength is: In the formula, k represents the diffraction order, λ represents the wavelength of the incident light, and d represents the grating constant; When the focal lengths of the collimating objective and the imaging objective are the same, the spectral lines of the diffraction grating are bent by Δy′ according to the spectral line bending characteristics of the prism and grating. g and the spectral line bending Δy′ after prism dispersion p Adding together the spectral bending Δy of the central wavelength: Δy=Δy′ p +Δy′ g In the formula, x represents the slit length, f′ represents the focal length of the objective lens, and α=γ+β represents the dispersion vertex angle of the main cross section of the prism; Let Δy = 0 and simplify the prism parameters to:

7. The hyperspectral imaging system based on prism-grating spectrometry according to claim 6, characterized in that: The spectral resolution of the grating is: In the formula, represents the grating constant, N represents the number of grating lines, p represents the pixel size, m represents the diffraction order, and f′2 represents the focal length of the imaging objective lens; Based on the pixel size used, the diffraction order, and the required spectral resolution, the grating line bundle is determined by the focal length of the imaging objective.

8. The hyperspectral imaging system based on prism-grating spectrometry according to claim 6, characterized in that: The system also includes a collimating objective lens arranged between the slit and the prism, and an imaging objective lens arranged between the grating and the detector; after the light emitted from the same field of view point on the slit is dispersed by the optical system, light of different wavelengths is imaged at different positions on the image plane, and since the light of the same field of view point has an angular component relative to the slit direction during the collimation and imaging process, unchanged, so the half-width of the spectral line at different wavelengths is expressed as: In the formula, ε′ represents the angular component of the slit direction, x represents the slit length, f′1 represents the focal length of the collimating objective lens, and f′2 represents the focal length of the imaging objective lens. The larger the angle between the system's outgoing light and the central wavelength light, the larger the half-width of the spectrum line, and the spectrum bending distortion is generated in the spatial dimension. The spectrum bending amount of the same field of view on the image plane is the difference between the spatial coordinates of the non-central wavelength and the central wavelength: Δx′=x′ λ -x′0=f′2tanε′[sec(φ′ λ -φ′0)-1] In the formula, x′0 represents the half-width of the spectrum line at the central wavelength, and φ′0 represents the grating diffraction angle of the central wavelength; When sec(φ′ λ -φ′0) hours, which can be expressed approximately in radians as Tanε′ is approximately expressed in radians as get: In the formula, Representing the magnification of the imaging system, the spectral curvature is proportional to the focal length of the objective, the square of the difference in the exit angle between the non-central wavelength and the central wavelength, and the slit length.

9. The hyperspectral imaging system based on prism-grating spectrometry according to claim 8, characterized in that: When the focal lengths of the collimating objective and the imaging objective are the same, m=1, and the spectral curvature has nothing to do with the focal length of the objective, but is only proportional to the square of the difference between the exit angles of the non-central wavelength and the central wavelength and the slit length.

10. The hyperspectral imaging system based on prism-grating spectrometry according to claim 6, characterized in that: The system further comprises a collimating objective lens arranged between the slit and the prism, and an imaging objective lens arranged between the grating and the detector; the incident surface of the grating is parallel to the exit surface of the prism, and the refraction angle i′2 of the incident surface of the prism is equal to the incident angle φ of the grating. According to the refraction angle of the incident surface of the prism, the light dispersion angle range corresponding to a certain wavelength range at the central field of view of the slit is obtained, so that the focal length of the collimating objective lens is consistent with that of the imaging objective lens, and the main light of the central wavelength is used as the optical axis of the imaging objective lens, that is, the imaging objective lens is tilted and the optical axis forms a certain angle with the optical axis of the collimating objective lens, and this angle is the dispersion angle of the central wavelength. According to the dispersion angle and the imaging area of ​​the detector, the actual focal length of the objective lens is obtained: In the formula, θ max Indicates the maximum value of the light dispersion angle, θ min represents the minimum value of the light dispersion angle, and θ0 represents the dispersion angle of the central wavelength.

Images