Push-broom spectral imaging system and imaging method thereof

Abstract

The present invention discloses a push-broom spectral imaging system and an exposure method thereof. The push-broom spectral imaging system includes a main imaging channel and an auxiliary imaging channel. The main imaging channel obtains spectral information and spatial information of a first field of view, and the auxiliary imaging channel obtains a second image of a second field of view. The first field of view of the main imaging channel can be mapped to the second field of view of the auxiliary imaging channel. The method includes: synchronously imaging the main imaging channel and the auxiliary imaging channel while moving in a set direction to obtain a second image at the current imaging moment; obtaining a first region of interest and a second region of interest from the second image at the current imaging moment, and adjusting the exposure duration of the main imaging channel at the current imaging moment based on the pixel response values ​​of the first region of interest and the second region of interest to obtain the exposure duration of the imaging at the next imaging moment. The present invention realizes dynamic adjustment of the exposure integration time of the push-broom spectral imaging system.

Description

Technical Field

[0001] The present invention relates to the field of imaging technology, and in particular to a push-broom spectral imaging system and an imaging method thereof. Background Art

[0002] Hyperspectral imaging technology can simultaneously acquire both spectral and spatial information about a target, and has been widely used in agriculture, forestry, ecology, and industry. Currently, hyperspectral imaging uses a fixed exposure method. When spectral imaging equipment is mounted on a drone platform for dynamic imaging during flight missions, the reflectivity of certain scenes varies significantly. This fixed exposure method can cause the acquired spectral image to appear partially overexposed or dark, affecting the accuracy of subsequent data inversion. Summary of the Invention

[0003] The purpose of the present invention is to provide a push-broom spectral imaging system and an imaging method thereof, which can realize dynamic adjustment of the exposure time of spectral imaging.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] An imaging method for push-broom spectral imaging, wherein the push-broom spectral imaging system includes a primary imaging channel and an auxiliary imaging channel, wherein the primary imaging channel is used to obtain spectral information and spatial information of a first field of view, and the auxiliary imaging channel is used to obtain a second image of a second field of view, wherein the first field of view of the primary imaging channel can be mapped to the second field of view of the auxiliary imaging channel, so that a region of interest corresponding to the first field of view exists in the second image;

[0006] The imaging method for push-broom spectral imaging includes:

[0007] When the main imaging channel and the auxiliary imaging channel move along the set direction, the main imaging channel and the auxiliary imaging channel are synchronously imaged to obtain the second image acquired by the auxiliary imaging channel at the current imaging moment;

[0008] Acquire a first region of interest and a second region of interest from the second image at a current imaging moment according to a mapping relationship between the first field of view of the primary imaging channel and the second field of view of the auxiliary imaging channel, wherein the first region of interest is a region of interest in the second image at the current imaging moment mapped from the first field of view at the current imaging moment, and the second region of interest is a region of interest in the second image at the current imaging moment mapped from the first field of view at a next imaging moment;

[0009] According to the pixel response value of the first region of interest and the pixel response value of the second region of interest, the exposure duration of the main imaging channel at the current imaging moment is adjusted, and the adjusted exposure duration is used as the exposure duration of the main imaging channel at the next imaging moment, so that the main imaging channel performs imaging at the next imaging moment.

[0010] Optionally, the imaging surface of the main imaging channel includes at least one strip, the strip includes a plurality of picture elements arranged in sequence along the length direction of the strip, the length direction of the strip corresponds to the spatial dimension, and the first field of view is the field of view corresponding to the strip.

[0011] Optionally, the spatial field of view of the primary imaging channel is mapped to the horizontal field of view of the auxiliary imaging channel, and the setting direction is perpendicular to the length direction of the strip;

[0012] Acquiring a first region of interest and a second region of interest from the second image at a current imaging moment according to a mapping relationship between the first field of view of the primary imaging channel and the second field of view of the auxiliary imaging channel includes:

[0013] In the second image at the current imaging moment, the first region of interest is represented as [(i, j), (i+kl, j+k-1)], and the second region of interest is represented as [(i, j+ck), (i+kl, j+k-1+ck)];

[0014] The number of pixels in the strip is 1×1, iFoV 主 ≈k×iFoV 辅 , iFOV 主 The spectral field of view of the pixel of the main imaging channel is the same as the spatial field of view of the pixel of the main imaging channel. iFOV 辅 Represents the vertical field of view of the pixel of the auxiliary imaging channel. The horizontal field of view of the pixel of the auxiliary imaging channel is consistent with its vertical field of view. c represents that the main imaging channel scans the spectral field of view of c pixels along the set direction from the current imaging moment to the next imaging moment.

[0015] Optionally, wherein

[0016] , ;

[0017] Among them, A H represents the pixel size of the main imaging channel, f H represents the focal length of the lens of the main imaging channel, the number of pixels on the imaging surface of the auxiliary imaging channel is p×q, and A M represents the pixel size of the auxiliary imaging channel, f Mrepresents the focal length of the lens of the auxiliary imaging channel, the center distance between the main imaging channel and the auxiliary imaging channel is expressed as (b, d), and h represents the imaging distance.

[0018] Optionally, wherein

[0019] , ;

[0020] Among them, A H represents the pixel size of the main imaging channel, f H represents the focal length of the lens of the main imaging channel, the number of pixels on the imaging surface of the auxiliary imaging channel is p×q, and A M represents the pixel size of the auxiliary imaging channel, f M represents the focal length of the lens of the auxiliary imaging channel, the center distance between the main imaging channel and the auxiliary imaging channel is expressed as (b, d), and h represents the imaging distance.

[0021] Optionally, adjusting the exposure duration of the main imaging channel at the current imaging moment according to the pixel response value of the first region of interest and the pixel response value of the second region of interest includes:

[0022] Adjusting the exposure duration of the main imaging channel at the current imaging moment according to the spectral response relationship between the main imaging channel and the auxiliary imaging channel, the pixel response value of the first region of interest, and the pixel response value of the second region of interest, wherein the spectral response relationship between the main imaging channel and the auxiliary imaging channel describes the relationship between the spectral response of all spectral-dimensional pixels corresponding to each spatial-dimensional pixel of the main imaging channel and the spectral response of the corresponding pixel of the auxiliary imaging channel, and the corresponding pixel of the auxiliary imaging channel refers to the mapping of all spectral-dimensional pixels corresponding to each spatial-dimensional pixel of the main imaging channel to the pixel corresponding to the auxiliary imaging channel;

[0023] Obtaining the spectral response relationship between the primary imaging channel and the auxiliary imaging channel includes:

[0024] For each spatial dimension pixel of each spectral dimension of the main imaging channel, obtaining a pixel response value of the spatial dimension pixel of the spectral dimension according to the spectral radiance of the spectral dimension at the entrance pupil of the main imaging channel, the spectral response function of the spatial dimension pixel of the spectral dimension, and the wavelength range of the spectral dimension;

[0025] For any spatial dimension pixel of the main imaging channel, the sum of the response values ​​of the pixels corresponding to the spatial dimension pixel in all spectral dimensions is obtained, which is expressed as the first response value sum;

[0026] For any spatial dimension pixel of the main imaging channel, determine that the pixels corresponding to the spatial dimension pixel in all spectral dimensions are mapped to the pixels corresponding to the auxiliary imaging channel, and obtain the sum of the response values ​​of the pixels corresponding to the auxiliary imaging channel, which is expressed as a second response value sum;

[0027] A spectral response relationship between the main imaging channel and the auxiliary imaging channel is obtained according to the first response value sum and the second response value sum.

[0028] Optionally, adjusting the exposure duration of imaging of the main imaging channel at a current imaging moment according to the pixel response value of the first region of interest and the pixel response value of the second region of interest, and using the adjusted exposure duration as the exposure duration of imaging of the main imaging channel at a next imaging moment includes:

[0029] Obtaining a first preset statistical value of the pixel response value of the first region of interest and a second preset statistical value of the pixel response value of the second region of interest;

[0030] According to the linear relationship satisfied by the first preset statistical value and the second preset statistical value, the exposure duration of the main imaging channel at the current imaging moment is adjusted, and the adjusted exposure duration is used as the exposure duration of the main imaging channel at the next imaging moment.

[0031] A push-broom spectral imaging system, comprising:

[0032] A main imaging channel, used to obtain spectral information and spatial information of the first field of view;

[0033] an auxiliary imaging channel for acquiring a second image of a second field of view, wherein the first field of view of the primary imaging channel can be mapped into the second field of view of the auxiliary imaging channel so that a region of interest corresponding to the first field of view exists in the second image;

[0034] A control device is used to synchronize imaging of the main imaging channel and the auxiliary imaging channel as they move along a set direction, obtain the second image acquired by the auxiliary imaging channel at a current imaging moment, obtain a first region of interest and a second region of interest from the second image at the current imaging moment based on a mapping relationship between the first field of view of the main imaging channel and the second field of view of the auxiliary imaging channel, adjust the exposure duration of imaging of the main imaging channel at the current imaging moment based on pixel response values ​​of the first region of interest and the pixel response values ​​of the second region of interest, and use the adjusted exposure duration as the exposure duration of imaging of the main imaging channel at a next imaging moment, wherein the first region of interest is a region of interest mapped from the first field of view at the current imaging moment to the second image at the current imaging moment, and the second region of interest is a region of interest mapped from the first field of view at the next imaging moment to the second image at the current imaging moment.

[0035] Optionally, the primary imaging channel includes a first optical lens, a spectrum acquisition device, and a first image sensor, wherein the first optical lens is used to collect light, the spectrum acquisition device is used to separate the light acquired by the first optical lens into different spectral lights, and the first image sensor is used to receive the different spectral lights to generate an image;

[0036] The auxiliary imaging channel includes a second optical lens and a second image sensor, the second optical lens is used to collect light from the second field of view, and the second image sensor is used to receive the light acquired by the second optical lens to generate the second image;

[0037] The first image sensor and the second image sensor are respectively connected to the control device.

[0038] Optionally, the system further comprises: a preview display device connected to the control device, configured to display the second image and to superimpose a region of interest corresponding to the first field of view on the second image.

[0039] It can be seen from the above technical solution that the present invention provides a push-broom spectral imaging system and an imaging method thereof. The push-broom spectral imaging system includes a main imaging channel and an auxiliary imaging channel. The main imaging channel is used to obtain spectral information and spatial information of a first field of view, and the auxiliary imaging channel is used to obtain a second image of a second field of view. The first field of view of the main imaging channel can be mapped to the second field of view of the auxiliary imaging channel, so that there is an area of ​​interest corresponding to the first field of view in the second image. The method includes: when the main imaging channel and the auxiliary imaging channel move along a set direction, the main imaging channel and the auxiliary imaging channel are synchronously imaged to obtain a second image obtained by the auxiliary imaging channel at the current imaging moment; according to the mapping relationship between the first field of view of the main imaging channel and the second field of view of the auxiliary imaging channel, a first region of interest and a second region of interest are obtained from the second image at the current imaging moment, the first region of interest being the region of interest in the second image mapped from the first field of view at the current imaging moment, and the second region of interest being the region of interest in the second image mapped from the first field of view at the next imaging moment; further, according to the pixel response value of the first region of interest and the pixel response value of the second region of interest, the exposure duration of the main imaging channel at the current imaging moment is adjusted, and the adjusted exposure duration is used as the exposure duration of the main imaging channel at the next imaging moment, so that the main imaging channel performs imaging at the next imaging moment. The push-broom spectral imaging system and the imaging method thereof of the present invention realize dynamic adjustment of the exposure duration of the push-broom spectral imaging system. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A flowchart of an imaging method for push-broom spectral imaging provided by one embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the pixel mapping relationship between the imaging plane of the primary imaging channel and the imaging plane of the auxiliary imaging channel of a push-broom spectral imaging system provided by one embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the field of view matching relationship between the main imaging channel and the auxiliary imaging channel of a push-broom spectral imaging system provided in one embodiment of the present invention;

[0044] Figure 4A spectral response characteristic diagram of a main imaging channel and an auxiliary imaging channel of a push-broom spectral imaging system provided by one embodiment of the present invention;

[0045] Figure 5 A flowchart of an imaging method for push-broom spectral imaging provided by yet another embodiment of the present invention;

[0046] Figure 6 A schematic diagram of a push-broom spectral imaging system provided by one embodiment of the present invention;

[0047] Figure 7 A schematic diagram of field of view mapping of a push-broom spectral imaging system provided by one embodiment of the present invention.

[0048] The reference numerals in the drawings of the specification include:

[0049] 1-first optical lens, 2-spectrum acquisition device, 3-first image sensor, 4-second optical lens, 5-second image sensor, 6-processor. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0051] This embodiment provides an imaging method for push-broom spectral imaging. The push-broom spectral imaging system used includes a main imaging channel and an auxiliary imaging channel. The main imaging channel is used to obtain spectral information and spatial information of a first field of view, and the auxiliary imaging channel is used to obtain a second image of a second field of view. The first field of view of the main imaging channel can be mapped to the second field of view of the auxiliary imaging channel, so that a region of interest corresponding to the first field of view exists in the second image.

[0052] The primary imaging channel can acquire spectral and spatial information, achieving spectral imaging. The secondary imaging channel's second field of view (FOV) is used to obtain an image of the second field of view, known as the secondary image. Furthermore, the primary imaging channel's first FOV can be mapped to the secondary imaging channel's second FOV, resulting in a region in the second image corresponding to the first FOV, known as the region of interest.

[0053] For reference Figure 1 , Figure 1A flowchart of an imaging method for push-broom spectral imaging is provided in accordance with an embodiment. As shown in the figure, the imaging method for push-broom spectral imaging includes the following steps:

[0054] S11: When the main imaging channel and the auxiliary imaging channel move along a set direction, the main imaging channel and the auxiliary imaging channel perform synchronous imaging to obtain the second image acquired by the auxiliary imaging channel at the current imaging moment.

[0055] When the main imaging channel and the auxiliary imaging channel move along the set direction, the main imaging channel and the auxiliary imaging channel perform imaging synchronously every time an imaging moment is reached.

[0056] S12: According to the mapping relationship between the first field of view of the main imaging channel and the second field of view of the auxiliary imaging channel, obtain a first region of interest and a second region of interest from the second image at the current imaging moment, the first region of interest is the region of interest in the second image at the current imaging moment mapped from the first field of view at the current imaging moment, and the second region of interest is the region of interest in the second image at the next imaging moment mapped from the first field of view at the current imaging moment.

[0057] The first field of view of the main imaging channel and the second field of view of the auxiliary imaging channel satisfy that in the second image at the current imaging moment, there is an area corresponding to the first field of view of the main imaging channel at the current imaging moment, and there is also an area corresponding to the first field of view of the main imaging channel at the next imaging moment.

[0058] The first region of interest and the second region of interest are obtained from the second image obtained at the current imaging moment. The first region of interest is the first field of view of the main imaging channel at the current imaging moment mapped to the corresponding region in the second image at the current imaging moment. The second region of interest is the first field of view of the main imaging channel at the next imaging moment mapped to the corresponding region in the second image at the current imaging moment.

[0059] The first and second regions of interest can be acquired from the second image at the current imaging moment based on the mapping relationship between the first field of view of the main imaging channel and the second field of view of the auxiliary imaging channel and based on the movement of the main imaging channel and the auxiliary imaging channel along the set direction.

[0060] S13: According to the pixel response value of the first region of interest and the pixel response value of the second region of interest, adjust the exposure duration of the main imaging channel at the current imaging moment, and use the adjusted exposure duration as the exposure duration of the main imaging channel at the next imaging moment, so that the main imaging channel performs imaging at the next imaging moment.

[0061] The imaging method for push-broom spectral imaging of this embodiment uses a push-broom spectral imaging system that includes not only a main imaging channel but also an auxiliary imaging channel. When the main imaging channel and the auxiliary imaging channel move along a set direction, the main imaging channel and the auxiliary imaging channel are synchronously imaged. In the second image obtained at the current imaging moment, the area of ​​the first field of view of the main imaging channel corresponding to the current imaging moment and the area of ​​the first field of view of the main imaging channel corresponding to the next imaging moment are obtained. The exposure duration of the main imaging channel imaging is adjusted according to the pixel response values ​​of the two areas, thereby realizing dynamic adjustment of the exposure duration of the spectral imaging.

[0062] In some embodiments, the imaging surface of the main imaging channel includes at least one strip, and the strip includes a plurality of pixels sequentially arranged along the length direction of the strip, the length direction of the strip corresponds to the spatial dimension, and the first field of view is the field of view corresponding to the strip. Any strip can correspond to any spectral dimension of the main imaging channel, and each pixel of any strip obtains spatial information of the spectral dimension corresponding to the strip. In this embodiment, in the second image obtained by the auxiliary imaging channel, the area of ​​the field of view mapping corresponding to a single strip of the main imaging channel is obtained to adjust the exposure time of the main imaging channel according to the pixel response value of the mapping area, which can reduce the amount of calculation and help improve the efficiency of dynamically adjusting the exposure time of the main imaging channel. The field of view corresponding to a single strip of the main imaging channel can be called an instantaneous field of view.

[0063] In some embodiments, the spatial dimension field of view of the primary imaging channel is mapped to the horizontal field of view of the auxiliary imaging channel. If the imaging plane of the primary imaging channel includes at least one stripe, the stripe includes a plurality of pixels arranged sequentially along the length of the stripe, and the length of the stripe corresponds to the spatial dimension, then the length of the stripe corresponds to the lateral direction of the imaging plane of the auxiliary imaging channel.

[0064] In some embodiments, the following conditions are met: HFoV 辅 ≥FoV 主 ; Among them, HFoV 辅 Represents the horizontal field of view of the auxiliary imaging channel, FoV 主 Denotes the spatial dimension field of view of the main imaging channel. Satisfying HFoV 辅 ≥FoV 主 To ensure that the spatial dimensional field of view of the main imaging channel can be mapped to the field of view of the auxiliary imaging channel.

[0065] In some embodiments, the following conditions are met: iFoV 主 ≥iFoV 辅 Among them, iFOV 辅 Represents the vertical field of view of the pixel of the auxiliary imaging channel, iFOV 主 Represents the spectral field of view of the pixel of the main imaging channel. 主≥iFoV 辅 To ensure that the main imaging channel can obtain a wider range of spectral information.

[0066] For example, you can refer to Figure 2 , Figure 2 A schematic diagram of the pixel mapping relationship between the imaging plane of the main imaging channel and the imaging plane of the auxiliary imaging channel of a push-broom spectral imaging system provided in one embodiment, wherein HSI represents the imaging plane of the main imaging channel and MONO represents the imaging plane of the auxiliary imaging channel. Each row of pixels on the imaging plane HSI of the main imaging channel represents a spatial dimension, and each column of pixels represents a spectral dimension. The λ marked on the pixel 11 ,λ 12 ,λ 1m ,λ 21 ,λ 22 or n1 Indicates the central wavelength of light that can cause the pixel to respond. Figure 3 , Figure 3 A schematic diagram of the field of view matching relationship between the main imaging channel and the auxiliary imaging channel of a push-broom spectral imaging system provided in one embodiment is provided. The spatial dimension field of view of the main imaging channel corresponds to the horizontal field of view of the auxiliary imaging channel, the spectral dimension field of view of the main imaging channel corresponds to the vertical field of view of the auxiliary imaging channel, and the length direction of the strip of the imaging surface of the main imaging channel corresponds to the lateral direction of the imaging surface of the auxiliary imaging channel.

[0067] In some embodiments, the spatial dimension field of view of the main imaging channel is mapped to the horizontal field of view of the auxiliary imaging channel. The imaging surface of the main imaging channel includes at least one strip, and the strip includes a plurality of pixels sequentially arranged along the length direction of the strip. The length direction of the strip corresponds to the spatial dimension, and the first field of view is the field of view corresponding to the strip. The setting direction is perpendicular to the length direction of the strip. Accordingly, according to the mapping relationship between the first field of view of the main imaging channel and the second field of view of the auxiliary imaging channel, obtaining the first region of interest and the second region of interest from the second image at the current imaging moment includes: in the second image at the current imaging moment, the first region of interest is expressed as [(i, j), (i+kl, j+k-1)], and the second region of interest is expressed as [(i, j+ck), (i+kl, j+k-1+ck)]; wherein the number of pixels in the strip is 1×l, iFoV 主 ≈k×iFoV 辅 , iFOV 主 The spectral field of view of the pixel of the main imaging channel is the same as the spatial field of view of the pixel of the main imaging channel. iFOV 辅Represents the vertical field of view of the pixel of the auxiliary imaging channel. The horizontal field of view of the pixel of the auxiliary imaging channel is consistent with its vertical field of view. c represents that the main imaging channel scans the spectral field of view of c pixels along the set direction from the current imaging moment to the next imaging moment.

[0068] The first ROI is represented as [(i, j), (i+kl, j+k-1)], where the starting pixel of the first ROI is (i, j), representing the pixel in the i-th column and j-th row, and the ending pixel of the first ROI is (i+kl, j+k-1), representing the pixel in the i+kl-th column and j+k-1-th row. The second ROI′ is represented as [(i, j+ck), (i+kl, j+k-1+ck)], where the starting pixel of the second ROI′ is (i, j+ck), representing the pixel in the i-th column and j+ck-th row, and the ending pixel of the second ROI′ is (i+kl, j+k-1+ck), representing the pixel in the i+kl-th column and j+k-1+ck-th row. The pixels correspond to the pixels on the imaging surface.

[0069] In some embodiments, k is a positive integer greater than or equal to 1, iFoV 主 ≈k×iFoV 辅 Indicates iFoV 主 / iFoV 辅 This ratio is rounded to k. When designing the auxiliary imaging channel, it can be made to meet iFoV 主 ≈k×iFoV 辅 For any pixel of the primary imaging channel, its spatial field of view is consistent with its spectral field of view. For any pixel of the auxiliary imaging channel, its horizontal field of view is consistent with its vertical field of view. According to the positions of the first region of interest and the second region of interest in the second image at the current imaging moment, it can be seen that in the process from the current imaging moment to the next imaging moment, the corresponding area of ​​any band of the primary imaging channel in the second image moves vertically by ck rows of pixels along the second image.

[0070] In some embodiments, c may be a positive integer greater than or equal to 1. Exemplarily, c may be 1. Accordingly, in the second image at the current imaging moment, the second region of interest ROI′ is expressed as [(i, j+k), (i+k1, j+2k-1)].

[0071] (i, j) can be determined and obtained based on the mapping relationship between the first field of view of the primary imaging channel and the second field of view of the auxiliary imaging channel. In some embodiments:

[0072] , ;

[0073] Among them, AH represents the pixel size of the main imaging channel, f H represents the focal length of the lens of the main imaging channel, the number of pixels on the imaging surface of the auxiliary imaging channel is p×q, and A M represents the pixel size of the auxiliary imaging channel, f M represents the focal length of the lens of the auxiliary imaging channel, the center distance between the main imaging channel and the auxiliary imaging channel is expressed as (b, d), and h represents the imaging distance.

[0074] The imaging plane of the main imaging channel and the imaging plane of the auxiliary imaging channel are on the same reference, which makes it easy to determine the relative position of the two imaging channels and to find the region of interest corresponding to the first field of view of the main imaging channel in the second image obtained by the auxiliary imaging channel. For example, an o-xyz three-dimensional coordinate system is established, where the imaging plane of the main imaging channel is parallel to the xoy plane, and the imaging plane of the auxiliary imaging channel is parallel to the xoy plane. The imaging planes of the two are at the same position on the z axis, that is, in the z direction, H =z M , z H Indicates the position of the imaging surface of the main imaging channel on the z axis, z M Indicates the position of the imaging plane of the auxiliary imaging channel on the z-axis. The center distance between the imaging planes of the primary and auxiliary imaging channels in the x and y directions is (d, b).

[0075] Assumptions:

[0076] The pixel size of the imaging surface of the main imaging channel is A H , the number of strip pixels is 1×l, and the focal length of the lens of the main imaging channel is f H .

[0077] The pixel size of the imaging surface of the auxiliary imaging channel is A M , the number of pixels is p×q, and the focal length of the auxiliary imaging channel lens is f M .

[0078] When designing the auxiliary imaging channel, the following requirements must be met:

[0079] 1) iFoV 主 ≈k×iFoV 辅 ,Right now . Where k = 1, 2, 3, .... Generally A H <<f H , A M <<f M ,Right now .

[0080] 2) HFoV 辅 ≥FoV 主 ,Right now .

[0081] The center distance between the main imaging channel and the auxiliary imaging channel is (b, d), the imaging distance is h, and the field of view corresponding to a band of the main imaging channel at the current imaging moment is mapped to the region of interest in the second image at the current imaging moment, namely the first region of interest, whose starting pixel position is (i, j).

[0082] According to the geometric relationship: image plane size / focal length = object plane size corresponding to image plane size / imaging distance, we can get:

[0083] , .

[0084] Right now: , .

[0085] If b<<h, d<<h during imaging, then we can get:

[0086] , .

[0087] For example, the push-broom spectral imaging system is mounted on a UAV. Generally, when the UAV performs aerial photography, b<<h and d<<h.

[0088] In some embodiments, adjusting the exposure time of the main imaging channel at the current imaging moment based on the pixel response value of the first region of interest and the pixel response value of the second region of interest includes: adjusting the exposure time of the main imaging channel at the current imaging moment based on the spectral response relationship between the main imaging channel and the auxiliary imaging channel, the pixel response value of the first region of interest, and the pixel response value of the second region of interest, wherein the spectral response relationship between the main imaging channel and the auxiliary imaging channel describes the relationship between the spectral response of all spectral dimensional pixels corresponding to each spatial dimensional pixel of the main imaging channel and the spectral response of the corresponding pixel of the auxiliary imaging channel, and the corresponding pixel of the auxiliary imaging channel refers to the mapping of all spectral dimensional pixels corresponding to each spatial dimensional pixel of the main imaging channel to the pixel corresponding to the auxiliary imaging channel.

[0089] In some embodiments, the spectral response relationship between the primary imaging channel and the auxiliary imaging channel may be obtained by the following process, including the following steps:

[0090] S21: For each spatial dimension pixel of each spectral dimension of the main imaging channel, obtain a pixel response value of the spatial dimension pixel of the spectral dimension according to the spectral radiance of the spectral dimension at the entrance pupil of the main imaging channel, the spectral response function of the spatial dimension pixel of the spectral dimension, and the wavelength range of the spectral dimension;

[0091] S22: For any spatial dimension pixel of the main imaging channel, obtain the sum of response values ​​of pixels corresponding to the spatial dimension pixel in all spectral dimensions, which is expressed as a first response value sum;

[0092] S23: For any spatial dimension pixel of the main imaging channel, determine that pixels corresponding to the spatial dimension pixel in all spectral dimensions are mapped to pixels corresponding to the auxiliary imaging channel, and obtain a sum of response values ​​of the pixels corresponding to the auxiliary imaging channel, expressed as a second response value sum;

[0093] S24: Obtaining a spectral response relationship between the main imaging channel and the auxiliary imaging channel according to the first response value sum and the second response value sum.

[0094] Obtaining the spectral response relationship between the main imaging channel and the auxiliary imaging channel can be considered as constructing a mapping relationship between the spectral responses of the main imaging channel and the auxiliary imaging channel. For example, in actual construction, since the distance between the drone and the ground surface is far, ignoring the influence of the angle, the spectral radiance of the ground surface reflected energy reaching the entrance pupil of the two imaging channels is the same. Assuming that the spectral radiance at the entrance pupil is L(λ), the spectral response function S(λ) of the spatial dimension pixel of any instantaneous field of view of the main imaging channel is i ), any instantaneous field of view of the main imaging channel is the field of view of the strip corresponding to a spectral dimension of the main imaging channel. Then the pixel response value of any spatial dimension pixel of any spectral dimension of the main imaging channel is expressed as:

[0095] ;

[0096] Among them, DN H_i represents the pixel response value of any spatial dimension pixel in the ith spectral dimension of the main imaging channel, L(λ i ) represents the spectral radiance of the i-th spectral dimension at the entrance pupil of the main imaging channel, S(λ i ) represents the spectral response function of any spatial pixel in the i-th spectral dimension of the main imaging channel, K H_i 、B H_i Represent parameters, [λ i_l ,λ i_h ] represents the wavelength range of the i-th spectral dimension of the main imaging channel, i=1,2,…,N, and N represents the number of spectral dimensions of the main imaging channel.

[0097] Assuming that the first quantization bit number is α, for any spatial dimension pixel of the main imaging channel, the sum of the response values ​​of the pixels corresponding to the spatial dimension pixel in all spectral dimensions is obtained, and after normalization, the result is:

[0098] .

[0099] According to the mapping relationship between the field of view of the main imaging channel and the field of view of the auxiliary imaging channel, for any spatial dimension pixel of the main imaging channel, the pixel corresponding to the spatial dimension pixel in all spectral dimensions is determined to be mapped to the pixel corresponding to the auxiliary imaging channel. The response value of the pixel corresponding to the auxiliary imaging channel is expressed as:

[0100] ;

[0101] Among them, DN M_j It represents the response value of all spectral pixels corresponding to any spatial pixel of the main imaging channel mapped to the pixel corresponding to the auxiliary imaging channel, L(λ) represents the spectral radiance at the entrance pupil of the auxiliary imaging channel, S(λ) represents the spectral response function of the pixel of the auxiliary imaging channel, K M_j 、B M_j Represent parameters, [λ origin ,λ end ] represents the spectral range of the auxiliary imaging channel, j=1,2,…,M, M represents the number of all spectral-dimensional pixels corresponding to any spatial-dimensional pixel of the main imaging channel mapped to the pixels corresponding to the auxiliary imaging channel.

[0102] When mapping multiple pixels, normalization is also required. Assuming that the second quantization bit is β, we have:

[0103] .

[0104] The relationship between the spectral responses of all spectral-dimensional pixels corresponding to each spatial-dimensional pixel of the main imaging channel and the spectral responses of the corresponding pixels of the auxiliary imaging channel is constructed as follows:

[0105] .

[0106] This relationship can be measured by fitting the spectrum calibration of the whole machine:

[0107] .

[0108] For example, you can refer to Figure 4 , Figure 4 A graph showing the spectral response characteristics of the primary and secondary imaging channels of a push-broom spectral imaging system, provided in one embodiment, shows wavelength on the horizontal axis and relative response on the vertical axis. The response wavelength range of the primary and secondary imaging channels is consistent.

[0109] In some embodiments, adjusting the exposure duration of imaging of the main imaging channel at the current imaging moment according to the pixel response value of the first region of interest and the pixel response value of the second region of interest, and using the adjusted exposure duration as the exposure duration of imaging of the main imaging channel at the next imaging moment, may include the following steps:

[0110] S131: Acquire a first preset statistical value of the pixel response value of the first region of interest and a second preset statistical value of the pixel response value of the second region of interest;

[0111] S132: According to the linear relationship satisfied by the first preset statistical value and the second preset statistical value, adjust the exposure duration of the main imaging channel at the current imaging moment, and use the adjusted exposure duration as the exposure duration of the main imaging channel at the next imaging moment.

[0112] Preset statistics are performed on the response values ​​of each pixel in the first region of interest to obtain a first preset statistical value, and preset statistics are performed on the response values ​​of each pixel in the second region of interest to obtain a second preset statistical value. In this embodiment, the specific method of the preset statistics is not limited and can be, but is not limited to, calculating the average of the response values ​​of each pixel in the region of interest.

[0113] According to the first preset statistical value and the second preset statistical value, the linear relationship satisfied by the two is obtained, which can be obtained:

[0114] ;

[0115] Among them, DN M_cur Indicates the first preset statistical value, DN M_pre represents the second preset statistical value, and ρ represents a coefficient. ρ can be called an adjustment coefficient.

[0116] The exposure time of the main imaging channel at the next imaging moment can be expressed as: , Indicates the exposure time of the main imaging channel at the next imaging moment. Indicates the exposure time of the main imaging channel at the current imaging moment.

[0117] For example, the mean response value of each pixel in the first region of interest ROI can be obtained, which is expressed as:

[0118] .

[0119] The mean response value of each pixel in the second region of interest ROI′ can be obtained, which is expressed as:

[0120] .

[0121] available: .

[0122] Because the spectral response relationship between the primary and secondary imaging channels is obtained by spectrally calibrating the primary and secondary imaging channels, a linear relationship exists between the spectral responses of all spectral-dimensional pixels corresponding to each spatial-dimensional pixel of the primary imaging channel and the spectral responses of the corresponding pixels of the secondary imaging channels. Based on this, the exposure duration of the primary imaging channel at the current imaging moment is adjusted based on the linear relationship satisfied by the first and second preset statistical values ​​to obtain the exposure duration of the primary imaging channel at the next imaging moment. The exposure duration may be the integral duration of the exposure.

[0123] In some embodiments, for the second image obtained by the auxiliary imaging channel, a histogram can be used to statistically analyze the pixel response value distribution of each pixel in the first region of interest / second region of interest, and an exposure strategy for strong light suppression can be implemented to ensure that there is no overexposure in the area.

[0124] For example, you can refer to Figure 5 , Figure 5 A flowchart of an imaging method for push-broom spectral imaging is provided for another embodiment, including first performing radiation calibration and geometric calibration on the push-broom spectral imaging system. Radiation calibration includes determining the radiation response model of the main and auxiliary imaging channels, and establishing the spectral radiance response relationship between the two. Geometric calibration includes determining the spatial mapping relationship between the instantaneous field of view of the main imaging channel and the full field of view of the auxiliary imaging channel, that is, matching the pixels corresponding to the instantaneous field of view, and clarifying the pixel positions of the region of interest of the auxiliary imaging channel through coordinate system transformation. The main and auxiliary image sensors are triggered synchronously to ensure that the acquired images are synchronized in time. The image sensors corresponding to the main and auxiliary imaging channels are required to operate in slave mode or trigger mode. The trigger signal sent by the processor ensures that the two channels are triggered and exposed at the same time and have the same AE statistical benchmark.

[0125] This embodiment further provides a push-broom spectral imaging system, comprising:

[0126] A main imaging channel, used to obtain spectral information and spatial information of the first field of view;

[0127] an auxiliary imaging channel for acquiring a second image of a second field of view, wherein the first field of view of the primary imaging channel can be mapped into the second field of view of the auxiliary imaging channel so that a region of interest corresponding to the first field of view exists in the second image;

[0128] A control device is used to synchronize imaging of the main imaging channel and the auxiliary imaging channel as they move along a set direction, obtain the second image acquired by the auxiliary imaging channel at a current imaging moment, obtain a first region of interest and a second region of interest from the second image at the current imaging moment based on a mapping relationship between the first field of view of the main imaging channel and the second field of view of the auxiliary imaging channel, adjust the exposure duration of imaging of the main imaging channel at the current imaging moment based on pixel response values ​​of the first region of interest and the pixel response values ​​of the second region of interest, and use the adjusted exposure duration as the exposure duration of imaging of the main imaging channel at a next imaging moment, wherein the first region of interest is a region of interest mapped from the first field of view at the current imaging moment to the second image at the current imaging moment, and the second region of interest is a region of interest mapped from the first field of view at the next imaging moment to the second image at the current imaging moment.

[0129] In addition to the main imaging channel, the push-broom spectral imaging system of this embodiment also includes an auxiliary imaging channel. When the main imaging channel and the auxiliary imaging channel move along the set direction, the main imaging channel and the auxiliary imaging channel are synchronously imaged. In the second image obtained at the current imaging moment, the area of ​​the first field of view of the main imaging channel corresponding to the current imaging moment and the area of ​​the first field of view of the main imaging channel corresponding to the next imaging moment are obtained. The exposure time of the main imaging channel imaging is adjusted according to the pixel response values ​​of the two areas, thereby realizing dynamic adjustment of the exposure time of spectral imaging.

[0130] In some embodiments, the main imaging channel includes a first optical lens, a spectrum acquisition device, and a first image sensor, wherein the first optical lens is used to collect light, the spectrum acquisition device is used to separate the light acquired by the first optical lens into different spectral lights, and the first image sensor is used to receive the different spectral lights to generate an image; the auxiliary imaging channel includes a second optical lens and a second image sensor, the second optical lens is used to collect light from the second field of view, and the second image sensor is used to receive the light acquired by the second optical lens to generate the second image; the first image sensor and the second image sensor are respectively connected to the control device. For example, reference can be made to Figure 6 , Figure 6This is a schematic diagram of a push-broom spectral imaging system provided in one embodiment. The main imaging channel includes a first optical lens 1, a spectrum acquisition device 2, and a first image sensor 3. The first optical lens 1 collects light from the outside world. The collected light is separated into different spectral lights by the spectrum acquisition device 2. The different spectral lights are received by the first image sensor 3, and images of different spectral dimensions are generated by each strip of the first image sensor 3. The auxiliary imaging channel includes a second optical lens 4 and a second image sensor 5. The second optical lens 4 collects light from the outside world. The collected light is incident on the second image sensor 5, and the second image sensor 5 generates a second image based on the received light. The first optical lens 1 can be a telescope, allowing the push-broom spectral imaging system to be applied in the field of remote sensing. The first image sensor 3 can be an area array detector. The second image sensor 5 can be an area array detector or a multi-line array detector. The second image sensor 5 can be a full-color image sensor, such as a full-color CMOS image sensor. The second optical lens 4 can use a low-distortion wide-angle lens. The push-broom spectral imaging system uses a line push-broom method, with the motion direction perpendicular to the strip direction of the main imaging channel. The main imaging channel obtains spatial information of the one-dimensional line field of view, while the spectral information of the line field of view is obtained in the second dimension of the main imaging channel imaging surface. The control device can be a processor 6.

[0131] In some embodiments, the push-broom spectral imaging system further includes: a preview display device connected to the control device, for displaying the second image, and superimposing the display of the region of interest corresponding to the first field of view in the second image. Exemplarily, the push-broom spectral imaging system is mounted on a drone. Before the aircraft takes off or after landing, the imaging spectrum system generally has to shoot a calibration plate or calibration cloth. The calibration plate or calibration cloth has a standard reflectivity and is uniform, that is, the calibration plate correction link, which contains image data of the calibration plate or calibration cloth for later data preprocessing to invert the true reflectivity of the ground object. The push-broom spectral imaging system can be referred to Figure 7 , Figure 7 This is a schematic diagram of the field of view mapping of a push-broom spectral imaging system, provided in one embodiment. The marked strip represents the field of view corresponding to a strip on the imaging plane of the primary imaging channel, expressed as iFOV × FOV; the black-framed area represents the field of view of the auxiliary imaging channel (HFOV × VFOV). During system assembly, the relative positional relationship between the two fields of view can be determined. In the real-time video preview of the auxiliary imaging channel (which can be sent to a ground station for preview via a drone image transmission link, or the imaging spectral system itself has wireless transmission capabilities and accesses an internal web server for preview), the iFOV × FOV field of view position of the primary imaging channel can be superimposed on the video display. This effectively determines whether the calibration plate is within the imaging field of view of the primary detector (i.e., the focal plane of the spectral imaging system), ensuring the accuracy and efficiency of the calibration plate calibration process.

[0132] The push-broom spectral imaging system and imaging method of this embodiment can improve the automatic exposure accuracy during dynamic imaging, enhancing scene adaptability. The combination of the panchromatic auxiliary detector and the main detector not only enables pre-flight gray-plate radiometric calibration plate positioning but also enables post-flight fusion of hyperspectral and panchromatic images acquired during flight, improving the spatial resolution of remote sensing images.

[0133] The push-broom spectral imaging system and exposure method provided by the present invention are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is intended only to facilitate understanding of the method and core concept of the present invention. It should be noted that those skilled in the art may make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. An imaging method for push-broom spectral imaging, characterized in that: The push-broom spectral imaging system used in the application includes a main imaging channel and an auxiliary imaging channel, wherein the main imaging channel is used to obtain spectral information and spatial information of a first field of view, and the auxiliary imaging channel is used to obtain a second image of a second field of view. The first field of view of the main imaging channel can be mapped to the second field of view of the auxiliary imaging channel, so that a region of interest corresponding to the first field of view exists in the second image; The imaging method for push-broom spectral imaging includes: When the main imaging channel and the auxiliary imaging channel move along the set direction, the main imaging channel and the auxiliary imaging channel are synchronously imaged to obtain the second image acquired by the auxiliary imaging channel at the current imaging moment; Acquire a first region of interest and a second region of interest from the second image at a current imaging moment according to a mapping relationship between the first field of view of the primary imaging channel and the second field of view of the auxiliary imaging channel, wherein the first region of interest is a region of interest in the second image at the current imaging moment mapped from the first field of view at the current imaging moment, and the second region of interest is a region of interest in the second image at the current imaging moment mapped from the first field of view at a next imaging moment; According to the pixel response value of the first region of interest and the pixel response value of the second region of interest, the exposure duration of the main imaging channel at the current imaging moment is adjusted, and the adjusted exposure duration is used as the exposure duration of the main imaging channel at the next imaging moment, so that the main imaging channel performs imaging at the next imaging moment.

2. The imaging method for push-broom spectral imaging according to claim 1, characterized in that: The imaging surface of the main imaging channel includes at least one strip, and the strip includes a plurality of picture elements arranged in sequence along the length direction of the strip. The length direction of the strip corresponds to the spatial dimension, and the first field of view is the field of view corresponding to the strip.

3. The imaging method for push-broom spectral imaging according to claim 2, characterized in that: The spatial field of view of the main imaging channel is mapped to the horizontal field of view of the auxiliary imaging channel, and the setting direction is perpendicular to the length direction of the strip; Acquiring a first region of interest and a second region of interest from the second image at a current imaging moment according to a mapping relationship between the first field of view of the primary imaging channel and the second field of view of the auxiliary imaging channel includes: In the second image at the current imaging moment, the first region of interest is represented as [(i, j), (i+kl, j+k-1)], and the second region of interest is represented as [(i, j+ck), (i+kl, j+k-1+ck)]; The number of pixels in the strip is 1×1, iFoV 主 ≈k×iFoV 辅 , iFOV 主 The spectral field of view of the pixel of the main imaging channel is the same as the spatial field of view of the pixel of the main imaging channel. iFOV 辅 Represents the vertical field of view of the pixel of the auxiliary imaging channel. The horizontal field of view of the pixel of the auxiliary imaging channel is consistent with its vertical field of view. c represents that the main imaging channel scans the spectral field of view of c pixels along the set direction from the current imaging moment to the next imaging moment.

4. The imaging method for push-broom spectral imaging according to claim 3, characterized in that: in, ， ； Among them, A H represents the pixel size of the main imaging channel, f H represents the focal length of the lens of the main imaging channel, the number of pixels on the imaging surface of the auxiliary imaging channel is p×q, and A M represents the pixel size of the auxiliary imaging channel, f M represents the focal length of the lens of the auxiliary imaging channel, the center distance between the main imaging channel and the auxiliary imaging channel is expressed as (b, d), and h represents the imaging distance.

5. The imaging method for push-broom spectral imaging according to claim 3, characterized in that: in, ， ； Among them, A H represents the pixel size of the main imaging channel, f H represents the focal length of the lens of the main imaging channel, the number of pixels on the imaging surface of the auxiliary imaging channel is p×q, and A M represents the pixel size of the auxiliary imaging channel, f M represents the focal length of the lens of the auxiliary imaging channel, the center distance between the main imaging channel and the auxiliary imaging channel is expressed as (b, d), and h represents the imaging distance.

6. The imaging method for push-broom spectral imaging according to claim 1, characterized in that: Adjusting the exposure time of the main imaging channel at the current imaging moment according to the pixel response value of the first region of interest and the pixel response value of the second region of interest includes: Adjusting the exposure duration of the main imaging channel at the current imaging moment according to the spectral response relationship between the main imaging channel and the auxiliary imaging channel, the pixel response value of the first region of interest, and the pixel response value of the second region of interest, wherein the spectral response relationship between the main imaging channel and the auxiliary imaging channel describes the relationship between the spectral response of all spectral-dimensional pixels corresponding to each spatial-dimensional pixel of the main imaging channel and the spectral response of the corresponding pixel of the auxiliary imaging channel, and the corresponding pixel of the auxiliary imaging channel refers to the mapping of all spectral-dimensional pixels corresponding to each spatial-dimensional pixel of the main imaging channel to the pixel corresponding to the auxiliary imaging channel; Obtaining the spectral response relationship between the primary imaging channel and the auxiliary imaging channel includes: For each spatial dimension pixel of each spectral dimension of the main imaging channel, obtaining a pixel response value of the spatial dimension pixel of the spectral dimension according to the spectral radiance of the spectral dimension at the entrance pupil of the main imaging channel, the spectral response function of the spatial dimension pixel of the spectral dimension, and the wavelength range of the spectral dimension; For any spatial dimension pixel of the main imaging channel, the sum of the response values ​​of the pixels corresponding to the spatial dimension pixel in all spectral dimensions is obtained, which is expressed as the first response value sum; For any spatial dimension pixel of the main imaging channel, determine that the pixels corresponding to the spatial dimension pixel in all spectral dimensions are mapped to the pixels corresponding to the auxiliary imaging channel, and obtain the sum of the response values ​​of the pixels corresponding to the auxiliary imaging channel, which is expressed as a second response value sum; A spectral response relationship between the main imaging channel and the auxiliary imaging channel is obtained according to the first response value sum and the second response value sum.

7. The imaging method for push-broom spectral imaging according to claim 1, characterized in that: Adjusting the exposure duration of the main imaging channel at a current imaging moment according to the pixel response value of the first region of interest and the pixel response value of the second region of interest, and using the adjusted exposure duration as the exposure duration of the main imaging channel at a next imaging moment includes: Obtaining a first preset statistical value of the pixel response value of the first region of interest and a second preset statistical value of the pixel response value of the second region of interest; According to the linear relationship satisfied by the first preset statistical value and the second preset statistical value, the exposure duration of the main imaging channel at the current imaging moment is adjusted, and the adjusted exposure duration is used as the exposure duration of the main imaging channel at the next imaging moment.

8. A push-broom spectral imaging system, characterized in that: include: A main imaging channel, used to obtain spectral information and spatial information of the first field of view; an auxiliary imaging channel for acquiring a second image of a second field of view, wherein the first field of view of the primary imaging channel can be mapped into the second field of view of the auxiliary imaging channel so that a region of interest corresponding to the first field of view exists in the second image; A control device is used to synchronize imaging of the main imaging channel and the auxiliary imaging channel as they move along a set direction, obtain the second image acquired by the auxiliary imaging channel at a current imaging moment, obtain a first region of interest and a second region of interest from the second image at the current imaging moment based on a mapping relationship between the first field of view of the main imaging channel and the second field of view of the auxiliary imaging channel, adjust the exposure duration of imaging of the main imaging channel at the current imaging moment based on pixel response values ​​of the first region of interest and the pixel response values ​​of the second region of interest, and use the adjusted exposure duration as the exposure duration of imaging of the main imaging channel at a next imaging moment, wherein the first region of interest is a region of interest mapped from the first field of view at the current imaging moment to the second image at the current imaging moment, and the second region of interest is a region of interest mapped from the first field of view at the next imaging moment to the second image at the current imaging moment.

9. The push-broom spectral imaging system according to claim 8, characterized in that: The primary imaging channel includes a first optical lens, a spectrum acquisition device, and a first image sensor. The first optical lens is used to collect light. The spectrum acquisition device is used to separate the light acquired by the first optical lens into different spectral lights. The first image sensor is used to receive the different spectral lights to generate an image. The auxiliary imaging channel includes a second optical lens and a second image sensor, the second optical lens is used to collect light from the second field of view, and the second image sensor is used to receive the light acquired by the second optical lens to generate the second image; The first image sensor and the second image sensor are respectively connected to the control device.

10. The push-broom spectral imaging system according to claim 8, characterized in that: Also includes: A preview display device is connected to the control device and is used to display the second image and to superimpose a region of interest corresponding to the first field of view on the second image.

Images