Snapshot spectral camera crosstalk correction method, device and system

Abstract

The present application relates to a method, device and system for crosstalk correction of a snapshot spectral camera, and belongs to the technical field of instrumentation. The method comprises: obtaining light intensity information of different bands collected by a radiometer, and image information of different bands collected by a coated spectral camera; converting the image information into current information; for each band, determining a quantum efficiency curve based on the light intensity information and current information corresponding to the band; performing Gaussian fitting on the quantum efficiency curve of each band to obtain an expected Gaussian curve of each band; determining a regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve; using the regression coefficient to perform crosstalk removal processing on the quantum efficiency curve to obtain a quantum efficiency curve after crosstalk removal; the influence of the light source, monochromator and integrating sphere on the curve can be eliminated; the crosstalk of different bands can be well suppressed, thereby solving the spectral crosstalk problem of the snapshot spectral camera and reducing the difficulty of manufacturing pixel-level filters.

Description

Technical Field

[0001] The present application relates to a method, device and system for crosstalk correction of a snapshot spectral camera, and belongs to the technical field of instrumentation. Background Art

[0002] Spectral cameras can use multispectral and hyperspectral imaging technology to obtain and analyze the spectral information of the surface of an object. It not only captures the spatial information of the object, but also obtains the brightness value of each pixel at different wavelengths to generate a spectral image containing spatial and spectral information. Traditional spectral cameras are divided into grating type, wheel filter type and FT type.

[0003] Among them, the grating type spectral camera uses a prism grating prism (Prism-Grating-Prism, PGP) as a spectroscopic device, and uses an external push-scan or an internal push-scan to obtain three-dimensional spectral data, which sacrifices its time and is not time-effective.

[0004] The filter wheel type spectral camera rotates through a mechanical structure to place filters of different wavelengths in front of the area array camera to obtain three-dimensional spectral data, which sacrifices its time and is not time-effective.

[0005] The FT spectral camera generates an optical path difference by swinging the corner mirror. The area array detector collects data continuously. Each pixel collects an interference pattern and obtains three-dimensional spectral data through Fourier transform. This sacrifices its time and is not time-effective.

[0006] In order to improve timeliness, snapshot spectral cameras are usually used to collect spectral images. Currently, there are two mainstream snapshot spectral cameras: pixel-level filter-based spectral cameras and computational imaging spectral cameras. Both spectral cameras sacrifice spatial resolution to achieve snapshot effects.

[0007] However, pixel-level filters are subject to processing technology, and wavelength overlap and wavelength leakage may occur, that is, some bands of light will enter other pixels, resulting in spectral crosstalk. Even if the process is optimized, such as optimizing the filter design and adding edge filtering, the above problems cannot be completely solved, and the process difficulty will be greatly increased and the yield rate will be reduced. Summary of the invention

[0008] The present application provides a snapshot spectral camera crosstalk correction method, device and system, which can effectively solve the spectral crosstalk problem of the snapshot spectral camera, reduce the difficulty of manufacturing pixel-level filters, and have universality and simplicity. The present application provides the following technical solutions:

[0009] In a first aspect, a snapshot spectral camera crosstalk correction method is provided, the method comprising:

[0010] Obtaining light intensity information of different bands collected by a radiometer and image information of different bands collected by a coating spectrum camera;

[0011] converting the image information into current information;

[0012] For each wavelength band, determining a quantum efficiency curve based on light intensity information and current information corresponding to the wavelength band;

[0013] Perform Gaussian fitting on the quantum efficiency curve of each band to obtain the expected Gaussian curve of each band;

[0014] Determining a regression coefficient for each band based on the quantum efficiency curve and the expected Gaussian curve;

[0015] The quantum efficiency curve is de-crosstalked using the regression coefficient to obtain a quantum efficiency curve after de-crosstalking.

[0016] Optionally, converting the image information into current information includes:

[0017] Calculate the average value of all pixels of the image corresponding to each band;

[0018] The average value is converted into a current value to obtain the current information.

[0019] Optionally, for each wavelength band, determining a quantum efficiency curve based on light intensity information and current information corresponding to the wavelength band includes:

[0020] For each wavelength band, the ratio of the current information corresponding to the wavelength band to the light intensity information is determined to obtain the quantum efficiency curve.

[0021] Optionally, performing Gaussian fitting on the quantum efficiency curve of each band to obtain an expected Gaussian curve of each band includes:

[0022] Performing Gaussian fitting on the quantum efficiency curve to determine the half-peak width;

[0023] Generate the expected Gaussian curve for each band based on its half-maximum width.

[0024] Optionally, determining the regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve includes:

[0025] The regression coefficients for each band were determined using ridge regression based on the quantum efficiency curve and the expected Gaussian curve.

[0026] Optionally, the regression coefficient of each band is determined based on the quantum efficiency curve and the expected Gaussian curve using ridge regression, which is expressed by the following formula:

[0027] W=(X T X+λI) -1 X T Y

[0028] Wherein, X is the quantum efficiency curve, Y represents the expected Gaussian curve, λ is the regularization coefficient, and I is the unit matrix.

[0029] Optionally, the obtaining of light intensity information of different wavelength bands collected by a radiometer and image information of different wavelength bands collected by a coating spectrum camera includes:

[0030] The monochromator's wavelength band can be adjusted in steps of 1 nm from 550 nm to 900 nm.

[0031] Every time a wavelength band is reached, the radiometer is triggered to collect the light intensity information of the wavelength band, and the coating spectrum camera is triggered to collect the image information of the wavelength band.

[0032] In a second aspect, a snapshot spectral camera crosstalk correction device is provided, the device comprising:

[0033] A data acquisition module, used to acquire light intensity information of different bands collected by a radiometer, and image information of different bands collected by a coating spectrum camera;

[0034] A data processing module, used for converting the image information into current information;

[0035] A curve generating module, for determining a quantum efficiency curve for each wavelength band based on light intensity information and current information corresponding to the wavelength band;

[0036] Gaussian fitting module, used to perform Gaussian fitting on the quantum efficiency curve of each band to obtain the expected Gaussian curve of each band;

[0037] A coefficient determination module, used to determine the regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve;

[0038] The crosstalk processing module is used to use the regression coefficient to perform crosstalk removal processing on the quantum efficiency curve to obtain a quantum efficiency curve after crosstalk removal.

[0039] In a third aspect, a snapshot spectral camera crosstalk correction system is provided, the system comprising:

[0040] light source;

[0041] a monochromator for receiving light incident from the light source through the incident slit and inputting the selected monochromatic light into the integrating sphere through the exit slit;

[0042] The integrating sphere is used to homogenize the received monochromatic light;

[0043] The radiometer located at the light output end of the integrating sphere is used to: collect light intensity information of different wavelength bands;

[0044] The coated spectral camera located at the light output end of the integrating sphere is used to: collect image information of different wavelength bands; and

[0045] The correction device comprises the snapshot spectral camera crosstalk correction device described in the second aspect.

[0046] Optionally, the monochromator is used to:

[0047] The wavelength can be adjusted in steps of 1 nm from 550 nm to 900 nm.

[0048] Every time a wavelength band is reached, the radiometer is triggered to collect the light intensity information of the wavelength band, and the coating spectrum camera is triggered to collect the image information of the wavelength band.

[0049] This application can eliminate the influence of the light source, monochromator, and integrating sphere on the curve by converting the image information into a quantum efficiency curve; then, an ideal Gaussian curve is obtained through Gaussian fitting, and the crosstalk between different bands can be well suppressed, solving the spectral crosstalk problem of snapshot spectral cameras and reducing the difficulty of manufacturing pixel-level filters.

[0050] In addition, the use of ridge regression improves the robustness and generalization ability of the regression coefficients.

[0051] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a structural schematic diagram of a snapshot spectral camera crosstalk correction system provided by an embodiment of the present application;

[0053] Figure 2 is a schematic diagram of a 3*3 pixel-level filter provided in one embodiment of the present application;

[0054] Figure 3 is a flow chart of a snapshot spectral camera crosstalk correction method provided by an embodiment of the present application;

[0055] Figure 4 is a schematic diagram of a crosstalk processing process provided by an embodiment of the present application;

[0056] Figure 5 It is a block diagram of a snapshot spectral camera crosstalk correction device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0057] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but are not intended to limit the scope of the present application.

[0058] Figure 1 is a schematic diagram of a snapshot spectral camera crosstalk correction system provided by an embodiment of the present application, such as Figure 1 As shown, the system at least includes: a light source 1, a monochromator 2, an integrating sphere 3, a radiometer 4, a coating spectrum camera 5 and a correction device 6.

[0059] The light source 1 is used to emit light to the monochromator 2 .

[0060] The monochromator 2 is used to receive the light incident from the light source 1 through the incident slit, and input the selected monochromatic light into the integrating sphere 3 through the exit slit.

[0061] The integrating sphere 3 is used to homogenize the received monochromatic light.

[0062] The radiometer 4 and the coating spectrum camera 5 are both located at the light output end of the integrating sphere. The radiometer 4 is used to collect light intensity information of different wavelengths; the coating spectrum camera is used to collect image information of different wavelengths.

[0063] Specifically, the monochromator sets the automatic switching range and interval time of the band (for example, set it to 1s), and adjusts the band in steps of 1nm intervals from 550nm to 900nm (including the minimum center wavelength and maximum center wavelength of the filter in the coating spectrum camera 5 and extending 100nm); each time a band is reached, the radiometer is triggered to collect the light intensity information of the band, and the coating spectrum camera is triggered to collect the image information of the band. Specifically, the radiometer and the coating spectrum camera can be triggered through I / O to collect data. Among them, Figure 2 Schematic diagram of a 3*3 pixel-level filter.

[0064] In addition, in actual implementation, the wavelength adjustment range of the monochromator may also be other adjustment ranges determined based on the minimum center wavelength and the maximum center wavelength of the filter, and this embodiment does not limit the adjustment range.

[0065] The correction device 6 is used to perform crosstalk removal on the data collected by the snapshot spectral camera, and details are shown in the following method embodiment.

[0066] The correction device 6 can be implemented in the coating spectrum camera 5, or can also be implemented as a device independent of the coating spectrum camera 5. In this case, the coating spectrum camera 5 and the radiometer 4 are both communicatively connected to the correction device 6 so that the correction device 6 obtains the light intensity information and image information.

[0067] Figure 3A flowchart of a snapshot spectral camera crosstalk correction method provided in an embodiment of the present application. This embodiment is described by taking the method used for correction of a device as an example. Figure 3 As shown, the method includes at least the following steps:

[0068] Step 310, obtaining light intensity information of different wavelength bands collected by the radiometer and image information of different wavelength bands collected by the coating spectrum camera.

[0069] In one example, the wavelength band of the monochromator is adjusted step by step from 550nm to 900nm at intervals of 1nm; each time a wavelength band is reached, the radiometer is triggered to collect the light intensity information of the wavelength band, and the coating spectrum camera is triggered to collect the image information of the wavelength band. For details, please refer to the above embodiment.

[0070] The light intensity information can be the light power I collected by the radiometer. mW (mW).

[0071] The image information may be all pixels of the area array detector in each band selected from the coated spectral camera.

[0072] Step 320: convert the image information into current information.

[0073] The correction device calculates the average value of all pixels of the image corresponding to each band; converts the average value into a current value I mA (mA) to obtain current information.

[0074] Among them, the average value of the pixel is the pixel remote sensing image pixel brightness (Digital Number, DN) value.

[0075] Step 330: For each wavelength band, determine a quantum efficiency curve based on the light intensity information and current information corresponding to the wavelength band.

[0076] Specifically, for each band, the ratio of the current information to the light intensity information corresponding to the band is determined to obtain a quantum efficiency curve, so that the influence of the light source, monochromator and integrating sphere on the curve can be eliminated.

[0077] The calculation formula of the quantum efficiency curve is as follows:

[0078]

[0079] Where QE represents the quantum efficiency curve, I mA Indicates the current information corresponding to each band, I mW Indicates the image information corresponding to each band.

[0080] Step 340 , performing Gaussian fitting on the quantum efficiency curve of each band to obtain an expected Gaussian curve of each band.

[0081] Specifically, the correction device performs Gaussian fitting on the quantum efficiency curve to determine the half-peak width; and generates an expected Gaussian curve for each band according to the half-peak width of each band.

[0082] Step 350 , determining the regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve.

[0083] In one example, ridge regression is used to determine the regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve. In this example, by using ridge regression to determine the regression system, the robustness and generalization ability of the regression coefficient are improved.

[0084] Specifically, the regression coefficient of each band is determined based on the quantum efficiency curve and the expected Gaussian curve using ridge regression, which is expressed by the following formula:

[0085] W=(X T X+λI) -1 X T Y

[0086] Where X is the quantum efficiency curve, Y is the expected Gaussian curve, λ is the regularization coefficient, and I is the unit matrix.

[0087] Step 360: Use the regression coefficient to perform crosstalk removal processing on the quantum efficiency curve to obtain a quantum efficiency curve after crosstalk removal.

[0088] Specifically, the process of crosstalk removal can be expressed by the following formula:

[0089] X*W=X new

[0090] X represents the quantum efficiency curve, W represents the regression coefficient, and X new This is the quantum efficiency curve after crosstalk removal.

[0091] In one example, after the original quantum efficiency curve is converted into the expected Gaussian curve, the process of generating the quantum efficiency curve after crosstalk removal is as follows: Figure 4 shown.

[0092] In summary, the snapshot spectral camera crosstalk correction method provided in this embodiment can eliminate the influence of the light source, monochromator, and integrating sphere on the curve by converting image information into a quantum efficiency curve; then, an ideal Gaussian curve is obtained through Gaussian fitting, and crosstalk in different bands can be well suppressed, thereby solving the spectral crosstalk problem of the snapshot spectral camera and reducing the difficulty of manufacturing pixel-level filters.

[0093] In addition, the use of ridge regression improves the robustness and generalization ability of the regression coefficients.

[0094] Figure 5 is a block diagram of a snapshot spectral camera crosstalk correction device provided by an embodiment of the present application. Figure 1 The correction device shown in the figure is used as an example for explanation. The device includes at least the following modules:

[0095] A data acquisition module 510 is used to acquire light intensity information of different wavelengths collected by a radiometer and image information of different wavelengths collected by a coating spectrum camera;

[0096] A data processing module 520, configured to convert the image information into current information;

[0097] A curve generating module 530, for determining a quantum efficiency curve for each wavelength band based on light intensity information and current information corresponding to the wavelength band;

[0098] A Gaussian fitting module 540 is used to perform Gaussian fitting on the quantum efficiency curve of each band to obtain an expected Gaussian curve of each band;

[0099] A coefficient determination module 550, for determining a regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve;

[0100] The crosstalk processing module 560 is used to perform crosstalk removal processing on the quantum efficiency curve using the regression coefficient to obtain a quantum efficiency curve after crosstalk removal.

[0101] For relevant details, refer to the above method embodiment.

[0102] It should be noted that: the snapshot spectral camera crosstalk correction device provided in the above embodiment only uses the division of the above functional modules as an example when performing snapshot spectral camera crosstalk correction. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the snapshot spectral camera crosstalk correction device is divided into different functional modules to complete all or part of the functions described above. In addition, the snapshot spectral camera crosstalk correction device provided in the above embodiment and the snapshot spectral camera crosstalk correction method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0103] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the snapshot spectral camera crosstalk correction method of the above method embodiment.

[0104] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium, wherein the computer-readable storage medium stores a program, and the program is loaded and executed by a processor to implement the snapshot spectral camera crosstalk correction method of the above method embodiment.

[0105] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A snapshot spectral camera crosstalk correction method, characterized in that: The method comprises: Obtaining light intensity information of different bands collected by a radiometer and image information of different bands collected by a coating spectrum camera; converting the image information into current information; For each wavelength band, determining a quantum efficiency curve based on light intensity information and current information corresponding to the wavelength band; Perform Gaussian fitting on the quantum efficiency curve of each band to obtain the expected Gaussian curve of each band; Determining a regression coefficient for each band based on the quantum efficiency curve and the expected Gaussian curve; The quantum efficiency curve is de-crosstalked using the regression coefficient to obtain a quantum efficiency curve after de-crosstalking.

2. The method according to claim 1, characterized in that The converting the image information into current information comprises: Calculate the average value of all pixels of the image corresponding to each band; The average value is converted into a current value to obtain the current information.

3. The method according to claim 1, characterized in that For each wavelength band, determining a quantum efficiency curve based on light intensity information and current information corresponding to the wavelength band includes: For each wavelength band, the ratio of the current information corresponding to the wavelength band to the light intensity information is determined to obtain the quantum efficiency curve.

4. The method according to claim 1, characterized in that: The Gaussian fitting of the quantum efficiency curve of each band to obtain the expected Gaussian curve of each band includes: Performing Gaussian fitting on the quantum efficiency curve to determine the half-peak width; Generate the expected Gaussian curve for each band based on its half-maximum width.

5. The method according to claim 1, characterized in that Determining the regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve includes: The regression coefficients for each band were determined using ridge regression based on the quantum efficiency curve and the expected Gaussian curve.

6. The method according to claim 5, characterized in that The ridge regression is used to determine the regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve, which is expressed by the following formula: W=(X T X+λI) -1 X T Y Wherein, X is the quantum efficiency curve, Y represents the expected Gaussian curve, λ is the regularization coefficient, and I is the unit matrix.

7. The method according to any one of claims 1 to 6, characterized in that: The obtaining of light intensity information of different wavebands collected by the radiometer and image information of different wavebands collected by the coating spectrum camera includes: The monochromator's wavelength band can be adjusted in steps of 1 nm from 550 nm to 900 nm. Every time a wavelength band is reached, the radiometer is triggered to collect the light intensity information of the wavelength band, and the coating spectrum camera is triggered to collect the image information of the wavelength band.

8. A snapshot spectral camera crosstalk correction device, characterized in that: The device comprises: A data acquisition module, used to acquire light intensity information of different bands collected by a radiometer, and image information of different bands collected by a coating spectrum camera; A data processing module, used for converting the image information into current information; A curve generating module, for determining a quantum efficiency curve for each wavelength band based on light intensity information and current information corresponding to the wavelength band; Gaussian fitting module, used to perform Gaussian fitting on the quantum efficiency curve of each band to obtain the expected Gaussian curve of each band; A coefficient determination module, used to determine the regression coefficient of each band based on the quantum efficiency curve and the expected Gaussian curve; The crosstalk processing module is used to use the regression coefficient to perform crosstalk removal processing on the quantum efficiency curve to obtain a quantum efficiency curve after crosstalk removal.

9. A snapshot spectral camera crosstalk correction system, characterized in that: The system comprises: light source; a monochromator for receiving light incident from the light source through the incident slit and inputting the selected monochromatic light into the integrating sphere through the exit slit; The integrating sphere is used to homogenize the received monochromatic light; The radiometer located at the light output end of the integrating sphere is used to: collect light intensity information of different wavelength bands; The coated spectral camera located at the light output end of the integrating sphere is used to: collect image information of different wavelength bands; and A correction device, comprising the snapshot spectral camera crosstalk correction device as described in claim 8.

10. The system according to claim 9, characterized in that The monochromator is used for: The wavelength can be adjusted in steps of 1 nm from 550 nm to 900 nm. Every time a wavelength band is reached, the radiometer is triggered to collect the light intensity information of the wavelength band, and the coating spectrum camera is triggered to collect the image information of the wavelength band.

Images