A video-level spectral image acquisition method

Abstract

The present application relates to the technical field of spectral imaging, and particularly relates to a video-level spectral image acquisition method.The method comprises the following steps: S1, acquiring imaging rule information of a pixel-level spectral detector integrated with a light splitting; S2, continuously acquiring a plurality of target mosaic multispectral images of an application scene; S3, performing effective spectral signal extraction on the target mosaic multispectral images obtained in S2 according to the imaging rule obtained in S1, to obtain effective target mosaic multispectral images; S4, cutting the effective target mosaic multispectral images; S5, extracting a plurality of target single-channel spectral images from the same positions of the effective target mosaic multispectral images; S6, performing super-resolution reconstruction on each target single-channel spectral image; and S7, repeating steps S3-S6 on each frame of the mosaic multispectral images obtained in S2.The present application can convert the mosaic multispectral images into video-level multispectral images in real time and efficiently, and realize real-time dynamic detection in the spectral dimension.

Description

Technical Field

[0001] This invention relates to the field of spectral imaging technology, and in particular to a video-level spectral image acquisition method. Background Technology

[0002] Spectral imaging technology is a novel multidimensional information acquisition technology that combines spectral analysis and imaging techniques. It is a research hotspot in modern remote sensing and has been widely applied in meteorology, oceanography, biomedicine, and military fields. Traditional hyperspectral imaging systems typically require point-by-point or line-by-line scanning of the target to acquire spectral information, resulting in lengthy image acquisition times. Snapshot imaging spectrometers, however, can directly acquire scene information within a shutter's integration time and effectively suppress reconstruction errors caused by pushbroom scanning and blurring due to rapid target movement. Their ability to rapidly acquire target spatial spectral information makes them widely applicable to video-level spectral image acquisition. However, the large data volume and complex processing required by video-level spectral imaging technology present challenges for real-time data analysis and processing. Therefore, an efficient video-level spectral image acquisition method is needed to meet practical application requirements. Summary of the Invention

[0003] To achieve the above objectives, the technical solution created by this invention is: a video-level spectral image acquisition method, which is based on an on-chip integrated pixel-level spectral detector and includes the following steps:

[0004] S1: Obtain imaging rule information from the on-chip integrated pixel-level spectral detector;

[0005] S2: Send a continuous acquisition signal to the pixel-level spectral detector to continuously acquire multi-frame target mosaic multispectral images of the application scenario;

[0006] S3: Extract spectral signals from the target mosaic multispectral image according to the imaging rules obtained in S1, extract effective spectral signal values, and obtain an effective target mosaic multispectral image.

[0007] S4: Crop the effective target mosaic multispectral image to ensure the integrity of each spectral period;

[0008] S5: Extract target pixels at the same position in each spectral period of the effective target mosaic multispectral image to form multiple target single-channel spectral images;

[0009] S6: Perform super-resolution reconstruction on each target single-channel spectral image to restore the texture details of the image;

[0010] S7: Repeat the operations of S3 to S6 for each frame of the target mosaic multispectral image obtained in S2, and transform the multiple frames of the target mosaic multispectral image into a video-level multispectral image.

[0011] Preferably, between S1 and S2, S11 is further included: placing the pixel-level spectrometer in front of the monochromator's output port, using the monochromator to traverse the pixel-level spectrometer, with the traversal range covering the entire spectral range of the pixel-level spectrometer, to obtain a set of single-wavelength mosaic spectral images whose wavelength range includes the entire spectral range of the pixel-level spectrometer.

[0012] Preferably, after S11, S12 is also included: acquiring the spectral signal of each pixel in the single-wavelength mosaic spectral image, and analyzing the spectral signal to derive the imaging rule most suitable for the current pixel-level spectral detector.

[0013] Preferably, the mosaic multispectral image contains A×B pixels, and each k×k pixel constitutes a mosaic unit. Multiple mosaic units are arranged according to a periodic pattern. Each mosaic unit contains n×n spectral channels, that is, each spectral channel is composed of m=(k÷n)×(k÷n) pixels. The n×n spectral channels in the same mosaic unit respectively represent n×n spectral characteristics at the same spatial location, where A≥1, B≥1, k≥1, n≥1, m≥1, and A, B, k, n, and m are all integers.

[0014] Preferably, the spectral signal extraction method in S12 is to normalize and output the spectral signals of m pixels covered by each spectral channel.

[0015] Preferably, the spectral signal extraction method in S12 is to select the spectral signal of the same position pixel in each of the m pixels covered by each spectral channel as the spectral signal value of that spectral channel.

[0016] Preferably, the spectral signal extraction method in S12 is to select the spectral signals of pixels at different positions in each spectral channel from the m pixels covered by each spectral channel as the spectral signal values ​​of that spectral channel.

[0017] Preferably, the target mosaic multispectral image is ultimately converted into n 2 A single-channel spectral image with a resolution of (A÷k)×(B÷k), n 2 The target number of spectral channels is used. When the values ​​of A÷k and B÷k are decimals, they are rounded down to the nearest integer.

[0018] Preferably, a computer device is characterized by comprising:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the video-level spectral image acquisition methods described above.

[0022] Preferably, a non-transitory computer-readable storage medium storing computer instructions is characterized in that the computer instructions are used to cause the computer to execute any of the above-described video-level spectral image acquisition methods.

[0023] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0024] The present invention provides a video-level spectral image acquisition method that can convert continuously acquired multi-frame mosaic multispectral images into video-level multispectral images in real time and efficiently, enabling real-time dynamic detection of spectral dimensions. This is of great significance for scenarios requiring rapid response, such as environmental monitoring and industrial production. Attached Figure Description

[0025] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0026] Figure 1 This is a flowchart of a video-level spectral image acquisition method according to the present invention;

[0027] Figure 2 This invention provides a single-wavelength mosaic spectral image of a video-level spectral image acquisition method.

[0028] Figure 3 The wavelength range of the video-level spectral image acquisition method of the present invention includes λ. min ~λ max Single-channel spectral images;

[0029] Figure 4 The target mosaic multispectral image is provided by the video-level spectral image acquisition method of the present invention.

[0030] Figure 5 This refers to the imaging rule information formed by normalizing spectral response values ​​in a video-level spectral image acquisition method of the present invention.

[0031] Figure 6 This invention provides imaging rule information formed by selecting spectral response values ​​at the same location in a video-level spectral image acquisition method.

[0032] Figure 7 This invention provides imaging rule information formed by selecting spectral response values ​​at different locations in a video-level spectral image acquisition method.

[0033] Figure 8 This is a schematic diagram illustrating the conversion of a video-level spectral image acquisition method of the present invention into multiple target single-channel spectral images;

[0034] Figure 9 This is a mosaic multispectral image of a video-level spectral image acquisition method according to the present invention.

[0035] Explanation of reference numerals in the attached figures:

[0036] 1. Single-wavelength mosaic spectral image; 2. λ min ~λ max 1. Single-channel spectral image; 2. Target mosaic multispectral image; 3. Target mosaic unit; 4. Target spectral channel; 5. Target mosaic multispectral image; 6. Target single-channel spectral image; 7. Mosaic multispectral image; 8. Mosaic unit; 9. Spectral channel; 10. Target pixel; 11. Effective target mosaic unit; 12. Effective target mosaic multispectral image; 13. Target single-channel spectral image; 14. Mosaic multispectral image; 15. Mosaic unit; 26. Spectral channel; 37. Pixel. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0038] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0039] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0040] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] This invention provides a video-level spectral image acquisition method, which is based on an on-chip integrated pixel-level spectral detector, such as... Figures 1-8 As shown, it includes the following steps:

[0042] S1: Obtain imaging rule information from the on-chip integrated pixel-level spectral detector, including:

[0043] S11: Place the pixel-level spectrometer in front of the monochromator's output port, and use the monochromator to traverse the pixel-level spectrometer. The spectral range of the pixel-level spectrometer is λ. min ~λ max Adjust the initial wavelength of the monochromator emitted as λ. min A λ image was acquired using a pixel-level spectrometer. min Mosaic spectral image; adjust the wavelength of the monochromator's emitted monochromatic light to λ. min+1 Collect another λ image min+1 Mosaic spectral image; repeat the above steps, adjusting the wavelength of the monochromator's emitted monochromatic light to λ. max-1 Collect a λ max-1 Mosaic spectral image; until the wavelength of the monochromatic light emitted by the monochromator is λ. max Collect a λ max Mosaic spectral images; ultimately, a set of single-wavelength mosaic spectral images was obtained, with wavelengths including λ. min ~λ max .

[0044] S12: Obtain the spectral signals of each single-wavelength pixel of the single-wavelength mosaic spectral image 1 obtained in S11, and use known imaging rules (including but not limited to the three existing imaging rules) to separate each single-wavelength mosaic spectral image 1 (λ min Mosaic spectral image, λ min+1 Mosaic spectral image, ..., λ max-1 Mosaic spectral image, λ max The spectral channels of each period of the mosaic spectral image are extracted, forming a wavelength range including λ.min ~λ max Single-channel spectral image 2, plotting λ min ~λ max The spectral response curve of single-channel spectral image 2 is shown. Known imaging rules are evaluated by manually assessing the spectral response curves, and the most suitable imaging rule for the current pixel-level spectrometer is selected as the imaging rule for this pixel-level spectrometer.

[0045] Specifically, in this embodiment, the method of manual evaluation is as follows: λ min ~λ max The spectral response curve of the single-channel spectral image 2 is compared with the spectral response curve of the spectrophotometer system of the pixel-level spectral detector, λ min ~λ max The closer the spectral response curve of the single-channel spectral image 2 is to the spectral response curve of the spectrophotometer system, the more suitable the imaging rule is as the imaging rule for the current pixel-level spectral detector.

[0046] Specifically, in this embodiment, such as Figure 9 As shown, the mosaic multispectral image 7 contains A×B pixels 73. Every k×k pixels 73 form a mosaic unit 71. The mosaic multispectral image 7 is composed of multiple mosaic units 71 arranged in a periodic pattern. Each mosaic unit 71 contains n×n spectral channels 72, that is, each spectral channel 72 is composed of m=(k÷n)×(k÷n) pixels 73. The n×n spectral channels 72 in the same mosaic unit 71 respectively represent n×n spectral characteristics at the same spatial location, where A≥1, B≥1, k≥1, n≥1, m≥1, and A, B, k, n, and m are all integers.

[0047] Specifically, in this embodiment, the three existing imaging rules are as follows:

[0048] like Figure 5 As shown, the spectral signal values ​​DN1, DN2, DN3...DN of each spectral channel covering m pixels are... m After normalization, the spectral signal value of this spectral channel is:

[0049]

[0050] Finally, a mosaic multispectral image with extracted effective spectral signal values ​​is obtained.

[0051] like Figure 6As shown, among the m pixels covered by each spectral channel, the DN value of the pixel that best matches the actual spectral signal is selected as the spectral signal value of that spectral channel. In each spectral channel, the extracted pixels are all in the same position, and finally a mosaic multispectral image with extracted effective spectral signal values ​​is obtained.

[0052] like Figure 7 As shown, among the m pixels covered by each spectral channel, the DN value of the pixel that best matches the actual spectral signal is selected as the spectral signal value of that spectral channel. In each spectral channel, the extracted pixels are located at different positions, and finally a mosaic multispectral image with extracted effective spectral signal values ​​is obtained.

[0053] After these three imaging rules were manually evaluated, the imaging rule most suitable for the pixel-level spectral detector of this invention was obtained.

[0054] S2: Send continuous acquisition signals to the pixel-level spectral detector to continuously acquire multi-frame target mosaic multispectral images of the application scenario.

[0055] Specifically, in this embodiment, such as Figure 4 As shown, each frame of the target mosaic multispectral image 3 contains A×B target pixels 33. Every k×k target pixels 33 form a target mosaic unit 31. The target mosaic multispectral image 3 is composed of multiple target mosaic units 31 arranged in a periodic pattern. Each target mosaic unit 31 contains n×n target spectral channels 32, that is, each target spectral channel 32 is composed of m=(k÷n)×(k÷n) target pixels 33. The n×n target spectral channels 32 in the same target mosaic unit 31 respectively represent n×n spectral characteristics of the same spatial location, where A≥1, B≥1, k≥1, n≥1, m≥1, and A, B, k, n, and m are all integers.

[0056] S3: Extract spectral signals from the target mosaic multispectral image 3 according to the imaging rules obtained in S1, extract effective spectral signal values, and obtain an effective target mosaic multispectral image 5 composed of effective target mosaic units 4.

[0057] S4: Crop the effective target mosaic multispectral image 5 to ensure the integrity of each spectral period. One spectral period is one target mosaic unit 31. Ensuring the integrity of each spectral period means ensuring the integrity of each target mosaic unit 31. The purpose of cropping is to remove those parts of the target mosaic multispectral image 3 with incomplete spectral periods due to imaging equipment, transmission process or other factors, so as to ensure that each target mosaic unit 31 in the target mosaic multispectral image 3 is complete.

[0058] S5: Each target mosaic unit 31 extracts an effective target mosaic unit 4 to form an effective spectral period. Extract the target pixels 33 at the same position in each effective spectral period to form multiple target single-channel spectral images 6.

[0059] S6: Perform super-resolution reconstruction on each target single-channel spectral image 6 to restore the texture details of the image.

[0060] S7: Repeat the operations of S3 to S6 for each frame of target mosaic multispectral image 3 obtained in S2, and transform the multi-frame target mosaic multispectral image 3 into a video-level multispectral image.

[0061] Specifically, in this embodiment, the target mosaic multispectral image 3 is ultimately converted into n 2 6,n single-channel spectral images with a resolution of (A÷k)×(B÷k). 2 The target number of spectral channels is A÷k. When A÷k and B÷k are decimals, they need to be rounded down.

[0062] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A video-level spectral image acquisition method, the method being based on an on-chip integrated pixel-level spectral detector, characterized in that, Specifically, the steps include the following: S1: Obtain imaging rule information from the on-chip integrated pixel-level spectral detector; S11: Place the pixel-level spectrometer in front of the monochromator's output port, and use the monochromator to traverse the pixel-level spectrometer. The traversal range covers the entire spectral range of the pixel-level spectrometer, resulting in a set of single-wavelength mosaic spectral images whose wavelength range includes the entire spectral range of the pixel-level spectrometer. S12: Obtain the spectral signal of each single-wavelength pixel of the single-wavelength mosaic spectral image obtained in S11. Use three imaging rules to extract the spectral channels of each period of the single-wavelength mosaic spectral image to form a single-channel spectral image whose wavelength range includes the entire spectral range of the pixel-level spectral detector. Plot the spectral response curve of the single-channel spectral image of the entire spectral range of the pixel-level spectral detector. The spectral response curves of single-channel spectral images covering the entire spectral range of a pixel-level spectrometer are compared with the spectral response curves of the spectrometer's spectral film system. The closer the spectral response curve of the single-channel spectral image covering the entire spectral range of a pixel-level spectrometer is to the spectral response curve of the spectrometer's spectral film system, the more suitable the imaging rule is as the current imaging rule for the pixel-level spectrometer. The imaging rule in S12 is to normalize and output the spectral signals of the m pixels covered by each spectral channel. The imaging rule in S12 is to select the spectral signal of the same pixel location as the spectral signal value of that spectral channel among the m pixels covered by each spectral channel. The imaging rule in S12 is to select the spectral signal of a pixel at a different position as the spectral signal value of that spectral channel among the m pixels covered by each spectral channel. S2: Send a continuous acquisition signal to the pixel-level spectral detector to continuously acquire multi-frame target mosaic multispectral images of the application scenario; S3: Extract spectral signals from the target mosaic multispectral image according to the imaging rules obtained in S1, extract effective spectral signal values, and obtain an effective target mosaic multispectral image. S4: Crop the effective target mosaic multispectral image to ensure the integrity of each spectral period; S5: Extract target pixels at the same position in each spectral period of the effective target mosaic multispectral image to form multiple target single-channel spectral images; S6: Perform super-resolution reconstruction on each target single-channel spectral image to restore the texture details of the image; S7: Repeat the operations of S3 to S6 for each frame of the target mosaic multispectral image obtained in S2, and transform the multiple frames of the target mosaic multispectral image into a video-level multispectral image.

2. The video-level spectral image acquisition method according to claim 1, characterized in that: A mosaic multispectral image contains A×B pixels. Each k×k pixel constitutes a mosaic unit. Multiple mosaic units are arranged according to a periodic pattern. Each mosaic unit contains n×n spectral channels, that is, each spectral channel is composed of m=(k÷n)×(k÷n) pixels. The n×n spectral channels in the same mosaic unit represent n×n spectral characteristics at the same spatial location, where A≥1, B≥1, k≥1, n≥1, m≥1, and A, B, k, n, and m are all integers.

3. The video-level spectral image acquisition method according to claim 1, characterized in that: The target mosaic multispectral image was ultimately converted into Zhang resolution is Single-channel spectral images, For the target number of spectral channels, when , When the value is a decimal, it is rounded down to the nearest integer.

4. A computer device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the video-level spectral image acquisition method according to any one of claims 1 to 3.

5. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the video-level spectral image acquisition method according to any one of claims 1 to 3.

Images