Spectral image demodulation method and spectral image demodulation device

Abstract

The present application provides a kind of spectral image demodulation method, spectral image demodulation equipment and computer readable storage medium.Spectral image demodulation method includes: S1: obtaining first spectral image;S2: the first spectral image is handled, and second spectral image is obtained;S3: the second spectral image is divided into spectral correlation area, and a plurality of spectral correlation areas are obtained;S4: according to the plurality of spectral correlation areas, a plurality of spectral curves are obtained;S5: according to target wave band, the plurality of spectral curves and the first spectral image, the spectral image of the target wave band is obtained.The spectral image demodulation method of the present application can break through the limitation of prior art, can realize spectral image demodulation, realize spectral image inversion, with high precision, high accuracy, high reliability and other advantages, can be widely applied in various fields and various scenes.

Description

TECHNICAL FIELD

[0001] The present application generally relates to the technical field of image processing, and particularly relates to a spectral image demodulation method and a spectral image demodulation device. BACKGROUND

[0002] A spectral image includes combined data of spatial and spectral information, and can reflect the appearance characteristics and internal composition of an object at the same time, and is widely applied in various fields and various scenes. In some application scenarios, it is necessary to demodulate the spectral image. However, in some existing spectral image demodulation schemes, there are defects such as low precision, low accuracy, and low reliability, which limit the application of the spectral image. Therefore, how to improve the precision, accuracy, and reliability of the spectral image demodulation is a technical problem to be solved by the present application.

[0003] The content in the background section is only the technology known by the inventor, and does not necessarily represent the prior art in the field. SUMMARY

[0004] In view of one or more of the problems in the prior art, the present application provides a spectral image demodulation method, a spectral image demodulation device, and a computer readable storage medium. The spectral image demodulation method of the present application can break through the limitations of the prior art, can realize spectral image demodulation, can realize spectral image inversion, has advantages such as high precision, high accuracy, and high reliability, and can be widely applied in various fields and various scenes.

[0005] According to a first aspect of the present application, a spectral image demodulation method is provided. The spectral image demodulation method comprises: S1: obtaining a first spectral image; S2: processing the first spectral image to obtain a second spectral image; S3: dividing the second spectral image into a plurality of spectral correlation regions to obtain a plurality of spectral correlation regions; S4: obtaining a plurality of spectral curves according to the plurality of spectral correlation regions; and S5: obtaining a spectral image of a target waveband according to the target waveband, the plurality of spectral curves, and the first spectral image.

[0006] Optionally, the first spectral image includes a plurality of pixels, each pixel including spectral information of one channel, and the plurality of pixels including spectral information of a plurality of channels.

[0007] Optionally, S2 comprises: performing interpolation processing on the first spectral image to obtain the second spectral image, each pixel of the second spectral image including spectral information of a plurality of channels, and the spectral information being related to a gray value.

[0008] Optionally, S3 comprises: S31, determining a gray scale reference value of each channel according to gray scale values of the corresponding channel of the plurality of pixels of the second spectral image; S32, determining a difference between the gray scale value of each channel of each pixel of the second spectral image and the corresponding gray scale reference value; and S33, performing spectral correlation region division on the second spectral image according to the difference.

[0009] Optionally, S31 comprises: determining an average value of the gray scale values of the corresponding channel of the plurality of pixels of the second spectral image, and taking the average value as the gray scale reference value of the corresponding channel.

[0010] Optionally, S32 comprises: determining a mean square error between the gray scale values of the plurality of channels of each pixel of the second spectral image and the gray scale reference value of the corresponding channel.

[0011] Optionally, S33 comprises: dividing the pixels with the mean square error less than a first threshold value into a first spectral correlation region.

[0012] Optionally, S3 further comprises: performing the following operations on the remaining pixels of the second spectral image: S34, determining an average value of the gray scale values of the corresponding channel of the remaining pixels, and taking the average value as the gray scale reference value of the corresponding channel of the remaining pixels; S35, determining a mean square error between the gray scale values of the plurality of channels of each pixel in the remaining pixels and the gray scale reference value of the corresponding channel; S36, dividing the pixels with the mean square error less than a second threshold value into a second spectral correlation region; and repeating the operations S34-S36 until the plurality of spectral correlation regions are obtained.

[0013] Optionally, S4 comprises: determining an average value of the gray scale values of the corresponding channel of each pixel of each spectral correlation region; and performing spectral inversion on each spectral correlation region according to the average value of the gray scale values of the corresponding channel of each pixel of each spectral correlation region to obtain a spectral curve of each spectral correlation region.

[0014] Optionally, S3 comprises: performing spectral correlation region division on the second spectral image based on a clustering algorithm; and the clustering algorithm comprises a K-means clustering algorithm.

[0015] Optionally, S5 comprises: S51, determining a weight of the plurality of spectral correlation regions according to the target waveband and the plurality of spectral curves; and S52, obtaining a spectral image of the target waveband according to the weight of the plurality of spectral correlation regions and the gray scale values of the pixels of the first spectral image corresponding to the plurality of spectral correlation regions.

[0016] Optionally, S51 comprises: determining an area proportion of the target waveband in the plurality of spectral curves; and determining the weight of the plurality of spectral correlation regions according to the area proportion of the target waveband in the plurality of spectral curves.

[0017] Optionally, S52 includes: multiplying the weights of the plurality of spectral correlation regions and the gray values ​​of the pixels of the first spectral image corresponding to the plurality of spectral correlation regions to obtain the spectral image of the target band.

[0018] According to a second aspect of the present invention, a spectral image demodulation apparatus is provided. The spectral image demodulation apparatus includes: a microlens array configured to converge incident light; a filter unit array disposed downstream of the optical path of the microlens array, the filter unit including a filter sub-unit array configured to allow incident light of a preset wavelength band to pass through; a photoelectric sensor array disposed downstream of the optical path of the filter unit array, the photoelectric sensor configured to receive the incident light of the preset wavelength band and perform photoelectric conversion; and a processor coupled to the photoelectric sensor array and configured to execute the spectral image demodulation method as described above.

[0019] According to a second aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes computer-executable instructions stored thereon, which, when executed by a processor, implement the spectral demodulation method as described above.

[0020] The spectral image demodulation method and spectral image demodulation device of the present invention can realize spectral image demodulation and spectral image inversion, and have advantages such as high precision, high accuracy and high reliability. They can be widely used in various fields and scenarios. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the following description of the embodiments will be provided as examples. The drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation of the present invention.

[0022] Figure 1 A flowchart illustrating an exemplary spectral image demodulation method consistent with some embodiments of the present invention is shown.

[0023] Figure 2A A schematic diagram of an exemplary spectral image demodulation device consistent with some embodiments of the present invention is shown.

[0024] Figure 2B A schematic diagram of an exemplary filter unit consistent with some embodiments of the present invention is shown.

[0025] Figure 3AA schematic diagram of an exemplary first spectral image is shown, consistent with some embodiments of the application.

[0026] Figure 3B A schematic diagram of an exemplary second spectral image is shown, consistent with some embodiments of the application.

[0027] Figure 4 A schematic diagram of an exemplary operation flow of step S3 is shown, consistent with some embodiments of the application.

[0028] Figure 5 A schematic diagram of an exemplary spectral curve is shown, consistent with some embodiments of the application.

[0029] Figure 6 A schematic diagram of an exemplary operation flow of step S5 is shown, consistent with some embodiments of the application.

[0030] Figure 7A A schematic diagram of an exemplary determining weight of spectral related region is shown, consistent with some embodiments of the application.

[0031] Figure 7B A schematic diagram of an exemplary determining weight of spectral related region is shown, consistent with some embodiments of the application.

[0032] Figure 7C A schematic diagram of an exemplary determining weight of spectral related region is shown, consistent with some embodiments of the application.

[0033] Figure 8A A schematic diagram of an exemplary obtaining spectral image of target waveband is shown, consistent with some embodiments of the application.

[0034] Figure 8B A schematic diagram of an exemplary obtaining spectral image of target waveband is shown, consistent with some embodiments of the application.

[0035] Figure 9 A schematic diagram of an exemplary effect of spectral image of target waveband is shown, consistent with some embodiments of the application. DETAILED DESCRIPTION

[0036] Hereinafter, only some exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present application. Therefore, the drawings and the description are considered to be exemplary in nature rather than limiting.

[0037] In the description of the application, it is to be understood that the orientations or positional relationships indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the devices or elements indicated thereby must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application. In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated thereby. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified and limited.

[0038] In the description of the application, it should be noted that, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "coupling" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication or interaction relationship of two elements. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0039] In the present application, unless otherwise explicitly specified and limited, the "upper" or "lower" of the first feature to the second feature can include the direct contact of the first and second features, or can include the contact of the first and second features through another feature between them. Moreover, the "upper", "upper" and "upper" of the first feature to the second feature include the vertical height of the first feature above and oblique to the second feature, or only indicates that the horizontal height of the first feature is higher than that of the second feature. The "below", "below" and "below" of the first feature to the second feature include the vertical height of the first feature below and oblique to the second feature, or only indicates that the horizontal height of the first feature is less than that of the second feature.

[0040] Many different embodiments or examples of the application are provided below to illustrate different structures of the application. To simplify the present application, components and settings of specific examples are described below. Of course, they are only examples and the purpose is not to limit the application. In addition, the application can refer to the same reference numerals and / or reference letters in different examples, and such repetition is for the purpose of simplification and clarity, which does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the application provides examples of various specific processes and materials, but those skilled in the art can realize the application of other processes and / or the use of other materials.

[0041] The preferred embodiments of the application are described below in conjunction with the accompanying drawings, it should be understood that the preferred embodiments described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0042] The application provides a spectral image demodulation method. Figure 1 A flowchart of an exemplary spectral image demodulation method 10 consistent with some embodiments of the application is shown. As shown in Figure 1 The spectral image demodulation method 10 includes steps S1-S5. Step S1, a first spectral image is obtained. Step S2, the first spectral image is processed to obtain a second spectral image. Step S3, the second spectral image is divided into a plurality of spectral correlation regions to obtain a plurality of spectral correlation regions. Step S4, a plurality of spectral curves are obtained according to the plurality of spectral correlation regions. Step S5, a spectral image of a target waveband is obtained according to the target waveband, the plurality of spectral curves and the first spectral image.

[0043] The spectral image demodulation method of the application can realize spectral image demodulation, realize spectral image inversion, has advantages such as high precision, high accuracy, high reliability, etc., and can be widely applied in various fields and various scenes.

[0044] The application also provides a spectral image demodulation device. Figure 2A A schematic diagram of an exemplary spectral image demodulation device 20 consistent with some embodiments of the application is shown. As shown in Figure 2A The spectral image demodulation device 20 includes a microlens array 21, a filter unit array 22, a photosensor array 23 and a processor 24. The processor 24 can execute the spectral image demodulation method 10. Before the spectral image demodulation method 10 is described in detail, the spectral image demodulation device 20 of the application is introduced.

[0045] As Figure 2AAs shown, the microlens array 21 includes a plurality of microlenses 210. The plurality of microlenses 210 can be arranged in a one-dimensional array or a two-dimensional array. The microlenses 210 can converge the incident light L. In some embodiments, the incident surface of the microlenses 210 may include an antireflection film, which can increase light throughput, reduce reflection, suppress noise, and help improve the signal-to-noise ratio of the output signal of the photoelectric sensor array 23 and improve the accuracy of spectral image demodulation.

[0046] like Figure 2A As shown, the filter unit array 22 is disposed downstream of the optical path of the microlens array 21. The filter unit array 22 includes multiple filter units 220. Each filter unit 220 includes multiple filter sub-units 2202. The multiple filter sub-units 2202 can be arranged in a one-dimensional array or a two-dimensional array. In some embodiments, the multiple filter sub-units 2202 in each filter unit 220 can be arranged in an n*n array, where n is a positive integer. For example, referring to the partially enlarged view of the filter unit array 22 shown above in FIG2, the filter unit 220 includes 9 filter sub-units 2202, and the 9 filter sub-units 2202 are arranged in a 3*3 array. For example, the filter unit 220 may include 4 filter sub-units 2202, and the 4 filter sub-units 2202 can be arranged in a 2*2 filter sub-unit array. For example, the filter unit 220 may include 16 filter sub-units 2202, which can be arranged in a 4*4 array. In some embodiments, the multiple filter sub-units 2202 of the filter unit 220 may be arranged in an n*m array, where n and m are positive integers, and n≠m. For example, the filter unit 220 may include 6 filter sub-units 2202, which can be arranged in a 2*3 filter sub-unit array. For example, the filter unit 220 may include 12 filter sub-units 2202, which can be arranged in a 3*4 filter sub-unit array. It should be noted that the present invention does not limit the number and arrangement of filter sub-units in each filter unit; in practical applications, the arrangement can be set according to requirements.

[0047] In some embodiments, the filter subunit 2202 may have a specific transmittance profile. The filter subunit 2202 may allow incident light of a preset wavelength band to pass through. One filter subunit 2202 may form a channel. One channel may allow incident light of a preset wavelength band to pass through. In some embodiments, the filter subunit 2202 may include a filter film, filter, nanoarray, grating, or similar device to achieve wavelength selectivity.

[0048] In some embodiments, the transmittance profiles of each filter subunit 2202 in the same filter unit 220 may be different. For example, the transmittance profiles of each filter subunit 2202 in the same filter unit 220 may be different. Figure 2BA schematic diagram of an exemplary filter unit 220 is shown, which is consistent with some embodiments of the present application. As shown in Figure 2A , 2B The filter unit 220 includes nine filter sub-units 2202, each of which has a different transmittance curve. In this configuration, one filter sub-unit 2202 can form one channel. The nine filter sub-units 2202 of the filter unit 220 can form nine channels CH1-CH9, each of which allows incident light of one of the nine wavelength bands λ1-λ9 to pass through.

[0049] In an example, the transmittance curves of the filter sub-units 2202 in the same filter unit 220 can be partially the same and partially different. Continuing to refer to Figure 2A For example, the filter unit 220 includes nine filter sub-units 2202, two of which have the same transmittance curve and the other seven of which have different transmittance curves. In this configuration, the nine filter sub-units 2202 of the filter unit 220 can form eight channels, each of which allows incident light of one of the eight wavelength bands to pass through.

[0050] It should be noted that the above embodiments are only exemplary and are described by way of example with the filter unit 220 including nine filter sub-units 2202. The present application is not limited thereto. In actual applications, the number, arrangement, transmittance, channel number, and device type of the filter sub-units in the filter unit can be configured according to requirements.

[0051] Continuing to refer to Figure 2A The photoelectric sensor array 23 is disposed downstream of the optical path of the filter unit array 22. The photoelectric sensor array 23 includes a plurality of photoelectric sensors 230. The plurality of photoelectric sensors 230 can be arranged in a one-dimensional array or a two-dimensional array. The photoelectric sensors 230 can receive incident light of a predetermined wavelength band that passes through the filter sub-units 2202 and perform photoelectric conversion. In some embodiments, the photoelectric sensors 230 can include one or more pixels 2303. The plurality of pixels 2303 can be arranged in a one-dimensional array or a two-dimensional array. The pixels 2303 can serve as the smallest light sensing unit of the photoelectric sensor array 23 and convert the light signal incident thereon into an electrical signal. In some embodiments, the pixels 2303 can include a light sensing element such as a photodiode or a phototransistor, for example, a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) device.

[0052] As shown in Figure 2AAs shown, the processor 24 is coupled to the photoelectric sensor array 23. The processor 24 can generate a spectral image based on the electrical signals output by the photoelectric sensor array 23. In some embodiments, one microlens 210 can correspond to one filter subunit 2202. One filter subunit 2202 can correspond to one pixel element 2303. One pixel element 2303 can correspond to one pixel in the spectral image. The processor 24 can determine the gray value of the pixel based on the electrical signals output by the pixel element 2303 and the wave band corresponding to the filter subunit 2202. It should be noted that the present application does not limit the correspondence between the microlens 210, the filter subunit 2202, the pixel element 2303 and the pixel. Alternatively, one microlens 210 can correspond to a plurality of filter subunits 2202. Alternatively, one filter subunit 2202 can correspond to a plurality of pixel elements 2303. Alternatively, a plurality of pixel elements 2303 can correspond to one pixel. In actual applications, the configuration can be made according to requirements.

[0053] In some embodiments, the processor 24 can include processing circuitry, a central processing unit (CPU), a microcontroller unit (MCU), a digital signal processor (DSP), a graphics processing unit (GPU), an accelerator, a neural processing unit (NPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a gate device or a transistor logic device, and the like.

[0054] Although not shown in the figure, the spectral image demodulation device 20 can further include a memory. The memory can be coupled to the processor 24 and used to store data, program instructions and the like collected by the spectral image demodulation device 20.

[0055] In some embodiments, the memory can include random access memory (RAM) and can also include non-volatile memory (NVM). Further, the memory can include at least one of phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), read-only memory (ROM), electrically erasable programmable read only memory (EEPROM), and the like.

[0056] In some embodiments, the spectral image demodulation device 20 can be a spectral camera. In some embodiments, the microlens array 21, the light filtering unit array 22, the photosensor array 23, and the processor 24 can be disposed on a single spectral chip, which can improve the integration level and reduce the overall size. Although not shown in the figure, the spectral image demodulation device 20 can also include communication circuitry, a housing, and the like.

[0057] The spectral image demodulation device 20 of the present application can capture the spectral information of an object at different wavelengths and generate a three-dimensional image containing spectral data, i.e., a spectral image, by processing the electrical signals output by the photosensor array 23 using the processor 24. The spectral image not only contains two-dimensional spatial information (e.g., object shape, contour, texture, position, etc.), but also includes spectral information. Each pixel of the spectral image includes three-dimensional data (x, y, λ), where x and y are image coordinates, and λ is the wavelength. Each pixel corresponds to a spectral curve. The spectral image can reflect the reflection, absorption, or emission characteristics of an object at different wavelengths, providing a comprehensive reference for the analysis and identification of the object.

[0058] In some application scenarios, the spectral image can be demodulated to extract spectral images at different wavebands to obtain a plurality of waveband spectral images (referred to as multi-waveband spectral images). The multi-waveband spectral image includes combined data of spatial and spectral information, and can reflect the appearance characteristics and internal composition of an object at the same time, and is widely used in various fields and various scenarios. For example, in the medical field, the multi-waveband spectral image can be used for disease diagnosis. Although the diseased tissue can be roughly determined according to a spectral curve, it is difficult to realize accurate lesion positioning. The multi-waveband spectral image can well solve such a problem. Different tissues and lesions have different spectral characteristics at different wavebands, and the multi-waveband spectral image can help doctors accurately find the lesion area and realize accurate lesion positioning. For another example, in the military field, the multi-waveband spectral image can be used for camouflage recognition. Although the target has been camouflaged, the spectral characteristics at different wavebands are difficult to completely change. Although the camouflage material can be roughly determined according to a spectral curve, it is still difficult to realize accurate positioning of the camouflaged target. The spectral characteristics of the camouflaged target at different wavebands are different, and the multi-spectral image can help identify the camouflaged target and realize accurate positioning. The multi-waveband spectral image can improve more comprehensive and more accurate material information and spatial information, so as to realize more efficient and more accurate object detection and recognition in more fields. In addition, the multi-waveband spectral image can also be used in the fields of aerospace, agriculture, environmental monitoring, consumer electronics, etc.

[0059] The spectral image demodulation device 20 of the present application can obtain a spectral image by processing the electrical signal output by the photoelectric sensor array 23 through the processor 24 executing the spectral image demodulation method 10. By demodulating the spectral image, the multi-waveband spectral image can be obtained by extracting the spectral image at different wavebands, so as to provide more comprehensive and more accurate material information and spatial information for the detection and recognition of objects and other applications, provide more reliable reference basis, and realize more efficient, more accurate and more reliable recognition and detection in more application fields.

[0060] The spectral image demodulation method 10 will be described in detail below by taking the processor 24 executing the spectral image demodulation method 10 as an example. It should be understood that the spectral image demodulation method 10 can also be executed by a processor other than the spectral image demodulation device 20.

[0061] Figure 3A A schematic diagram of an exemplary first spectral image IMG1 consistent with some embodiments of the present application is shown. Figure 3B A schematic diagram of an exemplary second spectral image IMG2 consistent with some embodiments of the present application is shown. As Figures 1-3BAs shown, at step S1, the processor 24 can acquire a first spectral image IMG1. At step S2, the processor 24 can process the first spectral image IMG1 to obtain a second spectral image IMG2. In some embodiments, the processor 24 can generate the first spectral image IMG1 according to the electrical signal output by the photosensor array 23. The first spectral image IMG1 includes raw spectral image data acquired by the spectral image demodulation device 20. The spatial resolution of the first spectral image IMG1 corresponds to the resolution of the spectral image demodulation device 20. For example, if the spatial resolution of the spectral image demodulation device 20 is 1200*1600, then the spatial resolution of the first spectral image IMG1 is 1200*1600. The first spectral image IMG1 includes a plurality of pixels. Each pixel includes spectral information of one channel. The plurality of pixels includes spectral information of a plurality of channels. In other words, each pixel includes spectral information of one waveband. The plurality of pixels includes spectral information of a plurality of wavebands. The spectral information includes wavelengths of light. The wavelengths are related to light intensity. The light intensity is related to the gray value of the pixel. The spectral information is related to the gray value. The processor 24 can determine the gray value of the pixel based on the electrical signal output by the pixel 2303. The gray value of the pixel corresponds to the waveband of the light filtering subunit 2202.

[0062] In some embodiments, the processor 24 can determine the spectral information of the channel according to the gray value of the pixel. Referring to Figures 1-3A The light filtering unit 220 includes 9 light filtering subunits 2202. The 9 light filtering subunits 2202 are arranged in a 3*3 array. The 9 light filtering subunits 2202 have different transmittance curves, respectively allowing 9 wavebands λ1-λ9 of incident light to pass through. Each light filtering subunit 2202 can form one channel. The 9 light filtering subunits 2202 form 9 channels CH1-CH9. The 9 channels CH1-CH9 respectively allow wavebands λ1-λ9 of incident light to pass through. The incident light passing through the light filtering subunit 2202 can be sensed and photoelectrically converted by the pixel 2303. The processor 24 can form one pixel of the first spectral image IMG1 based on the electrical signal output by the pixel 2303. In the first spectral image IMG1, each pixel includes one gray value, each pixel includes spectral information of one channel, the plurality of pixels includes a plurality of gray values, the plurality of pixels includes spectral information of a plurality of channels, and the plurality of pixels includes spectral information of a plurality of wavebands. The first spectral image IMG1 includes spectral information of a plurality of wavebands.

[0063] In some embodiments, at step S2, the processor 24 can perform interpolation processing on the first spectral image IMG1 to obtain the second spectral image IMG2. It should be noted that the first spectral image IMG1 and the second spectral image IMG2 both include a plurality of pixels, have the same number of pixels, have the same spatial resolution, and have the same resolution as the spectral image demodulation device 20, for example, both are 1200*1600, and both include spectral information of a plurality of wavebands. The difference is that, before interpolation processing, each pixel in the first spectral image IMG1 includes information of one channel, each pixel includes a gray value, and each pixel includes spectral information of one waveband, for example, includes spectral information corresponding to one filter sub-unit 2202. After interpolation processing, each pixel of the second spectral image IMG2 includes information of a plurality of channels, each pixel includes a plurality of gray values, and each pixel includes spectral information of a plurality of wavebands, for example, includes spectral information corresponding to a plurality of filter sub-units 2202. The plurality of filter sub-units 2202 can come from the same filter unit 220, or can come from a plurality of adjacent filter units 220. The purpose of interpolation processing is to make one pixel not only include waveband information of the corresponding channel, but also include waveband information of other channels adjacent to the channel. In other words, the purpose of interpolation processing is to change one pixel from including information of one waveband corresponding to one channel to including information of a plurality of wavebands corresponding to a plurality of channels. For example, referring to FIGS. 3A and 3B, before interpolation processing, the pixel P1 of the first spectral image IMG1 includes waveband information λ1 of the channel CH1. After interpolation processing, the pixel P2 of the second spectral image IMG2 at the corresponding position of the first spectral image IMG1 includes not only waveband information λ1 of the channel CH1, but also waveband information λ2-λ9 of the channels CH2-CH9. It should be noted that only nine channels and nine wavebands are exemplarily introduced here, and the present application is not limited thereto, and is similar to other numbers of channels and wavebands. Figure 2B 、 3A 、3B, before interpolation processing, the pixel P1 of the first spectral image IMG1 includes waveband information λ1 of the channel CH1. After interpolation processing, the pixel P2 of the second spectral image IMG2 at the corresponding position of the first spectral image IMG1 includes not only waveband information λ1 of the channel CH1, but also waveband information λ2-λ9 of the channels CH2-CH9. It should be noted that only nine channels and nine wavebands are exemplarily introduced here, and the present application is not limited thereto, and is similar to other numbers of channels and wavebands.

[0064] In some embodiments, at step S3, the processor 24 can perform spectral correlation region division on the second spectral image IMG2 to obtain a plurality of spectral correlation regions. It should be understood that performing spectral correlation region division on the second spectral image IMG2 is actually to select pixels with high spectral information correlation in the second spectral image IMG2 and to classify them. The range of the gray value can be 0 (black) to 255 (white). The gray value of the pixel can reflect the intensity of light. The intensity of light can reflect the wavelength information. In some embodiments, the processor 24 can perform spectral correlation region division on the second spectral image IMG2 according to the gray value of the pixel. Details are described below.

[0065] Figure 4 An exemplary operation flowchart of step S3 is shown, which is consistent with some embodiments of the present application. As shown in FIG. 3C, the processor 24 can perform spectral correlation region division on the second spectral image IMG2 to obtain a plurality of spectral correlation regions.Figure 4 As shown, step S3 includes operations S31-S33. In operation S31, the processor 24 can determine a gray scale reference value for each channel according to the gray scale values of the corresponding channels of the plurality of pixels of the second spectral image IMG2. For example, the second spectral image IMG2 includes 1200*1600 pixels. Each pixel includes information of 9 channels. Each pixel includes 9 gray scale values. Accordingly, the second spectral image IMG2 includes 1200*1600*9 gray scale values. That is, the second spectral image IMG2 includes 9 gray scale images with a resolution of 1200*1600. The processor 24 can determine a gray scale reference value for each channel according to the gray scale values of the corresponding channels of the plurality of pixels of the second spectral image IMG2. For example, the processor 24 can determine a gray scale reference value G1 for the channel CH1 according to the gray scale values of the channel CH1 of the 1200*1600 pixels. For another example, the processor 24 can determine a gray scale reference value G2 for the channel CH2 according to the gray scale values of the channel CH2 of the 1200*1600 pixels. For yet another example, the processor 24 can determine a gray scale reference value G3 for the channel CH3 according to the gray scale values of the channel CH3 of the 1200*1600 pixels. In this way, the processor 24 can determine the gray scale reference values G1-G9 for the 9 channels respectively by traversing the 9 channels of the 1200*1600 pixels of the second spectral image IMG2.

[0066] In some embodiments, in operation S32, the processor 24 can determine the difference between the gray scale value of each channel of each pixel of the second spectral image IMG2 and the corresponding gray scale reference value. For example, the processor 24 can determine the difference ΔCH1 between the gray scale value of the channel CH1 of each pixel of the second spectral image IMG2 and the gray scale reference value G1. The processor 24 can determine the difference ΔCH2 between the gray scale value of the channel CH2 of each pixel of the second spectral image IMG2 and the gray scale reference value G2. The processor 24 can determine the difference ΔCH3 between the gray scale value of the channel CH3 of each pixel of the second spectral image IMG2 and the gray scale reference value G3. In this way, the processor 24 can determine the differences ΔCH1-ΔCH9 between the gray scale values of the 9 channels of each pixel of the second spectral image IMG2 and the corresponding gray scale reference values by traversing the 9 channels of the 1200*1600 pixels of the second spectral image IMG2.

[0067] In some embodiments, the processor 24 can perform a spectral correlation region division on the second spectral image IMG2 according to the difference between the gray value of each channel of each pixel of the second spectral image IMG2 and the corresponding gray reference value. It should be understood that the spectral correlation region division can be understood as screening and classifying the pixels with high spectral information correlation in the second spectral image IMG2. The smaller the difference is, the higher the spectral correlation is. The processor 24 can divide the pixels with smaller difference into one spectral correlation region. Then, the processor 24 performs similar operations as operations S31-S33 on the remaining pixels, and iterates multiple times until all the pixels of the second spectral image IMG2 are divided into various spectral correlation regions, thereby obtaining multiple spectral correlation regions. It can be understood that the number of spectral correlation regions can be different for different shooting objects. For scenes with large differences in image spectral characteristics, the number of spectral correlation regions is small. For scenes with small differences in image spectral characteristics, the number of spectral correlation regions is large. Each spectral correlation region includes multiple pixels. The number of pixels in different spectral correlation regions can be the same or different.

[0068] In some embodiments, the processor 24 can determine the average value of the gray values of the corresponding channels of the multiple pixels of the second spectral image IMG2 as the gray reference value of the corresponding channel. For example, the processor 24 can determine the average value of the gray values of the channel CH1 of the 1200*1600 pixels of the second spectral image IMG2 as the gray reference value G1 of the channel CH1. For another example, the processor 24 can determine the average value of the gray values of the channel CH2 of the 1200*1600 pixels of the second spectral image IMG2 as the gray reference value G2 of the channel CH2. For another example, the processor 24 can determine the average value of the gray values of the channel CH3 of the 1200*1600 pixels of the second spectral image IMG2 as the gray reference value G3 of the channel CH3. In this way, the processor 24 can determine the gray reference values G1-G9 of the 9 channels respectively by iterating through the 9 channels of the 1200*1600 pixels of the second spectral image IMG2.

[0069] In some embodiments, the processor 24 can determine the mean square error between the gray values of the multiple channels of each pixel of the second spectral image IMG2 and the gray reference value of the corresponding channel. For example, the processor 24 can determine the mean square error between the gray values g1-g9 of the 9 channels CH1-CH9 of each pixel of the second spectral image IMG2 and the gray reference values G1-G9. In this way, the processor 24 can determine the mean square error between the gray values of the multiple channels of all the pixels of the second spectral image IMG2 and the gray reference value of the corresponding channel by iterating through the 1200*1600 pixels of the second spectral image IMG2.

[0070] In some embodiments, the processor 24 can divide, in operation S33, the pixels of the second spectral image IMG2 having a mean square error of the gray scale values of the multiple channels of each pixel and the gray scale reference values of the corresponding channels less than a first threshold value into the first spectral correlation region. For example, the processor 24 compares the mean square error of the gray scale values g1-g9 of the 9 channels CH1-CH9 of each pixel in the second spectral image IMG2 and the gray scale reference values G1-G9 with the first threshold value, and divides the pixels less than the first threshold value into the first spectral correlation region.

[0071] In some embodiments, as shown in FIG. 3, the step S3 further includes operations S34-S36. The step S3 further includes performing operations S34-S36 on the remaining pixels of the second spectral image IMG2. The remaining pixels are the pixels of the second spectral image IMG2 that are not divided into the spectral correlation region. For example, the remaining pixels include the pixels of the second spectral image IMG2 having a mean square error of the gray scale values of the multiple channels of each pixel and the gray scale reference values of the corresponding channels not less than the first threshold value, i.e., the pixels not divided into the first spectral correlation region. Figure 4

[0072] In some embodiments, the processor 24 can determine, in operation S34, an average value of the gray scale values of the corresponding channels of the remaining pixels, and take the average value as the gray scale reference value of the corresponding channels of the remaining pixels. Operation S34 is similar to operation S31. The processor 24 can determine, in operation S35, the mean square error of the gray scale values of the multiple channels of each pixel in the remaining pixels and the gray scale reference values of the corresponding channels. Operation S35 is similar to operation S32. The processor 24 can divide, in operation S36, the pixels having a mean square error less than a second threshold value into the second spectral correlation region. Operation S36 is similar to operation S33. Operations S34-S36 are repeatedly performed until all the pixels of the second spectral image IMG2 are divided into the spectral correlation regions, and a plurality of spectral correlation regions are obtained.

[0073] It should be noted that the first threshold value and the second threshold value do not necessarily have a size relationship, and can be the same or different. It can be understood that the higher the threshold value of the mean square error, the fewer the number of spectral correlation regions, and the lower the accuracy of the spectral image. The lower the threshold value of the mean square error, the more the number of spectral correlation regions, and the higher the accuracy of the spectral image. The present application does not limit the number of spectral correlation regions and the threshold value of the mean square error, and in actual application, it can be set according to the accuracy of the spectral image and other requirements. In addition, the threshold value of the mean square error can be constant or can change during the division of the spectral correlation region, depending on the actual requirements.

[0074] ​The foregoing described an example of processor 24 dividing the second spectral image IMG2 into spectral correlation regions based on the grayscale values ​​of pixels. Alternatively, in some embodiments, in step S3, processor 24 can also divide the second spectral image IMG2 into spectral correlation regions based on a clustering algorithm. In some embodiments, the clustering algorithm may include K-means clustering, etc. Processor 24 can divide the second spectral image IMG2 into k spectral correlation regions based on a set value of k, where k is a positive integer. It should be noted that the present invention does not limit the method of determining the value of k. In some embodiments, the value of k can be determined by the elbow method or the contour coefficient method. In some embodiments, the value of k can also be determined based on user input instructions.

[0075] In some embodiments, in step S4, the processor 24 can obtain multiple spectral curves based on multiple spectral correlation regions. One spectral correlation region includes one spectral curve. The processor 24 can obtain spectral curves for each of the multiple spectral correlation regions. For example, the processor 24 can obtain spectral curve S1 based on spectral correlation region A1. The processor 24 can obtain spectral curve S2 based on spectral correlation region A2. The processor 24 can obtain spectral curve S3 based on spectral correlation region A3.

[0076] In some embodiments, in step S4, the processor 24 can determine the average gray value of the corresponding channel for each pixel in each spectral correlation region. For example, the processor 24 divides the second spectral image IMG2 into spectral correlation regions, obtaining three spectral correlation regions A1, A2, and A3. Each spectral correlation region A1, A2, and A3 includes multiple pixels. The number of pixels in different spectral correlation regions may be the same or different. Each pixel includes data from nine channels CH1 to CH9. The processor 24 can determine the average gray value g1' of channel CH1 for each pixel in spectral correlation region A1. A1 The average grayscale value g2' of channel CH2 of each pixel A1 The average grayscale value g3' of channel CH3 of each pixel A1 This process continues until the average grayscale value g9' of channel CH9 for each pixel is determined. A1 Thus, processor 24 can obtain the average grayscale value g1' of the nine channels CH1~CH9 of all pixels in the spectral correlation region A1. A1 ~ g9' R9 A set of average gray values ​​GA1 (g1') was obtained. A1 g2' A1 g3' A1 ,…,g9' A1 Similarly, processor 24 can determine the average gray value of each pixel's corresponding channel in the spectral correlation region A2, thus obtaining g1'.A2 g9 A2 , obtaining another set of average gray values GA2(g1 A2 , g2 A2 , g3 A2 , …, g9 A2 ). Similarly, the processor 24 can determine the average gray values of the corresponding channels of each pixel in the spectral correlation region A3, obtaining g1 A3 g9 A3 , obtaining another set of average gray values GA3(g1 A3 , g2 A3 , g3 A3 , …, g9 A3 ). By traversing the N spectral correlation regions, the processor 24 can obtain N sets of average gray values.

[0077] In some embodiments, in step S4, the processor 24 can perform spectral inversion on each spectral correlation region according to the average gray values of the corresponding channels of each pixel in each spectral correlation region, to obtain the spectral curve of each spectral correlation region. For example, for the spectral correlation region Al, the processor 24 can perform spectral inversion on the spectral correlation region Al according to the average gray values GA1(g1 A1 , g2 A1 , g3 A1 , …, g9 A1 ), to obtain the spectral curve S1 of the spectral correlation region Al. For example, for the spectral correlation region A2, the processor 24 can perform spectral inversion on the spectral correlation region A2 according to the average gray values GA2(g1 A2 , g2 A2 , g3 A2 , …, g9 A2 ), to obtain the spectral curve S2 of the spectral correlation region A2. For example, for the spectral correlation region A3, the processor 24 can perform spectral inversion on the spectral correlation region A3 according to the average gray values GA3(g1 A3 , g2 A3 , g3 A3 , …, g9 A3 ), to obtain the spectral curve S3 of the spectral correlation region A3. Figure 5 A schematic diagram of an exemplary spectral curve is shown, which is consistent with some embodiments of the present application. As shown in FIG. 2, the abscissa represents the wavelength, and the ordinate represents the reflectivity. Figure 5As shown, the horizontal axis represents wavelength, and the vertical axis represents light intensity. The spectral curve S1 represents the spectral curve of the spectral correlation region A1. The spectral curve S2 represents the spectral curve of the spectral correlation region A2. The spectral curve S3 represents the spectral curve of the spectral correlation region A3. The wavelength range of the spectral curve S1 is 350-900 nm. The wavelength range of the spectral curve S2 is 350-900 nm. The wavelength range of the spectral curve S3 is 350-900 nm. It should be noted that, for the sake of convenience, Figure 5 The horizontal axis of the spectral curve S1 schematically shows a partial range 400-650 nm of the full wavelength range 350-900 nm.

[0078] In some embodiments, the processor 24 can perform spectral inversion on each spectral correlation region according to a spectral inversion formula (Formula 1) to obtain a spectral curve of each spectral correlation region. For example, for the spectral correlation region A1, the processor 24 can respectively perform spectral inversion according to the average value g1' A1 , g2' A1 , g3' A1 , …, g9' A1 According to the spectral inversion formula (Formula 1), nine relationship formulas are determined. Based on the nine relationship formulas, the processor 24 can obtain nine reflectances R1-R9. Based on the nine reflectances R1-R9, the processor 24 can obtain the spectral curve S1 of the spectral correlation region A1.

[0079] I x R x Tg x η = GAN … (Formula 1)

[0080] wherein I represents the light source spectrum. R represents the reflectance. Tg represents the transmittance curve of the corresponding channel. η represents the quantum efficiency of the photoelectric sensor. GAN represents the average value of the gray values of the pixels of the spectral correlation region corresponding to the channel.

[0081] Similarly, the processor 24 can perform spectral inversion on the spectral correlation regions A2 and A3 to obtain the spectral curves S2 and S3 of the spectral correlation regions A2 and A3. For N spectral correlation regions, the processor 24 performs similar operations to obtain N spectral curves.

[0082] In some embodiments, at step S5, the processor 24 can obtain a spectral image of the target waveband according to the target waveband, the plurality of spectral curves and the first spectral image. It is to be noted that the target waveband can be set according to requirements. The minimum target waveband range can be 1 nm. The maximum target waveband range can be the full waveband range (e.g. 350~900 nm) in which the spectral image demodulation device 20 operates. For example, the target waveband can be 350~400 nm, 400~500 nm, 550~600 nm, 725~750 nm, 800~900 nm, etc. The target waveband can be determined according to actual requirements. In some embodiments, the processor 24 can determine the target waveband according to the input instruction of the user. The input instruction of the user can include at least one of a voice instruction, a gesture instruction, an image instruction or a text instruction, etc.

[0083] Figure 6 An exemplary operation flow diagram of step S5 consistent with some embodiments of the present application is shown. As shown in Figure 6 , step S5 includes operations S51~S52. In some embodiments, at operation S51, the processor 24 can determine the weight of the plurality of spectral correlation regions according to the target waveband and the plurality of spectral curves. For example, the processor 24 can determine the weight w1 of the spectral correlation region A1 according to the target waveband 550~600 nm and the spectral curve S1. For example, the processor 24 can determine the weight w2 of the spectral correlation region A2 according to the target waveband 550~600 nm and the spectral curve S2. For example, the processor 24 can determine the weight w3 of the spectral correlation region A3 according to the target waveband 550~600 nm and the spectral curve S3.

[0084] In some embodiments, at operation S51, the processor 24 can determine the area proportion of the target waveband in the plurality of spectral curves. According to the area proportion of the target waveband in the plurality of spectral curves, the processor 24 can determine the weight of the plurality of spectral correlation regions. Figures 7A-7C An exemplary diagram of determining the weight of the spectral correlation region consistent with some embodiments of the present application is shown. Referring to Figures 7A-7C , the horizontal axis represents wavelength and the vertical axis represents light intensity. For example, the target waveband is 550~600 nm. For example, the full spectral waveband is 350~900 nm. For example, the spectral curves S1, S2, S3 represent the spectral curves of the spectral correlation regions A1, A2, A3 respectively. The processor 24 can determine the ratio of the target waveband area to the full spectral area under each spectral curve. As shown in Figure 7AAs shown, the processor 24 can calculate the area A11 (area of the shaded part) of the target waveband 550~600nm under the spectral curve S1, calculate the area A12 (area enclosed by the orange line S1 and the horizontal axis 350~900nm interval) of the full spectrum 350~900nm, and calculate the ratio A11 / A12 of the area A11 and the area A12. The ratio A11 / A12 is the area ratio of the target waveband 550~600nm relative to the spectral curve S1, and is the weight w1 of the spectral correlation region A1.

[0085] Similarly, as shown in FIG. 6, Figure 7B the processor 24 can calculate the area A21 (area of the shaded part) of the target waveband 550~600nm under the spectral curve S2, calculate the area A22 (area enclosed by the yellow line S2 and the horizontal axis 350~900nm interval) of the full spectrum 350~900nm, and calculate the ratio A21 / A22 of the area A21 and the area A22. The ratio A21 / A22 is the area ratio of the target waveband 550~600nm relative to the spectral curve S2, and is the weight w2 of the spectral correlation region A2.

[0086] Similarly, as shown in FIG. 7, Figure 7C the processor 24 can calculate the area A31 (area of the shaded part) of the target waveband 550~600nm under the spectral curve S3, calculate the area A32 (area enclosed by the blue line S3 and the horizontal axis 350~900nm interval) of the full spectrum 350~900nm, and calculate the ratio A31 / A32 of the area A31 and the area A32. The ratio A31 / A32 is the area ratio of the target waveband 550~600nm relative to the spectral curve S3, and is the weight w3 of the spectral correlation region A3.

[0087] It should be noted that, for the sake of convenience, Figure 7A , Figure 7B , Figure 7C the horizontal axis of FIG. 8 schematically shows a partial range 400~650nm of the full waveband range 350~900nm.

[0088] For example, the weight w1 is 0.12. For example, the weight w2 is 0.15. For example, the weight w3 is 0.1. The weights w1, w2, and w3 are the weighted values of the spectral correlation regions A1, A2, and A3. It should be noted that the numerical values of the weights w1, w2, and w3 herein are only exemplary, and the present application is not limited thereto. Based on one or more factors such as different shooting targets, different target wavebands, different full wavebands, and different light sources, the numerical values of the weights may

[0089] It should be noted that the spectral correlation regions A1, A2, A3, the spectral curves S1, S2, S3, the target waveband 550-600 nm, the full spectrum 350-900 nm, and the weights w1, w2, w3 are only for convenient description and are only exemplary. The present application is not limited thereto. For other target wavebands, full spectra, spectral correlation regions, and spectral curves, the weights of the spectral correlation regions can be determined in a similar manner.

[0090] In some embodiments, the processor 24 can obtain a spectral image of a target waveband according to the weights of the plurality of spectral correlation regions and the gray values of the pixels of the first spectral image corresponding to the plurality of spectral correlation regions, in operation S52. For example, the processor 24 can obtain a spectral image of a target waveband according to the weights w1, w2, w3 of the spectral correlation regions A1, A2, A3 and the gray values of the pixels of the first spectral image IMG1 corresponding to the spectral correlation regions A1, A2, A3.

[0091] Figure 8A An exemplary schematic diagram of obtaining a spectral image of a target waveband is shown, which is consistent with some embodiments of the present application. Figure 8B An exemplary schematic diagram of obtaining a spectral image of a target waveband is shown, which is consistent with some embodiments of the present application. Figure 8A 、 8B As shown in FIG. 6, the first spectral image IMG1 includes 1200*1600 pixels, each pixel includes a gray value, and the 1200*1600 pixels include 1200*1600 gray values. The second spectral image IMG2 includes 1200*1600 pixels. The positions of the pixels of the first spectral image IMG1 correspond to the positions of the pixels of the second spectral image IMG2.

[0092] The second spectral image IMG2 includes the spectral correlation regions A1, A2, A3. The spectral correlation region A1 includes the pixels P21, P22. The pixels P21, P22 correspond to the pixels P21', P22' of the first spectral image IMG1, respectively. The gray values of the pixels P21', P22' are g21', g22', respectively. The weight of the spectral correlation region A1 is w1. The spectral correlation region A2 includes the pixels P23, P24. The pixels P23, P24 correspond to the pixels P23', P24' of the first spectral image IMG1, respectively. The gray values of the pixels P23', P24' are g23', g24', respectively. The weight of the spectral correlation region A2 is w2. The spectral correlation region A3 includes the pixels P25, P26. The pixels P25, P26 correspond to the pixels P25', P26' of the first spectral image IMG1, respectively. The weight of the spectral correlation region A3 is w3. The processor 24 can obtain a spectral image of a target waveband according to the weights w1, w2, w3 and the gray values g21', g22', g23', g24', P25', P26'.

[0093] In some embodiments, the processor 24 can multiply the weight of the plurality of spectral correlation regions and the gray value of the pixel of the first spectral image corresponding to the plurality of spectral correlation regions to obtain the spectral image of the target waveband, in operation S52. For example, the processor 24 multiplies the weight w1 with the gray value g21', g22', respectively. The processor 24 multiplies the weight w2 with the gray value g23', g24', respectively. The processor 24 multiplies the weight w3 with the gray value P25', P26', respectively. In this way, the processor 24 can obtain the spectral image of the target waveband. It can be understood that the weight corresponds to the proportion of the target waveband in the full-spectrum waveband, and the gray value corresponds to the intensity of the full waveband. The intensity of the target waveband can be obtained by multiplying the proportion of the target waveband and the intensity of the full waveband, and the spectral image of the target waveband can be obtained. Figure 9 An example effect diagram of the spectral image of the target waveband is shown, which is consistent with some embodiments of the present application. Figure 9 In some embodiments, the spectral image IMG3 is a spectral image of the target waveband of 550-600 nm. It should be noted that, Figure 9 The spectral image IMG3 shown is only an example for introduction, and the present application is not limited thereto. In actual applications, the target waveband can be customized, and the spectral image of any target waveband mentioned above can be obtained through the above series of operations.

[0094] In the present application, the processor 24 obtains the spectral image of the target waveband according to the weight of the spectral correlation region of the second spectral image and the gray value of the pixel of the first spectral image corresponding to the spectral correlation region, which can avoid the loss of spatial information of the first spectral image, avoid the loss of original image data, avoid insufficient information, and help to achieve high-precision and high-reliability spectral image of the target waveband.

[0095] It should be noted that in the above embodiments, the resolution of the first spectral image and the second spectral image, the number of spectral correlation regions, the number of pixels of the spectral correlation region, the weight of the spectral correlation region, the target waveband, the full-spectrum waveband, the number of channels, and the waveband corresponding to the channel are all example for convenient description, and the present application is not limited thereto. It should be understood that for other resolutions of the first spectral image and the second spectral image, the number of spectral correlation regions, the number of pixels of the spectral correlation region, the weight of the spectral correlation region, the target waveband, the full-spectrum waveband, the number of channels, and the waveband corresponding to the channel, the processor can perform similar operations to complete spectral image demodulation, so as to obtain the desired spectral image of the target waveband.

[0096] It should be noted that the processor 24 of the spectral image demodulation device 20 is taken as an example to illustrate the spectral demodulation method 10 in the above embodiments. In some embodiments, the spectral demodulation method 10 of the present application can also be executed by a processor external to the spectral image demodulation device 20, such as a processor of a computer or the like.

[0097] The present application also provides a computer-readable storage medium. The computer-readable storage medium comprises computer-executable instructions stored thereon, which, when executed by a processor, implement the spectral demodulation method 10 as described above.

[0098] In some embodiments, the present application can take the form of a computer program product embodied in one or more storage media having stored thereon program code. The computer-usable storage media include permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, PRAM, SRAM, DRAM, other types of RAM, ROM, EEPROM, flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical storage, magnetic cassette, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0099] The spectral image demodulation method and the spectral image demodulation device of the present application can realize spectral image demodulation, can realize spectral image inversion, have the advantages of high precision, high accuracy, high reliability, etc., and can be widely applied in various fields and various scenarios.

[0100] It should be noted that the present specification provides method operation steps as embodiments or flowcharts, but more or fewer operation steps can be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one of the many execution orders, and does not represent the only execution order. In actual system or device product execution, the method order shown in the embodiments or flowcharts can be executed in sequence or in parallel.

[0101] It should be noted that although the spectral image demodulation device / some modules are mentioned in the above detailed description, such division is merely not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules described above can be implemented in one module. Conversely, the features and functions of one module described above can be further divided into specific embodiments by multiple modules.

[0102] It should be noted that the present application can include any one or more features of any one or more embodiments of the present application. In other words, not all of the features shown in the figures need to be implemented in the spectral image demodulation device / spectral image demodulation method of the present application. Figures 1-9

[0103] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.​

Claims

1. A spectral image demodulation method characterized by, The method comprises: S1: obtaining a first spectral image; S2: performing interpolation processing on the first spectral image to obtain a second spectral image; S3: performing spectral correlation region division on the second spectral image to obtain a plurality of spectral correlation regions; S4: obtaining a plurality of spectral curves according to the plurality of spectral correlation regions; S5: obtaining a spectral image of a target waveband according to the target waveband, the plurality of spectral curves, and the first spectral image; S3 S31: determining a gray reference value of each channel according to a gray value of a corresponding channel of each pixel of the second spectral image; S32: determining a difference between a gray value of each channel of each pixel of the second spectral image and a corresponding gray reference value; S33: performing spectral correlation region division on the second spectral image according to the difference; S4 comprises: determining an average value of a gray value of a corresponding channel of each pixel of each spectral correlation region; and performing spectral inversion on each spectral correlation region according to the average value of the gray value of the corresponding channel of each pixel of each spectral correlation region to obtain a spectral curve of each spectral correlation region; S5 comprises: S51: determining weights of the plurality of spectral correlation regions according to the target waveband and the plurality of spectral curves, including: determining an area proportion of the target waveband in the plurality of spectral curves; and determining the weights of the plurality of spectral correlation regions according to the area proportion of the target waveband in the plurality of spectral curves; and S52: obtaining a spectral image of the target waveband according to the weights of the plurality of spectral correlation regions and gray values of pixels of the first spectral image corresponding to the plurality of spectral correlation regions, including: multiplying the weights of the plurality of spectral correlation regions and the gray values of the pixels of the first spectral image corresponding to the plurality of spectral correlation regions to obtain the spectral image of the target waveband.

2. The spectral image demodulation method according to claim 1, characterized in that, The first spectral image comprises a plurality of pixels, and each pixel comprises spectral information of one channel.

3. The spectral image demodulation method of claim 2, wherein, Each pixel of the second spectral image comprises spectral information of a plurality of channels, and the spectral information is related to a gray value.

4. The spectral image demodulation method according to any one of claims 1-3, wherein S31 comprises: determining an average value of the gray values of the corresponding channels of the plurality of pixels of the second spectral image as the gray reference value of the corresponding channel; S32 comprises: determining a mean square error between the gray values of the plurality of channels of each pixel of the second spectral image and the gray reference value of the corresponding channel; S33 comprises: dividing a pixel with a mean square error less than a first threshold value to a first spectral correlation region.

5. The spectral image demodulation method according to any one of claims 1 to 3, characterized in that, S3 further comprises performing the following operations on the remaining pixels of the second spectral image: S34: determining an average value of the gray values of the corresponding channels of the remaining pixels as the gray reference value of the corresponding channel of the remaining pixels; S35: determining a mean square error between the gray values of the plurality of channels of each pixel in the remaining pixels and the gray reference value of the corresponding channel; S36: dividing a pixel with a mean square error less than a second threshold value to a second spectral correlation region; The operations S34-S36 are repeatedly performed until the plurality of spectral correlation regions are obtained.

6. The spectral image demodulation method according to any one of claims 1 to 3, characterized in that, S3 comprises: performing spectral correlation region division on the second spectral image based on a clustering algorithm; the clustering algorithm comprises a K-means clustering algorithm.

7. A spectral image demodulation device, characterized by, comprises: a microlens array, the microlenses being configured to converge incident light; a filter unit array, disposed downstream of the optical path of the microlens array, the filter unit comprising a filter subunit array, the filter subunits being configured to allow incident light of a preset waveband to pass through; a photosensor array, disposed downstream of the optical path of the filter unit array, the photosensors being configured to receive incident light of the preset waveband and perform photoelectric conversion; and a processor, coupled to the photosensor array, configured to perform the spectral image demodulation method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, comprises computer executable instructions stored thereon, the executable instructions, when executed by a processor, implement the spectral image demodulation method according to any one of claims 1-6.

Images