Spectral super-resolution method, device, equipment and storage medium based on spectral library

Abstract

The present invention belongs to the field of surveying and mapping remote sensing technology, and discloses a spectral super-resolution method, device, equipment and storage medium based on a spectral library. The method includes: obtaining land type information of a target area; determining additional spectral samples according to the land type information; establishing a private spectral library according to a preset spectral library and additional spectral samples; performing spectral super-resolution on a multi-spectral image corresponding to the target area collected in real time according to the private spectral library, and obtaining a target hyperspectral image corresponding to the target area. In the above manner, additional spectral samples are determined based on the land type information of the target area, and then combined with the preset spectral library to form a private spectral library for the target area. Finally, the multi-spectral image collected in real time is super-resolved based on the private spectral library, so that spectral super-resolution can be achieved without the need for low spatial resolution hyperspectral images and high spatial resolution multi-spectral images of the same scene in the target area, thereby reducing the difficulty of implementing spectral super-resolution and maintaining the implementation effect.

Description

Technical Field

[0001] The present invention relates to the field of surveying and mapping remote sensing technology, and in particular to a spectrum super-resolution method, device, equipment and storage medium based on a spectrum library. Background Art

[0002] Hyperspectral imaging technology can divide the spectral range from visible light to infrared bands into hundreds to thousands of bands. Therefore, compared with multispectral images, hyperspectral remote sensing images can provide richer spectral information and are widely used in precision agriculture, environmental monitoring, pest and disease monitoring and other fields.

[0003] With the widespread application of hyperspectral in various industries, the demand for obtaining high spatial resolution and high spectral resolution image data to meet more refined applications is becoming more and more urgent. Among them, the representative directions are fusion, sparse expression, and deep learning. However, the data sources required by the existing spectral super-resolution methods are mostly low spatial resolution hyperspectral images and high spatial resolution multispectral images based on the same scene. In practical applications, it is difficult to obtain the above-mentioned image pairs. Although some studies have proposed a method that only requires one multispectral image and a public hyperspectral library to complete the spectral super-resolution of images, this method is based on the premise that the public spectral library contains all land types in the target area, which still limits the large-scale application. In summary, to obtain the spectral super-resolution image of a region, hyperspectral images and multispectral images of the region at the same time are required. The existing mainstream multispectral satellites in orbit are relatively diverse in payload design and orbit design, and there are relatively few on-board hyperspectral sensors. Therefore, it is difficult to obtain the above-mentioned images, which has become the main obstacle to the large-scale application of spectral super-resolution technology.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide a spectral super-resolution method, device, equipment and storage medium based on a spectral library, aiming to solve the technical problem that the data source required for image super-resolution in the prior art is difficult to obtain, resulting in difficulty in achieving implementation conditions.

[0006] To achieve the above object, the present invention provides a spectrum super-resolution method based on a spectrum library, the method comprising the following steps:

[0007] Obtain land type information of the target area;

[0008] determining additional spectral samples according to the land type information;

[0009] Establishing a private spectral library according to the preset spectral library and the additional spectral samples;

[0010] The multispectral image corresponding to the target area collected in real time is spectrally super-resolved according to the private spectral library to obtain a target hyperspectral image corresponding to the target area.

[0011] Optionally, the establishing of a private spectrum library according to the preset spectrum library and the additional spectrum samples comprises:

[0012] Obtaining preset wavelength information of a preset spectrum library;

[0013] Matching the preset wavelength information with the additional wavelength information of the additional spectrum sample to obtain target reflectivity data;

[0014] The target reflectance data is added to the preset spectrum library to form a private spectrum library.

[0015] Optionally, matching the preset wavelength information with the additional wavelength information of the additional spectrum sample to obtain target reflectivity data includes:

[0016] Determine the preset wavelength and the preset wavelength range of the preset spectrum library according to the preset wavelength information;

[0017] Acquire additional wavelength information corresponding to the additional spectrum sample;

[0018] Determine an additional wavelength and an additional wavelength range according to the additional wavelength information;

[0019] Determine a target matching wavelength according to the preset wavelength and the additional wavelength;

[0020] Determining a target matching range according to the preset wavelength range and the additional wavelength range;

[0021] Extracting matching reflectance data that conforms to the target matching wavelength and the target matching range from the additional spectrum samples;

[0022] The matching reflectivity data is screened to obtain target reflectivity data.

[0023] Optionally, performing spectral super-resolution on the multispectral image corresponding to the target area acquired in real time according to the private spectral library includes:

[0024] Get multispectral images collected in real time;

[0025] extracting observed image information of the multispectral image and spectral library information of a private spectral library;

[0026] Determine a spectral reconstruction expression of the multispectral image and the target hyperspectral image according to the observed image information and the spectral library information;

[0027] Spectral super-resolution is performed according to the spectral library information and the spectral reconstruction expression to obtain a target hyperspectral image.

[0028] Optionally, obtaining a target spectral dictionary and a target sparse coefficient matrix corresponding to the target hyperspectral image;

[0029] Obtaining a hyperspectral image linear expression according to the target spectral dictionary and the target sparse coefficient matrix;

[0030] Determine a library spectrum dictionary and a library sparse coefficient matrix according to the spectrum library information;

[0031] Determine a spectral library linear relationship according to the library spectral dictionary and the library sparse coefficient matrix;

[0032] A target hyperspectral image is obtained according to the hyperspectral image linear expression, the spectral library linear relationship and the spectral reconstruction expression.

[0033] Optionally, obtaining a target hyperspectral image according to the hyperspectral image linear expression, the spectral library linear relationship and the spectral reconstruction expression comprises:

[0034] Performing band matching on the private spectral library through a band matching matrix to obtain a matched spectral library;

[0035] extracting a matching library spectrum dictionary of the matched spectrum library;

[0036] Determining a first subordinate relationship between the private spectral library and the target hyperspectral image;

[0037] Determining a second subordinate relationship between the matched spectral library and the target hyperspectral image according to the first subordinate relationship;

[0038] Determine an equivalent sparse coefficient matrix according to the matching library spectral dictionary, the second subordinate relationship, the spatial transformation matrix, the reconstructed image linear expression and the spectral library linear relationship;

[0039] The target hyperspectral image is obtained according to the equivalent sparse coefficient matrix and the spectral reconstruction expression.

[0040] Optionally, obtaining a target hyperspectral image according to the equivalent sparse coefficient matrix and the spectral reconstruction expression includes:

[0041] Get the spectral response function and zero-mean Gaussian noise;

[0042] Determine a reconstruction super-resolution model of the target hyperspectral image according to the spectral response function, the zero-mean Gaussian noise and the spectral reconstruction expression;

[0043] The reconstructed super-resolution model is solved according to the equivalent sparse coefficient matrix to obtain a target hyperspectral image.

[0044] In addition, to achieve the above object, the present invention further proposes a spectrum super-resolution device based on a spectrum library, the spectrum super-resolution device based on a spectrum library comprising:

[0045] A land type acquisition module is used to obtain land type information of a target area;

[0046] A sample determination module, used for determining additional spectral samples according to the land type information;

[0047] A spectrum library establishment module, used for establishing a private spectrum library according to a preset spectrum library and the additional spectrum sample;

[0048] The spectral super-resolution module is used to perform spectral super-resolution on the multi-spectral image corresponding to the target area collected in real time according to the private spectral library to obtain a target hyper-spectral image corresponding to the target area.

[0049] In addition, to achieve the above-mentioned purpose, the present invention also proposes a spectral super-resolution device based on a spectral library, and the spectral super-resolution device based on a spectral library comprises: a memory, a processor, and a spectral super-resolution program based on the spectral library stored in the memory and executable on the processor, and the spectral super-resolution program based on the spectral library is configured to implement the steps of the spectral super-resolution method based on the spectral library as described above.

[0050] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a spectral super-resolution program based on a spectral library is stored. When the spectral super-resolution program based on the spectral library is executed by a processor, the steps of the spectral super-resolution method based on the spectral library as described above are implemented.

[0051] The present invention obtains land type information of a target area; determines additional spectral samples according to the land type information; establishes a private spectral library according to a preset spectral library and the additional spectral samples; performs spectral super-resolution on a multispectral image corresponding to the target area collected in real time according to the private spectral library, and obtains a target hyperspectral image corresponding to the target area. In this way, additional spectral samples are determined based on the land type information of the target area under study, so that the additional spectral samples are combined with the preset spectral library to form a private spectral library for the target area, and finally, image super-resolution is performed on the multispectral image collected in real time based on the private spectral library, so that spectral super-resolution can be achieved without the need for low spatial resolution hyperspectral images and high spatial resolution multispectral images of the same scene in the target area, thereby reducing the difficulty of implementing spectral super-resolution and maintaining the implementation effect of spectral super-resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1It is a structural schematic diagram of a spectrum super-resolution device based on a spectrum library in a hardware operating environment involved in an embodiment of the present invention;

[0053] Figure 2 It is a flow chart of the first embodiment of the spectrum super-resolution method based on the spectrum library of the present invention;

[0054] Figure 3 It is a schematic diagram of a process of establishing a private spectrum library in an embodiment of a spectrum super-resolution method based on a spectrum library of the present invention;

[0055] Figure 4 It is a flow chart of a second embodiment of a spectrum super-resolution method based on a spectrum library of the present invention;

[0056] Figure 5 It is a structural block diagram of the first embodiment of the spectrum super-resolution device based on the spectrum library of the present invention.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0059] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of a spectrum super-resolution device based on a spectrum library in the hardware operating environment involved in the embodiment of the present invention.

[0060] like Figure 1 As shown, the spectrum super-resolution device based on the spectrum library may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0061] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the spectral super-resolution device based on the spectral library, and may include more or less components than those shown in the figure, or a combination of certain components, or a different arrangement of components.

[0062] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a spectrum super-resolution program based on a spectrum library.

[0063] exist Figure 1 In the spectral super-resolution device based on spectral library shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the spectral super-resolution device based on spectral library of the present invention can be set in the spectral super-resolution device based on spectral library, and the spectral super-resolution device based on spectral library calls the spectral super-resolution program based on spectral library stored in the memory 1005 through the processor 1001, and executes the spectral super-resolution method based on spectral library provided in an embodiment of the present invention.

[0064] The embodiment of the present invention provides a spectrum super-resolution method based on a spectrum library, referring to Figure 2 , Figure 2 The figure is a flow chart of a first embodiment of a spectrum super-resolution method based on a spectrum library according to the present invention.

[0065] In this embodiment, the spectrum super-resolution method based on the spectrum library includes the following steps:

[0066] Step S10: Obtaining land type information of the target area.

[0067] It should be noted that the executor of this embodiment is a server, which can be a physical server or a cloud server, mainly a server for implementing the spectral super-resolution method based on the spectral library, or other devices that can achieve this function, and this embodiment does not limit this.

[0068] It should be understood that the current spectral super-resolution methods for remote sensing images all require the real-time acquisition of image pairs of low spatial resolution high spectral images and high spatial resolution multispectral images of the same scene, and then deep learning is performed based on the image pairs to complete the spectral super-resolution. However, the conditions for acquiring the image pairs are very harsh, which makes the implementation of spectral super-resolution very difficult. The solution of this embodiment determines additional spectral samples based on the land type information of the target area, which are combined with a preset spectral library to form a private spectral library for the target area. Finally, the real-time acquired multispectral images are super-resolved based on the private spectral library, which achieves spectral super-resolution without the need for low spatial resolution high spectral images and high spatial resolution multispectral images of the same scene in the target area, thereby reducing the difficulty of implementing spectral super-resolution and maintaining the implementation effect of spectral super-resolution.

[0069] In a specific implementation, the target area refers to an area defined by a user that needs to be spectrally super-resolved, and may be an area with any contour, any area, and any position, which is not limited in this embodiment.

[0070] It should be noted that land type information refers to the land type related information of the target area. The land type can be agricultural land, construction land, unused land, etc. Therefore, the land type information of the target area refers to the land type and other related information corresponding to each location in the target area.

[0071] Step S20: determining additional spectral samples according to the land type information.

[0072] It should be understood that determining additional spectral samples according to the land type information means: after determining the land type information of the target area, spectral sampling is performed, and the spectrum should be collected in clear and cloudless weather as much as possible, and the collection work is carried out between 10:30 and 13:30. During the collection, the probe is tilted at 90° and is about 1m away from the sample height; the reference plate is placed perpendicular to the measurement direction of the scanning probe, and the properties of the observed target (plant name, health, coverage, surrounding vegetation), instrument model weather conditions, measurement time, observation and recording personnel, spectral naming and recording number, geographic coordinates and elevation, field photo number and other auxiliary parameters are recorded in detail during the measurement process; the collection site is the vegetation canopy, and 5 typical 2m×2m plots are selected for each vegetation. The spectral data finally collected is the spectral reflectance. Then, based on the land type information and the collected spectral data, the data that needs to be added to the private spectral library is extracted as an additional spectral sample. The additional spectral sample is a ground feature spectral sample obtained by taking the land type T of the target area as the class of interest.

[0073] Step S30: establishing a private spectrum library according to the preset spectrum library and the additional spectrum samples.

[0074] In a specific implementation, after the additional spectrum samples are determined, the additional spectrum samples are first wavelength matched with the preset spectrum library, so as to be merged into a private spectrum library.

[0075] It should be noted that the preset spectral library is the USGS spectral library, which is an open source public spectral library.

[0076] It should be understood that if Figure 3 The figure shows the process of establishing a private spectral library. A private spectral library is established based on the USGG spectral library plus extended land types, that is, additional spectral samples of the target area.

[0077] Furthermore, in order to accurately expand the spectral library to obtain a private spectral library, step S30 includes: obtaining preset wavelength information of the preset spectral library; matching the preset wavelength information with the additional wavelength information of the additional spectral sample to obtain target reflectance data; adding the target reflectance data to the preset spectral library to form a private spectral library.

[0078] It should be understood that the preset wavelength information refers to the wavelength, range and other related information corresponding to the preset spectrum library.

[0079] In a specific implementation, matching the preset wavelength information with the additional wavelength information of the additional spectral sample to obtain the target reflectance data means: matching and extracting the preset wavelength information with the additional wavelength information corresponding to the additional spectral sample, thereby extracting the corresponding data in the additional spectral sample that matches the wavelength of the preset spectral library as the target reflectance data.

[0080] It should be noted that after the target reflectance data is obtained, the target reflectance data is added to the preset spectrum library, thereby expanding the data of the preset spectrum library to become a private spectrum library.

[0081] In this way, when sampling in the field, one type of land will often be collected at multiple points, and the data at each point may be affected by lighting conditions, weather conditions, etc. Therefore, it is necessary to screen suitable samples from the additional spectral samples and pre-process them before adding them to the spectral library as the reflectance data of the land type, thus forming a private spectral library.

[0082] Furthermore, in order to complete wavelength matching, the preset wavelength information is matched with the additional wavelength information of the additional spectral sample, and the step of obtaining target reflectivity data includes: determining the preset wavelength and preset wavelength range of the preset spectral library according to the preset wavelength information; obtaining the additional wavelength information corresponding to the additional spectral sample; determining the additional wavelength and the additional wavelength range according to the additional wavelength information; determining the target matching wavelength according to the preset wavelength and the additional wavelength; determining the target matching range according to the preset wavelength range and the additional wavelength range; extracting matching reflectivity data that meets the target matching wavelength and the target matching range from the additional spectral sample; and performing data screening on the matching reflectivity data to obtain target reflectivity data.

[0083] It should be understood that determining the preset wavelength and the preset wavelength range of the preset spectral library according to the preset wavelength information means: determining the relevant information of the data corresponding to the wavelength corresponding to the preset spectral library and the value range of the wavelength range according to the preset wavelength information as the preset wavelength and the preset wavelength range respectively.

[0084] In a specific implementation, the additional wavelength information refers to the wavelength related information corresponding to the additional spectrum sample. The additional wavelength and the additional wavelength range refer to the data corresponding to each wavelength of the additional spectrum sample and the wavelength value range.

[0085] It should be noted that the target matching wavelength is determined based on the preset wavelength and the additional wavelength; the target matching range is determined based on the preset wavelength range and the additional wavelength range, which respectively correspond to the wavelength range in which the additional spectral sample matches the preset spectral library and the wavelength-related information in which there is an overlapping part.

[0086] It should be understood that extracting matching reflectance data that matches the target matching wavelength and the target matching range from the additional spectral samples means: after determining the target matching range and the target matching wavelength, all reflectance data that matches the target matching range and the target matching wavelength in the additional spectral samples are used as matching reflectance data.

[0087] In a specific implementation, after the matching reflectivity data is obtained, data screening is performed, and the data at the same location is screened and one item is retained, so that non-repetitive reflectivity data is finally obtained, which is the target reflectivity data.

[0088] In this way, the data matching and data screening operations are completed accurately, making the establishment of a private spectral library more accurate and free of redundant data.

[0089] Step S40: performing spectral super-resolution on the multispectral image corresponding to the target area collected in real time according to the private spectral library to obtain a target hyperspectral image corresponding to the target area.

[0090] It should be noted that after the private spectral library is established, the multispectral image of the target area collected in real time is spectrally super-resolved according to the private spectral library, and finally the target hyperspectral image corresponding to the target area is obtained.

[0091] This embodiment obtains the land type information of the target area; determines additional spectral samples according to the land type information; establishes a private spectral library according to the preset spectral library and the additional spectral samples; and performs spectral super-resolution on the multispectral image corresponding to the target area collected in real time according to the private spectral library, so as to obtain the target hyperspectral image corresponding to the target area. In this way, additional spectral samples are determined based on the land type information of the target area under study, so that the additional spectral samples are combined with the preset spectral library to form a private spectral library for the target area. Finally, the multispectral image collected in real time is super-resolved based on the private spectral library, so that the spectral super-resolution can be achieved without the need for low spatial resolution hyperspectral images and high spatial resolution multispectral images of the same scene in the target area, thereby reducing the difficulty of implementing spectral super-resolution and maintaining the implementation effect of spectral super-resolution.

[0092] refer to Figure 4 , Figure 4 The figure is a flow chart of a second embodiment of a spectrum super-resolution method based on a spectrum library according to the present invention.

[0093] Based on the first embodiment, the spectrum super-resolution method based on the spectrum library in this embodiment includes in step S40:

[0094] Step S401: Acquire multispectral images collected in real time.

[0095] It should be noted that the multispectral image collected in real time is a remote sensing image obtained through remote sensing technology. Specifically, the multispectral image is a remote sensing image corresponding to the target area.

[0096] Step S402: extracting observed image information of the multispectral image and spectral library information of a private spectral library.

[0097] It should be understood that the observed image information refers to the relevant information corresponding to the multispectral image, which may include the wavelength of the multispectral image and other related information. The spectral library information includes the library spectral dictionary and library sparse coefficient matrix related information corresponding to the private spectral library.

[0098] Step S403: determining a spectral reconstruction expression of the multispectral image and the target hyperspectral image according to the observed image information and the spectral library information.

[0099] In a specific implementation, the spectral reconstruction expression refers to an expression for spectral reconstruction of a multispectral image, specifically:

[0100] X=SH+N X

[0101] Among them, X is the multispectral image, H is the target hyperspectral image, S is the spectral response function, and Nx is the zero-mean Gaussian noise.

[0102] Step S404: performing spectral super-resolution according to the spectral library information and the spectral reconstruction expression to obtain a target hyperspectral image.

[0103] It should be noted that after the spectral reconstruction expression is determined, the library spectral dictionary and the library sparse coefficient matrix determined by the spectral library information are processed and the equation is solved to obtain the target hyperspectral image.

[0104] Further, in order to determine the current expression of the hyperspectral image and the spectral library linear relationship for solving the reconstruction super-resolution model, step S404 includes: obtaining a target spectral dictionary and a target sparse coefficient matrix corresponding to the target hyperspectral image; obtaining a hyperspectral image linear expression according to the target spectral dictionary and the target sparse coefficient matrix; determining a library spectral dictionary and a library sparse coefficient matrix according to the spectral library information; determining a spectral library linear relationship according to the library spectral dictionary and the library sparse coefficient matrix; and obtaining a target hyperspectral image according to the hyperspectral image linear expression, the spectral library linear relationship and the spectral reconstruction expression.

[0105] It should be understood that the target spectral dictionary and the target sparse coefficient matrix are respectively the spectral dictionary and the sparse coefficient matrix corresponding to the target hyperspectral image.

[0106] In the specific implementation, the linear expression of hyperspectral image is:

[0107] H=D H A H

[0108] Where H is the target hyperspectral image, is the target spectrum dictionary, A H ∈R K×N is the target sparse coefficient matrix. Specifically, λ H is the number of spectral bands of the target hyperspectral image, N is the number of pixels in each band of the target hyperspectral image, and K is the number of columns in the target spectral dictionary.

[0109] In a specific implementation, the linear relationship of the spectral library is:

[0110] L=D L A L

[0111] It should be noted that L is a private spectral library. is the library spectral dictionary, A L∈R K×M is the library sparse coefficient matrix. Specifically, λ L is the number of spectral bands of the target hyperspectral image, M is the number of pixels in each band of the target hyperspectral image, and K is the number of columns in the library spectral dictionary.

[0112] It should be understood that obtaining the target hyperspectral image according to the hyperspectral image linear expression, the spectral library linear relationship and the spectral reconstruction expression means: after the hyperspectral image linear expression is obtained, band matching is performed to obtain the reconstructed image linear expression, thereby obtaining the target hyperspectral image in combination with the spectral library linear relationship.

[0113] In this way, the spectral super-resolution operation is realized based on the linear expression of hyperspectral image and the linear relationship of spectral library combined with the spectral reconstruction expression, making the result of spectral super-resolution more accurate.

[0114] Furthermore, in order to obtain an equivalent sparse coefficient matrix, the step of obtaining a target hyperspectral image according to the hyperspectral image linear expression, the spectral library linear relationship and the spectral reconstruction expression includes: performing band matching on the private spectral library through a band matching matrix to obtain a matched spectral library; extracting a matching library spectral dictionary of the matched spectral library; determining a first subordinate relationship between the private spectral library and the target hyperspectral image; determining a second subordinate relationship between the matched spectral library and the target hyperspectral image according to the first subordinate relationship; determining an equivalent sparse coefficient matrix according to the matching library spectral dictionary, the second subordinate relationship, the spatial transformation matrix, the reconstructed image linear expression and the spectral library linear relationship; and obtaining the target hyperspectral image according to the equivalent sparse coefficient matrix and the spectral reconstruction expression.

[0115] In a specific implementation, the private spectrum library is band matched by a band matching matrix, and the band matching matrix is ​​obtained in the matched spectrum library. Then, the private spectral library L is band matched through P to obtain the matched spectral library PL.

[0116] It should be noted that in terms of space, the private spectral library L covers all types of objects in H. The relationship between H and L can be regarded as a subordinate relationship. H and D L can represent H and PL respectively, so the subordinate relationship also exists in D H and D L So the first subordinate relationship is the subordinate relationship between H and L, and then the first subordinate relationship is extended to D H and D L In the spectral dictionary D, it is the corresponding second subordinate relationship. H and D LEach atom in can be considered as a spectral feature without physical meaning, i.e., they are no longer pure spectral endmembers.

[0117] It should be understood that determining the equivalent sparse coefficient matrix according to the matching library spectral dictionary, the second subordinate relationship, the spatial transformation matrix, the reconstructed image linear expression and the spectral library linear relationship refers to: introducing the spatial transformation matrix Q∈R K×K , then the equivalent sparse coefficient matrix can be written as:

[0118] A q =QA H

[0119] Among them, A q is the equivalent sparse coefficient matrix.

[0120] In the specific implementation, when there is an equivalent sparse coefficient matrix, the linear expression of the hyperspectral image can be expressed as:

[0121] H=PD L QA H =D p A q

[0122] Among them, D H =PD L Q, D p =PD L is the matched spectrum dictionary.

[0123] In this way, the general spectral library is mapped to a specific spectral library corresponding to the target hyperspectral image band through the spectral dictionary, that is, an equivalent sparse coefficient matrix is ​​obtained, which can be substituted into the spectral reconstruction expression to calculate the target hyperspectral image.

[0124] Furthermore, in order to obtain the target hyperspectral image, the step of obtaining the target hyperspectral image according to the equivalent sparse coefficient matrix and the spectral reconstruction expression includes: obtaining a spectral response function and zero-mean Gaussian noise; determining a reconstruction super-resolution model of the target hyperspectral image according to the spectral response function, the zero-mean Gaussian noise and the spectral reconstruction expression; solving the reconstruction super-resolution model according to the equivalent sparse coefficient matrix to obtain the target hyperspectral image.

[0125] It should be noted that determining the reconstructed super-resolution model of the target hyperspectral image according to the spectral response function, the zero-mean Gaussian noise and the spectral reconstruction expression means: substituting the spectral response function and the zero-mean Gaussian noise into the spectral reconstruction expression to obtain the reconstructed super-resolution model.

[0126] Specifically, the reconstructed super-resolution model is:

[0127] X=SH+N X

[0128] in, is the spectral response function, N X is the zero-mean Gaussian noise in the model, is the target hyperspectral image, is a multispectral image, N and M (usually N>>M in general application scenarios), respectively represent the number of pixels in each band of H and L, λ H , X and λ L (λ L ≥λ H >>λ X ) represent the spectral band numbers of H, X and L respectively.

[0129] It should be understood that the multispectral image X can be expressed as the spectral degradation of H, and the estimation of H in the reconstruction super-resolution model is an ill-posed inverse problem, which can be solved by introducing sparse expression, that is, substituting the equivalent sparse coefficient matrix into the reconstruction super-resolution model to solve the target hyperspectral image.

[0130] This embodiment obtains multispectral images collected in real time; extracts observed image information of the multispectral image and spectral library information of a private spectral library; determines spectral reconstruction expressions of the multispectral image and the target hyperspectral image based on the observed image information and the spectral library information; performs spectral super-resolution based on the spectral library information and the spectral reconstruction expression to obtain the target hyperspectral image. In this way, accurate and efficient spectral super-resolution based on the spectral library information of the private spectral library is achieved, so that the spectral super-resolution of the multispectral image can be completed by using the existing public hyperspectral library and samples of some classes of interest. Compared with other spectral super-resolution research directions, the requirements for image sources are lower. It is not required to obtain hyperspectral and multispectral image pairs in the same area and the same period, only multispectral images and field sampling samples are required.

[0131] In addition, an embodiment of the present invention further proposes a storage medium, on which a spectral super-resolution program based on a spectral library is stored. When the spectral super-resolution program based on the spectral library is executed by a processor, the steps of the spectral super-resolution method based on the spectral library as described above are implemented.

[0132] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0133] Reference Figure 5 , Figure 5 It is a structural block diagram of the first embodiment of the spectrum super-resolution device based on the spectrum library of the present invention.

[0134] like Figure 5 As shown, the spectrum super-resolution device based on the spectrum library proposed in the embodiment of the present invention includes:

[0135] The land type acquisition module 10 is used to acquire land type information of a target area.

[0136] The sample determination module 20 is used to determine additional spectral samples according to the land type information.

[0137] The spectrum library establishment module 30 is used to establish a private spectrum library according to the preset spectrum library and the additional spectrum samples.

[0138] The spectral super-resolution module 40 is used to perform spectral super-resolution on the multi-spectral image corresponding to the target area collected in real time according to the private spectral library to obtain a target hyper-spectral image corresponding to the target area.

[0139] This embodiment obtains the land type information of the target area; determines additional spectral samples according to the land type information; establishes a private spectral library according to the preset spectral library and the additional spectral samples; and performs spectral super-resolution on the multispectral image corresponding to the target area collected in real time according to the private spectral library, so as to obtain the target hyperspectral image corresponding to the target area. In this way, additional spectral samples are determined based on the land type information of the target area under study, so that the additional spectral samples are combined with the preset spectral library to form a private spectral library for the target area. Finally, the multispectral image collected in real time is super-resolved based on the private spectral library, so that the spectral super-resolution can be achieved without the need for low spatial resolution hyperspectral images and high spatial resolution multispectral images of the same scene in the target area, thereby reducing the difficulty of implementing spectral super-resolution and maintaining the implementation effect of spectral super-resolution.

[0140] In one embodiment, the spectral library establishment module 30 is further used to obtain preset wavelength information of the preset spectral library; match the preset wavelength information with the additional wavelength information of the additional spectral sample to obtain target reflectance data; and add the target reflectance data to the preset spectral library to form a private spectral library.

[0141] In one embodiment, the spectral library establishment module 30 is also used to determine the preset wavelength and the preset wavelength range of the preset spectral library according to the preset wavelength information; obtain the additional wavelength information corresponding to the additional spectral sample; determine the additional wavelength and the additional wavelength range according to the additional wavelength information; determine the target matching wavelength according to the preset wavelength and the additional wavelength; determine the target matching range according to the preset wavelength range and the additional wavelength range; extract matching reflectance data that meets the target matching wavelength and the target matching range from the additional spectral sample; and perform data screening on the matching reflectance data to obtain target reflectance data.

[0142] In one embodiment, the spectral super-resolution module 40 is also used to obtain a multispectral image collected in real time; extract observation image information of the multispectral image and spectral library information of a private spectral library; determine a spectral reconstruction expression of the multispectral image and the target hyperspectral image based on the observation image information and the spectral library information; perform spectral super-resolution based on the spectral library information and the spectral reconstruction expression to obtain a target hyperspectral image.

[0143] In one embodiment, the spectral super-resolution module 40 is further used to obtain a target spectral dictionary and a target sparse coefficient matrix corresponding to the target hyperspectral image; obtain a hyperspectral image linear expression according to the target spectral dictionary and the target sparse coefficient matrix; determine a library spectral dictionary and a library sparse coefficient matrix according to the spectral library information; determine a spectral library linear relationship according to the library spectral dictionary and the library sparse coefficient matrix; and obtain a target hyperspectral image according to the hyperspectral image linear expression, the spectral library linear relationship and the spectral reconstruction expression.

[0144] In one embodiment, the spectral super-resolution module 40 is further used to perform band matching on the private spectral library through a band matching matrix to obtain a matched spectral library; extract a matching library spectral dictionary of the matched spectral library; determine a first subordinate relationship between the private spectral library and the target hyperspectral image; determine a second subordinate relationship between the matched spectral library and the target hyperspectral image based on the first subordinate relationship; determine an equivalent sparse coefficient matrix based on the matching library spectral dictionary, the second subordinate relationship, the spatial transformation matrix, the reconstructed image linear expression and the spectral library linear relationship; and obtain the target hyperspectral image based on the equivalent sparse coefficient matrix and the spectral reconstruction expression.

[0145] In one embodiment, the spectral super-resolution module 40 is also used to obtain a spectral response function and a zero-mean Gaussian noise; determine a reconstruction super-resolution model of a target hyperspectral image according to the spectral response function, the zero-mean Gaussian noise and the spectral reconstruction expression; solve the reconstruction super-resolution model according to the equivalent sparse coefficient matrix to obtain the target hyperspectral image.

[0146] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0147] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0148] In addition, for technical details not described in detail in this embodiment, reference can be made to the spectrum super-resolution method based on the spectrum library provided in any embodiment of the present invention, which will not be described again here.

[0149] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0150] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0152] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A spectrum super-resolution method based on a spectrum library, characterized in that: The spectrum super-resolution method based on the spectrum library comprises: Obtain land type information of the target area; determining additional spectral samples according to the land type information; Obtaining preset wavelength information of a preset spectrum library; Matching the preset wavelength information with the additional wavelength information of the additional spectrum sample to obtain target reflectivity data; Adding the target reflectance data to the preset spectrum library to form a private spectrum library; Get multispectral images collected in real time; extracting observed image information of the multispectral image and spectral library information of a private spectral library; The spectrum reconstruction expression of the multispectral image and the target hyperspectral image is determined according to the observed image information and the spectrum library information, and the spectrum reconstruction expression is X=SH+N X , where X is the multispectral image, H is the target hyperspectral image, S is the spectral response function, and Nx is the zero-mean Gaussian noise; Obtain the target spectral dictionary and target sparse coefficient matrix corresponding to the target hyperspectral image; The target hyperspectral image linear expression is obtained according to the target spectral dictionary and the target sparse coefficient matrix. The target hyperspectral image linear expression is: H = D H A H , where D H is the target spectrum dictionary, A H is the target sparse coefficient matrix; Determine a library spectrum dictionary and a library sparse coefficient matrix according to the spectrum library information; The linear relationship of the private spectrum library is determined according to the library spectrum dictionary and the library sparse coefficient matrix. The linear relationship of the private spectrum library is L=D L A L , where L is a private spectral library, D L is the library spectral dictionary, A L is the library sparse coefficient matrix; Performing band matching on the private spectral library through a band matching matrix to obtain a matched spectral library; extracting a matching library spectrum dictionary of the matched spectrum library; Determining a first subordinate relationship between the private spectral library and the target hyperspectral image; Determining a second subordinate relationship between the matched spectral library and the target hyperspectral image according to the first subordinate relationship; According to the matching library spectral dictionary, the second subordinate relationship, the spatial transformation matrix, the linear expression of the target hyperspectral image and the linear relationship of the target spectral library, the equivalent sparse coefficient matrix is ​​determined, wherein the first subordinate relationship is the subordinate relationship between H and L, and the first subordinate relationship is extended to D H and D L The corresponding second subordinate relationship is in the space transformation matrix Q∈R K×K , where R is a real number and K is the number of columns in the library spectrum dictionary. Then the equivalent sparse coefficient matrix is ​​written as: A q =QA H , A q is the equivalent sparse coefficient matrix; The target hyperspectral image is obtained according to the equivalent sparse coefficient matrix and the spectral reconstruction expression.

2. The method according to claim 1, characterized in that The matching of the preset wavelength information with the additional wavelength information of the additional spectrum sample to obtain target reflectivity data includes: Determine the preset wavelength and the preset wavelength range of the preset spectrum library according to the preset wavelength information; Acquire additional wavelength information corresponding to the additional spectrum sample; Determine an additional wavelength and an additional wavelength range according to the additional wavelength information; Determine a target matching wavelength according to the preset wavelength and the additional wavelength; Determining a target matching range according to the preset wavelength range and the additional wavelength range; Extracting matching reflectance data that conforms to the target matching wavelength and the target matching range from the additional spectrum samples; The matching reflectivity data is screened to obtain target reflectivity data.

3. The method according to claim 1, characterized in that The step of obtaining a target hyperspectral image according to the equivalent sparse coefficient matrix and the spectral reconstruction expression includes: Get the spectral response function and zero-mean Gaussian noise; Determine a reconstruction super-resolution model of the target hyperspectral image according to the spectral response function, the zero-mean Gaussian noise and the spectral reconstruction expression; The reconstructed super-resolution model is solved according to the equivalent sparse coefficient matrix to obtain a target hyperspectral image.

4. A spectrum super-resolution device based on a spectrum library, characterized in that: The spectrum super-resolution device based on the spectrum library corresponds to the spectrum super-resolution method based on the spectrum library as described in any one of claims 1 to 3, and the spectrum super-resolution device based on the spectrum library comprises: A land type acquisition module is used to obtain land type information of a target area; A sample determination module, used for determining additional spectral samples according to the land type information; A spectrum library establishment module is used to obtain preset wavelength information of a preset spectrum library; match the preset wavelength information with the additional wavelength information of the additional spectrum sample to obtain target reflectance data; and add the target reflectance data to the preset spectrum library to form a private spectrum library; The spectral super-resolution module is used to obtain multispectral images collected in real time; extract the observed image information of the multispectral image and the spectral library information of the private spectral library; determine the spectral reconstruction expression of the multispectral image and the target hyperspectral image according to the observed image information and the spectral library information, and the spectral reconstruction expression is X=SH+N X , where X is a multispectral image, H is a target hyperspectral image, S is a spectral response function, and Nx is zero-mean Gaussian noise; obtain a target spectral dictionary and a target sparse coefficient matrix corresponding to the target hyperspectral image; obtain a linear expression of the target hyperspectral image according to the target spectral dictionary and the target sparse coefficient matrix, and the linear expression of the target hyperspectral image is: H = D H A H , where D H is the target spectrum dictionary, A H is the target sparse coefficient matrix; according to the spectral library information, a library spectrum dictionary and a library sparse coefficient matrix are determined; according to the library spectrum dictionary and the library sparse coefficient matrix, a private spectral library linear relationship is determined, and the private spectral library linear relationship is L=D L A L , where L is a private spectral library, D L is the library spectral dictionary, A L is a library sparse coefficient matrix; band matching is performed on the private spectral library through a band matching matrix to obtain a matched spectral library; a matching library spectral dictionary of the matched spectral library is extracted; a first subordinate relationship between the private spectral library and the target hyperspectral image is determined; a second subordinate relationship between the matched spectral library and the target hyperspectral image is determined based on the first subordinate relationship; an equivalent sparse coefficient matrix is ​​determined based on the matching library spectral dictionary, the second subordinate relationship, the spatial transformation matrix, the linear expression of the hyperspectral image, and the linear relationship of the spectral library, wherein the first subordinate relationship is the subordinate relationship between H and L, and the first subordinate relationship is generalized to D H and D L The corresponding second subordinate relationship is in the space transformation matrix Q∈R K×K , where R is a real number and K is the number of columns in the library spectrum dictionary. Then the equivalent sparse coefficient matrix is ​​written as: A q =QA H , A q is an equivalent sparse coefficient matrix; a target hyperspectral image is obtained according to the equivalent sparse coefficient matrix and the spectral reconstruction expression.

5. A spectrum super-resolution device based on a spectrum library, characterized in that: The device comprises: a memory, a processor, and a spectral super-resolution program based on a spectral library stored in the memory and executable on the processor, wherein the spectral super-resolution program based on a spectral library is configured to implement the spectral super-resolution method based on a spectral library as described in any one of claims 1 to 3.

6. A storage medium, characterized in that: The storage medium stores a spectrum super-resolution program based on a spectrum library, and when the spectrum super-resolution program based on a spectrum library is executed by a processor, the spectrum super-resolution method based on a spectrum library according to any one of claims 1 to 3 is implemented.

Images