A method, device, equipment and storage medium for constructing spectral images

Abstract

The present application provides a method, device, equipment and storage medium for constructing a spectral image, the method comprising: obtaining a spectral image to be processed and a color image corresponding to the spectral image to be processed; preprocessing the spectral image to be processed to obtain a preprocessed spectral image; inputting the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image, wherein the multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing. The above-mentioned spectral image construction method, device, equipment and storage medium fully exploit all features of the color image from deep to shallow layers, and realize rapid reconstruction of high-precision, high-resolution, hyperspectral images.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device, equipment and storage medium for constructing a spectral image. Background Art

[0002] Existing image fusion methods can be divided into traditional physical model methods and deep learning methods. Traditional physical models are modeled by studying the imaging principles of RGB cameras and hyperspectral imagers, extracting the spatial structure information of high-resolution RGB images and the spectral information of low-resolution hyperspectral images, and fusing them based on the established physical model. However, in practical applications, due to the complexity of the real world and imaging equipment, it is very difficult to take all influencing factors into account when modeling, and most traditional model algorithms based on image fusion have hyperparameters, and their reconstruction accuracy is sensitive to hyperparameters.

[0003] The deep learning method uses a convolutional neural network to learn the residual between a low-resolution hyperspectral image and a high-resolution hyperspectral image using a high-resolution RGB image as a template to complete the structural edge information of the low-resolution hyperspectral image. However, the deep learning method requires a lot of computing power and data support, and the network parameters need to be readjusted and trained between different data sets.

[0004] Therefore, the process of image fusion using traditional physical models and deep learning methods is complicated, and it is difficult to quickly reconstruct high-precision, high-resolution hyperspectral images. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method, device, equipment and storage medium for constructing a spectral image, so as to solve the problems existing in the prior art and realize the rapid reconstruction of a high-resolution hyperspectral image.

[0006] In a first aspect, a method for constructing a spectral image is provided, comprising:

[0007] Acquire a spectral image to be processed and a color image corresponding to the spectral image to be processed;

[0008] Preprocessing the spectral image to be processed to obtain a preprocessed spectral image;

[0009] The preprocessed spectral image and the color image are input into a feature learner for multi-layer feature learning processing to obtain a target spectral image, wherein the multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing.

[0010] The above construction method obtains the spectral image to be processed and the color image corresponding to the spectral image to be processed, preprocesses the spectral image to be processed, obtains the preprocessed spectral image, and inputs the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing, and finally obtains the target spectral image. This spectral image construction method does not require a large amount of data set support and training, fully mines all features of the color image from deep to shallow layers, and can realize rapid reconstruction of high-resolution hyperspectral images with high precision.

[0011] In one embodiment, the inputting the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image includes:

[0012] Inputting the preprocessed spectral image and the color image into a feature learner for deep feature iterative processing to obtain a deep spectral image;

[0013] The deep spectral image and the color image are input into a feature learner for shallow feature iterative processing to obtain a target spectral image.

[0014] In the above embodiment, deep feature iterative processing is first performed in the feature learner, and then shallow feature iterative processing is performed through the deep feature learner, so that the preprocessed spectral image sequentially learns the deep semantic information and shallow texture information of the color image, and finally obtains the target spectral image.

[0015] In one embodiment, the feature learner includes a spectral feature extraction unit, a color feature extraction unit and an error calculation unit;

[0016] The step of inputting the preprocessed spectral image and the color image into a feature learner for deep feature iterative processing to obtain a deep spectral image comprises:

[0017] Inputting the intermediate deep spectral image into the spectral feature extraction unit to perform the Mth extraction of the Nth layer feature to obtain the Mth spectral feature corresponding to the Nth layer feature, and in the first iteration, the intermediate deep spectral image is the preprocessed spectral image;

[0018] The color image is input into the color feature extraction unit to perform the Mth extraction of the Nth layer feature, and the Mth color feature corresponding to the Nth layer feature is obtained;

[0019] Inputting the Mth spectral feature corresponding to the Nth layer feature and the Mth color feature corresponding to the Nth layer feature into an error calculation unit to obtain an error;

[0020] updating the intermediate deep spectral image according to the error to obtain an updated intermediate deep spectral image;

[0021] When the number of iterations reaches the deep target number, the updated intermediate deep spectral image is used as the deep spectral image, and the deep target number is the product of the number of layers of deep features that need to be iterated and the number of iterations of each deep feature.

[0022] In the above embodiment, it is necessary to complete deep feature iteration in the feature learner, input the intermediate deep spectral image and the color image into the feature learner, extract the deep features of the color image and the deep features of the intermediate deep spectral image, calculate the errors of the corresponding spectral features and color features, and update the intermediate deep spectral image according to the errors. When the number of iterations reaches the deep target number, the intermediate deep spectral image is updated, and the update of the intermediate deep spectral image is realized by calculating the errors between the spectral features and the color features. The deep feature image can be obtained by calculating the errors multiple times, and the learning of the deep semantics in the color image is completed to obtain the deep spectral image.

[0023] In one embodiment, the step of inputting the deep spectral image and the color image into a feature learner for shallow feature iterative processing to obtain a target spectral image comprises:

[0024] Inputting the intermediate shallow spectral image into the spectral feature extraction unit to perform the j-th extraction of the i-th layer feature to obtain the j-th spectral feature corresponding to the i-th layer feature, and in the first iteration, the intermediate shallow spectral image is the deep spectral image;

[0025] The color image is input into the color feature extraction unit to perform the j-th extraction of the i-th layer feature, and the j-th color feature corresponding to the i-th layer feature is obtained;

[0026] Inputting the j-th spectral feature corresponding to the i-th layer feature and the j-th color feature corresponding to the i-th layer feature into an error calculation unit to obtain an error;

[0027] updating the intermediate shallow spectral image according to the error to obtain an updated intermediate shallow spectral image;

[0028] When the number of iterations reaches the shallow target number, the updated intermediate shallow spectral image is used as the target spectral image. The shallow target number is the product of the number of shallow feature layers that need to be iterated and the number of iterations of each shallow feature.

[0029] In the above embodiment, it is necessary to complete shallow feature iteration in the feature learner, input the intermediate shallow spectral image and the color image into the feature learner, extract the shallow features of the color image and the shallow features of the intermediate shallow spectral image, calculate the errors of the corresponding spectral features and color features, and update the intermediate shallow spectral image according to the errors. When the number of iterations reaches the deep target number, the intermediate shallow spectral image is updated, and the update of the intermediate shallow spectral image is realized by calculating the errors between the spectral features and the color features. The deep feature image can be obtained by calculating the errors multiple times, and the learning of the shallow semantics in the color image is completed to obtain the target image.

[0030] In one embodiment, the function used by the error calculation unit to perform error calculation is:

[0031]

[0032] represents the Mth color feature corresponding to the Nth layer feature or the jth color feature corresponding to the i-th layer feature; x represents the Mth spectral feature corresponding to the Nth layer feature or the jth spectral feature corresponding to the i-th layer feature, Indicates the The mean square error between and x; Indicates the The error between and x; express The transpose of x T represents the transpose of x; A is a constant.

[0033] In the above embodiment, the error between the spectral feature and the color feature is calculated by an error function. The error is an important influencing factor affecting the updating of the intermediate deep spectral image and the intermediate shallow spectral image. The larger the error value, the greater the difference between the spectral feature and the color feature. According to the error value, the intermediate deep spectral image is updated to obtain the target spectral image, thereby reducing the difference between the spectral feature and the color feature.

[0034] In one embodiment, after inputting the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image, the method further includes:

[0035] The target spectral image is evaluated according to a preset evaluation index to obtain an evaluation result of the target spectral image.

[0036] In the above embodiment, the target spectral image is evaluated by using the evaluation index to determine whether the effect of the reconstruction of the target spectral image meets the requirements.

[0037] In a second aspect, a device for constructing a spectral image is provided, comprising:

[0038] An acquisition module, used for acquiring a spectral image to be processed and a color image corresponding to the spectral image to be processed;

[0039] A preprocessing module, used for preprocessing the spectral image to be processed to obtain a preprocessed spectral image;

[0040] The feature learning module is used to input the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image. The multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing.

[0041] In one embodiment, the feature learning module is specifically used to:

[0042] Inputting the preprocessed spectral image and the color image into a feature learner for deep feature iterative processing to obtain a deep spectral image;

[0043] The deep spectral image and the color image are input into a feature learner for shallow feature iterative processing to obtain a target spectral image.

[0044] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for constructing a spectral image as described above when executing the computer program.

[0045] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps of the method for constructing a spectral image as described above are executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A schematic diagram of the implementation process of the method for constructing a spectral image provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the process of step S300 in the method for constructing a spectral image provided in an embodiment of the present application;

[0049] Figure 3A schematic diagram of the process of step S310 in the method for constructing a spectral image provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of the process of step S320 in the method for constructing a spectral image provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of the composition structure of a device for constructing a spectral image provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0053] Figure 7 A comparison chart of the spectral image constructed in the embodiment of the present application and the real spectral image. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] In one embodiment, a target detection method is provided. The execution subject of the target detection method described in the embodiment of the present invention is a device capable of implementing the target detection method described in the embodiment of the present invention. The device may include but is not limited to a terminal and a server. Among them, the terminal includes a desktop terminal and a mobile terminal. The desktop terminal includes but is not limited to a desktop computer and a car computer; the mobile terminal includes but is not limited to a mobile phone, a tablet, a laptop computer and a smart watch. The server includes a high-performance computer and a high-performance computer cluster.

[0056] Figure 1 The method for constructing a spectral image provided in the embodiment of the present application includes:

[0057] Step 100: Obtain a spectral image to be processed and a color image corresponding to the spectral image to be processed.

[0058] The spectral image to be processed is an image whose resolution needs to be improved, for example, a low-resolution high-spectral image; the color image is a color image corresponding to the image whose resolution needs to be improved, for example, a high-resolution RGB image.

[0059] Step 200: preprocess the spectral image to be processed to obtain a preprocessed spectral image.

[0060] The spectral image to be processed is preprocessed. The preprocessing may be upsampling the spectral image to be processed so that the spatial resolution of the spectral image to be processed is consistent with the spatial resolution of the color image, and a preprocessed spectral image is obtained, that is, the preprocessed image is a spectral image with the same spatial resolution as the color image. For example, assuming that the dimension of the high-resolution RGB image is 512×512×3 and the dimension of the low-resolution hyperspectral image is 16×16×31, the low-resolution hyperspectral image is interpolated and upsampled using Bicubic interpolation, so that the spatial resolution of the low-resolution hyperspectral image is enhanced from 16×16 to 512×512, and a high-resolution hyperspectral image is obtained.

[0061] Step 300, input the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image, wherein the multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing.

[0062] The preprocessed spectral image and the color image are input into the feature learner for multi-layer feature learning. Multi-layer feature learning refers to learning multiple deep features in the color image and multiple shallow features in the color image from the preprocessed image, where the deep features are semantic information, which is the information presented in the visual layer, object layer and conceptual layer of the image information, and the shallow features are texture information and color information, to obtain the target spectral image, which is the spectral image output after the preprocessed spectral image completes multi-layer feature learning.

[0063] The above-mentioned target detection method obtains the spectral image to be processed and the color image corresponding to the spectral image to be processed, preprocesses the spectral image to be processed, obtains the preprocessed spectral image, and inputs the preprocessed spectral image and the color image into the feature learner for multi-layer feature learning processing, and finally obtains the target spectral image. This spectral image construction method does not require a large amount of data set support and training, fully mines all the features of the color image from deep to shallow layers, and can realize the rapid reconstruction of high-resolution hyperspectral images with high precision.

[0064] In one embodiment, Figure 2 As shown, step 300 includes:

[0065] Step 301: Input the preprocessed spectral image and the color image into a feature learner for deep feature iterative processing to obtain a deep spectral image.

[0066] The preprocessed spectral image and the color image are input into the feature learner for deep feature iterative processing, that is, the preprocessed spectral image learns the deep semantic information of the color image to obtain a deep spectral image, which is a spectral image that completes the learning of the deep semantic information of the color image.

[0067] Step 302: Input the deep spectral image and the color image into a feature learner for shallow feature iterative processing to obtain a target spectral image.

[0068] The deep spectral image and the color image are input into the feature learner for shallow feature iterative processing, that is, the deep spectral image learns the shallow texture information of the color image to obtain the target spectral image, which is the spectral image that completes the learning of the shallow texture information of the color image.

[0069] In the above embodiment, deep feature iterative processing is first performed in the feature learner, and then shallow feature iterative processing is performed through the deep feature learner, so that the preprocessed spectral image sequentially learns the deep semantic information and shallow texture information of the color image, and finally obtains the target spectral image.

[0070] In one embodiment, the feature learner includes a spectral feature extraction unit, a color feature extraction unit and an error calculation unit;

[0071] like Figure 3 As shown, step 301 includes:

[0072] Step 3011, input the intermediate deep spectral image into the spectral feature extraction unit to perform the Mth extraction of the Nth layer feature to obtain the Mth spectral feature corresponding to the Nth layer feature. In the first iteration, the intermediate deep spectral image is the preprocessed spectral image.

[0073] A deep neural network is provided in the spectral feature extraction unit. It is assumed that the outputs of the 29th, 22nd, 18th and 15th layers of the deep neural network output layer are selected as deep features. The output of the 29th layer of the neural network output layer is taken as an example to illustrate the process of deep feature extraction of the intermediate deep spectral image. The intermediate deep spectral image is a changing spectral image, and the intermediate deep spectral image is updated once each iteration.

[0074] The intermediate deep spectral image is input into the spectral feature extraction unit. The intermediate deep spectral image input into the spectral feature extraction unit in the first iteration is the preprocessed spectral image. The first extraction of the 29th layer features is performed in the deep neural network to obtain the first spectral features corresponding to the 29th layer features.

[0075] Step 3012: Input the color image into the color feature extraction unit to perform the Mth extraction of the Nth layer feature to obtain the Mth color feature corresponding to the Nth layer feature.

[0076] The color image is input into the color feature extraction unit, and the first extraction of the 29th layer features is performed in the deep neural network to obtain the first color features corresponding to the 29th layer features.

[0077] Step 3013: Input the Mth spectral feature corresponding to the Nth layer feature and the Mth color feature corresponding to the Nth layer feature into an error calculation unit to obtain an error.

[0078] The first spectral feature corresponding to the 29th layer feature and the first color feature corresponding to the 29th layer feature are input into the error calculation unit to obtain the error.

[0079] Step 3014: Update the intermediate deep spectral image according to the error to obtain an updated intermediate deep spectral image.

[0080] According to the error, the intermediate deep spectral image is updated. The error value includes the gradient direction and distance. According to the gradient method and distance, the network weights of the intermediate deep spectral image are updated. When the network weights are updated, the updated intermediate deep spectral image can be obtained. The intermediate deep spectral image of the first iteration is the preprocessed spectral image, and what is obtained at this time is the intermediate deep spectral image after the first update.

[0081] Step 3015, when the number of iterations reaches the deep target number, the updated intermediate deep spectral image is used as the deep spectral image, and the deep target number is the product of the number of layers of deep features that need to be iterated and the number of iterations of each deep feature.

[0082] Iteration refers to completing an intermediate deep spectral image update. When the number of iterations reaches the deep target number, the updated intermediate deep spectral image is used as the deep spectral image. The deep target number is the product of the number of deep feature layers that need to be iterated and the number of iterations of each deep feature. Assuming that the number of deep feature layers that need to be iterated is 4 and the iteration step of each deep feature is 800, the deep target number is 3200.

[0083] For better explanation, after obtaining the intermediate deep spectral image after the first update, the intermediate deep spectral image after the first update is input into the spectral feature extraction unit as the intermediate deep spectral image to perform the second extraction of the 29th layer feature, and obtain the second spectral feature corresponding to the 29th layer feature; the color image is input into the color feature extraction unit to perform the second extraction of the 29th layer feature, and obtain the second color feature corresponding to the 29th layer feature; the second spectral feature corresponding to the 29th layer feature and the second color feature corresponding to the 29th layer feature are input into the error calculation unit to obtain the error; according to the error, the intermediate deep spectral image after the first update is updated, The intermediate deep spectral image after the second update is obtained, and then the intermediate deep spectral image after the second update is input into the spectral feature extraction unit as the intermediate deep spectral image to perform the third extraction of the 29th layer feature, and so on, until the number of iterations of the 29th layer feature reaches 800 times, and then the 22nd, 18th and 15th layers are iterated in the same way. After the 800th iteration of the 15th layer feature, the updated intermediate deep spectral image is obtained. At this time, the number of iterations has reached the deep target number, so the updated intermediate deep spectral image obtained after the 800th iteration of the 15th layer feature is used as the deep spectral image.

[0084] In the above embodiment, it is necessary to complete deep feature iteration in the feature learner, input the intermediate deep spectral image and the color image into the feature learner, extract the deep features of the color image and the deep features of the intermediate deep spectral image, calculate the errors of the corresponding spectral features and color features, and update the intermediate deep spectral image according to the errors. When the number of iterations reaches the deep target number, the intermediate deep spectral image is updated, and the update of the intermediate deep spectral image is realized by calculating the errors between the spectral features and the color features. The deep feature image can be obtained by calculating the errors and updating them multiple times, thereby completing the learning of the deep semantics in the color image and obtaining the deep spectral image.

[0085] In one embodiment, Figure 4 As shown, step 302 includes:

[0086] Step 3021, input the intermediate shallow spectral image into the spectral feature extraction unit to perform the j-th extraction of the i-th layer feature to obtain the j-th spectral feature corresponding to the i-th layer feature. In the first iteration, the intermediate shallow spectral image is the deep spectral image.

[0087] A deep neural network is provided in the spectral feature extraction unit. It is assumed that the outputs of the 13th, 10th, 5th, 3rd and 2nd layers of the deep neural network output layer are selected as shallow features. The output of the 13th layer of the neural network output layer is taken as an example to illustrate the process of shallow feature extraction of the intermediate shallow spectral image. The intermediate shallow spectral image is a changing spectral image, and the intermediate shallow spectral image is updated once each iteration.

[0088] The intermediate shallow spectral image is input into the spectral feature extraction unit. At this time, the intermediate deep spectral image input into the spectral feature extraction unit is a deep spectral image. The first extraction of the 13th layer features is performed in the deep neural network to obtain the first spectral features corresponding to the 13th layer features.

[0089] Step 3022, input the color image into the color feature extraction unit to perform the j-th extraction of the i-th layer feature to obtain the j-th color feature corresponding to the i-th layer feature.

[0090] The color image is input into the color feature extraction unit, and the first extraction of the 13th layer features is performed in the deep neural network to obtain the first color features corresponding to the 13th layer features.

[0091] Step 3023: Input the j-th spectral feature corresponding to the i-th layer feature and the j-th color feature corresponding to the i-th layer feature into an error calculation unit to obtain an error.

[0092] The first spectral feature corresponding to the 13th layer feature and the first color feature corresponding to the 13th layer feature are input into the error calculation unit to obtain the error.

[0093] Step 3024: Update the intermediate shallow spectral image according to the error to obtain an updated intermediate shallow spectral image.

[0094] According to the error, the intermediate shallow spectral image is updated. The error value includes the gradient direction and distance. According to the gradient method and distance, the network weight of the intermediate shallow spectral image is updated. When the network weight update is completed, the updated intermediate shallow spectral image can be obtained. The first iteration input intermediate shallow spectral image is the deep spectral image. At this time, what is obtained is the intermediate shallow spectral image after the first update.

[0095] Step 3025, when the number of iterations reaches the shallow target number, the updated intermediate shallow spectral image is used as the target spectral image, and the shallow target number is the product of the number of shallow feature layers that need to be iterated and the number of iterations of each shallow feature.

[0096] When the number of iterations reaches the shallow target number, the updated intermediate shallow spectral image is used as the shallow spectral image. The shallow target number is the product of the number of deep feature layers that need to be iterated and the number of iterations of each deep feature. Assuming that the number of shallow feature layers that need to be iterated is 5 and the iteration step of each deep feature is 1200, the deep target number is 6000.

[0097] For better explanation, after obtaining the intermediate shallow spectral image after the first update, the intermediate shallow spectral image after the first update is input into the spectral feature extraction unit as the intermediate shallow spectral image to perform the second extraction of the 13th layer features, and obtain the second spectral features corresponding to the 13th layer features; the color image is input into the color feature extraction unit to perform the second extraction of the 13th layer features, and obtain the second color features corresponding to the 13th layer features; the second spectral features corresponding to the 13th layer features and the second color features corresponding to the 13th layer features are input into the error calculation unit to obtain the error; according to the error, the intermediate shallow spectral image after the first update is updated to obtain The intermediate shallow spectrum image after the second update is then input as the intermediate shallow spectrum image into the spectral feature extraction unit for the third extraction of the 13th layer feature, and so on, until the iteration number of the 13th layer feature reaches 1200 times, and then the iterations of the 10th, 5th, 3rd and 2nd layers are performed in the same way. After the 1200th iteration of the 2nd layer feature, the updated intermediate shallow spectrum image is obtained. At this time, the iteration number has reached the shallow target number, and thus the updated intermediate shallow spectrum image obtained after the 1200th iteration of the 2nd layer feature is used as the target spectrum image.

[0098] In the above embodiment, it is necessary to complete shallow feature iteration in the feature learner, input the intermediate shallow spectral image and the color image into the feature learner, extract the shallow features of the color image and the shallow features of the intermediate shallow spectral image, calculate the errors of the corresponding spectral features and color features, and update the intermediate shallow spectral image according to the errors. When the number of iterations reaches the deep target number, the intermediate shallow spectral image is updated, and the update of the intermediate shallow spectral image is realized by calculating the errors between the spectral features and the color features. The deep feature image can be obtained by calculating the errors multiple times, and the learning of the shallow semantics in the color image is completed to obtain the target image.

[0099] In one embodiment, the function used by the error calculation unit to perform error calculation is:

[0100]

[0101] represents the Mth color feature corresponding to the Nth layer feature or the jth color feature corresponding to the i-th layer feature; x represents the Mth spectral feature corresponding to the Nth layer feature or the jth spectral feature corresponding to the i-th layer feature, Indicates the The mean square error between and x; Indicates the The error between and x; express The transpose of x T represents the transpose of x; A is a constant.

[0102] In the process of iterative processing of deep features, represents the Mth color feature corresponding to the Nth layer feature, x represents the Mth spectral feature corresponding to the Nth layer feature, where N is 29 or 22 or 18 or 15, and M is an integer value in [1,800]. In the iterative processing of shallow features, represents the j-th color feature corresponding to the i-th layer feature, x represents the j-th spectral feature corresponding to the i-th layer feature, where i is 13 or 10 or 5 or 3 or 2, and j is an integer value in [1,1200]; A is a constant in deep feature iterative processing or shallow iterative processing, and the value of A is, for example, 100.

[0103] In the above embodiment, the error between the spectral feature and the color feature is calculated by an error function. The error is an important influencing factor affecting the updating of the intermediate deep spectral image and the intermediate shallow spectral image. The larger the error value, the greater the difference between the spectral feature and the color feature. According to the error value, the intermediate deep spectral image is updated to obtain the target spectral image, thereby reducing the difference between the spectral feature and the color feature.

[0104] In one embodiment, after inputting the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image, the method further includes:

[0105] The target spectral image is evaluated according to a preset evaluation index to obtain an evaluation result of the target spectral image.

[0106] The target spectral image can be evaluated by four evaluation indicators: average structural similarity (ASSIM), peak signal-to-noise ratio (PSNR), spectral angle mapping (SAM) and relative dimensionless global error (ERGAS). ASSIM measures image similarity from three aspects: brightness, contrast and structure; PSNR evaluates image quality based on the error between corresponding pixels; SAM measures the similarity between spectra by calculating the angle between two vectors. The smaller the angle, the more similar the two spectra are, and the greater the possibility that they belong to the same type of objects; ERGAS is a commonly used evaluation indicator in image fusion, which measures the accuracy of reconstructed spectra from the spatial structure level. See Table 1 for details.

[0107] Table 1 Evaluation results of target spectral images

[0108] ASSIM PSNR ERGAS SAM Target spectral image 0.988 43.87 0.091 1.30

[0109] You can also compare the target spectrum image with the real spectrum image, see Figure 7 . Figure 7 In the figure, the horizontal axis is wavelength and the vertical axis is spectral reflectance. It can be seen from the figure that the target spectrum image and the real spectrum image are basically consistent.

[0110] In the above embodiment, the target spectral image is evaluated by using the evaluation index to determine whether the effect of the reconstruction of the target spectral image meets the requirements.

[0111] In one embodiment, Figure 5 As shown, a spectral image construction device 500 is provided, comprising:

[0112] An acquisition module 501 is used to acquire a spectral image to be processed and a color image corresponding to the spectral image to be processed;

[0113] A preprocessing module 502 is used to preprocess the spectral image to be processed to obtain a preprocessed spectral image;

[0114] The feature learning module 503 is used to input the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image. The multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing.

[0115] Optionally, the feature learning module is specifically used for:

[0116] Inputting the preprocessed spectral image and the color image into a feature learner for deep feature iterative processing to obtain a deep spectral image;

[0117] The deep spectral image and the color image are input into a feature learner for shallow feature iterative processing to obtain a target spectral image.

[0118] Optionally, the deep feature learning module includes a spectral feature extraction unit, a color feature extraction unit and an error calculation unit;

[0119] The spectral feature extraction unit is used to input the intermediate deep spectral image into the spectral feature extraction unit to perform the Mth extraction of the Nth layer feature to obtain the Mth spectral feature corresponding to the Nth layer feature. In the first iteration, the intermediate deep spectral image is the preprocessed spectral image;

[0120] The color feature extraction unit is used to input the color image into the color feature extraction unit to perform the Mth extraction of the Nth layer feature to obtain the Mth color feature corresponding to the Nth layer feature;

[0121] The error calculation unit is used to input the Mth spectral feature corresponding to the Nth layer feature and the Mth color feature corresponding to the Nth layer feature into the error calculation unit to obtain an error;

[0122] The feature learning module is further used to update the intermediate deep spectral image according to the error to obtain an updated intermediate deep spectral image;

[0123] The feature learning module is also used to use the updated intermediate deep spectral image as the deep spectral image when the number of iterations reaches the deep target number, and the deep target number is the product of the number of layers of deep features that need to be iterated and the number of iterations of each deep feature.

[0124] Optionally, the spectral feature extraction unit is further used to input the intermediate shallow spectral image into the spectral feature extraction unit to perform the j-th extraction of the i-th layer feature to obtain the j-th spectral feature corresponding to the i-th layer feature, and in the first iteration, the intermediate shallow spectral image is the deep spectral image;

[0125] The color feature extraction unit is further used to input the color image into the color feature extraction unit to perform the j-th extraction of the i-th layer feature to obtain the j-th color feature corresponding to the i-th layer feature;

[0126] The error calculation unit is further used to input the j-th spectral feature corresponding to the i-th layer feature and the j-th color feature corresponding to the i-th layer feature into the error calculation unit to obtain an error;

[0127] The feature learning module is further used to update the intermediate shallow spectral image according to the error to obtain an updated intermediate shallow spectral image;

[0128] The feature learning module is also used to use the updated intermediate shallow spectral image as the target spectral image when the number of iterations reaches the shallow target number, and the shallow target number is the product of the number of layers of shallow features that need to be iterated and the number of iterations of each shallow feature.

[0129] Optionally, the function used by the error calculation unit to perform error calculation is:

[0130]

[0131] represents the Mth color feature corresponding to the Nth layer feature or the jth color feature corresponding to the i-th layer feature; x represents the Mth spectral feature corresponding to the Nth layer feature or the jth spectral feature corresponding to the i-th layer feature, express The mean square error between and x; Indicates the The error between and x; express The transpose of x T represents the transpose of x; A is a constant.

[0132] Optionally, the spectral image construction device 500 further includes an evaluation module 504;

[0133] The evaluation module 504 is used to evaluate the target spectral image according to a preset evaluation index to obtain an evaluation result of the target spectral image.

[0134] In one embodiment, Figure 6As shown, a computer device is provided, which can be a terminal or a server. The computer device includes a processor, a memory and a network interface connected by a system bus, and the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and can also store a computer program. When the computer program is executed by the processor, the processor can implement a method for constructing a spectral image. The non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM) or a flash memory. The volatile memory may include a random access memory (RAM) or an external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM). The internal memory may also store a computer program, which, when executed by the processor, enables the processor to perform the target detection method. Those skilled in the art will appreciate that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] The method for constructing a spectral image provided in the present application can be implemented in the form of a computer program. The computer program can be used in Figure 6 The computer device shown in FIG. 5 is run on the computer device. The memory of the computer device can store multiple program templates constituting the target detection device, such as the acquisition module 501, the preprocessing module 502 and the feature learning module 503.

[0136] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the following steps:

[0137] Acquire a spectral image to be processed and a color image corresponding to the spectral image to be processed;

[0138] Preprocessing the spectral image to be processed to obtain a preprocessed spectral image;

[0139] The preprocessed spectral image and the color image are input into a feature learner for multi-layer feature learning processing to obtain a target spectral image, wherein the multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing.

[0140] In one embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the processor performs the following steps:

[0141] Acquire a spectral image to be processed and a color image corresponding to the spectral image to be processed;

[0142] Preprocessing the spectral image to be processed to obtain a preprocessed spectral image;

[0143] The preprocessed spectral image and the color image are input into a feature learner for multi-layer feature learning processing to obtain a target spectral image, wherein the multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing.

[0144] It should be noted that the above-mentioned spectral image construction method, spectral image construction device, computer equipment and computer-readable storage medium belong to a general inventive concept, and the contents of the spectral image construction method, spectral image construction device, computer equipment and computer-readable storage medium embodiments are applicable to each other.

[0145] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. In addition, the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Furthermore, the functional modules in multiple embodiments of the present application can be integrated together to form an independent part, or multiple modules can exist separately, or two or more modules can be integrated to form an independent part. In this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The above is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should all be included in the scope of protection of the present application.

Claims

1. A method for constructing a spectral image, characterized in that: include: Acquire a spectral image to be processed and a color image corresponding to the spectral image to be processed; Preprocessing the spectral image to be processed to obtain a preprocessed spectral image; Inputting the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image, wherein the multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing; The step of inputting the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image includes: Inputting the preprocessed spectral image and the color image into a feature learner for deep feature iterative processing to obtain a deep spectral image; Inputting the deep spectral image and the color image into a feature learner for shallow feature iterative processing to obtain a target spectral image; Wherein, the feature learner includes a spectral feature extraction unit, a color feature extraction unit and an error calculation unit; The step of inputting the preprocessed spectral image and the color image into a feature learner for deep feature iterative processing to obtain a deep spectral image comprises: Inputting the intermediate deep spectral image into the spectral feature extraction unit to perform the Mth extraction of the Nth layer feature to obtain the Mth spectral feature corresponding to the Nth layer feature, and in the first iteration, the intermediate deep spectral image is the preprocessed spectral image; The color image is input into the color feature extraction unit to perform the Mth extraction of the Nth layer feature, and the Mth color feature corresponding to the Nth layer feature is obtained; Inputting the Mth spectral feature corresponding to the Nth layer feature and the Mth color feature corresponding to the Nth layer feature into an error calculation unit to obtain an error; updating the intermediate deep spectral image according to the error to obtain an updated intermediate deep spectral image; When the number of iterations reaches the deep target number, the updated intermediate deep spectral image is used as the deep spectral image, where the deep target number is the product of the number of layers of deep features to be iterated and the number of iterations of each deep feature; The step of inputting the deep spectral image and the color image into a feature learner for shallow feature iterative processing to obtain a target spectral image includes: Inputting the intermediate shallow spectral image into the spectral feature extraction unit to perform the j-th extraction of the i-th layer feature to obtain the j-th spectral feature corresponding to the i-th layer feature, and in the first iteration, the intermediate shallow spectral image is the deep spectral image; The color image is input into the color feature extraction unit to perform the j-th extraction of the i-th layer feature, and the j-th color feature corresponding to the i-th layer feature is obtained; Inputting the j-th spectral feature corresponding to the i-th layer feature and the j-th color feature corresponding to the i-th layer feature into an error calculation unit to obtain an error; updating the intermediate shallow spectral image according to the error to obtain an updated intermediate shallow spectral image; When the number of iterations reaches the shallow target number, the updated intermediate shallow spectral image is used as the target spectral image. The shallow target number is the product of the number of shallow feature layers that need to be iterated and the number of iterations of each shallow feature.

2. The construction method according to claim 1, characterized in that: The function of the error calculation unit for performing error calculation is: represents the Mth color feature corresponding to the Nth layer feature or the jth color feature corresponding to the i-th layer feature; x represents the Mth spectral feature corresponding to the Nth layer feature or the jth spectral feature corresponding to the i-th layer feature, express The mean square error between and x; Indicates the The error between and x; express The transpose of x T represents the transpose of x; A is a constant.

3. The construction method according to claim 1, characterized in that: After the pre-processed spectral image and the color image are input into a feature learner for multi-layer feature learning processing to obtain a target spectral image, the method further includes: The target spectral image is evaluated according to a preset evaluation index to obtain an evaluation result of the target spectral image.

4. A device for constructing a spectral image, characterized in that: include: An acquisition module, used for acquiring a spectral image to be processed and a color image corresponding to the spectral image to be processed; A preprocessing module, used for preprocessing the spectral image to be processed to obtain a preprocessed spectral image; A feature learning module, used for inputting the preprocessed spectral image and the color image into a feature learner for multi-layer feature learning processing to obtain a target spectral image, wherein the multi-layer feature learning processing includes multiple deep feature learning processing and multiple shallow feature learning processing; Wherein, the deep feature learning module includes a spectral feature extraction unit, a color feature extraction unit and an error calculation unit; The spectral feature extraction unit is used to input the intermediate deep spectral image into the spectral feature extraction unit to perform the Mth extraction of the Nth layer feature to obtain the Mth spectral feature corresponding to the Nth layer feature. In the first iteration, the intermediate deep spectral image is the preprocessed spectral image; The color feature extraction unit is used to input the color image into the color feature extraction unit to perform the Mth extraction of the Nth layer feature to obtain the Mth color feature corresponding to the Nth layer feature; The error calculation unit is used to input the Mth spectral feature corresponding to the Nth layer feature and the Mth color feature corresponding to the Nth layer feature into the error calculation unit to obtain an error; The feature learning module is further used to update the intermediate deep spectral image according to the error to obtain an updated intermediate deep spectral image; The feature learning module is further configured to use the updated intermediate deep spectral image as the deep spectral image when the number of iterations reaches a deep target number, wherein the deep target number is the product of the number of layers of deep features to be iterated and the number of iterations of each deep feature; The spectral feature extraction unit is further used to input the intermediate shallow spectral image into the spectral feature extraction unit to perform the j-th extraction of the i-th layer feature to obtain the j-th spectral feature corresponding to the i-th layer feature, and in the first iteration, the intermediate shallow spectral image is the deep spectral image; The color feature extraction unit is further used to input the color image into the color feature extraction unit to perform the j-th extraction of the i-th layer feature to obtain the j-th color feature corresponding to the i-th layer feature; The error calculation unit is further used to input the j-th spectral feature corresponding to the i-th layer feature and the j-th color feature corresponding to the i-th layer feature into the error calculation unit to obtain an error; The feature learning module is further used to update the intermediate shallow spectral image according to the error to obtain an updated intermediate shallow spectral image; The feature learning module is also used to use the updated intermediate shallow spectral image as the target spectral image when the number of iterations reaches the shallow target number, and the shallow target number is the product of the number of layers of shallow features that need to be iterated and the number of iterations of each shallow feature.

5. The construction device according to claim 4, characterized in that The feature learning module is specifically used to: input the preprocessed spectral image and the color image into a feature learner for deep feature iterative processing to obtain a deep spectral image; The deep spectral image and the color image are input into a feature learner for shallow feature iterative processing to obtain a target spectral image.

6. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 3 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps of the method according to any one of claims 1 to 3 are executed.

Images