RGB picture hyperspectral information reconstruction method based on machine learning

Abstract

The invention relates to an RGB picture hyperspectral information reconstruction method based on machine learning, and the method comprises the steps: collecting an RGB image and an HSI image of a plant, respectively building indexes for storage, and obtaining image data; performing cross validation distribution on the image data to obtain a data set comprising a training set and a validation set; based on the index, obtaining corresponding RGB image data and HSI image data from the data set, and obtaining training data; establishing a data loader, loading the training data through the data loader, and training the basic model to obtain a reconstruction model; and inputting the RGB picture to be reconstructed into the reconstruction model to obtain hyperspectral information used for calculating the hyperspectral characteristic parameters. According to the invention, the cost and complexity of hyperspectral image acquisition are reduced, and the efficiency and precision of hyperspectral information reconstruction are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of spectral reconstruction, and in particular to a method for reconstructing hyperspectral information of RGB images based on machine learning. Background Art

[0002] Hyperspectral imaging is an emerging, non-destructive, advanced optical technology that uses an imaging spectrometer to obtain continuous and detailed spectral information of target objects in the visible and near-infrared bands. Unlike traditional multispectral remote sensing, hyperspectral imaging has a very high spectral resolution within the detectable electromagnetic spectrum, and can usually distinguish dozens to hundreds of narrow and continuous bands, which enables hyperspectral data to provide extremely rich and detailed spectral features, providing unprecedented technical means for the analysis of object characteristics.

[0003] Hyperspectral imaging has many versatile applications due to its ability to capture rich spatial and spectral information. For example, hyperspectral technology is used to obtain spectral information of plants at different wavelengths, to achieve accurate analysis of plant physiological and biochemical characteristics, and to measure phenotypic data such as plant pigment composition, biochemical composition, nitrogen content, and water content. However, the equipment used to obtain hyperspectral images is usually expensive, which hinders its promotion and application in plant research. The current general method of mathematical modeling is difficult to accurately reconstruct low-dimensional data into high-dimensional data, and has low accuracy and low robustness. Therefore, the development of a new method for low-cost acquisition and reconstruction of hyperspectral information has theoretical significance and application value. Summary of the invention

[0004] The present invention provides a method for reconstructing hyperspectral information of RGB images based on machine learning, so as to solve the defects of the prior art.

[0005] The present invention provides a method for reconstructing hyperspectral information of RGB images based on machine learning, comprising: S1: Collecting RGB images and HSI images of plants, and indexing the RGB images and the HSI images respectively for storage to obtain image data; S2: performing cross-validation allocation on the image data to obtain a data set including a training set and a validation set; S3: Based on the index, obtain corresponding RGB image data and HSI image data from the data set to obtain training data; S4: establishing a data loader, loading the training data through the data loader, training the basic model, and obtaining a reconstructed model; S5: Inputting the RGB image to be reconstructed into the reconstruction model to obtain hyperspectral information for calculating hyperspectral characteristic parameters.

[0006] According to a method for reconstructing hyperspectral information of an RGB image based on machine learning provided by the present invention, the step of acquiring the RGB image in step S1 specifically includes: S11: Selecting a visible light band from the HSI image; S12: performing weighted synthesis on the visible light bands to obtain a synthetic image; S13: Perform gamma conversion on the synthesized image to obtain an RGB image.

[0007] According to a method for reconstructing hyperspectral information of RGB images based on machine learning provided by the present invention, the cross-validation distribution performed on the image data in step S2 is 10 times.

[0008] According to a method for reconstructing hyperspectral information of an RGB image based on machine learning provided by the present invention, the data loader in step S4 is a DataLoader class, and the data loader is used to package the data from the training set and the validation set in the training data to obtain an iterable object.

[0009] According to a method for reconstructing hyperspectral information of an RGB image based on machine learning provided by the present invention, the parameters of the DataLoader class include: `dataset`: used to specify the source of the dataset; `batch_size`: used to specify the size of data loaded at a time; `shuffle`: used to specify sequential loading, `shuffle` is set to True; `num_workers`: Used to specify the number of threads used to load data.

[0010] According to a method for reconstructing hyperspectral information of an RGB image based on machine learning provided by the present invention, step S4 further comprises: S41: Loading training data from the same sample through the data loader; S42: normalizing the HSI image data and the RGB image data in the training data to obtain normalized HSI image data and normalized RGB image data respectively; S43: performing feature dimension splicing on the normalized HSI image data and the normalized RGB image data to obtain a feature vector; S44: The feature vector is updated through the Adam optimizer to train the basic model for a preset number of rounds to obtain a reconstructed model.

[0011] According to a method for reconstructing hyperspectral information of an RGB image based on machine learning provided by the present invention, step S42 further includes: S421: Perform dimension conversion on the HSI image data to obtain dimensional HSI image data, where the dimension of the dimensional HSI image data is [channel, width, height]; S422: Flatten the dimensional HSI image data to obtain an HSI two-dimensional array; flatten the RGB image data to obtain an RGB two-dimensional array; S423: scaling the HSI two-dimensional array according to the global maximum value of the HSI image to obtain normalized HSI image data; scaling the RGB two-dimensional array to obtain normalized RGB image data.

[0012] According to a method for reconstructing hyperspectral information of an RGB image based on machine learning provided by the present invention, the step of obtaining the global maximum value of the HSI image in step S423 further includes: S4231: traverse all HSI image data; S4232: Read pixel data of a single HSI image data to obtain the maximum pixel extreme value of the pixel data in the single HSI image; S4233: Compare the pixel extreme values ​​of each HSI image data, output the maximum value of the pixel extreme values ​​of all HSI image data, and obtain the global maximum value of the HSI image.

[0013] According to a method for reconstructing hyperspectral information of an RGB image based on machine learning provided by the present invention, the loss function of the reconstruction model in step S4 is expressed as:

[0014] in, is the total loss function of the reconstruction model, is the L1 loss, is the relative L1 loss; The expression of the L1 loss is:

[0015] in, is the sample index value, is the total number of samples, For the The target output value of samples, For the The actual output value of samples; The expression of the relative L1 loss is:

[0016] in, is the feature index value of a single sample, is the total number of features of a single sample, For the The sample in The target output value on each channel, For the The sample in The actual output value on each channel.

[0017] According to a method for reconstructing hyperspectral information of an RGB image based on machine learning provided by the present invention, the hyperspectral characteristic parameters in step S5 specifically include: Conventional normalized spectral index, the expression of the conventional normalized spectral index is:

[0018] in, is the conventional normalized spectral index, is the reflectance of the hyperspectral image at a wavelength of 800nm, is the reflectance of the hyperspectral image at a wavelength of 670nm; Green normalized spectral index, the expression of the green normalized spectral index is:

[0019] in, is the green normalized spectral index, is the reflectance of the hyperspectral image at a wavelength of 560nm; Green-red vegetation index, the expression of the green-red vegetation index is:

[0020] in, is the green-red vegetation index; Red edge normalized difference vegetation index, the expression of the red edge normalized difference vegetation index is:

[0021] in, is the red edge normalized difference vegetation index, is the reflectance of the hyperspectral image at a wavelength of 715 nm; Photochemical vegetation index, the expression of the photochemical vegetation index is:

[0022] in, is the photochemical vegetation index, is the reflectance of the hyperspectral image at a wavelength of 531 nm, is the reflectance of the hyperspectral image at a wavelength of 570nm.

[0023] The invention provides a method for reconstructing hyperspectral information of RGB images based on machine learning. By fusing RGB image information and HSI image information, training a model, and based on a deep learning method, a high-quality hyperspectral image can be reconstructed from an RGB image, thereby avoiding the complexity and high cost of traditional hyperspectral image acquisition equipment and improving the efficiency of obtaining hyperspectral information. In the data collection stage, the invention establishes indexes for RGB images and HSI images, ensures the accuracy and consistency of the data, normalizes the training data, improves the training efficiency and accuracy of the model, and realizes batch loading and sequential loading of data through a data loader, thereby improving the flexibility and efficiency of data processing. In the model training process, the invention splices the normalized HSI image data and RGB image data in feature dimensions, obtains a feature vector containing rich information, and helps the model better learn the mapping relationship between RGB images and hyperspectral images. In addition, in the data normalization stage, the invention adopts a scaling method based on the global maximum value of the HSI image, ensures the comparability of data at different scales, not only improves the robustness of the model, but also helps the model maintain stable performance when processing RGB images under different lighting conditions and noise interference. In general, the present invention can reconstruct hyperspectral information directly from widely available RGB images based on artificial intelligence, which can eliminate the use of cumbersome spectral camera hardware while maintaining high spectral resolution and imaging performance. It has the advantages of high efficiency, low cost and easy operation. The reconstructed spectral information can be used to determine phenotypic data such as plant pigment composition, biochemical composition, nitrogen content, water content and vegetation index calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A schematic diagram of a method for reconstructing hyperspectral information of an RGB image based on machine learning provided in an embodiment of the present invention; Figure 2 A schematic diagram of a method for acquiring an RGB image provided by an embodiment of the present invention; Figure 3 A schematic flow chart of a method for obtaining a reconstruction model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0027] Combine the following Figures 1 to 3 Embodiments of the present invention are described.

[0028] like Figure 1 As shown, the present invention provides a method for reconstructing hyperspectral information of RGB images based on machine learning, comprising: S1: Collecting RGB images and HSI images of plants, and indexing the RGB images and the HSI images respectively for storage to obtain image data.

[0029] like Figure 2 As shown, the step of acquiring the RGB image in step S1 specifically includes: S11: Select a visible light band from the HSI image.

[0030] S12: Perform weighted synthesis on the visible light bands to obtain a synthetic image.

[0031] S13: Perform gamma conversion on the synthesized image to obtain an RGB image.

[0032] In a specific embodiment, the RGB image is obtained by weighted synthesis of the visible light band 400-700nm selected from the corresponding hyperspectral image using CIE standard illuminant D65 and CIE 1931 2° standard observer viewing angle, and gamma correction (γ≈1.4) is performed. All image data consist of corresponding 21×31 pixel image pairs randomly cropped from the RGB and hyperspectral images of each tomato.

[0033] The above-mentioned CIE standard illuminant D65 is a recognized standard illuminant for color measurement and matching. It represents the spectral power distribution of average daylight and is commonly used to calibrate display devices and cameras to ensure color accuracy. The CIE 19312° standard observer viewing angle is a mathematical model used to describe the human visual system's perception of color. It defines the color matching functions of a standard observer at different wavelengths. The 2° viewing angle means that these matching functions are measured within 2° of the center of the field of view.

[0034] S2: Perform cross-validation allocation on the image data to obtain a data set including a training set and a validation set.

[0035] The cross-validation distribution performed on the image data in step S2 is 10 times.

[0036] In a specific embodiment, firstly, data for implementing K-fold cross validation is imported, and then a 10-fold cross validator is initialized to divide the data set into 10 subsets (folds). Subsequently, the number of segmentations of the data set is obtained, specifically 10 times. Then, the training set and validation set lists are initialized to lists containing 0 to 9, respectively, for storing the training set and validation set indexes of each fold. The data is traversed using a `for` loop and `enumerate`. Each iteration returns a pair of training data and validation data, which correspond to the training set and validation set indexes of the current fold, respectively. The training set index and validation set index of each fold are stored separately, and finally the training set and validation set outputs of the first fold are extracted.

[0037] During validation, the entire dataset is divided into 10 subsets (folds) of similar size. Each subset will be used as a validation set during the entire validation process, and the remaining 9 subsets will be used as training sets. For each fold iteration, the 9 subsets other than the current fold will be combined as training data, and the current fold will be used as validation data. The model will be trained on the training set for evaluation.

[0038] The 10-fold cross validation selected by the present invention not only makes each piece of data serve as part of the training set but also as part of the validation set. This method of fully utilizing data is particularly suitable for situations where the amount of data is limited, ensuring that the model learns from all available data.

[0039] S3: Based on the index, corresponding RGB image data and HSI image data are obtained from the data set to obtain training data.

[0040] Specifically, in the aforementioned indexing process, a loop variable is set. When the loop variable is "training", the program will use the file index in the training data and create a file to store the training data. When the loop variable is "verification", the program will use the file index in the verification data and create a file to store the verification data.

[0041] In the above storage, first of all, it is necessary to ensure that the order of the lists is consistent: in order to ensure the validity of the above index correspondence, the two lists must be arranged in the same order, which is guaranteed by a unified naming rule or sorting method. In addition, it is also necessary to ensure the unique identifier of the data storage. When storing, the name of the data set is used uniquely and sequentially to ensure that each data point has a unique identifier, which helps to accurately load and match the corresponding HSI and RGB data in the subsequent model training process.

[0042] When reading, in the program, the two lists of training data and validation data contain the paths of all hyperspectral and RGB image files. The two lists are accessed through the same index to ensure that the read HSI and RGB data belong to the same sample.

[0043] S4: Establish a data loader, load the training data through the data loader, train the basic model, and obtain a reconstructed model.

[0044] Among them, the data loader in step S4 is a DataLoader class, and the data loader is used to package the data from the training set and the verification set in the training data to obtain an iterable object.

[0045] Furthermore, the DataLoader class provided by the present invention packages the training and validation datasets into iterable objects and loads the data in batches according to the specified batch size, which can greatly improve the efficiency of data input during training, especially when the data volume is large and frequent disk reading and writing are required.

[0046] The parameters of the DataLoader class include: `dataset`: used to specify the source of the dataset; `batch_size`: used to specify the size of data loaded at a time; `shuffle`: used to specify sequential loading, `shuffle` is set to True; `num_workers`: Used to specify the number of threads used to load data.

[0047] The loss function of the reconstruction model in step S4 is expressed as:

[0048] in, is the total loss function of the reconstruction model, is the L1 loss, is the relative L1 loss; The expression of the L1 loss is:

[0049] in, is the sample index value, is the total number of samples, For the The target output value of samples, For the The actual output value of samples; The expression of the relative L1 loss is:

[0050] in, is the feature index value of a single sample, is the total number of features of a single sample, For the The sample in The target output value on each channel, For the The sample in The actual output value on each channel.

[0051] The present invention selects L1 loss and relative L1 loss, considering that L1 loss provides a robust error measurement and is suitable for tasks that are insensitive to outliers, while relative L1 loss can more accurately reflect the proportional relationship between the predicted value and the true value by normalizing the error, and is suitable for tasks that need to consider errors of different scales or proportions. Based on their respective advantages and fit with the model objectives, the present invention combines the two loss functions to help the model optimize its performance in different aspects and achieve better training results.

[0052] like Figure 3 As shown, step S4 further includes: S41: Loading training data from the same sample through the data loader.

[0053] S42: Normalizing the HSI image data and the RGB image data in the training data to obtain normalized HSI image data and normalized RGB image data respectively.

[0054] Wherein, step S42 further includes: S421: Perform dimension conversion on the HSI image data to obtain dimensional HSI image data, where the dimension of the dimensional HSI image data is [channel, width, height].

[0055] S422: Flatten the dimensional HSI image data to obtain an HSI two-dimensional array; flatten the RGB image data to obtain an RGB two-dimensional array.

[0056] S423: scaling the HSI two-dimensional array according to the global maximum value of the HSI image to obtain normalized HSI image data; scaling the RGB two-dimensional array to obtain normalized RGB image data.

[0057] In steps S421 to S423, for HSI data, the dimensions of the HSI data are first converted from `[height, width, channel]` to `[channel, width, height]`, and then flattened into a two-dimensional array `[number of pixels, number of channels]`, and finally, the data is divided by the pre-calculated global maximum `hsi_max` for normalization so that its value range is between [0, 1]. For RGB data, the RGB image is flattened into a two-dimensional array `[number of pixels, number of channels]`, and then divided by a preset value of 255.0 for normalization, and the value range is also scaled to [0, 1].

[0058] The step of obtaining the global maximum value of the HSI image in step S423 further includes: S4231: Traverse all HSI image data.

[0059] S4232: Read pixel data of a single HSI image data to obtain the maximum pixel extreme value of the pixel data in the single HSI image.

[0060] S4233: Compare the pixel extreme values ​​of each HSI image data, output the maximum value of the pixel extreme values ​​of all HSI image data, and obtain the global maximum value of the HSI image.

[0061] Furthermore, in steps S4231 to S4233, firstly, each file is looped through to read the data of each hyperspectral image one by one. Secondly, for each image, the maximum pixel value in the image is found, and the maximum value of each image is compared with the currently recorded global maximum value. If the maximum value of the current image is larger, it is updated. After the traversal is completed, the last recorded global maximum value is the maximum value of all hyperspectral images.

[0062] S43: performing feature dimension splicing on the normalized HSI image data and the normalized RGB image data to obtain a feature vector.

[0063] Furthermore, after obtaining the normalized RGB data and HSI data in step S42, in step S43, the normalized HSI data and RGB data are concatenated in the feature dimension, i.e., the column direction, to form a comprehensive feature vector containing hyperspectral and RGB information.

[0064] S44: The feature vector is updated through the Adam optimizer to train the basic model for a preset number of rounds to obtain a reconstructed model.

[0065] In a specific implementation, the preset total number of training rounds is set to 1000 rounds, the learning rate is initially set to 1e-4, and decays according to the accuracy under certain conditions. If it exceeds 100, the learning rate will be multiplied by 0.98 for decay. The Adam optimizer is subsequently used, and its weight decay rate is `1e-4` to help control overfitting.

[0066] During training, each round of training uses the training set provided by the DataLoader. Each time, a set of data is taken from the training set and forward propagated. Then, the loss function is calculated and backpropagation is performed to update the model weights. In each training iteration, the model output is limited to [1e-8, 1.0-1e-8] to ensure the stability of the model output.

[0067] During the training process, the model parameters are continuously adjusted to make the model converge further. When the loss on the validation set reaches the optimal value, the current model is saved, and the reconstructed model is obtained.

[0068] The reconstruction model of the present invention mainly includes a residual block for introducing skip connections in the network to alleviate the gradient vanishing problem in deep networks, and the entire spectral mapping model, which uses multiple convolutional layers, batch normalization layers, activation functions, residual blocks and fully connected layers to process input data and generate output.

[0069] The specific network structure is as follows.

[0070] Convolutional layer: Conv1 has 3 input channels, 200 output channels, and a convolution kernel size of 3; Conv2 has 200 input channels, 200 output channels, and a convolution kernel size of 3; Conv3 has 200 input channels, 204 output channels, and a convolution kernel size of 3.

[0071] Batch normalization layer: corresponds to the number of output channels of each convolutional layer: 200, 200, 204 respectively.

[0072] Activation function: ReLU activation function is used after each convolutional layer.

[0073] Residual block, each ResBlock contains a convolution layer (the number of channels is the same as the input), a batch normalization layer and a ReLU activation function. The input is added to the output after convolution and activation in a certain proportion through a skip connection.

[0074] Fully connected layer: input dimension is 2043, output dimension is 204.

[0075] Dropout layer: Added after the residual block with a dropout rate of 0.5 to prevent overfitting.

[0076] S5: Inputting the RGB image to be reconstructed into the reconstruction model to obtain hyperspectral information for calculating hyperspectral characteristic parameters.

[0077] In step S5, firstly, the model preparation and weight loading of the reconstruction model obtained in step S4 need to be performed. Specifically, the obtained reconstruction model is instantiated and loaded into the GPU, and then the previously trained and saved model weights are loaded. After loading the trained model weights, the RGB image in the test folder is read and normalized (the pixel value is divided by 255.0). The RGB image data is converted into a format suitable for the input model, and forward propagation is performed to generate a predicted hyperspectral image. The predicted hyperspectral image is limited to a reasonable range of [1e-8, 1.0-1e-8] to avoid data overflow or unreasonable values. Finally, the output hyperspectral image is converted into a NumPy array for subsequent processing.

[0078] The hyperspectral characteristic parameters in step S5 specifically include: Conventional normalized spectral index, the expression of the conventional normalized spectral index is:

[0079] in, is the conventional normalized spectral index, is the reflectance of the hyperspectral image at a wavelength of 800nm, is the reflectance of the hyperspectral image at a wavelength of 670nm; Green normalized spectral index, the expression of the green normalized spectral index is:

[0080] in, is the green normalized spectral index, is the reflectance of the hyperspectral image at a wavelength of 560nm; Green-red vegetation index, the expression of the green-red vegetation index is:

[0081] in, is the green-red vegetation index; Red edge normalized difference vegetation index, the expression of the red edge normalized difference vegetation index is:

[0082] in, is the red edge normalized difference vegetation index, is the reflectance of the hyperspectral image at a wavelength of 715 nm; Photochemical vegetation index, the expression of the photochemical vegetation index is:

[0083] in, is the photochemical vegetation index, is the reflectance of the hyperspectral image at a wavelength of 531 nm, is the reflectance of the hyperspectral image at a wavelength of 570nm.

[0084] After reconstructing hyperspectral data from RGB images, different bands can be used to further calculate a variety of vegetation indices, which are widely used in remote sensing and agricultural research to measure plant health, crop monitoring, etc. Each index is saved as an image file after calculation for further analysis and display.

[0085] After the hyperspectral information is reconstructed, the present invention can be applied in plant research through the obtained hyperspectral information.

[0086] Used for plant physiological status monitoring: The physiological status of plants, such as chlorophyll content, water status, nutritional deficiencies, etc., will cause changes in their spectral reflectance characteristics, which are often reflected in the increase or decrease of the reflectivity of the characteristic bands. By analyzing and modeling hyperspectral data, non-destructive diagnosis and monitoring of plant physiological status can be achieved. For example, by utilizing the characteristics of nitrogen-containing compounds that absorb energy in the blue and red light bands and have high reflectivity in the green light band, a vegetation index model that is highly correlated with chlorophyll concentration can be established; by utilizing the correlation between the bands near 1450nm and 1940nm and the water content of plant leaves, a moisture index for drought stress monitoring can be developed.

[0087] Plant species identification and classification: Different species will show unique "spectral fingerprints" on hyperspectral images due to differences in their inherent biochemical composition. Automatic species identification and classification can be achieved by building a library of characteristic spectra of known species and applying pattern recognition algorithms. This technology has broad application prospects in the fields of invasive species monitoring, crop variety identification, and precision weed control.

[0088] Biomass and yield estimation: There is a good quantitative correlation between plant productivity indicators such as biomass and leaf area index and their hyperspectral characteristics. By performing correlation analysis modeling on ground measurement data and hyperspectral data, combined with data mining algorithms such as machine learning, high-precision biomass and yield estimation of large areas of crops and other vegetation can be achieved. This is crucial for the implementation of precision agriculture.

[0089] Farmland nutrition diagnosis and fertilization decision-making: Deficiencies in different nutrients such as nitrogen, phosphorus, and potassium can cause changes in related pigments or structures in plants, thereby causing abnormal changes in spectral reflectance at characteristic bands. By quantitatively establishing a spectral index model and analyzing hyperspectral data, the nutritional status of crops can be diagnosed, and precise and variable fertilization plans can be further formulated to improve agricultural production efficiency and reduce resource consumption and environmental pollution.

[0090] Crop pest and disease monitoring: When crops are attacked by pests and diseases, the structure and metabolites of their leaves will change, causing abnormal spectral reflectance. Based on hyperspectral data analysis, early detection of pests and diseases can be achieved, and specific types can be identified based on their spectral characteristics. This is critical for formulating prevention and control measures to avoid serious losses to crops.

[0091] The present invention provides a method for reconstructing hyperspectral information of RGB images based on machine learning, which reduces the cost and complexity of acquiring hyperspectral images. Traditional hyperspectral image acquisition usually requires expensive and complex equipment, while the present invention provides a method for reconstructing hyperspectral information from easily acquired RGB images, which significantly reduces the cost and complexity of acquiring hyperspectral images, thereby broadening the application scope of hyperspectral images. Moreover, through the method of machine learning, the mapping relationship between RGB images and HSI image data is learned from a large amount of RGB images and HSI image data, thereby realizing accurate reconstruction of hyperspectral information in RGB images, which not only improves the accuracy of reconstruction, but also improves efficiency through automated processing flow.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reconstructing hyperspectral information of RGB images based on machine learning, characterized in that: include: S1: Collecting RGB images and HSI images of plants, and indexing and storing the RGB images and the HSI images respectively to obtain image data; S2: performing cross-validation allocation on the image data to obtain a data set including a training set and a validation set; S3: Based on the index, obtain corresponding RGB image data and HSI image data from the data set to obtain training data; S4: establishing a data loader, loading the training data through the data loader, training the basic model, and obtaining a reconstructed model; S5: Inputting the RGB image to be reconstructed into the reconstruction model to obtain hyperspectral information for calculating hyperspectral characteristic parameters.

2. The method for reconstructing hyperspectral information of RGB images based on machine learning according to claim 1, characterized in that: The step of acquiring the RGB image in step S1 specifically includes: S11: Selecting a visible light band from the HSI image; S12: performing weighted synthesis on the visible light bands to obtain a synthetic image; S13: performing gamma conversion on the synthesized image to obtain an RGB image.

3. The method for reconstructing RGB image hyperspectral information based on machine learning according to claim 1, characterized in that: The cross-validation distribution performed on the image data in step S2 is 10 times.

4. The method for reconstructing hyperspectral information of RGB images based on machine learning according to claim 1, characterized in that: The data loader in step S4 is a DataLoader class, and the data loader is used to package the data from the training set and the validation set in the training data to obtain an iterable object.

5. The method for reconstructing RGB image hyperspectral information based on machine learning according to claim 4, characterized in that: The parameters of the DataLoader class include: `dataset`: used to specify the source of the dataset; `batch_size`: used to specify the size of data loaded at a time; `shuffle`: used to specify sequential loading, `shuffle` is set to True; `num_workers`: Used to specify the number of threads used to load data.

6. The method for reconstructing hyperspectral information of RGB images based on machine learning according to claim 1, characterized in that: Step S4 further comprises: S41: Loading training data from the same sample through the data loader; S42: normalizing the HSI image data and the RGB image data in the training data to obtain normalized HSI image data and normalized RGB image data respectively; S43: performing feature dimension splicing on the normalized HSI image data and the normalized RGB image data to obtain a feature vector; S44: The feature vector is updated through the Adam optimizer to train the basic model for a preset number of rounds to obtain a reconstructed model.

7. The method for reconstructing hyperspectral information of RGB images based on machine learning according to claim 6, characterized in that: Step S42 further includes: S421: Perform dimension conversion on the HSI image data to obtain dimensional HSI image data, where the dimension of the dimensional HSI image data is [channel, width, height]; S422: Flatten the dimensional HSI image data to obtain an HSI two-dimensional array; flatten the RGB image data to obtain an RGB two-dimensional array; S423: scaling the HSI two-dimensional array according to the global maximum value of the HSI image to obtain normalized HSI image data; scaling the RGB two-dimensional array to obtain normalized RGB image data.

8. The method for reconstructing RGB image hyperspectral information based on machine learning according to claim 7, characterized in that: The step of obtaining the global maximum value of the HSI image in step S423 further includes: S4231: traverse all HSI image data; S4232: Read pixel data of a single HSI image data to obtain the maximum pixel extreme value of the pixel data in the single HSI image; S4233: Compare the pixel extreme values ​​of each HSI image data, output the maximum value of the pixel extreme values ​​of all HSI image data, and obtain the global maximum value of the HSI image.

9. The method for reconstructing RGB image hyperspectral information based on machine learning according to claim 1, characterized in that: The loss function of the reconstruction model in step S4 is expressed as: in, is the total loss function of the reconstruction model, is the L1 loss, is the relative L1 loss; The expression of the L1 loss is: in, is the sample index value, is the total number of samples, For the The target output value of samples, For the The actual output value of samples; The expression of the relative L1 loss is: in, is the feature index value of a single sample, is the total number of features of a single sample, For the The sample in The target output value on each channel, For the The sample in The actual output value on each channel.

10. The method for reconstructing RGB image hyperspectral information based on machine learning according to claim 1, characterized in that: The hyperspectral characteristic parameters in step S5 specifically include: Conventional normalized spectral index, the expression of the conventional normalized spectral index is: in, is the conventional normalized spectral index, is the reflectance of the hyperspectral image at a wavelength of 800nm, is the reflectance of the hyperspectral image at a wavelength of 670nm; Green normalized spectral index, the expression of the green normalized spectral index is: in, is the green normalized spectral index, is the reflectance of the hyperspectral image at a wavelength of 560nm; Green-red vegetation index, the expression of the green-red vegetation index is: in, is the green-red vegetation index; Red edge normalized vegetation index, the expression of the red edge normalized vegetation index is: in, is the red edge normalized difference vegetation index, is the reflectance of the hyperspectral image at a wavelength of 715 nm; Photochemical vegetation index, the expression of the photochemical vegetation index is: in, is the photochemical vegetation index, is the reflectance of the hyperspectral image at a wavelength of 531 nm, is the reflectance of the hyperspectral image at a wavelength of 570nm.

Images