Solid-state fermented barley bran matrix feature detection method based on hyperspectral image reconstructed by RGB and application of solid-state fermented barley bran matrix feature detection method

Abstract

The invention discloses a solid-state fermented barley bran matrix characteristic detection method based on an RGB reconstructed hyperspectral image and application, 31 spectrum channels related to the water content and pH value of barley bran are screened out by using a competitive adaptive reweighting algorithm (CARS), and the RGB image of the fermented barley bran is collected to obtain the characteristic of the solid-state fermented barley bran matrix characteristic detection method based on the RGB reconstructed hyperspectral image. And an image processing technology and a deep learning algorithm are utilized to reconstruct a hyperspectral image, so that rapid and nondestructive detection of matrix characteristics such as moisture content and pH value in the solid-state fermented barley bran is realized. The method is an economical, simple, convenient and accurate solid state fermentation food agricultural product raw material nondestructive testing technology, is beneficial to development of a real-time and efficient solid state fermentation accurate control method, and is particularly suitable for being applied to the field of food quality, quality and safety control based on a solid state fermentation technology so as to optimize the fermentation process and improve the product quality; and the practical value is good.

Description

Technical Field

[0001] The invention belongs to the intersecting field of fermentation engineering and information technology, and relates to a method for detecting characteristics of a solid-state fermentation barley bran matrix, and in particular to a method for detecting characteristics of a solid-state fermentation barley bran matrix based on RGB reconstruction of a hyperspectral image. Background Art

[0002] In the field of agricultural production and food processing, monitoring of the fermentation process is a key link to ensure product quality and safety. Unlike liquid fermentation, solid-state fermentation has the characteristics of no liquid matrix, high viscosity, low moisture content and high density, which makes it difficult to quickly monitor the material matrix characteristics during the fermentation process in real time. Especially in the fermentation process of grains such as barley bran, moisture content and pH value are important factors affecting microbial growth, metabolic rate and final product quality. In addition, due to the small particle size and strong water absorption of powdery food agricultural raw materials such as grain bran, the change of moisture content during solid-state fermentation can easily affect the uniformity of the material during the fermentation process, which is a key influencing factor for solid-state fermentation efficiency and quality control. For traditional monitoring methods, such as chemical analysis and microbial counting, although they can provide accurate measurement results, these methods are often complex, costly, time-consuming, and usually involve destructive testing, and cannot achieve real-time monitoring.

[0003] In recent years, hyperspectral imaging technology has been widely used in the field of food safety and quality control due to its non-destructive and high efficiency. Hyperspectral imaging technology can capture the spatial and spectral information of fermented bran. By adopting a specific method to establish a suitable solid-state fermentation barley bran matrix characteristic evaluation model, the moisture and pH content in the fermented bran can be obtained, and a two-dimensional content distribution map of moisture and pH in a specific area can be generated. However, the practical application of this technology is limited by several key factors.

[0004] First, hyperspectral imaging equipment is usually expensive and has a slow imaging speed, which restricts its application in actual production environments. Second, the data generated by hyperspectral imaging is very high in dimensionality, and the computational complexity of processing these high-dimensional data is extremely high, which increases the difficulty and cost of data processing. In addition, the existing image reconstruction algorithm based on convolutional neural network (CNN) is insufficient in capturing the autocorrelation and long-range dependence of the spectrum, further exacerbating the challenge of data processing.

[0005] These technical problems limit the widespread application of hyperspectral imaging technology and make it impractical in real-time monitoring and large-scale production environments. Therefore, developing a new method to reduce equipment costs, simplify data processing, and improve the effectiveness of reconstruction algorithms is an urgent need for current research and application. Summary of the invention

[0006] In view of the problems of high equipment cost, complex data processing and insufficient reconstruction accuracy in the application of traditional hyperspectral imaging technology in the field of fermentation monitoring and detection, and in order to achieve the purpose of reducing equipment cost, simplifying data processing and supporting rapid detection, the present invention provides a solid-state fermentation barley bran matrix feature detection method and application based on RGB reconstruction of hyperspectral images, by collecting RGB images of fermented barley bran, and using image processing technology and deep learning algorithms to reconstruct hyperspectral images, to achieve accurate prediction of moisture content and pH value in the solid-state fermentation process. The specific technical scheme is as follows:

[0007] First, the present invention provides a method for detecting the matrix characteristics of solid-state fermentation barley bran based on RGB reconstruction hyperspectral images, comprising the following steps:

[0008] S1: Use a hyperspectral camera to collect hyperspectral data of solid-state fermented barley bran samples to be tested;

[0009] S2: The competitive adaptive reweighting algorithm is used to reduce the spectral dimension of the collected hyperspectral data of the solid-state fermented barley bran sample to 31 spectral channels as the target spectral bands;

[0010] S3: Under the experimental conditions of step S1, an industrial camera is used to collect an RGB image of the solid-state fermented barley bran sample to be tested;

[0011] S4: Reconstruction of the spectral model of fermented barley bran using a multi-level spectral Transformer model;

[0012] S5: inputting the RGB image collected in step S3 into the fermented barley bran spectral model reconstructed in step S3 to generate 31 characteristic bands of the hyperspectral image consistent with the target spectral band in step S2;

[0013] S6: Predicting the matrix characteristics of fermented barley bran based on the 31 characteristic bands of the hyperspectral image reconstructed in step S5 through a partial least squares regression model.

[0014] In the aforementioned solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image, in step S1, a hyperspectral camera collects hyperspectral data of the solid-state fermentation barley bran sample to be tested in a collection wavelength range of 616 bands in the region of 373 to 1034 nm, and the size of the collected hyperspectral image is 480×480×616 pixels.

[0015] In the above-mentioned solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image, in step S2, the 31 target spectral bands selected by the competitive adaptive reweighting algorithm are as follows: 376.51nm, 387.42nm, 453.94nm, 516.88nm, 518.96nm, 522.09nm, 638.62nm, 642.91nm, 643.98nm, 740.33nm, 741.42nm, 820.1 8nm, 827.86nm, 858.61nm, 859.71nm, 861.91nm, 881.70nm, 882.80nm, 887.20nm, 900.39nm, 905.89nm, 93 7.78nm, 939.98nm, 944.37nm, 960.86nm, 961.96nm, 966.35nm, 968.55nm, 978.43nm, 980.62nm, 982.82nm.

[0016] The aforementioned solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image, using competitive adaptive reweighting algorithm analysis process is mainly as follows:

[0017] S2.1: Using the moisture content and pH value of fermented barley bran as label values, the CARS model was run to screen out the characteristic bands related to moisture content and pH value;

[0018] S2.2: By comparing the RMSE value differences of the characteristic bands of moisture content and pH value, 31 characteristic bands were finally selected for the subsequent reconstruction model to reduce the computational complexity and maintain the prediction accuracy.

[0019] The aforementioned solid-state fermentation barley bran matrix feature detection method based on RGB reconstructed hyperspectral images adopts Monte Carlo sampling when selecting characteristic wavelengths related to moisture content and pH value. The number of sampling variables decreases rapidly when the sampling times increases from 0 to 50, and then decreases slowly and tends to be stable; 31 characteristic bands related to moisture content and 37 characteristic bands related to pH value are screened out, all of which are located near 400 and 1000 nm.

[0020] In the aforementioned solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image, in step S3, the RGB image is collected under specific lighting conditions with a color temperature of 6500K, and the collected image size is 3000×3000×3 pixels.

[0021] The aforementioned solid-state fermented barley bran matrix feature detection method based on RGB reconstruction of hyperspectral images is characterized in that: in step S4, the fermented barley bran spectral model reconstructed using the multi-level spectral Transformer model is to pair the hyperspectral image of the characteristic band selected in step S2 with the RGB image collected in step S3, and then input them as a data set into the Transformer model neural network model for training, and the hyperspectral image characteristic band of the fermented barley bran is reconstructed using the multi-level spectral Transformer model; specifically:

[0022] S4.1: The HSI-RGB data pair obtained by pairing the hyperspectral image of the characteristic band selected in step S2 and the RGB image collected in step S3 is used as a data set, and the data set is divided into a training set and a test set;

[0023] S4.2: During the training phase of the MST++ model, the pixel values ​​of the RGB image are normalized to the range of [0, 1] and the spatial dimension is downsampled to 480×480 pixels. The batch size is set to 8, and the Adam optimizer is used as the parameter optimizer. The initial learning rate is set to 0.0004, and the cosine annealing strategy is used as the learning rate decay strategy. MRAE is used as the loss function of the model, and the mean relative absolute error MRAE, root mean square error RMSE, and correlation coefficient CC are selected as evaluation indicators of the model reconstruction performance. In addition, the number of parameters Params and the number of floating-point operations Flops are selected as evaluation indicators of the computational complexity of the reconstruction model. The model is trained for a total of 300 epochs, with 1000 iterations in each epoch. The training data is enhanced by random rotation and flipping, and trained for 36 hours on a single RTX 4070Ti Suepr GPU.

[0024] The above-mentioned solid-state fermented barley bran matrix feature detection method based on RGB reconstruction of hyperspectral image is characterized in that: in step S5, the RGB image collected in step S3 is input into the fermented barley bran spectral model reconstructed in step S3, and the process of generating 31 characteristic bands of the hyperspectral image consistent with the target spectral band in step S2 is:

[0025] The RGB image collected in the input step S3 is embedded and mapped through a 3x3 convolutional layer, and trained with a hyperspectral image of H×W×31 shape as a label; the encoder part includes 1 spectral attention module and 1 convolution layer with a stride of 4x4 for downsampling and feature compression; the bottleneck part consists of a SAB module; the decoder adopts a symmetrical structure, and upsampling is performed through multiple layers of SAB and deconvolution layers with a stride of 2 to gradually restore the spatial dimensions of the feature map; finally, a multi-level spectral Transformer is constructed by cascading multiple single-level spectral Transformers to realize the conversion of RGB images to hyperspectral feature band images.

[0026] The aforementioned solid-state fermented barley bran matrix feature detection method based on RGB reconstruction of hyperspectral images is characterized in that: the process of reconstructing the fermented barley bran spectral model by the multi-level spectral Transformer model is:

[0027] S5.1, in the encoder, the feature map extracted by the embedding layer is converted into an embedding vector, and position encoding is added to retain the spatial position information; the spectral multi-head self-attention module is used to process the input features in the internal structure of the spectral attention module SAB to model the global dependencies between features; the output of the self-attention mechanism is then nonlinearly transformed through a feedforward neural network; residual connections and layer normalization are applied after each encoder sublayer to improve training stability and gradient propagation efficiency;

[0028] S5.2, the decoder input is the feature representation generated by the encoder, and the position encoding is added; the decoder structure is symmetrical with the encoder, and the SAB module interacts the encoder features with the decoder features through the attention mechanism, so that the decoder uses the encoder's context information to generate a hyperspectral image; the feedforward network performs a nonlinear transformation on each position feature;

[0029] S5.3, the final output of the decoder is mapped to each band of the hyperspectral image through a linear layer, and the softmax activation function is used to generate the final hyperspectral image pixel value.

[0030] In addition, the present invention also provides an application of the aforementioned solid-state fermentation barley bran matrix feature detection method based on RGB image reconstruction hyperspectral technology, that is, using 31 characteristic bands of the reconstructed hyperspectral image to predict the moisture content and pH value of the solid-state fermentation barley bran sample for optimizing the fermentation process of barley bran, thereby improving product quality and safety.

[0031] The beneficial effects of the present invention are:

[0032] 1) The present invention significantly reduces the equipment cost by using commonly available RGB imaging equipment instead of traditional hyperspectral imaging equipment. This makes the application of hyperspectral imaging technology more extensive, especially in resource-limited environments.

[0033] 2) The present invention uses the competitive adaptive reweighting algorithm (CARS) to perform band selection, effectively reducing the spectral dimension of the hyperspectral data from the original multiple bands to 31 characteristic bands. This process significantly reduces the dimension of the data, reduces the computational complexity and storage requirements, and thus simplifies the data processing process. This dimensionality reduction process makes the analysis of hyperspectral images more efficient and reduces processing costs.

[0034] 3) The present invention adopts a multi-level spectral Transformer model for image reconstruction, which improves the shortcomings of traditional convolutional neural networks (CNN) in capturing spectral autocorrelation and long-range dependencies. The Transformer model can better model the global dependencies between spectral features through the spectral attention mechanism (SAB) and the self-attention module (S-MSA), thereby improving the accuracy of hyperspectral image reconstruction. This improvement not only improves the accuracy of the prediction results, but also enhances the reliability of real-time monitoring and analysis.

[0035] 4) The present invention can realize real-time monitoring of key parameters in the fermentation process, such as water content and pH value, through software and hardware development and design. This provides data support for timely adjustment of fermentation conditions, which helps to optimize product quality and production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the solid-state fermentation barley bran matrix feature detection method of RGB reconstruction hyperspectral image of the present invention;

[0037] Figure 2 The CARS algorithm of the present invention is used to screen the moisture content prediction characteristic wavelength distribution (a and b) and the pH value prediction characteristic wavelength distribution (c and d) of fermented barley bran;

[0038] Figure 3 It is a schematic diagram of the band selection process of the present invention;

[0039] Figure 4 This is a schematic diagram of the overall structure of the RGB reconstruction hyperspectral network of the present invention;

[0040] Figure 5 Schematic diagram of the structure of the spectral attention module (SAB) in the reconstruction network structure of the present invention;

[0041] Figure 6 The diagrams are the moisture (a) and pH value (b) distribution diagrams of barley bran at different fermentation times of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with embodiments.

[0043] Example 1

[0044] This embodiment is a method for detecting the matrix characteristics of solid-state fermented barley bran based on RGB reconstruction of hyperspectral images. Figure 1 As shown, the following steps are included:

[0045] S1: Use a hyperspectral camera to collect hyperspectral data of solid-state fermented barley bran samples to be tested;

[0046] S2: The competitive adaptive reweighting algorithm is used to reduce the spectral dimension of the collected hyperspectral data of the solid-state fermented barley bran sample to 31 spectral channels as the target spectral bands;

[0047] S3: Under the experimental conditions of step S1, an industrial camera is used to collect an RGB image of the solid-state fermented barley bran sample to be tested;

[0048] S4: Reconstruction of the spectral model of fermented barley bran using a multi-level spectral Transformer model;

[0049] S5: inputting the RGB image collected in step S3 into the fermented barley bran spectral model reconstructed in step S3 to generate 31 characteristic bands of the hyperspectral image consistent with the target spectral band in step S2;

[0050] S6: Predicting the matrix characteristics of fermented barley bran based on the 31 characteristic bands of the hyperspectral image reconstructed in step S5 through a partial least squares regression model.

[0051] Among them, in step S1, the hyperspectral camera collects hyperspectral data of the solid-state fermented barley bran sample to be tested, specifically as follows: the acquisition wavelength range corresponding to the hyperspectral camera is 616 bands in the region of 373-1034nm (373.7858-1033.5949nm), the platform movement range is 170-240mm, and the movement speed is set to 2.85mm / s. As the platform moves, the hyperspectral image is collected, and the size of the collected hyperspectral image is 480×480×616 pixels.

[0052] In order to eliminate the influence of dark current, environmental noise and uneven distribution of light source on the original hyperspectral image (Rr), this embodiment also uses reference images with white and black backgrounds to calibrate the collected hyperspectral image. The white reference image (Rw) is obtained by a reflective white board with a reflectivity of 0.99, and the black reference image (Rd) is obtained by covering the lens. The final calibrated hyperspectral image (Rc) is calculated using the following formula:

[0053]

[0054] In step S2, the spectral dimension of the collected hyperspectral data of the solid-state fermented barley bran sample to be tested is reduced to 31 spectral channels using a competitive adaptive reweighting algorithm. The analysis process is mainly as follows:

[0055] S2.1: Using the moisture content and pH value of fermented barley bran as label values, the CARS model was run to screen out the characteristic bands related to moisture content and pH value;

[0056] S2.2: By comparing the RMSE value differences of the characteristic bands of moisture content and pH value, 31 characteristic bands were finally selected for the subsequent reconstruction model to reduce the computational complexity and maintain the prediction accuracy.

[0057] The details are as follows:

[0058] Monte Carlo sampling was used, the sampling times were set to 50, and the root mean square error of cross validation (RMSECV) was calculated through 10-fold cross validation, and the RMSECV value was used as the basis for wavelength screening. The moisture content and pH value of fermented barley bran were used as label values, and the CARS model was run to screen out the characteristic bands related to moisture content and pH value. Figure 2 As shown in , when the characteristic wavelength of moisture content is selected, the number of sampling variables decreases rapidly when the sampling times increases from 0 to 50, and then decreases slowly and tends to be stable, as shown in Figure 2 (b) In addition, as the number of Monte Carlo sampling increases, the RMSECV value first decreases and then increases, reaching the lowest point after 27 samplings (blue vertical asterisk line), corresponding to the extraction of 31 characteristic wavelengths, as shown in Figure 2 (a). Similarly, when the characteristic wavelength of pH is selected, the RMSECV value is Figure 2 (d) blue vertical asterisk line) reaches the lowest point, corresponding to the extraction of 37 characteristic wavelengths, such as Figure 2 (c) as shown.

[0059] The moisture and pH values ​​of solid-state fermented barley bran samples were predicted using 31 and 37 bands extracted, respectively. Figure 3The modeling results of the PLSR model with different numbers of spectral bands are shown in Table 1.

[0060] Table 1. Comparative analysis of prediction model evaluation indicators with different numbers of spectral channels

[0061]

[0062] From the results in Table 1, it can be seen that for the prediction of moisture content, the 31 and 37 ( The prediction accuracy and robustness of the spectral bands extracted by the CARS model (RMSEP = 0.0458) are not significantly different from those of the full-band modeling results, that is, although the 37 bands are screened out using pH as the target physicochemical value, they can also be used to predict moisture content. For pH value prediction, the 37 bands extracted by the CARS model perform well ( RMSEP=0.0823), while the pH prediction using the extracted 31 characteristic bands also showed excellent performance ( RMSEP=0.0935). Figure 2 (a) and (c) show that the characteristic wavelengths of moisture content and pH value extracted by CARS are different, but their distribution positions are roughly the same, both located near 400 and 1000 nm. That is, the performance of the models established for 31 and 37 characteristic bands extracted for moisture content and pH value is not significantly different. In order to improve the calculation efficiency of the subsequent spectral reconstruction model, 31 bands were finally selected, and their corresponding wavelengths (nm) are: 376.51, 387.42, 453.94, 516.88, 518.96, 522.09, 638.62, 642.91, 643.98, 740 .33, 741.42, 820.18, 827.86, 858.61, 859.71, 861.91, 881.70, 882.80, 887.20, 900.39, 905.89, 937.78, 939.98, 944.37, 960.86, 961.96, 966.35, 968.55, 978.43, 980.62, 982.82. These characteristic wavelengths cover the entire spectral range well and can effectively fit the data, indicating that they can represent the complete spectral information and achieve effective compression of information. The subsequent spectral reconstruction model and the training and inference process of the barley bran moisture content detection model will be based on this dataset.

[0063] In step S3, the RGB image of the solid-state fermented barley bran sample to be tested is collected by using an industrial camera, and the RGB image of the fermented barley bran is captured and collected by using an industrial camera in a light box with a color temperature of 6500K, and the image size is 3000×3000×3 pixels. The 6500K light source is considered to be white light close to natural light, and this color temperature can simulate the hue of sunlight. The solid-state numerical light intensity is used here mainly to effectively remove the influence of ambient light, eliminate interference caused by changes in ambient light, and ensure that the measurement or observation results are more accurate and reliable.

[0064] In step S4, the fermented barley bran spectral model is reconstructed using the multi-level spectral Transformer model: the hyperspectral image of the characteristic band selected in step S2 and the RGB image collected in step S3 are paired (file paired) and then input as a data set into the Transformer model neural network model for training. The specific calculation process is referred to Figure 4 and Figure 5 , a multi-level spectral Transformer model is used to reconstruct the characteristic bands of the hyperspectral image of fermented barley bran; specifically:

[0065] S4.1: The HSI-RGB data pairs obtained by sequentially pairing the hyperspectral image acquired in step S1 and the RGB image acquired in step S3 are used as a data set, and the data sets are divided into a training set and a test set;

[0066] S4.2: During the training phase of the MST++ model, the pixel values ​​of the RGB image are normalized to the range of [0, 1] and the spatial dimension is downsampled to 480×480 pixels. The batch size is set to 8, and the Adam optimizer is used as the parameter optimizer. The initial learning rate is set to 0.0004, and the cosine annealing strategy is used as the learning rate decay strategy. MRAE is used as the loss function of the model (Equation 3), and the mean relative absolute error MRAE, root mean square error RMSE (Equation 4), and correlation coefficient CC (Equation 5) are selected as the evaluation indicators of the model reconstruction performance. In addition, the number of parameters Params and the number of floating-point operations Flops are selected as the computational complexity evaluation indicators of the reconstruction model. The model is trained for a total of 300 epochs, with 1000 iterations in each epoch. The training data is augmented by random rotation and flipping. It takes about 36 hours to train the network on a single RTX 4070Ti Suepr GPU.

[0067]

[0068]

[0069]

[0070] Where:

[0071] N represents the number of pixels, X i and Represent the reflectance values ​​of the real image and the reconstructed image at the i-th pixel, μ and Represents X and The mean of .

[0072] In step S5, the RGB image collected in step S3 is input into the fermented barley bran spectral model reconstructed in step S3 to generate 31 characteristic bands of the hyperspectral image consistent with the target spectral band in step S2, specifically:

[0073] The RGB image (H×W×3) collected in the input step S3 is embedded and mapped through a 3x3 convolutional layer, and trained with a H×W×31-shaped hyperspectral image as a label; the encoder part includes a spectral attention module (SAB) layer and a convolution layer with a stride of 4x4 for downsampling and feature compression; the bottleneck part consists of a SAB module; the decoder adopts a symmetric structure, and upsampling is performed through multiple layers of SAB and deconvolution layers with a stride of 2 to gradually restore the spatial dimensions of the feature map; finally, a multi-level spectral Transformer is constructed by cascading multiple single-level spectral Transformers (SST) to realize the conversion of RGB images to hyperspectral feature band images.

[0074] In the encoder, the feature map extracted by the embedding layer is converted into an embedding vector, and position encoding is added to retain the spatial position information; the spectral multi-head self-attention module (S-MSA) is used to process the input features in the internal structure of the spectral attention module SAB to model the global dependencies between features; the output of the self-attention mechanism is then nonlinearly transformed through a feedforward neural network; residual connections and layer normalization are applied after each encoder sublayer to improve training stability and gradient propagation efficiency;

[0075] The decoder input is the feature representation generated by the encoder, and the position encoding is added; the decoder structure is symmetrical with the encoder. The SAB module interacts the encoder features with the decoder features through the attention mechanism, so that the decoder uses the encoder's context information to generate hyperspectral images; the feedforward network performs nonlinear transformation on each position feature;

[0076] The final output of the decoder is mapped to each band of the hyperspectral image through a linear layer (fully connected layer), and a softmax activation function is used to generate the final hyperspectral image pixel value.

[0077] In step S6, the matrix characteristics of the fermented barley bran are predicted based on the 31 characteristic bands of the hyperspectral image reconstructed in step S5 by using a partial least squares regression model, specifically:

[0078] The partial least squares regression model was used to predict the moisture content and pH value of barley bran based on the reconstructed hyperspectral images, and two types of fermented barley bran matrix feature detection datasets were constructed. The first dataset contained the real barley bran feature HSI and its average spectral reflectance value as input, and the corresponding moisture content and pH were used as prediction labels. The second dataset used the reconstructed barley bran HSI as spectral data to construct the fermented barley bran feature prediction dataset, and each dataset contained 150 barley bran samples. The dataset was divided into training set and test set in a ratio of 8:2.

[0079] The moisture content and pH value detection of fermented barley bran can be regarded as a regression task. The average spectral value is used as input, and PLSR, SVR and BPNN are selected as learning algorithms. The dataset is divided into training set and test set in a ratio of 8:2. The goal is to predict the moisture content and pH value of fermented barley bran. In order to intuitively demonstrate the performance of the model, the root mean square error (RMSE) and the coefficient of determination (R 2 ) as the evaluation indicator, and based on the above experimental settings, the model is trained. The experimental results are shown in Table 2.

[0080] Table 2. Quantitative analysis results of water content and pH value on real and reconstructed spectra

[0081]

[0082] The results showed that although the prediction performance of the reconstructed spectrum was lower than that of the true spectrum, it still maintained a high accuracy, which indicated that RGB-based hyperspectral reconstruction had the potential to predict the moisture content and pH value during barley bran fermentation.

[0083] Therefore, this embodiment obtains the prediction of moisture content and pH value based on the spectrum reconstructed by RGB, obtains the moisture and pH distribution of each pixel, and displays the moisture distribution of barley bran at different fermentation time points through pseudo-color images, such as Figure 6 As shown, the pH value changes linearly with the fermentation time, and the change between regions at the same time point is small, indicating that the fermentation process is stable.

[0084] Therefore, RGB reconstructed hyperspectral technology can accurately predict moisture and pH values ​​and achieve fine-grained quality monitoring. Through the visualization of pseudo-color images, the distribution and changes of moisture and pH during the fermentation process can be clearly displayed, thereby effectively identifying agglomeration and other problem areas. This method has high real-time and cost-effectiveness and has important application potential for the optimization and quality control of solid-state fermentation processes.

[0085] In general, the present invention significantly reduces the equipment cost by replacing the traditional hyperspectral imaging equipment with the commonly available RGB image equipment. This makes the application of hyperspectral imaging technology more extensive, especially in resource-limited environments. And the present invention effectively reduces the spectral dimension of the hyperspectral data from the original multiple bands to 31 characteristic bands by using the competitive adaptive reweighting algorithm (CARS) for band selection. This process significantly reduces the dimensionality of the data, reduces the computational complexity and storage requirements, and thus simplifies the data processing process. This dimensionality reduction process makes the analysis of hyperspectral images more efficient and reduces the processing cost.

[0086] In addition, the present invention adopts a multi-level spectral Transformer model for image reconstruction, which improves the shortcomings of traditional convolutional neural networks (CNN) in capturing spectral autocorrelation and long-range dependence. The Transformer model can better model the global dependencies between spectral features through the spectral attention mechanism (SAB) and the self-attention module (S-MSA), thereby improving the accuracy of hyperspectral image reconstruction. This improvement not only improves the accuracy of the prediction results, but also enhances the reliability of real-time monitoring and analysis. Therefore, based on the concept of the present invention, real-time monitoring of key parameters in the fermentation process, such as moisture content and pH value, can be achieved through software and hardware development and design. This provides data support for timely adjustment of fermentation conditions, which helps to optimize product quality and production efficiency.

[0087] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be regarded as exemplary and non-restrictive from any point of view. In addition, it should be understood that although this specification is described in accordance with the implementation mode, it does not include only one technical solution. This narrative mode of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in the embodiments can also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for detecting matrix features of solid-state fermented barley bran based on RGB reconstruction of hyperspectral images, characterized in that: The following steps are involved: S1: A hyperspectral camera is used to collect hyperspectral data of the solid-state fermented barley bran sample to be tested; S2: The competitive adaptive reweighting algorithm is used to reduce the spectral dimension of the collected hyperspectral data of the solid-state fermented barley bran sample to 31 spectral channels as the target spectral bands; S3: Under the experimental conditions of step S1, an industrial camera is used to collect an RGB image of the solid-state fermented barley bran sample to be tested; S4: Reconstruction of the spectral model of fermented barley bran using a multi-level spectral Transformer model; S5: inputting the RGB image collected in step S3 into the fermented barley bran spectral model reconstructed in step S3 to generate 31 characteristic bands of the hyperspectral image consistent with the target spectral band in step S2; S6: Predicting the matrix characteristics of fermented barley bran based on the 31 characteristic bands of the hyperspectral image reconstructed in step S5 through a partial least squares regression model.

2. The solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image according to claim 1, characterized in that: In step S1, the hyperspectral camera collects hyperspectral data of the solid-state fermented barley bran sample to be tested in a wavelength range of 616 bands in the region of 373 to 1034 nm, and the size of the collected hyperspectral image is 480×480×616 pixels.

3. The solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image according to claim 1, characterized in that: In step S2, the 31 target spectral bands selected by the competitive adaptive reweighting algorithm are as follows: 376.51nm, 387.42nm, 453.94nm, 516.88nm, 518.96nm, 522.09nm, 638.62nm, 642.91nm, 643.98nm, 740.33nm, 741.42nm, 820.18nm, 827.86nm, 858 .61nm, 859.71nm, 861.91nm, 881.70nm, 882.80nm, 887.20nm, 900.39nm, 905.89nm, 937.78nm, 9 39.98nm, 944.37nm, 960.86nm, 961.96nm, 966.35nm, 968.55nm, 978.43nm, 980.62nm, 982.82nm.

4. The solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image according to claim 3, characterized in that: The analysis process using the competitive adaptive reweighting algorithm is mainly as follows: S2.1: Using the moisture content and pH value of fermented barley bran as label values, the CARS model was run to screen out the characteristic bands related to moisture content and pH value; S2.2: By comparing the RMSE value differences of the characteristic bands of moisture content and pH value, 31 characteristic bands were finally selected for the subsequent reconstruction model to reduce the computational complexity and maintain the prediction accuracy.

5. The solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image according to claim 4, characterized in that: When selecting characteristic wavelengths related to moisture content and pH value, Monte Carlo sampling detection was used, and 31 characteristic bands related to moisture content and 37 characteristic bands related to pH value were screened out, all of which were located near 400 and 1000 nm.

6. The solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image according to claim 1, characterized in that: In step S3, the RGB image is collected under a specific lighting condition of a color temperature of 6500K, and the collected image size is 3000×3000×3 pixels.

7. The method for detecting the matrix characteristics of solid-state fermentation barley bran based on RGB reconstruction hyperspectral images according to claim 1, characterized in that: In step S4, the fermented barley bran spectral model reconstructed by using the multi-level spectral Transformer model is to pair the hyperspectral image of the characteristic band selected in step S2 with the RGB image collected in step S3, and then input them into the Transformer model neural network model as a data set for training, and reconstruct the hyperspectral image characteristic band of the fermented barley bran by using the multi-level spectral Transformer model; specifically: S4.1: The HSI-RGB data pair obtained by sequentially pairing the hyperspectral image of the characteristic band selected in step S2 and the RGB image collected in step S3 is used as a data set, and the data set is divided into a training set and a test set; S4.2: During the training phase of the MST++ model, the pixel values ​​of the RGB image are normalized to the range of [0, 1], and the spatial dimension is downsampled to 480×480 pixels; the batch size is set to 8, and the Adam optimizer is used as the parameter optimizer; the initial learning rate is set to 0.0004, and the cosine annealing strategy is used as the learning rate decay strategy; MRAE is used as the loss function of the model, and the mean relative absolute error MRAE, root mean square error RMSE, and correlation coefficient CC are selected as evaluation indicators of the model reconstruction performance. In addition, the number of parameters Params and the number of floating-point operations Flops are selected as evaluation indicators of the computational complexity of the reconstruction model; the model is trained for a total of 300 epochs, with 1000 iterations in each epoch; the training data is enhanced by random rotation and flipping, and trained for 36 hours on a single RTX 4070Ti Suepr GPU.

8. The solid-state fermentation barley bran matrix feature detection method based on RGB reconstruction hyperspectral image according to claim 1, characterized in that: In step S5, the process of inputting the RGB image collected in step S3 into the fermented barley bran spectral model reconstructed in step S3 to generate 31 characteristic bands of the hyperspectral image consistent with the target spectral band in step S2 is as follows: The RGB image collected in step S3 is input, embedded and mapped through a 3x3 convolutional layer, and trained with a H×W×31 hyperspectral image as a label; the encoder part includes a spectral attention module and a convolutional layer with a stride of 4x4 for downsampling and feature compression; the bottleneck part consists of a SAB module; The decoder adopts a symmetrical structure and performs upsampling through multiple layers of SAB and deconvolution layers with a stride of 2 to gradually restore the spatial dimensions of the feature map. Finally, a multi-level spectral Transformer is constructed by cascading multiple single-level spectral Transformers to achieve the conversion of RGB images to hyperspectral feature band images.

9. The method for detecting the matrix characteristics of solid-state fermentation barley bran based on RGB reconstruction hyperspectral images according to claim 8, characterized in that: The process of reconstructing the fermented barley bran spectral model by the multi-level spectral Transformer model is as follows: S5.1, in the encoder, the feature map extracted by the embedding layer is converted into an embedding vector, and position encoding is added to retain the spatial position information; the spectral multi-head self-attention module is used to process the input features in the internal structure of the spectral attention module SAB to model the global dependencies between features; the output of the self-attention mechanism is then nonlinearly transformed through a feedforward neural network; residual connections and layer normalization are applied after each encoder sublayer to improve training stability and gradient propagation efficiency; S5.2, the decoder input is the feature representation generated by the encoder, and the position code is added; The decoder structure is symmetrical with the encoder. The SAB module interacts the encoder features with the decoder features through the attention mechanism, so that the decoder can generate hyperspectral images using the encoder's contextual information. The feedforward network performs nonlinear transformation on each position feature; S5.3, the final output of the decoder is mapped to each band of the hyperspectral image through a linear layer, and the softmax activation function is used to generate the final hyperspectral image pixel value.

10. An application of the solid-state fermentation barley bran matrix feature detection method based on RGB image reconstruction hyperspectral technology according to any one of claims 1 to 9, characterized in that: The 31 characteristic bands of the reconstructed hyperspectral images were used to predict the moisture content and pH value of solid-state fermented barley bran samples to optimize the fermentation process of barley bran, thereby improving product quality and safety.

Images