A method and system for reconstructing hyperspectral data and predicting the internal quality of fruit.

Abstract

This application belongs to the field of nondestructive testing technology and discloses a method and system for reconstructing hyperspectral data and predicting the internal quality of fruit. The method includes: pre-training a spectral autoencoder to extract a nonlinear, high-information-content latent space feature from real hyperspectral data as the learning target for spectral reconstruction, and obtaining a fixed spectral decoder; constructing a multi-task deep learning master model including a shared encoder, a spectral reconstruction head, and a quality prediction head. This model takes RGB three-band values ​​as input and simultaneously optimizes the spectral reconstruction task and the internal quality prediction task through a joint loss function; only RGB values ​​need to be input to simultaneously obtain the predicted internal quality and the hyperspectral data reconstructed from the predicted latent space feature output by the spectral reconstruction head through the fixed spectral decoder. This invention avoids error accumulation and introduces chemical constraints through multi-task learning, improving the accuracy of spectral reconstruction and quality prediction.

Description

Technical Field

[0001] This invention relates to the field of non-destructive testing technology in computer vision, and in particular to a method and system for reconstructing hyperspectral data and predicting the internal quality of fruit based on spectral autoencoders and multi-task learning. Background Technology

[0002] Hyperspectral imaging technology can simultaneously acquire rich spatial and spectral information of the object being measured, and has broad application prospects in the field of non-destructive testing of agricultural products, such as assessing the sugar content, acidity, and ripeness of fruits. However, the high cost of hyperspectral cameras, slow data acquisition speed, and complex data processing procedures limit their widespread adoption and application in consumer-grade devices or large-scale automated sorting production lines.

[0003] In contrast, standard RGB cameras or sensors offer advantages such as low cost, fast response time, and small size, and are widely used in various devices such as smartphones and industrial cameras. Therefore, reconstructing complete hyperspectral data from readily available RGB three-band values ​​using algorithms has become a research direction with significant application value.

[0004] However, existing technical solutions suffer from the following drawbacks in achieving this goal: First, many traditional methods employ a two-stage sequential approach. The first step uses a model to reconstruct hyperspectral data from RGB values; the second step uses the reconstructed hyperspectral data to predict fruit quality using a chemometric model (such as Partial Least Squares (PLS) or Support Vector Machine (SVM)). This "reconstruct first, then predict" approach means that errors generated in the first step of spectral reconstruction are directly carried over and potentially amplified in the second step of quality prediction, severely impacting the accuracy of the final quality prediction. Second, most existing RGB-to-hyperspectral reconstruction models typically optimize by minimizing the overall statistical error between the reconstructed and true spectra. This approach lacks effective constraints on the spectral chemical meaning. While the model can learn the overall shape of the spectral curve, it often fails to guarantee high-precision reconstruction in key absorption peak regions closely related to the fruit's internal chemical components (such as sugar and water). This results in reconstructed spectra that are "similar in form but not in essence," leading to poor performance in quantitative analysis of chemical components. Furthermore, some techniques employ linear methods such as Principal Component Analysis (PCA) to reduce the dimensionality of high-dimensional hyperspectral data in order to simplify spectral reconstruction. The model is then trained to reconstruct the principal components after dimensionality reduction. However, PCA, as a linear dimensionality reduction method, may not effectively capture complex nonlinear structures such as absorption peak changes caused by the nonlinear superposition of multiple chemical components. This results in insufficient information representation capability of the learning target itself, fundamentally limiting the upper limit of the accuracy of the final reconstructed spectrum.

[0005] Therefore, there is an urgent need for a new technical solution to address the problems of error accumulation caused by traditional two-stage methods, low accuracy of key bands in traditional spectral reconstruction methods due to the lack of chemical constraints, and loss of spectral nonlinear features caused by the use of linear dimensionality reduction as a learning objective in traditional methods. The solution should be to achieve synchronous, efficient, and accurate output of reconstructed hyperspectral data and key internal quality indicators using a single RGB three-band numerical input. Summary of the Invention

[0006] The main objective of this invention is to provide a method and system for reconstructing hyperspectral data and predicting the internal quality of fruit based on spectral autoencoders and multi-task learning. This aims to solve the problems of error accumulation, lack of chemical constraints, and limitations of linear targets in the background technology, and to achieve end-to-end, high-precision synchronous prediction from RGB three-band values ​​to hyperspectral data and internal quality.

[0007] In a first aspect, the present invention provides a method for reconstructing hyperspectral data and predicting the internal quality of fruit, comprising the following steps:

[0008] Step 1: Obtain the training dataset, which includes RGB three-band values, hyperspectral data, and internal quality true values ​​that correspond one-to-one with the fruit sample points;

[0009] Step 2: Pre-train a spectral autoencoder to generate nonlinear target features and a spectral decoder; the spectral autoencoder includes a spectral encoder and a spectral decoder; using hyperspectral data from the training dataset as input, the spectral autoencoder is trained by minimizing the reconstruction error between the reconstructed spectrum and the original hyperspectral data to obtain the trained spectral encoder and the trained spectral decoder; using the trained spectral encoder, the hyperspectral data from the training dataset is input to generate nonlinear latent space features that serve as the learning target for the spectral reconstruction task;

[0010] Step 3: Construct and train a multi-task deep learning master model, which includes a shared encoder, a spectral reconstruction head connected to the shared encoder, and a quality prediction head connected to the shared encoder; input the RGB three-band values ​​from the training dataset into the shared encoder to extract shared latent space feature vectors; input the shared latent space feature vectors into the spectral reconstruction head and the quality prediction head respectively; train the master model using a joint loss function, which includes: a spectral reconstruction loss to measure the difference between the predicted nonlinear features output by the spectral reconstruction head and the nonlinear latent space features, and a quality prediction loss to measure the difference between the predicted internal quality output by the quality prediction head and the true internal quality value;

[0011] Step 4: Perform synchronous prediction based on the trained master model and the trained spectral decoder; input the RGB three-band values ​​of the fruit sample points to be tested into the trained master model, and directly output the predicted internal quality through the quality prediction head; at the same time, input the predicted nonlinear features output by the spectral reconstruction head into the trained spectral decoder, and output the reconstructed hyperspectral data of the fruit by the spectral decoder.

[0012] As an optional implementation of the first aspect of this application, in step two, the spectral autoencoder includes a spectral encoder and a spectral decoder, wherein: the network structure of the spectral encoder is: the input layer dimension is the original number of bands M; after passing through at least P fully connected hidden layers, the output is a nonlinear latent space feature with a dimension of k; the network structure of the spectral decoder is: the input layer dimension is k; after passing through at least Q fully connected hidden layers, the reconstructed output dimension is M; wherein M is a preset number of original bands, k is a preset dimension less than M, and P and Q are integers greater than or equal to 1.

[0013] As an optional implementation of the first aspect of this application, the spectral autoencoder is constructed based on a multilayer perceptron; the network structure of the spectral encoder is: an input layer of dimension M, followed by a first fully connected layer with an output dimension N1; then a second fully connected layer with an output dimension N2; then a third fully connected layer, outputting a nonlinear latent space feature dimension K, wherein the dimensions satisfy a layer-by-layer decreasing relationship of M > N1 > N2 > K; the network structure of the spectral decoder is: an input layer of dimension K, followed by a first fully connected layer with an output dimension N2; then a second fully connected layer with an output dimension N1; then a third fully connected layer with an output dimension M; wherein the dimensions satisfy K < N2 < N1 < M; the fully connected layers in the spectral encoder and spectral decoder use the ReLU activation function.

[0014] As an optional implementation of the first aspect of this application, the specific structure of the multi-task deep learning master model is as follows: the shared encoder is a multilayer perceptron or a fully connected network, which receives 3D RGB three-band values ​​as input and outputs a high-dimensional shared latent space feature vector; the spectral reconstruction head is another independent multilayer perceptron, which receives the shared latent space feature vector as input and outputs predicted nonlinear features; the quality prediction head is another independent multilayer perceptron, which receives the shared latent space feature vector as input and outputs predicted internal quality scalar values.

[0015] As an optional implementation of the first aspect of this application, the specific network hierarchy structure of the multi-task deep learning master model is as follows: the network structure of the shared encoder is: the input layer has a dimension of 3 RGB channels, followed by a first fully connected layer, and the output dimension is... Then, a second fully connected layer is added, with an output dimension of... Then, a third fully connected layer is added, with the output dimension being... To generate The shared latent space feature vector of dimension; wherein, dimension satisfies The hierarchical increasing relationship; the network structure of the spectral reconstruction head is: the input layer dimension is Following this is the first fully connected layer, with an output dimension of... Then, a second fully connected layer is added, with an output dimension of... To generate The predicted nonlinear features of dimension; wherein, dimension satisfies The quality prediction head exhibits a progressively decreasing relationship; its network structure is as follows: the input layer dimension is... Following this is the first fully connected layer, with an output dimension of... Then, a second fully connected layer is added, with an output dimension of... , usually 1; where the dimension satisfies The layer-by-layer decreasing relationship; the ReLU activation function is used between the fully connected layers in the shared encoder, spectral reconstruction head and quality prediction head.

[0016] As an optional implementation of the first aspect of this application, in step three, the joint loss function is defined as the weighted sum of the spectral reconstruction loss and the quality prediction loss: , where λ is a hyperparameter used to balance the two loss weights; The spectral reconstruction loss represents the mean square error between the predicted nonlinear feature and the nonlinear latent space feature. The quality prediction loss is the mean square error between the predicted internal quality and the true value of the internal quality.

[0017] As an optional implementation of the first aspect of this application, in step three, when training the main model, the Adam optimizer is used to update the parameters; and during the training of the main model, all parameters of the spectral autoencoder remain fixed.

[0018] Secondly, embodiments of this application provide a system for reconstructing hyperspectral data and predicting the internal quality of fruit, including:

[0019] The data acquisition module is used to acquire the training dataset, which includes RGB three-band values, hyperspectral data and internal quality true values ​​that correspond one-to-one with the fruit sample points.

[0020] A spectral autoencoder pre-training module is used to pre-train a spectral autoencoder to generate nonlinear target features and a spectral decoder. The spectral autoencoder includes a spectral encoder and a spectral decoder. Using hyperspectral data in the training dataset as input, the spectral autoencoder is trained by minimizing the reconstruction error between the reconstructed spectrum and the original hyperspectral data to obtain the trained spectral encoder and the trained spectral decoder. Using the trained spectral encoder, the hyperspectral data in the training dataset is input to generate nonlinear latent space features that serve as the learning target for the spectral reconstruction task.

[0021] The main model construction and training module is used to construct and train a multi-task deep learning main model. The main model includes a shared encoder, a spectral reconstruction head connected to the shared encoder, and a quality prediction head connected to the shared encoder. The RGB three-band values ​​from the training dataset are input to the shared encoder to extract a shared latent space feature vector. The shared latent space feature vector is then input to the spectral reconstruction head and the quality prediction head, respectively. The main model is trained using a joint loss function, which includes: a spectral reconstruction loss to measure the difference between the predicted nonlinear features output by the spectral reconstruction head and the nonlinear latent space features, and a quality prediction loss to measure the difference between the predicted internal quality output by the quality prediction head and the true internal quality value.

[0022] The synchronous prediction module is used to perform synchronous prediction based on the trained master model and the trained spectral decoder; the RGB three-band values ​​of the fruit sample points to be tested are input into the trained master model, and the predicted internal quality is directly output through the quality prediction head; at the same time, the predicted nonlinear features output by the spectral reconstruction head are input into the trained spectral decoder, and the spectral decoder outputs the reconstructed hyperspectral data of the fruit.

[0023] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0024] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. Superior Nonlinear Feature Representation: This invention innovatively uses a pre-trained nonlinear spectral autoencoder to define the learning target of the spectrum, replacing the traditional linear PCA. Compared with PC scores of several dimensions, the k-dimensional nonlinear latent space features extracted by this autoencoder can more compactly, efficiently, and accurately represent the complex nonlinear structure in the 176-dimensional spectrum, providing a higher quality target for subsequent learning.

[0027] 2. Higher restoration accuracy: Due to the higher quality and richer information of the learning target (k-dimensional nonlinear features) of the main model, the main model can learn a more accurate mapping relationship from RGB to spectral features. This fundamentally improves the upper limit of spectral restoration accuracy, especially in absorption peak regions with complex nonlinear relationships determined by multiple chemical components.

[0028] 3. Accuracy Enhancement and Chemical Constraints: By introducing a quality prediction task (such as sugar content) and sharing a feature extraction layer with the spectral reconstruction task, the shared encoder is forced to learn the deep nonlinear mapping relationship between the RGB three-band values ​​and the sugar content absorption peak (located in a specific hyperspectral band). This "chemical prior" constraint from the quality prediction task indirectly guides the spectral reconstruction process to pay more attention to key bands useful for quality prediction during the optimization process, making the reconstructed spectrum more accurate in these chemical absorption peak regions, truly achieving both "form and spirit" accuracy.

[0029] 4. Avoiding Error Accumulation: This invention employs an end-to-end multi-task learning framework where the two tasks (spectral reconstruction and quality prediction) share a feature extraction layer and are jointly optimized. The accuracy of quality prediction, in turn, improves the accuracy of spectral reconstruction, and vice versa. This collaborative optimization mechanism completely avoids the problem of reconstruction errors propagating to and accumulating in the prediction task, as seen in traditional two-stage methods, thus improving the overall performance and robustness of the system.

[0030] 5. High efficiency and synchronization: Requiring only a low-cost RGB sensor and a trained model, 176-band hyperspectral data and sugar content predictions can be obtained "instantly" through a single forward computation. This greatly improves detection efficiency, significantly reduces hardware costs, and paves the way for the widespread application of hyperspectral technology. Attached Figure Description

[0031] Figure 1 This is a flowchart of a method for reconstructing hyperspectral data and predicting the internal quality of fruit according to an embodiment of the present invention;

[0032] Figure 2This is a schematic diagram of the structure of a system for reconstructing hyperspectral data and predicting the internal quality of fruit according to an embodiment of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0035] Example 1

[0036] Please see Figure 1 This is a flowchart illustrating a method for reconstructing hyperspectral data and predicting the internal quality of fruit, provided by an embodiment of the present invention. The method may include the following steps:

[0037] Step 1: Obtain the training dataset, which includes RGB three-band values, hyperspectral data, and internal quality true values ​​that correspond one-to-one with the fruit sample points.

[0038] It should be noted that in this embodiment, RGB refers to the three spectral channels of red, green, and blue; the RGB three-band values ​​specifically refer to the vector composed of the reflectance values ​​of one characteristic band selected from the wavelength ranges corresponding to the red, green, and blue channels, respectively.

[0039] Specifically, images of the fruit sample surface were acquired using an industrial RGB camera, while spectral data of 176 bands (e.g., the 400-1000 nm range) at the same measurement point were acquired using a hyperspectral analyzer such as ASD FieldSpec. The average RGB three-band values ​​of the corresponding region at the hyperspectral measurement point were extracted from the RGB images. Necessary preprocessing, such as black-and-white correction and Savitzky-Golay smoothing, was performed on the acquired raw hyperspectral data. Immediately after the non-destructive measurement, a handheld digital saccharimeter was used to sample the soluble solids content (Brix value) at the measurement point, which was then used as the true internal quality value. A total of 2000 pairs of data (RGB three-band values, 176-band spectrum, saccharimeter Brix value) were collected for subsequent model training and validation.

[0040] Step 2: Pre-train a spectral autoencoder to generate nonlinear target features and a spectral decoder; the spectral autoencoder includes a spectral encoder and a spectral decoder; using hyperspectral data in the training dataset as input, the spectral autoencoder is trained by minimizing the reconstruction error between the reconstructed spectrum and the original hyperspectral data to obtain the trained spectral encoder and the trained spectral decoder; using the trained spectral encoder, the hyperspectral data in the training dataset is input to generate nonlinear latent space features that serve as the learning target for the spectral reconstruction task.

[0041] Specifically, 176 spectral bands are extracted from all training samples to form a spectral matrix. (sample size) (band); and constructing a spectral autoencoder (SpectrumAE): In this example, a multilayer perceptron (MLP) based autoencoder is used, which includes a "spectral encoder" and a "spectral decoder".

[0042] The spectral encoder has the following network structure: Input layer (176-dimensional) -> Fully connected layer (128-dimensional output) -> ReLU activation function -> Fully connected layer (64-dimensional output) -> ReLU activation function -> Fully connected layer (12-dimensional output). The final output is a 12-dimensional nonlinear latent space feature (i.e., k=12).

[0043] The spectral decoder has the following network structure: Input layer (12-dimensional) -> Fully connected layer (64-dimensional output) -> ReLU activation function -> Fully connected layer (128-dimensional output) -> ReLU activation function -> Fully connected layer (176-dimensional output). The final output is a 176-dimensional reconstructed spectrum.

[0044] Training a spectral autoencoder: converting the spectral matrix The autoencoder is trained using mean squared error (MSE) as input and as a loss function to optimize the target. The process continues until the MSE loss between the reconstructed spectrum and the original spectrum is minimized and convergence occurs.

[0045]

[0046] in The number of training samples. This represents the number of spectral bands (e.g., 176). For the first The sample at the th The true spectral values ​​of each band For the first The sample at the th The reconstructed spectral values ​​of each band, and .

[0047] Next, a nonlinear target is generated: after training, all parameters of the spectral encoder are fixed. The spectral matrix is ​​input into the trained spectral encoder to obtain... The "nonlinear latent space features" of dimensionality ( ). This will serve as the "learned ground truth" (Target) for the "spectral reconstruction task" in the main model of this invention, replacing the traditional PCA method. Principal component scores.

[0048] Finally, save the decoder: save the trained, fixed-parameter spectral decoder for use in the subsequent inference phase of the main model.

[0049] Step 3: Construct and train a multi-task deep learning master model, which includes a shared encoder, a spectral reconstruction head connected to the shared encoder, and a quality prediction head connected to the shared encoder; input the RGB three-band values ​​from the training dataset into the shared encoder to extract shared latent space feature vectors; input the shared latent space feature vectors into the spectral reconstruction head and the quality prediction head respectively; train the master model using a joint loss function, which includes: a spectral reconstruction loss to measure the difference between the predicted nonlinear features output by the spectral reconstruction head and the nonlinear latent space features, and a quality prediction loss to measure the difference between the predicted internal quality output by the quality prediction head and the true internal quality value.

[0050] First, construct the main model:

[0051] ① Shared Encoder: Employs a multilayer perceptron (MLP) or fully connected network (FCN) structure to extract nonlinear features from the RGB three-band values ​​and map them into a high-dimensional "shared latent space feature vector". The input is a 3-dimensional RGB vector.

[0052] An example of its network structure is: fully connected layer (3 -> 64) -> ReLU activation -> fully connected layer (64 -> 128) -> ReLU activation -> fully connected layer (128 -> 512). The final output is a 512-dimensional shared latent space feature vector.

[0053] ② Spectrum Head: Employs a multilayer perceptron (MLP) that receives the shared latent space feature vectors mentioned above. The network structure is: fully connected layer (512->256) -> ReLU activation -> fully connected layer (256->12). The final output is a... A dimensional vector representing the predicted A “nonlinear latent space feature”.

[0054] ③ Quality Head: A separate MLP is used, which receives the same 512-dimensional shared latent space feature vector. The network structure is: fully connected layer (512->128) -> ReLU activation -> fully connected layer (128->1). The final output is a scalar value representing the predicted internal quality index of the fruit (such as "predicted Brix sugar content").

[0055] It should be noted that the output of the "spectral reconstruction head" The “predicted nonlinear features” are input into the pre-trained, fixed “spectral decoder” in step two. The output of the “spectral decoder” is the reconstructed (decoded) complete 176-band predicted spectrum.

[0056] Next, the main model is trained:

[0057] Set a joint loss function to optimize both tasks simultaneously. Joint Loss Defined as spectral reconstruction loss And quality prediction loss Weighted sum:

[0058]

[0059] in, It is a hyperparameter weight used to balance the importance of the two tasks.

[0060] Spectral reconstruction loss Defined as the "prediction" output of the "spectral reconstruction head" "Nonlinear characteristics" and the "real" generated in step two Nonlinear characteristics ( ) Mean squared error (MSE) between )

[0061]

[0062] in This refers to the batch size or the total number of samples. Dimension of the nonlinear latent space features; For the first The first sample Dimensional “true nonlinear features” (from SpectrumEncoder); For the first The first sample The dimension is "predicting nonlinear features" (from the spectral reconstruction head of the master model).

[0063] Quality prediction loss Mean squared error (MSE) is defined as the difference between the predicted sugar content output by the quality prediction head and the actual sugar content collected from the sample.

[0064]

[0065] in For the first The "true sugar content" value of each sample; For the first The "predicted sugar content" value for each sample (from the quality prediction head of the main model).

[0066] It should be noted that the Adam optimizer is used, with a learning rate of 1e-4, and the backpropagation algorithm is employed. Simultaneously, all trainable parameters of the main model (i.e., the shared encoder, spectral reconstruction head, and quality prediction head) are updated, and end-to-end training is performed until the total loss converges on the validation set. During this process, the parameters of the spectral autoencoder (SpectrumAE) remain fixed during the main model training phase.

[0067] Step 4: Perform synchronous prediction based on the trained master model and the trained spectral decoder; input the RGB three-band values ​​of the fruit sample points to be tested into the trained master model, and directly output the predicted internal quality through the quality prediction head; at the same time, input the predicted nonlinear features output by the spectral reconstruction head into the trained spectral decoder, and output the reconstructed hyperspectral data of the fruit by the spectral decoder.

[0068] Specifically, after the main model is trained, for a brand new fruit sample to be tested, the RGB three-band values ​​of its surface are first collected. This 3D RGB vector is then input into the trained main model for one forward propagation.

[0069] Finally, the synchronous outputs are as follows: 1. The quality prediction head outputs the predicted sugar content of the apple. 2. The spectral reconstruction head outputs a 12-dimensional predicted nonlinear feature vector, which is then input into a saved, fixed spectral decoder. The output of the spectral decoder is the reconstructed 176-band hyperspectral data.

[0070] Example 2

[0071] Please see Figure 2 The diagram shown is a structural schematic of a system for reconstructing hyperspectral data and predicting the internal quality of fruit, as proposed in the second embodiment of this application. The system includes the following key modules:

[0072] The data acquisition module 100 is used to acquire a training dataset, which includes RGB three-band values, hyperspectral data and internal quality true values ​​that correspond one-to-one with the fruit sample points.

[0073] A spectral autoencoder pre-training module 200 is used to pre-train a spectral autoencoder to generate nonlinear target features and a spectral decoder. The spectral autoencoder includes a spectral encoder and a spectral decoder. Using hyperspectral data in the training dataset as input, the spectral autoencoder is trained by minimizing the reconstruction error between the reconstructed spectrum and the original hyperspectral data to obtain the trained spectral encoder and the trained spectral decoder. Using the trained spectral encoder, the hyperspectral data in the training dataset is input to generate nonlinear latent space features that serve as the learning target for the spectral reconstruction task.

[0074] The main model construction and training module 300 is used to construct and train a multi-task deep learning main model. The main model includes a shared encoder, a spectral reconstruction head connected to the shared encoder, and a quality prediction head connected to the shared encoder. The RGB three-band values ​​from the training dataset are input to the shared encoder to extract a shared latent space feature vector. The shared latent space feature vector is then input to the spectral reconstruction head and the quality prediction head, respectively. The main model is trained using a joint loss function, which includes: a spectral reconstruction loss to measure the difference between the predicted nonlinear features output by the spectral reconstruction head and the nonlinear latent space features, and a quality prediction loss to measure the difference between the predicted internal quality output by the quality prediction head and the true internal quality value.

[0075] The synchronous prediction module 400 is used to perform synchronous prediction based on the trained master model and the trained spectral decoder; the RGB three-band values ​​of the fruit sample points to be tested are input into the trained master model, and the predicted internal quality is directly output through the quality prediction head; at the same time, the predicted nonlinear features output by the spectral reconstruction head are input into the trained spectral decoder, and the spectral decoder outputs the reconstructed hyperspectral data of the fruit.

[0076] The system for reconstructing hyperspectral data and predicting the internal quality of fruit in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc. This application embodiment does not specifically limit the scope of the system.

[0077] The system for reconstructing hyperspectral data and predicting the internal quality of fruit in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0078] The system provided in this application for reconstructing hyperspectral data and predicting the internal quality of fruit can achieve... Figure 1 The various processes implemented in the method embodiment for reconstructing hyperspectral data and predicting the internal quality of fruit are not described in detail here to avoid repetition.

[0079] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described method embodiment for reconstructing hyperspectral data and predicting the internal quality of fruit, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0080] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described method for reconstructing hyperspectral data and predicting the internal quality of fruit, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0081] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0082] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0084] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for reconstructing hyperspectral data and predicting the internal quality of fruit, characterized in that, Includes the following steps: Step 1: Obtain the training dataset, which includes RGB three-band values, hyperspectral data, and internal quality true values ​​that correspond one-to-one with the fruit sample points; Step 2: Pre-train a spectral autoencoder to generate nonlinear target features and a spectral decoder; the spectral autoencoder includes a spectral encoder and a spectral decoder; using hyperspectral data from the training dataset as input, the spectral autoencoder is trained by minimizing the reconstruction error between the reconstructed spectrum and the original hyperspectral data to obtain the trained spectral encoder and the trained spectral decoder; using the trained spectral encoder, the hyperspectral data from the training dataset is input to generate nonlinear latent space features that serve as the learning target for the spectral reconstruction task; Step 3: Construct and train a multi-task deep learning master model, which includes a shared encoder, a spectral reconstruction head connected to the shared encoder, and a quality prediction head connected to the shared encoder; input the RGB three-band values ​​from the training dataset into the shared encoder to extract shared latent space feature vectors; input the shared latent space feature vectors into the spectral reconstruction head and the quality prediction head respectively; train the master model using a joint loss function, which includes: a spectral reconstruction loss to measure the difference between the predicted nonlinear features output by the spectral reconstruction head and the nonlinear latent space features, and a quality prediction loss to measure the difference between the predicted internal quality output by the quality prediction head and the true internal quality value; Step 4: Perform synchronous prediction based on the trained master model and the trained spectral decoder; The RGB three-band values ​​of the fruit sample points to be tested are input into the trained main model, and the predicted internal quality is directly output through the quality prediction head; at the same time, the predicted nonlinear features output by the spectral reconstruction head are input into the trained spectral decoder, and the spectral decoder outputs the reconstructed hyperspectral data of the fruit.

2. The method according to claim 1, characterized in that, In step two, the spectral autoencoder includes a spectral encoder and a spectral decoder, wherein: The network structure of the spectral encoder is as follows: the input layer dimension is the original number of bands M; after passing through at least P fully connected hidden layers, the output dimension is a nonlinear latent space feature of dimension k; The network structure of the spectral decoder is as follows: the input layer has a dimension of k; after passing through at least Q fully connected hidden layers, the reconstructed output has a dimension of M. Where M is the preset number of original bands, k is the preset dimension less than M, and P and Q are integers greater than or equal to 1.

3. The method according to claim 2, characterized in that, The spectral autoencoder is based on a multilayer perceptron. The network structure of the spectral encoder is as follows: the input layer has a dimension of M, followed by a first fully connected layer with an output dimension of N1; then a second fully connected layer with an output dimension of N2; then a third fully connected layer with an output dimension of K for nonlinear latent space features, where the dimensions satisfy a decreasing relationship of M > N1 > N2 > K. The network structure of the spectral decoder is as follows: the input layer has a dimension of K, followed by a first fully connected layer with an output dimension of N2; then a second fully connected layer with an output dimension of N1; then a third fully connected layer with an output dimension of M; wherein the dimensions satisfy K < N2 < N1 < M; The ReLU activation function is used between the fully connected layers in the spectral encoder and spectral decoder.

4. The method according to claim 1, characterized in that, In step three, the specific structure of the multi-task deep learning master model is as follows: The shared encoder is a multilayer perceptron or a fully connected network that receives 3D RGB three-band values ​​as input and outputs a high-dimensional shared latent space feature vector. The spectral reconstruction head is another independent multilayer perceptron that receives the shared latent space feature vector as input and outputs the predicted nonlinear features. The quality prediction head is another independent multilayer perceptron that receives the shared latent space feature vector as input and outputs the predicted internal quality scalar value.

5. The method according to claim 4, characterized in that, The specific network hierarchy structure of the multi-task deep learning master model is as follows: The network structure of the shared encoder is as follows: the input layer has 3 RGB channels, followed by a first fully connected layer, and the output layer has... Then, a second fully connected layer is added, with an output dimension of... Then, a third fully connected layer is added, with the output dimension being... To generate The shared latent space feature vector of dimension; wherein, dimension satisfies A hierarchical increasing relationship; The network structure of the spectral reconstruction head is as follows: the input layer dimension is... Following this is the first fully connected layer, with an output dimension of... Then, a second fully connected layer is added, with an output dimension of... To generate The predicted nonlinear features of dimension; wherein, dimension satisfies The relationship of decreasing order at each level; The network structure of the quality prediction head is as follows: the input layer dimension is... Following this is the first fully connected layer, with an output dimension of... Then, a second fully connected layer is added, with an output dimension of... , usually 1; where the dimension satisfies The relationship of decreasing order at each level; The ReLU activation function is used between the fully connected layers in the shared encoder, spectral reconstruction head, and quality prediction head.

6. The method according to claim 1, characterized in that, In step three, the joint loss function is defined as the weighted sum of the spectral reconstruction loss and the quality prediction loss: Where λ is a hyperparameter used to balance the two loss weights; The spectral reconstruction loss represents the mean square error between the predicted nonlinear feature and the nonlinear latent space feature. The quality prediction loss is the mean square error between the predicted internal quality and the true value of the internal quality.

7. The method according to any one of claims 1, 3, or 6, characterized in that, In step three, when training the main model, the Adam optimizer is used to update the parameters; and during the training of the main model, all parameters of the spectral autoencoder remain fixed.

8. A system for reconstructing hyperspectral data and predicting the internal quality of fruit, characterized in that, include: The data acquisition module is used to acquire the training dataset, which includes RGB three-band values, hyperspectral data and internal quality true values ​​that correspond one-to-one with the fruit sample points. A spectral autoencoder pre-training module is used to pre-train a spectral autoencoder to generate nonlinear target features and a spectral decoder. The spectral autoencoder includes a spectral encoder and a spectral decoder. Using hyperspectral data in the training dataset as input, the spectral autoencoder is trained by minimizing the reconstruction error between the reconstructed spectrum and the original hyperspectral data to obtain the trained spectral encoder and the trained spectral decoder. Using the trained spectral encoder, the hyperspectral data in the training dataset is input to generate nonlinear latent space features that serve as the learning target for the spectral reconstruction task. The main model construction and training module is used to construct and train a multi-task deep learning main model. The main model includes a shared encoder, a spectral reconstruction head connected to the shared encoder, and a quality prediction head connected to the shared encoder. The RGB three-band values ​​from the training dataset are input to the shared encoder to extract a shared latent space feature vector. The shared latent space feature vector is then input to the spectral reconstruction head and the quality prediction head, respectively. The main model is trained using a joint loss function, which includes: a spectral reconstruction loss to measure the difference between the predicted nonlinear features output by the spectral reconstruction head and the nonlinear latent space features, and a quality prediction loss to measure the difference between the predicted internal quality output by the quality prediction head and the true internal quality value. The synchronous prediction module is used to perform synchronous prediction based on the trained master model and the trained spectral decoder; the RGB three-band values ​​of the fruit sample points to be tested are input into the trained master model, and the predicted internal quality is directly output through the quality prediction head; at the same time, the predicted nonlinear features output by the spectral reconstruction head are input into the trained spectral decoder, and the spectral decoder outputs the reconstructed hyperspectral data of the fruit.

9. An electronic device, characterized in that, The method includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of a method for reconstructing hyperspectral data and predicting the internal quality of fruit as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of a method for reconstructing hyperspectral data and predicting the internal quality of fruit as described in any one of claims 1-7.

Images