Nondestructive testing method for internal quality and maturity of strawberries based on MPCD-Mmba model

Abstract

The invention discloses a strawberry internal quality and maturity nondestructive testing method based on an MPCD-Mama model. The strawberry internal quality and maturity nondestructive testing method comprises the following steps: acquiring a strawberry sample, acquiring a hyperspectral image by using a hyperspectral reflection imaging system, measuring SSC, PH and hardness, and marking maturity; correcting the hyperspectral image, performing spectrum extraction to form a data set, and dividing the data set into a training set, a test set and a verification set; the method comprises the following steps: constructing an MPCD-Mama model, training the MPCD-Mama model by using a training set, testing the performance of the MPCD-Mama model by using a verification set on the basis of a cross validation method, predicting related indexes on the basis of the pre-trained MPCD-Mama model, generating heat maps of corresponding components at different positions of strawberries, and displaying the distribution conditions of the different components. According to the method, nondestructive testing of multiple quality parameters of strawberries is achieved, compared with a traditional method, the prediction precision is remarkably improved, and technical support is provided for nondestructive testing of the quality of the strawberries.

Description

Technical Field

[0001] The invention relates to the technical field of image classification, in particular to a nondestructive detection method for internal quality and maturity of strawberries based on an MPCD-Mamba model. Background Art

[0002] Strawberry is a popular fruit worldwide, favored by consumers for its unique aroma, sweetness and rich nutritional value. Quality assessment and maturity detection of strawberries are crucial for harvesting, trade and consumption. Common quality indicators of strawberries include soluble solids content (SSC), pH, hardness and maturity. Over-mature strawberries are not only difficult to transport, but also have low nutritional value. These factors not only affect consumer acceptance, but also determine the shelf life and storage requirements of strawberries. Therefore, the agricultural and food industries urgently need rapid and non-destructive detection technologies to evaluate the quality and maturity of strawberries.

[0003] Traditional destructive evaluation methods, such as laboratory chemical analysis for soluble solids content (SSC) and pH, or texture analysis for hardness, are time-consuming and laborious. Summary of the invention

[0004] In order to overcome the above problems existing in the prior art, the present invention proposes a nondestructive detection method for the internal quality and maturity of strawberries based on the MPCD-Mamba model.

[0005] The technical solution adopted by the present invention to solve the technical problem is: a nondestructive detection method for the internal quality and maturity of strawberries based on the MPCD-Mamba model, comprising the following steps: Step 1, collect strawberry samples, use a hyperspectral reflectance imaging system to obtain hyperspectral images, measure SSC, pH, and hardness, and mark the maturity; Step 2, extracting the spectrum of the hyperspectral image obtained in step 1 after correction to form a data set, and dividing the data set into a training set, a test set, and a validation set; Step 3, constructing the MPCD-Mamba model, using the training set obtained in step 2 to train the MPCD-Mamba model, using the validation set based on the cross-validation method to test the performance of the MPCD-Mamba model, and finally obtaining the MPCD-Mamba model; Step 4, collecting strawberry hyperspectral images, extracting average spectral data, inputting the extracted average spectral data into the MPCD-Mamba model obtained in step 3, obtaining the predicted value of the relevant index of each pixel, generating a heat map of the corresponding components at different positions of the strawberry, and displaying the distribution of different components; The MPCD-Mamba model in step 3 includes an input layer, a backbone network, and a prediction layer. The input layer performs data preprocessing through moving average smoothing and principal component analysis; the backbone network is used to perform convolution operations and output features, which is achieved through three-layer convolution, and sparse jump connections are introduced during the convolution process; a Mamba module with dynamic focus is introduced into the prediction layer, and the Mamba module with dynamic focus selectively focuses on the multi-level features output by the backbone network, and a residual structure is added to the outermost layer of the Mamba module with dynamic focus; the prediction layer introduces a multi-task learning method, sets regression and classification detection heads, and realizes simultaneous detection of SSC, pH, hardness and maturity.

[0006] The above-mentioned nondestructive detection method for internal quality and maturity of strawberries based on the MPCD-Mamba model, the step 2 is specifically: manually selecting a region of interest, randomly selecting a box of size n*n in the selected area, and calculating the average value of the spectral data in the box as the spectral data, extracting m data from each strawberry sample, and corresponding to the corresponding label data.

[0007] In the above-mentioned nondestructive detection method for strawberry internal quality and maturity based on the MPCD-Mamba model, the specific calculation formula for hyperspectral image correction in step 2 is: Among them, R is the rectified image, R raw is the original hyperspectral image, R w is a white reference image obtained from a white board with a reflectivity of 99%, R d is a dark reference image obtained by turning off the lights.

[0008] In the above-mentioned nondestructive detection method for internal quality and maturity of strawberries based on the MPCD-Mamba model, the Mamba module with dynamic focusing uses adaptive average pooling to calculate the average value of each channel; through two convolution operations, the number of channels is first compressed and then restored; the result is multiplied by the original data through a sigmoid function to achieve the focus on important information channels. The specific calculation formula is as follows: Among them, x is the input data, Conv is a one-dimensional convolution, BN is batch normalization, It is element multiplication, out is the output feature, AdaptiveAvgPoolld is one-dimensional adaptive average pooling, SiLu is the activation function, and Conv is convolution.

[0009] In the above-mentioned nondestructive detection method for internal quality and maturity of strawberries based on the MPCD-Mamba model, the output features of all convolutional layers in the backbone network of the MPCD-Mamba model are only spliced ​​with the last layer features of the backbone network.

[0010] The above-mentioned nondestructive detection method for strawberry internal quality and maturity based on the MPCD-Mamba model, the specific calculation formula of the multi-task learning method is: in, is the mean square error, is the cross entropy loss, is the total loss, is the number of samples, is the true value of the i-th sample, is the predicted value of the i-th sample, is the number of categories, is the actual label, is the probability that the model predicts that the current sample belongs to category c.

[0011] The beneficial effects of the present invention are as follows: (1) the present invention proposes a Mamba model with a dynamic focusing module, which selectively focuses on the multi-level features of the backbone output, reducing the impact of noise and useless information on the final result.

[0012] (2) A multi-task learning method was introduced. By adding additional detection heads, simultaneous detection of SSC, pH, hardness and maturity was achieved, which solved the problem that traditional methods could not detect multiple quality parameters simultaneously. Through visualization, an intuitive display of multiple quality parameters and maturity was achieved, which not only achieved more complex simultaneous detection, but also achieved higher accuracy.

[0013] (3) A method for random mean sampling of hyperspectral images is proposed, which can reduce the impact of some outliers on the final performance of the model and improve the ability to resist interference. It is suitable for lower-precision hyperspectral images and can reduce the cost of image acquisition and model training.

[0014] (4) The skip connection method of DenseNet was improved, giving full play to its advantage of feature reuse and reducing the number of parameters by 11,152. The inference speed of a single sample is 3.10ms, and the model effect is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the present invention; Figure 2This is a visualization of part of the strawberry hyperspectral data set in the embodiment of the present invention, where (a) is the original RGB image, and (b) is a region of interest (ROI) annotation map of the hyperspectral data; Figure 3 for hyperspectral data visualization; Figure 4 It is a schematic diagram of the network structure of the MPCD-Mamba model of the present invention; Figure 5 This is a prediction heat map of some strawberries of the present invention, wherein (a) is an RGB image after hyperspectral acquisition, (b) is a prediction heat map of SSC, (c) is a prediction heat map of pH, (d) is a prediction heat map of hardness, and (e) is a prediction heat map of maturity; Figure 6 Schematic diagram of the performance of the MPCD-Mamba model on the test set in an embodiment of the present invention, wherein (a) is a graph showing the relationship between the measured value and the predicted value of the soluble fixative; (b) is a graph showing the relationship between the measured value and the predicted value of pH; and (c) is a graph showing the relationship between the measured value and the predicted value of hardness. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0017] This embodiment discloses a multi-task nondestructive detection method for the internal quality and maturity of strawberries based on a convolutional neural network fused with D-Mamba. Figure 1 As shown, the specific steps include: Step 1: The data set of this embodiment is collected from the Hongshiya strawberry planting base in Huangdao District, Qingdao City, Shandong Province (121.49° east longitude, 36.77° north latitude, 17m above sea level). The base was built in 1991 and has more than 360 planting greenhouses with a total area of ​​more than 500 hectares. Samples were collected from different farmers to reduce the impact of location deviation.

[0018] The strawberry samples were brought back to the laboratory on the day of collection, and hyperspectral images were acquired using a hyperspectral reflectance imaging system. The system consists of a hyperspectrometer, a charge-coupled device (CCD) camera, two halogen lamps, a mobile platform, a black box, and a computer with image acquisition software. The distance between the hyperspectral lens and the sample is 1m, and the camera resolution is 1700*900. After collecting the HSI image, SSC, PH and hardness were measured, and maturity was marked. Due to the uneven light distribution of the camera and the influence of dark current, the hyperspectral image needs to be corrected before spectral extraction. The calculation formula is as follows:

[0019] Among them, R is the rectified image, R raw is the original hyperspectral image, R w is a white reference image obtained from a white board with a reflectivity of 99%, R d is a dark reference image obtained by turning off the lights.

[0020] The distribution statistics of each index of the sample are shown in Table 1, including the minimum, maximum, mean and standard deviation. Among them, the range of SSC is from 3.7 to 10.3, and the range of hardness is from 0.6 to 4.4. This dataset has a certain degree of versatility and is suitable for non-destructive detection tasks of strawberries in different environments. The dataset contains 200 samples, covering mature and unripe strawberries at different growth stages.

[0021] Table 1 Step 2: In this embodiment, the region of interest (ROI) is first manually selected to ensure data accuracy. Figure 2 Visualization of part of the strawberry hyperspectral dataset, where (a) is the original RGB image, each acquisition contains hyperspectral data of 20 strawberry samples, and (b) is the region of interest (ROI) annotation map of the hyperspectral data, represented by red.

[0022] The data includes information of 300 bands with wavelengths ranging from 381.4nm to 1020.69nm. In order to eliminate interference, the strawberry sample was placed on a black background plate. Subsequently, a 5*5 sized box was randomly selected in the ROI area, and the average value of the spectral data in the box was calculated as the spectral data of the sample. Five such data were extracted for each strawberry sample and corresponded to the corresponding label data to form training data. This method effectively suppresses the influence of noise and outliers, while reducing the bias effect of a single data point on the prediction results, making the data more stable and reliable. In addition, this method ensures that the spectral data extracted from different positions have certain differences, increases the diversity of the data, and helps the model better learn the spectral characteristics of strawberries under different circumstances. All extracted spectral data such as Figure 3 As shown, Figure 3 The horizontal axis represents 300 bands, and the vertical axis represents the reflectance of the strawberry sample.

[0023] Step 3, construct the MPCD-Mamba model. The MPCD-Mamba of this embodiment uses moving average smoothing (MA) and principal component analysis (PCA) for data preprocessing, and integrates the D-mamba algorithm. This model can simultaneously predict SSC, PH, hardness and maturity classification. The overall structure of MPCD-Mamba (MA-Pca-Conv-D-Mamba) is as follows Figure 4 The input-output relationship of each layer is shown in Table 2.

[0024] Table 2 The MPCD-Mamba model includes an input layer, a backbone network, and a prediction layer. The input layer performs data preprocessing through moving average smoothing (MA) and principal component analysis; the backbone network is implemented through three layers of convolution, and sparse jump connections are introduced during the convolution process to achieve efficient fusion of different features. Compared with traditional dense connections, this design effectively reduces the number of parameters and calculations. The backbone network is used to perform convolution operations and output features; the prediction layer introduces a Mamba module with dynamic focus, which selectively focuses on the multi-level features output by the backbone network, and the outermost layer of the Mamba module with dynamic focus adds a residual structure; the prediction layer introduces a multi-task learning method, sets regression and classification detection heads, and realizes the simultaneous detection of SSC, pH, hardness, and maturity. The following are the key improvements of the model:

[0025] 1. Use random data sampling to increase the quantity and diversity of data.

[0026] 2. Introduce the Mamba algorithm with dynamic focusing to improve the model's ability to filter key information.

[0027] 3. By reducing densely connected branches, efficient feature extraction is achieved, avoiding the problem of the model being too large.

[0028] 4. In terms of multi-task learning, regression and classification detection heads are designed, and the loss function is changed accordingly. The calculation formula is as follows:

[0029] in, is the number of samples, It is The true value of the samples, It is the first The predicted value of the sample, is the number of categories, is the actual label, is the probability that the model predicts that the current sample belongs to category c.

[0030] Selective State Space Models (SSMs) combine the features of recurrent neural networks (RNNs) and convolutional neural networks (CNNs) and are inspired by classic state space models. Mamba is an innovative sequence modeling architecture that combines the power of SSMs with a simplified end-to-end neural network design. By introducing content-based reasoning and selective information propagation, Mamba solves the computational efficiency problem of Transformer when processing long sequences and demonstrates excellent performance in multiple fields such as language, audio, and genomics. With hardware-aware algorithms and SSMs, Mamba has leading-edge performance and scalability, making it a promising general-purpose base model architecture for processing sequence data.

[0031] This embodiment proposes a dynamic focusing module based on convolution, which can selectively focus on the multi-level features of the backbone output. By learning the weight of each channel, the model can automatically focus on the most useful channel in the current task, thereby improving the expressiveness of the model. In contrast, convolution treats all channels equally during the feature extraction process, and cannot actively select and emphasize important feature channels, resulting in information interference. In addition, the dynamic focusing module also learns the correlation information between different channels, and provides a richer feature representation through the weighted combination of different channel features. At the same time, the module can embed spatial information into channel attention, so that the model can better utilize the spatial structure of the input features and improve the perception of spatial information.

[0032] This module first uses adaptive average pooling to calculate the average value of each channel. Then, through two convolution operations, the number of channels is first compressed and then restored to promote information interaction. Finally, the result is multiplied with the original data through the sigmoid function to achieve the effect of focusing on important information channels. The calculation formula is as follows:

[0033] Among them, x is the input data, Conv is a one-dimensional convolution, BN is batch normalization, It is element multiplication, out is the output feature, AdaptiveAvgPoolld is one-dimensional adaptive average pooling, SiLu is the activation function, and Conv is convolution.

[0034] Adding a residual structure to the outermost layer of the Mamba structure can solve the gradient vanishing and exploding problems in deep neural networks. Through cross-layer connections, gradients can be more easily propagated in the network, thereby accelerating the convergence process and improving training results. In addition, the residual structure increases the nonlinear expression ability of the model, adapts to complex data distribution and extracts abstract features, which helps to optimize the fitting of training data.

[0035] DenseNet has the advantages of feature reuse and gradient flow mechanism. Dense connections allow each layer to access the feature maps of all previous layers, promote information transfer, and alleviate the gradient vanishing problem, which helps the convergence of deep networks. The generated feature maps are denser, and each layer obtains the feature information of all previous layers, which is conducive to rich feature representation. However, DenseNet has disadvantages. High memory consumption, and the feature maps of all previous layers need to be retained, especially when processing large-scale data. The feature map connection method leads to long inference time. High management and optimization complexity, each layer processes inputs from all previous layers. For large network depth or input size, video memory requirements limit scalability on some hardware.

[0036] This embodiment makes improvements based on the advantages and disadvantages of DenseNet. Unlike DenseNet, the output features of all convolutional layers no longer interact with all subsequent layers, but are only spliced ​​with the last layer features of the backbone network. This adjustment brings the following benefits: first, it ensures the fusion and feature propagation of different levels of information; second, it simplifies the network structure and reduces the model size; finally, it improves the inference speed. Figure 4 The backbone network part.

[0037] Step 4: Use ENVI 5.3 software to remove the background of the strawberry and traverse all pixels of the image. For each pixel, extract the average spectral data in the 5*5 area around it. Then, input the extracted strawberry spectral data into the pre-trained MPCD-Mamba model to obtain the predicted value of the relevant index for each pixel. Finally, generate a heat map of the corresponding components at different positions of the strawberry to show the distribution of different components, such as Figure 5 As shown, (a) is the RGB image after hyperspectral acquisition, (b) is the predicted heat map of SSC, (c) is the predicted heat map of pH, (d) is the predicted heat map of hardness, and (e) is the predicted heat map of maturity. In (b), (c), and (d), blue represents low values ​​and red represents high values. The background part is set to the minimum value of the corresponding indicator, which is 3.7, 3.02, and 0.6, respectively. In the maturity classification heat map (e), blue is the category set separately for the background, green represents immature, and red represents mature.

[0038] The visualization of the distribution of internal indicators of strawberries provides an intuitive and comprehensive evaluation of the internal indicators of each pixel. From Figures (b), (c), and (d), it can be observed that the SSC and pH of unripe strawberries are low, appearing blue, and the hardness is high, appearing red. In the maturity classification (e), the maturity classification of strawberries is correct. It can be seen from the figure that the second row and third column is an unripe strawberry, which shows lower SSC and pH, as well as higher hardness in the heat map, and the classification result correctly classifies it as an unripe strawberry. The fourth row and third column in the figure is a ripe strawberry, which is opposite to the aforementioned strawberry in the heat map, indicating that the model of this embodiment has accurately learned the potential relationship between hyperspectral data and each indicator.

[0039] In order to comprehensively evaluate the performance of the MPCD-Mamba model proposed in this embodiment in strawberry quality prediction, a series of comparative experiments were designed to compare with current advanced methods, including partial least squares (PLS), artificial neural network (ANN), support vector regression (SVR), extreme learning machine (ELM), ResNet and DeepSpectra. All models were trained and tested on the same data set to ensure fairness. The same data preprocessing method MA, data dimensionality reduction method PCA, and unified training and evaluation methods were used in the experiment. Table 3 is a comparative experiment. This embodiment uniformly adopts MA data preprocessing method, PCA data dimensionality reduction method and spxy data partitioning method. The data in Table 1 are all spectral data obtained using an improved data extraction method.

[0040] The experimental results show that compared with the traditional machine learning algorithms SVR and PLS, MPCD-Mamba in this embodiment has a higher average R 2 The average RPD increased by 2.2453 and 2.2431, and the maturity classification accuracy increased by 0.06 and 0.055, respectively. This shows that the model can more accurately predict the SSC, PH, firmness index and maturity of strawberries when dealing with complex nonlinear relationships and high-dimensional data.

[0041] In the comparative experiment, machine learning adopted the implementation method of single-task learning, while deep learning adopted the implementation method of multi-task learning. From the data in the table, it can be observed that in more complex multi-task learning, deep learning is superior to single-task machine learning in performance. Through the results of the comparative experiment, it is observed that the MPCD-Mamba of this embodiment is superior to the traditional model in multiple evaluation indicators. This advantage stems from its strong generalization ability, which provides possibilities for the future in a wider range of agricultural quality detection fields.

[0042] Table 3 In order to deeply understand the contribution of each component in the MPCD-Mamba model and verify the necessity of model design, this embodiment conducts a series of ablation experiments, as shown in Table 4. This embodiment adopts the MA data preprocessing method, the PCA data dimension reduction method and the spxy data partitioning method.

[0043] Table 4 Specific features are added from the baseline model one by one, and the modified models are retrained and evaluated on the same training and test sets. The performance differences of each model are measured and compared through R2, RMSEP, MAE, RDP, maturity classification accuracy, inference speed, number of parameters, and FLOPS indicators to quantify the impact of each component.

[0044] The results show that after the introduction of the improved data extraction method, the average R2 increased by 0.2741 and the average RPD increased by 1.5479. Although the improvement in maturity classification accuracy is not obvious, in the regression task, both the average and individual indicators have achieved significant improvements.

[0045] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A nondestructive detection method for internal quality and maturity of strawberries based on the MPCD-Mamba model, characterized in that: The steps include: Step 1, collect strawberry samples, use a hyperspectral reflectance imaging system to obtain hyperspectral images, measure SSC, pH, and hardness, and mark the maturity; Step 2, extracting the spectrum of the hyperspectral image obtained in step 1 after correction to form a data set, and dividing the data set into a training set, a test set, and a validation set; Step 3, constructing the MPCD-Mamba model, using the training set obtained in step 2 to train the MPCD-Mamba model, using the validation set based on the cross-validation method to test the performance of the MPCD-Mamba model, and finally obtaining the MPCD-Mamba model; Step 4, collecting strawberry hyperspectral images, extracting average spectral data, inputting the extracted average spectral data into the MPCD-Mamba model obtained in step 3, obtaining the predicted value of the relevant index of each pixel, generating a heat map of the corresponding components at different positions of the strawberry, and displaying the distribution of different components; The MPCD-Mamba model in step 3 includes an input layer, a backbone network, and a prediction layer. The input layer performs data preprocessing through moving average smoothing and principal component analysis; the backbone network is used to perform convolution operations and output features, which is achieved through three-layer convolution, and sparse jump connections are introduced during the convolution process; a Mamba module with dynamic focus is introduced into the prediction layer, and the Mamba module with dynamic focus selectively focuses on the multi-level features output by the backbone network, and a residual structure is added to the outermost layer of the Mamba module with dynamic focus; the prediction layer introduces a multi-task learning method, sets regression and classification detection heads, and realizes simultaneous detection of SSC, pH, hardness and maturity.

2. The nondestructive detection method for strawberry internal quality and maturity based on the MPCD-Mamba model according to claim 1, characterized in that: The step 2 is specifically as follows: manually selecting an area of ​​interest, randomly selecting a box of size n*n in the selected area, and calculating the average value of the spectral data in the box as the spectral data, extracting m data from each strawberry sample and corresponding to the corresponding label data.

3. The nondestructive detection method for strawberry internal quality and maturity based on the MPCD-Mamba model according to claim 1, characterized in that: The specific calculation formula for hyperspectral image correction in step 2 is: Among them, R is the rectified image, R raw is the original hyperspectral image, R w is a white reference image obtained from a white board with a reflectivity of 99%, R d is a dark reference image obtained by turning off the lights.

4. The nondestructive detection method for strawberry internal quality and maturity based on the MPCD-Mamba model according to claim 1, characterized in that: The Mamba module with dynamic focus uses adaptive average pooling to calculate the average value of each channel; through two convolution operations, the number of channels is first compressed and then restored; the result is multiplied with the original data through a sigmoid function to achieve the focus on important information channels. The specific calculation formula is as follows: Among them, x is the input data, Conv is a one-dimensional convolution, BN is batch normalization, It is element multiplication, out is the output feature, AdaptiveAvgPoolld is one-dimensional adaptive average pooling, SiLu is the activation function, and Conv is convolution.

5. The nondestructive detection method for strawberry internal quality and maturity based on MPCD-Mamba model according to claim 1, characterized in that: The output features of all convolutional layers in the backbone network of the MPCD-Mamba model are only concatenated with the last layer features of the backbone network.

6. The nondestructive detection method for strawberry internal quality and maturity based on the MPCD-Mamba model according to claim 1, characterized in that: The specific calculation formula of the multi-task learning method is: in, is the mean square error, is the cross entropy loss, is the total loss, is the number of samples, is the true value of the i-th sample, is the predicted value of the i-th sample, is the number of categories, is the actual label, is the probability that the model predicts that the current sample belongs to category c.

Images