Seedless and seeded watermelon classification method based on transmission spectrum and feature fusion deep learning model

Abstract

The invention discloses a seedless and seeded watermelon classification method based on a transmission spectrum combined feature fusion deep learning model. The method comprises the following steps: acquiring a transmission spectrum data set of watermelons by using a transmission spectrum acquisition device; then constructing a feature fusion model, and inputting the transmission spectrum data set of the watermelon into the feature fusion model for training to obtain a trained feature fusion model; and finally, acquiring a transmission spectrum of a watermelon to be detected, and inputting the transmission spectrum of the watermelon to be detected into the trained feature fusion model for classification and identification processing to obtain a seedless or seed type of the watermelon to be detected. According to the invention, the effective combination of the transmission spectrum method, the image conversion method and the convolutional neural network method is realized, and the advantage that the rapid and accurate classification result of seedless and seeded watermelons is directly obtained by collecting the transmission spectrum of the watermelons and carrying out model processing is realized.

Description

Technical Field

[0001] The present invention belongs to the field of rapid fruit quality detection, and in particular relates to a method for classifying seedless and seeded watermelons based on a transmission spectrum combined with a feature fusion deep learning model. Background Art

[0002] Watermelon is rich in water, minerals, vitamins, etc., and is a popular horticultural crop worldwide. With the advancement of technology, seedless watermelon has become more and more popular among consumers and has a larger share of the consumer market. The popularity of seedless watermelon can be attributed to its convenience, disease resistance, and nutritional advantages. Seedless watermelon has a hard and crisp flesh, is sweet and juicy, and has become the first choice of many consumers. In addition, seedless watermelon has higher tissue hardness, is less likely to develop internal fruit diseases, and is easier to store and transport, which helps to extend its shelf life. However, the cultivation process of seedless watermelon is laborious and time-consuming, and the seed production cost is high, so the price is higher than that of seeded watermelon. There are usually a large number of watermelons on the market, including seedless and seeded watermelons. Sometimes there is a situation where the two types of watermelons are mixed, or there are situations where they need to be centrally classified according to different needs. Therefore, it is of great significance to explore the high-throughput and high-precision classification of seedless and seeded watermelons.

[0003] Transmission spectroscopy can obtain information about the inside of a watermelon and can meet the needs of high-throughput online testing, so it is more suitable for large-sized fruits. Deep learning uses deep neural networks to extract advanced features and has demonstrated important information extraction capabilities in various fields. Based on a certain number of images, deep learning can capture local and global features of images and fuse features from low dimensions to high dimensions. At present, watermelon quality detection based on the combination of two-dimensional images and deep learning needs further research.

[0004] At present, most studies are based on the detection of watermelon sugar content or maturity based on transmission spectroscopy technology, and there is no research on the classification of seedless and seeded watermelons based on transmission spectroscopy technology. In addition, most traditional models are based on one-dimensional spectral data and machine learning models, and the accuracy of watermelon quality detection needs to be improved. Summary of the invention

[0005] In view of the defects and shortcomings in the prior art, the object of the present invention is to provide a method for classifying seedless and seeded watermelons based on transmission spectroscopy combined with a feature fusion deep learning model.

[0006] The technical solution adopted by the present invention is as follows:

[0007] The present invention discloses a method for classifying seedless and seeded watermelons based on transmission spectrum combined with a feature fusion deep learning model, comprising the following steps:

[0008] S1. Use a transmission spectrum acquisition device to acquire a transmission spectrum dataset of watermelon.

[0009] S2. Construct a feature fusion model, and input the watermelon transmission spectrum dataset into the feature fusion model for training to obtain a trained feature fusion model.

[0010] S3. Collect the transmission spectrum of the watermelon to be tested, input the transmission spectrum of the watermelon to be tested into the trained feature fusion model for classification and recognition processing, and obtain the category of the watermelon to be tested, whether it is seedless or seeded.

[0011] The transmission spectrum combined with feature fusion model includes a feature band selection module, a dimension conversion module and a dual-input convolutional neural network module connected in series.

[0012] The characteristic band selection module uses a competitive adaptive reweighted sampling method to process the transmission spectrum data set of the watermelon obtained in step S1 to obtain a one-dimensional characteristic band data set of the watermelon.

[0013] The dimension conversion module uses the Gram angular field algorithm to process the obtained one-dimensional characteristic band data set of the watermelon to obtain a two-dimensional Gram sum field image data set and a Gram difference field image data set.

[0014] The dual-input convolutional neural network module is specifically as follows: two parallel first feature extraction channels and second feature extraction channels are set, and then the output features of the two channels are preliminarily fused through a fully connected layer, and finally classified through another fully connected layer to obtain a classification result; the obtained two-dimensional Grammar sum field image data set is input into the first feature extraction channel, and the obtained Grammar difference field image data set is input into the second feature extraction channel.

[0015] The Gram angular field algorithm is specifically:

[0016]

[0017] Where GASF represents the two-dimensional Grammar sum field image data value, GADF represents the two-dimensional Grammar difference field image data value, α, α1, α2 and α n are all different polar angles converted from one-dimensional characteristic band data to polar coordinates, r is the polar diameter converted from one-dimensional characteristic band data to polar coordinates, X represents the transmission intensity value in the one-dimensional characteristic band data set and X={x1,x2,x3,...,x i ,...,x n}, xi represents the i-th transmission intensity value in the one-dimensional characteristic band data set, n represents the number of wavelengths in the one-dimensional characteristic band data set, x i Represents the i-th transmission intensity value in the normalized one-dimensional feature band dataset.

[0018] The first feature extraction channel and the second feature extraction channel both include a plurality of extraction and dimension reduction units connected in series and a fully connected layer.

[0019] Each of the extraction and dimensionality reduction units includes two or three convolutional layers and one pooling layer connected in series.

[0020] The convolution layers all use 3×3 convolution, and the pooling layers all use 2×2 maximum pooling.

[0021] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0022] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0023] The innovation of the present invention lies in the method of integrating the transmission spectrum method, image conversion method and convolutional neural network method to construct a feature fusion model, which realizes the advantage of collecting the transmission spectrum of watermelon for model processing and directly obtaining fast and accurate classification results of seedless and seeded watermelons, bringing the beneficial effect of combining optical recognition with neural network image processing.

[0024] The beneficial effects of the present invention are as follows:

[0025] (1) The present invention classifies seedless watermelons and seeded watermelons by integrating a transmission spectrum method, an image conversion method and a convolutional neural network method, thereby realizing lossless online classification of watermelons.

[0026] (2) The present invention achieves high accuracy in classifying seedless and seeded watermelons based on transmission spectroscopy technology combined with a feature fusion deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of a method for classifying seedless and seeded watermelons based on transmission spectrum combined with feature fusion deep learning model of the present invention;

[0028] Figure 2 : is a diagram of the Gram angular field conversion process used in the present invention; wherein (a) is a normalized curve in a rectangular coordinate system, (b) is a curve converted to a polar coordinate system, (c) is a GASF image, and (d) is a GADF image;

[0029] Figure 3 It is a structural diagram of a dual-input convolutional neural network module constructed by the present invention;

[0030] Figure 4 This is a watermelon classification result diagram according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The method proposed by the present invention is further described below in conjunction with the accompanying drawings and embodiments provided by the inventor.

[0032] The method proposed in the present invention realizes the effective fusion of the transmission spectrum method, the image conversion method and the convolutional neural network method, and realizes the rapid and accurate classification of seedless and seeded watermelons. First, the collected one-dimensional transmission spectrum of the watermelon is converted into two different two-dimensional images through the Gram angle field. The two-dimensional images are subjected to feature extraction and splicing through the constructed convolutional neural network model, and then the fusion feature information of different input images is obtained to obtain the final watermelon classification result.

[0033] The specific embodiments of the present invention are as follows:

[0034] Experimental preparation: The seedless and seeded watermelons used in the study were purchased from a local fruit wholesale market in Hangzhou. These watermelons came from different provinces in China and were all Qilin watermelons. After the watermelons were transported to the laboratory, they were stored for at least 10 hours to return to room temperature of 25°C to eliminate the influence of the temperature difference of the watermelons on their transmission spectra. In the end, a total of 163 seedless watermelons and 160 seeded watermelons were obtained in the experiment. After the spectrum was collected, each watermelon sample was cut from the equator to obtain the watermelon pulp image.

[0035] like Figure 1 As shown, the following is a flow chart of the method of the present invention.

[0036] S1. Use a transmission spectrum acquisition device to acquire a transmission spectrum dataset of watermelon.

[0037] As a large-sized and thick-skinned fruit, watermelon requires a higher-power light source to obtain a stable and accurate transmission spectrum. The light source includes 10 halogen tungsten lamps with a power of 150W each. The transmission spectrum acquisition range is 630-1000nm, and the integration time is 500ms.

[0038] S2. Construct a feature fusion model, and input the watermelon transmission spectrum dataset into the feature fusion model for training to obtain a trained feature fusion model.

[0039] The feature fusion model includes a feature band selection module, a dimension conversion module and a dual-input convolutional neural network module connected in series.

[0040] The characteristic band selection module uses a competitive adaptive reweighted sampling method to process the transmission spectrum data set of the watermelon obtained in step S1 to obtain a one-dimensional characteristic band data set of the watermelon.

[0041] The competitive adaptive reweighted sampling algorithm uses a ten-fold cross-validation method, with a maximum number of latent variables extracted of 30 and 100 sampling iterations.

[0042] The dimension conversion module uses the Gram angular field algorithm to process the one-dimensional characteristic band dataset of the watermelon to obtain a two-dimensional Gram sum field image dataset and a Gram difference field image dataset.

[0043] The obtained two-dimensional Grammar sum field image dataset and Grammar difference field image dataset can retain the time dependency of the data and intuitively display certain characteristics of the data.

[0044] The Gram angular field algorithm is specifically:

[0045]

[0046]

[0047] Where GASF represents the two-dimensional Grammar sum field image data value, GADF represents the two-dimensional Grammar difference field image data value, α, α1, α2 and α n are all different polar angles converted from one-dimensional characteristic band data to polar coordinates, r is the polar diameter converted from one-dimensional characteristic band data to polar coordinates, X represents the transmission intensity value in the one-dimensional characteristic band data set and X={x1,x2,x3,...,x i ,...,x n}, xi represents the i-th transmission intensity value in the one-dimensional characteristic band data set, n represents the number of wavelengths in the one-dimensional characteristic band data set, x i Represents the i-th transmission intensity value in the normalized one-dimensional feature band dataset.

[0048] In the specific implementation, the Kennard-Stone algorithm is used to divide the two-dimensional Grammar sum field image dataset and the Grammar difference field image dataset into a calibration set and a validation set in a ratio of 3:1.

[0049] In specific implementation, Figure 2 As shown, the Gram angular field conversion process diagram used in the present invention; wherein, (a) is the normalized curve in the rectangular coordinate system, (b) is the curve converted to the polar coordinate system, (c) is the GASF image, and (d) is the GADF image; the time for generating a sample GASF and GADF image using the original 501 wavelengths in this embodiment is 0.183 seconds, while the time for generating 84 wavelengths selected using the competitive adaptive reweighted sampling algorithm is 0.101 seconds. The image generation time after the optimal feature parameters is 55.19% of the original. Fewer feature wavelengths increase the speed of constructing the Gram angular field image.

[0050] The dual-input convolutional neural network module is specifically as follows: two parallel first feature extraction channels and second feature extraction channels are set up, and then the output features of the two channels are preliminarily fused through a fully connected layer, and finally classified through another fully connected layer to obtain the classification result; the obtained two-dimensional Grammar sum field image dataset is input into the first feature extraction channel, and the obtained Grammar difference field image dataset is input into the second feature extraction channel.

[0051] The first feature extraction channel and the second feature extraction channel both include several extraction and dimension reduction units connected in series and a final fully connected layer; each extraction and dimension reduction unit includes two or three convolutional layers connected in series and a final pooling layer. The convolutional layers all use 3×3 convolutions, and the pooling layers all use 2×2 maximum pooling.

[0052] In the specific implementation, the competitive adaptive reweighted sampling method adopts a ten-fold cross-validation method, the maximum number of latent variables extracted is 30, and 100 sampling iterations are performed. The competitive adaptive reweighted sampling method extracts 84 characteristic variables from the original 501 spectral data.

[0053] The learning rate of the dual-input convolutional neural network module is 0.001, the optimizer uses Adam to dynamically update the network weight parameters, the mini-batch processing size is 10, and the training iteration rounds are 100 times.

[0054] Each convolution layer includes a series of convolution operations and activation function operations. The activation function uses a linear rectifier unit function, which is a nonlinear activation function ReLU, which is used to zero out negative values ​​and only retain positive values ​​to enhance the nonlinear expression ability of the model.

[0055] The preset dimension of the fully connected layer for the initial fusion of the two channels is 100, that is, the number of output features is 100.

[0056] like Figure 3As shown, Conv represents the convolution layer and the linear rectifier unit function operation, Pooling represents the maximum pooling layer, and fc represents the fully connected layer. The CNN model for single GASF and GADF image input is the same. In this study, the size of the generated GASF and GADF images is 300×300×3. The present invention proposes a new convolutional neural network model, which increases the depth of the network by stacking multiple small-sized convolution kernels and pooling layers, gradually reduces the spatial dimension of the image, and improves the representation ability of image features. Relatively small 3×3 convolution kernels and 2×2 maximum pooling kernels are used, and linear rectifier unit functions are used after each convolution layer. In this embodiment, the linear rectifier unit function ReLU is a nonlinear activation function applied to the output of each convolution layer, which is used to zero negative values ​​and retain only positive values ​​to enhance the nonlinear expression ability of the model, thereby improving the model's learning ability for complex features. A fully connected layer is designed at the end of the network, and the number of nodes in the fully connected layers fc1 and fc2 is set to 100, which is used to map the features extracted by the convolution and pooling layers to the final classification output. The features extracted by the two branches are concatenated after being processed by the convolutional layer. This method can combine the feature information of different input images. Two fully connected layers fc3 and fc4 are designed at the end of the concatenated network, with 100 and 2 nodes respectively.

[0057] S3. Collect the transmission spectrum of the watermelon to be tested, input the transmission spectrum of the watermelon to be tested into the trained feature fusion model for classification and recognition processing, and obtain the category of the watermelon to be tested, whether it is seedless or seeded.

[0058] Figure 4 This is a watermelon classification result diagram of an embodiment of the present invention. The recognition accuracy rates of the calibration set and the verification set of this embodiment are 100% and 96.3% respectively.

[0059] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0060] The present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0061] The innovation of the present invention lies in the method of integrating the transmission spectrum method, image conversion method and convolutional neural network method to construct a feature fusion model, which realizes the advantage of collecting the transmission spectrum of watermelon for model processing and directly obtaining fast and accurate classification results of seedless and seeded watermelons, bringing the beneficial effect of combining optical recognition with neural network image processing.

[0062] It should be noted that the above content is only used to illustrate a technical solution of the present invention, rather than to limit the protection scope of the present invention. Simple modifications or equivalent substitutions of the technical solution of the present invention by those skilled in the art do not exceed the scope of the present invention.

Claims

1. A method for classifying seedless and seeded watermelons based on transmission spectroscopy combined with a feature fusion deep learning model, characterized in that: The following steps are involved: S1. Using a transmission spectrum acquisition device to acquire a transmission spectrum data set of watermelon; S2, constructing a feature fusion model, and inputting the watermelon transmission spectrum data set into the feature fusion model for training, to obtain a trained feature fusion model; S3. Collect the transmission spectrum of the watermelon to be tested, input the transmission spectrum of the watermelon to be tested into the trained feature fusion model for classification and recognition processing, and obtain the category of the watermelon to be tested, whether it is seedless or seeded.

2. The method for classifying seedless and seeded watermelons based on transmission spectrum combined with feature fusion deep learning model according to claim 1, characterized in that: The transmission spectrum combined with feature fusion model includes a feature band selection module, a dimension conversion module and a dual-input convolutional neural network module connected in series in sequence; The characteristic band selection module processes the watermelon transmission spectrum data set obtained in step S1 by using a competitive adaptive reweighted sampling method to obtain a one-dimensional characteristic band data set of the watermelon; The dimension conversion module uses the Gram angular field algorithm to process the obtained one-dimensional characteristic band data set of the watermelon to obtain a two-dimensional Gram sum field image data set and a Gram difference field image data set; The dual-input convolutional neural network module is specifically as follows: two parallel first feature extraction channels and second feature extraction channels are set, and then the output features of the two channels are preliminarily fused through a fully connected layer, and finally classified through another fully connected layer to obtain a classification result; the obtained two-dimensional Grammar sum field image data set is input into the first feature extraction channel, and the obtained Grammar difference field image data set is input into the second feature extraction channel.

3. The method for classifying seedless and seeded watermelons based on transmission spectrum combined with feature fusion deep learning model according to claim 2, characterized in that: The Gram angular field algorithm is specifically: Where GASF represents the two-dimensional Grammar sum field image data value, GADF represents the two-dimensional Grammar difference field image data value, α, α1, α2 and α n are all different polar angles converted from one-dimensional characteristic band data to polar coordinates, r is the polar diameter converted from one-dimensional characteristic band data to polar coordinates, X represents the transmission intensity value in the one-dimensional characteristic band data set and X={x1,x2,x3,...,x i ,...,x n }, xi represents the i-th transmission intensity value in the one-dimensional characteristic band data set, n represents the number of wavelengths in the one-dimensional characteristic band data set, x i Represents the i-th transmission intensity value in the normalized one-dimensional feature band dataset.

4. The method for classifying seedless and seeded watermelons based on transmission spectrum combined with feature fusion deep learning model according to claim 2, characterized in that: The first feature extraction channel and the second feature extraction channel both include a plurality of extraction and dimensionality reduction units connected in series and a fully connected layer; each of the extraction and dimensionality reduction units includes two or three convolutional layers connected in series and a pooling layer.

5. The method for classifying seedless and seeded watermelons based on transmission spectrum combined with feature fusion deep learning model according to claim 4, characterized in that: The convolution layers all use 3×3 convolution, and the pooling layers all use 2×2 maximum pooling.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images