Seedless and seeded watermelon classification method based on transmission spectrum combined with feature fusion deep learning model

Abstract

The application discloses a seedless and seeded watermelon classification method based on transmission spectrum combined with feature fusion deep learning model. The method comprises the following steps: firstly, using a transmission spectrum acquisition device to acquire a transmission spectrum data set of the watermelon; then, constructing a feature fusion model, inputting the transmission spectrum data set of the watermelon into the feature fusion model for training, and obtaining the trained feature fusion model; finally, collecting the transmission spectrum of the watermelon to be measured, inputting the transmission spectrum of the watermelon to be measured into the trained feature fusion model for classification and identification processing, and obtaining the category of the seedless or seeded watermelon to be measured. The application realizes the effective combination of the transmission spectrum method, the image conversion method and the convolutional neural network method, and realizes the advantages of collecting the transmission spectrum of the watermelon, processing the model and directly obtaining the rapid and accurate classification result of the seedless and seeded watermelon.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of rapid detection of fruit quality, and particularly relates to a seedless and seeded watermelon classification method based on transmission spectrum combined with feature fusion deep learning model. BACKGROUND

[0002] Watermelon is a popular horticultural crop worldwide due to its high water, mineral, and vitamin content. With the advancement of technology, seedless watermelons are becoming increasingly popular among consumers and are gaining a larger share of the consumer market. The popularity of seedless watermelons can be attributed to their convenience, disease resistance, and nutritional advantages. Seedless watermelons have a firm and crisp flesh, a sweet and juicy taste, and have become the first choice for many consumers. In addition, seedless watermelons have a higher tissue hardness, are less prone to internal fruit diseases, and are easier to store and transport, which helps to extend their shelf life. However, the cultivation process of seedless watermelons is laborious and time-consuming, and the cost of seed production is high, so the price is higher than that of seeded watermelons. There are usually a large number of watermelons on the market, including seedless and seeded watermelons. Sometimes there may be adulteration of the two types of watermelons, or there may be a need to concentrate classification according to different needs. Therefore, it is of great significance to explore high-throughput and high-precision classification of seedless and seeded watermelons.

[0003] Transmission spectrum technology can obtain information about the inside of the watermelon and can meet the needs of high-throughput online detection, so it is more suitable for large-sized fruits. Deep learning uses deep neural networks to extract high-level features and has shown important information extraction capabilities in various fields. Based on a certain number of images, deep learning can capture local and global features of the image and fuse features from low to high dimensions. Current research on watermelon quality detection based on two-dimensional images and deep learning needs further study.

[0004] Currently, most research is focused on detecting the sugar content or maturity of watermelons based on transmission spectrum technology, and there is no research on classifying seedless and seeded watermelons based on transmission spectrum technology. In addition, most traditional models are based on one-dimensional spectral data and machine learning models, and the accuracy of watermelon quality detection needs to be improved. SUMMARY

[0005] In view of the defects and shortcomings in the prior art, the purpose of the present application is to provide a seedless and seeded watermelon classification method based on transmission spectrum combined with feature fusion deep learning model.

[0006] The technical solution adopted by the present application is as follows:

[0007] The seedless and seeded watermelon classification method based on transmission spectrum combined with feature fusion deep learning model of the present application comprises the following steps:

[0008] S1, using a transmission spectrum acquisition device to acquire the transmission spectrum data set of the watermelon.

[0009] S2, a feature fusion model is constructed, and the transmission spectrum data set of watermelons is input into the feature fusion model for training to obtain a trained feature fusion model.

[0010] S3, the transmission spectrum of the to-be-tested watermelon is collected, and the transmission spectrum of the to-be-tested watermelon is input into the trained feature fusion model for classification and identification processing to obtain the category of the to-be-tested seedless or seeded watermelon.

[0011] The transmission spectrum combined with the feature fusion model comprises a feature waveband selection module, a dimension conversion module and a double-input convolutional neural network module connected in sequence.

[0012] The feature waveband selection module adopts a competitive adaptive reweighted sampling method to process the transmission spectrum data set of watermelons obtained in step S1 to obtain a one-dimensional feature waveband data set of watermelons.

[0013] The dimension conversion module adopts a Gram angle field algorithm to process the obtained one-dimensional feature waveband data set of watermelons to obtain a two-dimensional Gram sum field image data set and a Gram difference field image data set.

[0014] The double-input convolutional neural network module specifically comprises: setting two parallel first feature extraction channels and second feature extraction channels, then preliminarily fusing the output features of the two channels through a full connection layer, and finally classifying through another full connection layer to obtain a classification result; the obtained two-dimensional Gram sum field image data set is input into the first feature extraction channel, and the obtained Gram difference field image data set is input into the second feature extraction channel.

[0015] The Gram angle field algorithm specifically comprises:

[0016]

[0017] In the formula, GASF represents a two-dimensional Gram sum field image data value, GADF represents a two-dimensional Gram difference field image data value, and a, a1, a2 and a n are different polar angles of one-dimensional feature waveband data converted into polar coordinates, r is a polar radius of one-dimensional feature waveband data converted into polar coordinates, X represents a transmission intensity value in the one-dimensional feature waveband data set and X={x1, x2, x3,..., x i ,...,x n}, xi represents the i-th transmission intensity value in the one-dimensional feature waveband data set, n represents the number of wavelengths in the one-dimensional feature waveband data set, and x i represents the i-th transmission intensity value in the one-dimensional feature waveband data set after normalization.

[0018] The first feature extraction channel and the second feature extraction channel each comprise a plurality of extraction dimension reduction units and a full connection layer connected in series.

[0019] Each of the extraction dimension reduction units comprises two or three convolution layers and a pooling layer connected in series.

[0020] The convolution layers each adopt 3*3 convolution, and the pooling layers each adopt 2*2 maximum pooling.

[0021] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0022] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0023] The innovation of the present application is that the transmission spectrum method, image conversion method and convolution neural network method are fused together to construct a feature fusion model, which realizes the advantages of collecting the transmission spectrum of watermelons for model processing to directly obtain the fast and accurate classification results of seedless and seeded watermelons, and brings the beneficial effects of the combination of optical recognition and neural network image processing.

[0024] The beneficial effects of the present application are as follows:

[0025] (1) The present application adopts the fusion of the transmission spectrum method, image conversion method and convolution neural network method to classify seedless watermelons and seeded watermelons, and can realize the nondestructive online classification of watermelons.

[0026] (2) The present application realizes the high accuracy of the classification of seedless and seeded watermelons based on the transmission spectrum technology combined with the feature fusion deep learning model. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a flow chart of a method for classifying seedless and seeded watermelons based on the transmission spectrum combined with the feature fusion deep learning model of the present application;

[0028] Figure 2 is a Gram angle field conversion process diagram used in the present application; wherein (a) is a normalized curve in a rectangular coordinate system, (b) is a curve converted to polar coordinates, (c) is a GASF image, and (d) is a GADF image;

[0029] Figure 3 is a structure diagram of a double-input convolutional neural network module constructed in the present application;

[0030] Figure 4 is a watermelon classification result diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0031] The method proposed in the application will be further described in combination with the description of the accompanying drawings and examples given by the inventors.

[0032] The method proposed in the application realizes effective fusion of the transmission spectrum method, the image conversion method and the convolutional neural network method, and realizes rapid and accurate classification of seedless and seeded watermelons. First, the collected one-dimensional transmission spectrum of the watermelon is converted into two different two-dimensional images through a Gram angle field, and the two-dimensional images are subjected to feature extraction and splicing through the constructed convolutional neural network model, thereby obtaining fusion feature information of different input images, and finally obtaining the classification result of the watermelon.

[0033] The specific implementation of the application is as follows:

[0034] Experimental preparation: The seedless watermelons and seeded watermelons used in the study were purchased from a fruit wholesale market in Hangzhou. These watermelons came from different provinces in China and were all Kirin watermelons. After being transported to the laboratory, the watermelons were stored for at least 10 hours to recover to room temperature of 25℃, so as to eliminate the influence of temperature difference on the transmission spectrum of the watermelons. A total of 163 seedless watermelons and 160 seeded watermelons were finally obtained. After the spectrum was collected, each watermelon sample was cut open from the equatorial position to obtain the watermelon flesh image.

[0035] As shown in the flowchart of the method of the application. Figure 1

[0036] S1, using a transmission spectrum acquisition device to acquire a transmission spectrum data set of the watermelon.

[0037] As a large-sized thick-skinned fruit, a watermelon requires a larger power light source to obtain stable and accurate transmission spectrum. The light source includes 10 halogen lamps with a power of 150W, the transmission spectrum acquisition range is 630-1000nm, and the integration time is 500ms.

[0038] S2, constructing a feature fusion model and inputting the transmission spectrum data set of the watermelon into the feature fusion model for training to obtain a trained feature fusion model.

[0039] The feature fusion model includes a feature band selection module, a dimension conversion module and a double-input convolutional neural network module connected in sequence.

[0040] The feature band selection module adopts a competitive adaptive reweighted sampling method to process the transmission spectrum data set of the watermelon obtained in step S1 to obtain a one-dimensional feature band data set of the watermelon.

[0041] The competitive adaptive reweighted sampling algorithm adopts a ten-fold cross-validation method, the maximum potential variable extraction number is 30, and the sampling iteration is performed for 100 times.​

[0042] The dimensionality conversion module uses the Gram angular field algorithm to process the obtained one-dimensional characteristic band dataset of watermelon to obtain a two-dimensional Gram sum field image dataset and a Gram difference field image dataset.

[0043] Obtaining a two-dimensional Grammar sum field image dataset and a Grammar difference field image dataset can preserve the time dependency of the data and intuitively display certain characteristics of the data.

[0044] The Gram's angular field algorithm is specifically:

[0045]

[0046]

[0047] Where GASF represents the two-dimensional Gram sum field image data value, GADF represents the two-dimensional Gram difference field image data value, α, α1, α2 and α n All of them are different polar angles converted from one-dimensional characteristic band data to polar coordinates, r is the polar diameter converted from one-dimensional characteristic band data to polar coordinates, X represents the transmission intensity value in the one-dimensional characteristic band data set and X={x1,x2,x3,...,x i ,...,x n}, xi represents the i-th transmission intensity value in the one-dimensional characteristic band data set, n represents the number of wavelengths in the one-dimensional characteristic band data set, x i Represents the i-th transmission intensity value in the normalized one-dimensional feature band dataset.

[0048] In the specific implementation, the Kennard-Stone algorithm is used to divide the two-dimensional Grammar sum field image dataset and the Grammar difference field image dataset into a calibration set and a validation set in a ratio of 3:1.

[0049] In specific implementation, Figure 2 As shown in the figure, the Gram angular field conversion process used in the present invention; wherein (a) is the normalized curve in the rectangular coordinate system, (b) is the curve converted to polar coordinates, (c) is the GASF image, and (d) is the GADF image; in this embodiment, the time to generate a sample GASF and GADF image using the original 501 wavelengths is 0.183 seconds, while the time to generate the 84 wavelengths selected using the competitive adaptive reweighted sampling algorithm is 0.101 seconds. The image generation time after optimizing the characteristic parameters is 55.19% of the original time. Fewer characteristic wavelengths speed up the construction of the Gram angular field image.

[0050] The double-input convolutional neural network module specifically comprises: two parallel first feature extraction channels and second feature extraction channels are set, then the output features of the two channels are preliminarily fused through a full connection layer, and finally another full connection layer is used for classification to obtain a classification result; the obtained two-dimensional Gram and field image dataset is input into the first feature extraction channel, and the obtained Gram difference field image dataset is input into the second feature extraction channel.

[0051] The first feature extraction channel and the second feature extraction channel each comprise a plurality of extraction dimension reduction units and a last full connection layer connected in sequence; each extraction dimension reduction unit comprises two or three convolutional layers and a last pooling layer connected in sequence. The convolutional layers all adopt 3*3 convolution, and the pooling layers all adopt 2*2 max pooling.

[0052] In a specific implementation, the competition adaptive reweighted sampling method adopts a ten-fold cross-validation method, the maximum latent variable extraction number is 30, and 100 sampling iterations are performed. The competition adaptive reweighted sampling method extracts 84 characteristic variables from the original 501 spectral data.

[0053] The learning rate of the double-input convolutional neural network module is 0.001, the optimizer adopts Adam to dynamically update the network weight parameters, the small batch processing size is 10, and the training iteration round is 100 times.

[0054] Each convolutional layer comprises a convolution operation and an activation function operation connected in series, the activation function adopts a linear rectifier unit function, the linear rectifier unit function is a nonlinear activation function ReLU, which is used for zeroization processing of negative values and only retains positive values to enhance the nonlinear expression ability of the model.

[0055] The preset dimension of the full connection layer for preliminary fusion of the two channels is 100, that is, the output feature number is 100.

[0056] As Figure 3As shown, Conv represents the convolution layer and linear rectifier unit function operation, Pooling represents the maximum pooling layer, and fc represents the fully connected layer. The CNN model for single GASF and GADF image input is the same. In this study, the generated GASF and GADF images are both 300×300×3 in size. The present invention proposes a new convolutional neural network model that increases the depth of the network by stacking multiple small-sized convolution kernels and pooling layers, gradually reducing the spatial dimension of the image while improving the representation ability of image features. Relatively small 3×3 convolution kernels and 2×2 maximum pooling kernels are used, and linear rectifier unit functions are used after each convolution layer. In this embodiment, the linear rectifier unit function ReLU is a nonlinear activation function applied to the output of each convolution layer to zero out negative values ​​and retain only positive values ​​to enhance the nonlinear expression ability of the model, thereby improving the model's ability to learn complex features. A fully connected layer is designed at the end of the network, and the number of nodes in the fully connected layers fc1 and fc2 is set to 100 to map the features extracted by the convolution and pooling layers to the final classification output. The features extracted by the two branches are concatenated after processing in the convolutional layer. This method can combine feature information from different input images. Two fully connected layers, fc3 and fc4, are designed at the end of the concatenated network, with 100 and 2 nodes respectively.

[0057] S3. Collect the transmission spectrum of the watermelon to be tested, input the transmission spectrum of the watermelon to be tested into the trained feature fusion model for classification and recognition processing, and obtain the category of the watermelon to be tested as seedless or seeded.

[0058] Figure 4 This is a watermelon classification result diagram of an embodiment of the present invention. The recognition accuracy rates of the calibration set and the validation set of this embodiment are 100% and 96.3% respectively.

[0059] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0060] The present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0061] The innovation of the present invention lies in the method of integrating the transmission spectrum method, image conversion method and convolutional neural network method to construct a feature fusion model, which realizes the advantage of collecting the transmission spectrum of watermelon for model processing to directly obtain the fast and accurate classification results of seedless and seeded watermelons, bringing the beneficial effect of combining optical recognition with neural network image processing.

[0062] It is to be explained that the above is only used to illustrate one technical solution of the present application, and is not a limitation on the protection scope of the present application, and simple modifications or equivalent replacements of the technical solution of the present application by those skilled in the art do not exceed the scope of the present application.

Claims

1. A method for classifying seedless and seeded watermelons based on transmission spectroscopy combined with a feature fusion deep learning model, characterized in that: The following steps are involved: S1. Using a transmission spectrum acquisition device to acquire a transmission spectrum dataset of watermelon; S2. Build a feature fusion model and input the watermelon transmission spectrum dataset into the feature fusion model for training to obtain a trained feature fusion model. S3, collecting the transmission spectrum of the watermelon to be tested, inputting the transmission spectrum of the watermelon to be tested into the trained feature fusion model for classification and recognition processing, and obtaining the category of the watermelon to be tested as seedless or seeded; The transmission spectrum combined with feature fusion model includes a feature band selection module, a dimension conversion module and a dual-input convolutional neural network module connected in series; The characteristic band selection module processes the watermelon transmission spectrum dataset obtained in step S1 using a competitive adaptive reweighted sampling method to obtain a one-dimensional characteristic band dataset of the watermelon; The dimensionality conversion module processes the obtained one-dimensional characteristic band dataset of the watermelon using a Gram angular field algorithm to obtain a two-dimensional Gram sum field image dataset and a Gram difference field image dataset; The dual-input convolutional neural network module specifically comprises: setting two parallel first feature extraction channels and second feature extraction channels, then preliminarily fusing the output features of the two channels through a fully connected layer, and finally performing classification through another fully connected layer to obtain a classification result; inputting the obtained two-dimensional Grammar sum field image dataset into the first feature extraction channel, and inputting the obtained Grammar difference field image dataset into the second feature extraction channel.

2. The method for classifying seedless and seeded watermelons based on transmission spectroscopy combined with a feature fusion deep learning model according to claim 1, characterized in that: The Gram's angular field algorithm is specifically: Where GASF represents the two-dimensional Gram sum field image data value, GADF represents the two-dimensional Gram difference field image data value, α, α1, α2 and α n All of them are different polar angles converted from one-dimensional characteristic band data to polar coordinates, r is the polar diameter converted from one-dimensional characteristic band data to polar coordinates, X represents the transmission intensity value in the one-dimensional characteristic band data set and X={x1,x2,x3,...,x i ,...,x n }, x i represents the i-th transmission intensity value in the one-dimensional characteristic band data set, n represents the number of wavelengths in the one-dimensional characteristic band data set, x i Represents the i-th transmission intensity value in the normalized one-dimensional feature band dataset.

3. The method for classifying seedless and seeded watermelons based on transmission spectroscopy combined with a feature fusion deep learning model according to claim 1, characterized in that: The first feature extraction channel and the second feature extraction channel both include a plurality of extraction and dimensionality reduction units connected in series and a fully connected layer; each of the extraction and dimensionality reduction units includes two or three convolutional layers and a pooling layer connected in series.

4. The method for classifying seedless and seeded watermelons based on transmission spectroscopy combined with a feature fusion deep learning model according to claim 3, characterized in that: The convolution layers all use 3×3 convolution, and the pooling layers all use 2×2 maximum pooling.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images