Width learning shiny-leaved yellowhorn fruit maturity identification method based on characteristic excitation

Abstract

The invention discloses a width learning shiny-leaved yellowhorn fruit maturity identification method based on characteristic excitation, and the method comprises the steps: obtaining a shiny-leaved yellowhorn fruit image, and carrying out the preprocessing; manually marking the preprocessed shiny-leaved yellowhorn fruit image, and dividing the shiny-leaved yellowhorn fruit image into a training set and a test set according to a ratio of 7: 3 to obtain a shiny-leaved yellowhorn fruit data set; constructing a shiny-leaved yellowhorn fruit maturity identification model, inputting the shiny-leaved yellowhorn fruit data set into the shiny-leaved yellowhorn fruit maturity identification model for training and testing, and obtaining a shiny-leaved yellowhorn fruit maturity identification model based on feature excitation; and inputting the shiny-leaved yellowhorn fruit data set into a shiny-leaved yellowhorn fruit maturity identification model based on feature excitation for training verification to obtain a shiny-leaved yellowhorn fruit maturity identification result. The Xanthoceras sorbifolia Bunge fruit maturity identification and classification detection model based on the SeNet attention mechanism module is used to improve the traditional width learning, and the method still has good speed performance under the condition of ensuring high accuracy.

Description

Technical Field

[0001] The invention relates to the field of intelligent detection technology, and in particular to a width learning Xanthoceras sorbifolia fruit maturity recognition method based on feature excitation. Background Art

[0002] The maturity detection of Xanthoceras sorbifolia fruit is a key link in the production management of Xanthoceras sorbifolia products, and is of great significance for the timely picking of fruits. At present, my country's Xanthoceras sorbifolia planting industry has transitioned from the traditional scattered planting model to the large-scale and intensive planting model, but most planting bases still rely on the traditional manual naked eye observation method to judge the maturity of Xanthoceras sorbifolia fruit. This method relies entirely on the experience of the planting staff and the intensity of the ripening performance of Xanthoceras sorbifolia fruit. It has high labor intensity and low accuracy. Due to the inaccurate judgment of some boundary conditions, the wrong picking time is chosen, and the cracking of the fruit will directly cause the waste of Xanthoceras sorbifolia fruit, which in turn affects the economic benefits of the planting base.

[0003] In recent years, image processing and deep learning, such as K-means clustering algorithm, threshold segmentation algorithm, support vector machine, fuzzy clustering and other methods, have been used to detect the maturity of fruits. However, there is no efficient and accurate detection method for Xanthoceras sorbifolia. At present, it is still necessary to manually judge whether the fruit has reached the standard for picking based on experience, which consumes a lot of time and energy, and the operator's experience and technical level will affect the picking situation and thus affect the product yield. Therefore, how to automatically and accurately identify and detect the maturity of Xanthoceras sorbifolia fruit is a problem that needs to be solved.

[0004] Method content

[0005] In view of the above problems, the present invention proposes a feature-excitation-based width learning method for identifying the maturity of Xanthoceras sorbifolia fruit, which can automatically analyze the collected image information of Xanthoceras sorbifolia fruit, reduce the labor intensity of related operators, and at the same time improve the accuracy of fruit maturity and find the best time for picking.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for recognizing the maturity of Xanthoceras sorbifolia fruit based on width learning of feature excitation comprises the following steps:

[0008] Step 1, obtaining a Xanthoceras sorbifolia fruit image, and preprocessing the Xanthoceras sorbifolia fruit image;

[0009] Step 2: manually annotate the Xanthoceras sorbifolia fruit images preprocessed in step 1, and divide the manually annotated Xanthoceras sorbifolia fruit images into a training set and a test set in a ratio of 7:3 to obtain a Xanthoceras sorbifolia fruit dataset;

[0010] Step 3, constructing a Xanthoceras sorbifolia fruit maturity recognition model, inputting a Xanthoceras sorbifolia fruit data set into the Xanthoceras sorbifolia fruit maturity recognition model for training and testing, and obtaining a Xanthoceras sorbifolia fruit maturity recognition model based on feature excitation and neural width learning;

[0011] Step 4, inputting the Xanthoceras sorbifolia fruit data set into the feature-excitation-based neural width learning Xanthoceras sorbifolia fruit maturity recognition model of step 3 for training and verification, and obtaining the Xanthoceras sorbifolia fruit maturity recognition result;

[0012] Manually labeling the pre-processed Xanthoceras sorbifolia fruit images includes: classifying the processed Xanthoceras sorbifolia fruit images into three categories: immature, ready for picking, and mature and cracked;

[0013] The Xanthoceras sorbifolia fruit maturity recognition model comprises four parts: an input layer, a feature extraction layer, a feature conversion layer and a feature excitation layer;

[0014] The input layer preprocesses the input Xanthoceras sorbifolia fruit data set;

[0015] The feature extraction layer performs preliminary feature extraction on the Xanthoceras sorbifolia fruit data set preprocessed by the input layer;

[0016] The feature conversion layer generates a nonlinear mapping of the Xanthoceras sorbifolia fruit data set through random weights and biases, constructs enhancement nodes, strengthens the preliminary features extracted through the feature nodes, and performs linear or nonlinear conversion on the output of the feature nodes;

[0017] The feature excitation layer excites and weights the converted features output by the feature conversion layer to enhance important features and suppress unimportant features.

[0018] In the above technical solution, the preprocessing of the Xanthoceras sorbifolia fruit image includes:

[0019] The acquired Xanthoceras sorbifolia fruit images are toned and numbered, and the numbered Xanthoceras sorbifolia fruit images are data cleaned to obtain initial sample images; if the number of initial sample images does not reach the number required for training, the initial sample images are data enhanced.

[0020] In the above technical scheme, the Xanthoceras sorbifolia fruit dataset includes three file packages, namely Annotations, JPEGimages and ImageSet, wherein Annotations is an xml format file; JPEGimages is a jpg format file, and all images in the jpg format file are named according to numbers; ImageSet is four txt format files, and the four txt format files are training verification set, training set, verification set and test set respectively.

[0021] In the above technical solution, the linear or nonlinear conversion of the output of the feature node includes: using a sparse autoencoder method to perform sparse processing on the output of the feature node and the enhancement node, using an alternating direction multiplier method to optimize the weight coefficients between the feature representation and the feature node; and using a feature conversion function F(X in ) Combine the outputs of feature nodes and enhancement nodes into an input feature matrix.

[0022] In the above technical solution, the step of stimulating and weighting the converted features output by the feature conversion layer includes the following steps:

[0023] S3.1: Construct a feature excitation network and perform global average pooling on the feature map of each feature channel in the global average pooling layer;

[0024] S3.2: Learn the importance of each feature channel after global average pooling through two fully connected layers;

[0025] S3.3: Reweight the features according to the importance of the learned feature channels.

[0026] In the above technical solution, the formula for global average pooling of the feature map of each feature channel is as follows:

[0027]

[0028] Among them, Z c is the feature average of feature channel c, H and W are the height and width of the feature map, i and j are the row and column indexes of each position in the feature map, and they traverse all positions of the feature map respectively.

[0029] In the above technical solution, the learning formula of the feature channel after global average pooling and two fully connected layers is:

[0030] s=σ(W 2 δ(W 1 z))

[0031] Among them, s is the feature vector obtained after global average pooling, σ is the Sigmoid activation function, δ is the ReLU activation function, and W 1 and W 2 is the weight matrix of the fully connected layer, and z is the final output vector after two layers of fully connected layers and activation function processing.

[0032] In the above technical solution, the formula for re-weighting the features is as follows:

[0033]

[0034] in, is the weighted feature map, u c is the feature map of each channel, S c is the weighted weight.

[0035] The beneficial effects of the present invention are as follows:

[0036] The present invention proposes a Xanthoceras sorbifolia fruit maturity recognition and detection method based on feature excitation and width learning, and improves the traditional Xanthoceras sorbifolia fruit maturity recognition and classification detection model based on SeNet attention mechanism module. It has good speed performance while ensuring a high accuracy rate, thereby reducing the labor intensity of planting personnel and improving the accuracy and efficiency of Xanthoceras sorbifolia fruit detection. It can be used for the immediate picking of fruits in large-scale Xanthoceras sorbifolia planting bases. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 Shown is a module framework diagram of the feature excitation width learning system of the present invention.

[0039] Figure 2 Shown is a flow chart of the Xanthoceras sorbifolia fruit maturity recognition method based on feature excitation and width learning of the present invention.

[0040] Figure 3 Shown is a schematic diagram of the feature construction and conversion process of the present invention.

[0041] Figure 4 Shown is a schematic diagram of the feature-motivated width learning network process of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] like Figure 1 As shown, the present embodiment provides a Xanthoceras sorbifolia fruit maturity detection system based on feature excitation width learning, including two parts: a BLS width learning system module and a SENet (feature excitation network) module.

[0044] The BLS width learning system module uses the random vector functional link network (RVFLNN) as the basis, uses feature nodes and enhancement nodes to build a simple two-layer network, inputs the acquired data sets into the two-layer network respectively, and maps and processes them through feature nodes and enhancement nodes. Among them, the feature nodes are used to extract the features of the input data set, and the enhancement nodes are used to further strengthen the features extracted by the feature nodes. The sparse autoencoder is used to perform sparse representation and optimization of the mapped and converted features to enhance the identifiability of the features; the alternating direction multiplier method (ADMM) is used to solve the optimized feature formula to optimize the feature representation and the weight coefficients between the feature nodes.

[0045] The SENet module performs channel re-weighting on feature channels, enhances important features, and suppresses unimportant features, which can greatly improve the accuracy of feature representation.

[0046] A feature-based width learning method for recognizing Xanthoceras sorbifolia fruit maturity, see Figure 2 , including the following steps:

[0047] Step 1: obtain the Xanthoceras sorbifolia fruit image and preprocess the Xanthoceras sorbifolia fruit image.

[0048] Among them, the acquired Xanthoceras sorbifolia fruit images are growth images of Xanthoceras sorbifolia fruit at different growth stages; the format of the Xanthoceras sorbifolia fruit images is JPG format.

[0049] The preprocessing of the Xanthoceras sorbifolia fruit image comprises:

[0050] The acquired X. sorbifolia fruit images are toned and numbered, and the numbered X. sorbifolia fruit images are data cleaned (i.e., images without fruit or blurred images are deleted) to obtain initial sample images. If the number of initial sample images does not reach the number required for training, data enhancement is performed on the initial sample images (i.e., the number of image samples is increased by flipping, enlarging, reducing the images, etc.). The initial sample images of this embodiment are preferably 1,100, including 500 mature X. sorbifolia fruit sample images and 600 immature X. sorbifolia fruit sample images.

[0051] Step 2: Manually annotate the Xanthoceras sorbifolia fruit images preprocessed in step 1 (use the annotation software labelme to annotate the initial sample images after data enhancement), and divide the manually annotated Xanthoceras sorbifolia fruit images into training set and test set in a ratio of 7:3 (according to maturity and degree of harvestability), to obtain the Xanthoceras sorbifolia fruit dataset.

[0052] Manually labeling the pre-processed Xanthoceras sorbifolia fruit images includes: classifying the processed Xanthoceras sorbifolia fruit images into three categories: immature, ready for picking, and mature and cracked.

[0053] The Xanthoceras sorbifolia fruit dataset includes three file packages, namely Annotations, JPEGimages and ImageSet, wherein Annotations is an xml format file; JPEGimages is a jpg format file, and all images (training set and test set) in the jpg format file are named by number; ImageSet is four txt format files, and the four txt format files are training verification set, training set, verification set and test set.

[0054] Step 3, construct a Xanthoceras sorbifolia fruit maturity recognition model, input the Xanthoceras sorbifolia fruit data set into the Xanthoceras sorbifolia fruit maturity recognition model for training and testing (the Xanthoceras sorbifolia fruit annotated images manually annotated in the training set are used as input, and the maturity of the Xanthoceras sorbifolia fruit annotated images is used as output to train the Xanthoceras sorbifolia fruit maturity recognition model), and obtain the Xanthoceras sorbifolia fruit maturity recognition model based on feature excitation and neural width learning.

[0055] Among them, the Xanthoceras sorbifolia fruit maturity recognition model of this embodiment is a breadth learning (BLS) network with a SeNet attention mechanism module added, which enables the computer to fully learn the features in the Xanthoceras sorbifolia fruit dataset; through the shallow neural network such as BLS, the features in the Xanthoceras sorbifolia fruit dataset are quickly extracted, and the features in the dataset are captured and processed through feature nodes and enhancement nodes, avoiding the high computational cost of deep learning networks.

[0056] See also Figure 3 , Figure 4 The Xanthoceras sorbifolia fruit maturity recognition model includes four parts: an input layer, a feature extraction layer, a feature conversion layer and a feature excitation layer.

[0057] The input layer preprocesses the input Xanthoceras sorbifolia fruit data set; the preprocessing of the input Xanthoceras sorbifolia fruit data set includes: parsing the three file packages of the Xanthoceras sorbifolia fruit data set into a form in which an iterator can return a series of (img, bbox, label) tuples, performing channel conversion, pixel normalization, scaling, flipping (horizontally and vertically) and cropping of the training set.

[0058] The feature extraction layer performs preliminary feature extraction on the Xanthoceras sorbifolia fruit data set preprocessed by the input layer; the preliminary feature extraction includes: constructing feature nodes (Feature Nodes), and extracting preliminary features of the Xanthoceras sorbifolia fruit data set preprocessed by the input layer through the feature nodes.

[0059] The feature conversion layer generates a nonlinear mapping of the Xanthoceras sorbifolia fruit data set through random weights and biases, constructs enhancement nodes, enhances the preliminary features extracted through the feature nodes, and performs linear or nonlinear conversion on the output of the feature nodes (converting the enhanced features into a format that can be processed by the feature excitation network).

[0060] The linear or nonlinear conversion of the output of the feature node includes: using the sparse autoencoder method to sparse the output of the feature node and the enhanced node, using the alternating direction multiplier method (ADMM) to optimize the weight coefficient between the feature representation and the feature node, which can improve the feature representation ability of the feature node and the enhanced node and reduce redundant information; and using the feature conversion function F(X in ) Combine the outputs of feature nodes and enhancement nodes into an input feature matrix.

[0061] In order to further improve the performance of the Xanthoceras sorbifolia fruit maturity recognition model, this embodiment cyclically searches different node combinations to find the node configuration that performs best between classification accuracy and test speed; and determines the best feature node and enhanced node combination through experiments.

[0062] The feature excitation layer excites and weights the converted features (input feature matrix) output by the feature conversion layer to enhance important features and suppress unimportant features, which can greatly improve the accuracy of feature representation.

[0063] The weight calculation formula for stimulating the input feature matrix output by the feature conversion layer is as follows:

[0064] s=F ex (z,W)

[0065] Among them, s is the output after activation function processing, z is the intermediate result after linear transformation, and W is the weight matrix used to control the mapping relationship from input to output.

[0066] The step of stimulating and weighting the converted features output by the feature conversion layer comprises the following steps:

[0067] S3.1: Construct a feature excitation network (SENet) and perform global average pooling on the feature map of each feature channel in the global average pooling layer (compress the feature map of each feature channel into a scalar to obtain a global feature description). The formula for global average pooling of the feature map of each feature channel is as follows:

[0068]

[0069] Among them, Z cis the feature average of feature channel c, H and W are the height and width of the feature map, respectively, and i and j are the row and column indexes of each position in the feature map, which are used to traverse all positions of the feature map respectively.

[0070] S3.2: Learn the importance (weight) of each feature channel after global average pooling through two layers of fully connected layers. The learning formula of the feature channel after global average pooling through two layers of fully connected layers is:

[0071] s=σ(W 2 δ(W 1 z))

[0072] Among them, s is the feature vector obtained after global average pooling, σ is the Sigmoid activation function, δ is the ReLU activation function, and W 1 and W 2 is the weight matrix of the fully connected layer, z is the final output vector after two layers of fully connected layers and activation function processing. It is the feature representation obtained by the model after a series of transformations and learning of the input data, which can be used for subsequent tasks such as classification.

[0073] S3.3: Reweight the features according to the importance of the learned feature channels. The formula for reweighting the features is as follows:

[0074]

[0075] in, is the weighted feature map, u c is the feature map of each channel, S c is the weighted weight.

[0076] Furthermore, in order to improve the performance (classification speed) of the Xanthoceras sorbifolia fruit maturity recognition model, the features are recalibrated and classified, the excitation and weighted feature matrix is ​​input into the classifier, and the final classification is performed according to the reweighted feature matrix. The classification formula is:

[0077] Y=X b W m

[0078] Among them, W m is the weight matrix of the classifier, X bis the recalibrated (or weighted) feature matrix, which represents the features finally input into the classifier. It is usually the feature matrix output from the previous layer (e.g., the feature conversion layer after excitation and weighting). These features have been processed in some form, such as feature normalization, weighting, recalibration, etc., for better classification. Y is the output matrix of the classifier, which represents the classification result. Specifically, Y can be the predicted value of each sample in different categories. It is usually a probability distribution or classification label (depending on the specific output form of the classifier).

[0079] The feature-inspired deep learning model for identifying Xanthoceras sorbifolia fruit maturity in this embodiment achieved 100% accuracy on the training set and more than 80% accuracy on the test set. In addition, compared with other deep learning models, it has faster training and classification speeds and can meet the needs of real-time recognition.

[0080] Step 4, input the Xanthoceras sorbifolia fruit data set into the feature-excitation-based neural width learning Xanthoceras sorbifolia fruit maturity recognition model in step 3 for training and verification, and obtain the Xanthoceras sorbifolia fruit maturity recognition result.

[0081] The feature-excitation-based neural width learning method for recognizing the maturity of Xanthoceras sorbifolia fruits in this embodiment performs non-contact recognition of Xanthoceras sorbifolia fruits, and realizes the recognition of the maturity of Xanthoceras sorbifolia fruits by collecting images of the Xanthoceras sorbifolia fruits, thereby achieving the purpose of convenient, economical and efficient judgment of the picking conditions of Xanthoceras sorbifolia fruits.

[0082] The above is a preferred implementation mode of the present invention and should not be regarded as any form of limitation on the present invention. The preferred embodiments and related technologies in the present invention have been described above, but they do not limit the present invention. For relevant technical personnel familiar with the technical field, without departing from the scope of the technical principles of the present invention, the technical principles described above can be used to make some improvements or substitutions, including modifications and equivalent changes based on the core content of the present invention, all of which are within the scope of protection of the present invention.

Claims

1. A method for recognizing the maturity of Xanthoceras sorbifolia fruit based on width learning of feature excitation, characterized in that: The following steps are involved: Step 1, obtaining a Xanthoceras sorbifolia fruit image, and preprocessing the Xanthoceras sorbifolia fruit image; Step 2: manually annotate the Xanthoceras sorbifolia fruit images preprocessed in step 1, and divide the manually annotated Xanthoceras sorbifolia fruit images into a training set and a test set in a ratio of 7:3 to obtain a Xanthoceras sorbifolia fruit dataset; Step 3, constructing a Xanthoceras sorbifolia fruit maturity recognition model, inputting a Xanthoceras sorbifolia fruit data set into the Xanthoceras sorbifolia fruit maturity recognition model for training and testing, and obtaining a Xanthoceras sorbifolia fruit maturity recognition model based on feature excitation and neural width learning; Step 4, inputting the Xanthoceras sorbifolia fruit data set into the feature-excitation-based neural width learning Xanthoceras sorbifolia fruit maturity recognition model of step 3 for training and verification, and obtaining the Xanthoceras sorbifolia fruit maturity recognition result; Manually labeling the pre-processed Xanthoceras sorbifolia fruit images includes: classifying the processed Xanthoceras sorbifolia fruit images into three categories: immature, ready for picking, and mature and cracked; The Xanthoceras sorbifolia fruit maturity recognition model comprises four parts: an input layer, a feature extraction layer, a feature conversion layer and a feature excitation layer; The input layer preprocesses the input Xanthoceras sorbifolia fruit data set; The feature extraction layer performs preliminary feature extraction on the Xanthoceras sorbifolia fruit data set preprocessed by the input layer; The feature conversion layer generates a nonlinear mapping of the Xanthoceras sorbifolia fruit data set through random weights and biases, constructs enhancement nodes, strengthens the preliminary features extracted through the feature nodes, and performs linear or nonlinear conversion on the output of the feature nodes; The feature excitation layer excites and weights the converted features output by the feature conversion layer to enhance important features and suppress unimportant features.

2. The width learning method for recognizing Xanthoceras sorbifolia fruit maturity according to claim 1, characterized in that: The preprocessing of the Xanthoceras sorbifolia fruit image comprises: The acquired Xanthoceras sorbifolia fruit images are toned and numbered, and the numbered Xanthoceras sorbifolia fruit images are data cleaned to obtain initial sample images; if the number of initial sample images does not reach the number required for training, the initial sample images are data enhanced.

3. The width learning method for recognizing Xanthoceras sorbifolia fruit maturity according to claim 1, characterized in that: The Xanthoceras sorbifolia fruit dataset includes three file packages, namely Annotations, JPEGimages and ImageSet, wherein Annotations is an xml format file; JPEGimages is a jpg format file, and all images in the jpg format file are named by number; ImageSet is four txt format files, and the four txt format files are training verification set, training set, verification set and test set.

4. The width learning method for recognizing Xanthoceras sorbifolia fruit maturity according to claim 1, characterized in that: The linear or nonlinear transformation of the output of the feature node includes: using the sparse autoencoder method to sparse the output of the feature node and the enhancement node, using the alternating direction multiplier method to optimize the weight coefficient between the feature representation and the feature node; and using the feature conversion function F(X in ) Combine the outputs of feature nodes and enhancement nodes into an input feature matrix.

5. The width learning method for recognizing Xanthoceras sorbifolia fruit maturity according to claim 1, characterized in that: The step of stimulating and weighting the converted features output by the feature conversion layer comprises the following steps: S3.1: Construct a feature excitation network and perform global average pooling on the feature map of each feature channel in the global average pooling layer; S3.2: Learn the importance of each feature channel after global average pooling through two fully connected layers; S3.3: Reweight the features according to the importance of the learned feature channels.

6. The width learning method for recognizing Xanthoceras sorbifolia fruit maturity according to claim 5, characterized in that: The formula for global average pooling of the feature map of each feature channel is as follows: Among them, Zc is the feature average of feature channel c, H and W are the height and width of the feature map, respectively, and i and j are the row and column indexes of each position in the feature map, respectively traversing all positions of the feature map.

7. The width learning method for recognizing Xanthoceras sorbifolia fruit maturity according to claim 5, characterized in that: The learning formula of the feature channel after global average pooling and two fully connected layers is: s=σ(W2δ(W1z)) Among them, s is the feature vector obtained after global average pooling, σ is the Sigmoid activation function, δ is the ReLU activation function, W1 and W2 are the weight matrices of the fully connected layer, and z is the final output vector after two layers of fully connected layers and activation function processing.

8. The width learning method for recognizing Xanthoceras sorbifolia fruit maturity according to claim 5, characterized in that: The formula for reweighting features is as follows: in, is the weighted feature map, u c is the feature map of each channel, S c is the weighted weight.

Images