Apple multi-feature online classification method

By using the ResNet50 model in Apple classification recognition technology and integrating the synergistic attention mechanism, the problems of high complexity and poor applicability of manual feature extraction are solved, and Apple multi-feature online classification with high accuracy and stability are achieved, suitable for embedded equipment.

CN118941847APending Publication Date: 2024-11-12XIAN UNIV OF TECH
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202410965404.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing Apple classification recognition technology requires manual extraction of features and labeling, which is highly complex and poorly applicable, and is susceptible to external factors, resulting in unsatisfactory identification results under natural conditions.

Method used

Using ResNet50 as the basic model, adjust network parameters, optimize activation functions, and integrate the collaborative attention mechanism into the model to build a Scoord-ResNet37 classification model based on deep learning, optimize the data set through data augmentation and labeling, and improve the stability of the model. and applicability.

Benefits of technology

It realizes high accuracy and stability of Apple multi-feature online classification, reducing the difficulty of feature extraction and calculation complexity, and is suitable for online classification requirements of embedded devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941847A_ABST
    Figure CN118941847A_ABST
Patent Text Reader

Abstract

The invention discloses an apple multi-feature online classification method, which comprises the steps of collecting apple sample images and performing preprocessing, forming an apple data set, dividing the data set into a training set and a verification set, using ResNet50 as a basic model, adjusting network parameters, optimizing an activation function, fusing a collaborative attention mechanism into the ResNet50 model, constructing a Scoord-ResNet37 classification model based on deep learning, and classifying the apple multi-feature online classification model. And carrying out training and result analysis on the constructed Scoord-ResNet37, and classifying apple images to be classified by utilizing a Scoord-ResNet37 model of which the classification result meets the evaluation index. The apple multi-feature online classification method provided by the invention solves the problems that the existing classification identification technology needs to manually extract and mark features, is high in complexity and poor in applicability, and is easily influenced by external factors.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of agricultural fruit classification and recognition methods, and in particular relates to an apple multi-feature online classification method. Background Art

[0002] When selling or buying apples, effective classification of apple varieties can meet consumers' demand for different apple varieties, so that they can understand the price, quality and flavor of specific varieties, and classify them finely for packaging. In order to avoid economic losses caused by improper classification of varieties due to misclassification or omission, domestic and foreign scholars have developed many technologies that are widely used in fruit classification.

[0003] Regarding the apple classification problem, traditional classification and recognition technologies, such as spectral detection technology, dielectric property detection technology, CT detection technology, electronic nose technology, computer vision technology, evidence theory fusion, support vector machine and other methods, all require manual feature extraction and labeling. This process is highly complex and has poor applicability and is easily affected by external factors. The main manifestation is that when the environment and position of the same object change, the classification result also changes accordingly.

[0004] Traditional fruit recognition algorithms mostly rely on visual features such as the size, color, texture, and shape of the fruit, and have achieved good recognition results under specific environments. However, due to the complexity and variability of the agricultural planting environment, the recognition of target fruits is easily affected by interference factors such as uneven lighting, occlusion of branches and leaves, and fruit overlap. Its appearance characteristics will change greatly with environmental changes. Due to the influence of interference factors, the recognition effect of apple recognition algorithms based on image segmentation is not ideal under natural conditions. Therefore, how to improve the model based on a limited data set and essentially improve the classification stability and reliability has become one of the urgent problems to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide an apple multi-feature online classification method, which solves the problem that the existing classification and recognition technology requires manual feature extraction and marking, has high complexity and poor applicability, and is easily affected by external factors.

[0006] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present invention is: an apple multi-feature online classification method, collecting and preprocessing apple sample images to form an apple data set, dividing the data set into a training set and a validation set, using ResNet50 as the basic model, adjusting the network parameters, optimizing the activation function and integrating the collaborative attention mechanism into the ResNet50 model, constructing a Scoord-ResNet37 classification model based on deep learning, training the constructed Scoord-ResNet37 and analyzing the results, and using the Scoord-ResNet37 model whose classification results meet the evaluation indicators to classify the apple images to be classified.

[0007] The technical solution adopted by the present invention is also characterized in that:

[0008] Furthermore, the apple multi-feature online classification method is implemented in the following steps:

[0009] Step 1, data collection: collect data and select apple sample images without background interference and with complete fruit shape to form an apple dataset;

[0010] Step 2: Data enhancement: Perform data enhancement on the original apple dataset obtained in step 1;

[0011] Step 3, data annotation: randomly divide the data set obtained after data enhancement into training set and validation set in a ratio of 8:2, use the annotation software Labelimg to annotate, and save the information as an .xml file, and finally make a VOC format data set, which contains information such as image name, width, height, class name, object, bounding box coordinates, etc.;

[0012] Step 4: Adjust the bottleneck parameters based on the ResNet50 model, replace the ordinary convolution with the depthwise separable convolution, and obtain the improved ResNet50 model;

[0013] Step 5: Introduce the Silu activation function into the model in step 4;

[0014] Step 6: Integrate the collaborative attention mechanism into the model in step 5 to build a Scoord-ResNet37 classification model based on deep learning;

[0015] Step 7: Use the training set to train the Scoord-ResNet37 classification model obtained in step 6;

[0016] Step 8: Use the multi-category classification evaluation index as the evaluation index to evaluate the classification effect of the Scoord-ResNet37 network model after training in step 7;

[0017] Step 9: Construct the Grad-CAM mathematical model to visualize each layer of the Scoord-ResNet37 network model, present the network's focus area in the form of a heat map to determine whether the feature information learned by the layer is correct, and then evaluate the classification performance of the Scoord-ResNet37 network model. If the model passes the evaluation, proceed to step 10, otherwise return to step 4;

[0018] Step 10: Preprocess the apple image to be classified, input the preprocessed apple image into the Scoord-ResNet37 network model and obtain the classification result.

[0019] Furthermore, in step 1, the specific operation is: collect apple images with no background interference and complete fruit shape through on-site collection, online crawling and public data sets, quantitatively analyze the color characteristics of apples, and classify them according to different coloring rates, and divide red, yellow and green apples into three grades, namely Ⅰ, Ⅱ and Ⅲ.

[0020] Furthermore, in step 1, the original data set is enhanced by flipping, rotating, adding Gaussian noise and gamma transformation.

[0021] Furthermore, in step 3, the specific operation is: randomly divide the data set obtained after data enhancement into a training set and a validation set, use the labeling software Labelimg to label it, the labeling information includes image name, width, height, class name, object, bounding box coordinates, and save the information as an .xml file, and finally make it into a VOC format data set.

[0022] Furthermore, in step 4, the specific operations are as follows: adjust some parameters of ResNet50, first traverse 1×1 convolution after the feature image is input to reduce the channel dimension, and then use batch normalization function and activation function to alleviate gradient disappearance and improve the generalization ability of the model, and then replace the standard convolution with a depth-separable convolution with a convolution kernel size of 3×3 to speed up feature extraction, and finally perform 1×1 convolution on the fused feature map to perform channel scaling, and comprehensively improve the basic module of the residual network;

[0023] The specific operation of depth-separable convolution is as follows: Assume that the input feature map size is H×W×C i , First, in the channel-by-channel convolution process, a 3×3 convolution kernel is used to perform point multiplication and summation. The calculation amount of this process is as shown in formula (1):

[0024] S1=H×W×C i ×3×3 (1)

[0025] Secondly, perform point-by-point convolution and output the feature map H×W×C after channel convolution. iPerform point-by-point convolution with a convolution kernel size of 1×1, fuse the features of different channels and change the number of channels of the output feature map. The output feature map has a size of H×W×C. The calculation amount of this process is as shown in formula (2):

[0026] S2=1×1×C i ×H×W×C (2)

[0027] Among them, H and W are the height and width of the input feature map, C i is the number of channels of the input feature map, and C is the number of channels of the output feature map.

[0028] Furthermore, in step 5, the specific operation is: perform a convolution operation with a convolution kernel size of 7×7 on the input image, then traverse the BN function to normalize the input data, then use the Silu nonlinear activation function to replace the original activation function, and then perform subsequent operations. The Silu nonlinear activation function and its derivative function are specifically shown in formulas (3), (4), and (5):

[0029] Silu=x·sigmoid(x) (3)

[0030]

[0031] Silu=Silu+sigmoid(1-Silu) (5)

[0032] Where sigmoid(x) is the sigmoid activation function.

[0033] Furthermore, in step 6, the collaborative attention mechanism CA module is integrated into the model of step 5. The specific operation is as follows: for the input feature map F∈R C×H×W , first perform average pooling operations in the horizontal and vertical directions to obtain a 1-dimensional vector and Then perform Concat on the spatial dimension to obtain the new feature map C h and C w , and then encode the spatial information in the vertical and horizontal directions through BN and Non-linear functions to solve the problem of data distribution changes in the middle layer during training and prevent the gradient from disappearing or exploding. At this time, the feature map is shown as follows:

[0034]

[0035] Then perform the split operation to decompose f2 into and Each convolution operation with a convolution kernel size of 1×1 is performed to standardize the number of feature map channels to make it consistent with the input, and finally perform normalization and weighting to obtain:

[0036]

[0037] f y =f x ×f h ×f w (9)

[0038] in, is the activation function, is a 1×1 convolution transformation function, f2 is the intermediate feature map of spatial information in the vertical and horizontal directions, and are two tensors decomposed by f2, f x is the attention weight distribution value, f h and f w are transformed into tensors with the same number of channels as the input X using equations (8) and (9), respectively. y is the output value obtained by the final expanded output formula (9).

[0039] Furthermore, in step 6, the Scoord-ResNet37 classification model is specifically as follows: the input data is normalized using the first BN batch normalization function, that is, the input variables of each neuron in each layer of the neural network are forced to return their distribution characteristics to the standard normal distribution; so that the activation input value can generate effective feedback in the sensitive area of ​​the activation function, thereby avoiding the problem of gradient disappearance and accelerating the convergence speed and training speed in the model learning process;

[0040] Silu activation function is used for feature extraction to accelerate the convergence of the model. Maximum pooling with a step size of 2 and a pooling kernel size of 3×3 is used to perform feature screening and information filtering operations on the feature map output by the convolutional layer to improve the generalization ability and computing performance of the model.

[0041] Input the Resbolock feature extraction module of Layer1. This layer uses two bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs.

[0042] The first CA collaborative attention mechanism is used to capture cross-channel interactive information, so that the model extracts multi-scale information, emphasizes key details and suppresses redundant details, enhances representation strength and avoids feature loss caused by dimensionality reduction;

[0043] Input the Resbolock feature extraction module of Layer2. This layer uses three bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs.

[0044] The second CA collaborative attention mechanism is used to enhance the representation strength and avoid feature loss caused by dimensionality reduction. The Resbolock feature extraction module of Layer3 is input. This layer uses a 5-bottleneck residual structure to improve classification and recognition accuracy and reduce computational costs.

[0045] Input the Resbolock feature extraction module of Layer4. This layer uses two bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs. It uses activation functions to promote the stable propagation of gradients and accelerate the training process of the model. It uses global average pooling with a pooling kernel size of 7×7.

[0046] Information is extracted from the data output by the last bottleneck, and the image size of the output data is 1×1×2048; the output of the pooling layer is expanded into a one-dimensional vector using a fully connected layer. The output layer consists of a fully connected layer of 1000 nodes. Softmax is used as a classifier to divide the target variable into multiple categories. After accepting the feature matrix input by the fully connected layer, the output input image corresponds to the respective probability values ​​of each category.

[0047] Furthermore, in step 6, the multi-category classification evaluation indicators include selecting Top1accuracy or Top5accuracy, precision, recall and F1 score as evaluation indicators. Top1accuracy indicates the accuracy of the first-ranked category consistent with the actual result, and Top5 accuracy indicates the accuracy of the top five categories including the actual result. The calculation formulas of other evaluation indicators are as follows:

[0048]

[0049] Among them, TP is true positive, FP is false positive, and FN is false negative.

[0050] The present invention has the following beneficial effects:

[0051] The multi-feature online classification method for apples proposed in the present invention adjusts network parameters based on the Resnet50 network, optimizes activation functions and integrates attention mechanisms, and uses data enhancement and other methods to collect 17 types of apple data sets, and verifies the classification effect through a test set. The algorithm meets the online classification requirements of post-harvest classification robots and has high apple classification accuracy and stability. The improved Scoord-ResNet37 model can be verified by experiments and found to have better performance in extracting features of apple color, size and texture. It not only has high accuracy and low computational complexity, but also meets the needs of online classification of embedded devices. The improved Scoord-ResNet37 model of the classification method of the present invention greatly improves the prediction accuracy of target recognition by virtue of its effective feature extraction capability, and is improved in the accuracy of apple classification detection compared with traditional machine learning methods, and greatly reduces the difficulty of feature extraction and computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is the ResNe50 network structure diagram of the present invention;

[0053] Figure 2 It is the network structure diagram of Scoord-ResNet37 of the present invention;

[0054] Figure 3 It is a depth-separable convolution structure diagram of the present invention;

[0055] Figure 4 These are three activation function curves of the present invention;

[0056] Figure 5 is the Coordinate Attention network flow chart of the present invention;

[0057] Figure 6 It is the Coordinate Attention network structure diagram of the present invention;

[0058] Figure 7 It is a collection of 17 apple types in the present invention;

[0059] Figure 8 It is a loss curve diagram of the model training and verification stage before and after the improvement of the present invention;

[0060] Fig. 9 This is a graph showing the average detection accuracy of classification recognition of the present invention;

[0061] Fig.10 This is the average recall rate result diagram of the classification recognition of the present invention;

[0062] Fig.11 It is the confusion matrix result diagram of the present invention;

[0063] Fig.12 It is a visualized heat map of the test samples before and after the improvement of the present invention. DETAILED DESCRIPTION

[0064] The present invention will be further described below through the accompanying drawings and specific embodiments.

[0065] The invention provides an apple multi-feature online classification method. Apple sample images are collected and preprocessed to form an apple data set, the data set is divided into a training set and a validation set, ResNet50 is used as a basic model, network parameters are adjusted, activation functions are optimized, and a collaborative attention mechanism is integrated into the ResNet50 model, a Scoord-ResNet37 classification model based on deep learning is constructed, the constructed Scoord-ResNet37 is trained and the results are analyzed, and the apple images to be classified are classified using the Scoord-ResNet37 model whose classification results meet the evaluation index.

[0066] Please follow the steps below to implement:

[0067] Step 1, data collection: collect data and select apple sample images without background interference and with complete fruit shape to form an apple dataset;

[0068] The specific operation is: collect apple images with no background interference and complete fruit shape through field collection, online crawling and public data sets, quantitatively analyze the color characteristics of apples, and classify them according to the different coloring rates, and divide red, yellow and green apples into three levels, namely Ⅰ, Ⅱ and Ⅲ;

[0069] Step 2: Data enhancement: The original apple data set obtained in step 1 is enhanced. The original data set in step 1 is enhanced by flipping, rotating, adding Gaussian noise and gamma transformation.

[0070] Step 3, data annotation: randomly divide the data set obtained after data enhancement into training set and validation set in a ratio of 8:2, use the annotation software Labelimg to annotate, and save the information as an .xml file, and finally make a VOC format data set, which contains information such as image name, width, height, class name, object, bounding box coordinates, etc.;

[0071] The specific operation is as follows: the data set obtained after data enhancement is randomly divided into a training set and a validation set, and the labeling software Labelimg is used to label the image. The labeling information includes image name, width, height, class name, object, and bounding box coordinates. The information is saved as an .xml file, and finally a VOC format data set is produced.

[0072] Step 4: Based on the ResNet50 model, adjust the bottleneck parameters and replace the ordinary convolution with the depthwise separable convolution to obtain the improved ResNet50 model; Figure 1 and Figure 2 As shown;

[0073] The specific operation is as follows: adjust some parameters of ResNet50, first traverse 1×1 convolution after inputting the feature image to reduce the channel dimension, then use batch normalization function and activation function to alleviate gradient vanishing and improve the generalization ability of the model, then replace the standard convolution with a depth-separable convolution with a convolution kernel size of 3×3 to speed up feature extraction, and finally perform 1×1 convolution on the fused feature map to perform channel scaling, and comprehensively improve the basic module of the residual network;

[0074] like Figure 3 As shown, the specific operation of the depth-separable convolution is as follows: Assume that the input feature map size is H×W×C i , First, in the channel-by-channel convolution process, a 3×3 convolution kernel is used to perform point multiplication and summation. The calculation amount of this process is as shown in formula (1):

[0075] S1=H×W×C i ×3×3 (1)

[0076] Secondly, perform point-by-point convolution and output the feature map H×W×C after channel convolution. i Perform point-by-point convolution with a convolution kernel size of 1×1, fuse the features of different channels and change the number of channels of the output feature map. The output feature map has a size of H×W×C. The calculation amount of this process is as shown in formula (2):

[0077] S2=1×1×C i ×H×W×C (2)

[0078] Among them, H and W are the height and width of the input feature map, C i is the number of channels of the input feature map, and C is the number of channels of the output feature map;

[0079] Step 5: Introduce the Silu activation function into the model in step 4;

[0080] The specific operation is: perform a convolution operation with a kernel size of 7×7 on the input image, then traverse the BN function to normalize the input data, then use the Silu nonlinear activation function to replace the original activation function, and then perform subsequent operations. The Silu nonlinear activation function and its derivative function are shown in formulas (3), (4), and (5):

[0081] Silu=x×sigmoid(x) (3)

[0082]

[0083] Silu=Silu+sigmoid(1-Silu) (5)

[0084] Where sigmoid(x) is the sigmoid activation function;

[0085] Step 6: Integrate the collaborative attention mechanism into the model in step 5 to build a Scoord-ResNet37 classification model based on deep learning;

[0086] The collaborative attention mechanism CA module is integrated into the model of step 5. The specific operation is as follows: for the input feature map F∈R C×H×W , first perform average pooling operations in the horizontal and vertical directions to obtain a 1-dimensional vector and Then perform Concat on the spatial dimension to obtain new feature maps Ch and C w , and then encode the spatial information in the vertical and horizontal directions through BN and Non-linear functions to solve the problem of data distribution changes in the middle layer during training and prevent the gradient from disappearing or exploding. At this time, the feature map is shown as follows:

[0087]

[0088] Then perform the split operation to decompose f2 into and Each convolution operation with a convolution kernel size of 1×1 is performed to standardize the number of feature map channels to make it consistent with the input, and finally perform normalization and weighting to obtain:

[0089]

[0090]

[0091]

[0092] in, is the activation function, is a 1×1 convolution transformation function, f2 is the intermediate feature map of spatial information in the vertical and horizontal directions, and are two tensors decomposed by f2, f x is the attention weight distribution value, f h and f w are transformed into tensors with the same number of channels as the input X using equations (8) and (9), respectively. y is the output value obtained by the final expanded output formula (9);

[0093] The Scoord-ResNet37 classification model is as follows: the first BN batch normalization function is used to normalize the input data, that is, the input variables of each neuron in each layer of the neural network are forced to return their distribution characteristics to the standard normal distribution; so that the activation input value can generate effective feedback in the sensitive area of ​​the activation function, thereby avoiding the problem of gradient disappearance and accelerating the convergence speed and training speed in the model learning process;

[0094] Silu activation function is used for feature extraction to accelerate the convergence of the model. Maximum pooling with a step size of 2 and a pooling kernel size of 3×3 is used to perform feature screening and information filtering operations on the feature map output by the convolutional layer to improve the generalization ability and computing performance of the model.

[0095] Input the Resbolock feature extraction module of Layer1. This layer uses two bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs.

[0096] The first CA collaborative attention mechanism is used to capture cross-channel interactive information, so that the model extracts multi-scale information, emphasizes key details and suppresses redundant details, enhances representation strength and avoids feature loss caused by dimensionality reduction;

[0097] Input the Resbolock feature extraction module of Layer2. This layer uses three bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs.

[0098] The second CA collaborative attention mechanism is used to enhance the representation strength and avoid feature loss caused by dimensionality reduction. The Resbolock feature extraction module of Layer3 is input. This layer uses a 5-bottleneck residual structure to improve classification and recognition accuracy and reduce computational costs.

[0099] Input the Resbolock feature extraction module of Layer4. This layer uses two bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs. It uses activation functions to promote the stable propagation of gradients and accelerate the training process of the model. It uses global average pooling with a pooling kernel size of 7×7.

[0100] Information is extracted from the data output by the last bottleneck, and the image size of the output data is 1×1×2048; the output of the pooling layer is expanded into a one-dimensional vector using a fully connected layer. The output layer consists of a fully connected layer of 1000 nodes. Softmax is used as a classifier to divide the target variable into multiple categories. After accepting the feature matrix input by the fully connected layer, the output input image corresponds to the respective probability values ​​of each category.

[0101] The multi-category classification evaluation indicators include Top1 accuracy or Top5 accuracy, precision, recall and F1 score as evaluation indicators. Top1 accuracy indicates the accuracy of the first-ranked category and the actual result. Top5 accuracy indicates the accuracy of the top five categories including the actual result. The calculation formulas of other evaluation indicators are as follows:

[0102]

[0103] Among them, TP is true positive, FP is false positive, and FN is false negative;

[0104] Step 7: Use the training set to train the Scoord-ResNet37 classification model obtained in step 6;

[0105] Step 8: Use the multi-category classification evaluation index as the evaluation index to evaluate the classification effect of the Scoord-ResNet37 network model after training in step 7;

[0106] Step 9: Construct the Grad-CAM mathematical model to visualize each layer of the Scoord-ResNet37 network model, present the network's focus area in the form of a heat map to determine whether the feature information learned by the layer is correct, and then evaluate the classification performance of the Scoord-ResNet37 network model. If the model passes the evaluation, proceed to step 10, otherwise return to step 4;

[0107] Step 10: Preprocess the apple image to be classified, input the preprocessed apple image into the Scoord-ResNet37 network model and obtain the classification result.

[0108] The technical solution of the present invention is further described below through embodiments and drawings.

[0109] Example 1

[0110] The multi-feature online classification method of apples collects and preprocesses apple sample images to form an apple dataset, divides the dataset into a training set and a validation set, uses ResNet50 as the basic model, adjusts the network parameters, optimizes the activation function and integrates the collaborative attention mechanism into the ResNet50 model, builds a Scoord-ResNet37 classification model based on deep learning, trains the constructed Scoord-ResNet37 and analyzes the results, and uses the Scoord-ResNet37 model whose classification results meet the evaluation indicators to classify the apple images to be classified.

[0111] Please follow the steps below to implement:

[0112] Step 1, data collection: collect data and select apple sample images without background interference and with complete fruit shape to form an apple dataset;

[0113] Step 2: Data enhancement: Perform data enhancement on the original apple dataset obtained in step 1;

[0114] Step 3, data annotation: randomly divide the data set obtained after data enhancement into training set and validation set in a ratio of 8:2, use the annotation software Labelimg to annotate, and save the information as an .xml file, and finally make a VOC format data set, which contains information such as image name, width, height, class name, object, bounding box coordinates, etc.;

[0115] Step 4: Adjust the bottleneck parameters based on the ResNet50 model, replace the ordinary convolution with the depthwise separable convolution, and obtain the improved ResNet50 model;

[0116] Step 5: Introduce the Silu activation function into the model in step 4;

[0117] Step 6: Integrate the collaborative attention mechanism into the model in step 5 to build a Scoord-ResNet37 classification model based on deep learning;

[0118] Step 7: Use the training set to train the Scoord-ResNet37 classification model obtained in step 6;

[0119] Step 8: Use the multi-category classification evaluation index as the evaluation index to evaluate the classification effect of the Scoord-ResNet37 network model after training in step 7;

[0120] Step 9: Construct the Grad-CAM mathematical model to visualize each layer of the Scoord-ResNet37 network model, present the network's focus area in the form of a heat map to determine whether the feature information learned by the layer is correct, and then evaluate the classification performance of the Scoord-ResNet37 network model. If the model passes the evaluation, proceed to step 10, otherwise return to step 4;

[0121] Step 10: Preprocess the apple image to be classified, input the preprocessed apple image into the Scoord-ResNet37 network model and obtain the classification result.

[0122] Example 2

[0123] The present invention provides an apple multi-feature online classification method. The method is implemented by the following steps:

[0124] Step 1, data collection: collect data and select apple sample images without background interference and with complete fruit shape to form an apple dataset;

[0125] The specific operation is: collect apple images with no background interference and complete fruit shape through field collection, online crawling and public data sets, quantitatively analyze the color characteristics of apples, and classify them according to the different coloring rates, and divide red, yellow and green apples into three levels, namely Ⅰ, Ⅱ and Ⅲ;

[0126] Step 2: Data enhancement: The original apple data set obtained in step 1 is enhanced. The original data set in step 1 is enhanced by flipping, rotating, adding Gaussian noise and gamma transformation.

[0127] Step 3, data annotation: randomly divide the data set obtained after data enhancement into training set and validation set in a ratio of 8:2, use the annotation software Labelimg to annotate, and save the information as an .xml file, and finally make a VOC format data set, which contains information such as image name, width, height, class name, object, bounding box coordinates, etc.;

[0128] The specific operation is as follows: the data set obtained after data enhancement is randomly divided into a training set and a validation set, and the labeling software Labelimg is used to label the image. The labeling information includes image name, width, height, class name, object, and bounding box coordinates. The information is saved as an .xml file, and finally a VOC format data set is produced.

[0129] Step 4: Based on the ResNet50 model, adjust the bottleneck parameters and replace the ordinary convolution with the depthwise separable convolution to obtain the improved ResNet50 model; Figure 1 and Figure 2 As shown;

[0130] The specific operation is as follows: adjust some parameters of ResNet50, first traverse 1×1 convolution after inputting the feature image to reduce the channel dimension, then use batch normalization function and activation function to alleviate gradient vanishing and improve the generalization ability of the model, then replace the standard convolution with a depth-separable convolution with a convolution kernel size of 3×3 to speed up feature extraction, and finally perform 1×1 convolution on the fused feature map to perform channel scaling, and comprehensively improve the basic module of the residual network;

[0131] like Figure 3 As shown, the specific operation of the depth-separable convolution is as follows: Assume that the input feature map size is H×W×C i , First, in the channel-by-channel convolution process, a 3×3 convolution kernel is used to perform point multiplication and summation. The calculation amount of this process is as shown in formula (1):

[0132] S1=H×W×C i×3×3 (1)

[0133] Secondly, perform point-by-point convolution and output the feature map H×W×C after channel convolution. i Perform point-by-point convolution with a convolution kernel size of 1×1, fuse the features of different channels and change the number of channels of the output feature map. The output feature map has a size of H×W×C. The calculation amount of this process is as shown in formula (2):

[0134] S2=1×1×C i ×H×W×C (2)

[0135] Among them, H and W are the height and width of the input feature map, C i is the number of channels of the input feature map, and C is the number of channels of the output feature map;

[0136] Step 5: Introduce the Silu activation function into the model in step 4;

[0137] The specific operation is: perform a convolution operation with a kernel size of 7×7 on the input image, then traverse the BN function to normalize the input data, then use the Silu nonlinear activation function to replace the original activation function, and then perform subsequent operations. The Silu nonlinear activation function and its derivative function are shown in formulas (3), (4), and (5):

[0138] Silu=x×sigmoid(x) (3)

[0139]

[0140] Silu=Silu+sigmoid(1-Silu) (5)

[0141] Where sigmoid(x) is the sigmoid activation function;

[0142] Step 6: Integrate the collaborative attention mechanism into the model in step 5 to build a Scoord-ResNet37 classification model based on deep learning;

[0143] The collaborative attention mechanism CA module is integrated into the model of step 5. The specific operation is as follows: for the input feature map F∈R C×H×W , first perform average pooling operations in the horizontal and vertical directions to obtain a 1-dimensional vector and Then perform Concat on the spatial dimension to obtain the new feature map C h and C w , and then encode the spatial information in the vertical and horizontal directions through BN and Non-linear functions to solve the problem of data distribution changes in the middle layer during training and prevent the gradient from disappearing or exploding. At this time, the feature map is shown as follows:

[0144]

[0145] Then perform the split operation to decompose f2 into and Each convolution operation with a convolution kernel size of 1×1 is performed to standardize the number of feature map channels to make it consistent with the input, and finally perform normalization and weighting to obtain:

[0146]

[0147] in, is the activation function, is a 1×1 convolution transformation function, f2 is the intermediate feature map of spatial information in the vertical and horizontal directions, and are two tensors decomposed by f2, f x is the attention weight distribution value, f h and f w are transformed into tensors with the same number of channels as the input X using equations (8) and (9), respectively. y is the output value obtained by the final expanded output formula (9);

[0148] The Scoord-ResNet37 classification model is as follows: the first BN batch normalization function is used to normalize the input data, that is, the input variables of each neuron in each layer of the neural network are forced to return their distribution characteristics to the standard normal distribution; so that the activation input value can generate effective feedback in the sensitive area of ​​the activation function, thereby avoiding the problem of gradient disappearance and accelerating the convergence speed and training speed in the model learning process;

[0149] Silu activation function is used for feature extraction to accelerate the convergence of the model. Maximum pooling with a step size of 2 and a pooling kernel size of 3×3 is used to perform feature screening and information filtering operations on the feature map output by the convolutional layer to improve the generalization ability and computing performance of the model.

[0150] Input the Resbolock feature extraction module of Layer1. This layer uses two bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs.

[0151] The first CA collaborative attention mechanism is used to capture cross-channel interactive information, so that the model extracts multi-scale information, emphasizes key details and suppresses redundant details, enhances representation strength and avoids feature loss caused by dimensionality reduction;

[0152] Input the Resbolock feature extraction module of Layer2. This layer uses three bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs.

[0153] The second CA collaborative attention mechanism is used to enhance the representation strength and avoid feature loss caused by dimensionality reduction. The Resbolock feature extraction module of Layer3 is input. This layer uses a 5-bottleneck residual structure to improve classification and recognition accuracy and reduce computational costs.

[0154] Input the Resbolock feature extraction module of Layer4. This layer uses two bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs. It uses activation functions to promote the stable propagation of gradients and accelerate the training process of the model. It uses global average pooling with a pooling kernel size of 7×7.

[0155] Information is extracted from the data output by the last bottleneck, and the image size of the output data is 1×1×2048; the output of the pooling layer is expanded into a one-dimensional vector using a fully connected layer. The output layer consists of a fully connected layer of 1000 nodes. Softmax is used as a classifier to divide the target variable into multiple categories. After accepting the feature matrix input by the fully connected layer, the output input image corresponds to the respective probability values ​​of each category.

[0156] The multi-category classification evaluation indicators include Top1 accuracy or Top5 accuracy, precision, recall and F1 score as evaluation indicators. Top1 accuracy indicates the accuracy of the first-ranked category and the actual result. Top5 accuracy indicates the accuracy of the top five categories including the actual result. The calculation formulas of other evaluation indicators are as follows:

[0157]

[0158] Among them, TP is true positive, FP is false positive, and FN is false negative;

[0159] Step 7: Use the training set to train the Scoord-ResNet37 classification model obtained in step 6;

[0160] Step 8: Use the multi-category classification evaluation index as the evaluation index to evaluate the classification effect of the Scoord-ResNet37 network model after training in step 7;

[0161] Step 9: Construct the Grad-CAM mathematical model to visualize each layer of the Scoord-ResNet37 network model, present the network's focus area in the form of a heat map to determine whether the feature information learned by the layer is correct, and then evaluate the classification performance of the Scoord-ResNet37 network model. If the model passes the evaluation, proceed to step 10, otherwise return to step 4;

[0162] Step 10: Preprocess the apple image to be classified, input the preprocessed apple image into the Scoord-ResNet37 network model and obtain the classification result.

[0163] Example 3

[0164] Step S1: Data collection and data preprocessing

[0165] The present invention collects data through on-site collection, online crawlers and public data sets, and selects 11,534 images with no background interference and complete fruit shape, 17 categories, including 561 defective apple sample images. In addition, this paper quantitatively analyzes the color characteristics of apples, and classifies them according to different coloring rates, dividing red, yellow and green apples into three levels. It ensures that the sources of apple data are diverse and the distribution area is wide; and the original data is enhanced by flipping, rotating, adding Gaussian noise and gamma transformation to optimize the model training effect, enhance its generalization ability and thus improve the classification accuracy.

[0166] When performing the gamma transformation, the image pixel values ​​are first mapped to [0,1], which ensures the stability of the transformation while avoiding pixel value overflow. After adjusting the parameters many times to achieve the best image enhancement effect, the final parameter values ​​are: c=1, σ=0.5 and σ=2. 115,340 apple images were collected through data enhancement; the data set obtained after data enhancement was randomly divided into training sets and validation sets, and run 10 times at ratios of 7:3, 8:2 and 9:1. When the ratio is 8:2, the variance of the verification accuracy is the smallest and is 0.0031. Therefore, the present invention selects the training set and validation set 8:2 for calculation; use the annotation software Labelimg for annotation, and save the information as an .xml file, and finally make it into a VOC format dataset. It contains information such as image name, width, height, class name, object, and bounding box coordinates;

[0167] Step S2, optimize the network model

[0168] In order to improve the accuracy and stability of the apple classification method and meet the lightweight requirements of embedded online classification, the present invention selects ResNet50 as the basic model to improve and develop a new classification detection model. Figure 1As shown in the figure, ResNet50 consists of six parts: input module, 4 layers and output module. It uses skip connection to learn network features to alleviate the problems of gradient explosion and gradient disappearance. The feature F(x) learned by the weight layer is added to the feature x mapped to the identity and then passed through the activation function to obtain:

[0169] y=g(F(x)+x)~g(x)=relu(x)=x

[0170] While satisfying the requirement of enhanced feature expression capability, the network time complexity is reduced and the model calculation efficiency is improved. On this basis, the network parameters are adjusted and a Scoord-ResNet37 classification model based on deep learning is proposed.

[0171] like Figure 2 As shown in the figure, deep separable convolution is used to replace ordinary convolution to reduce the computational cost and improve the computational speed; the Silu activation function is introduced to make the model more self-stable in judging categories; the collaborative attention mechanism is integrated into ResNet50 to achieve better performance of the classification model without changing the initial receptive field; Gamma transformation and other methods are used in data preprocessing.

[0172] The following steps are included:

[0173] Step S2-1, optimize the basic module residual network ResNet50, which consists of 50 convolutional layers and fully connected layers, and consists of six parts: input module, 4 layers and output module. In order to prevent the model from overfitting due to the deepening of the number of network layers, improve the classification and recognition accuracy and reduce the computing cost. Adjust some parameters of the residual network. When the number of bottlenecks in Layer1 to Layer4 is adjusted to 2, 3, 5, and 2 respectively, that is, when the number of residual network layers is adjusted to 37 layers, the model not only improves the calculation speed in the classification and recognition process, but also meets the requirements of embedded devices for a small number of parameters. The present invention improves the residual structure Resblock such as Figure 2 , the input features first traverse 1×1 convolution to reduce the channel dimension; then use batch normalization function and activation function to alleviate gradient vanishing and improve the generalization ability of the model; then replace the standard convolution with a depth-separable convolution with a convolution kernel size of 3×3 to speed up feature extraction; finally, perform 1×1 convolution on the fused feature map to scale the channels. Comprehensively improve the basic module of the residual network to improve the network computing efficiency while sacrificing less accuracy or even keeping the accuracy unchanged.

[0174] Step S2-2, optimize convolution

[0175] Depthwise separable convolution is a structure that separates standard convolution into depthwise convolution and pointwise convolution.

[0176] like Figure 3 As shown, the input feature map size is H×W×C i , First, in the channel-by-channel convolution process, a 3×3 convolution kernel is used to perform point multiplication and summation. The amount of calculation in this process is:

[0177] S1=H×W×C i ×3×3

[0178] Secondly, perform point-by-point convolution and output the feature map H×W×C after channel convolution. i , perform point-by-point convolution with a convolution kernel size of 1×1, fuse the features of different channels while changing the number of channels of the output feature map, and output a feature map of size H×W×C. The computational complexity of this process is:

[0179] S1=1×1×C i ×H×W×C

[0180] Now we compare the parameters of depthwise separable convolution and standard convolution, and we get

[0181]

[0182] Compared with depthwise separable convolution, the standard convolution process generates a large number of parameters, increases the difficulty of network training and is not suitable for processing large-scale features. In this experiment, the characteristic scale features of apples cannot extract enough information, making the classification task difficult. Therefore, the present invention adopts depthwise separable convolution for feature extraction.

[0183] Step S2-3, optimize the activation function

[0184] The neural network basic model selected by the present invention adopts the linear activation function Relu. Although in the convolutional neural network training process, the input and output of each layer are linear summation processes, and the output of the next layer inherits the linear transformation of the input function of the previous layer, the pure linear combination is prone to incomplete feature information extraction in the face of complex classification and recognition problems, resulting in the omission of certain key features in the learning process. Therefore, this aspect performs a convolution operation with a convolution kernel size of 7×7 on the input image, and then traverses the BN function to normalize the input data, and then uses the Silu nonlinear activation function to replace the original Relu activation function, and then performs subsequent operations. The Silu nonlinear activation function and its derivative function are:

[0185] Silu=x·sigmoid(x)

[0186]

[0187] Silu=Silu+Sigmoid(1-Silu)

[0188] Where sigmoid(x) is the sigmoid activation function.

[0189] Compared with the Sigmoid function and the Relu function, the Silu activation function is a relatively smoother, more continuous, differentiable, and easier to optimize nonlinear activation function. It also has good self-stabilizing properties, making it easier to optimize the neural network and reduce the gradient vanishing problem.

[0190] like Figure 4 As shown in the figure, the Silu derivative function has a global minimum of about -0.28, which retains negative inputs while suppressing the update of large weights. This feature effectively prevents gradient explosion in the model training process. The introduction of nonlinear activation functions also helps to pass the understanding of the complexity and similarity of the data obtained in the previous layer to the input of the next layer, accelerates the convergence of the model, and effectively prevents the model from overfitting due to the limited features extracted from limited training samples.

[0191] Step S2-3, fusion attention mechanism

[0192] Apple classification is a typical multi-feature classification problem, which makes a comprehensive judgment based on the shape, color, texture, and whether the fruit is defective. For the situation where the colors of different varieties are similar but the color fullness is different and the texture features are similar, the attention mechanism is added to make the model focus on important information in the input data during training, and give higher weights to key features to improve the classification accuracy of the model.

[0193] Compared with the SE and CBAM attention mechanism modules, the CA module has small computational complexity, is flexible and lightweight, and meets the needs of embedded devices for lightweight networks. SE only considers internal channel information but ignores the importance of position information. CBAM obtains local information but cannot obtain widely reliable information. The CA module captures cross-channel interaction information, making the model extract multi-scale information, and also includes direction-aware and position-sensitive information, so that the model can more accurately locate and identify target areas. Therefore, the invention adopts a coordinated attention mechanism (CoordinateAttention) to emphasize key details and suppress redundant details to demonstrate its unique capabilities and advantages in enhancing representation strength, thereby avoiding feature loss due to dimensionality reduction. The main structure is as follows Figure 5 As shown, the process is as follows Figure 6 shown.

[0194] The CA module takes the input feature map F∈R C×H×W , first perform average pooling operations in the horizontal and vertical directions to obtain a 1-dimensional vector and Then perform Concat on the spatial dimension to obtain the new feature map C h and C w , and then encode the spatial information in the vertical and horizontal directions through BN and Non-linear functions to solve the problem of data distribution changes in the middle layer during training and prevent the gradient from disappearing or exploding. At this time, the feature map is shown as follows:

[0195]

[0196] Then perform the split operation to decompose f2 into and Each convolution operation with a convolution kernel size of 1×1 is performed to standardize the number of feature map channels to make it consistent with the input, and finally perform normalization and weighting to obtain:

[0197]

[0198]

[0199] in, is the activation function, is a 1×1 convolution transformation function, f2 is the intermediate feature map of spatial information in the vertical and horizontal directions, and are two tensors decomposed by f2, f x is the attention weight distribution value, f h and f w are transformed into tensors with the same number of channels as the input X using equations (8) and (9), respectively. y is the output value obtained by the final expanded output formula (9).

[0200] Step S3, the feature extraction module process of the apple post-harvest multi-feature classification model based on the improved Scoord-ResNet37 algorithm of the present invention is as follows: input the data image of the apple fruit with a size of 224×224×3 after preprocessing into the network, connect 64 convolution kernels, where the convolution kernel size is 7×7 and the step size is 2, so as to perform convolution feature extraction and integrate local information to improve the image processing performance. The final output data image size is 112×112×64; the input data is normalized using the first BN batch normalization function. That is, for the input variables of each neuron in each layer of the neural network, their distribution characteristics are forced to return to the standard normal distribution. This allows the activation input value to generate effective feedback in the sensitive area of ​​the activation function, thereby avoiding the problem of gradient disappearance and accelerating the convergence speed and training speed in the model learning process;

[0201] The Silu activation function is used for feature extraction. This nonlinear activation function is easier to optimize and accelerates the convergence of the model. The maximum pooling with a step size of 2 and a pooling kernel size of 3×3 is used to implement feature screening and information filtering operations on the feature map output by the convolutional layer, which helps to improve the generalization ability and computing performance of the model. The Resbolock feature extraction module of Layer1 is input. This layer uses 2 bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs. The first CA collaborative attention mechanism is used to capture cross-channel interaction information, so that the model extracts multi-scale information, emphasizes key details and suppresses redundant details, enhances the representation strength and avoids feature loss caused by dimensionality reduction. The Resbolock feature extraction module of Layer2 is input. This layer uses 3 bottleneck residual structures. eck residual structure to improve classification and recognition accuracy and reduce computational costs; use the second CA collaborative attention mechanism to enhance representation strength and avoid feature loss caused by dimensionality reduction; input the Resbolock feature extraction module of Layer3, which uses 5 bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs; input the Resbolock feature extraction module of Layer4, which uses 2 bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs; use activation functions to promote stable propagation of gradients and accelerate the training process of the model; use global average pooling with a pooling kernel size of 7×7 to extract information from the data output by the last bottleneck, and the image size of the output data is 1×1×2048;

[0202] A fully connected layer is used to expand the output of the pooling layer into a one-dimensional vector. The output layer consists of a fully connected layer with 1000 nodes. Softmax is used as a classifier to divide the target variable into multiple categories. After accepting the feature matrix input by the fully connected layer, the output input image corresponds to the respective probability values ​​of the 17 categories in this experiment.

[0203] Step S4: Evaluation indicators and result analysis

[0204] To evaluate the performance of a multi-class classification model, accuracy (Top1 accuracy or Top5 accuracy), precision, recall and F1 score are usually selected as evaluation indicators. Top1 accuracy indicates the accuracy of the first-ranked category and the actual result. Top5 accuracy indicates the accuracy of the top five categories including the actual result. The calculation formulas for other evaluation indicators are:

[0205]

[0206] Among them, TP is true positive, FP is false positive, and FN is false negative.

[0207] like Fig. 9 , 10 As shown in Figures 11 and 12, the precision, recall and confusion matrix results of the improved Scoord-ResNet37 network model of the present invention on the validation set. Since the image data of 11 types of apples, including Fuhuapi, Jonagold, Red Delicious, Red Apple III, Red Apple II, Red Apple I, Granny Smith, Yellow-Green Apple I, Yellow-Green Apple II, Yellow-Green Apple III and Braeburn, are from open source data sets and background interference is eliminated, the classification accuracy of the Scoord-ResNet37 network model has reached 100%, which shows that the improved model classification performance is superior without background interference; and the accuracy of the self-made data of Luochuan, Huang Yuanshuai, Red Fuji, Guoguang, Green Apple and Defective Apple is between 93% and 99%;

[0208] Especially for the classification and identification of defective apples, the accuracy is 93%. From the confusion matrix, it is clear that the model misclassified 6 defective apples into 5 Red Fuji and 1 Fuhuapi, because although they are corrupt and black, some of their textures and colors are still similar to healthy apples. The accuracy of Guoguang is 99%, and one Guoguang was misclassified as Huang Yuanshuai by the model, because the two have similar shapes and texture features.

[0209] Based on this dataset, the performance of the current mainstream neural network classification model and the improved model in this paper for the apple multi-classification problem is compared. The comparative experiment includes the networks AlexNet, EfficientNetV2, GhostNet, Resnet50, MobileNetV3, SuffleNetV2 and VGG16, and the test results of F1, precision, recall rate, Top1 and Top5 accuracy are obtained.

[0210] It can be seen that the improved Scoord-ResNet37 model in this paper has an accuracy improvement of 1.17% from 97.34% to 98.51% compared with ResNet50; the recall rate has increased from 97.76% to 98.77%, an increase of 1.01%. Compared with AlexNet, EfficientNetV2, GhostNet, MobileNetV3, SuffleNetV2 and VGG16, the accuracy has increased by 0.7%, 0.67%, 0.87%, 0.88%, 0.11% and 0.83% respectively;

[0211] The recall rates increased by 0.61%, 0.41%, 0.75%, 0.83%, 0.36% and 0.8% respectively. Among them, MobileNetv3 has a poor effect and is not suitable for multi-category apple recognition.

[0212] Therefore, the Scoord-ResNet37 model of the present invention has better performance in extracting apple color, size and texture features, with high accuracy and low computational complexity, meeting the needs of online classification of embedded devices. The proposed lightweight Scoord-ResNet37 classification model has strong classification effectiveness and robustness for multi-feature apple classification problems, with an accuracy of 98.51% under the test set, an increase of 1.17% compared to before the improvement; the recall rate is 98.77%, an increase of 1.01%.

Claims

1. Apple multi-feature online classification method, It is characterized in that Apple sample images are collected and preprocessed to form an apple dataset, which is then divided into a training set and a validation set. ResNet50 is used as the basic model, network parameters are adjusted, activation functions are optimized, and the collaborative attention mechanism is integrated into the ResNet50 model. A Scoord-ResNet37 classification model based on deep learning is constructed, and the constructed Scoord-ResNet37 is trained and the results are analyzed. The Scoord-ResNet37 model whose classification results meet the evaluation indicators is used to classify the apple images to be classified.

2. The multi-feature online classification method for apples according to claim 1, It is characterized in that Please follow the steps below to implement: Step 1, data collection: collect data and select apple sample images without background interference and with complete fruit shape to form an apple dataset; Step 2: Data enhancement: Perform data enhancement on the original apple dataset obtained in step 1; Step 3, data annotation: randomly divide the data set obtained after data enhancement into training set and validation set in a ratio of 8:2, use the annotation software Labelimg to annotate, and save the information as an .xml file, and finally make a VOC format data set, which contains image name, width, height, class name, object, and bounding box coordinate information; Step 4: Adjust the bottleneck parameters based on the ResNet50 model, replace the ordinary convolution with the depthwise separable convolution, and obtain the improved ResNet50 model; Step 5: Introduce the Silu activation function into the model in step 4; Step 6: Integrate the collaborative attention mechanism into the model in step 5 to build a Scoord-ResNet37 classification model based on deep learning; Step 7: Use the training set to train the Scoord-ResNet37 classification model obtained in step 6; Step 8: Use the multi-category classification evaluation index as the evaluation index to evaluate the classification effect of the Scoord-ResNet37 network model after training in step 7; Step 9: Construct the Grad-CAM mathematical model to visualize each layer of the Scoord-ResNet37 network model, present the network's focus area in the form of a heat map to determine whether the feature information learned by the layer is correct, and then evaluate the classification performance of the Scoord-ResNet37 network model. If the model passes the evaluation, proceed to step 10, otherwise return to step 4; Step 10: Preprocess the apple image to be classified, input the preprocessed apple image into the Scoord-ResNet37 network model and obtain the classification result.

3. The multi-feature online classification method for apples according to claim 2, It is characterized in that In the step 1, the specific operation is: collect apple images with no background interference and complete fruit shape through on-site collection, online crawlers and public data sets, quantitatively analyze the color characteristics of apples, and classify them according to different coloring rates, and divide red, yellow and green apples into three grades, namely I, II and III.

4. The multi-feature online classification method for apples according to claim 2, It is characterized in that In step 2, the original data set in step 1 is subjected to data enhancement by flipping, rotating, adding Gaussian noise and performing gamma transformation.

5. The multi-feature online classification method for apples according to claim 2, It is characterized in that In step 3, the specific operation is: randomly divide the data set obtained after data enhancement into a training set and a validation set, use the labeling software Labelimg to label, the labeling information includes image name, width, height, class name, object, bounding box coordinates, and save the information as an .xml file, and finally make it into a VOC format data set.

6. The multi-feature online classification method for apples according to claim 5, It is characterized in that In step 4, the specific operation is as follows: adjust some parameters of ResNet50, first traverse 1×1 convolution after the feature image is input to reduce the channel dimension, then use batch normalization function and activation function to alleviate gradient disappearance and improve model generalization ability, then replace standard convolution with depthwise separable convolution with convolution kernel size of 3×3 to speed up feature extraction, and finally perform 1×1 convolution on the fused feature map to perform channel scaling, and comprehensively improve the basic module of residual network; The specific operation of the depth-separable convolution is as follows: Assume that the input feature map size is H×W×C i , First, in the channel-by-channel convolution process, a 3×3 convolution kernel is used to perform point multiplication and summation. The calculation amount of this process is as shown in formula (1): S 1 =H×W×C i ×3×3 (1) Secondly, perform point-by-point convolution and output the feature map H×W×C after channel convolution. i Perform point-by-point convolution with a convolution kernel size of 1×1, fuse the features of different channels and change the number of channels of the output feature map. The output feature map has a size of H×W×C. The calculation amount of this process is as shown in formula (2): S 2 =1×1×C i ×H×W×C (2) Among them, H and W are the height and width of the input feature map, C i is the number of channels of the input feature map, and C is the number of channels of the output feature map.

7. The multi-feature online classification method for apples according to claim 2, It is characterized in that In step 5, the specific operation is: perform a convolution operation with a convolution kernel size of 7×7 on the input image, then traverse the BN function to normalize the input data, then use the Silu nonlinear activation function to replace the original activation function, and then perform subsequent operations. The Silu nonlinear activation function and its derivative function are specifically shown in formulas (3), (4), and (5): Silu=x·sigmoid(x) (3) Silu=Silu+sigmoid(1-Silu) (5) Where sigmoid(x) is the sigmoid activation function.

8. The multi-feature online classification method for apples according to claim 2, It is characterized in that In step 6, the collaborative attention mechanism CA module is integrated into the model of step 5. The specific operation is: for the input feature map F∈R C×H×W , first perform average pooling operations in the horizontal and vertical directions to obtain a 1-dimensional vector and Then perform Concat on the spatial dimension to obtain the new feature map C h and C w , and then encode the spatial information in the vertical and horizontal directions through BN and Non-linear functions to solve the problem of data distribution changes in the middle layer during training and prevent the gradient from disappearing or exploding. At this time, the feature map is shown as follows: Then perform the split operation to f 2 Decompose into and Each convolution operation with a convolution kernel size of 1×1 is performed to standardize the number of feature map channels to make it consistent with the input, and finally perform normalization and weighting to obtain: f y =f x ×f h ×f w (9) in, is the activation function, is the 1×1 convolution transformation function, f 2 It is the intermediate feature map of spatial information in the vertical and horizontal directions. and Yes 2 The two decomposed tensors, f x is the attention weight distribution value, f h and f w are transformed into tensors with the same number of channels as the input X using equations (8) and (9), respectively. y is the output value obtained by the final expanded output formula (9).

9. The multi-feature online classification method for apples according to claim 2, It is characterized in that In step 6, the Scoord-ResNet37 classification model is specifically as follows: the input data is normalized using the first BN batch normalization function, that is, the input variables of each neuron in each layer of the neural network are forced to return their distribution characteristics to the standard normal distribution; so that the activation input value can generate effective feedback in the sensitive area of ​​the activation function, thereby avoiding the problem of gradient disappearance and accelerating the convergence speed and training speed in the model learning process; Silu activation function is used for feature extraction to accelerate the convergence of the model. Maximum pooling with a step size of 2 and a pooling kernel size of 3×3 is used to perform feature screening and information filtering operations on the feature map output by the convolutional layer to improve the generalization ability and computing performance of the model. Input the Resbolock feature extraction module of Layer1. This layer uses two bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs. The first CA collaborative attention mechanism is used to capture cross-channel interactive information, so that the model extracts multi-scale information, emphasizes key details and suppresses redundant details, enhances representation strength and avoids feature loss caused by dimensionality reduction; Input the Resbolock feature extraction module of Layer2. This layer uses three bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs. The second CA collaborative attention mechanism is used to enhance the representation strength and avoid feature loss caused by dimensionality reduction. The Resbolock feature extraction module of Layer3 is input. This layer uses a 5-bottleneck residual structure to improve classification and recognition accuracy and reduce computational costs. Input the Resbolock feature extraction module of Layer4. This layer uses two bottleneck residual structures to improve classification and recognition accuracy and reduce computational costs. It uses activation functions to promote the stable propagation of gradients and accelerate the training process of the model. Use global average pooling with a pooling kernel size of 7×7; Information is extracted from the data output by the last bottleneck, and the image size of the output data is 1×1×2048; the output of the pooling layer is expanded into a one-dimensional vector using a fully connected layer. The output layer consists of a fully connected layer of 1000 nodes. Softmax is used as a classifier to divide the target variable into multiple categories. After accepting the feature matrix input by the fully connected layer, the output input image corresponds to the respective probability values ​​of each category.

10. The multi-feature online classification method for apples according to claim 2, It is characterized in that In step 6, the multi-category classification evaluation index includes selecting Top1 accuracy or Top5accuracy, Precision, Recall and F1 Score as evaluation indicators. Top1 accuracy indicates the accuracy of the first-ranked category consistent with the actual result, and Top5 accuracy indicates the accuracy of the top five categories including the actual result. The calculation formulas of other evaluation indicators are as follows: Among them, TP is true positive, FP is false positive, and FN is false negative.

Citation Information

Cited By

  • Non-destructive detection method for content of chlorophyll and carotenoid in tobacco leaves

    CN120510153A

  • Pump cavitation state intelligent identification method based on multi-scale fusion DCNN

    CN120995201A

  • Jujube appearance detecting and grading method and system based on machine vision

    CN120997825A

  • Apple identification method, device and equipment, medium and product

    CN121119824A