A method and system for classifying silkworm cocoon images based on ResNet18 and attention mechanism

Abstract

This application discloses a method and system for silkworm cocoon image classification based on ResNet18 and an attention mechanism, relating to the field of silkworm cocoon classification. The method includes: acquiring silkworm cocoon images; constructing a silkworm cocoon classification model based on a ResNet18 residual network and incorporating channel and spatial attention mechanisms; inputting the acquired silkworm cocoon images into the silkworm cocoon classification model for feature recognition to obtain the cocoon classification. Currently, early machine vision recognition solutions utilize image processing algorithms, primarily based on color and area feature differences, which struggle to distinguish cocoons with similar shapes. The method provided by this invention extracts multiple features from silkworm cocoons, including color, shape, area, and texture, eliminating the need for human intervention, reducing the subjectivity of human factors, and improving the efficiency and accuracy of silkworm cocoon classification while classifying lightly cottony cocoons and internally dyed cocoons.

Description

Technical Field

[0001] This method relates to the field of silkworm cocoon classification, specifically to a silkworm cocoon image classification method and system based on ResNet18 and attention mechanism. Background Technology

[0002] The sericulture and silk industry is a traditional Chinese national industry, an important industry for people's livelihood, and an industry with international competitive advantage. In order to produce high-quality raw silk, silkworm cocoons need to be sorted to remove inferior and second-grade cocoons, ensuring the quality of the raw materials for silk reeling. Traditional silkworm cocoon sorting relies mainly on manual labor, which is labor-intensive, inefficient, and easily affected by subjective factors such as the skill level and visual fatigue of the sorting personnel, resulting in low accuracy and poor batch-to-batch consistency.

[0003] Currently, machine vision technology can replace manual sorting of silkworm cocoons, and has advantages such as high efficiency, reliability, non-destructive processing, and good repeatability. Machine vision based on image processing algorithms aims to identify the type of silkworm cocoons by extracting features of various types of cocoons. However, based on the feature differences of cocoon color and area, it is easy to encounter the problem of difficulty in identifying cocoons with indistinct features or small feature differences. While machine vision technology based on deep learning can achieve classification of multiple types of silkworm cocoons, with the addition of deep learning networks, model degradation may occur, leading to a decrease in the accuracy of cocoon classification. To address this, a silkworm cocoon image classification method and system based on ResNet18 and attention mechanism is proposed, which can achieve classification of multiple types of silkworm cocoons while avoiding the problem of model degradation. Summary of the Invention

[0004] The purpose of this invention is to address the problems that machine vision based on image processing algorithms struggles to identify silkworm cocoons with subtle feature differences, and that current machine vision methods based on deep learning algorithms for identifying silkworm cocoons suffer from model degradation as the network size increases, leading to a decrease in silkworm cocoon classification accuracy. This invention provides a silkworm cocoon image classification method and system based on ResNet18 and an attention mechanism, thereby increasing silkworm cocoon classification accuracy and improving cocoon selection efficiency.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution, the steps of which specifically include:

[0006] S1: Obtain silkworm cocoon images, including training images of silkworm cocoons with category labels and images of silkworm cocoons to be classified;

[0007] S2: Establish a silkworm cocoon classification model, which includes a ResNet18 residual network and an attention mechanism. Train the silkworm cocoon classification model using silkworm cocoon training images with species labels.

[0008] S3: Input the silkworm cocoon image to be classified into the trained silkworm cocoon classification model to obtain the distribution probability of the silkworm cocoon image to be classified in each category. Set a preset cocoon probability threshold. If the cocoon distribution probability of the silkworm cocoon image to be classified is higher than the preset cocoon probability threshold, then the silkworm cocoon image to be classified is identified as a cocoon.

[0009] The training steps for the silkworm cocoon classification model in step S2 include:

[0010] S21: The silkworm cocoon training images with category labels are divided into a training set, a cross-validation set, and a test set. The silkworm cocoon training images in the training set are used to train the silkworm cocoon classification model; the silkworm cocoon training images in the cross-validation set are used to verify the adaptability of the silkworm cocoon classification model during training; and the silkworm cocoon training images in the test set are used to evaluate the silkworm cocoon classification model after training.

[0011] Furthermore, the specific method for dividing the silkworm cocoon training images with category labels into training set, cross-validation set and test set includes: dividing the silkworm cocoon training images under each category label into sets in a ratio of 3:1:1, and then merging them into training set, cross-validation set and test set.

[0012] S22: Add a channel and spatial attention mechanism after the second residual layer in the ResNet18 residual network layer of the silkworm cocoon classification model, and also add a channel and spatial attention mechanism after the fourth residual layer; the function of the channel and spatial attention mechanism is to enable the silkworm cocoon classification model to have the characteristic of recognizing key features;

[0013] S23: Add an activation function layer and a regularization layer before the fully connected layer of the ResNet18 residual network layer in the silkworm cocoon classification model; both the activation function layer and the regularization layer reduce the number of features in the input feature map, which not only reduces the amount of data computation, but also increases the ability of the silkworm cocoon classification model to adapt to unknown silkworm cocoon images;

[0014] S24: Input the test set into the silkworm cocoon classification model, evaluate the trained silkworm cocoon classification model, and select the final silkworm cocoon classification model.

[0015] Furthermore, to improve the accuracy of the silkworm cocoon classification model, it needs to be repeatedly trained. Images from the training set are input into the model for silkworm cocoon image recognition training, and the cross-validation set is used during training to verify the model's ability to recognize unknown silkworm cocoon image data. At the end of each training session, the gradient of that training session is calculated based on the results, and the training parameters in the silkworm cocoon classification model are optimized using backpropagation. Backpropagation refers to the process of adjusting the training parameters used in the next training session of the silkworm cocoon classification model based on the results of one training session.

[0016] Furthermore, the structure of the ResNet18 residual network layer in step S22 includes: a convolutional layer, a max pooling layer, a residual network layer, an activation function layer, a regularization layer, and a fully connected layer.

[0017] The ResNet18 residual network is a residual network with 18 weight layers, specifically including one convolutional layer, a residual network layer with 16 convolutional blocks, and one fully connected layer.

[0018] After receiving the silkworm cocoon image to be classified, the ResNet18 residual network inputs it into a convolutional layer for downsampling, resulting in a first feature map set with half the image size and an increased number of feature maps. The first feature map set is then input into a max-pooling layer to extract the maximum feature information, resulting in a second feature map set. This second feature map set is then input into the residual network layer of the ResNet18 residual network for further downsampling and channel increase. Simultaneously, through skip connections within the residual network layer, residual features between the input and output of the residual network layer are obtained, resulting in a third feature map set. This third feature map set is then input into an activation function layer and a regularization layer to filter feature information, resulting in a fourth feature map set. Finally, the fourth feature map set is input into a fully connected layer corresponding to the number of silkworm cocoon classifications to fuse all feature information, obtaining the classification result of the silkworm cocoon image to be classified. The classification result is then combined with a normalized exponential function and a cross-entropy loss function to obtain the gradient of the silkworm cocoon classification model for the current training iteration. The Adam optimizer is used to adaptively adjust the gradient, and backpropagation is used to optimize the parameters used for training in the silkworm cocoon classification model.

[0019] Furthermore, the residual network layer consists of a first residual layer, a second residual layer, a third residual layer, and a fourth residual layer. Each residual layer includes two residual blocks, each containing two convolutional blocks, an activation function layer, a skip connection layer, and an addition layer. Upon receiving input, the residual block performs downsampling and feature extraction through the convolutional blocks and activation functions to obtain a first residual factor. Simultaneously, it maintains or increases the number of channels through the skip connection layer, ensuring that the number of channels in the output second residual factor is consistent with that of the first residual factor. Finally, the first and second residual factors are added together to obtain an output with superimposed residuals, enabling the cocoon classification model to obtain the residual between the input and output of the residual network layer.

[0020] Furthermore, the channel and spatial attention mechanism in step S22 includes a channel attention mechanism and a spatial attention mechanism; the channel attention mechanism is used to increase the weight coefficients on different channels of the fifth feature map set, and the spatial attention mechanism is used to increase the weight coefficients on different features of the fifth feature map set, so that the model has the characteristic of focusing on the main features of the feature map.

[0021] Furthermore, the evaluation step of the silkworm cocoon classification model in step S24 includes: inputting the silkworm cocoon images of the test set into the silkworm cocoon classification model for a generalization experiment to obtain a confusion matrix between the predicted values ​​of the silkworm cocoon classification model for the test set and the true values ​​of the test set; calculating the accuracy, precision, and recall of the model based on the confusion matrix; calculating the harmonic mean F1-score of the model based on the precision and recall. The silkworm cocoon classification model is then evaluated using the accuracy, precision, and recall to obtain the final silkworm cocoon classification model.

[0022] Furthermore, in step S3, the cocoons below the probability threshold for cocooning are re-collected, and the distribution probabilities of each type of cocoon predicted by the cocoon classification model are sorted from largest to smallest; the type ranked first is set as the primary label, and the type ranked second is set as the secondary label; the distinguishing features of the primary and secondary label cocoons are determined, and the images of the cocoons to be classified are re-collected based on the distinguishing features, with corresponding image acquisition preprocessing and image acquisition methods added, and the re-collected images are enhanced before being input into the cocoon classification model for re-classification to obtain the classification result.

[0023] Furthermore, when acquiring images of the silkworm cocoons again, the added preprocessing operations include: when the main distinguishing feature between the main label and the secondary label is the color of the silkworm cocoon, increasing the light intensity on the silkworm cocoon; when the main distinguishing feature between the main label and the secondary label is texture and surface markings, enlarging the silkworm cocoon image and acquiring the enlarged silkworm cocoon texture image; when the main distinguishing feature between the main label and the secondary label is the shape of the silkworm cocoon and the area of ​​defects, increasing the recognition weight of the size and / or area of ​​the silkworm cocoon in the model.

[0024] The clarity and contrast of the re-acquired silkworm cocoon images are increased to improve their feature representation. Image denoising and sharpening are used to enhance the clarity of the re-acquired silkworm cocoon images; logarithmic transformation in nonlinear transformation is used to enhance their contrast. The re-acquired silkworm cocoon images with improved feature representation are then input into the silkworm cocoon classification model to obtain the final classification result.

[0025] The aforementioned silkworm cocoon image classification system based on ResNet18 and attention mechanism includes:

[0026] The image acquisition module acquires silkworm cocoon images, including training images of silkworm cocoons with category labels and images of silkworm cocoons to be classified.

[0027] The model training module establishes a silkworm cocoon classification model, which includes a ResNet18 residual network and an attention mechanism. The silkworm cocoon classification model is trained using silkworm cocoon training images with species labels.

[0028] The model classification module inputs the silkworm cocoon image to be classified into the trained silkworm cocoon classification model to obtain the distribution probability of the silkworm cocoon image in each category. A preset cocoon probability threshold is set. If the distribution probability of the silkworm cocoon image to be classified is higher than the preset cocoon probability threshold, then the silkworm cocoon image to be classified is identified as a cocoon.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The ResNet18 residual network used in this invention adds channel and spatial attention mechanisms after the second residual layer and after the fourth residual layer. This enables the input silkworm cocoon image to have weight coefficients added to different channels and different features of the image, giving the silkworm cocoon classification model the ability to focus on key features and increasing classification accuracy.

[0031] 2. While existing deep learning algorithms can classify various types of silkworm cocoons, model degradation and decreased classification accuracy occur with the increase of deep learning model networks. This invention utilizes the convolutional properties of the ResNet18 residual network to fuse image features and leverages its skip connection properties to obtain residuals between the input and output images, achieving the fusion of original image and residual features. This enhances the feature representation of the input image, improves the accuracy of the silkworm cocoon classification model, and avoids model degradation.

[0032] 3. In practice, it was found that the collected silkworm cocoon images were affected by factors such as lighting, image acquisition angle, and minimal differences in the distinguishing features of the cocoons. The cocoon classification model sometimes resulted in the inclusion of other types of cocoons in the classified cocoons. This invention uses the cocoon classification model to reclassify cocoons below the probability threshold for inclusion in the classification. Image enhancement techniques, such as increasing image contrast and clarity, are applied to the input images to enhance the distinguishing features of the cocoons, thereby improving classification accuracy. Attached image description:

[0033] Figure 1 This is a schematic diagram of the structure of a silkworm cocoon classification model;

[0034] Figure 2 This is a schematic diagram of the residual module.

[0035] Figure 3 A schematic diagram showing the distribution of the silkworm cocoon dataset used for model training and validation;

[0036] Figure 4 A schematic diagram of the channel and space structure for the channel-space attention mechanism;

[0037] Figure 5 This is a schematic diagram of the channel attention mechanism.

[0038] Figure 6 This is a schematic diagram of the spatial attention mechanism.

[0039] Figure 7 This is a confusion matrix diagram obtained after the model generalizes to the test set;

[0040] Figure 8 A graph showing the evaluation index values ​​obtained after the model generalizes to the test set;

[0041] Figure 9 This is the graph of a general logarithmic function; Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0043] In the description of this application, it should be noted that the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0044] Example 1:

[0045] This embodiment provides a silkworm cocoon image classification method based on ResNet18 and an attention mechanism, which includes the following steps:

[0046] S1: Obtain silkworm cocoon images, including training images of silkworm cocoons with category labels and images of silkworm cocoons to be classified;

[0047] Furthermore, the image acquisition device structure consists of a light source module, an industrial camera module, an experimental frame, and a computer module. The experimental frame is used to fix the light source module and the industrial camera module; the industrial camera module comprises two industrial cameras, one model MS-U500C with 5 megapixels and an MV-LD-8-3M-A lens; the other model MV-CE200-10UC with 20 megapixels and a LOMOSEN ZX-SF3528T lens. The industrial cameras are used to acquire training images of silkworm cocoons with species labels and images of silkworm cocoons to be classified from different positions and angles. The computer module is used to store the acquired silkworm cocoon images and to preprocess the images. Its processor is an Intel Xeon Gold 5222 CPU, 3.8GHz; its GPU is an NVIDIA Quadro RTX5000, 16GB; and its memory is 128GB.

[0048] In this embodiment, the silkworm cocoons used to collect training images of silkworm cocoons with type labels are pre-classified and labeled manually. The silkworm cocoons selected are silkworm cocoon varieties mainly promoted in Ningnan County, Yibin City, and Nanchong City of Sichuan Province, such as Chuanshan Shushui, Yuntan Xiong Cocoon, Yuntan Chuanshan, Yuejiang Xiong Can, Jiaolin Chuanshan, and Golden Cocoon. Through manual labeling, nine types of silkworm cocoons are classified, including board-printed cocoons, firewood-printed cocoons, maggot-pierced cocoons, light cotton cocoons, internally dyed cocoons, double-cocoons, wrinkled cocoons, yellow-spotted cocoons, and upper cocoons.

[0049] Images of each silkworm cocoon were acquired using the image acquisition device. Images of the nine types of cocoons were captured from different positions and angles on a single cocoon. To further increase the dataset size and ensure its usability and breadth, data augmentation techniques were used to expand the images. These techniques employed random scaling and cropping, random rotation angles, and random horizontal flipping to preprocess the image data. The preprocessed image data was then input into the cocoon classification model for training. In this embodiment, a total of 5792 cocoon images were acquired.

[0050] S2: Establish a silkworm cocoon classification model, which includes a ResNet18 residual network and an attention mechanism, and train the silkworm cocoon classification model using training images with species labels.

[0051] In this embodiment, to ensure the consistency of the silkworm cocoon data distribution and avoid the impact of additional biases introduced during the data partitioning process on the final results, this study employs stratified sampling to select silkworm cocoon image data for the cross-validation set and test set, ensuring a balanced data distribution ratio. The silkworm cocoon training images in the cross-validation set are used to verify the adaptability of the silkworm cocoon classification model during training. The training images in the test set are used to evaluate the silkworm cocoon classification model after training. The silkworm cocoon image data for each class is partitioned in a 3:1:1 ratio and then merged into a training set, cross-validation set, and test set. The total number of silkworm cocoon image samples is 5792, the number of training set samples is 3470, the number of cross-validation set samples is 1161, and the number of test set samples is 1161. The silkworm cocoon image data and distribution are as follows: Figure 3 As shown. Specifically, the stratified sampling involves dividing the population into non-overlapping strata, then independently drawing a certain number of individuals from each stratum according to a certain proportion, and finally combining the individuals drawn from each stratum as a sample.

[0052] In this embodiment, the ResNet18 residual network structure is as follows: Figure 1 As shown, it consists of one 7x7 convolutional layer (conv1), one max pooling layer, one residual network layer, one activation function layer, one regularization layer, and one fully connected layer (FC). The ResNet18 residual network is a residual network with 18 weight layers, specifically including one convolutional layer, one residual network layer with 16 convolutional blocks, and one fully connected layer.

[0053] In this embodiment, the acquired image is a color image, typically a 3-channel RGB structure. Before being input into the silkworm cocoon classification model for training, the image needs to be uniformly adjusted to 224×224 pixels. Therefore, the size of an input image is recorded as 3*224*224.

[0054] After receiving an image with a pixel size of 3*224*224, the ResNet18 residual network extracts features from it and outputs the image classification result. The image is then processed by a convolutional layer with a kernel size of 3*7*7, a stride of 2, and 64 output channels, resulting in a first set of feature maps with 64 channels and a pixel size of 112*112. This first set of feature maps is then processed by a max-pooling layer of size 64*3*3 with a stride of 2 to extract the maximum eigenvalue within a certain range from each feature map, resulting in a second set of feature maps with 64 channels and a pixel size of 56*56.

[0055] The second feature map set is further processed through residual network layers for image feature extraction and fusion, ultimately resulting in a third feature map set with a size of 512*7*7 after compression and increased channel count. The residual network layers include a first residual layer, a second residual layer, a third residual layer, and a fourth residual layer, with each residual layer comprising two residual blocks. The general structure of each residual block is as follows... Figure 2 As shown, the residual block includes two convolutional blocks, an activation function layer, a skip connection layer, and an additive layer. After receiving the input x, the residual block performs downsampling and feature extraction on x using two 3x3 convolutional blocks and a ReLU activation function, outputting F(x). Simultaneously, it performs a 1x1 convolution operation on the image via the skip connection layer, outputting x again. The sum of F(x) + x, after filtering feature values ​​through the activation function, is then used as the output of the entire residual block. This allows the cocoon classification model to obtain the residual between the input x and the output F(x) of the residual network layer. The 1x1 convolution unifies the number of channels in F(x) and x, achieving either dimensionality increase or decrease for the input x.

[0056] The third feature map set temporarily discards some feature values ​​randomly through activation function layers and regularization layers. This reduces the amount of data while increasing the model's adaptability to unknown silkworm cocoon images, outputting the fourth feature map set. The fourth feature map set is then converted into a one-dimensional vector through global average pooling. This one-dimensional vector is then processed by a fully connected layer with the same number of cocoon categories, fusing all feature information to obtain the classification result for the silkworm cocoon image to be classified. The obtained classification result is then used to calculate the probability distribution of the silkworm cocoon image under each category using the softmax function, completing the silkworm cocoon classification.

[0057] To measure the performance of the current model, the inventors introduced a normalized exponential function and a cross-entropy loss function. The cross-entropy loss function is calculated based on the probability distribution of the classified cocoons under each category. The derivative of this cross-entropy loss function yields the gradient for the current training iteration. To quickly and accurately calculate the cross-entropy loss function and its derivative, the inventors employed the Adam optimizer to adaptively adjust the gradient. The Adam optimizer is an adaptive optimization algorithm that automatically adjusts the gradient used for the next model training iteration based on historical gradients. To continuously optimize the cocoon classification model, it needs to be repeatedly trained. The gradient values ​​are used through backpropagation to optimize the training parameters of the cocoon classification model, resulting in the final cocoon classification model.

[0058] In this embodiment, the channel and spatial attention mechanism includes a channel attention mechanism and a spatial attention mechanism, such as... Figure 4 As shown. The channel attention mechanism is used to increase the weight coefficients of the feature map set on different channels, and the spatial attention mechanism is used to increase the weight coefficients of different features on the feature map set, enabling the model to focus on the main features. Considering that adding the channel and spatial attention mechanisms at different positions in the residual network layer will lead to changes in the model evaluation index, the inventors found in practice that adding the channel and spatial attention mechanisms to the second and fourth residual layers, respectively, yielded the best classification and recognition results for silkworm cocoon image data.

[0059] The Channel Attention Mechanism (CAM) is constructed as follows: Figure 5 As shown, CAM performs global average pooling and global max pooling on the input feature map to fuse all information from the feature map. Then, the results of global average pooling and global max pooling are processed separately using a shared fully connected layer. The two processed results are then added together, and the sigmoid function is used to calculate the weight (between 0 and 1) for each channel of the input feature map. Finally, this weight is multiplied by the input feature map, thus adding weights to different channels of the feature map.

[0060] The spatial attention mechanism (SAM) is composed as follows: Figure 6 As shown, SAM performs max pooling and average pooling on each channel of the input feature map to fuse information from the same feature point across different channels. Then, the max-pooled and average-pooled feature maps are stacked, and the number of channels is adjusted using a single convolution with one channel. The Sigmoid function is then used to calculate the weight (between 0 and 1) for each feature in the input feature map. Finally, this weight is multiplied by the input feature map, thus adding weights to different features on the feature map.

[0061] In this embodiment, the test set is used to input the silkworm cocoon classification model for a generalization experiment. The trained silkworm cocoon classification model is evaluated, and the final silkworm cocoon classification model is selected. The generalization experiment verifies the adaptability of the silkworm cocoon classification model, and the steps include:

[0062] S21: Input the test set and obtain the confusion matrix between the predicted values ​​of the silkworm cocoon classification model and the actual values ​​of the test set. The confusion matrix shows the correspondence between the predicted and actual categories. For multi-class classification problems, the confusion matrix is ​​an N×N matrix, where N is the number of categories. In the classification model, TP (True Positive) represents successfully predicting a positive sample as positive; TN (True Negative) represents successfully predicting a negative sample as negative; FP (False Positive) represents incorrectly predicting a negative sample as positive; and FN (False Negative) represents incorrectly predicting a positive sample as negative. Using N... i，j Let represent the value in the i-th row and j-th column of the confusion matrix (i, j = 1, 2, 3, ..., n). Then, for the k-th class sample...

[0063] TP k =N i，i i = k

[0064] TN k =∑N j，j j = 1, 2, 3, ..., n and j ≠ k

[0065] i=1, 2, 3, ...., n and i≠k

[0066] j = 1, 2, 3, ..., n and j ≠ k

[0067] TP k TN k FP k FN k The numbers represent the sample numbers corresponding to positive-to-positive, negative-to-negative, negative-to-positive, and positive-to-negative classifications for the k-th class of samples, respectively. In this embodiment, the confusion matrix predicted by the model for 1161 test samples is as follows: Figure 7 As shown.

[0068] S22: Calculate the accuracy, precision, and recall of the model based on the confusion matrix.

[0069] In this embodiment, accuracy refers to the percentage of all correctly judged samples out of the total observations. The calculation formula is as follows:

[0070]

[0071] In this embodiment, precision refers to the proportion of samples that the model classifies as positive, but which are actually positive samples. Precision analyzes the actual source of the prediction results and corresponds to the columns of the confusion matrix. In a multi-class classification model, the precision (Precision) for the k-th class is... k for:

[0072]

[0073] In this embodiment, recall refers to the proportion of samples that the model classifies as positive out of all positive samples. Recall is an analysis of the predicted distribution of actual samples, corresponding to the rows of the confusion matrix. For a multi-class classification model, the recall rate for the k-th class is... k for:

[0074]

[0075] S23: Calculate the model's F1-score based on precision and recall. The F1-score is the harmonic mean of the model's precision and recall. For multi-class models, the macro-F1 score is generally used to measure model precision. First, calculate the F1-score for each class, such as the k-th class:

[0076]

[0077] Then take the mean to calculate macro-F1:

[0078]

[0079] In this embodiment, the precision, recall, and F1 score results obtained from model testing are as follows: Figure 8 As shown in the figure, out of the 1161 silkworm cocoon images in the test set, a total of 16 samples were misclassified. The precision rate for the identification of the following types of cocoons was 100%: * **Chaiyin cocoon:** * **Zhichuan cocoon:** * **Qingmian cocoon:** * **Yinye cocoon:** * **Yinye cocoon:** * **Zhichuan cocoon:** * * **Shangchuan cocoon:** * * **Zhang ...

[0080] S3: Input the silkworm cocoon image to be classified into the trained silkworm cocoon classification model to obtain the distribution probability of the silkworm cocoon image to be classified in each category. Set a preset cocoon probability threshold. If the cocoon distribution probability of the silkworm cocoon image to be classified is higher than the preset cocoon probability threshold, then the silkworm cocoon image to be classified is identified as a cocoon.

[0081] In this embodiment, as Figure 7As shown, among the samples predicted as cocoons, there are other abnormal cocoons with similar appearances to the predicted cocoons that are mistakenly identified. Therefore, a probability threshold for the predicted cocoon type is preset. When the distribution probability of a cocoon being predicted as a predicted cocoon is lower than the predicted cocoon probability threshold, the predicted cocoons need to be re-collected, and the distribution probabilities of each type of cocoon predicted by the cocoon classification model are sorted from largest to smallest. The type ranked first is set as the primary label, and the type ranked second is set as the secondary label. The distinguishing features of the primary and secondary label cocoons are determined, including the color, texture and surface marks, shape and defect area of ​​the cocoons. The cocoons are re-collected, and corresponding preprocessing is added to the images of the cocoons based on the distinguishing features. The corresponding image acquisition method is added, and the re-collected cocoon images are enhanced. The images are then input into the cocoon classification model again for re-classification to obtain the final classification result.

[0082] When acquiring images again, different preprocessing operations need to be performed based on the distinguishing features of the primary and secondary labels, including: when the distinguishing feature of the primary and secondary labels is the color of the cocoon, increasing the light intensity on the cocoon; when the distinguishing feature of the primary and secondary labels is texture and surface markings, enlarging the cocoon image and acquiring the enlarged cocoon texture image; when the distinguishing feature of the primary and secondary labels is the shape of the cocoon and the area of ​​defects, increasing the recognition weight of the size and / or area of ​​the cocoon in the model.

[0083] In this embodiment, during the training of the silkworm cocoon classification model, it was found that the probability of some cocoons landing as predicted by the model was lower than the landing probability threshold, resulting in misclassification. In practice, it was found that the distinguishing features between the misclassified cocoon images and the actual landing cocoons were relatively similar. Therefore, it is necessary to enhance the feature representation of these cocoons with a landing probability below the threshold. In practice, increasing the contrast and clarity of the misclassified cocoons is used to enhance their feature representation.

[0084] In this embodiment, the contrast of the misjudged silkworm cocoons is increased by using a logarithmic transformation. This is achieved by taking the logarithm of the pixel values ​​in the image of the misjudged silkworm cocoons. The formula for the logarithmic transformation is as follows:

[0085] g(x, y) = c * log(1 + f(x, y))

[0086] Where g(x, y) are the transformed pixel values, f(x, y) are the original pixel values, and c is a constant used to control the contrast gain. A typical graph of a logarithmic function is shown below. Figure 9As shown in the figure, the smaller the original pixel value f(x, y), i.e., the darker the area of ​​the image, the greater the rate of change of the transformed pixel value; conversely, the larger the original pixel value f(x, y), i.e., the brighter the area of ​​the image, the smaller the rate of change of the transformed pixel value, and the more flat the function graph becomes. The main function of the logarithmic transformation is to expand the low grayscale portion of the image, revealing more detail in the low grayscale area, while compressing the high grayscale portion, reducing the detail in the high grayscale area, thereby enhancing the image if it is too dark.

[0087] In this embodiment, image sharpening is also used to improve the clarity of the misjudged silkworm cocoon image and the details such as the edges of the silkworm cocoons in the image. Since image sharpening enhances the edge features of the silkworm cocoons in the image, it is generally necessary to remove or reduce noise in the image first, and then perform sharpening.

[0088] In this embodiment, K-nearest neighbor filtering is used to remove noise from the misjudged silkworm cocoons. When filtering the pixels of the misjudged silkworm cocoon image, the most similar pixel values ​​within the K nearest neighbors of a given pixel are found. These K pixel values ​​are then averaged using a weighted average method, and the average value replaces the original pixel value, thereby suppressing noise and smoothing the image. The value of K needs to be chosen appropriately based on the characteristics of the noise and the size of the image; generally, an odd value is selected. The weighting coefficients are calculated based on the distance between pixels; pixels that are farther apart are assigned lower weights.

[0089] In this embodiment, the image sharpening of the misjudged silkworm cocoons employs both unsharpening filtering and high-gain filtering. The unsharpening filtering involves subtracting an unsharpened layer from the original silkworm cocoon image, thereby enhancing details such as edges. The formula for the unsharpening filtering is as follows:

[0090] g(x, y) = f(x, y) - f s (x, y)

[0091] Where f(x, y) is the input silkworm cocoon image, f s (x, y) is the unsharpened image obtained from the input cocoon image, equivalent to the low-frequency part of the cocoon image. g(x, y) is the high-gain image of the obtained cocoon image, which retains the low-frequency part of the cocoon image. Adding the high-gain image g(x, y) to the original input image f(x, y) can highlight edge features while preserving the features of the original input image. The formula for high-gain filtering is as follows:

[0092] f hb (x, y)=A*f(x, y)+K*g(x, y)

[0093] Where A and K are two proportionality coefficients, A≥0, 0≤K≤1. After applying logarithmic transformation to increase contrast, K-nearest neighbor algorithm to remove image noise, and unsharpening and high-amplification filtering to sharpen the image of the misjudged cocoons, the features of the obtained sharpened cocoon image are more obvious. After receiving the sharpened cocoon image, the cocoon classification model can extract more prominent features, better identify and distinguish cocoon images with very small feature differences, and improve the accuracy of cocoon sorting.

[0094] In this embodiment, a silkworm cocoon image classification system based on ResNet18 and an attention mechanism includes:

[0095] The image acquisition module acquires silkworm cocoon images, which include training images with category labels and silkworm cocoon images to be classified.

[0096] The model training module establishes a silkworm cocoon classification model, which includes a ResNet18 residual network and an attention mechanism. The silkworm cocoon classification model is trained using training images with species labels.

[0097] The model classification module inputs the silkworm cocoon image to be classified into the trained silkworm cocoon classification model to obtain the distribution probability of the silkworm cocoon image in each category. A preset cocoon probability threshold is set. If the distribution probability of the silkworm cocoon image to be classified is higher than the preset cocoon probability threshold, then the silkworm cocoon image to be classified is identified as a cocoon.

[0098] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the specific embodiments described above. Therefore, any modifications or equivalent substitutions to the present invention, as well as all technical solutions and improvements that do not depart from the spirit and scope of the invention, are covered within the scope of the claims of the present invention.

Claims

1. A method for classifying silkworm cocoon images based on ResNet18 and an attention mechanism, characterized in that, Includes the following steps: S1: Obtain silkworm cocoon images, including training images of silkworm cocoons with category labels and images of silkworm cocoons to be classified; S2: Establish a silkworm cocoon classification model, which includes a ResNet18 residual network and an attention mechanism. Train the silkworm cocoon classification model using silkworm cocoon training images with species labels. S3: Input the silkworm cocoon image to be classified into the trained silkworm cocoon classification model to obtain the distribution probability of the silkworm cocoon image to be classified in each category. Set a preset cocoon probability threshold. If the cocoon distribution probability of the silkworm cocoon image to be classified is higher than the preset cocoon probability threshold, then the silkworm cocoon image to be classified is considered to be a cocoon. The step S2, establishing and training the silkworm cocoon classification model, specifically includes: S21: The silkworm cocoon training images with category labels are divided into a training set, a cross-validation set, and a test set. The silkworm cocoon training images in the training set are used to train the silkworm cocoon classification model; the silkworm cocoon training images in the cross-validation set are used to verify the adaptability of the silkworm cocoon classification model during training; and the silkworm cocoon training images in the test set are used to evaluate the silkworm cocoon classification model after training. S22: Add channel and spatial attention mechanisms after the second residual layer in the ResNet18 residual network layer of the silkworm cocoon classification model, and also add channel and spatial attention mechanisms after the fourth residual layer. S23: Add an activation function layer and a regularization layer before the fully connected layer of the ResNet18 residual network layer in the silkworm cocoon classification model; S24: Input the test set into the silkworm cocoon classification model, evaluate the trained silkworm cocoon classification model, and select the final silkworm cocoon classification model; Step S3 further includes: The cocoons below the probability threshold for cocooning are re-collected, and the distribution probabilities of each type of cocoon predicted by the cocoon classification model are sorted from largest to smallest. The type ranked first is set as the primary label, and the type ranked second is set as the secondary label. The distinguishing features of the primary and secondary label cocoons are determined, and the images of the cocoons to be classified are re-collected based on the distinguishing features. The image collection method is increased, the re-collected images are enhanced, and then input into the cocoon classification model for re-classification to obtain the classification result. Contrast is increased by applying a logarithmic transformation to the re-acquired images. The formula for the logarithmic transformation is as follows: , Where x is the horizontal coordinate of a pixel in the image, and y is the vertical coordinate of a pixel in the image. These are the transformed pixel values. Here, c represents the original pixel value, and c is a constant used to control the contrast gain. Image sharpening is achieved by applying unsharpening filtering and high-gain filtering to the re-acquired image, increasing image clarity. The formula for unsharpening filtering is as follows: , in, For the input image, This is the unsharpened image obtained from the input image. For high-gain images, the formula for high-gain filtering is as follows: , Where A and K are two proportionality coefficients, A≥0, 0≤K≤1. This is a sharpened image.

2. The silkworm cocoon image classification method based on ResNet18 and attention mechanism as described in claim 1, characterized in that, The ResNet18 residual network includes convolutional layers, max pooling layers, residual network layers, activation function layers, regularization layers, and fully connected layers. After receiving the silkworm cocoon image to be classified, the ResNet18 residual network inputs it into a convolutional layer for downsampling, resulting in a first feature map set with half the image size and an increased number of feature maps. The first feature map set is then input into a max-pooling layer to extract the maximum feature information, resulting in a second feature map set. This second feature map set is then input into the residual network layer of the ResNet18 residual network for further downsampling and channel increase. Simultaneously, through the skip connections of the residual network layer, residual features between the input and output of the residual network layer are obtained, resulting in a third feature map set. This third feature map set is then input into an activation function layer and a regularization layer to filter feature information, resulting in a fourth feature map set. Finally, the fourth feature map set is input into a fully connected layer corresponding to the number of silkworm cocoon classifications to fuse all feature information, thus obtaining the classification of the silkworm cocoon image to be classified.

3. The silkworm cocoon image classification method based on ResNet18 and attention mechanism as described in claim 2, characterized in that, The residual network layer includes a first residual layer, a second residual layer, a third residual layer, and a fourth residual layer; each residual layer includes two residual blocks, and each residual block includes two convolutional blocks; the silkworm cocoon classification model obtains the residual features between the first feature map set and the second feature map set through the residual network layer. The ResNet18 residual network is a residual network with 18 weight layers, specifically including one convolutional layer, the residual network layer with 16 convolutional blocks, and one fully connected layer.

4. The silkworm cocoon image classification method based on ResNet18 and attention mechanism as described in claim 3, characterized in that, The channel and spatial attention mechanism includes a channel attention mechanism and a spatial attention mechanism; the channel attention mechanism is used to increase the weight coefficients on different channels of the fifth feature map set, and the spatial attention mechanism is used to increase the weight coefficients on different features of the fifth feature map set.

5. The silkworm cocoon image classification method based on ResNet18 and attention mechanism as described in claim 1, characterized in that, The distinguishing features between the main label and the sub-label of the cocoon include the color, area of ​​defects, texture, and shape of the cocoon.

6. The silkworm cocoon image classification method based on ResNet18 and attention mechanism as described in claim 1, characterized in that, Preprocessing includes increasing the light intensity on the silkworm cocoons, enlarging the texture images of the silkworm cocoons, and increasing the recognition weights for the size and / or area of ​​the silkworm cocoons in the model; increasing the clarity and contrast of the re-acquired silkworm cocoon images input to the silkworm cocoon classification model, and improving the feature representation of the re-acquired silkworm cocoon images.

7. A silkworm cocoon classification system based on ResNet18 and attention mechanism, characterized in that, include: The image acquisition module acquires silkworm cocoon images, which include training images of silkworm cocoons with category labels and images of silkworm cocoons to be classified. The model training module establishes a silkworm cocoon classification model, which includes a ResNet18 residual network and an attention mechanism. The silkworm cocoon classification model is trained using silkworm cocoon training images with species labels. The model classification module inputs the silkworm cocoon image to be classified into the trained silkworm cocoon classification model to obtain the distribution probability of the silkworm cocoon image to be classified in each category. A preset cocoon probability threshold is set. If the distribution probability of the silkworm cocoon image to be classified is higher than the preset cocoon probability threshold, then the silkworm cocoon image to be classified is considered to be a cocoon. The model training module also includes: The silkworm cocoon training images with category labels are divided into a training set, a cross-validation set, and a test set. The silkworm cocoon training images in the training set are used to train the silkworm cocoon classification model; the silkworm cocoon training images in the cross-validation set are used to verify the adaptability of the silkworm cocoon classification model during training; and the silkworm cocoon training images in the test set are used to evaluate the silkworm cocoon classification model after training. A channel and spatial attention mechanism is added after the second residual layer in the ResNet18 residual network layer of the silkworm cocoon classification model, and a channel and spatial attention mechanism is also added after the fourth residual layer. An activation function layer and a regularization layer are added before the fully connected layer of the ResNet18 residual network layer in the silkworm cocoon classification model; The test set is input into the silkworm cocoon classification model, the trained silkworm cocoon classification model is evaluated, and the final silkworm cocoon classification model is selected. The model classification module also includes: The cocoons below the probability threshold for cocooning are re-collected, and the distribution probabilities of each type of cocoon predicted by the cocoon classification model are sorted from largest to smallest. The type ranked first is set as the primary label, and the type ranked second is set as the secondary label. The distinguishing features of the primary and secondary label cocoons are determined, and the images of the cocoons to be classified are re-collected based on the distinguishing features. The image collection method is increased, the re-collected images are enhanced, and then input into the cocoon classification model for re-classification to obtain the classification result. Contrast is increased by applying a logarithmic transformation to the re-acquired images. The formula for the logarithmic transformation is as follows: , Where x is the horizontal coordinate of a pixel in the image, and y is the vertical coordinate of a pixel in the image. These are the transformed pixel values. Here, c represents the original pixel value, and c is a constant used to control the contrast gain. Image sharpening is achieved by applying unsharpening filtering and high-gain filtering to the re-acquired image, increasing image clarity. The formula for unsharpening filtering is as follows: , in, For the input image, This is the unsharpened image obtained from the input image. For high-gain images, the formula for high-gain filtering is as follows: , Where A and K are two proportionality coefficients, A≥0, 0≤K≤1. This is a sharpened image.

Images