A Method for Identifying Yellow-Spotted Silkworm Cocoons by Combining Deep Learning and Image Processing

Abstract

The present invention discloses a method for identifying macular cocoon by combining deep learning and image processing, including: establishing a dataset of cocoon pictures containing good cocoons and macular cocoons; training the SE-ResNet network, and sending the pictures to be measured into the trained network for prediction; directly outputting the network recognition result when the confidence of the network recognition result is greater than or equal to 75%, and performing secondary image processing recognition when the confidence is less than 75%; converting the image into HSV format, classifying the image channels, performing threshold segmentation on the cocoon area in the S (saturation) single-channel picture, counting the area of the macular area, and counting the average pixel value of the macular area in the S-channel picture, that is, the color saturation of the macula; setting double thresholds for the macular area and a threshold for the macular color saturation, first performing double-threshold judgment on the area, then performing yellow saturation threshold judgment, and finally outputting the secondary image processing recognition result. The method for identifying macular cocoon by combining deep learning and image processing of the present invention can identify macular cocoons, save labor costs, and is a standardized detection method that is quantitative and qualitative and does not depend on the experience of the detector.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and digital image processing, and in particular to a yellow-spotted silkworm cocoon recognition method combining deep learning and image processing. Background Art

[0002] The silk industry in my country has always been an industry with rapid development and great demand. The market demand for silk cocoons, the raw materials of silk products, is also great. Therefore, the automated detection and identification of silk cocoon quality affects the production efficiency and product quality of the entire silk products industry. The silk reeling industry calls silk cocoons of qualified quality "top cocoons". During the process of silkworm breeding and cocooning, the excrement of silkworms will contaminate the cocoons, forming yellow spots on the clean and white cocoons. When the area of ​​the yellow spots is too large or the yellow color is too deep, the cocoons are called yellow spot cocoons, which are unqualified cocoons and need to be removed. Silk reeling cannot be carried out. At present, the silk reeling industry mainly uses manual observation to judge whether the cocoons with yellow spots are qualified cocoons. The industry generally believes that the cocoons with yellow spots greater than 0.5 square centimeters and deeper yellow spots are yellow spot cocoons, which cannot be reeled; the cocoons with yellow spots on the surface of the cocoons but not large in area (usually less than 1 square centimeter) and lighter in color are considered to be top cocoons, which can be reeled.

[0003] The identification method based on manual observation is inefficient, time-consuming, and greatly influenced by personal experience, which is not conducive to the standardization and qualitative and quantitative detection of silkworm cocoon quality in the industry. Summary of the invention

[0004] In order to solve the above technical problems existing in the prior art, the present invention proposes a yellow-spotted silkworm cocoon recognition method combining deep learning and image processing, comprising the following steps:

[0005] S1. Collect multiple images of qualified cocoons and unqualified yellow-spotted cocoons to establish a cocoon image dataset;

[0006] S2. Expand the cocoon image dataset;

[0007] S3. Send the silkworm cocoon image dataset into the SE-ResNet network for network training according to 80% training set and 20% test set;

[0008] S4. The cocoon image to be tested is input into the SE-ResNet network for recognition prediction. If the confidence of the prediction result is greater than 75%, the recognition result is directly output. If the confidence of the prediction result is not greater than 75%, digital image processing is performed for secondary recognition;

[0009] S5. For the image that needs secondary image processing, the area of ​​the macular area and the yellow saturation of the macular area are counted;

[0010] S6. Set double thresholds T2 and T1 for the macular area, a threshold T0 for yellow saturation. If the macular area is greater than T2, the secondary recognition result is a macular cocoon; if the macular area is less than T1, the secondary recognition result is an on-vehicle cocoon; if the macular area is greater than or equal to T1 and less than or equal to T2, then judge the yellow saturation. If the yellow saturation is greater than T0, the secondary recognition result is a macular cocoon; if the yellow saturation is less than or equal to T0, the secondary recognition result is an on-vehicle cocoon.

[0011] Further, in S2, for the expansion of the cocoon image dataset, image rotation, left-right mirroring of the image, up-down mirroring of the image, and cropping of the edge background are used. Finally, all image sizes are unified to 448*448 size to complete the establishment of the dataset.

[0012] Further, the number of layers of the SE-ResNet network in S3 is 50 layers. The recognition result is output for each picture and the confidence of the corresponding result is given.

[0013] Further, in S5, the RGB format cocoon picture is converted to HSV format, channel separation is performed, the S channel is analyzed, the S channel image is binarized, the area of the macular region is statistically calculated, and the average pixel value of the S channel in the macular region, that is, the yellow saturation, is recorded.

[0014] Further, in S6, set the macular area threshold T1 to 0.5 cm 2 , T2 to 1 cm 2 , and the color saturation threshold T0 to 75.

[0015] The method for identifying macular cocoons by combining deep learning and image processing of the present invention can identify macular cocoons, save labor costs, and is a standardized detection method that can be quantitatively and qualitatively determined regardless of the experience of the detector. Description of the Drawings

[0016] Figure 1 is the flowchart of the method for identifying macular cocoons by combining deep learning and image processing of the present invention;

[0017] Figure 2 is the framework diagram of the method for identifying macular cocoons by combining deep learning and image processing of the present invention.

[0018] Figure 3 is the structural diagram of the SE-ResNet network;

[0019] Figure 4 is the S channel diagram of the macular cocoon in HSV format. Specific Implementation Method

[0020] Combined with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present invention.

[0021] As Figure 1 and 2 shown, the method for identifying yellow-spotted silkworm cocoons by combining deep learning and image processing of the present invention includes the following steps:

[0022] (1) Establishment of silkworm cocoon image dataset

[0023] ① Collect a large number (more than 8,000) of images including qualified cocoons for processing and unqualified yellow-spotted cocoons with a black background to establish a silkworm cocoon image dataset. Manually identify and classify the silkworm cocoon image dataset according to the industry's identification standards and put them into two folders, indicating which are unqualified yellow-spotted cocoons and which are qualified cocoons for processing.

[0024] ② Expansion of the dataset. Perform left-right mirroring, up-down mirroring, image rotation, and cropping of the edge background on the collected images. Finally, unify the size of all the expanded datasets to 448*448 size to complete the expansion of the image dataset.

[0025] (2) Train the SE-ResNet neural network. Randomly extract 80% of the existing dataset as the training set and 20% as the validation set, and send them into the SE-ResNet network for training, and give the confidence level for the training results.

[0026] (3) The SE-ResNet network identifies the types of silkworm cocoons. Send the silkworm cocoon images to be identified and classified into the trained SE-ResNet network for prediction. For the images with a confidence level greater than 75% in the network prediction results, the final output identification result is the neural network identification result; for the images with a confidence level not greater than 75% in the network prediction results, perform secondary judgment by digital image processing.

[0027] (4) The results of deep learning are used for secondary classification by digital image processing. Images with a confidence level of the recognition results of the SE-ResNet network less than or equal to 75% are subjected to secondary image recognition. The area of the yellow spots in the cocoon images is statistically calculated, and double-threshold judgment is performed. If the area of the yellow spots is greater than T2 (1 square centimeter), then the cocoon is judged as a yellow-spotted cocoon; if the area of the yellow spots is less than T1 (0.5 square centimeter), then the cocoon is judged as a good-quality cocoon for processing; if the area of the yellow spots is greater than or equal to T1 (0.5 square centimeter) and less than or equal to T2 (1 square centimeter), then the yellow saturation of the yellow-spot area in the image is further judged. The image is converted into the HSV format, the channels of the HSV-format image are separated, an S-channel (color saturation) image is obtained, and the average color saturation of the yellow-spot area is judged. (The range of the S color saturation in the HSV space is 0-255). If the average color saturation of the S-channel component in the yellow area is greater than T0 (75), then it is judged as a yellow-spotted cocoon, otherwise it is judged as a good-quality cocoon for processing.

[0028] As Figure 3 shown, the SE-ResNet network has good results in multi-classification recognition. However, since the quality judgment of yellow-spotted cocoons is related to both the area of the yellow spots and the yellow saturation of the yellow spots, the accuracy of the results judged by the network only once is not high. By using the recognition results of the SE-ResNet network and performing secondary digital image processing on this recognition structure, the recognition accuracy and the overall robustness of the algorithm can be improved.

[0029] The main idea of ResNet is to add a direct connection channel in the network, that is, compared with ordinary networks, a short-circuit mechanism is added between every two layers. The neural network of the current layer can learn the residuals of the upper two layers of networks, which is similar to the short circuit in the circuit structure. It allows the original input information to be directly transmitted to the subsequent layers, that is, the network can perform residual learning.

[0030] SE-ResNet adds a Squeeze-and-Excitation feature compression and excitation module to the ResNet network. Adding this module has better non-linearity compared to directly using a fully connected layer network, and can reduce a lot of computational complexity and the number of parameters. The network used in this method has a total of 50 layers, and finally recognition and classification are performed through a fully connected layer and a softmax function.

[0031] After the cocoon image is input into the SE-ResNet network, the network will give a prediction result and the confidence level of this result. Since some cocoons with small yellow-spot areas or light yellow-spot colors also belong to good-quality cocoons for processing, the recognition rate of deep learning for images with small and light yellow-spot areas is not high, and the confidence level of the recognition results is low. A threshold needs to be set, and images with a confidence level less than 75% are subjected to secondary digital image processing.

[0032] Most of the pictures in a computer are displayed in RGB format. However, in some application scenarios, pictures in HSV format can better highlight certain features of an image. For example, in the S channel (i.e., color saturation) of an HSV image, the area with yellow spots on the surface of a white cocoon will have a significantly higher color saturation than the white part. An RGB image is converted through the following formula:

[0033] R′ = R / 255;

[0034] G′ = G / 255;

[0035] B′ = B / 255;

[0036] C max = max(R′, G′, B′);

[0037] C min = min(R′, G′, B′);

[0038] Calculation of H:

[0039]

[0040] Calculation of S:

[0041]

[0042] Calculation of V:

[0043] v = C max

[0044] According to the above conversion formula, an HSV format picture of the cocoon to be recognized for the second time is obtained. After channel separation, the S channel (i.e., color saturation channel) is analyzed separately. When there are yellow spots on a white-textured cocoon, the yellow spot area on the S-channel picture will be significantly different from the white area. The deeper the yellow color, that is, the higher the yellow saturation, the larger the corresponding pixel value on the S channel. The value of the white cocoon area on the S-channel picture is very small, while the value of the yellow spot area is relatively large. By counting the area of the region with high pixel values in the S channel, the area of the yellow spots on the surface of the cocoon can be obtained. According to the size of the pixel values of the corresponding regions of the yellow spots on the S-channel picture, the color saturation of the yellow spots can be obtained. The cocoon image is recognized for the second time based on the area of the yellow spots and the yellow color saturation.

Claims

1. A method for identifying macular silkworm cocoons by combining deep learning and image processing, characterized in that, It includes the following steps: S1. Collect multiple pictures of qualified cocoons for reeling and unqualified yellow-spotted cocoons to establish a cocoon image dataset; S2. Augment the cocoon image dataset; S3. Feed 80% of the cocoon image dataset as the training set and 20% as the test set into the SE-ResNet network for network training; S4. Input the pictures of cocoons to be tested into the SE-ResNet network for recognition and prediction. If the confidence level of the prediction result is greater than 75%, directly output the recognition result. If the confidence level of the prediction result is not greater than 75%, perform digital image processing for secondary recognition; S5. For the pictures that need secondary recognition by image processing, count the area of the yellow-spotted region and the yellow saturation of the yellow-spotted region; S6. Set the double thresholds T2 and T1 for the yellow-spotted area and the threshold T0 for the yellow saturation. If the yellow-spotted area is greater than T2, the secondary recognition result is a yellow-spotted cocoon; if the yellow-spotted area is less than T1, the secondary recognition result is a cocoon for reeling; if the yellow-spotted area is greater than or equal to T1 and less than or equal to T2, then judge the yellow saturation. If the yellow saturation is greater than T0, the secondary recognition result is a yellow-spotted cocoon. If the yellow saturation is less than or equal to T0, the secondary recognition result is a cocoon for reeling; The number of layers of the SE-ResNet network described in S3 is 50 layers. Output the recognition result for each picture and give the confidence level of the corresponding result. The SE-ResNet network is the ResNet network with a feature compression and excitation module added; In S5, convert the cocoon pictures in RGB format to HSV format, perform channel separation, analyze the S channel, binarize the S channel image, count the area of the yellow-spotted region, and record the average pixel value of the S channel in the yellow-spotted region, that is, the yellow saturation; 2. The method for identifying macular silkworm cocoons by combining deep learning and image processing according to claim 1, wherein: In S2, for the augmentation of the cocoon image dataset, use picture rotation, left-right mirroring of the picture, up-down mirroring of the picture, and crop the edge background. Finally, unify the size of all pictures to 448*448 size to complete the establishment of the dataset.

3. The method for identifying macular silkworm cocoons by combining deep learning and image processing according to claim 1, characterized in that: Set the macular area threshold T1 to 0.5 cm in S6 2 , T2 to 1 cm 2 , and the color saturation threshold T0 to 75.

Images