Real-time detection method for frozen dumpling surface defects based on light YOLO_CBAM network model

Abstract

This invention discloses a real-time detection method for surface defects of frozen dumplings based on the LightYOLO_CBAM network model, comprising the following steps: First, creating a labeled dataset, marking the location and category of each frozen dumpling in each image; second, data preprocessing, including image scaling and data augmentation; third, constructing the LightYOLO_CBAM network model; fourth, training the LightYOLO_CBAM network model; fifth, saving the LightYOLO_CBAM network model; and sixth, using the LightYOLO_CBAM network model to perform real-time surface defect detection on frozen dumpling images input through a camera device. This invention improves the YOLOv4 object detection model, significantly enhancing the detection and recognition accuracy for each class of frozen dumplings, while greatly reducing the model size. The average recognition accuracy for each category is improved by 19.14%, and the number of images detected per second is improved by 16.9%, meeting the needs of real-time surface defect detection for quick-frozen dumplings in industrial production.

Description

Technical Field

[0001] This invention relates to the field of frozen food testing technology, and more particularly to a method for real-time detection of surface defects in frozen dumplings. Background Technology

[0002] With the development of technology and the acceleration of the pace of life, the complex traditional food preparation process can no longer meet the needs of today's fast-paced life. Frozen foods, due to their convenience, speed, and rich nutrition, are favored by consumers, especially young people and urban wage earners. They not only save a lot of cooking time, but also meet people's diverse needs for life.

[0003] Frozen foods are divided into three main categories: frozen rice and flour products, frozen hot pot ingredients, and other frozen products. In my country, frozen rice and flour products account for over 50% of the total frozen food market, while frozen hot pot ingredients and other frozen products account for 33.3% and 14.3%, respectively. Frozen rice and flour products are further subdivided into dumplings, glutinous rice balls, zongzi (sticky rice dumplings), and fried dough sticks, among others. Frozen dumplings account for 50% of the total sales of frozen rice and flour products, and their production and sales volume rank among the top in related food categories.

[0004] Frozen dumplings are frozen at around -30℃ and packaged, distributed, and sold at temperatures between -20℃ and -18℃. During the production process, instability in equipment, technology, and processes can cause surface defects in some dumplings. These defects severely affect the appearance, taste, and internal microbial content (external organisms can easily enter the dumpling after it breaks). Therefore, the appearance of frozen dumplings needs to be inspected before packaging to prevent substandard products from entering the market.

[0005] Meanwhile, mining and analyzing the data from the detection of surface defects in frozen dumplings is beneficial for enterprises to improve and upgrade related equipment, technologies, and processes. Currently, frozen food companies mainly rely on manual inspection for surface defects in frozen dumplings. Workers need to wear professional thermal work clothes and manually remove defects from frozen dumplings in an environment of -18℃ using a combination of hand and eye. This method presents the following main problems:

[0006] (1) Due to the high speed of the frozen dumpling conveyor belt, the manual detection rate is very low, generally less than 30%;

[0007] (2) It requires a high level of skill from the workers;

[0008] (3) Manual testing is subjective, and it is difficult to unify the standards, which can easily lead to false detections;

[0009] (4) The workers have high labor intensity, and visual fatigue can easily lead to missed detections;

[0010] (5) Workers working in low-temperature environments for extended periods are more likely to experience health problems.

[0011] The patent "A Visual Inspection Method and System for Dumplings" from the Luoyang Advanced Manufacturing Industry R&D Base of Tsinghua University Tianjin Advanced Equipment Research Institute discloses a deep learning-based detection method. This method, based on existing algorithms, uses labeled images for training to obtain a defect detection model. However, it fails to adapt the algorithm to the needs of dumpling inspection, resulting in lower detection accuracy. The method requires a high degree of color difference between the dumpling and the background (conveyor belt). Paragraph 24 of the patent specification also indicates that if the color difference between the dumpling and the background (conveyor belt) is small, the image is difficult to segment and identify correctly. To better distinguish defects from dumplings, this patent requires a color camera, specifically an industrial camera with an accuracy of 0.2 mm / pixel, and also imposes strict requirements on the light source, resulting in high overall equipment costs.

[0012] The technical approach of this invention lies in making targeted improvements to existing algorithms to reduce hardware requirements and achieve more accurate recognition performance compared to existing algorithms under the same hardware conditions.

[0013] YOLOv4 is a popular object detection model consisting of a backbone network, a fusion layer, and a head.

[0014] The fusion layer (Neck) contains an SPP layer, which is used to address the problem of inconsistent input image sizes, increase the model's receptive field, and enhance the model's ability to understand the target.

[0015] The YOLOv4 processing procedure is as follows: The image to be detected is fed into the backbone network, which extracts features from the image to obtain the extracted feature map. The feature map extracted by the backbone network is then passed to the fusion layer, which fuses feature maps from different levels and scales to capture target information at different scales, generating a fused feature map. The fused feature map is then passed to the detection head, which further processes the fused feature map to generate the target's bounding box and class probability, outputting the recognition result, including the target's class and location information.

[0016] The purpose of this invention is to improve the backbone network, fusion layer, and detection head of the YOLOv4 target detection model, thereby reducing the requirements for detection hardware and improving the recognition performance in the identification of surface defects of frozen dumplings. Summary of the Invention

[0017] The purpose of this invention is to provide a real-time detection method for surface defects of frozen dumplings based on YOLOv4 deep network structure, thereby improving the accuracy of surface defect identification in frozen dumplings.

[0018] To achieve the above objectives, the real-time detection method for frozen dumpling surface defects based on the LightYOLO_CBAM network model of the present invention is carried out according to the following steps:

[0019] The first step is to create a labeled dataset, in which the location and category of each frozen dumpling are labeled in each image of the labeled dataset;

[0020] The second step is data preprocessing, which involves scaling and data augmentation of the image.

[0021] The third step is to construct a LightYOLO_CBAM network model for real-time detection of surface defects in frozen dumplings.

[0022] The fourth step is to train the LightYOLO_CBAM network model;

[0023] The fifth step is to save the LightYOLO_CBAM network model;

[0024] The sixth step is to use the LightYOLO_CBAM network model to perform real-time surface defect detection on the frozen dumpling images input through the camera device.

[0025] In the first step, the frozen dumpling images were manually labeled using the open-source deep learning annotation tool LabelImg;

[0026] The location of each frozen dumpling is marked in each image of the labeled dataset. The area where each frozen dumpling is located is marked using the rectangle annotation in the LabelImg toolbar.

[0027] When categorizing frozen dumplings, seven categories are preset: Normal, Leak, Broken_belly, Broken_ends, Crack, Adhesion, and Malformation.

[0028] Each frozen dumpling was manually labeled to distinguish its category, forming a labeled dataset, and finally the labeled file was saved.

[0029] The second step involves scaling the image:

[0030] The width and height of each captured image of frozen dumplings were scaled up by the same ratio and filled with RGB 128 colors. The final scaled images had the same width and height and the image content was not distorted.

[0031] The second step involves data augmentation of the image, specifically:

[0032] The Mosaic data augmentation method is used to randomly select four images and stitch them together to form a new image, which contains the corresponding label information.

[0033] Image data augmentation involves three steps:

[0034] The first step is to randomly select four images from the labeled dataset.

[0035] The second step is to flip and scale the four images respectively, and then place the four processed images in the top left, bottom left, top right and bottom right positions of the four original-size canvases.

[0036] The third step is to extract the image regions from the four images and stitch them together into a new image. The label information of the four images is also recalculated and merged into the label information of the new image.

[0037] The third step specifically involves modifying the existing YOLOv4 object detection model, including:

[0038] ① For the backbone network, MobileNetv1 is used as the backbone network of the LightYOLO_CBAM network model, and its core is depthwise separable convolution.

[0039] ② For the fusion layer, i.e. the Neck, standard convolutional and depthwise separable convolutional alternating stacked modules CD_3 and CD_5 are used to replace the continuous standard convolutional modules CBL×3 and CBL×5 in PANet, respectively, as the fusion layer, i.e. the Neck, of the LightYOLO_CBAM network model;

[0040] ③ For the fusion layer, i.e., the Neck, use Softpool to build the SPP layer;

[0041] ④ For the detection head, the original YOLO Head 52×52 and YOLO Head 26×26 modules in the YOLOv4 object detection model are retained, and a YOLO Head 104×104 module is added. The YOLO Head 13×13 module is deleted, but the 13×13 feature map is retained. The shallow feature maps such as 104×104, 52×52, and 26×26 contain strong localization information. The fusion layer, or Neck, upsamples and transfers the strong semantic information contained in the 13×13 feature map to the shallow feature map for feature fusion. Finally, the fused feature maps of 104×104, 52×52, and 26×26 sizes are input into the detection head to improve the head's performance in locating and classifying defective dumplings.

[0042] ⑤ In the fusion layer, i.e., the Neck, the ReaPAN_CBAM module is used to input the 104×104, 52×52, 26×26 and 13×13 feature maps output from the last four DW_1 of the backbone network built by MobileNetv1 into the fusion layer, i.e., the Neck. The 13×13 feature map is passed through the SPP layer built by Softpool, and after upsampling, it is fused with the 26×26, 52×52 and 104×104 feature maps. After residual connection and downsampling operations, the fused 104×104, 52×52 and 26×26 feature maps are finally input into the detection head, i.e., the Head.

[0043] In the fusion layer, i.e., the Neck, a CBAM attention mechanism is embedded after the concat operation of upsampling and downsampling.

[0044] The experimental training parameters in step four are configured as follows: the image input size is 416×416 pixels, and the pre-trained model is MobileNetv1_1_0_224_tf.h5 trained on ImageNet; the training process is divided into two stages: freezing and not freezing the backbone network. The parameter configurations for these two stages are shown in the table below:

[0045]

[0046] In steps one through four, the LightYOLO_CBAM network model was created using Python 3.6 on the PyCharm platform for a program to detect surface defects in frozen dumplings. The deep learning object detection network, namely the LightYOLO_CBAM network model, was built using TensorFlow 1.31.1 and Keras 2.1.5. In step five, the final structure and parameters of the LightYOLO_CBAM network model were saved as a .h5 file.

[0047] The sixth step is as follows:

[0048] For real-time images of frozen dumplings from the frozen dumpling production line captured by the camera device, the image size is changed to a uniform size after data augmentation, and then normalized and preprocessed as the input image of the LightYOLO_CBAM network model. Then, after convolution operation and downsampling, the feature maps calculated by the last four depthwise separable convolutions with a stride of 1 in the backbone network are extracted. These four feature maps are respectively input into the fusion layer (Neck) for multi-scale feature fusion. Then, the fused 104×104, 52×52 and 26×26 feature maps are input into the detection head (Head) to obtain the dumpling's position coordinates, category and confidence level, as well as defect statistics.

[0049] The present invention has the following advantages:

[0050] 1. In this invention, depthwise separable convolution operations are used instead of standard convolution operations in the backbone and fusion layer Neck parts of the LightYOLO_CBAM network. That is, a lightweight MobileNetv1 network is used as the backbone and a lightweight PANet network is constructed as the neck, which greatly reduces the number of model parameters and the size of the network model file, thus achieving model lightweighting.

[0051] 2. In the SPP structure of LightYOLO_CBAM, this invention uses Softpool to construct the SPP, obtaining richer multi-scale defect feature information. The core of Softpool is to pool the feature map using an exponentially weighted summation method. It calculates the weights of each position in the local feature map based on the information of the local feature map with the pooling kernel size, and then calculates the pooling value of the local feature map by weighted summation of each position. The pooling kernel traverses the entire feature map according to the stride, finally obtaining the pooled feature map. Therefore, it not only reduces the feature map size but also integrates more surface feature information, which is beneficial for more defect semantic information to participate in subsequent operations.

[0052] 3. In the LightYOLO_CBAM output network Head part, the present invention constructs a Head (104×104) module, deletes the YOLO Head (13×13) module, but retains the 13×13 feature map. By upsampling, the strong semantic information contained in the 13×13 feature map is transferred to the shallow feature layer for feature fusion, which enhances the model's accuracy in identifying defects and improves the performance of the detection head (Head) in locating and classifying defective dumplings.

[0053] 4. In the Neck (fusion layer) part of LightYOLO_CBAM, this invention constructs a lightweight ResPAN_CBAM feature fusion network, which is a feature fusion network built on the basis of lightweight PANet and improved Head. It achieves full fusion of feature location information and semantic information through residual connections, upsampling and downsampling operations, and uses the attention mechanism CBAM module after the concat operation to improve the network's attention to defect features, thereby enhancing the model's attention to important features in the fusion stage.

[0054] 5. Improvements to the detection head avoid the problem of defects being less than one pixel, which is beneficial for the Neck to fuse features and for the Head to classify and locate defective dumplings.

[0055] Compared to the standard YOLOv4 network model, as shown in Table 2, the LightYOLO_CBAM network model of this invention significantly improves the detection and recognition accuracy for each class of frozen dumplings. As shown in Table 3, the model size is greatly reduced, and the mAP (average accuracy per class) of the LightYOLO_CBAM network model is improved by 19.14%, and the FPS (frames per second) is improved by 16.9%, meeting the needs of real-time detection of surface defects of quick-frozen dumplings in industrial production. Attached Figure Description

[0056] Figure 1 This is a flowchart of a real-time detection method for surface defects of frozen dumplings based on the LightYOLO_CBAM network model.

[0057] Figure 2 This is a diagram showing the frozen dumplings from the first step;

[0058] Figure 3 These are four images from the frozen dumpling dataset in the second step;

[0059] Figure 4 It is in the second step that... Figure 3 Four images of a dataset of quick-frozen dumplings after scaling;

[0060] Figure 5 It is in the second step that... Figure 4 The composite image containing rectangular box labels is obtained by cropping, stitching and merging four images from the quick-frozen dumpling dataset.

[0061] Figure 6 This is the network structure diagram of the LightYOLO_CBAM network model finally constructed in the third step;

[0062] Figure 7 This is a schematic diagram of the principle of depthwise separable convolution in the third step;

[0063] Figure 8 The network structure obtained in the third step is a lightweight improvement on the Backbone and Neck. Figure 6 (Pre-intermediate network structure);

[0064] Figure 9 This is a structural diagram of the SPP layer constructed using Softpool in the third step;

[0065] Figure 10 This is a lightweight network structure diagram of the Head in this invention (constructed in the third step).

[0066] Figure 11 This is the network structure diagram of the ReaPAN_CBAM module in the third step;

[0067] Figure 12 This is a defect identification image of frozen dumplings generated using the LightYOLO_CBAM network model. Detailed Implementation

[0068] like Figures 1 to 12 As shown, the real-time detection method for frozen dumpling surface defects based on the LightYOLO_CBAM network model of the present invention is carried out according to the following steps:

[0069] The first step is to create a labeled dataset, in which the location and category of each frozen dumpling are labeled in each image of the labeled dataset;

[0070] The second step is data preprocessing, which involves scaling and data augmentation of the image.

[0071] The third step is to construct a LightYOLO_CBAM network model for real-time detection of surface defects in frozen dumplings.

[0072] The fourth step is to train the LightYOLO_CBAM network model;

[0073] The fifth step is to save the LightYOLO_CBAM network model;

[0074] The sixth step is to use the LightYOLO_CBAM network model to perform real-time surface defect detection on frozen dumpling images input through a camera device (such as a camera or video camera).

[0075] like Figure 2 As shown, in the first step, the frozen dumpling images are manually labeled using the open-source deep learning annotation tool LabelImg;

[0076] The location of each frozen dumpling is marked in each image of the labeled dataset. The area where each frozen dumpling is located is marked using the rectangle annotation in the LabelImg toolbar.

[0077] When categorizing frozen dumplings, seven preset categories are provided: Normal, Leak, Brokenbelly, Brokenends, Crack, Adhesion, and Malformation.

[0078] Each frozen dumpling was manually labeled to distinguish its category (i.e., the seven categories mentioned above), forming a labeled dataset. Finally, the labeled file was saved. The saved file format was XML, which contained the coordinates of the top left and bottom right corners of each frozen dumpling rectangle, as well as the category information, i.e., the label information, corresponding to that rectangle.

[0079] The second step involves scaling the image:

[0080] The width and height of each captured image of frozen dumplings were scaled up by the same ratio and filled with RGB 128 colors. The final scaled images had the same width and height and the image content was not distorted. Considering the graphics card and video memory, the width and height were scaled to 416×416, which was used as the input size of the LightYOLO_CBAM model.

[0081] The second step involves data augmentation of the image, specifically:

[0082] The Mosaic data augmentation method is used to randomly select four images and stitch them together to form a new image, which contains the corresponding label information.

[0083] Image data augmentation involves three steps:

[0084] The first step is to randomly select four images from the labeled dataset, such as... Figure 3 These are four images from a dataset of quick-frozen dumplings;

[0085] The second step is to flip and scale each of the four images, then place the four processed images in the top left, bottom left, top right, and bottom right positions on four canvases of the original size; for example... Figure 4 As shown.

[0086] Thirdly, a matrix is ​​used to extract image regions from the four images and stitch them together to form a new image. The label information of the four images is also recalculated and incorporated into the label information of the new image; for example... Figure 5 As shown, this is a composite image containing rectangular box labels.

[0087] The third step specifically involves modifying the existing YOLOv4 object detection model, including:

[0088] ① For the backbone network (also known as the feature extraction network, i.e., the backbone), MobileNetv1 (lightweight neural network) is used as the backbone network (i.e., the backbone) of the LightYOLO_CBAM network model. Its core is depthwise separable convolution; depthwise separable convolution uses depthwise convolution and pointwise convolution to separately compute the channels and spatial regions of the feature map. Depthwise convolution computes one channel per kernel, while pointwise convolution is 1×1×C_in (number of channels in the upper layer feature map)×C_out (number of channels in the output feature map). For example... Figure 7 The image shows a depthwise separable convolution.

[0089] ② For the fusion layer (also known as the feature fusion network, or Neck), the consecutive standard convolutional modules CBL×3 and CBL×5 in PANet (path aggregation network) are replaced by alternating stacked standard convolutional and depthwise separable convolutional modules CD_3 and CD_5 as the fusion layer (Neck) of the LightYOLO_CBAM network model; for example Figure 8 The diagram shows the network structure after lightweight improvements to Backbone and Neck.

[0090] ③ For the fusion layer, i.e., the Neck, the SPP layer is constructed using Softpool (replacing the original max pooling operation); Softpool is a pooling operation and is an existing technology; the structure of the constructed SPP layer is as follows. Figure 9 As shown.

[0091] The core of Softpool is to pool feature maps using an exponentially weighted summation method. It calculates the weights of each position in the local feature map based on the information of the local feature map within the pooling kernel size, and then calculates the pooling value of the local feature map by summing the weights of these positions. The pooling kernel traverses the entire feature map according to the stride, ultimately obtaining the pooled feature map. Therefore, it not only reduces the feature map size but also integrates more surface feature information, which is beneficial for incorporating more defect semantic information into subsequent operations.

[0092] ④ For the detection head, the original YOLO Head 52×52 (pixels, the same below) and YOLO Head 26×26 modules in the YOLOv4 object detection model are retained, and a YOLO Head 104×104 module is added. The YOLO Head 13×13 module is deleted, but the 13×13 feature map is retained. This feature map belongs to the deep feature map of the backbone network, so it has a large receptive field and contains strong semantic features. The shallow feature maps such as 104×104, 52×52 and 26×26 (all pixels) contain strong localization information. The fusion layer, or Neck, upsamples and transfers the strong semantic information contained in the 13×13 feature map to the shallow feature map for feature fusion. Finally, the fused feature maps of 104×104, 52×52 and 26×26 sizes are input into the detection head to improve the head's performance in locating and classifying defective dumplings. Figure 10 This is a diagram of the lightweight network structure of the Head in this invention.

[0093] This invention fuses the strong localization information in the shallow feature map with the strong semantic information contained in the 13×13 feature map to improve the performance of the detection head in locating and classifying defective dumplings.

[0094] The necessity of improving the detection head lies in the fact that the size of the dumplings and the size of defects occupy a very small proportion of the input image (416×416 pixels) of the LightYOLO_CBAM network model. After downsampling by 8x, 16x, and 32x on the backbone, the pixel size of the dumplings is 5×5, 2×3, and 1×1, respectively. After downsampling by 32x, the defects are less than one pixel. Defects less than one pixel are not conducive to the fusion of features by the neck and the classification and localization of defective dumplings by the head. Therefore, the head part needs improvement.

[0095] The improvements to the detection head avoid the problem of defects being less than one pixel, which is beneficial for the Neck to fuse features and for the Head to classify and locate defective dumplings.

[0096] ⑤ In the fusion layer, i.e., the Neck, the ReaPAN_CBAM module (a feature fusion network) is used to input the 104×104 (pixel, the same below), 52×52, 26×26 and 13×13 feature maps output by the last four DW_1 of the backbone network built by MobileNetv1 into the fusion layer, i.e., the Neck. The 13×13 feature map is passed through the SPP layer built by Softpool, and after upsampling, it is fused with the 26×26, 52×52 and 104×104 feature maps. After residual connection and downsampling operations, the fused 104×104, 52×52 and 26×26 feature maps are finally input into the detection head, i.e., the Head.

[0097] In the fusion layer, or Neck, a CBAM attention mechanism is embedded after the concat operation of upsampling and downsampling, thereby enhancing the model's attention to important features during the fusion stage.

[0098] Figure 11 This is the network structure of the ReaPAN_CBAM module. The LightYOLO_CBAM network model constructed in the third step is as follows: Figure 6 As shown.

[0099] The experimental training parameters in the fourth step are configured as follows: the image input size is 416×416 pixels, and the pre-trained model is MobileNetv1_1_0_224_tf.h5 trained on ImageNet; the training process is divided into two stages: freezing and not freezing the backbone network. The parameter configurations for these two stages are shown in the table below (Table 1 is the model training parameter configuration table):

[0100] Table 1

[0101]

[0102] In steps one through four, the LightYOLO_CBAM network model was created using Python 3.6 on the PyCharm platform for a program to detect surface defects in frozen dumplings. The deep learning object detection network, namely the LightYOLO_CBAM network model, was built using TensorFlow 1.31.1 and Keras 2.1.5. In step five, the final structure and parameters of the LightYOLO_CBAM network model (including weights, model structure, and configuration information) were saved as a .h5 file with a file size of 39.4M.

[0103] The sixth step is as follows:

[0104] For real-time images of frozen dumplings captured by a camera from the frozen dumpling production line, data augmentation is performed to resize the images to a uniform size. These images are then normalized and preprocessed before being used as input to the LightYOLO_CBAM network model. After a series of convolutional operations and downsampling, feature maps calculated from the last four depthwise separable convolutions with a stride of 1 in the backbone network are extracted. These four feature maps are then input into the fusion layer (Neck) for multi-scale feature fusion. The fused 104×104, 52×52, and 26×26 (all pixels) feature maps are then input into the detection head (Head) to obtain the dumpling's position coordinates, category, confidence level, and defect statistics. The recognition structure is as follows: Figure 12 As shown.

[0105] The LightYOLO_CBAM network model in this invention is ultimately evaluated using model size, F1-score, AP, mAP, and FPS. F1-score is the harmonic mean of precision (P) and recall (R), reflecting the model's detection accuracy for each class. AP is used to measure the recognition accuracy of the seven classes. mAP is the average of the AP of the seven classes. FPS is used to measure the detection speed of the optimized model.

[0106] Table 2 is a comparison table of F1-score and AP between the LightYOLO_CBAM network model and YOLOv4.

[0107] Table 2

[0108]

[0109] Table 3 compares the model size, mAP, and FPS of the LightYOLO_CBAM network model and YOLOv4.

[0110] Table 3

[0111] YOLOv4 245MB 69.57％ 22.12％ LightYOLO_CBAM 39.4MB 88.71％ 39.02％

[0112] As shown in Tables 2 and 3, LightYOLO_CBAM has a significantly improved accuracy in identifying the seven categories of frozen dumplings compared to the standard YOLOv4.

[0113] The above embodiments are only used to illustrate and not limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention without departing from the spirit and scope of the present invention. Any modifications or partial substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A real-time detection method for surface defects of frozen dumplings based on the LightYOLO_CBAM network model, characterized in that... Follow these steps: The first step is to create a labeled dataset, in which the location and category of each frozen dumpling are labeled in each image of the labeled dataset; The second step is data preprocessing, which involves scaling and data augmentation of the image. The third step is to construct a LightYOLO_CBAM network model for real-time detection of surface defects in frozen dumplings. The fourth step is to train the LightYOLO_CBAM network model; The fifth step is to save the LightYOLO_CBAM network model; The sixth step is to use the LightYOLO_CBAM network model to perform real-time surface defect detection on the frozen dumpling images input through the camera device; The third step specifically involves modifying the existing YOLOv4 object detection model, including: ① For the backbone network, MobileNetv1 is used as the backbone network of the LightYOLO_CBAM network model, and its core is depthwise separable convolution. ② For the fusion layer, i.e. the Neck, standard convolutional and depthwise separable convolutional alternating stacked modules CD_3 and CD_5 are used to replace the continuous standard convolutional modules CBL×3 and CBL×5 in PANet, respectively, as the fusion layer, i.e. the Neck, of the LightYOLO_CBAM network model; ③ For the fusion layer, i.e. Neck, Softpool is used to construct the SPP layer. The Softpool uses an exponential weighted summation method to pool the feature map. The pooling value is calculated by weighted summation based on the weights of each position of the local feature map within the pooling kernel. ④ For the detection head, the original YOLO Head 52×52 and YOLO Head 26×26 modules in the YOLOv4 target detection model are retained, and a YOLO Head 104×104 module is added. The YOLO Head 13×13 module is deleted, but the 13×13 feature map is retained. The shallow feature maps such as 104×104, 52×52 and 26×26 contain strong localization information. The fusion layer, or Neck, upsamples and transfers the strong semantic information contained in the 13×13 feature map to the shallow feature map for feature fusion. Finally, the fused feature maps of 104×104, 52×52 and 26×26 sizes are input into the detection head to improve the head's performance in locating and classifying defective dumplings. The necessity for improving the detection head lies in the fact that the size of dumplings and the size of defects account for a very small proportion of the input image. After downsampling by the backbone network, the defects are less than one pixel, which is not conducive to feature fusion and classification localization. ⑤ In the fusion layer, i.e., the Neck, the ReaPAN_CBAM module is used to input the 104×104, 52×52, 26×26 and 13×13 feature maps output from the last four DW_1 of the backbone network built by MobileNetv1 into the fusion layer, i.e., the Neck. The 13×13 feature map is passed through the SPP layer built by Softpool, and after upsampling, it is fused with the 26×26, 52×52 and 104×104 feature maps. After residual connection and downsampling operations, the fused 104×104, 52×52 and 26×26 feature maps are finally input into the detection head, i.e., the Head. In the fusion layer, i.e., the Neck, a CBAM attention mechanism is embedded after the concat operation of upsampling and downsampling.

2. The real-time detection method for frozen dumpling surface defects based on the LightYOLO_CBAM network model according to claim 1, characterized in that: In the first step, the frozen dumpling images were manually labeled using the open-source deep learning annotation tool LabelImg; The location of each frozen dumpling is marked in each image of the labeled dataset. The area where each frozen dumpling is located is marked using the rectangle annotation in the LabelImg toolbar. When categorizing frozen dumplings, seven categories are preset: Normal, Leak, Broken_belly, Broken_ends, Crack, Adhesion, and Malformation. Each frozen dumpling was manually labeled to distinguish its category, forming a labeled dataset, and finally the labeled file was saved.

3. The real-time detection method for frozen dumpling surface defects based on the LightYOLO_CBAM network model according to claim 2, characterized in that: The second step involves scaling the image: The width and height of each captured image of frozen dumplings were scaled up by the same ratio and filled with RGB 128 colors. The final scaled images had the same width and height and the image content was not distorted. The second step involves data augmentation of the image, specifically: The Mosaic data augmentation method is used to randomly select four images and stitch them together to form a new image, which contains the corresponding label information. Image data augmentation involves three steps: The first step is to randomly select four images from the labeled dataset. The second step is to flip and scale the four images respectively, and then place the four processed images in the top left, bottom left, top right and bottom right positions of the four original-size canvases. The third step is to extract the image regions from the four images and stitch them together to form a new image. The label information of the four images is also recalculated and merged into the label information of the new image.

4. The real-time detection method for frozen dumpling surface defects based on the LightYOLO_CBAM network model according to claim 3, characterized in that: The experimental training parameters in step four are configured as follows: the image input size is 416×416 pixels, and the pre-trained model is MobileNetv1_1_0_224_tf.h5 trained on ImageNet; the training process is divided into two stages: freezing and not freezing the backbone network. The parameter configurations for these two stages are shown in the table below: 。 5. The real-time detection method for frozen dumpling surface defects based on the LightYOLO_CBAM network model according to claim 4, characterized in that: In steps one through four, the LightYOLO_CBAM network model was created using Python 3.6 on the PyCharm platform for a program to detect surface defects in frozen dumplings. The deep learning object detection network, namely the LightYOLO_CBAM network model, was built using TensorFlow 1.31.1 and Keras 2.1.

5. In step five, the final structure and parameters of the LightYOLO_CBAM network model were saved as a .h5 file.

6. The real-time detection method for frozen dumpling surface defects based on the LightYOLO_CBAM network model according to claim 5, characterized in that: The sixth step is as follows: For real-time images of frozen dumplings from the frozen dumpling production line captured by the camera device, the image size is changed to a uniform size after data augmentation, and then normalized and preprocessed as the input image of the LightYOLO_CBAM network model. Then, after convolution operation and downsampling, the feature maps calculated by the last four depthwise separable convolutions with a stride of 1 in the backbone network are extracted. These four feature maps are respectively input into the fusion layer (Neck) for multi-scale feature fusion. Then, the fused 104×104, 52×52 and 26×26 feature maps are input into the detection head (Head) to obtain the dumpling's position coordinates, category and confidence level, as well as defect statistics.

Images