A food vacuum packaging defect detection system based on deep learning

Abstract

The present invention discloses a deep learning-based defect detection system for small vacuum food packages. The system includes an image acquisition module, an image processing module, and a motion control module. The image acquisition module uses an industrial camera to capture images of small vacuum food packages on a conveyor belt. The image processing module, deployed on an industrial computer, receives images transmitted by the image acquisition module, detects and locates defective packages in the images using a deep learning-based target detection algorithm, and transmits the location information of the defective packages to the motion control module after detecting the defective packages. The motion control module is used to eliminate defective packages detected by the image processing module. This system overcomes the shortcomings of traditional manual defective package screening, which suffers from low efficiency and high missed detection rates. Instead, it uses an improved deep learning method to detect and eliminate defective packages, achieving high detection efficiency, high accuracy, and low labor costs.

Description

Technical Field

[0001] The present invention relates to the field of computer vision target detection, and in particular to a food vacuum small package defect detection system based on deep learning. Background Art

[0002] Traditional small vacuum packaging of food, such as mustard tubers and snow vegetables, requires manual screening for defect detection and removal, which is inefficient and has a high rate of missed detection of defective packaging. The low efficiency of packaging defect removal on the production line will affect the orderly progress of subsequent packaging, sterilization, and boxing. The missed detection of defective packaging will also affect the final product quality.

[0003] Compared with traditional manual defective packaging rejection methods, computer vision inspection technology has the following advantages: the inspection results are not affected by the physical condition and mood of the inspector, and can detect defective packaging for a long time, ensuring the objectivity and credibility of the inspection results; it does not cause visual fatigue, and the inspection speed is 5-10 times that of the human eye, with high inspection efficiency and guaranteed inspection accuracy; it can directly perform statistical analysis on the inspection results, facilitating production automation. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned shortcomings of low efficiency of manual rejection of defective food vacuum packaging detection and high defective package missed detection rate, and to provide a food vacuum packaging defect detection system based on deep learning to improve the efficiency of food vacuum packaging defect detection and reduce its missed detection rate.

[0005] In order to achieve the above-mentioned object, the present invention provides the following technical solutions: a food vacuum small packaging defect detection system based on deep learning, the system comprising an image acquisition module, an image processing module and a motion control module;

[0006] The image acquisition module is used to acquire images of small vacuum packages of food on a conveyor belt through an industrial camera;

[0007] The image processing module is deployed on the industrial computer and is used to receive the image transmitted by the image acquisition module, detect and locate defective packages in the image using a deep learning-based target detection algorithm, and transmit the location information of the defective packages to the motion control module after detecting the defective packages;

[0008] The image processing module includes an image preprocessing submodule and a vacuum small package defect detection and positioning submodule;

[0009] The image preprocessing submodule is used to preprocess the food vacuum package image, including image size scaling and image denoising based on wavelet transform;

[0010] The vacuum small package defect detection and positioning submodule includes a model pre-training process and a real-time detection and positioning process;

[0011] The model pre-training process is specifically as follows:

[0012] (1) Simulate the scene of small vacuum-packed food being transported on a conveyor belt and collect images containing different defect categories, including missing packages due to vacuum packaging failure and defective packages due to incorrect vacuum packaging position; collect images of different packaging stacking methods containing defective packages, including multiple defective packages in the same image and defective packages with some defects obscured, to form a dataset;

[0013] (2) Label the dataset: Label the categories, including normal and defective; label the coordinates of the upper left corner and lower right corner of the defect location; divide the dataset into training set and test set;

[0014] (3) Performing Mosaic data augmentation on the data in the training set, splicing the four images by random scaling, random cropping, and random arrangement; then inputting them into the food vacuum packaging defect detection model for training;

[0015] (4) Using the test set to test the learning effect of the food vacuum packaging defect detection model;

[0016] The food vacuum packaging defect detection model adopts an improved YOLOv5s network model, and its specific structure is as follows:

[0017] (1) Introducing affine convolution (Ghost Module) to replace the ordinary convolution in the C3 module of the original model, while retaining the feature extraction capability, reducing the number of network parameters;

[0018] (2) Adding SE (Squeeze and Excitation) attention mechanism to the feature extraction network to enhance the model’s ability to extract features from different channels;

[0019] (3) Use the space-to-depth operation SPD (Space to Depth) and the convolution with a stride of 1 to replace the convolution with a stride of 2 in the original model for downsampling, thereby improving the model's feature extraction ability for low-resolution images;

[0020] The real-time detection and positioning process specifically includes: inputting the real-time image captured by the image acquisition module into a trained food vacuum packaging defect detection model to obtain a detection result; locating the position of the defective package on the image when the defective package is detected; and transmitting the position information of the defective package to the motion control module based on the mapping relationship between the position on the image and the actual position of the conveyor belt, combined with the model inference consumption time and the transportation speed of the conveyor belt;

[0021] The motion control module includes an industrial robot for rejecting defective packages detected by the image processing module.

[0022] Furthermore, the image acquisition module includes an industrial camera and a light source. The industrial camera is placed perpendicular to the conveyor belt, and the light source and the conveyor belt are placed at a 45-degree angle. Two industrial cameras are placed at the same acquisition point so that the observation range of the industrial camera covers the width of the conveyor belt.

[0023] Furthermore, the implementation of the food vacuum small packaging defect detection model includes: first, the initial image is subjected to feature fusion by a large convolution with a convolution kernel size of 6 and a stride of 2, and then down-sampled by an SPD operation and a convolution with a convolution kernel size of 3 and a stride of 1, and the CSP (Cross Stage Paritial) structure incorporating the affine convolution Ghost Module is used for feature extraction. After four SPD and CSP operations, the image is down-sampled by 32 times, and then the features are further extracted through the SE attention mechanism and SPPF (Spatial Pyramid Pooling Fast).

[0024] Furthermore, affine convolution is introduced into the food vacuum small packaging defect detection model, and its implementation includes: first, obtaining a set of features through ordinary convolution operations, using grouped convolution operations on the new features to obtain another set of features, and splicing the two sets of features in the channel dimension to obtain new features; there are a large number of redundant features in the features obtained by ordinary convolution, and the affine convolution Ghost Module obtains these redundant features through grouped convolution. On the basis of maintaining the network feature extraction capability, the number of network parameters is reduced through grouped convolution, thereby improving the hardware's inference speed of the model.

[0025] Furthermore, the SE attention mechanism is introduced into the food vacuum small packaging defect detection model. The SE attention mechanism includes two operations: compression and excitation. Compression compresses the input features into a size of 1×1 through global average pooling. Excitation performs convolution operation on the compressed features through convolution with a convolution kernel size of 1 to obtain the weights of all different channel features of the input. The channel weights after the excitation operation are multiplied by the input features to amplify important channel features, thereby increasing the feature extraction capability of the model.

[0026] Furthermore, in the food vacuum small packaging defect detection model, SPD-Conv is used to replace the convolution with a stride of 2 in the original model. The original image is spliced ​​on the channel according to pixel interval sampling through SPD, and then the channel features are fused through convolution with a convolution kernel size of 3 and a stride of 1. This makes full use of the detailed information on the feature map and enhances the model's feature extraction capability for low-resolution images and small objects.

[0027] Furthermore, the motion control module adopts a suction cup type industrial robot.

[0028] Furthermore, the industrial camera and the industrial robot constitute a basic food vacuum small packaging defect detection unit. During the defect detection process, the conveyor belt is non-stationary. Considering the time consumed by the defect detection model reasoning, the time spent by the industrial robot to eliminate defective packaging, and the real-time changes in the position of the inspected packaging, it is necessary to reasonably place the number of industrial robots and the distance from the industrial camera.

[0029] Furthermore, multiple groups of food vacuum small packaging defect detection units are placed on a complete conveyor belt to increase the success rate of defective packaging detection.

[0030] The beneficial effects of the present invention are as follows: the present invention adopts an improved YOLOv5s network model in the image processing module, introduces affine convolution (Ghost Module) to replace the ordinary convolution in the C3 module in the original model, while retaining the feature extraction capability, reducing the number of network parameters; adds the SE (Squeeze and Excitation) attention mechanism to the feature extraction network part, thereby increasing the model's feature extraction capability on different channels; uses the space-to-depth operation SPD (Space to Depth) and a convolution with a step size of 1 to replace the convolution with a step size of 2 in the original model for downsampling, thereby increasing the model's feature extraction capability for low-resolution images. The overall system reasonably arranges the position and number of industrial cameras and industrial robots to prevent missed detection of defective packaging. Existing small vacuum packaging defects in food are mostly eliminated through manual screening and elimination, and the degree of automation is low. The automatic defective packaging detection system provided by the present invention can meet the real-time detection and elimination of defective packaging on small food packaging conveyor belts. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a structural diagram of a food vacuum small packaging defect detection system based on deep learning in a preferred example of the present invention.

[0032] Figure 2 It is a schematic diagram of the structure of the improved YOLOv5s network model in the preferred embodiment of the present invention.

[0033] Figure 3 It is a schematic diagram of the Ghost Module method provided in a preferred embodiment of the present invention.

[0034] Figure 4 This is a schematic diagram of the SE attention mechanism method provided in the preferred example of the present invention.

[0035] Figure 5 This is a schematic diagram of the SPD-Conv method provided in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The present invention can also be applied through different specific embodiments, and the details in this specification can be modified in various ways based on actual application scenarios. The scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0037] The present invention provides a deep learning-based defect detection system for small vacuum food packages. The system includes an image acquisition module, an image processing module, and a motion control module. The image acquisition module uses an industrial camera to capture images of small vacuum food packages on a conveyor belt. The image processing module receives images transmitted by the image acquisition module and uses a deep learning-based target detection algorithm to detect and locate defective packages in the images. After detecting a defective package, the system transmits the location information of the defective package to the motion control module. The motion control module then removes defective packages detected by the image processing module.

[0038] In a specific embodiment, Figure 1 As shown, the image acquisition module includes an industrial camera and a light source. The industrial camera is placed perpendicular to the conveyor belt, and the light source is placed at a 45-degree angle to the conveyor belt. Two industrial cameras are placed at the same acquisition point, ensuring that their observation range covers the width of the conveyor belt. The image processing module is deployed on an industrial computer. The motion control module includes a suction cup-type industrial robot.

[0039] Furthermore, if Figure 1 As shown in the dotted box in the figure, an industrial camera and two industrial robots form a basic set of defect detection units for small vacuum packaging of food. During the defect detection process, the conveyor belt is non-stationary. Considering the time consumed by defect detection model inference, the time spent by industrial robots to remove defective packages, and the real-time changes in the position of the inspected packages, it is necessary to reasonably place the number of industrial robots and their distance from the industrial camera. At the same time, multiple sets of defect detection units for small vacuum packaging of food are placed on a complete conveyor belt to increase the success rate of defective packaging detection.

[0040] Furthermore, the image processing module includes an image preprocessing submodule and a vacuum packaging defect detection and positioning submodule. The image preprocessing submodule is used to preprocess images of vacuum packaging food, including image resizing and wavelet-based image denoising. The vacuum packaging defect detection and positioning submodule includes a model pretraining process and a real-time detection and positioning process.

[0041] In a specific embodiment, the model pre-training process is specifically as follows:

[0042] (1) Simulate the scene of small vacuum-packed food being transported on a conveyor belt and collect images containing different defect categories, including missing packages due to vacuum packaging failure and defective packages due to incorrect vacuum packaging position; collect images of different packaging stacking methods containing defective packages, including multiple defective packages in the same image and defective packages with some defects obscured, to form a dataset;

[0043] (2) Label the dataset: Label the categories, including normal and defective; label the coordinates of the upper left corner and lower right corner of the defect location; divide the dataset into training set and test set;

[0044] (3) Performing Mosaic data augmentation on the data in the training set, splicing the four images by random scaling, random cropping, and random arrangement; then inputting them into the food vacuum packaging defect detection model for training;

[0045] (4) Use the test set to detect the learning effect of the food vacuum small packaging defect detection model.

[0046] like Figure 2 The figure shows an improved YOLOv5s network model structure for small package defect detection. In the backbone feature extraction, the network first fuses the initial image features using a large convolution with a kernel size of 6 and a stride of 2. Downsampling is then performed through an SPD operation and a convolution with a kernel size of 3 and a stride of 1. Feature extraction is then performed using the CSP structure of the affine convolution Ghost Module. After four SPD and CSP passes, the image is downsampled by a factor of 32. Features are then further extracted using the SE attention mechanism and SPPF. The main improvements to the network include:

[0047] (1) Introduce affine convolution Ghost Module into the original CS structure to replace the conventional convolution. The principle of Ghost Module is as follows Figure 3As shown in the figure, a set of features is first obtained through a normal convolution operation. A grouped convolution operation is then performed on these new features to obtain another set of features. The two sets of features are then concatenated in the channel dimension to obtain the new features. The features obtained through normal convolution contain a large number of redundant features. The Ghost Module obtains these redundant features through grouped convolution. While maintaining the network's feature extraction capabilities, this method reduces the number of network parameters and improves the hardware's inference speed on the model.

[0048] (2) Introduce the SE attention mechanism module. The principle of the SE attention mechanism is as follows Figure 4 As shown in the figure, it includes two operations: compression and excitation. Compression compresses the input features into a size of 1×1 through global average pooling. Excitation performs convolution operation on the compressed features through convolution with a convolution kernel size of 1 to obtain the weights of all different channel features of the input. The channel weights after the excitation operation are multiplied by the input features to amplify important channel features and improve the feature extraction capability of the model.

[0049] (3) Use SPD-Conv to replace the convolution with a step size of 2 in the network. The principle of SPD-Conv is as follows Figure 5 As shown in the figure, the traditional convolution with a stride of 2 for downsampling will cause the loss of fine-grained information. The SPD module splices the original image on the channel according to the pixel interval sampling method, and then fuses the channel features through convolution with a convolution kernel size of 3 and a stride of 1, making full use of the detailed information on the feature map and improving the model's feature extraction ability for low-resolution images and small objects.

[0050] In a specific embodiment, the real-time detection and positioning process is specifically as follows: the real-time image captured by the image acquisition module is input into a trained food vacuum small packaging defect detection model to obtain the detection result. When the defective packaging is detected, the position of the defective packaging on the image is located. According to the mapping relationship between the position on the image and the actual position of the conveyor belt, combined with the model reasoning consumption time and the conveyor belt transportation speed, the position information of the defective packaging is transmitted to the motion control module.

[0051] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Any changes or replacements that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed in the present invention should be covered by the protection scope of the present invention.

Claims

1. A food vacuum packaging defect detection system based on deep learning, characterized in that: The system includes an image acquisition module, an image processing module and a motion control module; The image acquisition module is used to acquire images of small vacuum packages of food on a conveyor belt through an industrial camera; The image processing module is deployed on the industrial computer and is used to receive the image transmitted by the image acquisition module, detect and locate defective packages in the image using a deep learning-based target detection algorithm, and transmit the location information of the defective packages to the motion control module after detecting the defective packages; The image processing module includes an image preprocessing submodule and a vacuum small package defect detection and positioning submodule; The image preprocessing submodule is used to preprocess the food vacuum package image, including image size scaling and image denoising based on wavelet transform; The vacuum small package defect detection and positioning submodule includes a model pre-training process and a real-time detection and positioning process; The model pre-training process is specifically as follows: (1) Simulate the scene of small vacuum-packed food being transported on a conveyor belt and collect images containing different defect categories, including missing packages due to vacuum packaging failure and defective packages due to incorrect vacuum packaging position; collect images of different packaging stacking methods containing defective packages, including multiple defective packages in the same image and defective packages with some defects obscured, to form a dataset; (2) Label the dataset: Label the categories, including normal and defective; Mark the coordinates of the upper left corner and lower right corner of the defect location; Divide the dataset into training and testing sets; (3) Performing Mosaic data augmentation on the data in the training set, splicing the four images by random scaling, random cropping, and random arrangement; then inputting them into the food vacuum packaging defect detection model for training; (4) Using the test set to test the learning effect of the food vacuum packaging defect detection model; The food vacuum packaging defect detection model adopts an improved YOLOv5s network model, and its specific structure is as follows: (1) Introducing affine convolution (Ghost Module) to replace the ordinary convolution in the C3 module of the original model, while retaining the feature extraction capability, reducing the number of network parameters; (2) Adding SE (Squeeze and Excitation) attention mechanism to the feature extraction network to enhance the model’s ability to extract features from different channels; (3) Use the space-to-depth operation SPD (Space to Depth) and the convolution with a stride of 1 to replace the convolution with a stride of 2 in the original model for downsampling, thereby improving the model's feature extraction ability for low-resolution images; The real-time detection and positioning process specifically includes: inputting the real-time image captured by the image acquisition module into a trained food vacuum packaging defect detection model to obtain a detection result; locating the position of the defective package on the image when the defective package is detected; and transmitting the position information of the defective package to the motion control module based on the mapping relationship between the position on the image and the actual position of the conveyor belt, combined with the model inference consumption time and the transportation speed of the conveyor belt; The motion control module includes an industrial robot for rejecting defective packages detected by the image processing module.

2. A food vacuum packaging defect detection system based on deep learning according to claim 1, characterized in that: The image acquisition module includes an industrial camera and a light source. The industrial camera is placed perpendicular to the conveyor belt, and the light source and the conveyor belt are placed at a 45-degree angle. Two industrial cameras are placed at the same acquisition point so that the observation range of the industrial camera covers the width of the conveyor belt.

3. The food vacuum packaging defect detection system based on deep learning according to claim 1 is characterized in that: The implementation of the food vacuum packaging defect detection model includes: first, performing feature fusion on the initial image through a large convolution with a convolution kernel size of 6 and a stride of 2, then downsampling through an SPD operation and a convolution with a convolution kernel size of 3 and a stride of 1, and incorporating a CSP (Cross Stage Paritial) structure of an affine convolution Ghost Module for feature extraction. After four SPD and CSP operations, the image is downsampled by a factor of 32, and features are further extracted through the SE attention mechanism and SPPF (Spatial Pyramid Pooling Fast).

4. A food vacuum packaging defect detection system based on deep learning according to claim 1, characterized in that: Affine convolution is introduced into the food vacuum packaging defect detection model. Its implementation includes: first, obtaining a set of features through ordinary convolution operations, applying grouped convolution operations on these new features to obtain another set of features, and then concatenating the two sets of features in the channel dimension to obtain new features. The features obtained by ordinary convolution contain a large number of redundant features. The affine convolution Ghost Module obtains these redundant features through grouped convolution. While maintaining the network's feature extraction capabilities, the grouped convolution method reduces the number of network parameters and improves the hardware's inference speed on the model.

5. The food vacuum packaging defect detection system based on deep learning according to claim 1 is characterized in that: The SE attention mechanism is introduced into the food vacuum small packaging defect detection model. The SE attention mechanism includes two operations: compression and excitation. Compression compresses the input features into a 1×1 size through global average pooling. Excitation performs a convolution operation on the compressed features with a convolution kernel size of 1 to obtain the weights of all different channel features of the input. The channel weights after the excitation operation are multiplied by the input features to amplify important channel features and enhance the feature extraction capability of the model.

6. The food vacuum packaging defect detection system based on deep learning according to claim 1 is characterized in that: In the food vacuum packaging defect detection model, SPD-Conv is used to replace the convolution with a stride of 2 in the original model. The original image is spliced ​​on the channel by pixel interval sampling through SPD, and then the channel features are fused through convolution with a convolution kernel size of 3 and a stride of 1. This fully utilizes the detailed information on the feature map and improves the model's feature extraction capability for low-resolution images and small objects.

7. The food vacuum packaging defect detection system based on deep learning according to claim 1 is characterized in that: The motion control module adopts a suction cup type industrial robot.

8. The food vacuum packaging defect detection system based on deep learning according to claim 1 is characterized in that: The industrial camera and the industrial robot constitute a basic food vacuum packaging defect detection unit. During the defect detection process, the conveyor belt is non-stationary. Considering the time consumed by the defect detection model reasoning, the time spent by the industrial robot to remove defective packages, and the real-time changes in the position of the inspected packages, it is necessary to reasonably place the number of industrial robots and the distance between them and the industrial camera.

9. The food vacuum packaging defect detection system based on deep learning according to claim 8, characterized in that: Multiple groups of food vacuum packaging defect detection units are placed on a complete conveyor belt to increase the success rate of defective packaging detection.

Images