Filled food bottle cap defect detection method based on transfer learning

Abstract

The invention relates to a transfer learning-based filled food bottle cap defect detection method, which comprises the following steps of: obtaining a to-be-detected filled food bottle cap state image, inputting the image into a pre-trained bottle cap state detection model, and outputting a bottle cap state detection result, wherein the bottle cap state detection model is trained by adopting a pre-trained VGG-16 convolutional neural network model, the pre-trained VGG-16 convolutional neural network model comprises an input layer, a convolutional layer, a pooling layer, a full connection layer and an output layer, knowledge migration is carried out by combining weight parameters of the convolutional layer and the pooling layer, and the bottle cap state detection model is obtained. And carrying out fine adjustment on the weight parameters of the full connection layer to obtain a trained bottle cap state detection model. Compared with the prior art, the method has the advantages of realizing rapid and effective detection under the condition of small samples and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of food equipment detection, and in particular to a method for detecting defects in bottle caps of filled food based on transfer learning. Background Art

[0002] Ensuring food safety is a normalized goal in the field of food processing. The global food processing industry is developing in the direction of high technology, intelligence and sustainability. Among them, the liquid filling industry, which also belongs to food processing, is also true. For a complete food canning production line process, it includes filling, sealing, packaging and other processes. And each of these links should have a corresponding defect detection and rejection mechanism. However, the reality is that most of the manual detection methods are still retained in many processes to be inspected, and the few remaining defect detection devices can only detect the presence or absence of defects, but cannot classify and identify specific problems of the product.

[0003] In the liquid food filling production line, defect detection of filled food is an essential and important link, and the detection of caps is particularly important. It is directly related to the shelf life, safety and final consumer satisfaction of the product. With the increasing requirements of consumers for food safety and quality, the traditional cap detection method that relies mainly on manual labor has been unable to meet the needs of efficient and accurate production. This method is not only inefficient and easily affected by human factors, but may also lead to missed detection due to fatigue or negligence, thereby increasing the risk of product recalls and market complaints. Therefore, the introduction of advanced automated defect detection and intelligent recognition technology has become an inevitable trend in the upgrading of the liquid filling industry. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting bottle cap defects of filled food based on transfer learning for accurately identifying bottle cap defects.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A method for detecting defects in bottle caps of filled food based on transfer learning comprises the following steps:

[0007] Acquire a bottle cap state image of a bottled food to be detected, input it into a pre-trained bottle cap state detection model, and output a bottle cap state detection result, wherein the bottle cap state detection model is trained using a pre-trained VGG-16 convolutional neural network model;

[0008] The training steps of the bottle cap state detection model include:

[0009] Obtain various types of state data sets of bottle caps of filled food and perform preprocessing;

[0010] Retrieving a pre-trained VGG-16 convolutional neural network model, wherein the pre-trained VGG-16 convolutional neural network model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer;

[0011] Freeze the weight parameters of the convolution layer and the pooling layer, and perform knowledge transfer;

[0012] The preprocessed data sets of various types of states of bottle caps of filled food are input into the pretrained VGG-16 convolutional neural network model to fine-tune the weight parameters of the fully connected layer to obtain a trained bottle cap state detection model.

[0013] Furthermore, the types of the state of the bottle caps of the filled food bottles include three types, namely, no cap, crooked cap and normal cap.

[0014] Furthermore, the preprocessing operation includes using data enhancement technology to enhance the data set of various types of states of the bottle caps of filled food, using Gaussian filtering method to remove discrete point noise, and using normalization operation to compress all images to the same size.

[0015] Furthermore, the data enhancement technology includes one or more of flipping, color gamut conversion and shearing.

[0016] Furthermore, the expression of the Gaussian filtering method is:

[0017]

[0018] I(x,y)=[G(x)G(y)]*f(x,y)

[0019] Where G(x) and G(y) are Gaussian functions, σ is the standard deviation of the Gaussian function, which is used to adjust the degree of smoothing, f(x, y) is the original image, I(x, y) is the smoothed image, and x and y represent pixel positions.

[0020] Furthermore, the convolution layer and the pooling layer extract feature maps from the preprocessed data set of various types of states of bottle caps of filled food through convolution-pooling operations, wherein the convolution operation of the convolution layer is:

[0021]

[0022] In the formula, is the input value of the mth feature map of the lth layer during the convolution process, is the bias value of the feature graph convolution operation, k is the amount of neuron data, is the weight of the connection between the neuron and the i-th neuron in the l-1 layer, is the output value of the convolution operation of the ith neuron in the mth feature map of the l-1th layer, is the output value of the feature graph convolution operation, and f(·) is the ReLU activation function;

[0023] The pooling layer uses the maximum pooling operation, and the operation process is expressed as:

[0024]

[0025] In the formula, is the value of the i-th neuron output by the pooling layer, f max (·) is the maximum pooling function, is the output feature map of the l-1th layer of the pooling layer.

[0026] Furthermore, the fully connected layer stretches the extracted feature map into a one-dimensional vector through a fully connected operation, and randomly discards neurons using a Dropout strategy in the fully connected operation, wherein the one-dimensional vector integrates local features containing category information.

[0027] Furthermore, in the training process of the bottle cap state detection model, the cross entropy loss is used as the cost function, and the expression is:

[0028]

[0029] Where J(θ) is the cross entropy loss, θ is the model parameter, m is the number of training samples, and h is θ (·) is the output of the activation function, x (i) is the feature vector of the i-th sample, y (i) is the true label of the i-th sample, which takes a value of 0 or 1.

[0030] Furthermore, the Adam optimizer is used to update the parameters in the process of fine-tuning the weight parameters of the fully connected layer.

[0031] Furthermore, a testing step is included to test the detection performance of the trained bottle cap state detection model, and the accuracy evaluation index used in the testing step is:

[0032]

[0033] ACC = correct prediction / total number of samples

[0034] In the formula, ACC is the accuracy rate, TP is the number of correctly predicted positive samples, TN is the number of correctly predicted negative samples; N is the total number of samples.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) The present invention utilizes the powerful feature extraction capability and transfer learning advantages of the convolutional layer and pooling layer of the VGG-16 network, and realizes knowledge transfer by constructing a data set containing various states of bottle caps and retrieving a pre-trained VGG-16 network. The feature extraction capability of the convolutional layer and the pooling layer is guaranteed by freezing the weight parameters of the convolutional layer and the pooling layer, and the requirements of bottle cap state classification detection are met by fine-tuning the weight parameters of the fully connected layer, thereby realizing the rapid construction of a bottle cap state detection model, which can accurately identify the capping defects of canned products and improve the detection accuracy.

[0037] (2) The present invention combines deep learning technology with transfer learning technology, which can quickly train a model suitable for rapid detection tasks under small sample conditions, significantly shortening the development cycle. There is no need to develop an algorithm separately for each detection project, reducing the cost of algorithm development and maintenance. At the same time, it can also well meet the real-time detection needs of high-speed production lines, which is of great significance to improving food safety levels and corporate production efficiency.

[0038] (3) The present invention solves the problems of low production efficiency and high cost of manual classification of bottle cap status, improves classification and recognition efficiency, and reduces the consumption of manpower, material resources and time. Through image enhancement and Gaussian denoising, the model's adaptability to complex environments is improved, and the robustness of the system is enhanced.

[0039] (4) When detecting the state of the bottle cap, the present invention does not simply determine whether there is a state defect, but rather classifies the bottle cap state into three categories, which has a stronger recognition ability and expands the practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0041] Figure 2 The following are characteristic diagrams of different states of the bottle cap of the food filling bottle of the present invention, wherein (a) is a normal cap state, (b) is a crooked cap state, and (c) is an open cap state;

[0042] Figure 3 The structure diagram of the VGG-16 transfer learning algorithm reconstructed by the parameters of the present invention

[0043] Figure 4 This is a rendering of the method for detecting defects in bottle caps of filled food according to the present invention. DETAILED DESCRIPTION

[0044] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0045] In view of the actual status of finished product defect detection in the filling production line, the capping defect is the most likely defect in the finished product defect category. However, the traditional detection method that relies on manual labor is inefficient and has a high missed detection rate. Therefore, this embodiment provides a method for detecting defects in filled food bottle caps based on transfer learning. Figure 1 As shown, the method comprises the following steps:

[0046] S1: Data collection and preprocessing.

[0047] This embodiment uses the constructed VGG16 transfer learning detection model to detect bottle cap defects of filled food, and then discriminates the bottle cap state type through bottle cap feature information. To this end, it is necessary to build a bottle cap state image database and mark the bottle cap state, which is specifically divided into three states: normal cap, crooked cap and no cap, such as Figure 2 As shown, Figure 2 (a) shows the normal cover state, (b) shows the crooked cover state, and (c) shows the uncovered state. Then the training set and the test set are divided according to the preset ratio of 4:1, and the image dataset is enhanced using data enhancement techniques such as flipping, color gamut conversion, and shearing.

[0048] In the implementation step S1, if there is a large amount of noise in the captured image, it will inevitably affect the subsequent processing and the final result. Therefore, denoising is a very important preliminary operation step. Common noise removal methods include Gaussian filtering, median filtering, mean filtering, etc. Because it is necessary to remove discrete point noise and retain continuous point noise, the Gaussian filtering method is used here, and its function is as follows:

[0049]

[0050] I(x,y)=[G(x)G(y)]*f(x,y)

[0051] Where: σ is the standard deviation of the Gaussian function, which is used to adjust the degree of smoothing; f(x, y) is the original image; I(x, y) is the smoothed image.

[0052] Since the image size of the extracted region of interest is not uniform, and in order to meet the problem that the input sample size of the classification model needs to be uniform, the image after threshold segmentation is normalized. After many attempts, each image is compressed to a size of 128×128, which improves the training speed without losing too much feature information.

[0053] S2: Model building and transfer learning.

[0054] like Figure 3As shown in the figure, a pre-trained VGG-16 convolutional neural network model is selected, the weight parameters of its feature extraction part are frozen, and the parameters of the fully connected layer are optimized to adapt to the detection of cap defects in filled foods.

[0055] The VGG-16 deep convolutional model mainly consists of 1 input layer, 13 convolutional layers, 5 pooling layers, 3 fully connected layers and 1 output layer.

[0056] The specific network architecture is:

[0057] 1. Input layer: input original image data, size is 224*224*3 (RGB channels);

[0058] 2. Convolution layer (conv1): uses 64 convolution kernels of size 3×3, with an incremental step size of 1, generating 64 feature maps;

[0059] 3. Pooling layer (pool1): Use a 2×2 maximum pooling kernel with a step size of 2 to compress the Feature Map size by half;

[0060] 4. Convolution layer (conv2-conv3): uses 64 3×3 convolution kernels with a step size of 1 to generate 64 feature maps;

[0061] 5. Pooling layer (pool2): use 2×2 maximum pooling kernel, stride 2;

[0062] 6. Repeat the convolution-pooling layer block, and the number of output channels of conv4-conv7 is 128, 256, 512, 512 respectively;

[0063] 7. Three fully connected layers, outputting 4096, 4096, and 1000 features respectively;

[0064] 8. Softmax classifier.

[0065] The Softmax classifier is one of the most common classifiers in multi-classification problems. Its working principle is: the input is the feature vector output from the last fully connected layer, and the length of the feature vector is the number of categories; each element represents the "score" of the sample belonging to that category. The Softmax function can convert these classification scores into a probability distribution ranging from 0 to 1.

[0066] During training, the pre-trained VGG-16 model in the PyTorch library is used for knowledge transfer to extract features of bottle cap status data.

[0067] During the training process, the model inputs a 128×128 bottle cap state image, and after a series of operations such as convolution, maximum pooling, full connection, Dropout strategy and Softmax classifier, it finally outputs 3D features as the result of bottle cap state detection. During this period, the parameters of the model convolution layer are frozen, and only the parameters of the model's fully connected layer are fine-tuned, so that it has strong detection accuracy and maintains the model's generalization ability.

[0068] In the convolution layer, the convolution kernel will perform convolution operations on the input image. In VGG-16, all small 3×3 convolution kernels are used, which greatly reduces the parameter calculation of the model and can make the decision function more decisive. Its operation process can be expressed as follows:

[0069]

[0070] Where: is the output value after convolution operation of the ith neuron in the mth feature map of the l-1th layer; is the weight of the connection between this neuron and the i-th neuron in the l-1 layer; is the input value of the mth feature map of the lth layer during the convolution process; is the output value of the convolution operation of the feature map; is the bias value of the convolution operation of the feature map; f(·) is the activation function;

[0071] For the activation function, the ReLU function is used here, and its formula is as follows:

[0072] f(x)=max(0,x)

[0073] Although the pooling operation is different from the convolution operation, the essence of the pooling operation is also feature extraction. In addition, the pooling operation reduces the network parameters and reduces the amount of calculation of the model. Commonly used pooling methods include maximum pooling and average pooling. Average pooling calculates the average value of all values ​​in the local perception threshold as its output result, while maximum pooling selects the maximum value of all values ​​in the local perception threshold as its output result. This model adopts the latter, and the pooling kernel size is 2×2. Its formula can be expressed as:

[0074]

[0075] Where: f max (·) is the maximum pooling function.

[0076] The fully connected layer stretches the feature map after the last convolution-pooling operation into a one-dimensional feature vector, which integrates local features containing category information. In addition, in the operation of the fully connected layer, in order to make the trained model have a strong generalization ability, the Dropout strategy is used. This strategy can randomly discard neurons during the training process, thereby effectively preventing overfitting and enhancing the robustness of the network. The output layer is connected to the fully connected layer and uses the Softmax classifier. This study is a three-class classification, so a three-dimensional column vector will be obtained at the end of the network as the discrimination result.

[0077] S3: Training of bottle cap status detection model.

[0078] The divided data set is input into the constructed bottle cap state detection model, and the model forward propagates to calculate the predicted value corresponding to the input data; the model is trained under the PyTorch deep learning framework, and the model performance is optimized through back propagation and parameter update in each epoch.

[0079] When designing the bottle cap state detection model, cross entropy loss is used as the cost function. The cost function can characterize the quality of a model. The smaller the loss function value, the closer the model's prediction result is to the true label. During the model training process, the cost function is expected to be as small as possible. The formula is as follows:

[0080]

[0081] In step S3, a total of 50 generations are trained when training the model, and the Adam optimizer is selected to optimize the parameters of the model. The batch size is selected as 16 to reasonably utilize the computing power and video memory of the GPU.

[0082] The performance of the evaluation model on the test set is evaluated using the accuracy indicator, and the formula is:

[0083]

[0084] ACC = correct prediction / total number of samples

[0085] In the formula, ACC is the accuracy rate; TP is the number of correctly predicted positive samples; TN is the number of correctly predicted negative samples; N is the total number of samples.

[0086] S4: Bottle cap status detection.

[0087] The bottle cap status images taken by the camera or in the test set are pre-processed and input into the trained bottle cap status detection model based on transfer learning to obtain the predicted bottle cap status.

[0088] In view of the above method, this embodiment has carried out case testing, and the specific contents are as follows:

[0089] (1) Batch test: In this embodiment, the prediction accuracy of the model is obtained by testing on the test set of bottle cap states of bottled food, that is, the accuracy of the "Gaussian denoising + image enhancement + VGG-16" model in Table 1 is -89.50%. The test set consists of three different types of bottle cap states (crooked cap, normal cap, no cap), totaling 160 pictures. The model before and after adding image enhancement and Gaussian denoising is compared by ablation experiments to obtain different accuracy rates (ACC), that is, the accuracy rates obtained on the bottle cap state classification data set with different preprocessing operations. The experimental results are shown in Table 1.

[0090] Table 1 Accuracy achieved on the bottle cap status classification dataset with different preprocessing operations

[0091]

[0092] (2) Single image test: A random image taken by a camera is input into the trained transfer learning-based bottle cap state detection model for food bottles (here the normal cap state is selected) and predicted. The correct prediction result can be obtained and the result is marked on the graph, as shown in the figure. Figure 4 shown.

[0093] The present invention uses a constructed transfer learning-based filled food bottle cap defect detection model to detect feature information of filled food bottle caps, and obtains the measured bottle cap state type through the bottle cap feature information, which can quickly complete the filled food bottle cap state detection work, saving labor costs and improving classification and recognition efficiency.

[0094] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0095] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0096] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0099] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0100] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting defects in bottle caps of filled food based on transfer learning, characterized in that: The following steps are involved: Acquire a bottle cap state image of a bottled food to be detected, input it into a pre-trained bottle cap state detection model, and output a bottle cap state detection result, wherein the bottle cap state detection model is trained using a pre-trained VGG-16 convolutional neural network model; The training steps of the bottle cap state detection model include: Obtain various types of state data sets of bottle caps of filled food and perform preprocessing; Retrieving a pre-trained VGG-16 convolutional neural network model, wherein the pre-trained VGG-16 convolutional neural network model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; Freeze the weight parameters of the convolution layer and the pooling layer, and perform knowledge transfer; The preprocessed data sets of various types of states of bottle caps of filled food are input into the pretrained VGG-16 convolutional neural network model to fine-tune the weight parameters of the fully connected layer to obtain a trained bottle cap state detection model.

2. The method for detecting defects in bottle caps of filled food based on transfer learning according to claim 1, characterized in that: There are three types of bottle cap states for the filled food bottles, namely, no cap, crooked cap and normal cap.

3. The method for detecting defects in bottle caps of filled food based on transfer learning according to claim 1, characterized in that: The preprocessing operation includes using data enhancement technology to enhance the data set of various types of states of the filled food bottle caps, using Gaussian filtering method to remove discrete point noise, and using normalization operation to compress all images to the same size.

4. The method for detecting defects in bottle caps of canned food based on transfer learning according to claim 3, characterized in that: The data enhancement technology includes one or more of flipping, color gamut conversion and shearing.

5. The method for detecting defects in bottle caps of filled food based on transfer learning according to claim 3, characterized in that: The expression of the Gaussian filtering method is: l(x,y)=[G(x)G(y)]*f(x,y) Where G(x) and G(y) are Gaussian functions, σ is the standard deviation of the Gaussian function, which is used to adjust the degree of smoothing, f(x, y) is the original image, I(x, y) is the smoothed image, and x and y represent pixel positions.

6. The method for detecting defects in bottle caps of filled food based on transfer learning according to claim 1, characterized in that: The convolution layer and the pooling layer extract feature maps from the preprocessed state data set of various types of filled food bottle caps through convolution-pooling operations, wherein the convolution operation of the convolution layer is: In the formula, is the input value of the mth feature map of the lth layer during the convolution process, is the bias value of the feature graph convolution operation, k is the amount of neuron data, is the weight of the connection between the neuron and the i-th neuron in the l-1 layer, is the output value of the convolution operation of the ith neuron in the mth feature map of the l-1th layer, is the output value of the feature graph convolution operation, and f(·) is the ReLU activation function; The pooling layer uses the maximum pooling operation, and the operation process is expressed as: In the formula, is the value of the i-th neuron output by the pooling layer, f max (·) is the maximum pooling function, is the output feature map of the l-1th layer of the pooling layer.

7. The method for detecting defects in bottle caps of filled food based on transfer learning according to claim 6, characterized in that: The fully connected layer stretches the extracted feature map into a one-dimensional vector through a fully connected operation, and randomly discards neurons using a Dropout strategy in the fully connected operation, wherein the one-dimensional vector integrates local features containing category information.

8. The method for detecting defects in bottle caps of canned food based on transfer learning according to claim 1, characterized in that: During the training process of the bottle cap state detection model, cross entropy loss is used as the cost function, and the expression is: Where J(θ) is the cross entropy loss, θ is the model parameter, m is the number of training samples, and h is θ (·) is the output of the activation function, x (i) is the feature vector of the i-th sample, y (i) is the true label of the i-th sample, which takes a value of 0 or 1.

9. The method for detecting defects in bottle caps of filled food based on transfer learning according to claim 1, characterized in that: The Adam optimizer is used to update the parameters in the process of fine-tuning the weight parameters of the fully connected layer.

10. The method for detecting defects in bottle caps of filled food based on transfer learning according to claim 1, characterized in that: A testing step is also included to test the detection performance of the trained bottle cap state detection model. The accuracy evaluation index used in the testing step is: ACC = correct prediction / total number of samples In the formula, ACC is the accuracy rate, TP is the number of correctly predicted positive samples, TN is the number of correctly predicted negative samples; N is the total number of samples.

Images