A method for detecting the appearance of a shelled poultry egg

Abstract

The application discloses a kind of after shelling poultry egg appearance detection method, step one: deep learning target detection algorithm: egg is marked as image of region of interest in advance input convolutional neural network and is trained;Step two: deep learning classification algorithm: various defects of egg and good egg before use are classified, input convolutional network carries out feature extraction, using the SPPF module of the feature extraction module of space pyramid pooling SPP and full connection layer combination extracts multiscale feature information;Step three: according to the position of egg and the detection result of single shooting obtained according to deep learning target detection algorithm and classification detection algorithm, according to position information, judge which recess the egg belongs to, then all results of the egg belonging to this recess are merged;The method can save a lot of human resources, and can provide uninterrupted detection needs, and realize full detection, improve work efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of food quality detection, and particularly relates to a method for detecting the appearance of shelled poultry eggs. BACKGROUND

[0002] There are some problems in the process of eggshell removal: 1) eggshell residues, the eggshell may not be completely removed, resulting in eggshell residues on the surface of the egg; 2) egg breakage, the egg white breaks during the eggshell removal process, resulting in the exposure of the egg yolk; 3) egg yolk contamination, part of the egg yolk is contaminated on the egg white of other eggs due to the breakage of the egg white of other eggs; 4) excessive air chamber, the air chamber is more obvious after the eggshell removal; 5) black spots, there are some small black spots on the egg white; 6) concave, the surface of the egg is concave due to the extrusion during the processing; 7) cracking, the egg white is broken but the egg yolk is not exposed; 8) honeycomb egg, small and dense holes appear on the surface of the egg; 9) black egg, the surface is no longer smooth and white but black and dark, which is often related to the poor quality of the egg itself. The above problems seriously affect the quality and even the food safety.

[0003] At present, the appearance quality detection of eggs after eggshell removal in food processing equipment is mainly manual visual detection. In the existing literature, we only found the related research on the appearance detection and quality detection of egg products without eggshell removal. The research mainly focuses on the detection of egg cracks, shell dirt, internal egg quality including loose yolk eggs and fertilized eggs; no effective detection research on the eggshell removal process and egg quality after the eggshell removal of cooked and shelled egg products is found. Therefore, the application provides a method for detecting the appearance of shelled poultry eggs. SUMMARY

[0004] The application aims to provide a method for detecting the appearance of shelled poultry eggs to solve the problems in the background.

[0005] To achieve the above-mentioned purpose, the application provides the following technical scheme: a method for detecting the appearance of shelled poultry eggs, comprising the following steps: step one: a deep learning target detection algorithm: an egg marked as an image of a region of interest is input into a convolutional neural network for training, local spatial information is extracted, and the conversion and extraction of input features are realized; the specific steps are as follows:

[0006] S1: collect pictures of various distribution conditions of eggs that may occur in actual production;

[0007] S2: label the collected pictures, and label all the eggs in the pictures; the specific labeling means is that a positive rectangle slightly larger than the egg area is used to frame the egg, and the labeling name of the egg is added; all the egg labeling names are consistent;

[0008] S3: divide the labeled pictures into a training set and a test set according to a ratio of 9:1;

[0009] S4: input the pictures of the training set and the corresponding label information into the neural network, so that the internal weight of the neural network records the image information features of the eggs; use the FPN feature pyramid structure to fuse the feature maps of different levels together through upsampling and downsampling operations to generate a multi-scale feature pyramid;

[0010] S5: use stochastic gradient descent (SGD) to iteratively optimize and update the model parameters;

[0011] S6: finally, use the target boxes of different sizes and aspect ratios defined in advance to cluster the target boxes of the training set to obtain the final detection boxes for prediction;

[0012] Step two: deep learning classification algorithm: use the classified various defective eggs and good eggs in advance to input the convolutional network for feature extraction, use the SPPF module combined with the spatial pyramid pooling (SPP) and fully connected layer feature extraction module to extract multi-scale feature information without introducing additional parameters; the specific steps include:

[0013] A1: the spatial pyramid pooling algorithm divides the input image into multiple grids, each grid has different sizes;

[0014] A2: for each grid, perform average pooling operation to extract features within the region;

[0015] A3: concatenate the pooling results of all grids to form a fixed-size feature vector;

[0016] A4: input the pooled feature vector into the fully connected layer for classification or regression task;

[0017] Step three: according to the deep learning object detection algorithm and the classification detection algorithm, the positions of the eggs and the detection results of single shooting are obtained, according to the position information, it is judged which groove the egg belongs to, then all the results of the eggs belonging to this groove are merged, as long as one of them is judged as defective, it is considered that the egg in the groove belongs to defective egg, otherwise it is considered to belong to good egg.

[0018] Preferably, the FPN feature pyramid structure mainly includes three parts: Bottom-up, Top-down and Lateral connection; Bottom-up:

[0019] The process of Bottom-up is to input the picture into the neural network backbone to extract features, that is, to downsample the pixels layer by layer to extract feature pixels;

[0020] The top-down process is to upsample the feature map obtained from the high layer and then pass it down. Since the high-level features contain rich semantic information, the top-down propagation can make these semantic information propagate to the low-level features, so that the low-level features also have rich semantic information in the case of resolution improvement.

[0021] The lateral connection is a feature fusion process, that is, P1 is the fusion of the feature after 1*1 convolution of C1 and the upsampled feature of P2.

[0022] Preferably, in the S5, the specific algorithm is: ① randomly initialize the parameters of the model; ② in each training batch, calculate the gradient of the loss function with respect to the parameters of the model by the back propagation algorithm; the back propagation algorithm will start from the loss function, propagate the gradient in the reverse direction along the computation graph, until all parameters are updated; ③ use the calculated gradient to update the parameters of the model; the update rule of the SGD algorithm is:

[0023] wherein,

[0024] θ t represents the model parameters in the tth iteration, and a is the learning rate,

[0025] J(θ t ) is the loss function,

[0026] is the gradient of the loss function with respect to the parameters; the parameters are updated by subtracting the learning rate multiplied by the gradient from the current parameters; ④ repeat steps ② and ③ until the loss function converges or the maximum number of iterations is reached.

[0027] Preferably, in the A1, the size of each grid is fixed and predefined.

[0028] Preferably, in the A2, the average pooling: for each pooling region, calculate the average value of all values in the region in the input feature map, and take the average value as the output value of the region; the pooling region is a square and slides on the input feature map with a fixed step; the pooling operation is non-overlapping, that is, there is no overlap between the pooling regions; its function is to smooth the input feature map.

[0029] Preferably, in the A4, the fully connected layer usually includes one or more fully connected layers and activation functions, which are used to extract higher-level features from the features and generate the final output.

[0030] Compared with the prior art, the present application has the beneficial effects that: the present application designs a kind of appearance detection method for shelled poultry eggs, which can save a lot of human resources, and can provide uninterrupted detection needs, and realize full detection, improve work efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0031] Fig. 1 The flowchart of the present application is shown in the figure.

[0032] Fig. 2 The feature fusion process of the present application is shown in the figure. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0034] Please refer to Figs. 1-2 The embodiment provides a technical solution: an appearance detection method for shelled poultry eggs, comprising the following steps:

[0035] S1: collect pictures of various distribution of eggs that may occur in actual production;

[0036] S2: label the collected pictures, and label all the eggs in the picture. The specific labeling means is to frame the eggs with a larger rectangle than the area of the eggs, and add the label name of the eggs. All the egg label names are consistent.

[0037] S3: divide the labeled pictures into training set and test set according to the ratio of 9:1;

[0038] S4: input the pictures of the training set and the corresponding label information into the neural network, make the neural network internal weight record the image information characteristics of the eggs; use the FPN feature pyramid structure, fuse the feature maps of different levels together through upsampling and downsampling operations to generate a multi-scale feature pyramid; the high-level features of the feature pyramid contain rich semantic information (beneficial to classification), but the resolution is low, and it is difficult to accurately save the position information of the object; the low (shallow) layer features have less semantic information, but the resolution is high, and can contain accurate object position information (beneficial to detection and segmentation); the FPN is to fuse the low-level features and high-level features to obtain a target detection structure with accurate recognition and positioning; the feature pyramid is mainly realized by upsampling and stacking with features of coarser granularity from top to bottom to fuse features of different levels, and from bottom to top, a convolution layer is used to fuse feature maps from different levels;

[0039] S5: use stochastic gradient descent (SGD) to iteratively optimize and update the model parameters;

[0040] S6: finally, use the target box of different sizes and aspect ratios defined in advance to cluster the target box of the training set to obtain the final detection box generated during prediction; by this method, the target detection area of the egg can be output, which can effectively avoid the misjudgment of the empty groove, and does not need to judge each groove position separately, reducing the calculation amount and increasing the running speed of the device;

[0041] Step two: deep learning classification algorithm: use the classified eggs with various defects and perfect eggs in advance, input the convolution network for feature extraction, use the SPPF module combined with the spatial pyramid pooling (SPP) and fully connected layer feature extraction module to extract multi-scale feature information without introducing additional parameters; the specific steps include:

[0042] A1: the spatial pyramid pooling algorithm divides the input image into multiple grids, each grid has different sizes;

[0043] A2: for each grid, perform average pooling operation to extract features in the region;

[0044] A3: splice the pooling results of all grids together to form a fixed-size feature vector; since each grid has different sizes, features of different scales can be fully utilized. Therefore, regardless of the size of the input image, spatial pyramid pooling can generate a fixed-size feature vector. This allows the spatial pyramid to be connected with subsequent layers such as fully connected layers without the need for fixed-size adjustment of the input;

[0045] A4: input the pooled feature vector into the fully connected layer for classification or regression task;

[0046] Step three: according to the deep learning target detection algorithm and the classification detection algorithm, the position of the eggs and the detection result of single shooting are obtained, the position information is used to determine which groove the egg belongs to, then all the results of the eggs belonging to this groove are merged, and as long as one of them is determined as defective, the egg in the groove is considered as defective egg, otherwise it is considered as good egg.

[0047] In this embodiment, preferably, the FPN feature pyramid structure mainly includes three parts: Bottom-up, Top-down and Lateral connection; Bottom-up:

[0048] The process of Bottom-up is to input the picture into the neural network backbone to extract features, that is, to down-sample the pixels layer by layer and extract feature pixels;

[0049] The process of Top-down is to up-sample the feature map obtained by the high layer and then pass it down, because the features of the high layer contain rich semantic information, after the propagation of Top-down, the semantic information can be propagated to the low layer features, so that the low layer features also have rich semantic information in the case of resolution improvement;

[0050] Lateral connection is a feature fusion process, such as Fig. 2 that is, P1 is the fusion of the feature of C1 after 1*1 convolution and the up-sampled feature of P2.

[0051] In this embodiment, preferably, in the S5, the specific algorithm is: ①randomly initialize the parameters of the model; ②in each training batch, calculate the gradient of the loss function with respect to the parameters of the model by the back propagation algorithm; the back propagation algorithm will start from the loss function, propagate the gradient in the reverse direction along the computation graph, until all parameters are updated; ③use the calculated gradient to update the parameters of the model; the update rule of the SGD algorithm is:

[0052] wherein,

[0053] θ t represents the model parameters in the tth iteration, and a is the learning rate,

[0054] J(θ t ) is the loss function,

[0055] is the gradient of the loss function with respect to the parameters; the parameters are updated by subtracting the learning rate multiplied by the gradient from the current parameters; ④repeat steps ② and ③ until the loss function converges or the maximum number of iterations is reached.

[0056] In this embodiment, preferably, in A1, the size of each grid is fixed and predefined.

[0057] In this embodiment, preferably, in A2, the average pooling is: for each pooling region, the average value of all values in the region is calculated in the input feature map, and the average value is taken as the output value of the region; the pooling region is a square, and slides on the input feature map with a fixed step; the pooling operation is non-overlapping, i.e. there is no overlap between the pooling regions; its role is to smooth the input feature map.

[0058] In this embodiment, preferably, in A4, the fully connected layer usually includes one or more fully connected layers and activation functions, which are used to extract higher-level features from the features and generate the final output.

[0059] Although embodiments of the present application have been shown and described (see the detailed description above), it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made hereto without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method of detecting the appearance of a shelled poultry egg, characterized by: The method comprises the following steps: step one: a deep learning target detection algorithm; images of eggs marked as regions of interest are input into a convolutional neural network for training, local spatial information is extracted, and conversion and extraction of input features are realized; the specific steps are as follows: S1: collect pictures of various distribution conditions of eggs that may occur in actual production; S2: label the collected pictures, and label all the eggs in the pictures; the specific labeling method is to frame the eggs using a rectangle larger than the area of the egg, and add the label of the egg; all the egg labels are consistent; S3: divide the labeled pictures into a training set and a test set according to a 9:1 ratio; S4: input the pictures in the training set and the corresponding label information into the neural network, so that the internal weights of the neural network record the image information features of the eggs; use the FPN feature pyramid structure to fuse the feature maps of different levels together through up-sampling and down-sampling operations to generate a multi-scale feature pyramid; S5: use the stochastic gradient descent (SGD) to iteratively optimize and update the model parameters; S6: finally, use the predefined target box of different sizes and aspect ratios to cluster the target boxes of the training set to obtain the final detection box used for prediction; Step two: a deep learning classification algorithm; use the classified eggs with various defects and the intact eggs to input a convolutional network for feature extraction, use the spatial pyramid pooling (SPP) and the SPPF module combining a fully connected layer feature extraction module to extract multi-scale feature information without introducing additional parameters; The specific steps include: A1: the spatial pyramid pooling algorithm divides the input image into multiple grids, each grid having a different size; A2: for each grid, perform an average pooling operation to extract features within the region; A3: concatenate the pooling results of all grids to form a fixed-size feature vector; A4: input the pooled feature vector into a fully connected layer for classification or regression tasks; Step three: according to the deep learning target detection algorithm and the classification detection algorithm, the positions of the eggs and the detection results of a single shot are obtained, the positions are used to determine which groove the egg belongs to, then all the results of the eggs belonging to the groove are merged, and as long as one of the results is determined to be defective, the egg in the groove is considered to be a defective egg, otherwise, it is considered to be a good egg.

2. A method of detecting the appearance of a shelled poultry egg according to claim 1, characterized in that: The FPN feature pyramid structure mainly includes three parts: Bottom-up, Top-down and Lateral connection; Bottom-up: The process of Bottom-up is to input the picture into the main part of the neural network to extract features, that is, to perform down-sampling on the pixels layer by layer to extract feature pixels; The process of Top-down is to up-sample the feature map obtained by the high layer and then pass it down; since the high layer features contain rich semantic information, the semantic information can be propagated to the low layer features through top-down propagation, so that the low layer features also have rich semantic information in the case of resolution improvement; Lateral connection is a feature fusion process, that is, P1 is the feature after 1*1 convolution of C1 and the feature after up-sampling of P2.

3. A method of detecting the appearance of a shelled poultry egg according to claim 1, characterized in that: In S5, the specific algorithm is: ① randomly initialize the parameters of the model; ② in each training batch, calculate the gradient of the loss function with respect to the model parameters by the back propagation algorithm; the back propagation algorithm will start from the loss function, propagate the gradient along the computation graph in reverse, and update all parameters until all parameters are updated; ③ use the calculated gradient to update the parameters of the model; the update rule of the SGD algorithm is: wherein, θ t denotes the model parameters in the t-th iteration, and a is the learning rate, J(θ t ) is a loss function, is the gradient of the loss function with respect to the parameters; updating the parameters by subtracting the learning rate multiplied by the gradient from the current parameters; and repeating steps ② and ③ until the loss function converges or a maximum number of iterations is reached.

4. The method of claim 1, wherein: In A1, the size of each grid is fixed and predefined.

5. The method of claim 1, wherein: In A2, average pooling: for each pooling region, calculate the average value of all values in the region in the input feature map, and take the average value as the output value of the region; The pooling region is a square and slides on the input feature map with a fixed step; the pooling operation is non-overlapping, that is, there is no overlap between the pooling regions; Its role is to smooth the input feature map.

6. A method of inspecting the appearance of a shelled poultry egg according to claim 1, characterized in that: In A4, the fully connected layer usually includes one or more fully connected layers and activation functions, which are used to extract higher-level features from the features and generate the final output.

Images