Appearance detection method for unshelled poultry eggs

Abstract

The invention discloses a method for detecting the appearance of a shelled egg. The method comprises the following steps: step 1, a deep learning target detection algorithm: inputting an image of which an egg is marked as a region of interest into a convolutional neural network in advance for training; step 2, a deep learning classification algorithm: inputting classified eggs with various defects and intact eggs into a convolutional network for feature extraction in advance, and extracting multi-scale feature information by using an SPPF module combined with a spatial pyramid pooling SPP and a feature extraction module of a full connection layer; 3, according to a deep learning target detection algorithm and a classification detection algorithm, obtaining the positions of the eggs and a detection result of single shooting, according to the position information, judging which groove the eggs belong to, and then combining all results of the eggs belonging to the groove; according to the method, manpower resources can be greatly saved, uninterrupted detection requirements can be provided, full detection is realized, and the working efficiency is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of food quality detection, and in particular relates to a method for detecting the appearance of shelled poultry eggs. Background Art

[0002] There are some problems in the process of shelling eggs: 1) eggshell residue, the shelling may not be clean and the eggshell may remain on the surface of the egg; 2) egg cracking, the egg white cracks during the shelling process, causing the yolk to be exposed, 3) egg yolk contamination, due to the cracking of the egg white of other eggs, part of the yolk is contaminated on the egg white of other eggs, 4) the air chamber is too large, and the air chamber is more obvious after shelling, 5) black spots, there are some small black spots on the egg white, 6) depression, the egg is squeezed during the processing, causing depressions on the surface, 7) cracking, although the egg white is damaged, the yolk is not exposed, 8) honeycomb eggs, small and dense holes appear on the surface of the egg, 9) black eggs, the surface is no longer smooth white, but dark and black, which is often related to the poor quality of the egg itself. The above problems seriously affect the quality and even affect food safety.

[0003] At present, the appearance quality inspection of eggs after shelling in food processing equipment is mainly based on manual visual inspection. In the existing literature, we only found relevant research on the appearance inspection and quality inspection of unshelled egg products. It mainly focuses on the inspection of egg cracks, dirty shells, and the inspection of the internal quality of eggs, including loose yolk eggs and fertilized eggs; no research on the effective inspection of the shelling process and egg quality after the egg products are cooked and shelled is found. For this reason, the present invention proposes a method for the appearance inspection of poultry eggs after shelling. Summary of the invention

[0004] The object of the present invention is to provide a method for detecting the appearance of shelled poultry eggs to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for detecting the appearance of shelled poultry eggs, comprising step 1: a deep learning target detection algorithm: inputting an image of an egg marked as a region of interest into a convolutional neural network for training, extracting local spatial information, and realizing the conversion and extraction of input features; the specific steps are:

[0006] S1: Collect pictures of various egg distribution situations that may occur in actual production;

[0007] S2: Label the collected pictures and all the eggs in the pictures. The specific labeling method is to use a regular rectangle slightly larger than the egg area to frame the egg and add the egg's label name. All the eggs have the same label name.

[0008] S3: Divide the labeled images into training set and test set in a ratio of 9:1;

[0009] S4: Input the training set images and the corresponding annotation information into the neural network, so that the internal weights of the neural network record the image information characteristics of the egg; use the FPN feature pyramid structure to fuse the feature maps of different levels together through upsampling and downsampling operations to generate a multi-scale feature pyramid;

[0010] S5: Use stochastic gradient descent SGD to iteratively optimize and update model parameters;

[0011] S6: Finally, the target boxes of the training set are clustered using the pre-defined target boxes of different sizes and aspect ratios to generate the detection boxes for prediction;

[0012] Step 2: Deep learning classification algorithm: Use the classified eggs with various defects and intact eggs in advance, input them into the convolutional network for feature extraction, and use the SPPF module that combines the spatial pyramid pooling SPP and the fully connected layer feature extraction module to extract multi-scale feature information without introducing additional parameters; the specific steps include:

[0013] A1: The spatial pyramid pooling algorithm divides the input image into multiple grids, each with a different size;

[0014] A2: For each grid, perform an average pooling operation to extract features within the area;

[0015] A3: Concatenate the pooling results of all grids together to form a fixed-size feature vector;

[0016] A4: Input the pooled feature vector into the fully connected layer for classification or regression tasks;

[0017] Step 3: Use the deep learning target detection algorithm and classification detection algorithm to obtain the location of the egg and the detection results of a single shot. Determine which groove the egg belongs to based on the location information, and then merge all the results of the eggs belonging to this groove. If one of them is judged to be defective, the egg in the groove is considered to be a defective egg, otherwise it is considered to be a good egg.

[0018] Preferably, the FPN feature pyramid structure mainly includes three parts: Bottom-up, Top-down and Lateral connection; Bottom-up:

[0019] The bottom-up process is to input the image into the backbone of the neural network to extract features, that is, to downsample the pixels layer by layer and extract feature pixels;

[0020] The top-down process is to upsample the feature map obtained at the high level and then pass it downward. Since the high-level features contain rich semantic information, these semantic information can be propagated to the low-level features through top-down propagation, so that the low-level features also have rich semantic information when the resolution is improved;

[0021] Lateral connection is a feature fusion process, that is, P1 is the fusion of the features of C1 after 1*1 convolution and the features of P2 after upsampling.

[0022] Preferably, in S5, the specific algorithm is: ① randomly initialize the parameters of the model; ② in each training batch, calculate the gradient of the loss function with respect to the model parameters by the back propagation algorithm; the back propagation algorithm starts from the loss function and back propagates the gradient along the calculation graph until all parameters are updated; ③ use the calculated gradient to update the parameters of the model; the update rule of the SGD algorithm is:

[0023] in,

[0024] θ t represents the model parameters in the tth iteration, α is the learning rate,

[0025] J(θ t ) is the loss function,

[0026] It is the gradient of the loss function with respect to the parameters; update the parameters by subtracting the current parameters from the learning rate multiplied by the gradient; ④ Repeat steps ② and ③ until the loss function converges or the maximum number of iterations is reached.

[0027] Preferably, in A1, the size of each grid is fixed and predefined.

[0028] Preferably, in A2, average pooling: for each pooling area, the average value of all values ​​in the area is calculated in the input feature map, and the average value is used as the output value of the area; the pooling area is a square and slides on the input feature map with a fixed step; the pooling operation is non-overlapping, that is, there is no overlap between the pooling areas; its function is to smooth the input feature map.

[0029] Preferably, in A4, the fully connected layer generally includes one or more fully connected layers and activation functions, which are used to extract higher-level features from the features and generate the final output.

[0030] Compared with the prior art, the beneficial effects of the present invention are: the present invention designs a method for detecting the appearance of shelled poultry eggs, which can save a lot of human resources, provide uninterrupted detection needs, and realize full inspection, thereby improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the process of the present invention;

[0032] Figure 2 Schematic diagram of the feature fusion process of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1 to Figure 2 This embodiment provides a technical solution: a method for detecting the appearance of shelled poultry eggs, comprising step 1: a deep learning target detection algorithm: inputting an image of an egg marked as a region of interest into a convolutional neural network for training, extracting local spatial information, and realizing the conversion and extraction of input features; the specific steps are:

[0035] S1: Collect pictures of various egg distribution situations that may occur in actual production;

[0036] S2: Label the collected pictures and all the eggs in the pictures. The specific labeling method is to use a regular rectangle slightly larger than the egg area to frame the egg and add the egg's label name. All the eggs have the same label name.

[0037] S3: Divide the labeled images into training set and test set in a ratio of 9:1;

[0038] S4: Input the training set images and the corresponding annotation information into the neural network, so that the internal weights of the neural network record the image information features about the egg; use the FPN feature pyramid structure to fuse the feature maps of different levels through upsampling and downsampling operations to generate a multi-scale feature pyramid; the high-level features of the feature pyramid contain rich semantic information (conducive to classification), but their resolution is low and it is difficult to accurately save the location information of the object; although the low (shallow) layer features have less semantic information, their resolution is high and can contain accurate object location information (conducive to detection and segmentation); FPN is to fuse low-level features with high-level features to obtain a target detection structure that is accurate in both recognition and positioning; the top-down part of the feature pyramid mainly realizes the fusion of features at different levels through upsampling and superposition with feature maps of coarser granularity, while the bottom-up part is to fuse feature maps from different levels by using a convolutional layer;

[0039] S5: Use stochastic gradient descent SGD to iteratively optimize and update model parameters;

[0040] S6: Finally, the target frames of the training set with different sizes and aspect ratios are clustered using the pre-defined target frames to obtain the detection frames used for prediction. By outputting the target detection area of ​​the egg using this method, it is possible to effectively avoid misjudging the empty grooves, and the calculation amount is reduced without having to judge each groove position separately, thereby increasing the running speed of the device.

[0041] Step 2: Deep learning classification algorithm: Use the classified eggs with various defects and intact eggs in advance, input them into the convolutional network for feature extraction, and use the SPPF module that combines the spatial pyramid pooling SPP and the fully connected layer feature extraction module to extract multi-scale feature information without introducing additional parameters; the specific steps include:

[0042] A1: The spatial pyramid pooling algorithm divides the input image into multiple grids, each with a different size;

[0043] A2: For each grid, perform an average pooling operation to extract features within the area;

[0044] A3: The pooling results of all grids are concatenated together to form a fixed-size feature vector; since each grid has a different size, features at different scales can be fully utilized. Therefore, regardless of the size of the input image, spatial pyramid pooling can generate a fixed-size feature vector. This allows the spatial pyramid to be connected to subsequent layers such as the fully connected layer without the need to adjust the input to a fixed size.

[0045] A4: Input the pooled feature vector into the fully connected layer for classification or regression tasks;

[0046] Step 3: Use the deep learning target detection algorithm and classification detection algorithm to obtain the location of the egg and the detection results of a single shot. Determine which groove the egg belongs to based on the location information, and then merge all the results of the eggs belonging to this groove. If one of them is judged to be defective, the egg in the groove is considered to be a defective egg, otherwise it is considered to be a good egg.

[0047] In this embodiment, preferably, the FPN feature pyramid structure mainly includes three parts: Bottom-up, Top-down and Lateral connection; Bottom-up:

[0048] The bottom-up process is to input the image into the backbone of the neural network to extract features, that is, to downsample the pixels layer by layer and extract feature pixels;

[0049] The top-down process is to upsample the feature map obtained at the high level and then pass it downward. Since the high-level features contain rich semantic information, these semantic information can be propagated to the low-level features through top-down propagation, so that the low-level features also have rich semantic information when the resolution is improved;

[0050] Lateral connection is a feature fusion process, such as Figure 2 , that is, P1 is the fusion of the features of C1 after 1*1 convolution and the features of P2 after upsampling.

[0051] In this embodiment, preferably, in S5, the specific algorithm is: ① randomly initialize the parameters of the model; ② in each training batch, calculate the gradient of the loss function with respect to the model parameters by the back propagation algorithm; the back propagation algorithm starts from the loss function and back propagates the gradient along the calculation graph until all parameters are updated; ③ use the calculated gradient to update the parameters of the model; the update rule of the SGD algorithm is:

[0052] in,

[0053] θ t represents the model parameters in the tth iteration, α is the learning rate,

[0054] J(θ t ) is the loss function,

[0055] It is the gradient of the loss function with respect to the parameters; update the parameters by subtracting the current parameters from the learning rate multiplied by the gradient; ④ Repeat steps ② and ③ until the loss function converges or the maximum number of iterations is reached.

[0056] In this embodiment, preferably, in A1, the size of each grid is fixed and predefined.

[0057] In this embodiment, preferably, in A2, average pooling is performed: for each pooling area, the average value of all values ​​in the area is calculated in the input feature map, and the average value is used as the output value of the area; the pooling area is a square and slides on the input feature map with a fixed step; the pooling operation is non-overlapping, that is, there is no overlap between the pooling areas; its function is to smooth the input feature map.

[0058] In this embodiment, preferably, in A4, the fully connected layer generally includes one or more fully connected layers and activation functions, which are used to extract higher-level features from the features and generate the final output.

[0059] Although embodiments of the present invention have been shown and described (see the above detailed description for details), it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting the appearance of shelled poultry eggs, characterized in that: The method includes the following steps: Step 1: Deep learning target detection algorithm: input the image with the egg marked as the region of interest into the convolutional neural network for training, extract the local spatial information, and realize the conversion and extraction of input features; the specific steps are: S1: Collect pictures of various egg distribution situations that may occur in actual production; S2: Label the collected pictures and all the eggs in the pictures. The specific labeling method is to use a regular rectangle slightly larger than the egg area to frame the egg and add the egg's label name. All the eggs have the same label name. S3: Divide the labeled images into training set and test set in a ratio of 9:1; S4: Input the training set images and the corresponding annotation information into the neural network, so that the internal weights of the neural network record the image information characteristics of the egg; use the FPN feature pyramid structure to fuse the feature maps of different levels together through upsampling and downsampling operations to generate a multi-scale feature pyramid; S5: Use stochastic gradient descent SGD to iteratively optimize and update model parameters; S6: Finally, the target boxes of the training set are clustered using the pre-defined target boxes of different sizes and aspect ratios to generate the detection boxes for prediction; Step 2: Deep learning classification algorithm: Use the classified eggs with various defects and intact eggs in advance, input them into the convolutional network for feature extraction, and use the SPPF module that combines the spatial pyramid pooling (SPP) and the fully connected layer feature extraction module to extract multi-scale feature information without introducing additional parameters; The specific steps include: A1: The spatial pyramid pooling algorithm divides the input image into multiple grids, each with a different size; A2: For each grid, perform an average pooling operation to extract features within the area; A3: Concatenate the pooling results of all grids together to form a fixed-size feature vector; A4: Input the pooled feature vector into the fully connected layer for classification or regression tasks; Step 3: Use the deep learning target detection algorithm and classification detection algorithm to obtain the location of the egg and the detection results of a single shot. Determine which groove the egg belongs to based on the location information, and then merge all the results of the eggs belonging to this groove. If one of them is judged to be defective, the egg in the groove is considered to be a defective egg, otherwise it is considered to be a good egg.

2. The method for detecting the appearance of shelled poultry eggs according to claim 1, wherein: The FPN feature pyramid structure mainly includes three parts: Bottom-up, Top-down and Lateral connection; Bottom-up: The bottom-up process is to input the image into the backbone of the neural network to extract features, that is, to downsample the pixels layer by layer and extract feature pixels; The top-down process is to upsample the feature map obtained at the high level and then pass it downward. Since the high-level features contain rich semantic information, these semantic information can be propagated to the low-level features through top-down propagation, so that the low-level features also have rich semantic information when the resolution is improved; Lateral connection is a feature fusion process, that is, P1 is the fusion of the features of C1 after 1*1 convolution and the features of P2 after upsampling.

3. The method for detecting the appearance of shelled poultry eggs according to claim 1, wherein: In S5, the specific algorithm is: ① randomly initialize the parameters of the model; ② in each training batch, calculate the gradient of the loss function with respect to the model parameters through the back propagation algorithm; the back propagation algorithm starts from the loss function and back propagates the gradient along the calculation graph until all parameters are updated; ③ use the calculated gradient to update the parameters of the model; the update rule of the SGD algorithm is: in, θ t represents the model parameters in the tth iteration, α is the learning rate, J(θ t ) is the loss function, It is the gradient of the loss function with respect to the parameters; update the parameters by subtracting the current parameters from the learning rate multiplied by the gradient; ④ Repeat steps ② and ③ until the loss function converges or the maximum number of iterations is reached.

4. The method for detecting the appearance of shelled eggs according to claim 1, wherein: In A1, the size of each grid is fixed and pre-defined.

5. The method for detecting the appearance of shelled poultry eggs according to claim 1, wherein: In A2, average pooling: for each pooling area, the average value of all values ​​in the area is calculated in the input feature map, and the average value is used as the output value of the area; The pooling area is square and slides on the input feature map with a fixed stride; the pooling operation is non-overlapping, that is, there is no overlap between the pooling areas; Its function is to smooth the input feature map.

6. The method for detecting the appearance of shelled poultry eggs according to claim 1, characterized in that: In A4, the fully connected layer generally includes one or more fully connected layers and activation functions, which are used to extract higher-level features from the features and generate the final output.

Images