A method and system for detecting eggshell defects of poultry eggs based on MSFE-YOLO

Abstract

The application belongs to the technical field of defect detection, and discloses an eggshell defect detection method and system based on MSFE-YOLO, which constitutes a complete detection system through image acquisition, illumination enhancement, data transmission and defect detection, realizes integrated detection from image acquisition to defect recognition, can effectively replace the existing manual detection method in the application of eggshell defect detection, improves the automation level of egg quality screening, improves the ADEM module in the MSFE-YOLO network model, reduces the channel dimension, strengthens the attention to high-frequency features, adjusts the feature fusion network based on the PAN-FPN architecture, increases the attention to shallow features, introduces a large-scale convolution kernel to enhance the attention to the target context information, and finally realizes the targeted adjustment of the adjustable parameters N , realizes the targeted adjustment of the adjustable parameters IoU , to adapt to the needs of different detection tasks, and a good balance between detection accuracy and computational overhead is achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of defect detection, and particularly relates to a poultry eggshell defect detection method and system based on MSFE-YOLO. BACKGROUND

[0002] The poultry eggshell is the primary barrier for hatching, and has important functions such as maintaining the stability of egg contents, blocking the invasion of external microorganisms, and ensuring the integrity of nutritional components. Surface defects such as eggshell cracks, stains, and uneven calcification are directly related to the storage safety and commercial circulation qualification rate of egg products. At present, traditional defect screening methods mainly rely on manual observation, which is not only inefficient, but also easily affected by subjective factors. Developing automatic defect detection technology can provide real-time quality monitoring solutions for poultry breeding enterprises, reduce the loss of agricultural products, and has important people's livelihood value and economic significance. At present, image recognition-based non-destructive detection technology for eggshell defects has become a hot research direction for scholars.

[0003] Common defects of poultry eggshells mainly include cracks, cracks, and damage, and the defect accounts for a small proportion of the overall pixels, which is a typical small target detection problem. YOLOv8 adopts the PAN-FPN structure, which combines the advantages of feature pyramid network (FPN) and path aggregation network (PANet). FPN integrates multi-level features through a top-down strategy, enhancing the ability to capture context information, while PANet further introduces a bottom-up path aggregation mechanism, improving the utilization efficiency of shallow features, and showing good detection performance on small target datasets.

[0004] According to the design depth and width of the backbone network, YOLOv8 algorithm is divided into n, s, m, l, and x versions, and the model size can be adjusted by adjusting the parameters. With the increase of depth and width, the detection accuracy of the algorithm will also be improved, but at the same time, the huge parameter quantity and computing overhead cannot be ignored. YOLOv8s is the second smallest version of YOLOv8 algorithm parameters, and the parameter quantity of YOLOv8s algorithm is smaller, which is suitable for actual deployment and application, but its detection accuracy on the poultry egg defect dataset still needs to be further improved.

[0005] Therefore, it is of great market prospect to develop a method that has a good balance between detection accuracy and computing overhead and can achieve good results in the task of egg defect detection. SUMMARY

[0006] The purpose of the present application is to solve the problems of the prior art, provide an MSFE-YOLO-based poultry eggshell defect detection method and system, which combines a deep learning algorithm with an embedded device, realizes integrated detection from image acquisition to defect recognition, is suitable for small target detection scenes of poultry eggshell defects, balances detection accuracy and calculation overhead, and obtains good detection effect in the poultry egg screening scene.

[0007] To solve the technical problem, the technical scheme of the present application is: an MSFE-YOLO-based poultry eggshell defect detection method, comprising the following steps:

[0008] Step 1: collect poultry eggshell defect images, including cracks, fissures and damage defects;

[0009] Step 2: pre-process and data-label the collected poultry eggshell defect images to obtain a final poultry eggshell defect dataset;

[0010] Step 3: input the poultry eggshell defect dataset into an improved MSFE-YOLO network model for training, optimize the parameters of the improved MSFE-YOLO network model through a loss function, and obtain a trained defect detection model;

[0011] The improved MSFE-YOLO network model comprises a backbone network, a feature fusion network and a detection head, the feature extraction process of the backbone network is in turn a first CBS, a second CBS, a first C2f-ADEM, a third CBS, a second C2f-ADEM, a fourth CBS, a third C2f-ADEM, a fifth CBS, a fourth C2f-ADEM and a SPPF, the feature fusion network is an improved MFFN feature fusion network, the SPDConv in the improved MFFN feature fusion network is connected to the output of the second CBS, the output of the SPDConv is connected to the second Concat, the output of the second C2f-ADEM is also connected to the second Concat, the output of the SPPF is connected to the first Upsample, and the output of the first Upsample is connected to the first Concat; the output of the third C2f-ADEM is also connected to the first Concat, the first Concat is connected to the second Concat through the output of the fifth C2f-ADEM and the second Upsample, the output of the second Concat is connected to a large-scale convolution layer Conv, the output of the fifth C2f-ADEM is connected to the third Concat, the output of the SPPF is also connected to the fourth Concat, and the outputs of the large-scale convolution layer Conv, the third Concat and the fourth Concat are respectively connected to the detection head;

[0012] Step 4: Detecting the images of the eggshells of the poultry eggs to be detected using the trained defect detection model to determine whether there are related defects, and obtaining a detection result.

[0013] Step 5: According to the detection result, using the two-finger gripper to pick up the defective eggs to realize the real-time diversion of the defective eggs and the intact eggs.

[0014] Preferably, the step 1 specifically comprises: acquiring high-quality poultry eggshell defect images using a high-definition camera carrying a ring-shaped LCD light source module, adjusting the pose of the high-definition camera, so that the optical axis of the high-definition camera and the light source incidence angle have a normal coefficient relationship, and avoiding mirror reflection.

[0015] Preferably, the pre-processing in the step 2 includes geometric correction, denoising and equalization, and the specific process is: first, performing geometric correction on the poultry eggshell defect images to eliminate the deformation caused by the shooting angle; second, using Gaussian filtering or median filtering for denoising processing to reduce the interference of high-frequency noise on the learning of the improved MSFE-YOLO network model; and finally, uniformly scaling the poultry eggshell defect images to the input size of the improved MSFE-YOLO network model and performing normalization processing.

[0016] Preferably, the first C2f-ADEM, the second C2f-ADEM, the third C2f-ADEM and the fourth C2f-ADEM in the step 3 have the same module structure, which are all to pass the input features through a 1x1 CBS convolution layer, then split the output features into two parts in the channel dimension, one part passes through the ADEM module, the other part keeps the original features, then the output results of the two branches are spliced in the channel dimension, and finally the features are output after passing through a 1x1 CBS convolution layer.

[0017] Preferably, the ADEM module first inputs the input features into the conditional convolution, then splits the input features into two parts in the channel dimension, one part passes through the detail enhancement convolution and the point-wise convolution, then the other part is directly spliced with the other part in the feature dimension, and finally the features are output after channel shuffling.

[0018] Preferably, the detection head in the step 3 adopts an anchor-free detection mechanism, and three detection heads detect on three feature maps of different scales respectively, and output five physical quantities: predicted category, predicted frame center point x, y coordinates, predicted frame width and height.

[0019] Preferably, the poultry eggshell defect dataset in the step 3 is divided into a training set, a validation set and a test set according to a ratio of 7:2:1, and is input into the improved MSFE-YOLO network model for training.

[0020] Preferably, the loss function in step 3 includes three parts: a classification loss function, a bounding box loss function and a confidence loss function.

[0021] The bounding box loss function adopts N-CIoU a loss function, N-CIoU The expression of the loss function is:

[0022] ;

[0023] Wherein:

[0024] L N-CIoU indicates N-CIoU loss;

[0025] NIoU indicates N-IoU the proposed intersection over union loss;

[0026] indicates CIoU the bounding box loss of ; wherein indicates the square of the Euclidean distance between the center points of the predicted box and the real box, which is used to measure the position difference of the two box center points; a as a balance factor for adjusting the weight of the shape similarity penalty term; v indicates the measure of the aspect ratio difference between the predicted box and the real box;

[0027] The N-CIoU proposed intersection over union loss NIoU The expression is:

[0028] ;

[0029] Wherein:

[0030] inter indicates the intersection between the real box and the predicted box area;

[0031] union indicates the union of the real box and the predicted box area;

[0032] N indicates an adjustable parameter.

[0033] Preferably, an MSFE-YOLO-based poultry eggshell defect detection system, the system comprises:

[0034] An image acquisition device acquires high-quality poultry eggshell defect images through a high-definition camera;

[0035] An illumination enhancement device supplements light to the poultry eggshell through a ring-shaped LCD to enhance the recognizability of the poultry eggshell defect image;

[0036] The data transmission module realizes image transmission between the high-definition camera and the embedded device through Wi-Fi transmission.

[0037] The defect detection module realizes automatic detection of the input poultry eggshell image to be detected by deploying an improved MSFE-YOLO network model in the embedded device.

[0038] The egg clamping device clamps the defective eggs by using two finger clamps to realize real-time classification of the eggs.

[0039] Compared with the prior art, the application has the following advantages:

[0040] (1) The application provides a poultry eggshell defect detection method and system based on MSFE-YOLO, which constitutes a complete detection system through an image acquisition device, an illumination enhancement device, a data transmission module, and a defect detection module, realizes integrated detection from image acquisition to defect recognition, and can effectively replace the existing manual detection method in the application of poultry eggshell defect detection to improve the automation level of poultry egg quality screening.

[0041] (2) The application improves the ADEM module in the MSFE-YOLO network model by introducing conditional convolution and detail enhancement convolution to generate a condition vector for each input feature, and dynamically adjusts the weight combination of multiple basic convolution kernels using the vector, so that the CBS convolution layer can adapt to the feature requirements of different poultry eggshell defect samples, and improve the flexibility and expression ability of the model.

[0042] (3) The application improves the MFFN feature fusion network in the MSFE-YOLO network model based on the PAN-FPN architecture to improve the YOLOv8 feature fusion network, uses SPDConv to downsample the output of the second CBS, and fuses the output result with the P3 layer to enhance the attention to detail information in large-scale feature maps.

[0043] (4) The application improves the MSFE-YOLO network model by designing a C2f-ADEM module and constructing a MFFN multi-scale feature fusion network, which improves the feature extraction and feature fusion effect while greatly reducing the parameter amount, and achieves a good balance between detection accuracy and computational overhead. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The flowchart of the poultry eggshell defect detection method based on MSFE-YOLO of the application;

[0045] Figure 2 The arrangement diagram of the poultry eggshell defect detection system based on MSFE-YOLO of the application;

[0046] Figure 3 The architecture diagram of the improved MSFE-YOLO network model of the present application;

[0047] Figure 4 The C2f-ADEM module structure diagram of the present application;

[0048] Figure 5 The ADEM module structure diagram of the present application;

[0049] Figure 6 The information transmission flowchart of the improved MSFE-YOLO network model of the present application;

[0050] Explanation of reference signs:

[0051] 1, first CBS, 2, second CBS, 3, first C2f-ADEM, 4, third CBS, 5, second C2f-ADEM, 6, fourth CBS, 7, third C2f-ADEM, 8, fifth CBS, 9, fourth C2f-ADEM, 10, SPPF, 11, first Upsample, 12, first Concat, 13, fifth C2f-ADEM, 14, second Upsample, 15, second Concat, 16, SPDConv, 17, large-scale convolution layer Conv, 18, third Concat, 19, fourth Concat. DETAILED DESCRIPTION

[0052] The present application is described in detail below in conjunction with the drawings and specific embodiments, but the present application is not limited only to these embodiments. The present application covers any substitutions, modifications, equivalent methods and solutions made on the essence and scope of the present application. In order to make the public have a thorough understanding of the present application, specific details are described in detail in the following embodiments of the present application, and the present application can also be completely understood without the description of these details for those skilled in the art.

[0053] The present application proposes a poultry egg surface defect detection method based on MSFE-YOLO, as well as a non-contact detection method, system and detection equipment for poultry egg surface defect detection. The high-definition industrial camera with ring-shaped LED fill light is used to collect the egg surface defect image, the camera calibration parameters, light source configuration parameters, imaging synchronization timing and other data are obtained through initialization, the contrast of the defect area is improved through histogram equalization, frequency domain filtering enhancement and other preprocessing methods, and finally the improved MSFE-YOLO network model deployed in the embedded development board is used to complete the automatic recognition of defects. The present application effectively improves the intelligentization and automation degree of perfect egg and defective egg classification, improves the classification efficiency, and greatly reduces the cost of manual screening.

[0054] As Figure 1As shown, the present application discloses a kind of poultry eggshell defect detection method based on MSFE-YOLO, comprising the following steps:

[0055] Step 1: collect poultry eggshell defect image, including crack, crack and damage defect;

[0056] Step 2: the collected poultry eggshell defect image is preprocessed and data labeled, and the final poultry eggshell defect data set is obtained;

[0057] Step 3: the poultry eggshell defect data set is input into the improved MSFE-YOLO network model for training, the parameters of the improved MSFE-YOLO network model are optimized by loss function, the weight with highest accuracy is saved, which is used for subsequent defect detection task, and the trained defect detection model is obtained;

[0058] As Figure 3 shown, the improved MSFE-YOLO network model includes backbone network, feature fusion network and detection head, the feature extraction process of backbone network is first CBS, second CBS, first C2f-ADEM, third CBS, second C2f-ADEM, fourth CBS, third C2f-ADEM, fifth CBS, fourth C2f-ADEM and SPPF in turn, feature fusion network is improved MFFN feature fusion network, SPDConv in improved MFFN feature fusion network is connected with the output of second CBS, the output of SPDConv is connected with second Concat, the output of second C2f-ADEM is also connected with second Concat, the output of SPPF is connected with first Upsample, the output of first Upsample is connected with first Concat;The output of third C2f-ADEM is also connected with first Concat, first Concat is connected with second Concat through the output of fifth C2f-ADEM and second Upsample, the output of second Concat is connected with large scale convolution layer Conv, the output of fifth C2f-ADEM is connected with third Concat, the output of SPPF is also connected with fourth Concat, the output of large scale convolution layer Conv, third Concat and fourth Concat is connected with detection head respectively;

[0059] Step 4: using the trained defect detection model to detect poultry eggshell image to be detected, whether there is related defect is detected, and detection result is obtained;

[0060] Step 5: according to detection result, utilize two-fingered gripper to clamp defective egg, realize the real-time shunt of defective egg and intact egg.

[0061] Preferably, step 1 specifically involves: using a high-definition camera carrying a ring-shaped LCD light source module to acquire high-quality images of defects in the shells of poultry hatching eggs, adjusting the pose of the high-definition camera so that the optical axis of the high-definition camera and the incident angle of the light source are in a Norman coefficient relationship to avoid specular reflection.

[0062] Preferably, the preprocessing in step 2 includes geometric correction, denoising, and equalization. The specific process is as follows: First, geometric correction is performed on the images of defects in the shells of poultry hatching eggs to eliminate deformation caused by the shooting angle; second, Gaussian filtering or median filtering is used for denoising to reduce the interference of high-frequency noise on the learning of the improved MSFE-YOLO network model; finally, the images of defects in the shells of poultry hatching eggs are uniformly scaled to the input size of the improved MSFE-YOLO network model and normalized.

[0063] like Figure 4 As shown, preferably, the first C2f-ADEM, the second C2f-ADEM, the third C2f-ADEM and the fourth C2f-ADEM in step 3 have the same module structure. They all pass the input features through a 1×1 CBS convolutional layer, and then split the output features into two parts in the channel dimension. One part passes through the ADEM module, and the other part retains the original features. Then, the output results of the two branches are concatenated in the channel dimension, and finally the output features are output after passing through a 1×1 CBS convolutional layer.

[0064] like Figure 5 As shown, preferably, the ADEM module first inputs the input features into a conditional convolution, then divides them into two parts through channels. One part is subjected to detail enhancement convolution and pointwise convolution, and then the features are directly concatenated with the other part in the feature dimension. Finally, the features are output after channel shuffling.

[0065] Preferably, in step 3, the detection head adopts an anchor-free detection mechanism, with three detection heads performing detection on three feature maps of different scales (the three feature maps of different scales are all abstracted from the same image of defects in the shells of poultry hatching eggs), and outputting five physical quantities: predicted category, x and y coordinates of the center point of the predicted bounding box, and width and height of the predicted bounding box.

[0066] Preferably, in step 3, the dataset of defects in the shells of avian hatching eggs is divided into a training set, a validation set, and a test set in a ratio of 7:2:1, and then input into the improved MSFE-YOLO network model for training.

[0067] Preferably, the loss function in step 3 includes three parts: classification loss function, bounding box loss function, and confidence loss function;

[0068] The bounding box loss function adopts N-CIoU loss functionN-CIoU The expression of the loss function is:

[0069] ;

[0070] wherein:

[0071] L N-CIoU represents N-CIoU the loss;

[0072] NIoU represents N-IoU the proposed IoU loss;

[0073] represents CIoU the bounding box loss of ; wherein represents the square of the Euclidean distance between the center points of the predicted box and the real box, used to measure the position difference of the two box center points; a as a balance factor for adjusting the weight of the shape similar penalty term; v represents the measure of the aspect ratio difference between the predicted box and the real box;

[0074] The N-CIoU proposed IoU loss NIoU is expressed as:

[0075] ;

[0076] wherein:

[0077] inter represents the intersection between the real box and the predicted box area;

[0078] union represents the union of the real box and the predicted box area;

[0079] N represents an adjustable parameter, in the original IoU the numerator and denominator are added by N times inter , the value is 0, 1, 2, …, 15.

[0080] In order to realize the automatic and accurate grabbing of defective eggs in step 5, the present application first carries out hand-eye calibration based on the Eye-to-Hand mode, places a chessboard calibration plate at a fixed position of the conveying belt, calculates the camera external parameters through OpenCV, and combines the TCP position of the mechanical arm to complete the alignment of the coordinate system, so as to ensure the accurate conversion of the vision coordinates to the mechanical arm coordinates; for the problem of depth information missing of the monocular camera, the fixed-height conveying belt provides a known Z-axis reference value. For the original XYZ three-axis right-angle coordinate mechanical arm, the inverse kinematics analytical solution is directly used to plan the straight-line grabbing path, without complex iterative calculation, and at the same time, the FSR thin film force sensor is integrated on the gripper, and the force feedback closed-loop control is realized by the STM32 embedded controller, so as to avoid eggshell breakage and ensure reliable grabbing. The system transmits the egg defect detection frame coordinates output by the MSFE-YOLO to the mechanical arm controller in real time through the ROSModbus TCP protocol and superimposes the target size as additional information, and uses the calibration matrix for coordinate conversion. For the continuously moving conveying belt, the encoder pulse signal triggers a hardware interrupt, and the Kalman filtering algorithm is embedded in the mechanical arm controller to predict and compensate the dynamic position offset of the eggs, so as to realize the high-precision and high-reliability automatic grabbing function with minimal hardware modification.

[0081] As shown in Figure 2 Preferably, an MSFE-YOLO-based poultry eggshell defect detection system, the system comprises:

[0082] An image acquisition device acquires high-quality poultry eggshell defect images through a high-definition camera;

[0083] An illumination enhancement device enhances the recognizability of the poultry eggshell defect images by supplementing light to the poultry eggshell through a ring-shaped LCD;

[0084] A data transmission module realizes the transmission of poultry eggshell defect images between the high-definition camera and the embedded device through Wi-Fi transmission;

[0085] A defect detection module realizes the automatic detection of input poultry eggshell images to be detected by deploying an improved MSFE-YOLO network model in the embedded device; has the functions of gripper control, image processing and instruction sending; and transmits data to a computer device.

[0086] An egg gripping device receives the gripper control instruction, grips the defective eggs with a two-finger gripper, and realizes real-time classification of the eggs.

[0087] Example 1

[0088] As shown in Figure 1As shown, the embodiment discloses a poultry eggshell defect detection method based on MSFE-YOLO, comprising the following steps:

[0089] Step 1: Use a high-definition camera carrying an LCD light source module to obtain high-quality poultry eggshell defect images, adjust the camera pose to make the optical axis of the high-definition camera and the light source incidence angle have a normal coefficient relationship, and avoid mirror reflection.

[0090] Collect poultry eggshell defect images, including cracks, cracks, and damage defects. Among them, each type of defect has 1000 images as the initial data set.

[0091] Step 2: Preprocess the obtained poultry eggshell defect images: including image correction, denoising, equalization, etc.; and mark the data to obtain the final poultry eggshell defect data set;

[0092] Preprocess the collected poultry eggshell images. First, geometrically correct the image to eliminate the deformation caused by the shooting angle. Secondly, use Gaussian filtering or median filtering for denoising to reduce the interference of high-frequency noise on model learning. Finally, the image is uniformly scaled to the model input size and normalized. The above operations aim to improve data quality, reduce irrelevant variable interference, and thus enhance the model's ability to identify target features.

[0093] Step 3: Transfer the preprocessed poultry eggshell defect data set to the improved MSFE-YOLO network model constructed to obtain a trained defect detection model;

[0094] The high-definition camera is directly connected to the embedded device through the USB3.0 interface, uses the UVC protocol to encode real-time image data in MJPG / H.264, and then calls the deployed improved MSFE-YOLO network model for defect detection after the embedded device captures the video stream through OpenCV.

[0095] The improved MSFE-YOLO network model includes a backbone network, a feature fusion network, and a detection head; the backbone network includes a C2f-ADEM module, and the feature fusion network is a MFFN multi-scale feature fusion network.

[0096] The C2f-ADEM module is designed in the backbone network, which effectively improves the model capacity and enhances the network's ability to capture detailed features while reducing the parameter amount by introducing conditional convolution and detail enhancement convolution. In the feature fusion network part, the utilization rate of shallow features is strengthened, and large-scale convolution kernels are added to fully exploit the target context information. Specifically, first, the B2 layer (between the second CBS and the first C2f-ADEM) of the backbone network is fused with the P3 layer (between the second Upsample and the second Concat) of the feature fusion network. Since the output feature size of the B2 layer does not match the output feature size of the P3 layer, the output feature of the B2 layer needs to be down-sampled. By utilizing shallow features twice, the model's ability to capture detailed information can be effectively improved, which is beneficial to improving the model's recognition ability for small target defects. After realizing B2-P3 feature fusion, a large-scale convolution layer Conv is added to effectively obtain the context information of the defect target by increasing the model's receptive field, thereby better realizing the classification and positioning of defects. In the detection head, the MSFE-YOLO algorithm uses a decoupled detection head mechanism, which processes classification and regression tasks through two branches, allowing each task to use specialized feature learning and optimization, thereby improving performance and accuracy. The MSFE-YOLO algorithm designs the C2f-ADEM module and constructs the MFFN multi-scale feature fusion network, which improves the feature extraction and feature fusion effect while greatly reducing the parameter amount.

[0097] As shown in Figure 4 , the C2f-ADEM module first passes the input feature through a 1x1 CBS convolution layer, then splits the output feature into two parts in the channel dimension, one of which passes through the ADEM module, and the other part remains the original feature, then the output results of the two branches are spliced in the channel dimension, and finally the output feature is output after passing through a 1x1 CBS convolution layer.

[0098] As shown in Figure 5 , the ADEM module uses conditional convolution and detail enhancement convolution to expand the model capacity while enhancing the model's ability to perceive detailed texture information. The input feature is first input into the conditional convolution to dynamically generate convolution kernels for different samples, thereby enhancing the model's generalization. Then the output feature is split into two parts in the channel dimension (channel segmentation), one of which passes through the detail enhancement convolution and the 1x1 pointwise convolution to enhance the model's detail perception ability and strengthen the information exchange between channels, and then it is directly spliced with the other part in the feature dimension. Finally, the output feature is output after strengthening the information exchange between channels through channel shuffling.

[0099] In addition to reducing network parameters by channel segmentation, to further realize the lightweight of the module, the application applies different convolution kernels to each channel of the standard convolution and the difference convolution in the detail enhancement convolution to reduce the calculation complexity and improve the detection efficiency of the model.

[0100] The different scale feature maps output by the backbone network each contain information in different spatial scales. The shallow layer feature has the largest output size and also contains more abundant semantic information. The deep layer feature has a smaller output size, but has a better abstract expression effect on the whole input image and contains abundant spatial information. For the task of detecting the damage of the eggshell of poultry eggs, the detection objects include the breakage and cracks of the eggshell, which account for a small proportion in the whole background and belong to a typical small target detection task. Therefore, the detection of the defects of the eggshell of poultry eggs needs to rely more on the abundant semantic information contained in the shallow layer feature, such as Figure 6 As shown in the figure, the MFFN multi-scale feature fusion network realizes the full use of the shallow layer feature by fusing the B2 and P3 feature layers in depth, and enhances the capturing ability of the model to the micro target detail feature. By introducing a large-scale convolution kernel, the attention to the context information of the target is strengthened, so that the positioning of the target is better realized. Specifically, the implementation process is as follows:

[0101] Firstly, the B2 and P3 feature maps are fused, and after upsampling and downsampling, cross fusion is performed. For the standard input size of 640 x 640 picture, the output feature of the B2 layer is 160 x 160, and the output feature size of the P3 layer is 80 x 80. The output sizes of the two do not match, and cannot be spliced in the channel dimension. Therefore, a downsampling layer needs to be added after the B2 layer to match the size of the output feature of the P3 layer.

[0102] Further, the downsampling operation usually adopts a standard convolution with a step of 2 to process the feature map, but the standard convolution with a step of 2 for downsampling is easy to cause the loss of fine-grained information, which affects the feature representation. Therefore, the application introduces SPDConv to rearrange the spatial dimension information to the depth dimension, which completely retains all the information in the channel dimension, so as to realize lossless downsampling.

[0103] The receptive field of the 3x3 convolution kernel is relatively small, and can only cover a local area of the input feature map. In the small target detection task, the target context information is crucial for accurate positioning and classification. Therefore, after realizing the deep fusion of the B2-P3 layer, a large size convolution layer with K=31 is added in the feature fusion network to strengthen the attention to the target context information. To balance the high computational overhead brought by the large size convolution layer, the standard convolution in the large size layer is replaced by the designed deep dilated convolution in combination with the ideas of deep convolution and dilated convolution, which reduces the model parameter amount while ensuring the detection accuracy. Specifically, the deep dilated convolution parameters are as follows: the convolution kernel k = 7, padding padding= (k / 2 ） dilation , the dilated rate dilation= 5.

[0104] The detection head of the present application adopts an anchor-free detection mechanism, and three detection heads respectively detect on three feature maps of different scales to output five physical quantities: predicted category, predicted frame center point x, y coordinates, predicted frame width and height.

[0105] The loss function of the present application adopts N-CIoU loss function, N-CIoU The expression of the loss function is as follows:

[0106] ;

[0107] Among them:

[0108] L N-CIoU represents N-CIoU loss;

[0109] NIoU represents N-IoU the proposed intersection over union loss;

[0110] represents CIoU the bounding box loss of ; wherein represents the square of the Euclidean distance between the center points of the predicted frame and the real frame, which is used to measure the position difference of the two frame center points; a as a balance factor for adjusting the weight of the shape similar penalty term; v represents the measure of the aspect ratio difference between the predicted frame and the real frame;

[0111] The proposed intersection over union loss N-CIoU has the expression as follows: NIoU

[0112] ; ​

[0113] wherein:

[0114] inter represents the intersection between the real box and the predicted box region;

[0115] union represents the union between the real box and the predicted box region;

[0116] N represents an adjustable parameter.

[0117] the N-CIoU using N-IoU instead of CIoU in the loss function IoU loss, compared with IoU , the N-IoU by introducing an adjustable parameter N The intersection-over-union loss can be adjusted specifically. By changing the value of N , the boundary box of the low IoU value sample can be appropriately accelerated to converge, and the fine granularity of the boundary box regression of the high IoU value sample can be increased, improving the robustness and accuracy of the model.

[0118] The defects in the poultry eggshell defect data set are small and irregular in outline, and in the actual detection process, the IoU value is usually low. N-IoU An adjustable parameter N is introduced, and a series of comparative tests are set by changing the value of N , and the most suitable parameter value for the poultry eggshell defect detection task is selected, which greatly improves the generalization and flexibility of the model compared with N , so that the loss function can be adjusted specifically when facing different detection tasks.

[0119] Step 4: Use the trained defect detection model to detect the poultry eggshell image to be detected; judge whether there is a defect, output the detection result, and upload the result to the background monitoring system.

[0120] Through communication connection to the Wi-Fi transmission module, the poultry eggshell image to be detected and the detection result are transmitted to the background monitoring system in real time, so as to realize non-destructive detection and monitoring of the poultry eggshell.

[0121] Step 5: According to the detection result, use the two-finger gripper to clamp the defective egg, realize real-time shunting of defective eggs and intact eggs.

[0122] To further verify the superiority of the improved MSFE-YOLO network model, under the condition of ensuring the same training environment, the YOLOv8s detection algorithm was compared on the collected poultry eggshell defect data set. The evaluation indexes were mAP50 and parameter quantity, taking into account the detection accuracy and complexity of the model. Compared with the YOLOv8s algorithm, the improved MSFE-YOLO network model had mAP50 of 69.2%, 92.3%, 90.1%, 90.5% and 99.5% in the egg, cracked egg, white egg, dirty egg and spotted egg categories, respectively. Compared with YOLOv8s, the detection accuracy of the first four categories was improved by 11.7%, 0.6%, 2.8% and 0.2%, respectively, and the detection accuracy of the spotted egg category was the same as that of YOLOv8s. The improved MSFE-YOLO network model with a parameter quantity of 8.08M exceeded the YOLOv8s algorithm with a parameter quantity of 11.15M in mAP50, proving that the improved MSFE-YOLO network model of the application achieved a good balance between detection accuracy and computational overhead, and could achieve good results in the poultry egg defect detection task.

[0123] The preferred embodiments of the application are described in detail above, but the application is not limited to the above-mentioned embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the application.

[0124] Many other changes and modifications can be made without departing from the concept and scope of the application. It should be understood that the application is not limited to the specific embodiments, and the scope of the application is defined by the appended claims.

Claims

1. A method for detecting shell defects in poultry hatching eggs based on MSFE-YOLO, characterized in that, Includes the following steps: Step 1: Collect images of defects in the shells of poultry hatching eggs, including cracks, fissures, and broken defects; Step 2: Preprocess and label the collected images of poultry hatching egg shell defects to obtain the final poultry hatching egg shell defect dataset; Step 3: Input the dataset of defects in the shells of avian hatching eggs into the improved MSFE-YOLO network model for training. Optimize the parameters of the improved MSFE-YOLO network model through the loss function to obtain the trained defect detection model. The improved MSFE-YOLO network model includes a backbone network, a feature fusion network, and a detection head. The feature extraction process of the backbone network is as follows: first CBS, second CBS, first C2f-ADEM, third CBS, second C2f-ADEM, fourth CBS, third C2f-ADEM, fifth CBS, fourth C2f-ADEM, and SPPF. The feature fusion network is an improved MFFN feature fusion network. In the improved MFFN feature fusion network, SPDConv is connected to the output of the second CBS, the output of SPDConv is connected to the second Concat, the output of the second C2f-ADEM is also connected to the second Concat, the output of SPPF is connected to the first Upsample, and the output of the first Upsample is connected to the first Concat. The output of the third C2f-ADEM is also connected to the first Concat. The first Concat is connected to the second Concat through the outputs of the fifth C2f-ADEM and the second Upsample. The output of at is connected to the large-scale convolutional layer Conv, the output of the fifth C2f-ADEM is connected to the third Concat, and the output of SPPF is also connected to the fourth Concat. The outputs of the large-scale convolutional layer Conv, the third Concat, and the fourth Concat are connected to the detection head respectively. The module structures of the first C2f-ADEM, the second C2f-ADEM, the third C2f-ADEM, and the fourth C2f-ADEM are the same. They all pass the input features through a 1×1 CBS convolutional layer, and then split the output features into two parts in the channel dimension. One part passes through the ADEM module, and the other part retains the original features. Then, the outputs of the two branches are concatenated in the channel dimension, and finally, the features are output after passing through a 1×1 CBS convolutional layer. The ADEM module first inputs the input features into a conditional convolution, and then splits them into two parts through the channels. One part passes through detail enhancement convolution and pointwise convolution, and then is directly concatenated with the other part in the feature dimension. Finally, the features are output after channel shuffling. Step 4: Use the trained defect detection model to detect the shell images of the poultry hatching eggs to be tested, detect whether there are any related defects, and obtain the detection results; Step 5: Based on the test results, use two-finger grippers to pick up defective eggs, achieving real-time separation of defective eggs from intact eggs.

2. The method for detecting defects in the shells of poultry hatching eggs based on MSFE-YOLO according to claim 1, characterized in that, Step 1 specifically involves: using a high-definition camera equipped with a ring-shaped LCD light source module to acquire high-quality images of defects in the shells of poultry hatching eggs; adjusting the pose of the high-definition camera to make the optical axis of the high-definition camera and the incident angle of the light source have a Norman coefficient relationship to avoid specular reflection.

3. The method for detecting defects in the shells of poultry hatching eggs based on MSFE-YOLO according to claim 1, characterized in that, The preprocessing in step 2 includes geometric correction, denoising, and equalization. Specifically, the process is as follows: First, geometric correction is performed on the images of defects in the shells of poultry hatching eggs to eliminate deformation caused by the shooting angle; second, Gaussian filtering or median filtering is used for denoising to reduce the interference of high-frequency noise on the learning of the improved MSFE-YOLO network model; finally, the images of defects in the shells of poultry hatching eggs are uniformly scaled to the input size of the improved MSFE-YOLO network model and normalized.

4. The method for detecting defects in the shells of poultry hatching eggs based on MSFE-YOLO according to claim 1, characterized in that, In step 3, the detection head adopts an anchor-free detection mechanism, with three detection heads performing detection on feature maps at three different scales, and outputting five physical quantities: predicted category, x and y coordinates of the predicted bounding box center point, and width and height of the predicted bounding box.

5. The method for detecting defects in the shells of poultry hatching eggs based on MSFE-YOLO according to claim 1, characterized in that, In step 3, the dataset of defects in the shells of avian hatching eggs is divided into a training set, a validation set, and a test set in a ratio of 7:2:1, and then input into the improved MSFE-YOLO network model for training.

6. The method for detecting defects in the shells of poultry hatching eggs based on MSFE-YOLO according to claim 4, characterized in that, The loss function in step 3 includes three parts: classification loss function, bounding box loss function, and confidence loss function. The bounding box loss function adopts the N-CIoU loss function, and the expression of the N-CIoU loss function is: in: L N-CIoU Indicates N-CIoU loss; NIoU represents the crossover ratio loss proposed by N-IoU; Represents the bounding box loss of CIoU; where ρ 2 (b,b gt The square of the Euclidean distance between the center points of the predicted and ground truth boxes is used to measure the positional difference between the two box center points; 'a' is used as a balancing factor to adjust the weight of the shape similarity penalty term; 'v' represents a measure of the aspect ratio difference between the predicted and ground truth boxes; the N-CIoU proposed cross-union ratio loss NIoU expression is: in: inter represents the intersection of the ground truth bounding box and the predicted bounding box; Union represents the set of regions between the ground truth bounding boxes and the predicted bounding boxes. N represents an adjustable parameter.

7. A poultry hatching eggshell defect detection system based on MSFE-YOLO, characterized in that, For implementation of the MSFE-YOLO-based method for detecting defects in the shells of avian hatching eggs as described in any one of claims 1 to 6, the system comprises: The image acquisition device uses a high-definition camera to acquire high-quality images of defects in the shells of poultry hatching eggs. The light enhancement device uses a ring-shaped LCD to supplement the light on the shells of poultry hatching eggs, thereby enhancing the recognizability of images showing defects in the eggshells. The data transmission module enables the transmission of images of poultry eggshell defects between a high-definition camera and an embedded device via Wi-Fi. The defect detection module automatically detects the shells of poultry hatching eggs by deploying an improved MSFE-YOLO network model within an embedded device. The hatching egg grasping device uses two-finger grippers to grasp defective hatching eggs, enabling real-time classification of hatching eggs.

Images