A method for automatically detecting surface defects of chicken carcasses
Through the improved YOLOv7 model and feature pyramid module and other technologies, the subtle defect identification problem in chicken carcass surface defect detection is solved, and the rapid and accurate detection effect is achieved, meeting the efficient detection needs of broiler processing production lines.
Patent Information
- Application Number
- CN202410799582.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-20
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-06-20
AI Technical Summary
The existing chicken carcass surface defect detection methods are insufficient in terms of accuracy and efficiency, especially for subtle and irregular defects such as peanut defects, the detection effect is poor, and traditional methods are difficult to meet the rapid detection needs of broiler processing production lines.
Using the improved YOLOv7 object detection model, combined with the lightweight MobileViT network, SPPFS feature pyramid module, CBAM attention mechanism and Focal-EIoU Loss, we can detect chicken carcass surface defects to improve detection accuracy and real-time, and adaptively The weight adjustment mechanism improves the ability to identify subtle defects.
It realizes rapid, accurate and non-destructive testing of chicken carcass surface defects, significantly improves detection accuracy and efficiency, meets the real-time inspection needs of broiler processing production lines, and ensures product quality and consumer health and safety.
Smart Images

Figure CN118735873B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of deep learning and machine vision technology, and relates to a method for automatically detecting surface defects of chicken carcasses. Background Art
[0002] my country is the world's second largest producer and consumer of broiler chickens. In 2023, the total consumption reached 14.51 million tons, accounting for 14.38% of the world. Chicken has become the most important source of meat product supply in my country. With the increasing demand for chicken in the market, the requirements for chicken quality have also increased. Consumers will choose whether to buy chicken through sensory evaluation, and chicken carcasses with obvious defects in appearance will be regarded as defective products and will not be favored by consumers. Since the integrity and abnormal color spots of chicken carcasses are important features that affect the production, processing and sales of broilers, measures need to be taken to strengthen the inspection and management of chicken carcass quality. In the broiler production and processing industry, surface defects of chicken carcasses are important parameter indicators for assessing the grade of broilers. Both national standards and livestock and poultry production enterprises have corresponding standards for the grade of chicken carcass surface defects. The present invention mainly detects the three types of surface defects that appear most in the broiler market and production enterprises, namely, surface congestion, black breast and peanuts.
[0003] With the growth of broiler consumption in my country, consumers' demand for the quality of chicken carcasses continues to increase. my country's broiler slaughtering and processing industry is also developing rapidly, but there is a certain gap with developed countries in the detection and grading of surface defects of chicken carcasses. Foreign countries have developed automatic online detection systems to detect surface defects of chicken carcasses. The system is mainly based on dynamic simulation of visual images and traditional image processing technology. However, due to the large size, complexity and high price of the system, only a few large poultry slaughtering companies in China have introduced such systems; on the contrary, domestic small and medium-sized broiler slaughtering and processing companies mostly use manual naked eye observation to detect surface defects of chicken carcasses. This method requires a lot of human resources. One production line usually requires 4-5 workers. The labor intensity is high, and the accuracy and efficiency decrease with the increase of working hours, which seriously limits the processing capacity and stability.
[0004] Among the published patents, the following cases involve the detection of the surface integrity of chicken carcasses: Patent CN116563237B proposes a hyperspectral image detection method for chicken carcass defects based on deep learning. This method mainly detects skin damage defects on chicken carcasses, and does not make detailed defect distinctions on the surface defects of chicken carcasses. Although hyperspectral imaging technology can provide richer information and help to more accurately identify and distinguish targets in images, in actual production lines, camera acquisition is more suitable for the pursuit of fast and real-time surface defect detection. Among them, the combination of Faster R-CNN and U-net methods has a better detection effect on larger and obvious defects such as surface congestion and black breast, but for defects with unclear features, such as peanut defects with light color, dispersion and small shape, this method may not have a good detection effect.
[0005] Among the published patents, the following cases involve using YOLOv7 for target feature detection: Patent CN117237611A proposes a steel surface defect detection method based on improved YOLOv7. The core of this method is to improve and optimize the YOLOv7 model, aiming to effectively improve the detection accuracy of small target defects such as pitting and scratches by introducing coordinate convolution CoordConv, cutting CBS layer, reconstructing SPPCSPC using SPPF structure and Biformer attention mechanism, using new upsampling operation CARAFE and replacing CIoU loss function. Although this method performs well in steel defect detection, there are essential differences in the characteristics of steel surface defects and chicken carcass surface defects. Steel surface defects such as scratches and pitting usually have significant and recognizable physical features and sizes. Defects such as scale and inclusions may have different visual contrast on the steel surface from the surrounding materials, while chicken carcass surface defects such as congestion and flowers are more subtle, irregular in shape and have unclear features. The identification of these defects depends more on subtle changes in color and texture rather than obvious physical forms. Therefore, directly applying existing steel surface defect detection methods to chicken carcass defect detection may not achieve ideal detection results. In addition, steel defect detection usually pays more attention to real-time performance to meet the needs of efficient production line detection. However, in chicken carcass defect detection, although real-time performance is also a key factor, accuracy is the top priority. Any false detection or missed detection may have a serious impact on product quality and consumer health. Therefore, according to the characteristics of chicken carcass defect detection, the YOLOv7 model needs to be further optimized and adjusted.
[0006] Therefore, based on the above considerations, the present invention proposes an automatic detection method for chicken carcass surface defects based on improved YOLOv7. This method inherits the high efficiency of YOLOv7, and optimizes and improves the model specifically according to the characteristics of chicken carcass surface defects to ensure that it can accurately and quickly identify various subtle and irregular chicken carcass surface defects. Specifically for defects with unclear surface features of chicken carcasses, an adaptive weight adjustment mechanism is introduced, which can intelligently adjust the model's attention to different features according to the actual situation of the defects, thereby significantly improving the detection accuracy and ensuring that every subtle defect can be accurately captured. While meeting the rapid detection requirements of the production line, the model structure is simplified and the amount of calculation is reduced, so that the model can detect multiple objects in real time and quickly process large amounts of data, thereby meeting the dual requirements of efficiency and accuracy of the actual production line.
[0007] Therefore, how to quickly, non-destructively and accurately detect surface defects of chicken carcasses plays an important role in promoting the development of my country's poultry processing and production enterprises. Summary of the invention
[0008] To solve one or more of the above technical problems, the present invention proposes an automatic detection method for chicken carcass surface defects. The method is suitable for a chicken carcass processing and inspection production line, and realizes rapid, accurate and non-destructive detection of chicken carcass surface defects.
[0009] The technical solution of the present invention:
[0010] A method for automatically detecting surface defects of chicken carcasses, the steps are as follows:
[0011] The method for automatically detecting surface defects of chicken carcasses includes constructing an image acquisition module, an image processing and data set making module, a target detection module, and a defect classification module;
[0012] (1) Building an image acquisition module to obtain complete data on the surface of the chicken carcass;
[0013] S1.1 is designed to collect image data, ensure continuous transportation of chicken carcasses, and meet the requirements of body orientation, limb position and other postures of chicken carcasses at the inspection station, especially to facilitate the detection of underwing defects. A chicken carcass image acquisition device is designed to meet the production line rhythm of the enterprise and is suitable for clear and complete collection of surface defects of chicken carcasses. The overall operation mode of the device is: when the chicken carcass passes through the image acquisition station in the rotating area during transportation, that is, Figure 2Display position, by designing a guide rod of a certain width under the rotating mechanism, the chicken carcass is thrown away at an angle of 25° under the action of the rotating centrifugal force, so that the chicken wings and the chicken body can be fully separated, and then cooperate with the light source device composed of four equal-length high-brightness LED strip light sources spliced on a rectangular light group frame, so that the light is evenly irradiated to the surface of the chicken carcass. At the same time, each light source on the light group frame can be individually adjusted to a certain irradiation angle and brightness to match the surface morphology of the chicken carcass during transportation, so that different parts of the chicken carcass can achieve the best lighting brightness, thereby reducing the phenomenon of reflection and light spots, and then a clear and complete chicken carcass image is collected by an industrial camera, and the collected image is saved by a computer.
[0014] S1.2 The chicken carcass image acquisition device designed in step S1.1 contains a soft-trigger photography algorithm, which triggers photography based on the color difference between the chicken carcass and the background objects and the position of the chicken carcass in the image. First, the image size of each frame of the real-time video is fixed to 640×640 pixels; then, the image is converted from BGR to GRAY color format for subsequent processing; then, the image is corroded and expanded; then, the GRAY value of the chicken carcass color in the image is used as the color recognition threshold, and the chicken carcass and the background in the image are distinguished based on the color recognition method of threshold segmentation. According to the color recognition result, the position of the chicken carcass in the image is determined to provide accurate positioning information for subsequent trigger photography. After determining the position of the chicken carcass, the algorithm will select the center line of the image as the standard for triggering photography. When the chicken carcass image reaches the lower 2 / 3 of this standard position, the system will send a photo signal to trigger the camera to capture the image.
[0015] S1.3 After the triggering of the photo shooting in step S1.2 is completed, the image storage function in OpenCV is used to complete the image storage work, and the images are named according to the creation time and creation order. When the system receives the photo shooting trigger signal, a frame of the current video is saved as an image.
[0016] (2) Construct an image processing and data set production module, perform image preprocessing on the original images output by the image acquisition module, and use LabelImg software to annotate the preprocessed image data. Then perform data screening and data enhancement operations to obtain a data set, and divide it into a training set and a test set in a ratio of 7:3;
[0017] (3) Build a target detection module and use the improved YOLOv7 target detection model to detect surface defects of chicken carcasses;
[0018] S3.1 establishes a target detection network model based on the improved YOLOv7. This model specifically improves the YOLOv7 target detection model based on the characteristics of surface defects of chicken carcasses. Specifically, in order to better maintain the model detection accuracy while improving the detection speed, and to be more suitable for the scenario of chicken carcass surface defect detection with high real-time requirements, the backbone feature network of YOLOv7 is replaced with the lightweight MobileViT network; in order to better capture the subtle features of chicken carcass surface defects such as surface congestion, black breast and peanuts, in the pooling stage of feature fusion, the SPPCSPC feature pyramid module is replaced with the SPPFS feature pyramid module, which helps to reduce missed detections and false detections, thereby improving detection accuracy; in order to better capture the detailed information of chicken carcass surface defects, especially for defects such as peanuts that are relatively light in color, small and widely distributed, the CBAM attention mechanism is introduced in the neck, which can adaptively adjust the channel weights and spatial weights of the feature map, thereby optimizing the representation ability of specific defect features; in order to better comprehensively consider the problem of imbalanced defect categories and the uncertainty of target positioning, the Focal-EIoU Loss is added to the neck, which can better optimize the model performance and improve the accuracy and robustness of detection.
[0019] S3.2 sets the YOLOv7 model related training parameters, including input image size (img_size) of 640×640, training batch size (batch_size) of 32, starting epoch (init_Epoch) of 0, training epoch (Epoch) of 300, weight decay coefficient (weight_decay) of 0.0005, initial learning rate (lr) of 0.01, and uses SGD optimizer for optimization.
[0020] S3.3 uses the training set data in the image processing and data set production module to train the YOLOv7 target detection model after the parameters are set in step S3.2 to obtain the optimal target detection model;
[0021] S3.4 Use the optimal target detection model obtained in step S3.3 to perform target detection on the surface defects of the chicken carcass, and input the images in the test set into the trained target detection model to obtain the surface defect images containing the defect category, bounding box and confidence.
[0022] S3.5 summarizes and evaluates the improved YOLOv7 target detection model based on the model detection results in step S3.4. The evaluation indicators used for training and testing are mainly the mean average precision (mAP), which can be used to measure the performance of the model and more clearly reflect the improvement of the performance of the improved model.
[0023] (4) Construct a defect classification module and design a statistical method based on quantity and area to classify the defect information output by the target detection module;
[0024] S4.1 Count the number of defects on each body surface defect image containing defect category, bounding box and confidence obtained in step S3.4. First, traverse all bounding boxes, and extract the category information and location information for each bounding box. Then use a dictionary to store the number of defects of different categories, where the dictionary is an unordered set of key-value pairs, each key is unique and mapped to a value. In the present invention, the defect category is used as the key and the number of each category is used as the value. And when traversing each bounding box, the corresponding count is added to the dictionary according to its category label. Finally, the number of each defect is output and a corresponding defect statistics report is formed.
[0025] S4.2 Based on the surface defect images containing defect categories, bounding boxes and confidence obtained in step S3.4, the area of defects on each image is counted. Use python-OpenCV to extract the contours of the defects in the bounding box on each image, mark the contours with pixel values, and then calculate the pixel area of the defect based on the contour of the defect. Finally, the actual area of the chicken carcass defect is calculated based on the ratio between the pixel area of the established standard block and its actual area. Then the actual area of all defects is output to form a defect area statistics report. It should be noted that before taking the chicken carcass image, a standard white paper of known size is pasted on the surface of a selected chicken carcass. This standard white paper is used as a standard block to calculate the ratio of pixel area to actual area, and the pixel area of the defect is converted into the actual area, thereby improving the accuracy of the calculation.
[0026] S4.3 After the image is identified and located by the target detection module, it enters the defect grading module. In this module, firstly, according to step S4.1 and step S4.2, the number and area of defects are statistically calculated. Then, combined with the results obtained in step S4.1 and step S4.2, the grading standards are set and graded according to the number (Q), actual area (A) and distribution of defects on the surface of the chicken carcass. First, according to the data of step S4.1, the threshold of the number of defects (Q) is set to a small amount (Q<2), a medium amount (2≤Q<5) and a large amount (Q≥5). Next, according to the analysis results of step S4.2, the actual area (A) of the defects is subdivided. Defects within the same quantity level are further divided into different sub-levels according to their area size, and different area thresholds are set for different defects. Specifically, for defects of fresh and old flower types, the small area threshold is defined as (A<2cm 2 ), medium area (2cm 2 ≤A<5cm 2 ) and large area (A≥5cm2 For defects such as congestion and black chest, small area is defined as (A < 3cm 2 ), large area (A ≥ 3cm 2 ). According to the grading standards set above, defects are divided into level one, level two and level three based on the two dimensions of quantity and area. Level one defects are characterized by a small number of defects and a small area, which have little impact on the overall quality of the chicken carcass; level two defects are characterized by a medium number of defects or a medium area, which have a certain impact on the quality; level three defects are characterized by a large number of defects or a large area, which have a significant impact on the overall quality. For example, in a picture of a chicken carcass, 3 flower defects (Q=3) are detected, which falls within the threshold range of 2≤Q<5. At this time, it is first judged as a medium number of defects, and then the actual area (A) of two of the defects is detected to be 1.5cm 2 and 3.5cm 2 , which belong to the categories of small and medium areas respectively, while the area of the third defect (A) is 6cm 2 Combining the two dimensions of quantity and area, although the number of defects is at a medium level, there is a significant large area defect. Therefore, according to the grading standard, the flower defects in this picture are judged as level 3 defects, which means that these defects have a significant adverse effect on the overall quality of the chicken carcass.
[0027] Beneficial effects of the present invention:
[0028] The present invention provides a modular method for automatically detecting surface defects of chicken carcasses. Compared with other prior arts, the present invention has the following advantages:
[0029] (1) The present invention constructs a chicken carcass image acquisition device that meets the production line rhythm of the enterprise and is suitable for clear and complete acquisition of surface defects of chicken carcasses, which is used to realize the acquisition of chicken carcass data. During the transportation and acquisition process of the chicken carcass, in order to ensure that the chicken carcass meets the posture requirements such as body orientation and limb position at the detection station, especially to facilitate the detection of underwing defects, we specially designed a guide rod of a certain width under the rotating mechanism and on the inner side of the clamping mechanism, so that the clamping mechanism and the chicken carcass are thrown away at an angle of 25° under the action of the rotating centrifugal force generated by the guide rod and the transportation. This is the optimal acquisition posture obtained through repeated experiments. Through the designed guide rod, combined with the centrifugal force of the rotating transportation, the chicken wings are successfully thrown away to a large angle, so that the wings are fully separated from the chicken body. This innovative design ensures that the defects on the inner side of the wing can be completely collected, providing good image conditions for subsequent data collection and processing. The device not only improves the efficiency and accuracy of data acquisition, but also highlights the innovation and practicality in solving the problem of underwing defect collection.
[0030] (2) The present invention uses the improved YOLOv7 target detection network model to specifically predict chicken carcass defects. The improved YOLOv7 network model uses a lightweight MobileViT network to replace the backbone feature network, which can significantly improve the detection speed while maintaining the detection accuracy, making it more suitable for chicken carcass surface defect detection scenarios with high real-time requirements. In the pooling stage of feature fusion, the SPPCSPC feature pyramid module is replaced by the SPPFS feature pyramid module, which can better capture the subtle features of chicken carcass surface defects such as surface congestion, black breast and peanuts. At the same time, the CBAM attention mechanism is introduced in the neck, so that the network can better capture defects such as peanuts that are relatively light in color, small in shape and widely distributed, and can adaptively adjust the channel weights and spatial weights of the feature map. It helps to more accurately locate and identify such defects that are easily missed, thereby improving the overall detection accuracy. Focal-EIoU Loss is added to the neck, which comprehensively considers the imbalance problem of defect categories and the uncertainty of target positioning, and can better optimize the model performance and improve the accuracy and robustness of detection.
[0031] (3) The present invention innovatively combines the number and area of defects to achieve the classification of surface defects on chicken carcasses. The present invention can achieve real-time online detection, which is of great significance for promoting the automation and intelligence of the broiler slaughtering and processing industry and improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is the overall flow chart of the system of the present invention;
[0033] Figure 2 Figure 1. Hardware setup for chicken carcass image acquisition.
[0034] Figure 3 Software triggering program flow chart for chicken carcass;
[0035] Figure 4 A schematic diagram of an improved YOLOv7 target detection network in the present invention;
[0036] Figure 5 It is the structural diagram of the MV2Block module in the present invention;
[0037] Figure 6 It is the structural diagram of the MobileViTBlock module in the present invention;
[0038] Figure 7 It is the structural diagram of the SPPFS pyramid pooling module in the present invention;
[0039] Figure 8 This is a structural diagram of the CBAM attention mechanism module in the present invention;
[0040] In the figure: 1. Rotating mechanism; 2. Clamping mechanism; 3. Chicken carcass; 4. LED strip light source; 5. Light source device; 6. Industrial camera; 7. Display; 8. Industrial computer. DETAILED DESCRIPTION
[0041] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.
[0042] The present invention provides a method for detecting surface defects of chicken carcasses, comprising constructing an image acquisition module, an image processing and data set preparation module, a target detection module, and a defect classification module. Figure 1 The present invention is a flow chart of the method for automatically detecting surface defects of chicken carcasses;
[0043] Preferably, the image acquisition module comprises the following steps:
[0044] S1.1 In order to simulate the state of chicken carcasses hanging on the hanging conveyor line and obtain images, a chicken carcass image acquisition device (such as Figure 2 As shown), it includes: a rotating mechanism 1, a clamping mechanism 2, a chicken carcass 3, an LED strip light source 4, a light source device 5, an industrial camera 6, a display 7 and an industrial computer 8. Specifically, the industrial camera 6 is fixed on a vertically retractable bracket so that the camera lens is 150 cm from the ground, and the chicken carcass 3 is hung on the clamping mechanism 2 of the rotating mechanism 1. Due to the directional effect of the rotating mechanism, the chest or back of the chicken carcass can be directed toward the camera. In this experiment, the chicken breast is directed toward the camera. In addition, due to the effect of gravity, the suspended chicken carcass will naturally droop, so the chicken breast will be perpendicular to the optical axis of the camera. In order to ensure uniform illumination, four high-brightness LED strip light sources 4 of equal length are spliced on a rectangular light group frame to form a light source device 5, and are arranged in parallel at a position 0.8 m opposite to the measured chicken carcass. Each light source can be individually adjusted to a certain irradiation angle and brightness on the light assembly rack to match the body surface morphology of the chicken carcass hanging on the conveyor device, so that different parts of the chicken carcass can achieve the best lighting brightness, thereby reducing the phenomenon of reflection and light spots, so as to cooperate with the industrial camera 6 to collect a clear image of the chicken carcass. When the chicken carcass reaches the camera optical axis position, a trigger signal is given to the camera, and the camera automatically collects an image of the chicken carcass, and at the same time displays the collected image on the display 7 through the industrial computer 8, thereby completing the image collection work.
[0045] S1.2 The chicken carcass image acquisition device designed in step S1.1 contains a soft trigger photography algorithm, which triggers photography based on the color difference between the chicken carcass and the background objects and the position of the chicken carcass in the image. The specific process is as follows: Figure 3As shown in the figure. First, the size of each frame of the real-time video is fixed to 640×640 pixels; then, the image is converted from BGR to GRAY color format for subsequent processing; then, the image is corroded and expanded by image processing methods; then, the GRAY value of the chicken carcass color in the image is used as the color recognition threshold, and the chicken carcass and the background in the image are distinguished based on the color recognition method of threshold segmentation. According to the color recognition results, the position of the chicken carcass in the image is determined to provide accurate positioning information for subsequent triggering photography. After determining the position of the chicken carcass, the algorithm will select the data in the middle of the image as the standard for triggering photography. When the chicken carcass image reaches the lower 2 / 3 of this standard position, the system will send a photo signal to trigger the camera to take an image.
[0046] S1.3 After the triggering of the photo shooting in step S1.2 is completed, the image storage function in OpenCV is used to complete the image storage work, and the images are named according to the creation time and creation order. When the system receives the photo shooting trigger signal, a frame of the current video is saved as an image.
[0047] Preferably, the image processing and data set preparation module comprises the following steps:
[0048] S2.1 pre-processes the image output by the image acquisition module, mainly including cropping, random flipping, brightness adjustment and other operations. It can eliminate irrelevant information in the collected chicken carcass images, enhance the detectability of relevant information and simplify the data to the maximum extent.
[0049] S2.2 Use LabelImg software to mark the defect types and locations in the chicken carcass images in step 2.1, and use rectangular frames to frame the defective parts in each image and indicate the category to which they belong. Before using LabelImg software, you need to install pyqt5 and LabelImg in the environment you are using. After the installation is complete, you can enter LabelImg in the command line to classify and mark the images in the software interface that appears, and finally obtain a chicken carcass surface defect dataset.
[0050] S2.3 Types of surface defects on chicken carcasses may include surface congestion, black breast and peanuts. Surface congestion is manifested as bright red on the broken part of the chicken carcass legs or wings, and the congestion area under the wings is less than 3cm 2 , or pink under the wings; Black breast is specifically manifested as pink breast on the chicken carcass, with an area of no more than 3cm 2, slight water accumulation in the chest, pus is discharged after puncturing with scissors, and the puncture site does not exceed 0.5cm; peanuts include fresh flowers and old flowers. Fresh flowers are specifically manifested as the number of strip scars formed after damage to the skin of the chicken carcass is ≤5, and the length does not exceed 3cm, the number of large spot-like scars is ≤1, and the area of the dot-like flowers does not exceed 2CM*2CM, that is, the size of a 1 yuan coin, and it is bright red. The total scar area is <10cm 2 ; The specific manifestation of old spots is that the number of stripe scars formed by damage to the skin of the legs and back of the chicken carcass is ≤5, and the length does not exceed 3cm; or the number of small spot scars is ≤5, the number of large spot scars is ≤2, and they are yellow, and the total scar area is <10cm 2 The old spots on the chest are pale yellow cortical thickening or slight scabs, and the thickening and scab areas are less than 3cm 2 .
[0051] S2.4 improves the accuracy and efficiency of model training by screening the chicken carcass surface defect dataset obtained in step 2.2. The specific screening method is to screen according to the category and number of chicken carcass surface defects, which involves increasing the number of samples of certain defect categories and reducing the number of samples of other categories, so that the number of each defect in the chicken carcass image tends to be consistent.
[0052] S2.5 The chicken carcass surface defect dataset screened in step 2.4 is enhanced, which may include image translation, image scaling, horizontal flipping, vertical flipping, vertical and horizontal flipping, and adjusting the image brightness and contrast. The dataset after data enhancement has 2,000 images in total, which are divided into training set and test set in a ratio of 7:3, with 1,400 images in the training set and 600 images in the test set.
[0053] Preferably, the target detection module comprises the following steps:
[0054] S3.1 builds an improved YOLOv7 target detection model, which is mainly composed of the trunk, neck, and head. Figure 4 As shown in the figure, the following improvements are made to the YOLOv7 model:
[0055] 1) In order to better maintain the detection accuracy of the model while improving the detection speed, and to be more suitable for the scenario of chicken carcass surface defect detection with high real-time requirements, the backbone feature network is replaced with a lightweight MobileViT network. Mainly, the MobileViT network is based on the Transformer architecture, which significantly reduces the size of the model while maintaining performance, and is more efficient when processing large amounts of image data. It can not only understand global contextual information, but also effectively capture local details in the image, and has a good representation capability for features of different scales and positions. This greatly improves the accuracy of the specific location and shape of congestion, black breast and peanut defects on the surface of the chicken carcass, thereby improving the accuracy of detection.
[0056] 2) The MobileViT network mainly consists of two different blocks, namely MV2Block and MobileViTBlock. The MV2Block module (such as Figure 5 As shown in Figure 2, the dimension of the feature map is increased by 1×1 convolution, and then processed by 3×3 depthwise separable convolution. By modifying the dimension of the feature by first increasing and then decreasing, the loss of feature information is minimized and the extraction of target features is enhanced. In order to improve the model's ability to distinguish between lighter and smaller peanut defects and defective areas and normal areas, the SE attention mechanism is introduced in the MV2Block module to enhance the model's attention to defect features, which helps the model pay more attention to the texture features and color changes of the defective area. The MobileViTBlock module (as shown in Figure 2) Figure 6 The proposed method consists of three key components: local information encoding module, global information encoding module and feature fusion module. The feature map is locally reconstructed through 3×3 convolution and the number of channels is adjusted. The features of global information are then modeled. The channels are then adjusted to the original input size through the convolution layer and concatenated with the feature map of the original input. Finally, the output fused feature map is obtained through 3×3 convolution. It can make full use of local and global information, thereby improving the effect of extracting image features.
[0057] 3) In order to better capture the subtle features of chicken carcass surface defects such as surface congestion, black breast and peanuts, in the pooling stage of feature fusion, the SPPFS feature pyramid module is replaced by the SPPFS feature pyramid module. The SPPFS module structure is as follows: Figure 7As shown in the figure. Specifically, using the design concepts of SPPF and SPPCSPC, we introduced a new module called SPPFS spatial pyramid pooling that combines spatial pyramid pooling and feature selection. Its feature maps are sequentially aggregated through pooling kernels of different sizes, such as 5×5, 9×9, and 13×13, with a step size of 1. Finally, the obtained feature images are connected. Spatial pyramid pooling is performed by pooling kernels of different sizes, especially for small and scattered defects such as peanuts, which can better capture defect features ranging from small to large. Compared with the traditional SPPCSPC module, the SPPFS module can better handle multi-scale changes in the image, which is very important for simultaneously identifying large defects such as surface congestion and black chest and small defects such as peanuts. At the same time, the SPPFS module is more concise and efficient in structure, which can reduce the computational complexity of the model while maintaining the model performance.
[0058] 4) In order to better capture the detailed information of surface defects of chicken carcasses, especially for defects such as peanuts, which are relatively light in color, small in size and widely distributed, the CBAM attention mechanism is introduced at the neck, which can adaptively adjust the channel weights and spatial weights of the feature map, thereby optimizing the representation ability of specific defect features. The structure is as follows: Figure 8 The CBAM module mainly focuses on the location information of surface defects on the chicken carcass, multiplies the attention map by the input feature map to refine the self-attention features, improves the detail richness of the output features, and improves the accuracy of network detection.
[0059] 5) In order to better consider the imbalance of defect categories and the uncertainty of target positioning, the Focal-EIoU Loss is innovatively introduced at the neck to optimize the model, which can significantly improve the accuracy and robustness of detection. Specifically, EIoULoss integrates IoU loss (L IoU ), distance loss (L dis ) and width-height loss (L asp ), directly reduces the distance between the anchor point and the ground truth (gt), which can not only make accurate boundary predictions, but also timely capture these subtle defect differences for some relatively small or irregularly shaped defects such as congestion on the surface of the body and peanuts, thereby reducing the false detection rate and missed detection rate of the model. Focal Loss distinguishes high-quality and low-quality anchor boxes, improves the accuracy of the bounding box, helps the model better learn the characteristics of defective targets, and improves the accuracy and generalization ability of target detection. At the same time, it optimizes the problem of category imbalance, so that the model can better handle the loss weights of the three defects of congestion on the surface of the body, black chest and peanuts during training. In particular, for defects such as peanuts, which have a relatively large overall number compared to congestion on the surface of the body and black chest, this loss function can solve this problem well and improve the accuracy and precision of the model.
[0060] S3.2 sets the yolov7 related training parameters, including the input image size (img_size) of 640×640, the training batch size (batch_size) of 32, the starting generation (init_Epoch) of 0, the training generation (Epoch) of 300, the weight decay coefficient (weight_decay) of 0.0005, the initial learning rate (lr) of 0.01, and uses the SGD optimizer for optimization.
[0061] S3.3 uses the training set data in the image processing and dataset production module to train the YOLOv7 target detection model after the parameters are set in step S3.2 to obtain the optimal model.
[0062] S3.4 Use the optimal model obtained in step S3.3 to perform target detection on the surface defects of the chicken carcass, and input the images in the test set into the trained target detection model to obtain the surface defect images containing the defect category, bounding box and confidence.
[0063] S3.5 summarizes and evaluates the improved model detection results of step S3.4 for the sample data of step S2.5. The evaluation index used for training and testing is mainly the mean average precision (mAP), which can be used to measure the performance of the model and more clearly reflect the improvement of the performance of the improved model. The model test results in the following table show that the improved YOLOv7 target detection model has significantly improved the accuracy of detecting different defects compared to the original YOLOv7 target detection model.
[0064]
[0065] Preferably, the defect grading module comprises the following steps:
[0066] S4.1 Count the number of defects on each body surface defect image containing defect category, bounding box and confidence obtained in step S3.4. First, traverse all bounding boxes, and extract the category information and location information for each bounding box. Then use a dictionary to store the number of defects of different categories, where the dictionary is an unordered set of key-value pairs, each key is unique and mapped to a value. In the present invention, the defect category is used as the key and the number of each category is used as the value. And when traversing each bounding box, the corresponding count is added to the dictionary according to its category label. Finally, the number of each defect is output and a corresponding defect statistics report is formed.
[0067] S4.2 Based on the surface defect images containing defect categories, bounding boxes and confidence obtained in step S3.4, the area of defects on each image is counted. Use python-OpenCV to extract the contours of the defects in the bounding box on each image, mark the contours with pixel values, and then calculate the pixel area of the defect based on the contour of the defect. Finally, the actual area of the chicken carcass defect is calculated based on the ratio between the pixel area of the established standard block and its actual area. Then the actual area of all defects is output to form a defect area statistics report. It should be noted that before taking the chicken carcass image, a standard white paper of known size is pasted on the surface of a selected chicken carcass. This standard white paper is used as a standard block to calculate the ratio of pixel area to actual area, and the pixel area of the defect is converted into the actual area, thereby improving the accuracy of the calculation.
[0068] S4.3 After the image is identified and located by the target detection module, it enters the defect grading module. In this module, firstly, according to step S4.1 and step S4.2, the number and area of defects are statistically calculated. Then, combined with the results obtained in step S4.1 and step S4.2, the grading standards are set and graded according to the number (Q), actual area (A) and distribution of defects on the surface of the chicken carcass. First, according to the data of step S4.1, the threshold of the number of defects (Q) is set to a small amount (Q<2), a medium amount (2≤Q<5) and a large amount (Q≥5). Next, according to the analysis results of step S4.2, the actual area (A) of the defects is subdivided. Defects within the same quantity level are further divided into different sub-levels according to their area size, and different area thresholds are set for different defects. Specifically, for defects of fresh and old flower types, the small area threshold is defined as (A<2cm 2 ), medium area (2cm 2 ≤A<5cm 2 ) and large area (A≥5cm 2 For defects such as congestion and black chest, small area is defined as (A < 3cm 2 ), large area (A ≥ 3cm 2 ). According to the grading standards set above, defects are divided into level one, level two and level three based on the two dimensions of quantity and area. Level one defects are characterized by a small number of defects and a small area, which have little impact on the overall quality of the chicken carcass; level two defects are characterized by a medium number of defects or a medium area, which have a certain impact on the quality; level three defects are characterized by a large number of defects or a large area, which have a significant impact on the overall quality. For example, in a picture of a chicken carcass, 3 flower defects (Q=3) are detected, which falls within the threshold range of 2≤Q<5. At this time, it is first judged as a medium number of defects, and then the actual area (A) of two of the defects is detected to be 1.5cm 2 and 3.5cm2 , which belong to the categories of small and medium areas respectively, while the area of the third defect (A) is 6cm 2 Combining the two dimensions of quantity and area, although the number of defects is at a medium level, there is a significant large area defect. Therefore, according to the grading standard, the flower defects in this picture are judged as level 3 defects, which means that these defects have a significant adverse effect on the overall quality of the chicken carcass.
[0069] It should be understood that the contents not elaborated in this specification belong to the prior art. A person skilled in the art should understand that he can modify or replace the technical solution of the present invention, but any modification, replacement, improvement, etc. made within the spirit and principle of the present invention are within the protection scope of the present invention.
Claims
1. A method for automatically detecting surface defects of chicken carcasses, characterized in that: The method for automatically detecting surface defects of chicken carcasses includes constructing an image acquisition module, an image processing and data set making module, a target detection module, and a defect classification module; (1) Building an image acquisition module to obtain complete data on the surface of the chicken carcass; (2) Construct an image processing and data set production module, perform image preprocessing on the original images output by the image acquisition module, and use LabelImg software to annotate the preprocessed image data. Then perform data screening and data enhancement operations to obtain a data set, and divide it into a training set and a test set in a ratio of 7:3; (3) Build a target detection module and use the improved YOLOv7 target detection model to detect surface defects of chicken carcasses; S3.1 The improved YOLOv7 target detection network model is based on the YOLOv7 target detection network model and has the following improvements: 1) The backbone feature network of YOLOv7 is replaced with a lightweight MobileViT network; 2) In the pooling stage of feature fusion, the SPPCSPC feature pyramid module is replaced with the SPPFS feature pyramid module; 3) The CBAM attention mechanism is introduced in the neck; 4) Focal-EIoU Loss is added in the neck; S3.2 sets the training parameters of the improved YOLOv7 target detection network model; S3.3 uses the training set data in the image processing and data set preparation module to train the improved YOLOv7 target detection network model after the parameters are set in step S3.2 to obtain the optimal target detection model; S3.4 Using the optimal target detection model obtained in step S3.3, target detection is performed on the surface defects of the chicken carcass, and the images in the test set are input into the trained target detection model to obtain the surface defect image including the defect category, bounding box and confidence level; S3.5 summarizes and evaluates the improved YOLOv7 target detection model based on the body surface defect image including defect category, bounding box and confidence obtained in step S3.4; the evaluation index used for training and testing is mean average precision (mAP); (4) Construct a defect classification module and design a statistical method based on quantity and area to classify the defect information output by the target detection module; S4.1, based on the body surface defect images containing defect categories, bounding boxes and confidence levels obtained in step S3.4, count the number of defects on each image; first, traverse all bounding boxes, and for each bounding box, extract its category information and location information; Then use a dictionary to store the number of defects in different categories, where a dictionary is an unordered collection of key-value pairs, each key is unique and maps to a value, with the defect category as the key and the number of each category as the value; When traversing each bounding box, the corresponding count is added to the dictionary according to its category label; finally, the number of each defect is output and a corresponding defect statistics report is formed; S4.2 Count the area of defects on each image based on the surface defect images containing defect categories, bounding boxes and confidence levels obtained in step S3.4; use python-OpenCV to extract the contours of defects in the bounding boxes on each image, mark the contours with pixel values, and then calculate the pixel area of the defect based on the contours of the defect, and finally calculate the actual area of the chicken carcass defect based on the ratio between the pixel area of the established standard block and its actual area; then output the actual areas of all defects to form a defect area statistics report; before taking a chicken carcass image, stick a standard white paper of known size on the surface of a selected chicken carcass, use this standard white paper as a standard block, and use it to calculate the ratio of pixel area to actual area, and convert the pixel area of the defect into actual area; S4.3 After the image is identified and located by the target detection module, it enters the defect classification module; first, according to step S4.1 and step S4.2, the number and area of defects are statistically calculated; then, combined with the results of step S4.1 and step S4.2, the classification standards are set and classified according to the number Q, actual area A and distribution of defects on the surface of the chicken carcass; first, according to the data of step S4.1, the threshold of the number of defects Q is set to a small amount Q<2, a medium 2≤Q<5 and a large amount Q≥5; next, according to the analysis results of step S4.2, the actual area A of the defects is subdivided, and for defects of fresh and old flower types, the small area threshold is defined as A<2cm 2 , medium area is 2cm 2 ≤A<5cm 2 and large area A ≥ 5 cm 2 For defects such as congestion and black chest, small area is defined as A < 3 cm 2 , Large area: A ≥ 3cm 2 ; According to the grading standards set above, defects are divided into level one, level two and level three based on the two dimensions of quantity and area; level one defects are characterized by a small number of defects and a small area, which have little impact on the overall quality of the chicken carcass; level two defects are characterized by a medium number of defects or a medium area, which have a certain impact on the quality; level three defects are characterized by a large number of defects or a large area, which have a significant impact on the overall quality.
2. The method for automatically detecting surface defects of chicken carcasses according to claim 1, characterized in that: Step (1) constructs an image acquisition module, and the specific implementation process of obtaining complete data of the chicken carcass surface is as follows: S1.1 is to simulate the state of a chicken carcass hanging on a hanging conveyor line and obtain an image. According to the hanging conveyor line actually used in production, a chicken carcass image acquisition device is designed, including a rotating mechanism, a clamping mechanism, a chicken carcass, an LED strip light source, a light source device, an industrial camera, a display, and an industrial computer; the industrial camera is fixed on a vertically retractable bracket so that the lens of the industrial camera is 150 cm from the ground, and the chicken carcass is hung on the clamping mechanism of the rotating mechanism. Due to the directional effect of the rotating mechanism, the chest or back of the chicken carcass is facing Towards the industrial camera; due to gravity, the hanging chicken carcass will naturally droop, and the chicken breast will be perpendicular to the optical axis of the industrial camera; to ensure uniform lighting, four high-brightness LED strip light sources of equal length are spliced on a rectangular light assembly frame to form a light source device, and the light source device is arranged in parallel at a position 0.8m away from the opposite side of the measured chicken carcass; each LED strip light source can be individually adjusted on the light assembly frame to a certain irradiation angle and brightness to match the body surface morphology of the chicken carcass hanging on the conveying device, so that different parts of the chicken carcass can achieve the best lighting brightness, thereby reducing reflections and light spots, so as to cooperate with the industrial camera to collect clear chicken carcass images; S1.2 When the chicken carcass reaches the optical axis position of the industrial camera, a trigger signal is given to the industrial camera, and the camera automatically captures an image of the chicken carcass. At the same time, the captured image is displayed on the monitor through the industrial computer, thereby completing the image acquisition work; wherein, the chicken carcass image acquisition device designed in step S1.1 contains a soft trigger photography algorithm, which triggers photography based on the color difference between the chicken carcass and the background objects and the position of the chicken carcass in the image; first, the image size of each frame of the real-time video is fixed to 640×640 pixels; then, the image is converted from BGR to GRAY color. The color format is converted for subsequent processing; then, the image is corroded and expanded; subsequently, the gray value of the chicken carcass color in the image is used as the color recognition threshold, and the chicken carcass and the background in the image are distinguished based on the color recognition method of threshold segmentation; according to the color recognition result, the position of the chicken carcass in the image is determined to provide accurate positioning information for subsequent triggering photography; after determining the position of the chicken carcass, the soft trigger photography algorithm will select the center line of the image as the standard for triggering photography; when the chicken carcass image reaches the lower 2 / 3 of this standard position, the system will send a photography signal to trigger the camera to capture the image; S1.3 After the shooting is triggered in step S1.2, the image storage function in OpenCV is used to complete the image storage work, and the image is named according to the creation time and creation order; when the system receives the photo trigger signal, a frame of the current video is saved as an image.
3. The method for automatically detecting surface defects of chicken carcasses according to claim 1, characterized in that: The training parameters in step S3.2 include input image size of 640×640, training batch size of 32, starting epoch of 0, training epoch of 300, weight decay coefficient of 0.0005, initial learning rate of 0.01, and optimization using SGD optimizer.
