Methods, systems, inspection robots, and storage media for detecting dead chickens in cages

Abstract

This invention discloses a method, system, inspection robot, and storage medium for detecting dead chickens in cages. The method includes: acquiring chicken video streams via a gimbal camera during inspection; preprocessing single-frame chicken images to remove the cage railing area; inputting the chicken image with the railing area removed into a trained dead chicken detection model for detection, outputting chicken category, detection box location information, and confidence score; if a dead chicken is detected, saving the dead chicken information and controlling the gimbal camera to track and capture the dead chicken's location within a preset time; and uploading the saved dead chicken information to a management platform for display after inspection. This invention not only improves the efficiency and accuracy of dead chicken detection but also significantly reduces labor costs by reducing the need for manual inspection; simultaneously, it protects the health of the flock, thereby improving overall breeding efficiency and product quality.

Description

Technical Field

[0001] This invention relates to a method, system, inspection robot, and storage medium for detecting dead chickens in cages. It belongs to the field of dead chicken detection. Background Technology

[0002] In modern animal husbandry, improving production efficiency and animal welfare are among the core objectives. Especially in large-scale cage-raising environments, the rapid and accurate detection of dead chickens is crucial for maintaining flock health and preventing disease transmission. However, traditional detection methods often rely on manual inspections, which are not only time-consuming but also inefficient. Currently, the highest layer of stacked cages used in large-scale chicken houses can reach over 2 meters, making manual inspections particularly challenging for these high-rise cages. Summary of the Invention

[0003] In view of this, the present invention provides a method, system, inspection robot, and storage medium for detecting dead chickens in cages. It can automatically capture images of chickens and make predictions, effectively identify dead chickens and determine their locations. This not only improves the efficiency and accuracy of dead chicken detection, but also significantly reduces labor costs by reducing the need for manual inspections. At the same time, the rapid removal of dead chickens helps prevent the spread of diseases and protects the health of the flock, thereby improving overall breeding efficiency and product quality.

[0004] The first objective of this invention is to provide a method for detecting dead chickens in cages.

[0005] The second objective of this invention is to provide a system for detecting dead chickens in cages.

[0006] The third objective of this invention is to provide an inspection robot.

[0007] A fourth objective of this invention is to provide a computer-readable storage medium.

[0008] The first objective of this invention can be achieved by adopting the following technical solution:

[0009] A method for detecting dead chickens in cages, the method comprising:

[0010] During the inspection, video streams of chickens were captured using a gimbal camera.

[0011] Preprocess the single-frame chicken image to remove the chicken cage fence area from the single-frame chicken image;

[0012] The images of chickens with the cage railings removed are input into a trained dead chicken detection model for detection, and the chicken category is output, which includes healthy chickens and dead chickens;

[0013] If a dead chicken is detected, the information of the dead chicken is saved, and the gimbal camera is controlled to track and photograph the location of the dead chicken within a preset time.

[0014] After the inspection is completed, the information on the dead chickens is uploaded to the management platform for display.

[0015] Furthermore, the dead chicken detection model is an improved YOLOv7 model, with the backbone network replaced by a lightweight MobileNetv3 network. A CBAM attention module is added to the feature enhancement network, along with Repulsion Loss and DIoU-NMS non-maximum suppression.

[0016] Furthermore, the formula for the Repulsion Loss is as follows:

[0017] ;

[0018] in, and The formula is as follows:

[0019] ;

[0020] ;

[0021] in:

[0022] ;

[0023] in, It is a prediction box. It is the true bounding box with the highest IoU score. and The difference is evaluated using IoG loss: , It is a smoothing parameter that adjusts the sensitivity of Repulsion Loss to outliers. and Indicates different groups of prediction boxes. It is an identity function. It is a small constant when divided by zero.

[0024] Furthermore, the non-maximum suppression is obtained by calculating the IoU value between the predicted bounding box and the ground truth bounding box, as well as the distance between their center points. The calculation formula is as follows:

[0025] ;

[0026] Where M represents a high-confidence prediction box. This indicates the overlap between each bounding box and the bounding box with high confidence. It is calculated by the following formula:

[0027] ;

[0028] in, This represents the furthest diagonal distance between the two boxes. The distance between the center points of the two boxes.

[0029] Furthermore, before acquiring the chicken video stream via the pan-tilt camera during the inspection process, the procedure also includes:

[0030] Obtain images of caged chickens as training images;

[0031] Preprocess the training images to remove the chicken coop fence area.

[0032] Annotate and augment the training images after removing the chicken coop fence area;

[0033] Based on the labeled and data-enhanced training images, the dead chicken detection model is initially trained to obtain the weights of the dead chicken detection model.

[0034] Using the YOLOv7x model as the teacher model and the improved YOLOv7 model as the student model, knowledge distillation was used to retrain the dead chicken detection model to obtain a trained dead chicken detection model.

[0035] Furthermore, the student model is trained using the soft labels output by the teacher model, and the loss function of the student model is updated by combining the soft and hard labels.

[0036] The second objective of this invention can be achieved by adopting the following technical solution:

[0037] A system for detecting dead chickens in cages, the system comprising:

[0038] The acquisition unit is used to acquire video streams of chickens through an image acquisition device during the inspection process;

[0039] The preprocessing unit is used to perform preprocessing operations on a single frame of chicken images to remove the chicken cage fence area from the single frame of chicken images.

[0040] The detection unit is used to input images of chickens with the cage fence area removed into a trained dead chicken detection model for detection, and output chicken category, detection box position information and confidence score. The chicken category includes healthy chickens and dead chickens.

[0041] The tracking unit is used to save the information of the dead chicken if a dead chicken is detected, and to control the image acquisition device to track and photograph the location of the dead chicken within a preset time.

[0042] The upload unit is used to upload the saved information on dead chickens to the management platform for display after the inspection is completed.

[0043] The third objective of this invention can be achieved by adopting the following technical solution:

[0044] An inspection robot includes a robot body, a data acquisition device, and a computing device, wherein the data acquisition device and the computing device are mounted on the robot body and connected to each other.

[0045] The computing device includes a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the above-described method for detecting dead chickens in cages.

[0046] Furthermore, the acquisition device includes an electric push rod, a high-level support, and a camera assembly. The electric push rod is fixed to the robot body. The high-level support has an H-shaped structure and is fixed to the top of the electric push rod. The camera assembly is divided into two columns, with multiple camera assemblies in each column. Each camera assembly includes a camera bracket and a gimbal camera. The camera bracket is fixed to the high-level support, and the gimbal camera is fixed to the camera bracket.

[0047] The fourth objective of this invention can be achieved by adopting the following technical solution:

[0048] A computer-readable storage medium storing a program that, when executed by a processor, implements the above-described method for detecting dead chickens in cages.

[0049] The present invention has the following advantages over the prior art:

[0050] This invention can automatically capture and predict chicken images, effectively identify dead chickens and determine their location. It not only improves the efficiency and accuracy of dead chicken detection, but also significantly reduces labor costs by reducing the need for manual inspections. At the same time, the rapid removal of dead chickens helps prevent the spread of diseases and protects the health of the flock, thereby improving overall breeding efficiency and product quality. It can be reliably and stably applied in actual large-scale farms, and compared with existing technologies, it has higher detection accuracy and faster speed. It has sufficient innovation at the algorithm level, and some algorithms are original. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0052] Figure 1 This is a structural diagram of the inspection robot of Embodiment 1 of the present invention.

[0053] Figure 2 This is a simplified flowchart of the cage-raised dead chicken detection method of Embodiment 1 of the present invention.

[0054] Figure 3 This is a flowchart illustrating the specific process of detecting dead chickens in cages according to Embodiment 1 of the present invention.

[0055] Figure 4 This is a flowchart illustrating the training process of the dead chicken detection model in Embodiment 1 of the present invention.

[0056] Figure 5 This is a network structure diagram of the dead chicken detection model in Embodiment 1 of the present invention.

[0057] Figure 6 This is a schematic diagram of the management platform interface of Embodiment 1 of the present invention.

[0058] Figure 7 This is a structural block diagram of the cage-raised dead chicken detection system of Embodiment 2 of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0060] Example 1:

[0061] With the development of computer vision technology, new solutions have been provided for the problems existing in the current technology. This embodiment adopts a more innovative target detection algorithm, which is combined with a chicken house inspection robot and a gimbal camera on the inspection robot to detect dead chickens in the cage.

[0062] like Figure 1 As shown, this embodiment provides an inspection robot, which is a tracked robot. It includes a robot body 101, a data acquisition device, and a computing device. The data acquisition device and the computing device are mounted on the robot body 101 and are connected to each other.

[0063] The data acquisition equipment includes an electric push rod 102, a high-level support 103, and a camera assembly 104. The electric push rod 102 is fixed to the top plate of the robot body 101 by bolts. The high-level support 103 has an H-shaped structure and is fixed to the top of the electric push rod, which can avoid the light bulb in the middle during the inspection. The camera assembly 104 is divided into two rows, with four camera assemblies 104 in each row, which can acquire images of four layers of chicken coops on each side. Each camera assembly 104 includes a camera bracket 1041 and a gimbal camera 1042. The camera bracket 1041 is fixed to the high-level support 103 by bolts, and the gimbal camera 1042 is fixed to the camera bracket 1041 by shock-absorbing balls.

[0064] The computing device can be connected to a control panel and a monitor. The computing device can be an Nvidia Jetson orin nano edge device. The trained dead chicken detection model can be deployed on the Nvidia Jetson orin nano edge device. TensorRT inference acceleration is performed through the C++ language API interface, and detection is performed through Deepstream video stream. After a dead chicken is detected, the information of the dead chicken with the highest confidence will be saved within 2 seconds. The information of the dead chicken includes the image of the dead chicken, the location of the dead chicken, the detection time, etc. The location of the dead chicken is identified by recognizing the QR code information pasted under each chicken coop. After detecting a dead chicken, the gimbal camera 1042 will also track and capture the location of the dead chicken within 2 seconds through the tracking algorithm.

[0065] The computing device includes a processor and memory for storing processor-executable programs. When the processor executes the program stored in memory, it implements a method for detecting dead chickens in cages, such as... Figure 2 and Figure 3 As shown, the method includes the following steps:

[0066] S301. During the inspection process, video streams of chickens are acquired using a PTZ camera.

[0067] To obtain a trained model, this embodiment, before step S301, such as... Figure 4 As shown, it may also include:

[0068] S401. Obtain images of caged chickens as training images.

[0069] In this embodiment, 5000 images of caged chickens were acquired as training images.

[0070] S402. Perform preprocessing operations on the training images to remove the chicken coop fence area from the training images.

[0071] In this embodiment, the preprocessing operations include morphological operations and Gaussian filtering for noise reduction.

[0072] S403. Label and augment the training images after removing the chicken coop fence area.

[0073] In this embodiment, LabelImg software is used for labeling. Chickens are divided into two categories: "healthy" and "dead". Data augmentation is performed, including rotation, translation, scaling, mirroring, Mosaic, and Retinex data augmentation algorithms.

[0074] S404. Based on the labeled and data-enhanced training images, perform preliminary training on the dead chicken detection model to obtain the dead chicken detection model.

[0075] This embodiment is based on the You Only Look Once version 7 (YOLOv7) object detection algorithm for model training, such as Figure 5 As shown, the backbone network is replaced with the MobileNetv3 lightweight network, a CBAM attention module is added to the feature enhancement network, and Repulsion Loss and DIoU-NMS non-maximum suppression are introduced.

[0076] In this embodiment, the dead chicken detection network incorporates Repulsion Loss into its backpropagation loss function, building upon YOLOv7's approach. This allows the model to mitigate occlusion issues in crowded chicken coop environments. The formula for Repulsion Loss is as follows:

[0077] ;

[0078] in, and The formula is as follows:

[0079] ;

[0080] ;

[0081] in:

[0082] ;

[0083] in, It is a prediction box. It is the true bounding box with the highest IoU score. and The difference is evaluated using IoG loss: , It is a smoothing parameter that adjusts the sensitivity of Repulsion Loss to outliers. and Indicates different groups of prediction boxes. It is an identity function. It is a small constant when divided by zero.

[0084] In this embodiment, the dead chicken detection model predicts objects in an image. For each object, multiple predicted bounding boxes are generated. These boxes are then filtered using a non-maximum suppression (NMS) method. The NMS method calculates the IoU value between the predicted bounding box and the ground truth bounding box, along with the distance to the center point. The calculation formula is as follows:

[0085] ;

[0086] Where M represents a high-confidence prediction box. This indicates the overlap between each bounding box and the bounding box with high confidence. It is calculated by the following formula:

[0087] ;

[0088] in, This represents the furthest diagonal distance between the two boxes. The distance between the center points of the two boxes

[0089] S405. Using the YOLOv7x model as the teacher model and the improved YOLOv7 as the student model, knowledge distillation is used to retrain the dead chicken detection model to obtain a trained dead chicken detection model.

[0090] In this embodiment, the student model is trained using the soft labels output by the teacher model, and the loss function of the student model is updated by combining the soft labels and hard labels. In the detection task, the hard label is "belongs to" or "does not belong to" a certain category, that is, 1 or 0; the soft label uses a probability distribution to express the probability of belonging to a certain category.

[0091] S302. Perform preprocessing on the single-frame chicken image to remove the chicken cage fence area from the single-frame chicken image.

[0092] S303. Input the chicken images with the cage fence area removed into the trained dead chicken detection model for detection, and output the chicken category, detection box location information and confidence score.

[0093] In this embodiment, if a dead chicken is detected, proceed to step S304; otherwise, return to step S301 to reacquire the chicken video stream.

[0094] S304. Save the information of the dead chicken and control the gimbal camera to track and photograph the location of the dead chicken within a preset time (2s).

[0095] S305. After the inspection is completed, the information on the dead chickens will be uploaded to the management platform for display.

[0096] This embodiment uses the Qt language for software visualization. Information about dead chickens is uploaded to a cloud server after the inspection is completed. The management platform then retrieves this information from the cloud server and displays it in the visualization interface. Figure 6 As shown.

[0097] It should be noted that although the above-described method operations are depicted in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the illustrated operations must be performed to achieve the desired result. On the contrary, the order of execution of the depicted steps can be changed. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0098] Example 2:

[0099] like Figure 7 As shown, this embodiment provides a caged dead chicken detection system. The system includes an acquisition unit 701, a preprocessing unit 702, a detection unit 703, a tracking unit 704, and an uploading unit 705. The specific functions of each unit are as follows:

[0100] The acquisition unit 701 is used to acquire video streams of chickens through an image acquisition device during the inspection process;

[0101] Preprocessing unit 702 is used to perform preprocessing operations on a single frame chicken image to remove the chicken cage fence area from the single frame chicken image.

[0102] Detection unit 703 is used to input chicken images with the cage fence area removed into a trained dead chicken detection model for detection, and output chicken category, detection box position information and confidence score. The chicken category includes healthy chickens and dead chickens.

[0103] The tracking unit 704 is used to save the information of the dead chicken if a dead chicken is detected, and to control the image acquisition device to track and capture the position of the dead chicken within a preset time.

[0104] Upload unit 705 is used to upload the saved information on dead chickens to the management platform for display after the inspection is completed.

[0105] The specific implementation of each unit in this embodiment can be found in Embodiment 1 above, and will not be repeated here. It should be noted that the system provided in this embodiment is only illustrated by the division of the above functional units. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure can be divided into different functional units to complete all or part of the functions described above.

[0106] Example 3:

[0107] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the cage-raised dead chicken detection method of Embodiment 1 described above, as follows:

[0108] During the inspection, video streams of chickens are acquired using a gimbal camera. Preprocessing is performed on individual chicken images to remove the cage railing area. The chicken images with the railing removed are then input into a trained dead chicken detection model for detection. The model outputs the chicken category, detection box location information, and confidence score. The chicken category includes healthy chickens and dead chickens. If a dead chicken is detected, its information is saved, and the gimbal camera is controlled to track and capture its location within a preset time. After the inspection is completed, the saved dead chicken information is uploaded to the management platform for display.

[0109] It should be noted that the computer-readable storage medium in this embodiment can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0110] In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this embodiment, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0111] The computer-readable storage medium described above can be used to write computer programs for executing this embodiment in one or more programming languages ​​or combinations thereof. These programming languages ​​include object-oriented programming languages—such as Java, Python, and C++—and conventional procedural programming languages—such as C or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0112] In summary, this invention can automatically capture and predict chicken images, effectively identify dead chickens and determine their location. This not only improves the efficiency and accuracy of dead chicken detection but also significantly reduces labor costs by minimizing the need for manual inspections. Simultaneously, the rapid removal of dead chickens helps prevent the spread of disease, protects the health of the flock, and thus improves overall breeding efficiency and product quality. It can be reliably and stably applied in large-scale farms and, compared to existing technologies, offers higher accuracy and faster speed. Furthermore, it demonstrates sufficient innovation at the algorithm level, with some algorithms possessing originality.

[0113] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A method for detecting dead chickens in cages, characterized in that, The method includes: During the inspection, video streams of chickens were captured using a gimbal camera. Preprocess the single-frame chicken image to remove the chicken cage fence area from the single-frame chicken image; Images of chickens with the cage fence area removed are input into a trained dead chicken detection model for detection. The model outputs chicken category, detection box location information, and confidence score. The chicken category includes healthy chickens and dead chickens. The dead chicken detection model is an improved YOLOv7 model, with the backbone network replaced by a lightweight MobileNetv3 network. A CBAM attention module is added to the feature enhancement network, along with Repulsion Loss and DIoU-NMS non-maximum suppression. If a dead chicken is detected, the information of the dead chicken is saved, and the gimbal camera is controlled to track and photograph the location of the dead chicken within a preset time. After the inspection is completed, the information on the dead chickens is uploaded to the management platform for display. The formula for the Repulsion Loss is as follows: ； in, and The formula is as follows: ； ； in: ； in, It is a prediction box. It is the true bounding box with the highest IoU score. and The difference is evaluated using IoG loss: , It is a smoothing parameter that adjusts the sensitivity of Repulsion Loss to outliers. and Indicates different groups of prediction boxes. It is an identity function. It is a small constant when divided by zero.

2. The method for detecting dead chickens in cages according to claim 1, characterized in that, The non-maximum suppression is obtained by calculating the IoU value between the predicted bounding box and the ground truth bounding box, as well as the distance between their center points. The calculation formula is as follows: ； Where M represents a high-confidence prediction box. This indicates the overlap between each bounding box and the bounding box with high confidence. It is calculated by the following formula: ； in, This represents the furthest diagonal distance between the two boxes. The distance between the center points of the two boxes.

3. The method for detecting dead chickens in cages according to any one of claims 1-2, characterized in that, Before acquiring video streams of chickens via a PTZ camera during the inspection process, the process also includes: Images of caged chickens were obtained and used as training images. Preprocess the training images to remove the chicken coop fence area. Annotate and augment the training images after removing the chicken coop fence area; Based on the labeled and data-enhanced training images, the dead chicken detection model is initially trained to obtain the dead chicken detection model. Using the YOLOv7x model as the teacher model and the improved YOLOv7 model as the student model, knowledge distillation was used to retrain the dead chicken detection model to obtain a trained dead chicken detection model.

4. The method for detecting dead chickens in cages according to claim 3, characterized in that, The student model is trained using soft labels output by the teacher model, and the student model's loss function is updated using gradients that combine soft and hard labels.

5. A system for detecting dead chickens in cages, characterized in that, The system includes: The acquisition unit is used to acquire video streams of chickens through an image acquisition device during the inspection process; The preprocessing unit is used to perform preprocessing operations on a single frame of chicken images to remove the chicken cage fence area from the single frame of chicken images. The detection unit is used to input chicken images with the cage fence area removed into a trained dead chicken detection model for detection, and outputs chicken category, detection box location information and confidence score. The chicken category includes healthy chickens and dead chickens. The dead chicken detection model is an improved YOLOv7 model, with the backbone network replaced by a MobileNetv3 lightweight network. A CBAM attention module is added to the feature enhancement network, and Repulsion Loss and DIoU-NMS non-maximum suppression are also introduced. The tracking unit is used to save the information of the dead chicken if a dead chicken is detected, and to control the image acquisition device to track and photograph the location of the dead chicken within a preset time. The upload unit is used to upload the saved information on dead chickens to the management platform for display after the inspection is completed; The formula for the Repulsion Loss is as follows: ； in, and The formula is as follows: ； ； in: ； in, It is a prediction box. It is the true bounding box with the highest IoU score. and The difference is evaluated using IoG loss: , It is a smoothing parameter that adjusts the sensitivity of Repulsion Loss to outliers. and Indicates different groups of prediction boxes. It is an identity function. It is a small constant when divided by zero.

6. An inspection robot, characterized in that, It includes a robot body, a data acquisition device, and a computing device, wherein the data acquisition device and the computing device are mounted on the robot body and connected to each other. The computing device includes a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the cage-raised dead chicken detection method according to any one of claims 1-4.

7. The inspection robot according to claim 6, characterized in that, The data acquisition device includes an electric push rod, a high-level support, and a camera assembly. The electric push rod is fixed to the robot body. The high-level support has an H-shaped structure and is fixed to the top of the electric push rod. The camera assembly is divided into two columns, with multiple camera assemblies in each column. Each camera assembly includes a camera bracket and a gimbal camera. The camera bracket is fixed to the high-level support, and the gimbal camera is fixed to the camera bracket.

8. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the cage-raised dead chicken detection method according to any one of claims 1-4.

Images