A method and system for counting pigs in a pen

Abstract

The application discloses a kind of pig count method and system, belong to pig breeding technical field, including the following steps: S1: obtaining image;S2: image clipping;S3: algorithm processing;S4: information integration.The application can conveniently count the data volume of the pig in the pig crowded condition by using the rotating target detection network in combination with the semantic segmentation network, while reducing the computing cost, improving the counting efficiency, improving the user experience, solving the problem that the traditional target detection algorithm has low counting accuracy in the pig crowded condition, and the instance segmentation algorithm can better handle the pig crowded condition, but the computing cost is high, time is long, and the data annotation cost is extremely high, and is worth popularization and use.

Description

Technical Field

[0001] This invention relates to the field of pig farming technology, specifically to a method and system for counting pigs in stock. Background Technology

[0002] Pig farming is an important sector of my country's agriculture. It plays a vital role in ensuring the safe supply of meat. my country's pig farming industry is transforming from traditional to modern, with significant changes occurring in breeding models, regional distribution, production methods, and production capacity.

[0003] In the process of raising pigs, it is often necessary to count them. The existing counting methods mainly include manual counting, counting by ear tags, and counting by taking pictures with a fixed camera. Manual counting has the following problems, such as: pigs move around, making it easy to miss or count too many; low inventory efficiency. Counting by ear tags also has certain shortcomings, such as: radio frequency ear tags are required, which are expensive and easy to fall off. Because they can only be sensed at a very close distance (within half a meter), the inventory equipment needs to be very close to each pig, making large-pen inventory extremely troublesome. It is also difficult to solve the problem of low inventory efficiency caused by pig movement.

[0004] The main problems with traditional methods of counting pigs by taking photos with fixed cameras are the high cost and difficulty of installation; a single farmer typically needs at least 20-40 cameras. When images of pigs are acquired, traditional object detection or instance segmentation algorithms are used for processing. Traditional object detection algorithms have low accuracy when pigs are crowded, while instance segmentation algorithms, although handling crowded conditions better, are computationally expensive, time-consuming, and have extremely high data annotation costs. These problems urgently need to be addressed. Therefore, a new method for counting pigs in stock is proposed. Summary of the Invention

[0005] The technical problem to be solved by this invention is: how to solve the problem that traditional target detection algorithms have low counting accuracy when pigs are crowded, and that although instance segmentation algorithms can handle pigs crowded well, they have high computational cost, long time, and extremely high data labeling cost. This invention provides a method for counting pigs in stock, which can identify the number of pigs in each individual pigpen.

[0006] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0007] S1: Acquire image

[0008] Connect the camera to the terminal, acquire the video stream through the terminal, capture the data captured by the camera and convert it into an RGB image, which is an image of the pigs in the pen. The image contains the pigs and the pen.

[0009] S2: Image cropping

[0010] The RGB image is cropped to the set size and uploaded to the OSS server, while the OSS address of the image is sent to the cloud server.

[0011] S3: Algorithm Processing

[0012] The cloud server simultaneously calls the rotating target detection algorithm and the image semantic segmentation algorithm services. The rotating target detection algorithm performs rotating target detection processing on the image in step S2 through the rotating target detection network to obtain the number and location information of pigs; at the same time, the image semantic segmentation algorithm performs semantic segmentation processing on the image in step S2 through the semantic segmentation network to obtain the location information of the pen boundary.

[0013] S4: Information Integration

[0014] The number and location information of pigs obtained in step S3 are integrated with the location information of the pen boundary to calculate the number of pigs in the pen area segmented from the captured image, and the result is sent back to the terminal and displayed to the user.

[0015] Furthermore, in step S1, the camera is an ultra-wide-angle external camera, the terminal is a mobile phone, and the connection between the ultra-wide-angle external camera and the mobile phone is either wireless or wired.

[0016] Furthermore, in step S1, camera lens data is captured and converted into an RGB image using the UVC protocol.

[0017] Furthermore, in step S3, before performing rotation target detection and semantic segmentation, the cropped image is preprocessed to obtain the image data required by the rotation target detection network and the semantic segmentation network, respectively. The preprocessing methods include four techniques: resize, letterboxing, adaptive white balance, and histogram equalization.

[0018] Furthermore, in step S3, the rotated target detection network includes a first backbone module, a Neck module, and a head module. The backbone module is a Swin Transformer feature extraction framework used to extract features from the image. The Neck module is PANet used to aggregate features from different layers, maintaining feature integrity and diversity. The head module is a head module with added rotation angle information in the YOLOv5 model, used to make the target detection box fit the outline of the pig.

[0019] Furthermore, in the rotating target detection network, after the image is processed by the first backbone module, the feature information of the input image is obtained; after processing by the Neck module, the features obtained by the first backbone module are extracted and fused, thereby improving the network performance; after processing by the head module, the type of the detected target and its position in the image are finally determined based on the features.

[0020] Furthermore, after processing by the rotating target detection network, the output results are subjected to soft-nms processing.

[0021] Furthermore, in step S3, the semantic segmentation network includes a second backbone module, a semantic segmentation feature head module, and a semantic segmentation detection head module. The second backbone module is the Swin Transformer feature extraction framework, used to extract features from the image. The semantic segmentation feature head module is Uperhead, used for segmenting the features of the pigpen area. The semantic segmentation detection head module is FCNHead, used for pixel-by-pixel reconstruction of the deep features extracted by the second backbone module to obtain the complete pigpen area.

[0022] Furthermore, in the semantic segmentation network, after the image is processed by the second backbone module, a feature map of the input image is obtained; after processing by the semantic segmentation feature head module, the feature information is refined and combined; after processing by the semantic segmentation detection head module, the segmentation region location information of the predicted target is obtained using the features extracted by the semantic segmentation feature head module.

[0023] Furthermore, after processing by the semantic segmentation network, the envelope algorithm is used on the segmented pigpen area to obtain a polygonal region with neat edges and its location information, namely the pen boundary and its location information.

[0024] This invention also discloses a pig counting system, which uses the above-mentioned counting method to count the number of pigs in stock, including:

[0025] The image acquisition module is used to connect to the terminal using a camera, acquire video streams through the terminal, and capture data captured by the camera and convert it into RGB images.

[0026] The image cropping module is used to crop RGB images to a set size and upload them to the OSS server, while also sending the image's OSS address to the cloud server.

[0027] The algorithm processing module is used to simultaneously call the rotating target detection algorithm and the image semantic segmentation algorithm service through the cloud server. The rotating target detection algorithm performs rotating target detection processing on the image in step S2 through the rotating target detection network to obtain the number and location information of pigs; at the same time, the image semantic segmentation algorithm performs semantic segmentation processing on the image in step S2 through the semantic segmentation network to obtain the location information of the pen boundary.

[0028] The information integration module is used to integrate the number and location information of pigs obtained in step S3 with the location information of the pen boundary, calculate the number of pigs in the pen area segmented from the captured image, and transmit the result back to the terminal to be displayed to the user.

[0029] The control processing module is used to issue instructions to the other modules to complete the corresponding actions.

[0030] The image acquisition module, image cropping module, algorithm processing module, and information integration module are all communicatively connected to the control processing module.

[0031] Compared with the prior art, the present invention has the following advantages: by combining the rotating target detection network with the speech segmentation network, it is possible to conveniently count the amount of data of pigs in stock when pigs are crowded, while reducing the computing cost, improving the counting efficiency, and improving the user experience. It is worth promoting and using. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the pig counting method in an embodiment of the present invention;

[0033] Figure 2 a is a schematic diagram of the Backbone portion in the rotating target detection network in this embodiment of the invention;

[0034] Figure 2 b is a schematic diagram of the Neck portion in the rotating target detection network in this embodiment of the invention;

[0035] Figure 3 a is a schematic diagram of the Upperhead part of the semantic segmentation network in an embodiment of the present invention;

[0036] Figure 3 b is a schematic diagram of the structure of the FCNHead part in the semantic segmentation network in this embodiment of the invention. Detailed Implementation

[0037] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0038] like Figure 1 As shown, this embodiment provides a technical solution: a method for counting pigs in stock, including the following steps:

[0039] Step 1: Connect the ultra-wide-angle external camera to the mobile phone. There are two connection methods: wired and wireless. The mobile phone will display the video stream to the user. The user can click the shutter button to call the image capture interface and obtain the RGB image.

[0040] Step 2: The original RGB image is too large to be suitable for network transmission. It is cropped to 1024*1024 resolution and uploaded to the OSS server, i.e., distributed object storage cloud server. At the same time, the OSS address of the image is passed to the local cloud server to start the algorithm call.

[0041] Step 3: Preprocess the cropped image. Preprocessing methods include resizing, letterboxing, adaptive white balance, and histogram equalization. Resizing and letterboxing are used together to adjust the image size while maintaining the aspect ratio, and fill the surrounding area with black pixels as needed to maintain the feature map size and not discard the original image information. Adaptive white balance and histogram equalization can make the image clearer. The former can make white objects appear white under any light source, aiming to restore the true color of the object as much as possible, while the latter enhances the image contrast.

[0042] Step 4: The cloud server simultaneously starts two algorithm services: A. Rotational target detection, B. Image semantic segmentation;

[0043] Step 5: Use the Swing Transformer feature extraction framework to extract features from the image;

[0044] It should be noted that Swin Transformer is a general-purpose feature extraction framework (backbone) and currently the most powerful feature extractor in the ViT (Vision Transformer) field for detection. Although detecting an image uses the same backbone, different tasks can be achieved by adjusting different heads and necks. This invention, through this multi-headed snake structure, can significantly save computational power without affecting functionality.

[0045] The Swin Transformer architecture is as follows: Figure 2As shown in Figure a, the input RGB image is segmented into non-overlapping patches using a patch segmentation module. In stage 1, a line ar embedding is first applied, and then the image is fed into the Swing Transformer Block. Stages 2-4 involve patch merging, which downsamples the input by merging adjacent patches in a 2x2 pattern before feeding them into the Swing Transformer Block.

[0046] Step 6A: Select PANet as the Neck part of the rotating target detection network;

[0047] PANet is a framework based on Mask R-CNN and FPN that enhances information propagation. The feature extractor of this network adopts a novel enhanced bottom-up path FPN structure, which improves the propagation of low-level features, enhances the processing and analysis of target location information, and improves the recognition accuracy of crowded areas in pigs.

[0048] Step 6B: Select Upperhead as the semantic segmentation feature head for the semantic segmentation network;

[0049] Uperhead is a novel semantic segmentation feature head. Its advantage lies in its ability to collect various different information such as scene, region, object, and material from the same image. It has a good effect on segmentation of pig pens, where regional features are not obvious and materials are repetitive and similar.

[0050] Step 7A: Add rotation information to the head part of the rotating target detection network;

[0051] Based on the original YOLOv5 head part, a rotation angle dimension is added. That is, in addition to the four coordinates (x, y) of the target rectangle, an angle θ is added to make the target detection box fit the outline of the pig better and reduce the overlap of the target area.

[0052] Step 7B: Select FCNHead as the basic detection head for semantic segmentation in the semantic segmentation network;

[0053] The deep features extracted from the backbone are restored pixel by pixel using FCNHead, and finally the complete pigpen area is obtained.

[0054] Step 8A: Post-processing of target detection: The output result (i.e., the image including all detection boxes identified by the model) is processed with soft-NMS. Soft-NMS is an upgrade of basic NMS, which can better handle crowded targets. Soft-NMS is soft non-maximum suppression. The image obtains multiple detection boxes of the same target according to the model classifier. The detection boxes are sorted by score, and the box with the highest score is retained. At the same time, the confidence of boxes whose overlap area with the target is greater than a certain proportion of other boxes is reduced, and redundant boxes are removed according to the confidence threshold. At the same time, because a rotation angle is added, soft-NMS adds support for rotated rectangles. Finally, the image result containing the detection box information of all pigs in the entire image is obtained.

[0055] Step 8B: Image segmentation post-processing: The image segmentation region is processed using the envelope algorithm. Because the boundary shape of the segmented pen region is irregular, the segmented pen region needs to be envelope analyzed to find the smallest area convex polygon of the largest connected component of the region, and then obtain the polygon region with neat edges, i.e., the pen region.

[0056] Step 9A: Upload the center point, length, width, and rotation angle information of all pigs to the cloud server;

[0057] Step 9B: Upload the location information of the endpoints of the pigpen area envelope polygon obtained in Step 8B.

[0058] Step 10: Information integration, i.e., obtaining the number of pigs in the pigpen area; based on the position coordinates of the endpoints of the envelope polygon uploaded in 9B, the location of the pigpen area is determined by the result image obtained in step 8A, and the number of pigs in the area is transmitted back to the mobile phone and displayed to the user.

[0059] like Figure 2 Figure a shows a schematic diagram of the Backbone part (i.e., the Swin Transformer framework) in the rotating target detection network. Figure 2 Figure b shows a schematic diagram of the Neck part in the rotating target detection network.

[0060] like Figure 3 Figure a shows a schematic diagram of the Upperhead part in a semantic segmentation network; as shown in Figure a. Figure 3 Figure b shows a schematic diagram of the FCNHead part in a semantic segmentation network.

[0061] In summary, the pig counting method described above, by combining a rotating target detection network with a speech segmentation network, can conveniently count the amount of data on pigs in crowded conditions, while reducing computational costs, improving counting efficiency, and enhancing user experience. It is worthy of widespread adoption.

[0062] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for counting pigs in stock, characterized in that, Includes the following steps: S1: Acquire image Connect the camera to the terminal, acquire the video stream through the terminal, capture the data captured by the camera and convert it into an RGB image, which is an image of the pigs in the pen. The image contains the pigs and the pen. S2: Image cropping The RGB image is cropped to the set size and uploaded to the OSS server, while the OSS address of the image is sent to the cloud server. S3: Algorithm Processing The cloud server simultaneously calls the rotating target detection algorithm and the image semantic segmentation algorithm services. The rotating target detection algorithm performs rotating target detection processing on the image in step S2 through the rotating target detection network to obtain the number and location information of pigs; at the same time, the image semantic segmentation algorithm performs semantic segmentation processing on the image in step S2 through the semantic segmentation network to obtain the location information of the pen boundary. S4: Information Integration The number and location information of pigs obtained in step S3 are integrated with the location information of the pen boundary to calculate the number of pigs in the pen area segmented from the captured image, and the result is sent back to the terminal and displayed to the user. In step S3, the rotating target detection network includes a first backbone module, a Neck module, and a head module. The backbone module is a Swin Transformer feature extraction framework used to extract features from the image. The Neck module is PANet used to aggregate features from different layers. The head module is a head module with added rotation angle information in the YOLOv5 model used to make the target detection box fit the outline of the pig. In the rotating target detection network, the image is processed by the first backbone module to obtain the feature information of the input image; After processing by the Neck module, the features obtained by the first backbone module are extracted and fused. After processing by the head module, the type of the detected target and its location in the image are finally determined based on the features. After processing by the rotating target detection network, the output result is subjected to soft-nms processing; In step S3, the semantic segmentation network includes a second backbone module, a semantic segmentation feature head module, and a semantic segmentation detection head module. The second backbone module is the Swin Transformer feature extraction framework, which is used to extract features from the image. The semantic segmentation feature head module is Uperhead, which is used to segment the features of the pigpen area. The semantic segmentation detection head module is FCNHead, which is used to restore the deep features extracted by the second backbone module pixel by pixel to obtain the complete pigpen area. In the semantic segmentation network, after the image is processed by the second backbone module, a feature map of the input image is obtained; after processing by the semantic segmentation feature head module, the obtained feature information is refined and combined; after processing by the semantic segmentation detection head module, the segmentation region location information of the predicted target is obtained using the features extracted by the semantic segmentation feature head module. After processing by the semantic segmentation network, the envelope algorithm is used on the segmented pigpen area to obtain a polygonal region with neat edges and its location information, that is, the pen boundary and its location information.

2. The method for counting pigs in stock according to claim 1, characterized in that: In step S1, the camera is an ultra-wide-angle external camera, the terminal is a mobile phone, and the connection between the ultra-wide-angle external camera and the mobile phone is wireless or wired. The camera lens data is captured and converted into an RGB image through the UVC protocol.

3. The method for counting pigs in stock according to claim 1, characterized in that: In step S3, before performing rotation target detection and semantic segmentation, the cropped image is preprocessed to obtain the image data required by the rotation target detection network and the semantic segmentation network, respectively. The preprocessing methods include four techniques: resize, letterboxing, adaptive white balance, and histogram equalization.

4. A pig counting system, characterized in that, The counting of pigs in stock is performed using the counting method described in any one of claims 1-3, including: The image acquisition module is used to connect to the terminal using a camera, acquire video streams through the terminal, and capture data captured by the camera and convert it into RGB images. The image cropping module is used to crop RGB images to a set size and upload them to the OSS server, while also sending the image's OSS address to the cloud server. The algorithm processing module is used to simultaneously call the rotating target detection algorithm and the image semantic segmentation algorithm service through the cloud server. The rotating target detection algorithm performs rotating target detection processing on the image in step S2 through the rotating target detection network to obtain the number and location information of pigs; at the same time, the image semantic segmentation algorithm performs semantic segmentation processing on the image in step S2 through the semantic segmentation network to obtain the location information of the pen boundary. The information integration module is used to integrate the number and location information of pigs obtained in step S3 with the location information of the pen boundary, calculate the number of pigs in the pen area segmented from the captured image, and transmit the result back to the terminal to be displayed to the user. The control processing module is used to issue instructions to the other modules to complete the corresponding actions. The image acquisition module, image cropping module, algorithm processing module, and information integration module are all communicatively connected to the control processing module.

Images