A pig weight estimation method based on deep learning multimodal technology on a large pen track robot

Abstract

The present invention relates to the technical field of pig weight estimation, and specifically, to a method for pig weight estimation on a large pen track robot based on deep learning multimodal technology. In the present invention, by setting up a track-based robot inspection to perform weight estimation, not only the stress response of the pigs is avoided, but also the efficiency is greatly improved. At the same time, the present invention can obtain the characteristic information of the pigs more comprehensively by integrating information of multiple modes such as RGB, depth map, camera installation height and key points of the pigs, thereby improving the accuracy of pig weight estimation, and using a pig weight estimation algorithm based on deep learning multimodal fusion, the pig weight is output end-to-end, the actual application process is simple, the calculation complexity is greatly reduced, the real-time performance is improved, and the algorithm accuracy is high. Therefore, the present invention is very suitable for daily pig inspection and weight estimation in large pen scenes of pig farms, and has strong applicability in actual operation.

Description

Technical Field

[0001] The present invention relates to the technical field of pig weight estimation, and in particular to a pig weight estimation method on a large pen track robot based on deep learning multimodal technology. Background Art

[0002] After searching, it was found that a Chinese patent with publication number CN118366184A disclosed a method and device for estimating pig weight. The multimodal instance segmentation model was used to extract the pig's body size pixel information, and the pig image was a priori calibrated to obtain a priori calibration template. The pig's body size pixel information and the prior calibration template were input into a support vector regression model for training, so as to obtain the pig weight estimation result based on the trained model.

[0003] However, this patent is based on a specific pigpen design. A camera is installed at a specific position in the pigpen. After a pig is driven in, pictures of different modes are collected from different angles to estimate the weight. Obviously, this method can only estimate the weight of one pig. It is more complicated to apply and cannot be used in large pen scenes in pig houses.

[0004] Therefore, the patented technology also has the following defects:

[0005] 1) It is necessary to design a specific pig pen with different types of cameras installed at specific locations in the pig pen. The weight of the pigs can only be estimated by driving them into the pig pen. Driving the pigs may cause stress reactions in the pigs, which is not conducive to their growth;

[0006] 2) Calibration is required before weight estimation, and a priori calibration template is obtained, which is a complicated process;

[0007] 3) For visible light and infrared images, they are mainly used for segmentation algorithms. Based on the segmentation, the height and width of the pigs are obtained, and this information is used to calculate the weight of the pigs. Obviously, the multimodal information is not used sufficiently;

[0008] 4) Using the support vector product algorithm, the accuracy and robustness of the algorithm are greatly affected by external factors such as the environment;

[0009] 5) In actual pig pens, the postures of pigs vary greatly at different times. Therefore, this weight estimation based on standing pigs in a specific pigpen cannot be applied in large pen scenarios. Summary of the invention

[0010] The purpose of the present invention is to provide a pig weight estimation method based on deep learning multimodal technology on a large pen track robot, so as to solve the problems that the traditional method proposed in the above background technology can only estimate the weight of one pig, is relatively complicated to apply, and cannot be used in the large pen scene of the pig house. At the same time, it is not conducive to the growth of pigs, the process is complicated, the use of multimodal information is not sufficient, and the accuracy and robustness of the algorithm are greatly affected by external factors such as the environment.

[0011] To achieve the above object, the present invention provides a method for estimating the weight of pigs on a large pen track robot based on deep learning multimodal technology, comprising the following steps:

[0012] S1. Record altitude;

[0013] S2, inspection and flow pulling;

[0014] S3, target segmentation;

[0015] S4, posture estimation and key point acquisition;

[0016] S5. Design a weight estimation algorithm;

[0017] S6. Circular flow estimation.

[0018] As a further improvement of the technical solution, the specific operation method for recording the height in step S1 is: when deploying the track robot, record the height h_camera information from the robot lens to the ground.

[0019] As a further improvement of the present technical solution, the specific operation method of the inspection and streaming in step S2 is: the inspection robot starts the inspection and triggers the depth camera to pull the stream to obtain a frame of RGB and depth images.

[0020] As a further improvement of the technical solution, the specific operation method of the target segmentation in step S3 is: based on yolov10 segmentation of the pigs in the image, after segmentation, the segmentation boundary information of each pig in the image in the RGB image and the depth image can be obtained.

[0021] As a further improvement of the technical solution, the specific operation method of posture estimation and key point acquisition in step S4 is: based on yolov10, the posture information and key point information of each pig in the image are acquired, where the posture includes standing, lying, and lying on the side. There are three key points, which are respectively at point A of the pig's head, point B of the center of the pig's back, and point C of the pig's tail.

[0022] As a further improvement of the technical solution, the specific operation method of designing the weight estimation algorithm in step S5 is:

[0023] S51, introduction to multimodal information, the information that needs to be fused currently includes RGB, depth, h_camera and key point curvature information. The curvature information is the angle between BA and BC.

[0024] S52, network design of fusion algorithm. The network input is a 4-channel input of RGB+depth combination. The backbone network uses mobilenetV3 network for feature extraction. The regression head fuses h_camera and key point curvature c in turn, and finally inputs the pig weight weight.

[0025] S53. Independent model design based on different postures of branches. Considering that the same pig has great differences in different postures, different models need to be trained according to different postures, which can further improve the accuracy and robustness of the model.

[0026] S54, performing model training;

[0027] S55. Use the corresponding model to perform inference weight estimation according to different postures.

[0028] As a further improvement of the present technical solution, the specific operation method for model training in step S54 is: collecting pig data with weight labels in different postures, classifying pigs in different pictures through a posture estimation model, and training corresponding models for pigs in different postures respectively.

[0029] As a further improvement of the technical solution, the specific operation method of the cyclic flow estimation in step S6 is: during the robot inspection, the flow estimation is repeated until the inspection is completed.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. Non-contact weight estimation by inspection robots: The weight estimation method based on the inspection of track robots not only avoids the stress response of pigs, but also greatly improves the efficiency;

[0032] 2. Multimodal fusion technology based on deep learning: This invention can obtain more comprehensive characteristic information of pigs by fusing information of multiple modalities such as RGB, depth map, camera installation height and key points of pigs, thereby improving the accuracy of pig weight estimation. This innovation solves the problem that the existing technology mainly relies on single modal information, can make full use of multimodal information, and improve the upper limit of weight estimation accuracy;

[0033] 3. The pig weight estimation algorithm based on deep learning multimodal fusion outputs pig weight end-to-end. The practical application process is simple, which greatly reduces the calculation complexity, improves the real-time performance, and has high algorithm accuracy.

[0034] 4. The multi-model pig weight estimation method based on different postures not only simplifies the complexity of model design, but also ensures the accuracy of the algorithm;

[0035] 5. This method is very suitable for daily inspection and weight estimation of pigs in large pens of pig farms, and is highly applicable in actual operations. This innovation expands the application scope of the technology, making this technical solution more flexible and practical in actual applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the pig weight estimation method on a large pen track robot based on deep learning multimodal technology of the present invention.

[0037] Figure 2 This is a network design diagram of the fusion algorithm of the present invention.

[0038] Figure 3 It is a flow chart of the weight estimation algorithm of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In a specific embodiment, Figure 1 As shown, the present invention provides a pig weight estimation method on a large pen track robot based on deep learning multimodal technology, which mainly adopts the following technical means:

[0041] Step 1. Record the height: When deploying the track robot, record the height h_camera information from the robot lens to the ground;

[0042] Step 2: Inspection and streaming: The inspection robot starts inspection and triggers the depth camera to stream and obtain a frame of RGB and depth images.

[0043] The present invention proposes a method of using a patrol robot to estimate the weight of pigs without contact, which not only avoids the stress response of pigs but also greatly improves efficiency. In the field of animal breeding technology, this technical solution not only improves breeding efficiency and reduces labor costs by quickly and accurately estimating the weight of pigs, but also provides important data support for pig health management and disease prevention.

[0044] Step 3: Target segmentation: Segment the pigs in the image based on yolov10. After segmentation, the segmentation boundary information of each pig in the image in the RGB image and the depth image can be obtained;

[0045] Step 4, posture estimation and key point acquisition: Based on yolov10, the posture information and key point information of each pig in the image are obtained, where the posture includes standing, lying, and lying on the side. Three key points are designed, namely, point A of the pig's head, point B of the center of the pig's back, and point C of the pig's tail;

[0046] The present invention also proposes using RGB images, depth images, camera installation height, pig posture information, and pig key point information as input information, fully integrating various information related to pig weight.

[0047] Step 5: Design of weight estimation algorithm:

[0048] 1) Introduction to multimodal information: The information that needs to be fused currently includes RGB, depth, h_camera, and key point curvature information. The curvature information is the angle between BA and BC.

[0049] 2) Network design of fusion algorithm. The network input is a 4-channel input of RGB+depth combination. The backbone network uses mobilenetV3 network for feature extraction. The regression head fuses h_camera and key point curvature c in turn, and finally inputs the pig weight. The network design is detailed in the attached Figure 2 ;

[0050] 3) Independent model design based on different postures of branches. Considering that the same pig has great differences in different postures, different models need to be trained according to different postures, which can further improve the accuracy and robustness of the model;

[0051] 4) Training model: Collect pig data with weight labels in different postures, classify pigs in different pictures through posture estimation model, and train corresponding models for pigs in different postures;

[0052] 5) Use the corresponding model to perform inference weight estimation according to different postures: The overall process of weight estimation algorithm design is detailed in the attached Figure 3 .

[0053] The present invention proposes a pig weight estimation algorithm based on deep learning multimodal fusion, designs a neural network structure for multimodal weight estimation, and outputs pig weights end-to-end. The practical application process is simple and the algorithm has high accuracy. At the same time, a weight estimation algorithm design based on pig posture is proposed, which greatly reduces the complexity of network design and improves the robustness of the algorithm.

[0054] Step 6: Circular flow estimation: During the robot inspection, repeat the flow estimation until the inspection is completed.

[0055] Compared with the prior art, this technical solution mainly solves the following technical problems:

[0056] 1) The method of using inspection robots to estimate the weight of pigs without contact is proposed, which not only avoids the stress response of pigs but also greatly improves the efficiency;

[0057] 2) A method based on RGB images, depth images, camera installation height, pig posture information, and pig key point information is proposed as input information, which fully integrates various information related to pig weight;

[0058] 3) A pig weight estimation algorithm based on deep learning multimodal fusion is proposed, which outputs pig weight end-to-end. The practical application process is simple and the algorithm has high accuracy.

[0059] 4) This solution is very suitable for daily inspection and weight estimation of pigs in large pens on pig farms, and is highly applicable in actual operations.

[0060] In general, this technical solution has broad application prospects and market demands in many fields such as deep learning, computer vision and animal husbandry, and has important practical and commercial value.

[0061] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A pig weight estimation method based on deep learning multimodal technology on a large pen track robot, characterized in that: The following steps are involved: S1. Record altitude; S2, inspection and flow pulling; S3, target segmentation; S4, posture estimation and key point acquisition; S5. Design a weight estimation algorithm; S6. Circular flow estimation.

2. The method for estimating pig weights on a large pen track robot based on deep learning multimodal technology according to claim 1 is characterized in that: The specific operation method of recording the height in step S1 is: when deploying the track robot, record the height h_camera information from the robot lens to the ground.

3. The pig weight estimation method based on deep learning multimodal technology on a large pen track robot according to claim 1 is characterized in that: The specific operation method of the inspection and streaming in step S2 is: the inspection robot starts the inspection and triggers the depth camera to pull the stream to obtain a frame of RGB and depth images.

4. The pig weight estimation method based on deep learning multimodal technology on a large pen track robot according to claim 1 is characterized in that: The specific operation method of the target segmentation in step S3 is: segment the pigs in the image based on yolov10, and after segmentation, the segmentation boundary information of each pig in the image in the RGB image and the depth image can be obtained.

5. The method for estimating pig weight on a large pen track robot based on deep learning multimodal technology according to claim 1 is characterized in that: The specific operation method of posture estimation and key point acquisition in step S4 is: based on Yolov10, the posture information and key point information of each pig in the image are acquired, wherein the posture includes standing, lying, and lying on the side. Three key points are designed, namely, point A of the pig's head, point B of the center of the pig's back, and point C of the pig's tail.

6. The method for estimating pig weights on a large pen track robot based on deep learning multimodal technology according to claim 1, characterized in that: The specific operation method of designing the weight estimation algorithm in step S5 is: S51, introduction to multimodal information, the information that needs to be fused currently includes RGB, depth, h_camera and key point curvature information. The curvature information is the angle between BA and BC. S52, network design of fusion algorithm. The network input is a 4-channel input of RGB+depth combination. The backbone network uses mobilenetV3 network for feature extraction. The regression head fuses h_camera and key point curvature c in turn, and finally inputs the pig weight weight. S53. Independent model design based on different postures of branches. Considering that the same pig has great differences in different postures, different models need to be trained according to different postures, which can further improve the accuracy and robustness of the model. S54, performing model training; S55. Use the corresponding model to perform inference and weight estimation according to different postures.

7. The method for estimating pig weights on a large pen track robot based on deep learning multimodal technology according to claim 1, characterized in that: The specific operation method for model training in step S54 is: collecting pig data with weight labels in different postures, classifying pigs in different pictures through a posture estimation model, and training corresponding models for pigs in different postures respectively.

8. The method for estimating pig weights on a large pen track robot based on deep learning multimodal technology according to claim 1, characterized in that: The specific operation method of the cyclic flow estimation in step S6 is: during the robot inspection, the flow estimation is repeated until the inspection is completed.

Images