Pig health assessment method based on rail type inspection robot

Abstract

The invention discloses a pig health assessment method based on a rail-mounted inspection robot, and the method comprises the following steps: S1, collecting an RGB image, an infrared image and a depth image of a pig through the rail-mounted inspection robot, and recording the health condition of the pig; s2, extracting physiological and behavior characteristics of the pig through a deep learning model according to the RGB image, the infrared image and the depth image; s3, obtaining health indexes of the pigs according to the physiological and behavior characteristics of the pigs; s4, standardizing the health indexes of the pigs through z-score to obtain scores of the health indexes of the pigs; and S5, constructing a random forest model according to the health condition of the pig, taking the score of the health index of the pig as an independent variable, training through the random forest model to obtain the weight of the health index, establishing a pig health scoring model according to the weight of the health index, and calculating the health score of the pig, thereby realizing automatic pig health assessment.

Description

Technical Field

[0001] The invention belongs to the field of pig health assessment, and in particular relates to a pig health assessment method based on a track-type inspection robot. Background Art

[0002] In order to ensure the normal supply of pork, improve the risk resistance of pig farming and reduce the cost of breeding, pig farming has gradually developed in the direction of intensive, large-scale and automated. Data show that individual farmers are gradually decreasing. At present, my country's intensive pig farming model has developed rapidly, but due to its late start, low degree of intelligence, weak production foundation, and insufficient protection against diseases. At the same time, the small space and large individual base of the intensive farming model make pigs more susceptible to cross-infection, leading to large-scale outbreaks of diseases, causing heavy losses to farms. Therefore, in the process of pig farming under the intensive farming model, timely and effective evaluation and feedback of the health status of pigs and health warnings can help farms make decisions quickly to reduce losses. The health status of pigs is judged based on their various health indicators. For pig farming, by obtaining the various physiological and behavioral characteristics of pigs and then extracting the health indicators contained therein, their health status can be more comprehensively evaluated, and the feeding plan can be adjusted in a targeted manner, which is of great significance to the development of precision animal husbandry and animal welfare assessment.

[0003] Pig health monitoring is of great significance to pig farming management. At present, actual production still relies on manual inspections to monitor the health status of pigs. Pig farm staff make judgments by observing pig behavior and collecting some physiological indicators, but the manual inspection method is not only time-consuming, costly, and inefficient, but also relies on the personal experience of staff. The lack of unified standards makes it impossible to achieve industrial promotion. In addition, when pigs show obvious abnormalities, they are usually in the late stage of the disease, and pigs in sub-health or early stages of the disease are difficult to detect in time.

[0004] In recent years, researchers have explored the use of various wearable sensors and cameras to collect pig data to extract abnormal physiological and behavioral characteristics of pig health, and have made some progress. For example, Cang Yan et al. used the ReSpeaker Core v2.0 development board and MobileNetV2 network technology to classify pig sounds, with a recognition rate of 97.3%; Li Juxia et al. used the Yolov4 network to recognize the drinking behavior of pigs in images from different perspectives; Xue Yueju et al. proposed a deep video recognition algorithm based on the hidden Markov model to address the difficulty in identifying high-risk actions of lactating sows; Zhou Liping et al. proposed an improved Ostu algorithm based on thermal infrared images to detect the ear root area of ​​live pigs; Zhao Haitao used the human face key point detection technology to automatically identify key parts of infrared images of pig faces. However, the above research only obtains a certain physiological or behavioral characteristic of pigs, and does not consider the health status of pigs from the perspective of their overall condition. At the same time, the problem of fixed equipment being easily damaged and falling means that it has no obvious cost advantage over manual labor, and is difficult to promote and apply. However, the inspection robot equipped with a variety of detection equipment can inspect all pigs in the pig house and extract their characteristic information for analysis, which can effectively solve the problems of cost and single evaluation perspective. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a pig health assessment method based on a track-type inspection robot, which solves the problem that the pig health assessment in the prior art relies on manual inspection for judgment.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a pig health assessment method based on a track-type inspection robot, comprising the following steps:

[0007] S1. Collect RGB images, infrared images and depth images of pigs through the track inspection robot, and record the health status of pigs;

[0008] S2. Extract the physiological and behavioral characteristics of pigs based on RGB images, infrared images and depth images through a deep learning model, including pig weight characteristics, pig body temperature characteristics, pig body surface characteristics and pig behavior characteristics;

[0009] S3. Obtaining the health indicators of the pigs according to the physiological and behavioral characteristics of the pigs, including the health indicators of the pigs' weight, the health indicators of the pigs' body temperature, the health indicators of the pigs' body surface and the health indicators of the pigs' behavior;

[0010] S4. Standardizing the health indicators of the pigs by z-score to obtain the scores of the health indicators of the pigs, including the scores of the health indicators of the pigs' weight, the health indicators of the pigs' body temperature, the health indicators of the pigs' body surface, and the health indicators of the pigs' behavior;

[0011] S5. Construct a random forest model based on the health status of pigs, take the scores of the pigs' health indicators as independent variables, obtain the weights of the health indicators through random forest model training, establish a pig health scoring model based on the weights of the health indicators, and calculate the pigs' health scores.

[0012] Further: In S2, the method for extracting the weight characteristics of pigs is specifically:

[0013] Input the RGB image into the RGB branch to generate an RGB feature map;

[0014] Input the depth image into the depth branch to generate a depth feature map;

[0015] Generate a confidence map consistent with the size of the depth image through the confidence branch;

[0016] The confidence map is multiplied by the depth feature map of the corresponding proportion through the fusion module to generate an enhanced depth feature map, the enhanced depth feature map is merged with the RGB feature map, and a fused feature map is generated through a 1×1 convolution operation. The weight characteristics of the pigs are extracted through the fused feature map.

[0017] Furthermore, the RGB branch and the depth branch have the same structure, both of which are ResNet, wherein the method for ResNet to process the input image is specifically as follows:

[0018] The input image is input into the conv1 to conv4 convolutional layers in ResNet in sequence for convolution operations to generate the output feature map.

[0019] Further: The method for generating the confidence map is specifically:

[0020] Generate a matrix, identify the areas with reliable depth values ​​in the depth image in the matrix, and use 0 to represent the positions of the depth value 0 in the depth image in the matrix, generate a validity mask feature map based on the identified matrix, input the validity mask feature map into the confidence map estimator, and generate a confidence map after five convolution operations.

[0021] Further: In S2, the method for extracting the body temperature characteristics of pigs is specifically:

[0022] A set of matching feature points are selected on the infrared image and the RGB image through the body temperature extraction model, and the affine transformation matrix is ​​calculated based on the coordinate information of the matching feature points. The affine transformation matrix is ​​used to transform the coordinates on the RGB image into the coordinates on the infrared image.

[0023] The RTMDet target detection algorithm is used to obtain the position information of individual pigs and their ear base areas based on the RGB image, which is converted into position information on the infrared image through the affine transformation matrix. The infrared temperature matrix of the ear base area is obtained based on the position information, and the maximum temperature of the pig's ear base area is obtained based on the infrared temperature matrix, which is used as the pig's body temperature feature.

[0024] Further: In S2, the method for extracting the pig's body surface characteristics is specifically:

[0025] The body surface condition extraction model is used to identify the lesions on the pig's body surface in the RGB image according to the RTMDet target detection algorithm to generate the pig's body surface features.

[0026] Further: In S2, the method for extracting the behavior characteristics of pigs is specifically as follows:

[0027] The RTMPose key point detection algorithm is used to obtain the skeleton joint points of the pig according to the video frame of the RGB image, and the skeleton image sequence of the pig is generated. The skeleton image sequence and the corresponding RGB video frame sequence are input into the pig behavior recognition model to obtain the pig behavior characteristics;

[0028] Among them, the pig behavior recognition model includes a skeleton image processing branch and an RGB processing branch, both of which are connected to the feature fusion module, and the feature fusion module is also connected to the classification layer. The method for obtaining the pig behavior characteristics through the pig behavior recognition model is specifically as follows:

[0029] The skeleton graph processing branch extracts the spatiotemporal features of the skeleton graph sequence according to the spatiotemporal graph convolutional network to obtain skeleton features, and the RGB processing branch extracts the spatiotemporal features of the RGB video frame sequence according to the R(2+1)D network to obtain RGB features;

[0030] The skeleton features and RGB features are input into the feature fusion module, and the skeleton features are converted into a pixel with a length of C through a global average pooling operation. s The vector, C s is the number of channels of the skeleton feature, and the vector is adjusted to the dimension size C through the broadcast operation s ×W×H tensor, W is the width of the tensor, H is the length of the tensor, and the tensor and RGB features are sequenced from the first frame to the Tth frame. r Frame splicing operation, T r is the length of the input RGB video frame sequence. The concatenated tensor is input into a 1×1×1 convolution to adjust its channel size and obtain the fused feature.

[0031] The fused features are input into the classification layer to obtain the behavior classification results, and the pig behavior characteristics are obtained through statistics.

[0032] Further: In S3, the method for obtaining the pig weight health index is specifically:

[0033] Select a weight index model based on the current growth stage and breed of the pig, and output the pig weight health index based on the weight characteristics and age of the pig through the weight index model;

[0034] The specific method for obtaining the pig's body temperature health index is:

[0035] Select the corresponding temperature index model according to the type of pig house where the pigs are located, and output the pig temperature health index according to the temperature characteristics of the pigs and the ambient temperature and humidity data through the temperature index model;

[0036] The specific method for obtaining the pig's body health index is:

[0037] The body surface health index model detects abnormal symptoms and their severity of pigs’ body surface characteristics and outputs pigs’ body surface health indexes;

[0038] The specific method for obtaining the behavioral health indicators of pigs is:

[0039] The behavior indicator model is selected according to the current growth stage of the pigs. The behavior indicator model extracts the pig behavior type and frequency based on the pig behavior characteristics, and outputs the pig behavior health index.

[0040] Further: In S5, the weight of the health index includes the weight w of the pig weight health index Weight , the weight of pig body temperature health index w Temp , weights of pig surface health indicators w Skin and the weights of the pig behavioral health indicators w Behavior ;

[0041] The specific expression of the pig health score calculated by the pig health score model is:

[0042] H=w Weight ·S Weight +w Temp ·S Temp +w Skin ·S Skin +w Behavior ·S Behavior

[0043] In the formula, S Weight S is the score of pig weight health index, Temp S is the score of pig body temperature health index, Skin S is the score of pig surface health index, Behavior Scoring of behavioral health indicators for pigs.

[0044] The beneficial effects of the present invention are:

[0045] (1) The present invention provides a method for evaluating the health of pigs based on a track-mounted inspection robot. The method collects video images of pigs and environmental data of pig houses by designing a track-mounted inspection robot equipped with a variety of acquisition devices, extracts the physiological and behavioral characteristics of pigs by constructing a deep learning model, constructs health indicators of pigs based on the above characteristics, and then realizes the health score of pigs, thereby realizing an automated health assessment of pigs.

[0046] (2) The present invention uses a track-type inspection robot equipped with a variety of data acquisition devices to collect data and inspect the entire pig house. Compared with the traditional fixed camera acquisition strategy, the track-type inspection robot has higher flexibility, can cover a wider area, and can adjust the inspection route and time as needed, thereby ensuring the comprehensiveness and accuracy of data collection. In addition, the use of mobile devices makes the system deployment and maintenance costs more controllable.

[0047] (3) Accurate weight monitoring and health assessment: The present invention designs a weight estimation model that combines RGB and depth images to monitor the daily weight changes of pigs. By fusing the data of the two modalities, a more accurate pig weight estimation is achieved. At the same time, a health assessment index is designed to reflect the growth status and health status of pigs. This index combines the pig weight estimated by the algorithm with the age and type of the target pigs, providing a scientific health management basis for breeders, helping to timely detect growth abnormalities and optimize feeding strategies.

[0048] (4) Accurate temperature measurement and health assessment: This paper designs a pig temperature detection model that combines RGB images and infrared images. By improving the positioning accuracy of the target area and the accuracy of temperature reading, it realizes the accurate measurement of the pig's ear root area temperature. The temperature information and environmental temperature and humidity data are further combined to construct the pig's temperature health index, which enhances the comprehensiveness and reliability of health assessment.

[0049] (5) Automated lesion identification and health assessment: The present invention realizes the automatic identification and severity classification of pig surface lesions through the combination of target detection and classification models. It also designs a pig surface health assessment index based on the type and severity of pig surface lesions, which can help breeders quickly identify pig surface problems and effectively improve the efficiency and accuracy of disease prevention and control.

[0050] (6) Behavior recognition and health monitoring: The present invention designs a pig behavior recognition model that combines time-series pig skeleton information and RGB video streams, and improves the accuracy of pig behavior recognition by guiding RGB feature learning through skeleton features. The behavior recognition model counts the time and frequency of various behaviors of pigs, and constructs pig behavior health assessment indicators based on the age and type of pigs, making health monitoring more comprehensive, helping to detect behavioral abnormalities in a timely manner and prevent potential health problems.

[0051] (7) Calculation of health index weights: The present invention uses a random forest model to calculate the weights of various health indexes. The random forest model determines the weights of different health indexes by analyzing the impact of various health indexes on the health status of pigs, making the evaluation results of the health status of pigs more scientific and objective. This method not only improves the accuracy of the evaluation, but also provides a reliable decision-making basis for health management.

[0052] (8) Multi-dimensional health assessment system: The present invention constructs a comprehensive health assessment system by integrating multi-dimensional data such as pig weight, body temperature, body surface condition and behavior. This multi-dimensional data fusion method effectively avoids the limitations of single indicator evaluation and improves the comprehensiveness and accuracy of health status assessment. At the same time, it also provides a richer source of information and decision-making basis for disease prevention and management.

[0053] (9) Multi-scale health monitoring and early warning: Based on the track-type inspection robot, the present invention can realize health status assessment at multiple time and space scales, such as individual pigs, the entire pig house, single inspection results, and full-day inspection results. This system method can comprehensively evaluate the health status of the farm and provide multi-level health monitoring and early warning from individual to overall. Multi-angle assessment can timely discover potential health problems and improve the management level and production efficiency of the farm. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 The present invention is a flow chart of a pig health assessment method based on a track-type inspection robot. DETAILED DESCRIPTION

[0055] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0056] like Figure 1 As shown, in one embodiment of the present invention, a method for evaluating the health of pigs based on a track-type inspection robot comprises the following steps:

[0057] S1. Collect RGB images, infrared images and depth images of pigs through the track inspection robot, and record the health status of pigs;

[0058] S2. Extract the physiological and behavioral characteristics of pigs based on RGB images, infrared images and depth images through a deep learning model, including pig weight characteristics, pig body temperature characteristics, pig body surface characteristics and pig behavior characteristics;

[0059] S3. Obtaining the health indicators of the pigs according to the physiological and behavioral characteristics of the pigs, including the health indicators of the pigs' weight, the health indicators of the pigs' body temperature, the health indicators of the pigs' body surface and the health indicators of the pigs' behavior;

[0060] S4. Standardizing the health indicators of the pigs by z-score to obtain the scores of the health indicators of the pigs, including the scores of the health indicators of the pigs' weight, the health indicators of the pigs' body temperature, the health indicators of the pigs' body surface, and the health indicators of the pigs' behavior;

[0061] S5. Construct a random forest model based on the health status of pigs, take the scores of the pigs' health indicators as independent variables, obtain the weights of the health indicators through random forest model training, establish a pig health scoring model based on the weights of the health indicators, and calculate the pigs' health scores.

[0062] In this embodiment, a track-type inspection robot is equipped with collection equipment to obtain RGB, infrared, depth images and environmental data of pigs. The installation position of the track-type inspection robot is about 2 to 2.5 meters from the ground to ensure that the camera can clearly capture the pig's body, clean and organize it to obtain a standardized data set, and at the same time record the health status of pigs in the farm as a reference point for health assessment and divide it into three levels: healthy, sub-healthy and sick according to the severity of health abnormalities.

[0063] In S2, the method for extracting the weight characteristics of pigs is specifically as follows:

[0064] Input the RGB image into the RGB branch to generate an RGB feature map;

[0065] Input the depth image into the depth branch to generate a depth feature map;

[0066] Generate a confidence map consistent with the size of the depth image through the confidence branch;

[0067] The confidence map is multiplied by the corresponding proportion of the depth feature map through the fusion module to generate an enhanced depth feature map, which is then merged with the RGB feature map. A fusion feature map is generated through a 1×1 convolution operation, and the weight features of the pigs are extracted through the fusion feature map. This operation makes full use of the information in the RGB and depth data, greatly improving the overall performance of the model.

[0068] In this embodiment, in order to achieve contactless and accurate pig weight estimation, the present invention designs a pig weight estimation model, including an RGB branch, a depth branch, a confidence branch and a fusion module, wherein both the RGB and depth branches use an improved ResNet as a feature extraction network, and the computational complexity is reduced by modifying the number of groups of the residual blocks in the bottleneck structure of ResNet and introducing an inverted residual structure and a linear bottleneck layer. In order to improve the accuracy of the model, the present invention designs a multimodal feature fusion backbone network based on a confidence map, aiming to enhance the flexibility of feature learning and achieve more accurate pig weight estimation through the efficient fusion of RGB and depth features.

[0069] The structure of the RGB branch and the depth branch is the same, both are ResNet, wherein the method of ResNet processing the input image is specifically as follows:

[0070] The input image is input into the conv1 to conv4 convolutional layers in ResNet in sequence for convolution operation to generate the output feature map.

[0071] The method for generating a confidence map is as follows:

[0072] Generate a matrix, identify the areas with reliable depth values ​​in the depth image in the matrix, and use 0 to represent the positions of the depth value 0 in the depth image in the matrix, generate a validity mask feature map based on the identified matrix, input the validity mask feature map into the confidence map estimator, and generate a confidence map after five convolution operations.

[0073] In this embodiment, the confidence map estimator uses depth information to accurately disperse the validity mask feature map at the pixel level through five convolution operations to provide input to the fusion module.

[0074] In S2, the method for extracting the body temperature characteristics of pigs is specifically as follows:

[0075] A set of matching feature points are selected on the infrared image and the RGB image through the body temperature extraction model, and the affine transformation matrix is ​​calculated based on the coordinate information of the matching feature points. The affine transformation matrix is ​​used to transform the coordinates on the RGB image into the coordinates on the infrared image.

[0076] The RTMDet target detection algorithm is used to obtain the position information of individual pigs and their ear base areas based on the RGB image, which is converted into position information on the infrared image through the affine transformation matrix. The infrared temperature matrix of the ear base area is obtained based on the position information, and the maximum temperature of the pig's ear base area is obtained based on the infrared temperature matrix, which is used as the pig's body temperature feature.

[0077] In this embodiment, the present invention extracts the temperature at the base of the pig's ear as a representation of its body temperature based on the pig's RGB and infrared images. In order to extract the pig's body temperature, it is necessary to collect the pig's color and infrared image data in the early stage for the training of the body temperature extraction model. The data is used to train the target detection algorithm to detect individual pigs and the base of their ears from the image.

[0078] In S2, the method for extracting the pig's body surface characteristics is specifically as follows:

[0079] The body surface condition extraction model is used to identify the lesions on the pig's body surface in the RGB image according to the RTMDet target detection algorithm to generate the pig's body surface features.

[0080] In this embodiment, in order to extract the pig's body surface information, it is necessary to collect the pig's color image data in the early stage and mark the existing pathological features for training the body surface condition extraction model. The pathological features include skin redness, skin congestion, scars, erythema blisters, etc.

[0081] In S2, the method for extracting the behavior characteristics of pigs is specifically as follows:

[0082] The RTMPose key point detection algorithm is used to obtain the skeleton joint points of the pig according to the video frame of the RGB image, and the skeleton image sequence of the pig is generated. The skeleton image sequence and the corresponding RGB video frame sequence are input into the pig behavior recognition model to obtain the pig behavior characteristics;

[0083] Among them, the pig behavior recognition model includes a skeleton image processing branch and an RGB processing branch, both of which are connected to the feature fusion module, and the feature fusion module is also connected to the classification layer. The method for obtaining the pig behavior characteristics through the pig behavior recognition model is specifically as follows:

[0084] The skeleton graph processing branch extracts the spatiotemporal features of the skeleton graph sequence according to the spatiotemporal graph convolutional network to obtain skeleton features, and the RGB processing branch extracts the spatiotemporal features of the RGB video frame sequence according to the R(2+1)D network to obtain RGB features;

[0085] The skeleton features and RGB features are input into the feature fusion module, and the skeleton features are converted into a pixel with a length of C through a global average pooling operation. s The vector, C s is the number of channels of the skeleton feature, and the vector is adjusted to the dimension size C through the broadcast operation s ×W×H tensor, W is the width of the tensor, H is the length of the tensor, and the tensor and RGB features are sequenced from the first frame to the Tth frame. r Frame splicing operation, T ris the length of the input RGB video frame sequence. The concatenated tensor is input into a 1×1×1 convolution to adjust its channel size and obtain the fused feature.

[0086] The fused features are input into the classification layer to obtain the behavior classification results, and the pig behavior characteristics are obtained through statistics.

[0087] In this embodiment, the present invention identifies the behavior of pigs based on the video stream of their RGB images. In order to realize the identification of pig behaviors, it is necessary to collect the color video stream data of the pigs in the early stage and mark the behaviors appearing therein.

[0088] In S3, corresponding health indicators are established and scored for the pigs' weight, body temperature, body surface condition and behavior. The method for obtaining the pigs' weight health indicators is specifically as follows:

[0089] A weight index model is selected according to the current growth stage and breed of the pig, and the weight health index of the pig is output through the weight index model according to the weight characteristics of the pig and the age of the pig; the weight health index of the pig is divided into three intervals: thin weight, normal weight, and fat weight.

[0090] The specific method for obtaining the pig's body temperature health index is:

[0091] The temperature index model is selected according to the current growth stage of the pigs and the type of pig house they are in. The temperature index model outputs the pigs' body temperature health index based on the pigs' body temperature characteristics and the ambient temperature and humidity data. The pigs' body temperature health index is divided into three intervals: hypothermia, normal body temperature, and heat stress.

[0092] The specific method for obtaining the pig's body health index is:

[0093] The body surface health index model is used to detect abnormal body surface symptoms and their severity according to the body surface characteristics of the pigs, and the body surface health index of the pigs is output; the body surface health index of the pigs is divided into two intervals: abnormal body surface and normal body surface.

[0094] The specific method for obtaining the behavioral health indicators of pigs is:

[0095] The behavior indicator model is selected according to the current growth stage of the pig. The behavior indicator model extracts the pig behavior type and frequency according to the pig behavior characteristics, and outputs the pig behavior health index. The pig behavior health index is divided into two intervals: abnormal behavior and normal behavior.

[0096] In S4, after obtaining various health indicators, the Z-score is used to standardize the data to make the data from different sources comparable, and finally generate health indicator scores for 4 different areas of concern.

[0097] In S5, the weight of the health index includes the weight of the pig weight health index w Weight , the weight of pig body temperature health index w Temp , weights of pig surface health indicators w Skin and the weights of the pig behavioral health indicators w Behavior ;

[0098] The specific expression of the pig health score calculated by the pig health score model is:

[0099] H=w Weight ·S Weight +w Temp ·S Temp +w Skin ·S Skin +w Behavior ·S Behavior

[0100] In the formula, S Weight S is the score of pig weight health index, Temp S is the score of pig body temperature health index, Skin S is the score of pig surface health index, Behavior Scoring of behavioral health indicators for pigs.

[0101] In this embodiment, the above-established pig health index score is used as the independent variable, and the collected pig health status is used as the dependent variable. The random forest model is constructed by the scikit-learn library in python for training. 70% of the samples are randomly selected as the training set, and the remaining 30% are the test set to generate 1000 classic decision trees. After the training is completed, the feature_importances_ method is called to calculate the mean square error percentage and node purity increment of each health index, and the normalized average of the two is used as the weight of each health index. The pig health scoring model is established according to the weight of each index.

[0102] The beneficial effects of the present invention are as follows: the present invention provides a pig health assessment method based on a track-type inspection robot, which collects pig video images and pig house environmental data by designing a track-type inspection robot equipped with a variety of acquisition devices, and extracts the physiological and behavioral characteristics of pigs by constructing a deep learning model, and constructs pig health indicators based on the above characteristics, thereby achieving pig health scores, thereby realizing an automated pig health assessment.

[0103] In the description of the present invention, it is necessary to understand that the orientation or positional relationship indicated by the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", and "third" are used only for descriptive purposes, and cannot be understood as indicating or implying the relative importance or the number of implicitly specified technical features. Therefore, the features defined by "first", "second", and "third" may explicitly or implicitly include one or more of the features.

Claims

1. A pig health assessment method based on a track-type inspection robot, characterized in that: The following steps are involved: S1. Collect RGB images, infrared images and depth images of pigs through the track inspection robot, and record the health status of pigs; S2. Extract the physiological and behavioral characteristics of pigs based on RGB images, infrared images and depth images through a deep learning model, including pig weight characteristics, pig body temperature characteristics, pig body surface characteristics and pig behavior characteristics; S3. Obtaining the health indicators of the pigs according to the physiological and behavioral characteristics of the pigs, including the health indicators of the pigs' weight, the health indicators of the pigs' body temperature, the health indicators of the pigs' body surface and the health indicators of the pigs' behavior; S4. Standardizing the health indicators of the pigs by z-score to obtain the scores of the health indicators of the pigs, including the scores of the health indicators of the pigs' weight, the health indicators of the pigs' body temperature, the health indicators of the pigs' body surface, and the health indicators of the pigs' behavior; S5. Construct a random forest model based on the health status of pigs, take the scores of the pigs' health indicators as independent variables, obtain the weights of the health indicators through random forest model training, establish a pig health scoring model based on the weights of the health indicators, and calculate the pigs' health scores.

2. The pig health assessment method based on a track-type inspection robot according to claim 1 is characterized in that: In S2, the method for extracting the weight characteristics of pigs is specifically as follows: Input the RGB image into the RGB branch to generate an RGB feature map; Input the depth image into the depth branch to generate a depth feature map; Generate a confidence map consistent with the size of the depth image through the confidence branch; The confidence map is multiplied by the depth feature map of the corresponding proportion through the fusion module to generate an enhanced depth feature map, the enhanced depth feature map is merged with the RGB feature map, and a fused feature map is generated through a 1×1 convolution operation. The weight characteristics of the pigs are extracted through the fused feature map.

3. The pig health assessment method based on a track-type inspection robot according to claim 2 is characterized in that: The structure of the RGB branch and the depth branch is the same, both are ResNet, wherein the method of ResNet processing the input image is specifically as follows: The input image is input into the conv1 to conv4 convolutional layers in ResNet in sequence for convolution operations to generate the output feature map.

4. The pig health assessment method based on a track-type inspection robot according to claim 2 is characterized in that: The method for generating a confidence map is as follows: Generate a matrix, identify the areas with reliable depth values ​​in the depth image in the matrix, and use 0 to represent the positions of the depth value 0 in the depth image in the matrix, generate a validity mask feature map based on the identified matrix, input the validity mask feature map into the confidence map estimator, and generate a confidence map after five convolution operations.

5. The pig health assessment method based on a track-type inspection robot according to claim 1 is characterized in that: In S2, the method for extracting the body temperature characteristics of pigs is specifically as follows: A set of matching feature points are selected on the infrared image and the RGB image through the body temperature extraction model, and the affine transformation matrix is ​​calculated based on the coordinate information of the matching feature points. The affine transformation matrix is ​​used to transform the coordinates on the RGB image into the coordinates on the infrared image. The RTMDet target detection algorithm is used to obtain the position information of individual pigs and their ear base areas based on the RGB image, which is converted into position information on the infrared image through the affine transformation matrix. The infrared temperature matrix of the ear base area is obtained based on the position information, and the maximum temperature of the pig's ear base area is obtained based on the infrared temperature matrix, which is used as the pig's body temperature feature.

6. The pig health assessment method based on a track-type inspection robot according to claim 1 is characterized in that: In S2, the method for extracting the pig's body surface characteristics is specifically as follows: The body surface condition extraction model is used to identify the lesions on the pig's body surface in the RGB image according to the RTMDet target detection algorithm to generate the pig's body surface features.

7. The method for evaluating pig health based on a track-type inspection robot according to claim 1, characterized in that: In S2, the method for extracting the behavior characteristics of pigs is specifically as follows: The RTMPose key point detection algorithm is used to obtain the skeleton joint points of the pig according to the video frame of the RGB image, and the skeleton image sequence of the pig is generated. The skeleton image sequence and the corresponding RGB video frame sequence are input into the pig behavior recognition model to obtain the pig behavior characteristics; Among them, the pig behavior recognition model includes a skeleton image processing branch and an RGB processing branch, both of which are connected to the feature fusion module, and the feature fusion module is also connected to the classification layer. The method for obtaining the pig behavior characteristics through the pig behavior recognition model is specifically as follows: The skeleton graph processing branch extracts the spatiotemporal features of the skeleton graph sequence according to the spatiotemporal graph convolutional network to obtain skeleton features, and the RGB processing branch extracts the spatiotemporal features of the RGB video frame sequence according to the R(2+1)D network to obtain RGB features; The skeleton features and RGB features are input into the feature fusion module, and the skeleton features are converted into a pixel with a length of C through a global average pooling operation. s The vector, C s is the number of channels of the skeleton feature, and the vector is adjusted to the dimension size C through the broadcast operation s ×W×H tensor, W is the width of the tensor, H is the length of the tensor, and the tensor and RGB features are sequenced from the first frame to the Tth frame. r Frame splicing operation, T r is the length of the input RGB video frame sequence. The concatenated tensor is input into a 1×1×1 convolution to adjust its channel size and obtain the fused feature. The fused features are input into the classification layer to obtain the behavior classification results, and the pig behavior characteristics are obtained through statistics.

8. The method for evaluating pig health based on a track-type inspection robot according to claim 1, characterized in that: In S3, the method for obtaining the pig weight health index is specifically as follows: Select a weight index model based on the current growth stage and breed of the pig, and output the pig weight health index based on the weight characteristics and age of the pig through the weight index model; The specific method for obtaining the pig's body temperature health index is: Select the corresponding temperature index model according to the type of pig house where the pigs are located, and output the pig temperature health index according to the temperature characteristics of the pigs and the ambient temperature and humidity data through the temperature index model; The specific method for obtaining the pig's body health index is: The body surface health index model detects abnormal symptoms and their severity of pigs’ body surface characteristics and outputs pigs’ body surface health indexes; The specific method for obtaining the behavioral health indicators of pigs is: The behavior indicator model is selected according to the current growth stage of the pigs. The behavior indicator model extracts the pig behavior type and frequency based on the pig behavior characteristics, and outputs the pig behavior health index.

9. The method for evaluating pig health based on a track-type inspection robot according to claim 1, characterized in that: In S5, the weight of the health index includes the weight of the pig weight health index w Weight , the weight of pig body temperature health index w Temp , weights of pig surface health indicators w Skin and the weights of the pig behavioral health indicators w Behavior ; The specific expression of the pig health score calculated by the pig health score model is: H=w Weight ·S Weight +w Temp ·S Temp +w Skin ·S Skin +w Behavior ·S Behavior In the formula, S Weight S is the score of pig weight health index, Temp S is the score of pig body temperature health index, Skin S is the score of the pig's body health index. Behavior Scoring of behavioral health indicators for pigs.

Images