Pig health assessment method based on rail-mounted inspection robot

Abstract

Disclosed in the present invention is a pig health assessment method based on a rail-mounted inspection robot. The method comprises the following steps: S1, collecting RGB images, infrared images and depth images of pigs by means of a rail-mounted inspection robot, and recording the health condition of the pigs; S2, on the basis of the RGB images, the infrared images and the depth images, extracting physiological and behavioral features of the pigs by means of a deep learning model; S3, on the basis of the physiological and behavioral features of the pigs, obtaining health indicators of the pigs; S4, normalizing the health indicators of the pigs by means of z-score, so as to obtain scores of the health indicators of the pigs; and S5, constructing a random forest model on the basis of the health condition of the pigs, using the scores of the health indicators of the pigs as independent variables, performing training by means of the random forest model to obtain weights of the health indicators, establishing a pig health scoring model on the basis of the weights of the health indicators, and calculating health scores of the pigs. Thus, automatic health assessment of pigs is realized.

Description

Pig health assessment method based on orbital inspection robot TECHNICAL FIELD

[0001] The present application belongs to the field of pig health assessment, and particularly relates to a pig health assessment method based on an orbital inspection robot. BACKGROUND

[0002] To ensure normal supply of pork, improve the risk resistance of pig breeding and reduce breeding costs, pig breeding is gradually developing towards intensification, large-scale and automation. Data shows that individual breeders are gradually decreasing. At present, the intensive pig breeding mode in China is developing rapidly, but due to late start, low intelligence and weak production foundation, there are deficiencies in the protection against diseases. At the same time, the characteristics of small space and large individual base under the intensive breeding mode make pigs more likely to cross-infect and cause large-scale outbreaks of diseases, resulting in heavy losses for the breeding farm. Therefore, in the process of pig breeding under the intensive breeding mode, timely and effective assessment and feedback of the health status of pigs and health early warning are helpful for the breeding farm to make quick decisions and reduce losses. The health status of pigs is judged based on various health indicators. For pig breeding, the health status of pigs can be comprehensively evaluated by obtaining various physiological and behavioral characteristics of pigs and extracting the health indicators contained therein, and the feeding scheme can be adjusted accordingly, which is of great significance for the development of precision livestock breeding and animal welfare assessment.

[0003] Pig health monitoring is of great significance to pig breeding management. At present, the actual production still relies on manual patrol method to monitor the health status of pigs. Pig farm staff judge by observing the behavior of pigs and collecting part of the physiological indicators, but the manual patrol method is not only time-consuming, high-cost and low-efficient, but also depends on the personal experience of the staff, lacks unified standards and cannot realize industrialization promotion. In addition, when the pigs show obvious abnormalities, they are usually in the late stage of disease, and it is difficult to find pigs in sub-health state or in the early stage of disease in time.

[0004] In recent years, researchers have explored the use of various wearable sensors and cameras to collect pig data for abnormal physiological and behavioral characteristics of pigs. For example, Cangyan et al. used ReSpeaker Core v2.0 development board and MobileNetV2 network technology to classify pig sounds, with a recognition rate of 97.3%; Li Juxia et al. realized the recognition of pig drinking behavior in different view images through Yolov4 network; Xue Yujing et al. proposed a deep video recognition algorithm based on hidden Markov model to solve the problem of recognizing high-risk actions of lactating sows; Zhou Liping et al. proposed an improved Ostu algorithm based on thermal infrared images to detect the ear root area of live pigs; Zhao Haitao borrowed the key point detection technology of human face to automatically identify the key parts of the infrared image of pig face. However, the above research only obtains a certain physiological or behavioral characteristic of pigs, and does not consider the health status of pigs from the perspective of the overall condition of pigs. At the same time, the fixed equipment is easy to damage and fall, which makes the cost not obviously superior to manual work, and it is difficult to popularize and apply. The use of a patrol robot carrying multiple detection devices to patrol and extract feature information of all pigs in the pig house can effectively solve the problems of single cost and evaluation angle. SUMMARY

[0005] In view of the above problems in the prior art, the pig health evaluation method based on the track type patrol robot provided by the present application solves the problem that the pig health status evaluation in the prior art relies on manual inspection for judgment.

[0006] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: a pig health evaluation method based on a track type patrol robot, comprising the following steps:

[0007] S1, collecting RGB images, infrared images and depth images of pigs by a track type patrol robot, and recording the health status of pigs;

[0008] S2, extracting physiological and behavioral characteristics of pigs from RGB images, infrared images and depth images through a deep learning model, including pig weight characteristics, pig temperature characteristics, pig surface characteristics and pig behavior characteristics;

[0009] S3, obtaining health indicators of pigs according to the physiological and behavioral characteristics of pigs, including pig weight health indicators, pig temperature health indicators, pig surface health indicators and pig behavior health indicators;

[0010] S4, standardizing the health indicators of pigs by z-score to obtain the scores of the health indicators of pigs, including the scores of the pig weight health indicators, the scores of the pig temperature health indicators, the scores of the pig surface health indicators and the scores of the pig behavior health indicators;

[0011] S5, constructing a random forest model according to the health status of the pigs, taking the scores of the health indicators of the pigs as independent variables, training the health indicator weight through the random forest model, and establishing a pig health score model according to the health indicator weight to calculate the health score of the pigs.

[0012] Further, in the S2, the method for extracting the pig weight feature is specifically:

[0013] The RGB image is input into the RGB branch to generate an RGB feature map;

[0014] The depth image is input into the depth branch to generate a depth feature map;

[0015] A confidence map consistent with the size of the depth image is generated through the confidence branch;

[0016] The confidence map is multiplied with the corresponding proportional depth feature map through the fusion module to generate an enhanced depth feature map, the enhanced depth feature map and the RGB feature map are combined, a fusion feature map is generated through a 1x1 convolution operation, and the pig weight feature is extracted through the fusion feature map.

[0017] Further, the structure of the RGB branch and the depth branch is the same, both of which are ResNet, wherein the method for ResNet to process the input image is specifically:

[0018] The input image is sequentially input into the conv1 to conv4 convolution layers in the ResNet for convolution operation to generate an output feature map.

[0019] Further, the method for generating the confidence map is specifically:

[0020] A matrix is generated, in which the reliable area of the depth value in the depth image is marked, and the position of the depth value of 0 in the depth image is represented by 0 in the matrix, an effectiveness mask feature map is generated according to the marked matrix, the effectiveness mask feature map is input into a confidence map estimator, and a confidence map is generated through five convolution operations.

[0021] Further, in the S2, the method for extracting the pig temperature feature is specifically:

[0022] A set of matched feature points are selected on the infrared image and the RGB image through the temperature extraction model, an affine transformation matrix is calculated through the coordinate information of the matched feature points, and the affine transformation matrix is used to convert the coordinates on the RGB image into the coordinates on the infrared image.

[0023] The position information of the pig individual and the ear root region thereof is obtained from the RGB image by the RTMDet target detection algorithm, the position information is converted into the position information on the infrared image by an affine transformation matrix, the infrared temperature matrix of the region at the ear root is obtained according to the position information, the highest temperature of the region at the ear root of the pig is obtained according to the infrared temperature matrix, and the highest temperature is taken as the pig body temperature feature.

[0024] Further, in S2, the method for extracting the pig body surface feature is specifically:

[0025] The pig body surface feature is generated by the body surface condition extraction model according to the identification of the pig body surface lesions in the RGB image by the RTMDet target detection algorithm.

[0026] Further, in S2, the method for extracting the pig behavior feature is specifically:

[0027] The skeleton joints of the pig are obtained from the video frames of the RGB image according to the RTMPose key point detection algorithm, the skeleton graph sequence of the pig is generated, the skeleton graph sequence and the corresponding RGB video frame sequence are input into the pig behavior recognition model, and the pig behavior feature is obtained.

[0028] The pig behavior recognition model includes a skeleton graph processing branch and an RGB processing branch, the skeleton graph processing branch and the RGB processing branch are connected with a feature fusion module, the feature fusion module is further connected with a classification layer, and the method for obtaining the pig behavior feature by the pig behavior recognition model is specifically:

[0029] The skeleton feature is obtained by extracting the time-space feature of the skeleton graph sequence according to the space-time graph convolution network through the skeleton graph processing branch, and the RGB feature is obtained by extracting the time-space feature of the RGB video frame sequence according to the R(2+1)D network through the RGB processing branch.

[0030] The skeleton feature and the RGB feature are input into the feature fusion module, the skeleton feature is converted into a vector with a length of C s by a global average pooling operation, C s is the channel number of the skeleton feature, the vector is adjusted to a tensor with a dimension size of C s by a broadcast operation, W is the width of the tensor, and H is the length of the tensor, the tensor and the RGB feature are spliced according to the sequence from the first frame to the T r frame, T r is the length of the input RGB video frame sequence, the spliced tensor is input into a convolution with a size of 1*1*1 to adjust the channel size, and the fusion feature is obtained.

[0031] The fusion feature is input into the classification layer to obtain the behavior classification result, and the pig behavior feature is obtained by counting.

[0032] Further, in the S3, the method for obtaining the pig weight health index is specifically:

[0033] According to the current growth stage and breed of the pig, a weight index model is selected, and a pig weight health index is output by the weight index model according to pig weight characteristics and the age of the pig;

[0034] The method for obtaining the pig temperature health index is specifically:

[0035] According to the type of the pig house where the pig is located, a corresponding temperature index model is selected, and a pig temperature health index is output by the temperature index model according to pig temperature characteristics and environmental temperature and humidity data;

[0036] The method for obtaining the pig surface health index is specifically:

[0037] According to pig surface characteristics, a surface health index model is used to detect abnormal symptoms and their severity on the surface of the pig, and a pig surface health index is output;

[0038] The method for obtaining the pig behavior health index is specifically:

[0039] According to the current growth stage of the pig, a behavior index model is selected, and a pig behavior health index is output by the behavior index model according to the pig behavior type and occurrence frequency extracted from pig behavior characteristics.

[0040] Further, in the S5, the weights of the health indexes include the weight w Weight of the pig weight health index, the weight w Temp of the pig temperature health index, the weight w Skin of the pig surface health index, and the weight w Behavior of the pig behavior health index.

[0041] The expression for calculating the health score of the pig by the pig health score model is specifically: H = w Weight · S Weight + w Temp · S Temp + w Skin · S Skin + w Behavior · S Behavior

[0042] In the formula, S Weight is the score of the pig weight health index, S Temp is the score of the pig temperature health index, S Skin is the score of the pig surface health index, and S Behavior is the score of the pig behavior health index.

[0043] The beneficial effects of the present application are:

[0044] (1) The present application provides a pig health evaluation method based on a track inspection robot, which collects pig video images and pig house environment data through a track inspection robot carrying multiple collection devices, extracts physiological and behavioral characteristics of pigs through a deep learning model, constructs a pig health index based on the above characteristics, and realizes pig health scoring, thereby realizing an automatic pig health evaluation.

[0045] (2) The present application uses a track inspection robot carrying multiple data collection devices to collect data and patrol the entire pig house. Compared with the traditional fixed camera collection strategy, the track inspection robot has higher flexibility, can cover a wider area, and can adjust the patrol route and time as needed, thereby ensuring the comprehensiveness and accuracy of data collection. In addition, the use of mobile devices makes the system deployment and maintenance cost more controllable.

[0046] (3) Precise weight monitoring and health assessment: The present application designs a weight estimation model combining RGB and depth images to monitor the daily weight change of pigs, and realizes more accurate pig weight estimation by fusing data of two modalities. At the same time, a health assessment index is designed to reflect the growth state and health condition of pigs, which combines the pig weight estimated by the algorithm with the age and type of the target pig, providing scientific health management basis for breeders, and helping to discover abnormal growth in time and optimize feeding strategies.

[0047] (4) Accurate temperature measurement and health assessment: The present application designs a pig temperature detection model combining RGB images and infrared images, which realizes accurate measurement of the ear root area temperature of pigs by improving the positioning accuracy of the target area and the accuracy of temperature reading. Further combining temperature information and environmental temperature and humidity data to construct pig temperature health index, enhances the comprehensiveness and reliability of health assessment.

[0048] (5) Automatic lesion identification and health assessment: The present application realizes automatic identification and severity classification of pig surface lesions through the combination of target detection and classification model. And design a pig surface condition health assessment index constructed according to the type and severity of pig surface lesions, which can help breeders quickly identify pig surface problems, effectively improving the efficiency and accuracy of disease prevention and control.

[0049] (6) Behavior recognition and health monitoring: The application designs a pig behavior recognition model that combines skeletal information and RGB video stream. The model uses skeletal features to guide RGB feature learning, improving the accuracy of pig behavior recognition. By analyzing the time and frequency of each behavior, the application constructs a pig behavior health assessment index based on the pig's age and type, making health monitoring more comprehensive and helping to detect behavior abnormalities and prevent potential health problems.

[0050] (7) Health index weight calculation: The application uses a random forest model to calculate the weight of each health index. The random forest model analyzes the impact of each health index on the pig's health status, determining the weight of different health indexes, making the evaluation of the pig's health status more scientific and objective. This method not only improves the accuracy of the evaluation, but also provides a reliable basis for health management.

[0051] (8) Multi-dimensional health assessment system: The application integrates multi-dimensional data such as pig weight, body temperature, body surface condition, and behavior to construct a comprehensive health assessment system. This multi-dimensional data fusion method effectively avoids the limitations of single index evaluation, improving the comprehensiveness and accuracy of health status assessment. At the same time, it provides more abundant information sources and decision-making basis for disease prevention and management

[0052] (9) Multi-scale health monitoring and early warning: Based on the track inspection robot, the application can evaluate the health status from multiple time and space scales, such as individual pigs, pig houses, single inspection results, and all-day inspection results. This system can comprehensively evaluate the health status of the farm, providing multi-level health monitoring and early warning from individual to whole. Multi-angle evaluation can timely detect potential health problems and improve the management level and production efficiency of the farm. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a flowchart of a pig health assessment method based on a track inspection robot according to the application. DETAILED DESCRIPTION

[0054] The specific embodiments of the application are described below to facilitate understanding of the application by those skilled in the art, but it should be clear that the application is not limited to the scope of the specific embodiments. For those skilled in the art, it is obvious that all changes within the spirit and scope of the application defined and determined by the appended claims are obvious, and all applications utilizing the concept of the application are within the scope of protection.

[0055] As shown in Figure 1, in one embodiment of the application, a pig health assessment method based on a track inspection robot includes the following steps:

[0056] S1, collecting RGB images, infrared images and depth images of the pigs by the track inspection robot, and recording the health status of the pigs;

[0057] S2, extracting physiological and behavioral characteristics of the pigs from the RGB images, infrared images and depth images by a deep learning model, including pig weight characteristics, pig temperature characteristics, pig surface characteristics and pig behavior characteristics;

[0058] S3, obtaining health indicators of the pigs according to the physiological and behavioral characteristics of the pigs, including pig weight health indicators, pig temperature health indicators, pig surface health indicators and pig behavior health indicators;

[0059] S4, standardizing the health indicators of the pigs by z-score to obtain scores of the health indicators of the pigs, including scores of the pig weight health indicators, scores of the pig temperature health indicators, scores of the pig surface health indicators and scores of the pig behavior health indicators;

[0060] S5, constructing a random forest model according to the health status of the pigs, taking the scores of the health indicators of the pigs as independent variables, obtaining weights of the health indicators by training the random forest model, establishing a pig health score model according to the weights of the health indicators, and calculating a health score of the pigs.

[0061] In the embodiment, the track inspection robot is used to obtain RGB, infrared, depth images and environmental data of the pigs by carrying a collection device. The track inspection robot is installed at a position about 2-2.5 m away from the ground, so that the camera can clearly capture the body of the pig, and standardize the obtained data set. At the same time, the health status of the pigs in the farm is recorded as a reference point for health evaluation, and is divided into three levels of health, sub-health and disease according to the severity of health abnormalities.

[0062] In S2, the method for extracting pig weight characteristics is specifically as follows:

[0063] The RGB image is input into the RGB branch to generate an RGB feature map;

[0064] The depth image is input into the depth branch to generate a depth feature map;

[0065] A confidence map consistent with the size of the depth image is generated by the confidence branch;

[0066] The confidence map is multiplied by the corresponding proportion of the depth feature map by the fusion module to generate an enhanced depth feature map. The enhanced depth feature map and the RGB feature map are combined, and a fusion feature map is generated by 1x1 convolution operation. The pig weight characteristics are extracted by the fusion feature map. This operation fully utilizes the information in the RGB and depth data, greatly improving the overall performance of the model.

[0067] In the embodiment, in order to realize non-contact precise pig weight estimation, the pig weight estimation model is designed, including an RGB branch, a depth branch, a confidence branch and a fusion module, wherein the RGB branch and the depth branch both adopt an improved ResNet as a feature extraction network, and the grouping number of the residual block in the bottleneck structure in the ResNet is modified, and a reverse residual structure and a linear bottleneck layer are introduced to reduce the calculation complexity. In order to improve the precision of the model, a multi-modal feature fusion backbone network based on a confidence map is designed, which aims to enhance the flexibility of feature learning through efficient fusion of RGB and depth features, and realize more accurate pig weight estimation.

[0068] The RGB branch and the depth branch have the same structure, both of which are ResNet, wherein the method for the ResNet to process the input image is specifically as follows:

[0069] The input image is sequentially input into the conv1 to conv4 convolution layers in the ResNet for convolution operation to generate an output feature map.

[0070] The method for generating the confidence map is specifically as follows:

[0071] A matrix is generated, in which the reliable area of the depth value in the depth image is marked, and the position of the depth value of 0 in the depth image is represented by 0 in the matrix. An effectiveness mask feature map is generated according to the marked matrix, and the effectiveness mask feature map is input into a confidence map estimator to generate a confidence map through five convolution operations.

[0072] In the embodiment, the confidence map estimator utilizes the depth information to accurately disperse the effectiveness mask feature map at the pixel level through five convolution operations, and provides input for the fusion module.

[0073] In the S2, the method for extracting the pig temperature feature is specifically as follows:

[0074] A set of matched feature points are selected on the infrared image and the RGB image through the temperature extraction model, and an affine transformation matrix is calculated according to the coordinate information of the matched feature points, which is used to convert the coordinates on the RGB image into coordinates on the infrared image.

[0075] The position information of the pig individual and the ear root area is obtained from the RGB image through the RTMDet target detection algorithm, which is converted into the position information on the infrared image through the affine transformation matrix, and the infrared temperature matrix of the ear root area is obtained according to the position information, and the highest temperature of the ear root area of the pig is obtained as the pig temperature feature according to the infrared temperature matrix.

[0076] In the embodiment, the application extracts the temperature at the ear root of the pig as a representation of the body temperature of the pig based on the RGB and infrared images of the pig. In order to realize the extraction of the body temperature of the pig, the color and infrared image data of the pig are collected in advance for the training of the body temperature extraction model. The data is used to train a target detection algorithm to detect the pig individual and the ear root in the image.

[0077] In S2, the method for extracting the pig body surface feature is specifically:

[0078] The pig body surface feature is generated by the body surface condition extraction model according to the RTMDet target detection algorithm to identify the lesions on the pig body surface in the RGB image.

[0079] In the embodiment, in order to realize the extraction of the pig body surface information, the color image data of the pig is collected in advance and the lesion features existing thereon are labeled for training the body surface condition extraction model. The lesion features include skin redness, skin congestion, scars, erythema blisters, etc.

[0080] In S2, the method for extracting the pig behavior feature is specifically:

[0081] The skeleton joints of the pig are obtained from the video frames of the RGB image by the RTMPose key point detection algorithm to generate a skeleton graph sequence of the pig. The skeleton graph sequence and the corresponding RGB video frame sequence are input into the pig behavior recognition model to obtain the pig behavior feature.

[0082] The pig behavior recognition model includes a skeleton graph processing branch and an RGB processing branch. Both the skeleton graph processing branch and the RGB processing branch are connected with a feature fusion module. The feature fusion module is further connected with a classification layer. The method for obtaining the pig behavior feature by the pig behavior recognition model is specifically:

[0083] The skeleton feature is obtained by the skeleton graph processing branch according to the spatio-temporal graph convolution network to extract the spatio-temporal feature of the skeleton graph sequence. The RGB feature is obtained by the RGB processing branch according to the R(2+1)D network to extract the spatio-temporal feature of the RGB video frame sequence.

[0084] The skeleton feature and the RGB feature are input into the feature fusion module. The skeleton feature is converted into a vector with a length of C s by a global average pooling operation, C s is the number of channels of the skeleton feature. The vector is adjusted to a tensor with a dimension size of C s ×W×H by a broadcast operation, W is the width of the tensor, and H is the length of the tensor. The tensor and the RGB feature are spliced according to their sequences from the first frame to the T r frame. T rFor the length of the input RGB video frame sequence, the spliced tensor is input into a 1x1x1 size convolution to adjust the channel size, and a fusion feature is obtained.

[0085] The fusion feature is input into a classification layer to obtain a behavior classification result, and pig behavior characteristics are obtained by statistics.

[0086] In the embodiment, the behavior of pigs is identified based on a video stream of pig RGB images. In order to realize the identification of pig behavior, color video stream data of pigs needs to be collected in advance and behaviors appearing therein are labeled.

[0087] In the S3, corresponding health indicators are established for pig weight, body temperature, body surface condition and behavior, and scores are assigned, wherein the method for obtaining the pig weight health indicator is specifically:

[0088] According to the current growth stage and breed of the pig, a weight index model is selected, and the pig weight health indicator is output by the weight index model according to the pig weight characteristics and the age of the pig. The pig weight health indicator is divided into three intervals of lean weight, normal weight and obese weight.

[0089] The method for obtaining the pig temperature health indicator is specifically:

[0090] According to the current growth stage and the type of pig house, a temperature index model is selected, and the pig temperature health indicator is output by the temperature index model according to the pig temperature characteristics and the environmental temperature and humidity data. The pig temperature health indicator is divided into three intervals of hypothermia, normal temperature and heat stress.

[0091] The method for obtaining the pig surface health indicator is specifically:

[0092] The pig surface health indicator is output by the pig surface health index model according to the pig surface characteristics to detect abnormal symptoms and their severity of the pig surface. The pig surface health indicator is divided into two intervals of abnormal surface and normal surface.

[0093] The method for obtaining the pig behavior health indicator is specifically:

[0094] According to the current growth stage of the pig, a behavior index model is selected, and the pig behavior health indicator is output by the behavior index model according to the pig behavior type and the occurrence frequency extracted from the pig behavior characteristics. The pig behavior health indicator is divided into two intervals of abnormal behavior and normal behavior.

[0095] In the S4, after obtaining each health indicator, the data is standardized using Z-score to make the data from different sources comparable, and finally four health indicator scores in different attention areas are generated.

[0096] The weight of the health index in S5 includes the weight w of the pig weight health index Weight The weight w of the pig temperature health index Temp The weight w of the pig surface health index Skin And the weight w of the pig behavior health index Behavior ;

[0097] The expression of calculating the health score of the pig through the pig health score model is: H=w Weight ·S Weight +w Temp ·S Temp +w Skin ·S Skin +w Behavior ·S Behavior

[0098] In the formula, S Weight is the score of the pig weight health index, S Temp is the score of the pig temperature health index, S Skin is the score of the pig surface health index, and S Behavior is the score of the pig behavior health index.

[0099] In the embodiment, the scores of the health indexes of the pigs established above are taken as independent variables, and the health conditions of the pigs collected are taken as dependent variables, and a random forest model is constructed through the scikit-learn library in python to perform training. 70% of the samples are randomly selected as a training set, and the remaining 30% is a test set, and 1000 classical decision trees are generated. After training, the feature_importances_method is called to calculate the mean square error percentage and the node purity increment of each health index, and the normalized average value of the two is taken as the weight of each health index, and the pig health score model is established according to the weight of each index.

[0100] The pig health evaluation method based on the track inspection robot provided by the application has the beneficial effects that: the pig video image and the pig house environment data are collected through the track inspection robot carrying multiple collection devices, and the physiological and behavior characteristics of the pigs are extracted through the construction of a deep learning model, the health indexes of the pigs are constructed based on the above characteristics, and the health score of the pigs is realized, so as to realize an automatic pig health evaluation.

[0101] In the description of the application, it needs to be understood that the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implying the number of technical features indicated. Therefore, the features defined by "first", "second", "third" can explicitly or implicitly include one or more of the features.

Claims

1. A pig health assessment method based on a trackable inspection robot, characterized in that, The method comprises the following steps: S1, collecting RGB images, infrared images and depth images of pigs by a track inspection robot, and recording the health status of the pigs; S2, extracting physiological and behavioral characteristics of the pigs from the RGB images, infrared images and depth images by a deep learning model, including pig weight characteristics, pig temperature characteristics, pig surface characteristics and pig behavior characteristics; S3, obtaining health indicators of the pigs according to the physiological and behavioral characteristics of the pigs, including pig weight health indicators, pig temperature health indicators, pig surface health indicators and pig behavior health indicators; S4, standardizing the health indicators of the pigs by z-score to obtain scores of the health indicators of the pigs, including scores of the pig weight health indicators, scores of the pig temperature health indicators, scores of the pig surface health indicators and scores of the pig behavior health indicators; S5, constructing a random forest model according to the health status of the pigs, taking the scores of the health indicators of the pigs as independent variables, training the health indicators by the random forest model to obtain weights of the health indicators, establishing a pig health score model according to the weights of the health indicators, and calculating a health score of the pigs. 2.The pig health assessment method based on the track-type inspection robot according to claim 1, wherein, In the S2, the method for extracting the pig weight characteristics is specifically as follows: The RGB image is input into an RGB branch to generate an RGB feature map; The depth image is input into a depth branch to generate a depth feature map; A confidence map consistent with the size of the depth image is generated through a confidence branch; The confidence map is multiplied by the corresponding proportional depth feature map through a fusion module to generate an enhanced depth feature map, the enhanced depth feature map and the RGB feature map are combined, a fusion feature map is generated through a 1x1 convolution operation, and the pig weight characteristics are extracted through the fusion feature map. 3.The pig health assessment method based on the track-type inspection robot according to claim 2, wherein, The RGB branch and the depth branch have the same structure, both of which are ResNet, and the method for processing the input image by the ResNet is specifically as follows: The input image is sequentially input into conv1 to conv4 convolution layers in the ResNet for convolution operation to generate an output feature map. 4.The pig health evaluation method based on the track-type inspection robot according to claim 2, wherein, The method for generating the confidence map is specifically as follows: A matrix is generated, in which the reliable area of the depth value in the depth image is marked, and the position of the depth value of 0 in the depth image is represented by 0 in the matrix, and an effectiveness mask feature map is generated according to the marked matrix, the effectiveness mask feature map is input into a confidence map estimator, and the confidence map is generated through five convolution operations. 5.The pig health evaluation method based on the track-type inspection robot according to claim 1, wherein, In the S2, the method for extracting the pig temperature characteristics is specifically as follows: An affine transformation matrix is calculated through the coordinates of the matched feature points, and the affine transformation matrix is used to convert the coordinates on the RGB image into the coordinates on the infrared image; The position information of the pig individual and the ear root area is obtained from the RGB image by an RTMDet target detection algorithm, the position information is converted into the position information on the infrared image by the affine transformation matrix, the infrared temperature matrix of the ear root area is obtained according to the position information, and the highest temperature of the ear root area of the pig is obtained as the pig temperature characteristics according to the infrared temperature matrix.

6. The method of claim 1, wherein the method further comprises: In the S2, the method for extracting the pig surface characteristics is specifically as follows: The pig body surface feature is generated by identifying lesions appearing on the pig body surface in the RGB image according to an RTMDet target detection algorithm through the body surface condition extraction model.

7. The method of claim 1, wherein the method further comprises: determining a health score of the pig based on the detected health condition of the pig. The method for extracting the pig behavior feature in S2 is specifically as follows: The skeleton key points of the pig are acquired according to the video frames of the RGB image through an RTMPose key point detection algorithm, and a skeleton map sequence of the pig is generated, the skeleton map sequence and the corresponding RGB video frame sequence are input into a pig behavior recognition model, and the pig behavior feature is obtained; The pig behavior recognition model comprises a skeleton map processing branch and an RGB processing branch, the skeleton map processing branch and the RGB processing branch are connected with a feature fusion module, the feature fusion module is further connected with a classification layer, and the method for obtaining the pig behavior feature through the pig behavior recognition model is specifically as follows: The spatiotemporal features of the skeleton map sequence are extracted according to a spatiotemporal graph convolution network through the skeleton map processing branch, and skeleton features are obtained, the spatiotemporal features of the RGB video frame sequence are extracted according to an R(2+1)D network through the RGB processing branch, and RGB features are obtained; The skeleton feature and the RGB feature are input into a feature fusion module, the skeleton feature is converted into a vector with a length of C s by a global average pooling operation, C s is a channel number of the skeleton feature, the vector is adjusted to have a dimension size of C s ×W×H by a broadcast operation, W is a width of the tensor, H is a length of the tensor, the tensor is spliced with the RGB feature according to a sequence thereof from a first frame to a T r th frame, T r is a length of an input RGB video frame sequence, and the spliced tensor is input into a 1×1×1 size convolution to adjust a channel size, so that a fusion feature is obtained. The fusion features are input into the classification layer to obtain a behavior classification result, and the pig behavior feature is obtained by counting. 8.The pig health assessment method based on the track-type inspection robot according to claim 1, wherein, The method for obtaining the pig weight health index in S3 is specifically as follows: The pig weight health index is output by a weight index model according to the pig weight feature and the age of the pig through the weight index model according to the current growth stage and the breed of the pig; The method for obtaining the pig temperature health index is specifically as follows: The pig temperature health index is output by a temperature index model according to the pig temperature feature and the environmental temperature and humidity data through the temperature index model according to the type of the pig house where the pig is located; The method for obtaining the pig body surface health index is specifically as follows: The pig body surface health index is output by a body surface health index model according to the pig body surface feature, the abnormal symptoms of the pig body surface and the severity thereof are detected, and the pig body surface health index is output; The method for obtaining the pig behavior health index is specifically as follows: The pig behavior health index is output by a behavior index model according to the pig behavior type and the occurrence frequency of the pig behavior extracted according to the pig behavior feature according to the current growth stage of the pig. 9.The pig health assessment method based on the track-type inspection robot according to claim 1, wherein, The weight of the health index in S5 includes the weight w of the pig weight health index Weight , the weight w of the pig temperature health index Temp , the weight w of the pig surface health index Skin , and the weight w of the pig behavior health index Behavior ; The expression of the health score of the pig calculated through the pig health score model is specifically as follows: H = w Weight • S Weight + w Temp • S Temp + w Skin • S Skin + w Behavior • S Behavior where S Weight is the score of the pig's weight health indicator, S Temp is the score of the pig's temperature health indicator, S Skin is the score of the pig's skin health indicator, S Behavior is the score of the pig's behavior health indicator.

Images