A fence-aisle type cattle health data extraction device and intelligent extraction method thereof

Abstract

The present invention relates to a fence aisle type cattle health data extraction device and an intelligent extraction method thereof, which solves the defect that cattle health identification cannot meet actual use needs compared with the prior art. In the present invention, an electronic weight scale is installed at the bottom of the fence aisle, and cameras A, B, C, D, E and F, all of which are connected to a server, are installed on the fence aisle. The cameras A, B and C are all visible light image acquisition cameras, and the cameras D, E and F are all visible light and infrared image acquisition cameras. The present invention uses an image synthesis strategy to generate a frontal image of the cow, solving the problem that the frontal image acquisition device of the aisle type cattle health information acquisition device is easily damaged by the cow. Through the synthesized non-contact cow face temperature, the temperature at the cow's forehead can be accurately measured.

Description

Technical Field

[0001] The present invention relates to the field of animal husbandry information technology, and in particular to a fence-aisle type cattle health data extraction device and an intelligent extraction method thereof. Background Art

[0002] Animal husbandry is a vital component of my country's agricultural sector, and informatization of animal husbandry is a key approach to improving its efficiency. To achieve automated, information-based, and refined daily management of individual cattle on large-scale cattle farms, track the health of each individual cow, and trace the source of milk and meat products, it is essential to establish and improve a quality traceability system. The key lies in identifying individual cattle. Traditional cattle identification relies primarily on manual observation methods such as ear tags, branding, neck chains, and spike tags. These methods are not only time-consuming and labor-intensive, but can also easily trigger stress reactions, resulting in injuries to cattle and personnel. Therefore, it is necessary to develop a non-contact monitoring model.

[0003] Several renowned scholars and biometric technology companies, both domestically and internationally, have also begun research on contactless animal tracking and identification. Allen et al. abandoned traditional RFID technology and used cattle irises for identification. For their experiment, they collected 1,738 retinal images (taken from both eyes) from 869 cattle to identify their irises. Based on the uniqueness of the irises, they determined the cattle's type, achieving a maximum recognition rate of 98.30%. However, this method is difficult to obtain in the wild, and iris acquisition equipment is expensive, making it difficult to popularize. Xia et al. attempted to describe cattle facial features by combining sparse coding classification with principal component analysis and chi-square distance detection. However, this method only focuses on the front of the cattle's face and requires extensive initial data collection, making it difficult to implement in practical applications.

[0004] Kim et al. collected a facial dataset of 12 Japanese Wagyu cattle without obvious body markings. They first trained an associative memory neural network, then calculated its feature parameters and finally performed cow face recognition. This method demonstrated the feasibility of cow face recognition technology. Kumar et al. combined traditional feature extraction, feature dimensionality reduction, and classifier models to analyze and compare the effectiveness of these combinations in cow face recognition.

[0005] CN106778902A also discloses a method for identifying individual cows based on a convolutional neural network. The method uses an optical flow method or an inter-frame difference method to extract images of the cow's trunk, and uses a convolutional neural network to extract features, combined with the cow's texture features to achieve effective identification of individual cows.

[0006] Chen Juanjuan et al. developed an improved bag-of-features (BOF)-based algorithm for cow individual recognition. Using self-generated cow facial data, this method first extracts cow facial features using an optimized histogram of oriented gradients (HOC), followed by classification using the spatial pyramid matching principle (SPM), achieving a final recognition accuracy of 95.3%. Zhao Kaixuan et al. introduced deep learning to individual cattle recognition. They first extracted cow torso images and then fed them into a convolutional neural network to accurately identify individual cows. A test on 30 cows achieved a video segment recognition accuracy of 93.33%. Zhu Minling et al. proposed an algorithm and model for cow face recognition and detection based on a CNN, combining ResNet and SVM. Experimental results achieved an accuracy of over 95.1%. Xu et al. proposed a novel cow face recognition framework, CattleFaceNet, that integrates a lightweight RetinaFace-mobilenet with the additive angular margin loss (ArcFace), achieving a cow face recognition accuracy of 91.3%. Li et al. designed a lightweight neural network with six convolutional layers for cow face detection. Experimental results on 103 cows showed that the proposed model achieved an accuracy of 98.37%. This demonstrates the high application value of deep learning algorithms in individual cow detection.

[0007] In another invention, patent number CN202110952783.3, titled "An Intelligent Aisle Device for Acquiring Body Parameters and Identifying Movement Health of Beef Cattle," uses an infrared camera to capture three-dimensional infrared images of cattle. However, using only three-dimensional infrared images is difficult to locate and identify the cattle's body, and can only provide relatively general health monitoring. Another invention, patent number CN202110952774.4, titled "An Automatic Health Monitoring System for Beef Cattle," uses a camera to locate the cattle's face. However, a drive unit drives the camera and infrared temperature probe up and down in front of the cattle to capture the cattle's face and identify the cattle's body. However, in actual use, it has been found that after entering the aisle, cattle are easily startled by the camera moving down to capture objects, causing them to rush forward, potentially damaging the camera and infrared temperature probe on the drive unit. In particular, the infrared temperature probe cannot accurately locate the cattle's forehead and often measures the temperature of other parts of the face, making it unsuitable for practical applications.

[0008] Therefore, how to design a method that can meet practical applications and use cow side face images to identify the individual identity of cows has become a technical problem that needs to be solved urgently. Summary of the Invention

[0009] The purpose of the present invention is to solve the defect that the cattle health identification in the prior art cannot meet the actual use needs, and to provide a fence aisle type cattle health data extraction device and an intelligent extraction method thereof to solve the above problem.

[0010] In order to achieve the above object, the technical solution of the present invention is as follows:

[0011] A fence aisle type cattle health data extraction device includes a fence aisle, an electronic weight scale is installed at the bottom of the fence aisle, and cameras A, B, C, D, E and F are installed on the fence aisle, all of which are connected to a server. The cameras A, B and C are all visible light image acquisition cameras, and the cameras D, E and F are all visible light and infrared image acquisition cameras.

[0012] The camera A is arranged at the top middle part of the fence aisle and its camera range is vertically downward, the camera B is arranged on the left side of the fence aisle and its camera range is toward the camera C, the camera C is arranged on the right side of the fence aisle and its camera range is toward the camera B; the camera D is arranged at the top front part of the fence aisle and its camera range is 45° to the rear and lower part, the camera E is arranged at the lower left front part of the fence aisle and its camera range has an angle of 45° with the horizontal plane and the front view section, and the camera F is arranged at the lower right front part of the fence aisle and its camera range has an angle of 45° with the horizontal plane and the front view section.

[0013] An intelligent extraction method for a fence-aisle type cattle health data extraction device comprises the following steps:

[0014] Acquiring the status of cattle entering and exiting the fence: Real-time monitoring of electronic scale data. When the scale data exceeds the threshold, the time when the cattle enters the aisle is recorded as t1; when the scale data falls below the threshold, the time when the cattle leaves the aisle is recorded as t2. The cattle weight and image data collected by cameras A, B, C, D, E, and F between t1 and t2 are saved and recorded as the current cattle health data label;

[0015] Synthesis processing of the cow's frontal image: Using the visible light images of the cow's face from three angles collected by cameras D, E, and F, a frontal, non-tilted visible light image of the cow's face is synthesized;

[0016] Cow identity recognition: Based on the synthesized frontal, non-tilted cow face image, the yolov5 object detection model is used to locate the cow's face, and another yolov5 network is used to locate the cow's eyes, nose, ears, and mouth. The following eight indicators are calculated: the number of cow eye pixels, the average grayscale value of the cow eye area, the distance to the center of the cow eye, the number of cow nose pixels, the average grayscale value of the cow nose area, the number of cow mouth pixels, the average grayscale value of the cow mouth area, and the distance to the center of the cow ear. These indicators are then matched with the cow face images in the database to determine the cow's identity.

[0017] Calculation of cattle body parameters: Obtain images captured by cameras A, B, and C when the cattle pass through the fence. Use image segmentation to separate the cattle from the background. Calculate the length, width, and height of the cattle in each image. Take the maximum value as the length, width, and height of the cattle, and calculate the cattle body parameters. The expressions are as follows:

[0018] BL=max(bl1,bl2,...,bl n ),

[0019] BW=max(bw1, bw2,..., bw n ),

[0020] BH=max(bh1, bh2,..., bh n ),

[0021] Among them, BL, BW, and BH represent the measurement results of the body length, body width, and body height of the cattle respectively. n 、bw n 、bh n They represent the body length, body width, and body height obtained from the image analysis of each cow, and n is the number of images collected;

[0022] Measurement of cattle body temperature data:

[0023] Using the ITG network model, the infrared images collected by cameras D, E, and F at the same time are combined into a frontal infrared image of the cow's face. The cow's face and background are segmented using the u-net semantic segmentation model, and the length and width of the cow's face are calculated.

[0024] Align the synthesized infrared image of the cow's face with the optical cow's face image to directly locate the midpoints of the cow's eyes and nostrils in the cow's face image;

[0025] Take the line connecting the midpoints of the cow's eyes, 0.2 times the length of the cow's face, H, as the center of the circle and 0.4 times the width of the cow's face, W, as the radius, and use this as the cow's forehead position. Take the average value as the cow's body temperature in this frame image.

[0026] Analyze all infrared images of a cow's forehead, calculate the cow's body temperature in each frame according to the above method, and take the maximum value as the cow's body temperature measurement result. The formula is as follows:

[0027] BT=max(bt1, bt2, ... bt n ),

[0028] Among them, BT is the final measurement result of the cow's body temperature, bt n The body temperature of the cow calculated for each frame of the image, n is the number of image acquisitions;

[0029] Cattle weight measurement: After removing outliers from the measured cattle weight data, the data is fitted into a normal distribution curve, and the mean of the normal distribution curve is taken as the cattle weight data;

[0030] Storage of cattle health data: cattle identity, cattle body parameters, cattle temperature data, and cattle weight data are stored to form cattle health data.

[0031] The synthesis process of the cow front image comprises the following steps:

[0032] Build a dual-path fully convolutional network model;

[0033] Training of a two-way fully convolutional network model;

[0034] Construct an ITG network model for synthesizing cow face frontal images:

[0035] An image synthesis algorithm is designed based on a deep learning fully convolutional network model. The following is a synthesized frontal, non-tilted face image of a cow:

[0036] ITG(image_4,image_5,image_6)=image_front,

[0037] Among them, image_4, image_5 and image_6 are images collected by camera D, camera E and camera F respectively, ITG is a deep learning fully convolutional network model, image_front is a synthesized cow's front face image without tilt, and the visible light and infrared cow front images are synthesized;

[0038] The ITG network model is set to include three identical two-way fully convolutional network models, which process the cow face images taken from the upper side, left side, and right side respectively to obtain three feature maps, that is, generate three frontal feature maps of the cow face from different angles;

[0039] Set the frontal feature maps of the cow face at three different angles to be merged into a 9-channel feature map to be processed;

[0040] The 9-channel feature map to be processed is set to pass through 3 convolution layers and 2 pooling layers to extract and fuse the features of the three view images. It is then restored to its resolution through 1 deconvolution layer to obtain a synthesized frontal image of the cow's face.

[0041] The images captured by cameras D, E, and F are obtained and input into the ITG network model to output a synthesized real-time frontal image of the cow's face.

[0042] The identification of the cattle body comprises the following steps:

[0043] Based on the synthesized visible light image of the cow's face, the yoloV5 network model is used to detect the position of the cow's face in the synthesized image;

[0044] Another yoloV5 network model is used to locate the cow's eyes, nose, ears, and mouth in the image, and then match them with the cow face images in the database. The matching degree is calculated by calculating the following eight indicator features: the number of cow eye pixels, the average grayscale value of the cow eye area, the distance between the cow eye centers, the number of cow nose pixels, the average grayscale value of the cow nose area, the number of cow mouth pixels, the average grayscale value of the cow mouth area, and the distance between the cow ear centers;

[0045] Take the cow facial image for analysis, compare the above eight indicator features with the images of all cows in the database, and retain the results when all eight indicator features are above the threshold; analyze all cow facial images from time t1 to t2 according to the above steps, take the best result as the cow facial image, and identify the cow body from the database.

[0046] The construction of the dual-path fully convolutional network model includes the following steps:

[0047] The dual-path fully convolutional network model consists of two parts: a local feature extraction part and a global feature extraction part, which are arranged in parallel, and a fusion part. The input of the local feature extraction part is the labeled image, and the input of the global feature extraction part is the unlabeled image.

[0048] The local feature extraction part is set to include the input layer, convolution layer 1, convolution layer 2, convolution layer 3, convolution layer 4, deconvolution layer 1 and deconvolution layer 2 in sequence;

[0049] The global feature extraction part is set to include the input layer, convolution layer 5, convolution layer 6, convolution layer 7, convolution layer 8, and deconvolution layer 3 in sequence;

[0050] The fusion part is set to include an overlay layer, a convolution layer 9 and a convolution layer 10 in sequence; the overlay layer of the fusion part superimposes the local feature extraction result and the global feature extraction result to obtain a front feature map of the cow face at the current angle.

[0051] The training method of the dual-path fully convolutional network model is as follows:

[0052] The training dataset consists of no fewer than 3,000 images of cattle faces, including the front, upper profile, left side, and right side. The upper profile, left side, and right side images are used as input to the network. The trained results are compared with the frontal images. The error is calculated using a pixel-wise strategy, and then trained using the SGD gradient descent method. The initial learning rate is set to 0.1, the decay rate is set to 0.01, the inertia coefficient is set to 0.1, the maximum number of training times is set to 5,000, and the number of patients is set to 15.

[0053] After training, the model obtained will be verified using a prediction set containing no less than 1,000 sets of cow face images. When the error between the synthesized image and the true image is less than a threshold, the model training is completed.

[0054] Beneficial effects

[0055] The fence-aisle-type cattle health data extraction device and the intelligent extraction method thereof of the present invention use an image synthesis strategy to generate a frontal image of a cattle compared with the prior art, solving the problem that the frontal image acquisition device of aisle-type cattle health information acquisition equipment is easily damaged by cattle. By synthesizing non-contact cattle face body temperature, the temperature at the cattle forehead can be accurately measured; the stress response of the cattle is reduced, and damage to the cattle is avoided. The overall operation is simple and the practicality is high.

[0056] The present invention also has the following advantages:

[0057] 1. Based on deep learning technology, a technology was implemented to synthesize the frontal image of a cow using three cameras, solving the problem that the frontal image acquisition equipment of aisle-type cattle health information collection equipment is easily damaged by cattle;

[0058] 2. It realizes the contactless and rapid acquisition of beef cattle weight and body temperature data, improving the accuracy and efficiency of beef cattle health data collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a top view of the structure of the cattle health data extraction device involved in the present invention;

[0060] Figure 2 This is a method sequence framework diagram of the intelligent extraction method involved in the present invention;

[0061] Figure 3 This is a structural diagram of the dual-path fully convolutional network model involved in the present invention;

[0062] Figure 4 This is the composite image of the cow face involved in the present invention;

[0063] Among them, 1-Camera A, 2-Camera B, 3-Camera C, 4-Camera D, 5-Camera E, 6-Camera F, 7-Fence aisle, 8-Electronic weight scale. DETAILED DESCRIPTION

[0064] In order to provide a further understanding and appreciation of the structural features and effects achieved by the present invention, a detailed description is provided with reference to preferred embodiments and accompanying drawings as follows:

[0065] To automatically acquire beef cattle health data, this patent invention develops a fence-pass-type cattle health data extraction device and intelligent extraction method. The background technologies involved include animal husbandry, machine vision, deep learning, and sensor technology. Animal husbandry technology includes cattle access channel design technology; machine vision technology includes visible light and infrared camera image acquisition and image synthesis technology; deep learning technology includes target detection technologies such as convolutional neural networks; and sensor technology includes beef cattle weight measurement sensor technology.

[0066] like Figure 1 As shown, the fence aisle type cattle health data extraction device described in the present invention includes a fence aisle 7, and an electronic weight scale 8 is installed at the bottom of the fence aisle 7.

[0067] Cameras A1, B2, C3, D4, E5 and F6 are installed on the fence aisle 7, all of which are connected to the server. Cameras A1, B2 and C3 are all visible light image acquisition cameras, and cameras D4, E5 and F6 are all visible light and infrared image acquisition cameras.

[0068] Cameras placed on the front of cattle are easily damaged by collisions, yet the cow's face is essential for collecting health data. Therefore, a fenced aisle is used to restrict the cattle's movement, forcing them to move only forward. Cameras are then placed on the sides and tops of the cattle. An algorithm is then used to process these images to obtain facial images.

[0069] Camera A1 is positioned at the top center of the fence aisle 7, with its camera range facing vertically downward, capturing a top-down image of the cow's midsection. Camera B2 is positioned on the left side of the fence aisle 7, with its camera range facing camera C3. Camera C3 is positioned on the right side of the fence aisle 7, with its camera range facing camera B2, capturing images of the cow's left and right abdomens, respectively.

[0070] The camera D4 is positioned at the front top of the fence aisle 7, with its camera range extending 45° from the rear and lower sides, capturing an image of the top of the cow's head from the upper side. The camera E5 is positioned at the lower left front of the fence aisle 7, with its camera range extending at an angle of 45° to both the horizontal plane and the front view section, i.e., at an angle of 45° to both the ground (horizontal plane) and the longitudinal plane (vertical plane) based on the front end of the cow's face, capturing an image of the left side of the cow's face, with the nose as the approximate dividing line. The camera F6 is positioned at the lower right front of the fence aisle 7, with its camera range extending at an angle of 45° to both the horizontal plane and the front view section, capturing an image of the right side of the cow's face, with the nose as the approximate dividing line.

[0071] like Figure 2As shown, an intelligent extraction method of a fence-aisle-type cattle health data extraction device is also provided, comprising the following steps:

[0072] The first step is to obtain the status of cattle entering and leaving the fence: real-time monitoring of the data of the electronic scale 8. When the data of the scale exceeds the threshold, the time when the cattle enters the aisle is recorded as t1; when the data of the scale falls below the threshold, the time when the cattle leaves the aisle is recorded as t2. The cattle weight between t1 and t2 and the image data collected by cameras A1, B2, C3, D4, E5 and F6 are saved and recorded as the health data label of the current cattle body.

[0073] The second step is the synthesis processing of the cow's front face image: using the cow's face images from three angles captured by cameras D4, E5 and F6, a frontal cow face image without tilt is synthesized.

[0074] The cow frontal face image synthesis technology used in this invention is feature-level image fusion. This technology extracts feature information from the cow's side face image, then analyzes, processes, and integrates this feature information to produce a fused cow frontal face image. The accuracy of target recognition in this synthesized cow frontal face image is significantly higher than that of the original image. Feature-level fusion compresses the image information before computer analysis and processing. Compared to pixel-level image fusion, this method consumes less memory and time, improving the real-time performance of image processing.

[0075] The synthesis process of the cow front image comprises the following steps:

[0076] (1) Figure 3 As shown in Table 1, a dual-path fully convolutional network model is constructed.

[0077] A1) A dual-path fully convolutional network model is constructed, comprising two parts: a local feature extraction part and a global feature extraction part, which are arranged in parallel, and a fusion part. The input of the local feature extraction part is a labeled image, while the input of the global feature extraction part is an unlabeled image.

[0078] A2) setting the local feature extraction part to include the input layer, convolution layer 1, convolution layer 2, convolution layer 3, convolution layer 4, deconvolution layer 1 and deconvolution layer 2 in sequence;

[0079] A3) setting the global feature extraction part to include the input layer, convolution layer 5, convolution layer 6, convolution layer 7, convolution layer 8, and deconvolution layer 3 in sequence;

[0080] A4) setting the fusion part to include an overlay layer, a convolution layer 9, and a convolution layer 10 in sequence; the overlay layer of the fusion part superimposes the local feature extraction results and the global feature extraction results to obtain a frontal feature map of the cow face at the current angle.

[0081] Table 1 Dual-path fully convolutional network model structure

[0082]

[0083]

[0084] The relevant code algorithm is as follows:

[0085] a. Local feature extraction part

[0086] o1=Conv2D(filters=1024,kernel_size=(5,5),padding="same",activation="relu")(model.output)

[0087] o1=Conv2D(filters=512,kernel_size=(3,3),padding="same",activation="relu")(o1)

[0088] o1=Conv2D(filters=512,kernel_size=(3,3),padding="same",activation="relu")(o1)

[0089] o1=Dropout(rate=0.5)(o1)

[0090] o1=Conv2D(filters=512,kernel_size=(3,3),padding="same",activation="relu")(o1)

[0091] o1=Conv2DTranspose(filters=512,kernel_size=(32,32), strides=(4,4), padding="valid", activation=None, name="score2")(o1)

[0092] o1=Conv2DTranspose(filters=512,kernel_size=(32,32), strides=(4,4), padding="valid", activation=None, name="score2")(o1)

[0093] b. Global feature extraction part

[0094] o2 = Conv2D(filters = 128, kernel_size = (5, 5), padding = "same", activation = "relu")(model.output)

[0095] o2 = Conv2D(filters = 128, kernel_size = (3, 3), padding = "same", activation = "relu")(o2)

[0096] o2 = Conv2D(filters = 128, kernel_size = (3, 3), padding = "same", activation = "relu")(o2)

[0097] o2 = Dropout(rate = 0.5)(o2)

[0098] o2 = Conv2D(filters = 64, kernel_size = (3, 3), padding = "same", activation = "relu")(o2)

[0099] o2 = Conv2DTranspose(filters = 2, kernel_size = (32, 32), strides = (4, 4), padding = "valid", activation = None, name = "score2")(o2)

[0100] c. Fusion layer

[0101] o3 = torch.stack(o1, o2)

[0102] o3 = Conv2D(filters = 8, kernel_size = (3, 3), padding = "same", activation = "relu")(o3)

[0103] o3 = Conv2D(filters = 3, kernel_size = (3, 3), padding = "same", activation = "relu")(o3)

[0104] The above code explanations are as follows:

[0105] Conv2D: convolution layer; Conv2DTranspose: deconvolution layer; Filter: number of feature maps; kernel_size: convolution kernel size; padding: edge processing mode; activation: activation function;

[0106] Dropout: randomly freeze a certain proportion of nodes; stack: channel (feature map) stacking.

[0107] (2) Training of the dual-path fully convolutional network model.

[0108] B1) The training dataset consists of no less than 3,000 sets of cow facial images, including the front, upper profile, left side, and right side. The upper profile, left side, and right side images are used as input to the network. The trained results are compared with the front images. The error is calculated using a pixel-wise strategy, and then trained using the SGD gradient descent method. The learning rate is initially set to 0.1, the decay rate is set to 0.01, the inertia coefficient is set to 0.1, the maximum number of training times is set to 5,000, and the number of patients is set to 15.

[0109] B2) After training, the resulting model will be validated using a prediction set containing no less than 1,000 sets of cow face images. The model is considered trained when the error between the synthesized image and the ground truth image is less than a threshold.

[0110] (3) Constructing an ITG network model for synthesizing cow face frontal images:

[0111] An image synthesis algorithm is designed based on a deep learning fully convolutional network model. The following is a synthesized frontal, non-tilted face image of a cow:

[0112] ITG(image_4,image_5,image_6)=image_front,

[0113] Among them, image_4, image_5 and image_6 are images captured by camera D4, camera E5 and camera F6 respectively, ITG is a deep learning fully convolutional network model, image_front is a synthesized frontal non-tilted facial image of a cow, and visible light and infrared frontal images of a cow are synthesized.

[0114] C1) The ITG network model is configured to include three identical dual-path fully convolutional network models, which process the cow face images taken from the upper side, left side, and right side, respectively, to obtain three feature maps, i.e., generate three frontal feature maps of the cow face from different angles;

[0115] C2) merging the three cow face frontal feature maps at different angles into a 9-channel feature map to be processed;

[0116] The corresponding pytorch code is as follows:

[0117] p12 = torch.stack(p1,p2)

[0118] p123 = torch.stack(p12,p3)

[0119] p1, p2, and p3 represent the synthesis results of cow face images from three angles respectively, p123 represents the synthesis result, and stack represents the feature map stacking.

[0120] C3) Set the 9-channel feature map to be processed to pass through 3 convolution layers (the parameters of the multi-view image feature fusion convolution layer are shown in Table 2) and 2 pooling layers to extract and fuse the features of the three view images, and then pass it through 1 deconvolution layer (the parameters of the deconvolution layer are shown in Table 3) to restore the resolution, and obtain a synthesized cow face frontal image;

[0121] Table 2 Multi-view image feature fusion convolutional layer parameters

[0122] Kernel size Number of feature maps Convolutional layer 1 3*3 128 Convolutional layer 2 3*3 64 Convolutional layer 3 3*3 2

[0123] The Pytorch code is as follows:

[0124] s1=Conv2D(filters=128,kernel_size=(3,3),padding="same",activation="relu")(model.output)

[0125] s1=Conv2D(filters=64,kernel_size=(3,3),padding="same",activation="relu")(s1)

[0126] s1)=Conv2D(filters=8,kernel_size=(3,3),padding="same",activation="relu")(s1))

[0127] Table 3 Deconvolution layer parameters

[0128] Kernel size Feature Map Deconvolution layer 1 32*32 2

[0129] The Pytorch code is as follows:

[0130] T1=Conv2DTranspose(filters=2, kernel_size=(32,32), strides=(4,4), padding="valid", activation=None, name="score2") (model.output).

[0131] (4) Obtain the images captured by camera D4, camera E5 and camera F6, input them into the ITG network model, and output the synthesized real-time cow face frontal image.

[0132] The third step is to identify the cow’s body.

[0133] Based on the synthesized frontal, non-tilted cow face image, the traditional YOLOv5 target detection model is used to locate the cow's face, and another YOLOv5 network is used to locate the cow's eyes, nose, ears and mouth. The following eight indicators are calculated: the number of cow eye pixels, the average grayscale value of the cow eye area, the distance to the center of the cow eye, the number of cow nose pixels, the average grayscale value of the cow nose area, the number of cow mouth pixels, the average grayscale value of the cow mouth area, and the distance to the center of the cow ear. These indicators are then matched with the cow face images in the database to determine the cow's identity.

[0134] (1) Based on the synthesized visible light image of the cow's face, the yoloV5 network model is used to detect the position of the cow's face in the synthesized image.

[0135] (2) Use another yoloV5 network model to locate the positions of the cow's eyes, nose, ears and mouth in the image, and then match them with the cow face images in the database. The matching degree is calculated by calculating the following eight indicator features: the number of cow eye pixels, the average grayscale value of the cow eye area, the distance between the cow eye centers, the number of cow nose pixels, the average grayscale value of the cow nose area, the number of cow mouth pixels, the average grayscale value of the cow mouth area, and the distance between the cow ears centers;

[0136] Take the cow facial image for analysis, compare the above eight indicator features with the images of all cows in the database, and retain the results when all eight indicator features are above the threshold; analyze all cow facial images from time t1 to t2 according to the above steps, take the best result as the cow facial image, and identify the cow body from the database.

[0137] The fourth step is to calculate the cattle body parameters.

[0138] Obtain images captured by cameras A1, B2, and C3 when a cow passes through a fence. Use image segmentation to separate the cow from the background. Calculate the length, width, and height of the cow in each image. Take the maximum value as the length, width, and height of the cow. Calculate the cow's body shape parameters, which are expressed as follows:

[0139] BL=max(bl1,bl2,...,bl n ),

[0140] BW=max(bw1, bw2,..., bw n ),

[0141] BH=max(bh1, bh2,..., bh n ),

[0142] Among them, BL, BW, and BH represent the measurement results of the body length, body width, and body height of the cattle respectively. n 、bw n 、bh n They represent the body length, body width, and body height obtained from the image analysis of each cow, and n is the number of images collected.

[0143] Step 5: Measure the cow’s body temperature data.

[0144] In actual applications, it was found that because cattle's head movement is not restricted in the fenced aisle, it is impossible to use body temperature sensing equipment to obtain the temperature of the cattle's forehead, making it impossible to accurately measure body temperature. This method cleverly utilizes a composite image of the cow's face and uses clever processing between visible light and infrared images to obtain cattle body temperature data using infrared images, thus accurately measuring cattle body temperature data.

[0145] (1) Using the ITG network model, the infrared images collected by cameras D4, E5, and F6 at the same time are synthesized into a frontal infrared image of the cow's face. The cow's face and background are segmented using the u-net semantic segmentation model, and the length and width of the cow's face are calculated.

[0146] Here, the characteristics of cameras D4, E5, and F6 as visible light and infrared image acquisition cameras are utilized, and the same ITG network model (without increasing the complexity of industrial data processing) is used to perform synthesis processing of different images. In the synthesis processing step of the cow's front image, the ITG network model is used to input the visible light image to obtain the synthesized visible light cow face front image; then the yoloV5 network model is used to locate the cow's face feature points. In the step of measuring the cow's body temperature data, the ITG network model is used to input the infrared image to obtain the infrared cow face front image, and then the visible light cow face front image and the infrared cow face front image are aligned and processed to directly locate the various key positions in the cow's face (cow's eyes, the midpoint of the cow's nostrils, the cow's forehead, etc.). This solves the problem in the prior art that it is impossible to use temperature measuring equipment to accurately obtain the temperature of the exact position of the cow's face.

[0147] (2) Align the synthesized infrared image of the cow face with the optical cow face image to directly locate the midpoints of the cow's eyes and nostrils in the cow face image;

[0148] Take 0.2 times the length of the cow's face above the line connecting the midpoints of the cow's eyes as the center of the circle and 0.4 times the width of the cow's face as the radius, and make a circle. This is used as the position of the cow's forehead; the average value is taken as the cow's body temperature in this frame image.

[0149] (3) Analyze all infrared images of a cow's forehead, calculate the cow's body temperature in each frame according to the above method, and take the maximum value as the cow's body temperature measurement result. The formula is as follows:

[0150] BT=max(bt1, bt2, ... bt n ),

[0151] Among them, BT is the final measurement result of the cow's body temperature, bt n The body temperature of the cow calculated for each frame of the image, n is the number of image acquisitions;

[0152] Step 6: Measuring cattle weight: After removing outliers from the measured cattle weight data, fit it into a normal distribution curve, and take the mean of the normal distribution curve as the cattle weight data.

[0153] The seventh step is the storage of cattle health data: the cattle identity, cattle body parameters, cattle temperature data, and cattle weight data are stored to form cattle health data.

[0154] This device uses industrial cameras to collect image information of beef cattle, infrared cameras to collect body temperature information, and electronic scales 8 to measure weight information. All data are transmitted to the server via the gigabit network, and the body parameters, weight, and temperature data of the cattle are obtained through background algorithm analysis. Among them, the smart aisle is 2.6m long, 1.4m wide, and 2.0m high. Partitions are set at 0.2m on both sides of the width to protect the camera. A steel cavity is welded on the top of the smart aisle to facilitate the lifting and transportation of the device. Figure 4 As shown, the method of the present invention is used to synthesize and locate the cow's facial image, and then match it with the cow's health data to monitor the cow's body condition.

[0155] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent extraction method for a fence aisle type cattle health data extraction device, the device comprising a fence aisle (7), an electronic weight scale (8) installed at the bottom of the fence aisle (7), and cameras A (1), B (2), C (3), D (4), E (5) and F (6) all connected to a server installed on the fence aisle (7), wherein the cameras A (1), B (2) and C (3) are all visible light image acquisition cameras, and the cameras D (4), E (5) and F (6) are all visible light and infrared image acquisition cameras; the camera A (1) is arranged at the bottom of the fence aisle. The camera B (2) is arranged at the left side of the fence aisle (7) and its camera range is directed vertically downwards. The camera C (3) is arranged at the right side of the fence aisle (7) and its camera range is directed towards the camera B (2). The camera D (4) is arranged at the front top of the fence aisle (7) and its camera range is 45 degrees from the rear and downwards. The camera E (5) is arranged at the lower left front of the fence aisle (7) and its camera range has an angle of 45 degrees with the horizontal plane and the front view section. The camera F (6) is arranged at the lower right front of the fence aisle (7) and its camera range has an angle of 45 degrees with the horizontal plane and the front view section. The camera is characterized in that: An intelligent extraction method for a fence-aisle type cattle health data extraction device comprises the following steps: 11) Obtaining the data of cattle entering and leaving the fence aisle: Real-time monitoring of the data of the electronic scale (8). When the data of the electronic scale exceeds the threshold, the time when the cattle enters the fence aisle is recorded as t1; when the data of the electronic scale is lower than the threshold, the time when the cattle leaves the fence aisle is recorded as t2. The cattle weight between t1 and t2 and the image data collected by camera A (1), camera B (2), camera C (3), camera D (4), camera E (5) and camera F (6) are saved and recorded as the health data label of the current cattle body; 12) Synthesis of cow frontal image: Using the three-angle visible light images of the cow face collected by camera D (4), camera E (5) and camera F (6), synthesize a frontal non-tilted visible light image of the cow face; 13) Cow Identity Recognition: Based on a synthesized frontal, unobstructed cow face visible light image, the YOLOv5 object detection model is used to locate the cow's face. Another YOLOv5 network is used to locate the cow's eyes, nose, ears, and mouth. The following eight metrics are calculated: number of eye pixels, average grayscale value of the eye region, distance to the center of the eye, number of nose pixels, average grayscale value of the nose region, number of mouth pixels, average grayscale value of the mouth region, and distance to the center of the ear. These metrics are then matched with cow face images in the database to determine the cow's identity. 14) Calculation of cattle body parameters: Obtain images captured by camera A (1), camera B (2), and camera C (3) when the cattle pass through the fence aisle. Use image segmentation to separate the cattle from the background. Calculate the length, width, and height of the cattle in each image. Take the maximum value as the length, width, and height of the cattle, and calculate the cattle body parameters. The expressions are as follows: BL=max(bl1、bl2、……、bl n ), BW=max(bw1、bw2、……、bw n ) , BH=max(bh1、bh2、……、bh n ), Among them, BL, BW, and BH represent the measurement results of the body length, body width, and body height of the cattle respectively. n 、bw n 、bh n They represent the body length, body width, and body height obtained from the image analysis of each cow, and n is the number of images collected; 15) Measurement of cattle body temperature data: 151) Using the ITG network model, the infrared images collected by cameras D (4), E (5) and F (6) at the same time are synthesized into a frontal infrared image of the cow's face. The cow's face and background are segmented using the u-net semantic segmentation model, and the length and width of the cow's face are calculated. 152) Align the synthesized frontal infrared image of the cow’s face with the frontal visible light image of the cow’s face without tilt, and directly locate the midpoints of the cow’s eyes and nostrils in the cow’s face image; Take the line connecting the midpoints of the cow's eyes, 0.2 times the length of the cow's face, H, as the center of the circle and 0.4 times the width of the cow's face, W, as the radius, and use this as the cow's forehead position. Take the average value as the cow's body temperature in this frame image. 153) Analyze all infrared images of a cow's forehead, calculate the cow's body temperature in each frame according to the above method, and take the maximum value as the cow's body temperature measurement result. The formula is as follows: BT=max(bt1,bt2,……bt n ) , Among them, BT is the final measurement result of the cow's body temperature, bt n The body temperature of the cow calculated for each frame of the image, n is the number of image acquisitions; 16) Cattle weight measurement: After removing outliers from the measured cattle weight data, fit it into a normal distribution curve, and take the mean of the normal distribution curve as the cattle weight data; 17) Storage of cattle health data: cattle identity, cattle body parameters, cattle temperature data, and cattle weight data are stored to form cattle health data.

2. The intelligent extraction method of the fence-aisle type cattle health data extraction device according to claim 1 is characterized in that: The synthesis process of the cow front image comprises the following steps: 21) Construct a dual-path fully convolutional network model; 22) Training of a dual-path fully convolutional network model; 23) Constructing an ITG network model for synthesizing cow face frontal images: An image synthesis algorithm is designed based on a deep learning fully convolutional network model to synthesize a visible light image of a cow’s face without tilt from the front: ITG(image_4,image_5,image_6)=image_front, Among them, image_4, image_5 and image_6 are the images collected by camera D (4), camera E (5) and camera F (6) respectively, ITG is a deep learning fully convolutional network model, and image_front is a frontal non-tilted visible light image of the cow's face; 231) The ITG network model is set to include three identical two-way fully convolutional network models, which process the cow face images taken from the upper side, left side, and right side respectively to obtain three feature maps, that is, generate three frontal feature maps of the cow face at different angles; 232) Set the frontal feature maps of the cow face at three different angles to be merged into a 9-channel feature map to be processed; 233) Set the 9-channel feature map to be processed to pass through 3 convolution layers and 2 pooling layers to extract and fuse the features of the three perspective images, and then restore the resolution through 1 deconvolution layer to obtain a synthesized cow face frontal image; 24) Obtain the images captured by camera D (4), camera E (5) and camera F (6), input them into the ITG network model, and output a synthesized real-time cow face frontal image.

3. The intelligent extraction method of the fence-aisle type cattle health data extraction device according to claim 1, characterized in that: The identification of the cattle body comprises the following steps: 31) Based on the synthesized frontal non-tilted visible light image of the cow’s face, the YOLOv5 network model is used to detect the position of the cow’s face in the synthesized image; 32) Use another YOLOv5 network model to locate the cow's eyes, nose, ears, and mouth in the image, and then match it with the cow face images in the database. The matching degree is calculated by calculating the following eight indicators: the number of cow eye pixels, the average grayscale value of the cow eye area, the distance between the cow eye centers, the number of cow nose pixels, the average grayscale value of the cow nose area, the number of cow mouth pixels, the average grayscale value of the cow mouth area, and the distance between the cow ear centers; Take the cow facial image for analysis, compare the above eight indicator features with the images of all cows in the database, and retain the results when all eight indicator features are above the threshold; analyze all cow facial images from time t1 to t2 according to the above steps, take the best result as the cow facial image, and identify the cow body from the database.

4. The intelligent extraction method of the fence-aisle type cattle health data extraction device according to claim 2, characterized in that: The construction of the dual-path fully convolutional network model includes the following steps: 41) The dual-path fully convolutional network model consists of two parts: a local feature extraction part and a global feature extraction part, which are arranged in parallel, and a fusion part. The input of the local feature extraction part is the labeled image, and the input of the global feature extraction part is the unlabeled image; 42) The local feature extraction part is assumed to include the input layer, convolution layer 1, convolution layer 2, convolution layer 3, convolution layer 4, deconvolution layer 1 and deconvolution layer 2 in sequence; 43) The global feature extraction part is set to include the input layer, convolution layer 5, convolution layer 6, convolution layer 7, convolution layer 8, and deconvolution layer 3 in sequence; 44) The fusion part is set to include an overlay layer, a convolution layer 9, and a convolution layer 10 in sequence; the overlay layer of the fusion part superimposes the local feature extraction results and the global feature extraction results to obtain a frontal feature map of the cow face at the current angle.

5. The intelligent extraction method of the fence-aisle type cattle health data extraction device according to claim 2, characterized in that: The training method of the dual-path fully convolutional network model is as follows: 51) The training dataset should contain no less than 3,000 sets of cow facial images, including the front, upper side, left side, and right side. The upper side, left side, and right side images are used as input to the network. The trained results are compared with the frontal images. The error is calculated using a pixel-wise strategy, and then the SGD gradient descent method is used for training. The learning rate is initially set to 0.1, the decay rate is set to 0.01, the inertia coefficient is set to 0.1, the maximum number of training times is set to 5,000, and the number of patients is set to 15. 52) After training, the model will be validated using a prediction set containing no less than 1,000 sets of cow face images. When the error between the synthesized image and the true image is less than a threshold, the model training is complete.

Images