A method and system for identifying individual sheep in a sheep house based on YOLOv4
By applying a contactless flock individual identity recognition method based on the YOLOv4 neural network model in the sheep house, using multi-source video stream fusion technology and convolutional neural network for facial recognition, the problem of flock identity recognition in high-density and severe occlusion environments in the sheep house is solved, and efficient and accurate identity recognition effect is achieved.
Patent Information
- Application Number
- CN202210109354.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-28
- Publication Date
- 2025-03-28
- Estimated Expiration
- 2042-01-28
AI Technical Summary
The prior art is difficult to effectively realize the identification of individuals in the sheep flock, especially in the environment of high density and severe occlusion in the sheep house, the traditional contact method has problems of flag-waived and stress response.
The contactless flock individual identity recognition method based on the YOLOv4 neural network model is adopted, and the flock facial information is collected and processed through multi-source video stream fusion technology, a facial recognition neural network model is constructed, and a convolutional neural network is used for object detection and identity recognition.
It realizes efficient and accurate identification of individual identity of the sheep in the sheep house, avoids the problem of flag-waived and stress response of contact method, and has high stability and anti-interference ability.
Smart Images

Figure CN114529840B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of modern intelligent animal husbandry, and in particular to a method and system for identifying individual identities of sheep in a sheep house based on a YOLOv4 neural network model. Background Art
[0002] In recent years, animal husbandry at home and abroad is developing from traditional models to intelligent, precise and large-scale. Sheep, as an important livestock, is an important part of today's animal husbandry. The identification of individual sheep has been widely valued as an important aspect of large-scale and precise sheep farming. The main sheep identification method in the current sheep farming industry is the contact identification method, that is, the method of manual marking or ear tagging. The most commonly used sheep identification method in large-scale farms is the ear tag method based on wireless radio frequency. With the increase in the scale of farming, the phenomenon of ear tags falling off due to friction and biting between sheep has increased significantly. At the same time, ear tags will cause stress reactions in sheep, and inflammation of the tag hole will also have a certain impact on the health of sheep. In addition, the phenomenon of falling tags will increase the cost of sheep breeding for farmers. Therefore, in recent years, in the breeding industry, non-contact individual identification methods have gradually been valued by people. Among them, non-contact methods based on computer vision can effectively save labor costs and improve work efficiency. Especially in large-scale breeding environments, they can replace manual supervision of livestock growth around the clock, greatly reducing the labor costs of farmers and ranches, and improving breeding efficiency. Zhang Yu et al. published a patent "A method for identifying rumination behavior of dairy cows based on SSD convolutional neural network" to provide a method for identifying cattle, but compared with cattle, sheep herds are more difficult to collect data sets. The herds of cattle have a small living density, a small range of activities, and less mutual occlusion, while the breeding density in the sheep house is high, the activity volume is large, and the mutual occlusion phenomenon is serious; Liu Bing et al. published a patent "A chicken farm chicken identification algorithm based on Yolov4" to propose a detection method targeting chickens in a farm, but did not identify each individual chicken.
[0003] Therefore, how to achieve individual identification of sheep in a non-contact manner has become an urgent problem to be solved. Summary of the invention
[0004] In order to solve the above technical problems, the present invention provides a method and system for identifying individual identities of sheep in a sheep house based on a YOLOv4 neural network model.
[0005] The technical solution of the present invention is: a method for identifying individual sheep in a sheep house based on a YOLOv4 neural network model, comprising:
[0006] Step S1: Collect facial information of each sheep in the flock, pre-process the facial images, annotate the sheep faces with annotation boxes, obtain a data set, and divide it into a training set and a test set;
[0007] Step S2: construct a sheep face recognition neural network model based on YOLOv4, wherein the sheep face recognition neural network model includes: Input, Backbone, Neck and Head: Backbone is used as a backbone feature extraction network, CSPnet is added to Darknet53 to construct CSPDarknet53, and Mish activation function is used; Neck is used as an enhanced feature extraction network, SPP and PANet are used to extract context features and parameter aggregation respectively; Head recalculates the anchor box size based on the annotation box using K means++ algorithm, predicts based on the output features of Neck, and calculates the final target detection box after non-maximum suppression of the generated prediction box;
[0008] Step S3: construct a loss function, use sheep face data for pre-training, use the model parameters obtained after pre-training as the initial parameters of the sheep face recognition neural network model, use the training set to train it, and obtain a trained sheep face recognition neural network model; finally, input the test set into the trained sheep face recognition neural network model to evaluate its performance.
[0009] Compared with the prior art, the present invention has the following advantages:
[0010] The present invention discloses a method for identifying individual sheep in a sheep shed based on a YOLOv4 neural network model. The multi-source video stream fusion technology can effectively solve the problem of mutual occlusion between sheep in the sheep shed during data set collection. The method provided by the present invention is a non-contact identification method that will not cause the sheep to have only a stress response. Unlike traditional feature point extraction and classification methods, the target detection method based on a convolutional neural network provided by the present invention does not require manual feature extraction. It can obtain more essential and richer essential features of the target through repeated feature extraction and stacking. Moreover, the system scale required by the method provided by the present invention is small and the structure is relatively simple. Therefore, it has high stability and strong anti-interference ability, strong versatility, and has a strong recognition accuracy for the deformation and occlusion of the detected target. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 It is a flow chart of a method for identifying individual sheep in a sheep house based on a YOLOv4 neural network model in an embodiment of the present invention;
[0012] Figure 2This is a schematic diagram of the CSPDarknet53 module structure in an embodiment of the present invention;
[0013] Figure 3 This is a schematic diagram of the structure of a CSP in an embodiment of the present invention;
[0014] Figure 4 It is the Mish function image in the embodiment of the present invention;
[0015] Figure 5 Schematic diagram of the structure of a neural network model for sheep face recognition based on YOLOv4 in an embodiment of the present invention;
[0016] Figure 6 It is a schematic diagram of the overall process of the sheep face recognition neural network model based on YOLOv4 in an embodiment of the present invention;
[0017] Figure 7 It is the mAP index of the test result in the embodiment of the present invention;
[0018] Figure 8 The present invention is a structural block diagram of a sheep flock individual identity recognition system in a sheep house based on a YOLOv4 neural network model in an embodiment of the present invention. DETAILED DESCRIPTION
[0019] The present invention provides a method for identifying individual sheep in a sheep house based on a YOLOv4 neural network model. The method adopts a non-contact identification method with low cost, high accuracy, safety and effectiveness, thereby avoiding the problems of label loss and sheep being easily stressed in the existing contact technology. At the same time, the method also solves the problem that sheep are easily blocked and cannot be accurately identified in the existing non-contact technology.
[0020] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific implementations and in conjunction with the accompanying drawings.
[0021] Embodiment 1
[0022] like Figure 1 As shown, a method for identifying individual sheep in a sheep house based on a YOLOv4 neural network model provided by an embodiment of the present invention includes the following steps:
[0023] Step S1: Collect facial information of each sheep in the flock, pre-process the facial images, annotate the sheep faces with annotation boxes, obtain a data set, and divide it into a training set and a test set;
[0024] Step S2: construct a sheep face recognition neural network model based on YOLOv4, wherein the sheep face recognition neural network model includes: Input, Backbone, Neck and Head: Backbone is used as the backbone feature extraction network, CSPnet is added to Darknet53 to construct CSPDarknet53, and Mish activation function is used; Neck is used as an enhanced feature extraction network, SPP and PANet are used to extract context features and parameter aggregation respectively; Head uses Kmeans++ algorithm to recalculate the anchor box size according to the annotation box, predicts based on the output features of Neck, and calculates the final target detection box after non-maximum suppression of the generated prediction box;
[0025] Step S3: construct a loss function, use sheep face data for pre-training, use the model parameters obtained after pre-training as the initial parameters of the sheep face recognition neural network model, use the training set to train it, and obtain a trained sheep face recognition neural network model; finally, input the test set into the trained sheep face recognition neural network model to evaluate its performance.
[0026] In one embodiment, the above step S1: collecting facial information of each sheep in the flock, preprocessing the facial pictures, manually annotating the sheep faces with annotation boxes, obtaining a data set, and dividing it into a training set and a test set, specifically includes:
[0027] Step S11: collecting video information of sheep in the flock at different time periods;
[0028] First, the sheep are numbered so that the data sets can be classified later. Second, multiple cameras are placed 10-20 cm above the height of the sheep and 1-2 m outside the fence. The number of sheep in the camera field of view is preferably controlled within 20. 24-hour videos of the flock are taken at different times such as sunny and cloudy days, and the video files are transferred to the server using network cables, WIFI technology, and other methods.
[0029] Step S12: intercepting the video information according to a preset frame rate, obtaining sheep pictures, extracting and matching feature points, fusing the facial data of each sheep, and obtaining pictures containing the complete facial data of each sheep;
[0030] For the captured video files, use Adobe Premiere Pro 2019 software to cut out invalid data in the video captured by the camera and save it. Then use the OpenCV open source software library in python to extract video frames from the saved valid video, set the frame cut frequency to 30, and save it in JPG format.
[0031] Using multi-source video stream fusion technology, the video frames are extracted to obtain pictures for feature point extraction. According to the placement of each camera, the coordinate transformation and unification between multiple groups of cameras are performed using the principle of stereo vision. Combined with the time correlation characteristics of the shot video data, the feature point matching of multi-source video data is achieved, and the pictures containing the facial information of the sheep are effectively fused. Effective fusion of multi-source video data is achieved, and finally a picture containing the complete facial information of the sheep is obtained.
[0032] According to the sheep's identity tag, all pictures of the same sheep are stored in the same folder, and the folder name is the same as the sheep tag name. If there are multiple sheep in the picture, the pictures are stored in multiple folders, and then the data set is cleaned to remove blurry and overly similar pictures in the folder, and 600 pictures of each sheep are retained.
[0033] Step S13: Perform translation and rotation augmentation operations on the pictures of the complete facial data of each sheep in the folder, and then annotate the sheep faces to obtain a data set, which is divided into a training set and a test set according to a preset ratio.
[0034] Use the LabelImg visual image calibration tool to label the sheep faces in the photos in the folder, obtain the annotation boxes, and generate a data set in voc format. If there are many categories, the voc format can be converted into a coco data set format to obtain the final data, which is divided into a training set and a test set in a 4:1 ratio.
[0035] In one embodiment, the above step S2: constructs a sheep face recognition neural network model based on YOLOv4, wherein the sheep face recognition neural network model includes: Input, Backbone, Neck and Head: Backbone is used as a backbone feature extraction network, CSPnet is added to Darknet53 to construct CSPDarknet53, and Mish activation function is used; Neck is used as an enhanced feature extraction network, SPP and PANet are used to extract context features and parameter aggregation respectively; Head recalculates the anchor box size based on the annotation box using the K means++ algorithm, predicts based on the output features of Neck, and calculates the final target detection box after non-maximum suppression of the generated prediction box, specifically including:
[0036] Step S21: construct Backbone as the backbone feature extraction network, and add CSPnet to Darknet53 to construct CSPDarknet53, where CSPDarknet53 includes: 1 convolution layer and 5 Resblock_body modules, where the Resblock_body module includes: 2 convolution blocks, standardization and Mish activation function;
[0037] Backbone is constructed as the backbone feature extraction network. CSPnet is added to Darknet53 to construct CSPDarknet53. Its structure is as follows Figure 2 As shown in the figure, the network has a total of 53 layers. The input is a 416×416×3 image. After the first layer of convolution, the feature layer becomes 416×416×32. Then there are 5 large convolution blocks consisting of a series of convolutions, activation functions, and standardization. Resblock_body will continuously downsample the image, and the width and height of the image will be continuously compressed, and the number of channels will be continuously expanded. Among them, the first Resblock_body contains a ResBlock residual network, and the input will become a 208×208×64 feature layer. The second Resblock_body contains two ResBlock residual networks, and the input will become a 104×104×128 feature layer. The third Resblock_body contains eight ResBlock residual networks, and the input will become a 52×52×256 feature layer. The fourth Resblock_body contains eight ResBlock residual networks, and the input will become a 26×26×512 feature layer. The fifth Resblock_body contains four ResBlock residual networks, and the input becomes a 13×13×1024 feature layer.
[0038] Resblock_body is the core of building the CSPDarknet53 network, where the structure of CSP is as follows Figure 3As shown in the figure, part2 is the main part, which extracts features from the input after stacking a series of residual networks. On the other side, part1 is similar to a large residual edge structure, which can skip part2 to reduce the repeated calculation of gradients, and directly stack the input with the output after a small amount of processing. Resblock_body is essentially a large convolution block composed of one downsampling and multiple residual networks. First, use zero padding and a convolution block with a step size of 2x2 to compress the height and width, and then establish a large residual edge. This large residual edge can bypass many residual structures. The main part will be looped, and the residual structure is inside the loop. For the entire CSPdarknet structural block, it is a large residual block + multiple small residual blocks inside. The residual network based on Resblock_body is composed of Resblock. Resblock consists of convolution, standardization and activation function. Resblock has two convolution blocks. The first convolution fast convolution kernel size is 1×1, and the second convolution fast convolution kernel size is 3×3. When the residual network is forward, the residual block is composed of the input and the output after two convolutions. The activation function of Resblock is composed of the Mish activation function, and the formula is as follows:
[0039]
[0040] in, is the network parameter;
[0041] The image of the Mish function is as follows Figure 4 As shown. The embodiment of the present invention adopts the Mish activation function. Compared with ReLU, the Mish function is unbounded, avoiding saturation due to capping. In addition, the gradient of the Mish function is smoother, and the smooth activation function allows information to penetrate the neural network better, thereby obtaining better accuracy and generalization ability.
[0042] like Figure 5 As shown in the figure, after the input image passes through the Backbone feature extraction network, three effective feature layers are finally obtained, namely C3: (52×52×256) effective feature layer, C4: (26×26×512) effective feature layer, and C5: (13×13×1024) effective feature layer. These three effective feature layers are input into the Neck enhanced feature extraction network for subsequent operations.
[0043] Step S22: Construct Neck as an enhanced feature extraction network, using SPP to increase the receptive field and separate the most important context and PANet to aggregate the parameters of SPP and different Backbone feature layers, and adjust based on the existing structure of YOLOv4. S1 is obtained by convolving and stacking the output of SPP, and then convolving and upsampling it with one output of PANet P3 for 5 times and stacking it to output P5. P5 and the other two outputs of PANet P1 and P3 are used as the input of the subsequent Head together;
[0044] First, the effective feature layer C5: (13×13×1024) output by the fifth resblock_body in the backbone feature extraction network is convolved three times, and then the pooling layers with pooling kernel sizes of 5×5, 9×9, and 13×13 in SPP are used for maximum pooling and then stacked. After obtaining the pooled output, it is convolved three times and input into the PANet structure.
[0045] PANet is an instance segmentation network that can extract features repeatedly. In the YOLOv4 network, the PANet structure is used for the final output S1 of the SPP structure and the third effective feature layer C3: (52×52×256) and the fourth effective feature layer C4: (26×26×512) output in the backbone feature extraction network.
[0046] The final output S1 in the SPP structure enters PANet for convolution + upsampling operations. The effective feature layer C4 of the fourth output of the backbone feature extraction network: (26×26×512) is convolved once and then input into PANet. The above two effective feature layers S1 and C4 are stacked in PANet and convolved five times with 1×1, 3×3, 1×1, 3×3, and 1×1 to obtain the effective feature layer P2; the effective feature layer C3 of the third output of the backbone feature extraction network: (52×52×256) is convolved once and input into PANet for five convolutions with 1×1, 3×3, 1×1, 3×3, and 1×1. At the same time, P2 is convolved + upsampled, and then the two effective feature layers are stacked to obtain the effective feature layer P1. Two operations are performed on P1, namely, output to YOLO Head and downsampling. After downsampling, it is convolved with P2 five times with 1×1, 3×3, 1×1, 3×3, and 1×1 and stacked to obtain P3, which is output to Head. P3 is downsampled, and then convolved with S1 five times (1×1, 3×3, 1×1, 3×3, and 1×1) and stacked to obtain P4, which is output to the Head.
[0047] Step S23: Construct the Head, including the integration of several 1×1 convolutions and 3×3 convolutions, and recalculate multiple anchor box sizes of different sizes using the K means++ algorithm according to the manually labeled boxes, and assign them to the feature layers corresponding to the inputs P1, P3, and P5 of the Head from small to large. Each feature layer is assigned 3 anchor boxes, and the predicted box is output. After non-maximum suppression of the predicted box, the final target detection box is calculated.
[0048] YOLOv4 uses the same Head part as YOLOv3, which is an integration of a series of 1×1 convolutions and 3×3 convolutions. The function of 3x3 convolution is feature integration, and the function of 1x1 convolution is to adjust the number of channels.
[0049] like Figure 5 As shown in the YOLO Head section, the three feature layers of C3, C4 and C5 output by BackBone, i.e., the shapes of (52,52,256), (26,26,512) and (13,13,1024), are respectively connected to the YOLO Head after being strengthened with the feature extraction network Neck. The sizes of the obtained feature maps are 13×13, 26×26 and 52×52 respectively. If a Voc format dataset with 20 categories is used, the dimension is (5+20)×3=75; when there are more categories, the coco dataset with a total of 80 categories is used, and the dimension is (5+80)×3=255. The 5 represents 5 parameters: position x, y, width and height w, h and confidence.
[0050] When the detection frame obtained in the head is matched with the anchor frame, the size of the anchor frame is often selected by the K Means algorithm. The detection effect of objects of different sizes in some specific problems is not ideal, and it may cause false detection, missed detection or repeated detection. Therefore, the embodiment of the present invention uses the K means++ algorithm to retrain the size of the anchor frame.
[0051] In one embodiment, in the above step S23, the K means++ algorithm is used to recalculate the sizes of multiple anchor boxes of different sizes, and the feature layers corresponding to the inputs P1, P3, and P5 of the Head are assigned from small to large, and each feature layer is assigned 3 anchor boxes, and the prediction box is output. After the prediction box is subjected to non-maximum suppression, the final target detection box is calculated, which specifically includes:
[0052] Step S231: Randomly select a label box as the initial cluster center ;
[0053] Calculate the length and width of all annotation boxes, length = lower right corner horizontal coordinate - upper left corner horizontal coordinate, width = lower right corner vertical coordinate - upper left corner vertical coordinate, randomly select a annotation box as the initial cluster center ;
[0054] Step S232: For each sample point , respectively, The shortest distance between all existing cluster centers is expressed as Indicates that, then use Calculate the probability of each sample becoming the next cluster center, and finally select the next cluster center according to the roulette method;
[0055] Step S233: Repeat step S232 and stop when K cluster centers are obtained. The cluster center with the smallest distance will Classified into the category corresponding to the center;
[0056] Step S234: For each category ,use Recalculate the center of the category;
[0057] Step S235: looping steps S232 to S233, and stopping when the obtained cluster center no longer changes;
[0058] After the above steps, the anchor box is retrained, and the trained anchor box is used as the new prior box to improve the detection accuracy.
[0059] Step S236: According to the anchor frame, the output features of Neck are predicted, and the generated prediction frame is subjected to non-maximum suppression to obtain the final target detection frame.
[0060] In one embodiment, the above step S3: constructing a loss function, using sheep face data for pre-training, and using the model parameters obtained after pre-training as the initial parameters of the sheep face recognition neural network model, specifically includes:
[0061] Step S31: construct the mean square error loss MSE as shown in formula (1). During the model training process, in the confidence loss function and the classification loss function, the constructed mean square error loss MSE is used to replace the cross entropy loss BCE in the original YOLOv4 model to improve the detection and recognition ability of small targets in complex environments during the sheep face recognition neural network model training:
[0062] (1)
[0063] in, For real data, To fit the data, M is the number of samples;
[0064] Step S32: Use the sheep face data to pre-train the sheep face recognition neural network model so that it can better extract the essential features of the sheep face. Use the transfer learning idea during training in the YOLO model, use the pre-training results as the initial parameters for training the sheep face recognition model, and use the training set to train it to speed up the convergence of the model and obtain a trained sheep face recognition neural network model.
[0065] In order to accelerate the convergence speed of the model, the embodiment of the present invention first pre-trains the sheep face data. The present invention collects 29,000 facial images of 100 sheep of different sizes, genders, and fur colors, and performs 100 generations of training. After the model is pre-trained with a large number of sheep face pictures, the model at this time can better extract the essential features of the sheep face. Using the transfer learning idea during training in YOLO, the pre-training results are used as the initial parameters when training the sheep face recognition model. In this way, when training the sheep face recognition neural network model, the convergence speed of the model can be effectively accelerated and the model performance can be improved.
[0066] like Figure 6 As shown, it is a schematic diagram of the overall process of the sheep face recognition neural network model based on YOLOv4 in an embodiment of the present invention, from data set construction, model construction and training to final model verification.
[0067] The embodiment of the present invention collects 16 sheep, and 15,000 photos are used to form a test set. The sheep face recognition neural network model is tested and the results are as follows: Figure 7 As shown, the ordinate represents different sheep, the abscissa represents the mAP index, and the final average mAP value is 92.01%.
[0068] The present invention discloses a method for identifying individual sheep in a sheep shed based on a YOLOv4 neural network model. The multi-source video stream fusion technology can effectively solve the problem of mutual occlusion between sheep in the sheep shed during data set collection. The method provided by the present invention is a non-contact identification method that will not cause the sheep to have only a stress response. Unlike traditional feature point extraction and classification methods, the target detection method based on a convolutional neural network provided by the present invention does not require manual feature extraction. It can obtain more essential and richer essential features of the target through repeated feature extraction and stacking. Moreover, the system scale required by the method provided by the present invention is small and the structure is relatively simple. Therefore, it has high stability and strong anti-interference ability, strong versatility, and has a strong recognition accuracy for the deformation and occlusion of the detected target.
[0069] Embodiment 2
[0070] like Figure 8As shown, the embodiment of the present invention provides a sheep individual identification system in a sheep house based on a YOLOv4 neural network model, comprising the following modules:
[0071] The module 41 for acquiring data sets is used to collect facial information of each sheep in the flock, pre-process the facial images, annotate the sheep faces with annotation frames, obtain data sets, and divide them into training sets and test sets;
[0072] A model building module 42 is used to build a sheep face recognition neural network model based on YOLOv4, wherein the sheep face recognition neural network model includes: Input, Backbone, Neck and Head: Backbone is used as a backbone feature extraction network, CSPnet is added to Darknet53 to build CSPDarknet53, and Mish activation function is used; Neck is used as an enhanced feature extraction network, SPP and PANet are used to extract context features and parameter aggregation respectively; Head uses K means++ algorithm to recalculate the anchor box size according to the annotation box, predicts based on the output features of Neck, and calculates the final target detection box after non-maximum suppression of the generated prediction box;
[0073] The training model module 43 is used to build a loss function, use sheep face data for pre-training, use the model parameters obtained after pre-training as the initial parameters of the sheep face recognition neural network model, use the training set to train it, and obtain a trained sheep face recognition neural network model; finally, input the test set into the trained sheep face recognition neural network model to evaluate its performance.
[0074] The above embodiments are provided only for the purpose of describing the present invention, and are not intended to limit the scope of the present invention. The scope of the present invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the present invention should all be included within the scope of the present invention.
Claims
1. A method for identifying individual sheep in a sheep house based on a YOLOv4 neural network model, characterized in that: include: Step S1: Collect facial information of each sheep in the flock, pre-process the facial images, annotate the sheep faces with annotation boxes, obtain a data set, and divide it into a training set and a test set; Step S2: construct a sheep face recognition neural network model based on YOLOv4, wherein the sheep face recognition neural network model based on YOLOv4 includes: Input, Backbone, Neck and Head: Backbone is used as a backbone feature extraction network, CSPnet is added to Darknet53 to construct CSPDarknet53, and Mish activation function is used; Neck is used as an enhanced feature extraction network, SPP and PANet are used to extract context features and parameter aggregation respectively; Head recalculates the anchor box size based on the annotation box using the Kmeans++ algorithm, predicts based on the output features of Neck, and calculates the final target detection box after non-maximum suppression of the generated prediction box, specifically including: Step S21: construct Backbone as the backbone feature extraction network, and add CSPnet to Darknet53 to construct CSPDarknet53, wherein the CSPDarknet53 includes: 1 convolution layer and 5 Resblock_body modules, wherein the Resblock_body module includes: 2 convolution blocks, standardization and Mish activation function; Step S22: Neck is constructed as an enhanced feature extraction network, and SPP is used to increase the receptive field and separate the most important context and PANet is used to aggregate the parameters of SPP and different Backbone feature layers. The existing structure of YOLOv4 is adjusted, and the S1 obtained by convolving and stacking the output of SPP is convolved and upsampled with one output P3 of PANet for 5 times and stacked to output P5; P5 and the other two outputs P1 and P3 of PANet are used together as the input of the subsequent Head; Step S23: Constructing Head, including the integration of several 1×1 convolutions and 3×3 convolutions, and recalculating multiple anchor box sizes of different sizes using the Kmeans++ algorithm according to the manually labeled boxes, and assigning them to the feature layers corresponding to the inputs P1, P3, and P5 of Head from small to large, each of which is assigned 3 anchor boxes, outputting the predicted box, and calculating the final target detection box after non-maximum suppression of the predicted box, specifically including: Step S231: Randomly select one of the labeled boxes as the initial cluster center ; Step S232: For each sample point , respectively, The shortest distance between all existing cluster centers is expressed as Indicates that, then use Calculate the probability of each sample becoming the next cluster center, and finally select the next cluster center according to the roulette method; Step S233: Repeat step S232 and stop when K cluster centers are obtained. The cluster center with the smallest distance will Classified into the category corresponding to the center; Step S234: For each category ,use Recalculate the center of the category; Step S235: looping steps S232 to S233, and stopping when the obtained cluster center no longer changes; Step S236: predict the output features of Neck according to the anchor frame, and calculate the final target detection frame after non-maximum suppression of the generated prediction frame; Step S3: construct a loss function, use sheep face data for pre-training, use the model parameters obtained after pre-training as the initial parameters of the sheep face recognition neural network model, use the training set to train it, and obtain a trained sheep face recognition neural network model; finally, input the test set into the trained sheep face recognition neural network model to evaluate its performance.
2. The method for identifying individual sheep in a sheep house based on the YOLOv4 neural network model according to claim 1, characterized in that: The step S1: collecting facial information of each sheep in the flock, preprocessing the facial pictures, manually annotating the sheep faces with annotation frames, obtaining a data set, and dividing it into a training set and a test set, specifically includes: Step S11: collecting video information of sheep in the flock at different time periods; Step S12: intercepting the video information according to a preset frame rate, obtaining sheep pictures, extracting and matching feature points, fusing the facial data of each sheep, and obtaining pictures containing complete facial data of each sheep; Step S13: performing translation and rotation augmentation operations on the image of the complete facial data of each sheep, and then manually annotating the sheep faces to obtain a data set, which is divided into a training set and a test set according to a preset ratio.
3. The method for identifying individual sheep in a sheep house based on the YOLOv4 neural network model according to claim 1, characterized in that: In step S3, a loss function is constructed, sheep face data is used for pre-training, and the model parameters obtained after pre-training are used as the initial parameters of the sheep face recognition neural network model, specifically including: Step S31: construct the mean square error loss MSE as shown in formula (1). During the model training process, in the confidence loss function and the classification loss function, the constructed mean square error loss MSE is used to replace the cross entropy loss BCE in the original YOLOv4 model to improve the detection and recognition ability of the sheep face recognition neural network model in a complex environment: (1) in, For real data, To fit the data, M is the number of samples; Step S32: Use the sheep face data to pre-train the sheep face recognition neural network model so that it can better extract the essential features of the sheep face, use the transfer learning idea during training in the YOLO model, use the pre-training result as the initial parameter during training of the sheep face recognition model, use the training set to train it to speed up the convergence speed of the model, and obtain a trained sheep face recognition neural network model.
4. A sheep flock individual identity recognition system in a sheep house based on the YOLOv4 neural network model, characterized in that: Includes the following modules: The module for obtaining the data set collects the facial information of each sheep in the flock, pre-processes the facial images, annotates the sheep faces with annotation boxes, obtains the data set, and divides it into a training set and a test set; Construct a model module, and construct a sheep face recognition neural network model based on YOLOv4, wherein the sheep face recognition neural network model based on YOLOv4 includes: Input, Backbone, Neck and Head: Backbone is used as a backbone feature extraction network, CSPnet is added to Darknet53 to construct CSPDarknet53, and Mish activation function is used; Neck is used as an enhanced feature extraction network, and SPP and PANet are used to extract context features and parameter aggregation respectively; Head recalculates the anchor box size according to the annotation box using the Kmeans++ algorithm, predicts based on the output features of Neck, and calculates the final target detection box after non-maximum suppression of the generated prediction box, specifically including: Step S21: construct Backbone as the backbone feature extraction network, and add CSPnet to Darknet53 to construct CSPDarknet53, wherein the CSPDarknet53 includes: 1 convolution layer and 5 Resblock_body modules, wherein the Resblock_body module includes: 2 convolution blocks, standardization and Mish activation function; Step S22: Neck is constructed as an enhanced feature extraction network, and SPP is used to increase the receptive field and separate the most important context and PANet is used to aggregate the parameters of SPP and different Backbone feature layers. The existing structure of YOLOv4 is adjusted, and the S1 obtained by convolving and stacking the output of SPP is convolved and upsampled with one output P3 of PANet for 5 times and stacked to output P5; P5 and the other two outputs P1 and P3 of PANet are used together as the input of the subsequent Head; Step S23: Constructing Head, including the integration of several 1×1 convolutions and 3×3 convolutions, and recalculating multiple anchor box sizes of different sizes using the Kmeans++ algorithm according to the manually labeled boxes, and assigning them to the feature layers corresponding to the inputs P1, P3, and P5 of Head from small to large, each of which is assigned 3 anchor boxes, outputting the predicted box, and calculating the final target detection box after non-maximum suppression of the predicted box, specifically including: Step S231: Randomly select one of the labeled boxes as the initial cluster center ; Step S232: For each sample point , respectively, The shortest distance between all existing cluster centers is expressed as Indicates that, then use Calculate the probability of each sample becoming the next cluster center, and finally select the next cluster center according to the roulette method; Step S233: Repeat step S232 and stop when K cluster centers are obtained. The cluster center with the smallest distance will Classified into the category corresponding to the center; Step S234: For each category ,use Recalculate the center of the category; Step S235: looping steps S232 to S233, and stopping when the obtained cluster center no longer changes; Step S236: predict the output features of Neck according to the anchor frame, and calculate the final target detection frame after non-maximum suppression of the generated prediction frame; Training model module: used to construct a loss function, use sheep face data for pre-training, use the model parameters obtained after pre-training as the initial parameters of the sheep face recognition neural network model, use the training set to train it, and obtain a trained sheep face recognition neural network model; finally, input the test set into the trained sheep face recognition neural network model to evaluate its performance.
Citation Information
Patent Citations
Method for identifying complex behaviors in multi-source video
CN109977856A
Sheep face recognition system and method
CN113780207A