A livestock counting method based on frame filtering and target detection
By constructing overlap classification network and target detection network, using frame filtering and object detection algorithms, the problem of difficulty in accurately counting livestock in high-density environments and target occlusion scenarios in the prior art is solved, and higher counting accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202111144594.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-28
- Publication Date
- 2025-03-14
- Estimated Expiration
- 2041-09-28
AI Technical Summary
The prior art is difficult to accurately count livestock in high-density environments and target occlusion scenarios, and no methods such as object detection or frame filtering are used.
The livestock counting method based on frame filtering and target detection is adopted. By constructing an overlap classification network and a target detection network, the video frame is filtered and targeted to detect the video frames by using deep learning algorithms to achieve accurate counting of livestock.
It improves the accuracy and robustness of livestock counting, reduces the impact of livestock activities on counting results, and is suitable for complex scenarios.
Smart Images
Figure CN114882303B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of livestock target detection, and in particular relates to a livestock counting method based on frame filtering and target detection. Background Art
[0002] According to different counting methods, target counting methods can be divided into counting methods based on density estimation, counting methods based on clustering and counting methods based on regression.
[0003] Density estimation methods mainly include feature-based density estimation methods and texture-based density estimation methods. Feature-based quantity estimation methods generally estimate livestock density by the number of speed limit points of livestock and calculating the pixel ratio of pixels in the image. However, the overlap and shadow of livestock will bring certain deviations to livestock estimation. At the same time, some scholars have proposed to perform projection correction by assigning different weights to the foreground and background to further improve the accuracy of density estimation. However, in high-density environments, feature-based density estimation methods are still not good. In order to cope with this scenario, a counting method based on texture features has been proposed. The target (livestock) is regarded as a special texture, and the target texture features in the image are extracted, such as grayscale co-occurrence matrix, binary pattern features, etc.
[0004] The clustering-based target counting method combines the global optical flow method with Gaussian background modeling, acts on the input image, obtains the target's motion area, and uses the clustering method to segment the target's motion direction in the moving target area to obtain the motion direction information in the moving area. Finally, linear regression analysis is used to count the number of targets in each motion direction. The clustering-based counting method is widely used for crowd counting with clear motion directions. It can effectively avoid operations such as feature extraction of pixels in each frame of the video image, improve the efficiency and accuracy of target counting, and reduce computer resource overhead. However, the clustering counting method still does not deal with problems such as target occlusion and accurate counting in scenes with low target density.
[0005] Regression-based object counting methods are mostly used to detect the number of people in crowded scenes. Davies et al. first proposed a linear regression model to capture overall features from photos and estimate the number of people in a crowd. Chan et al. proposed a view normalization method and used multiple complementary features for feature extraction, which effectively improved the counting accuracy. At the same time, a counting method based on semi-supervised learning was also proposed, which can discover more unlabeled data through interactive learning. This method performs better when the video angle is complex and it is not easy to label.
[0006] At the same time, convolutional neural networks (CNNs) are also used in regression counting models. The neural network is trained with a given data set, and then the trained feature templates are used to identify unfamiliar scenes.
[0007] Although there are many counting methods today, including data methods based on density, clustering and regression, none of them have been applied to livestock counting, such as object detection or frame filtering. Summary of the invention
[0008] In view of the above-mentioned deficiencies in the prior art, the present invention provides a livestock counting method based on frame filtering and target detection, which solves the problem of detecting and counting livestock by using a deep learning frame filtering algorithm based on overlap and a target detection algorithm.
[0009] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:
[0010] The present invention provides a livestock counting method based on frame filtering and target detection, the method comprising the following steps:
[0011] S1. Obtain a livestock counting dataset, and evenly divide the livestock counting dataset into a training set, a validation set, and a test set;
[0012] S2, construct overlapping classification network;
[0013] S3, training the overlap classification network to obtain a trained overlap classification network;
[0014] S4, using the overlap to perform frame filtering on the trained overlap classification network to obtain a preset number of frames with the lowest overlap;
[0015] S5, build target detection network;
[0016] S6. Train the target detection network to obtain a trained target detection network;
[0017] S7. Use the trained target detection network to detect a preset number of frames with the lowest overlap, obtain the livestock position and livestock quantity in each frame, and perform weighted average counting fusion on the livestock quantity in each frame to obtain the measured livestock quantity.
[0018] The beneficial effects of the present invention are as follows: the livestock counting method based on frame filtering and target detection proposed in the present invention cleans the measured video at a practical overlap, selects frames with low livestock overlap for target detection and counting, and has higher practicality and robustness than traditional target detection methods, greatly improves the detection and counting accuracy, and effectively reduces the influence of livestock activities on counting results, and is also applicable to areas with high livestock activity intensity and complex scenes.
[0019] Furthermore, the specific steps of step S1 are as follows:
[0020] S11. Label the collected raw data of the pasture using labelme to obtain a livestock counting dataset including several pairs of pasture images and json annotations;
[0021] S12. Process the json annotation into five-tuples, and evenly divide the livestock counting dataset into training set, validation set and test set according to the target balance strategy, where the five-tuple includes X-axis coordinate, Y-axis coordinate, bounding box width, bounding box height and overlap.
[0022] The beneficial effects of adopting the above further scheme are: using labelme to label the original data of the ranch, that is, the video to be tested, and dividing the json annotations corresponding to the ranch pictures into five tuples, which are used to clean the video to be tested according to the overlap, obtain a preset number of frames that can be used for target detection, as well as target detection network training and actual livestock target detection and counting.
[0023] Furthermore, the step S2 constructs an overlap classification network including an input layer, five convolutional layers, two fully connected layers and an output layer which are arranged in sequence, wherein each convolutional layer is correspondingly followed by a pooling layer and a Relu activation function.
[0024] The beneficial effects of adopting the above further scheme are: the overlap classification network is a simple classification network, which extracts features through convolution operations, reduces the resolution and improves the receptive field of feature extraction through pooling operations, reduces the impact of gradient descent through the Relu activation function, inputs the feature extraction results into the fully connected layer for feature integration and outputs the corresponding overlap level.
[0025] Furthermore, the overlap classification network is used to classify each input frame into one of four overlap levels, wherein the overlap levels include no overlapping objects in the frame, a small number of overlapping objects in the frame, a large number of overlapping objects in the frame, and countless overlapping objects in the frame.
[0026] The beneficial effects of adopting the above further scheme are as follows: this scheme proposes the concept of overlap and constructs a convolutional neural network to complete the overlap classification of the target, dividing each frame into one of the four overlap levels, which helps to select a preset number of frames with low overlap levels for target detection, and finally obtains the number of livestock being tested in the tested video through weighted average summation.
[0027] Furthermore, the specific steps of step S3 are as follows:
[0028] S31, inputting the overlap degree in step S12 as the true value together with the training set into the overlap degree classification network for training to obtain an overlap degree label;
[0029] S32, using the standard cross entropy loss function as the loss function, and using Adam to optimize the overlap classification network;
[0030] The cross entropy L(u,v) loss function expression is as follows:
[0031]
[0032] Among them, u represents the true distribution of overlap labels, v represents the fitted distribution of overlap labels, and u i represents the true distribution of the i-th overlap label, v i The fitted distribution of the i-th overlap label;
[0033] S33, performing back propagation training on the overlap classification network, and modifying the weight parameters of the overlap classification network, iterating the training until the classification accuracy is greater than or equal to a preset value, and then stopping the training to obtain a trained overlap classification network.
[0034] The beneficial effects of adopting the above further scheme are: training the overlap classification network to obtain the overlap labels of each frame, and using the standard cross entropy loss function and the Adam optimizer for optimization to reduce the loss during network training, updating and modifying the network weight parameters through back propagation, iterating training until the classification accuracy reaches the preset value, saving the model, and using it as the final classification model.
[0035] Furthermore, the specific steps of step S4 are as follows:
[0036] S41, inputting the video to be tested into the trained overlap classification network frame by frame to obtain the overlap level corresponding to each frame, wherein each frame of the video to be tested includes 3 channels;
[0037] S42: Compare the overlap levels of the frames to obtain a preset number of frames with the lowest overlap in the video to be tested.
[0038] The beneficial effects of adopting the above further scheme are: according to the overlap classification level, each frame is filtered, and the frame filtering algorithm is to select frames with low overlap in the video to be tested, and then use these frames for target detection and calculation; corresponding to the video to be tested, each frame contains 3 channels, which are RGB channels, and each frame is input into the overlap classification network. The overlap classification network limits the classification results of each frame to four levels of overlap classification, and selects a preset number of frames with the lowest overlap for subsequent target detection and counting.
[0039] Furthermore, the specific steps of step S5 are as follows:
[0040] S51, use Faster RCNN as the initial target detection network, and use the region candidate network RPN instead of candidate region selection;
[0041] S52, extracting image features of each preset number of frames by layer-by-layer learning of the network layer, and processing the image features by using the middle convolutional layer of the VGG16 network to obtain a feature map;
[0042] S53, using the anchor to map the preset number of frames according to the size of the pooling layer, to obtain a bounding box of the same size and a category score of the corresponding target;
[0043] The bounding box size f(w,h) is expressed as follows:
[0044] f(w,h)=(w / r)×(h / r)
[0045] Wherein, w represents the width of each preset number of frame bounding boxes, h represents the height of each preset number of frame bounding boxes, r represents the downsampling rate of the pooling layer, and r is 16;
[0046] S54, using the RPN network, retaining and filtering bounding boxes and categories whose IOU is greater than a set ratio threshold, and obtaining proposals consisting of bounding boxes of a preset number of target categories;
[0047] S55. Use the Roi pooling layer to combine the feature map and proposals and input them into the softmax classification and regression layer to determine the final position of the target, thereby completing the construction of the target detection network.
[0048] The beneficial effects of adopting the above further scheme are as follows: constructing a target detection network, using Faster RCNN as the target detection network, Faster RCNN uses the region candidate network RPN (Region Proposal Networks) to replace the candidate region selection process of RCNN, and divides the entire training process into two trainable stages. At the same time, because RPN shares feature weights with subsequent training processes, there is no additional cost, and the processing speed is greatly improved; after layer-by-layer learning, a feature map is obtained, and its size is much smaller than the original input image, and the layer is deeper; after obtaining the feature map, it is necessary to find proposals in the image for classification, and in the process of extracting image features, equal-sized convolution is used, that is, the image size remains unchanged before and after the convolution, so in the process of mapping the anchor to the original image, it is necessary to consider the size change of the pooling layer, that is, the size of the downsampling, and finally obtain the bbox of the target; after obtaining the target regression bounding box and target category, the bounding box and the corresponding category with an IOU greater than a certain ratio (0.8) threshold are retained, and after screening, the bounding boxes of the preset number of bounding boxes are composed of proposals and sent to the ROIpooling layer; the ROI The pooling layer combines the feature map and proposals and inputs them into the subsequent classification and regression layers to obtain the bounding box position and bounding box category for subsequent livestock location statistics and livestock number counting.
[0049] Furthermore, the specific steps of step S6 are as follows:
[0050] S61, inputting the X-axis coordinate, Y-axis coordinate, bounding box width and bounding box height in the five-tuple as true values together with the training set into the pre-trained model to train the RPN network, thereby obtaining a trained RPN network;
[0051] S62. Use the trained RPN network to collect proposals and train the Fast RCNN network to obtain a trained model.
[0052] S63, iteratively training the trained model using the validation set to obtain an optimal training model;
[0053] S64. According to the optimal training model, a target detection network is trained using a training set plus a validation set. The accuracy of the target detection network is tested using a test set to achieve a preset accuracy. The target detection network training is completed to obtain a trained target detection network.
[0054] The beneficial effect of adopting the above further scheme is: the X-axis coordinates and Y-axis coordinates of the position information of each frame in the quintuple, the bounding box width and the bounding box height of the frame size information are used together with the training set to train the RPN network, and then the trained RPN network is used to collect proposals to train the Fast RCNN network, and the network model is continuously optimized and tested by the validation set and the test set to obtain a target detection network model with a preset recognition accuracy, which is used for detecting and counting the position of the tested livestock in the image.
[0055] Furthermore, the expression for weighted average count fusion of the number of livestock in each frame in step S7 is as follows:
[0056]
[0057] Among them, C f represents the number of livestock being measured, i represents the i-th frame, a i represents the weight of the i-th frame, c i represents the number of livestock obtained through target detection in the i-th frame, and m represents the value of the preset number.
[0058] The beneficial effect of adopting the above further scheme is that the livestock position and livestock quantity in each frame can be obtained through the target detection network, but a preset number of frames are selected for target detection, and the preset number of results need to be weighted summed. Considering that the lower the overlap between the detected targets, the higher the confidence level of target detection and counting, this scheme assigns higher weights to frames with lower levels of overlap, and then realizes the counting fusion of the detected livestock for the preset number of frames according to the weighted summation method. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 The present invention is a flowchart of the steps of a livestock counting method based on frame filtering and target detection in an embodiment of the present invention.
[0060] Figure 2 It is a network framework diagram of the overlap classification network in an embodiment of the present invention.
[0061] Figure 3 4 is a network framework diagram of a target detection network in an embodiment of the present invention.
[0062] Figure 4 4 is a network structure diagram of the ROI pooling layer in an embodiment of the present invention.
[0063] Figure 5 Schematic diagram of frame filtering and target detection and counting in an embodiment of the present invention. DETAILED DESCRIPTION
[0064] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0065] Before describing this embodiment, the meanings of the following parameters are described first. The parameters and their meanings are shown in Table 1:
[0066] Table 1
[0067]
[0068]
[0069] like Figure 1 As shown, in one embodiment of the present invention, the present invention provides a livestock counting method based on frame filtering and target detection, the method comprising the following steps:
[0070] S1. Obtain a livestock counting dataset, and evenly divide the livestock counting dataset into a training set, a validation set, and a test set;
[0071] The specific steps of step S1 are as follows:
[0072] S11. Label the collected raw data of the pasture using labelme to obtain a livestock counting dataset including several pairs of pasture images and json annotations;
[0073] S12, processing the json annotation into five-tuples, and evenly dividing the livestock counting dataset into a training set, a validation set, and a test set according to the target balance strategy, wherein the five-tuple includes the X-axis coordinate, the Y-axis coordinate, the bounding box width, the bounding box height, and the overlap;
[0074] Labelme is used to label the original data of the ranch, that is, the video to be tested, and the JSON annotations corresponding to the ranch pictures are divided into five tuples, which are used to clean the video to be tested according to the degree of overlap, obtain a preset number of frames that can be used for target detection, target detection network training, and actual livestock target detection and counting.
[0075] S2, construct overlapping classification network;
[0076] like Figure 2 As shown, the step S2 constructs an overlap classification network including an input layer, five convolutional layers, two fully connected layers and an output layer, wherein each convolutional layer is followed by a pooling layer and a Relu activation function;
[0077] The overlap classification network is a simple classification network that extracts features through convolution operations, reduces resolution and improves the receptive field of feature extraction through pooling operations, reduces the impact of gradient descent through Relu activation functions, inputs feature extraction results into a fully connected layer for feature integration, and outputs the corresponding overlap level;
[0078] The overlap classification network constructed in step S2 classifies each input frame into one of four overlap levels, wherein the overlap levels include no overlapping objects in the frame, a small number of overlapping objects in the frame, a large number of overlapping objects in the frame, and an uncountable number of overlapping objects in the frame, wherein a small number of overlapping objects in the frame means that the overlap of objects in the frame is less than or equal to 10%, and a large number of overlapping objects in the frame means that the overlap of objects in the frame is greater than 10% and less than or equal to 30%;
[0079] This scheme proposes the concept of overlap and constructs a convolutional neural network to complete the overlap classification of the target. Each frame is divided into one of the four overlap levels, which helps to select a preset number of frames with low overlap levels for target detection. Finally, the number of livestock tested in the tested video is obtained through weighted average summation;
[0080] S3, training the overlap classification network to obtain a trained overlap classification network;
[0081] The specific steps of step S3 are as follows:
[0082] S31, inputting the overlap degree in step S12 as the true value together with the training set into the overlap degree classification network for training to obtain an overlap degree label;
[0083] S32, using the standard cross entropy loss function as the loss function, and using Adam to optimize the overlap classification network;
[0084] The cross entropy L(u,v) loss function expression is as follows:
[0085]
[0086] Among them, u represents the true distribution of overlap labels, v represents the fitted distribution of overlap labels, and u i represents the true distribution of the i-th overlap label, v i The fitted distribution of the i-th overlap label;
[0087] S33, performing back propagation training on the overlap classification network, modifying the weight parameters of the overlap classification network, iterating the training until the classification accuracy is greater than or equal to a preset value, and then stopping the training to obtain a trained overlap classification network;
[0088] The overlap classification network is trained to obtain the overlap labels of each frame, and the standard cross entropy loss function is used. The Adam optimizer is used for optimization to reduce the loss during the network training process. The network weight parameters are updated and modified through back propagation. The classification accuracy reaches the preset value through iterative training, and the model is saved and used as the final classification model.
[0089] S4, using the trained overlap classification network to filter frames according to the overlap, and obtain a preset number of frames with the lowest overlap, such as Figure 5 As shown;
[0090] The specific steps of step S4 are as follows:
[0091] S41, inputting the video to be tested into the trained overlap classification network frame by frame to obtain the overlap level corresponding to each frame, wherein each frame of the video to be tested includes 3 channels;
[0092] S42, comparing the overlap levels of the frames to obtain a preset number of frames with the lowest overlap in the video to be tested. The maximum value of the preset number is set to M, and a total of M frames with the lowest overlap are obtained;
[0093] According to the overlap classification level, each frame is filtered. The frame filtering algorithm is to select frames with low overlap in the video to be tested, and then use these frames for target detection and counting; corresponding to the video to be tested, each frame contains 3 channels, usually RGB channels, and each frame is input into the overlap classification network. The overlap classification network limits the classification results of each frame to four levels of overlap classification, and selects a preset number of frames with the lowest overlap for subsequent target detection and counting;
[0094] S5, build target detection network;
[0095] like Figure 3 As shown, the specific steps of step S5 are as follows:
[0096] S51, use Faster RCNN as the initial target detection network, and use the region candidate network RPN instead of candidate region selection;
[0097] S52, extracting image features of each preset number of frames by layer-by-layer learning of the network layer, and processing the image features by using the middle convolutional layer of the VGG16 network to obtain a feature map;
[0098] S53, using the anchor to map the preset number of frames according to the size of the pooling layer, to obtain a bounding box of the same size and a category score of the corresponding target;
[0099] The bounding box size f(w,h) is expressed as follows:
[0100] f(w,h)=(w / r)×(h / r)
[0101] Wherein, w represents the width of each preset number of frame bounding boxes, h represents the height of each preset number of frame bounding boxes, r represents the downsampling rate of the pooling layer, and r is 16;
[0102] S54, using the RPN network, retaining and filtering bounding boxes and categories whose IOU is greater than a set ratio threshold, and obtaining proposals consisting of bounding boxes of a preset number of target categories;
[0103] S55. Use the Roi pooling layer to combine the feature map and proposals and input them into the softmax classification and regression layer to determine the final position of the target, thereby completing the construction of the target detection network.
[0104] S55, use the Roi pooling layer to combine the feature map and proposals and input them into the softmax classification and regression layer to determine the final position of the target, thereby completing the target detection network construction, such as Figure 4 As shown;
[0105] Construct a target detection network, and use Faster RCNN as the target detection network. Faster RCNN uses the region proposal network RPN (Region Proposal Networks) to replace the candidate region selection process of RCNN, and divides the entire training process into two trainable stages. At the same time, because RPN shares feature weights with subsequent training processes, there is no additional cost, and the processing speed is greatly improved. After layer-by-layer learning, the feature map is obtained, and its size is much smaller than the original input image, and the layer is deeper. After obtaining the feature map, it is necessary to find proposals in the image for classification. In the process of extracting image features, equal-sized convolution is used, that is, the image size remains unchanged before and after convolution. Therefore, in the process of mapping the anchor to the original image, it is necessary to consider the size change of the pooling layer, that is, the size of the downsampling, and finally obtain the bbox of the target. After obtaining the target regression bounding box and target category, the bounding box and the corresponding category with an IOU greater than a certain ratio (0.8) threshold are retained. After screening, the bounding boxes of the preset number of bounding boxes are composed of proposals and sent to the ROI pooling layer. The ROI The pooling layer combines the feature map and proposals and inputs them into the subsequent classification and regression layers to obtain the bounding box position and bounding box category for subsequent livestock position statistics and livestock number counting;
[0106] S6. Train the target detection network to obtain a trained target detection network;
[0107] The specific steps of step S6 are as follows:
[0108] S61, inputting the X-axis coordinate, Y-axis coordinate, bounding box width and bounding box height in the five-tuple as true values together with the training set into the pre-trained model to train the RPN network, thereby obtaining a trained RPN network;
[0109] S62. Use the trained RPN network to collect proposals and train the Fast RCNN network to obtain a trained model.
[0110] S63, iteratively training the trained model using the validation set to obtain an optimal training model;
[0111] S64, training a target detection network using a training set plus a validation set according to the optimal training model, testing the accuracy of the target detection network using a test set to achieve a preset accuracy, completing the target detection network training, and obtaining a trained target detection network;
[0112] The X-axis coordinates and Y-axis coordinates of the position information of each frame in the five-tuple, the bounding box width and bounding box height of the frame size information are used together with the training set to train the RPN network, and then the trained RPN network is used to collect proposals to train the Fast RCNN network. Through the continuous optimization and testing of the network model with the validation set and the test set, the target detection network model with the preset recognition accuracy is obtained, which is used to detect and count the position of the tested livestock in the image;
[0113] S7, using the trained target detection network to detect a preset number of frames with the lowest overlap, obtaining the livestock position and livestock number in each frame, and performing weighted average counting fusion on the livestock number in each frame to obtain the detected livestock number;
[0114] The expression for weighted average count fusion of the number of livestock in each frame in step S7 is as follows:
[0115]
[0116] Among them, C f represents the number of livestock being measured, i represents the i-th frame, a i represents the weight of the i-th frame, c i represents the number of livestock obtained through target detection in the i-th frame, and m represents the value of the preset number;
[0117] The position and number of livestock in each frame can be obtained through the target detection network, but a preset number of frames are selected for target detection, and the preset number of results need to be weighted summed. Considering that the lower the overlap between detected targets, the higher the confidence level of target detection and counting, this scheme assigns higher weights to frames with lower levels of overlap, and then realizes the counting fusion of the detected livestock for the preset number of frames according to the weighted summation method.
[0118] The beneficial effects of the present invention are as follows: the livestock counting method based on frame filtering and target detection proposed in the present invention cleans the measured video at a practical overlap, selects frames with low livestock overlap for target detection and counting, and has higher practicality and robustness than traditional target detection methods, greatly improves the detection and counting accuracy, and effectively reduces the influence of livestock activities on counting results, and is also applicable to areas with high livestock activity intensity and complex scenes.
Claims
1. A livestock counting method based on frame filtering and target detection, characterized in that: The method comprises the following steps: S1. Obtain a livestock counting dataset, and evenly divide the livestock counting dataset into a training set, a validation set, and a test set; S2, construct overlapping classification network; S3, training the overlap classification network to obtain a trained overlap classification network; S4, using the overlap to perform frame filtering on the trained overlap classification network to obtain a preset number of frames with the lowest overlap; S5, build target detection network; S6. Train the target detection network to obtain a trained target detection network; S7. Use the trained target detection network to detect a preset number of frames with the lowest overlap, obtain the livestock position and livestock quantity in each frame, and perform weighted average counting fusion on the livestock quantity in each frame to obtain the measured livestock quantity.
2. The livestock counting method based on frame filtering and target detection according to claim 1, characterized in that: The specific steps of step S1 are as follows: S11. Label the collected raw data of the pasture using labelme to obtain a livestock counting dataset including several pairs of pasture images and json annotations; S12. Process the json annotation into five-tuples, and evenly divide the livestock counting dataset into training set, validation set and test set according to the target balance strategy, where the five-tuple includes X-axis coordinate, Y-axis coordinate, bounding box width, bounding box height and overlap.
3. The livestock counting method based on frame filtering and target detection according to claim 2, characterized in that: The step S2 constructs an overlap classification network including an input layer, five convolutional layers, two fully connected layers and an output layer which are arranged in sequence, wherein each convolutional layer is correspondingly followed by a pooling layer and a Relu activation function.
4. The livestock counting method based on frame filtering and target detection according to claim 3, characterized in that: The overlap classification network is used to classify each input frame into one of four overlap levels, wherein the overlap levels include no overlapping objects in the frame, a small number of overlapping objects in the frame, a large number of overlapping objects in the frame, and countless overlapping objects in the frame.
5. The livestock counting method based on frame filtering and target detection according to claim 4, characterized in that: The specific steps of step S3 are as follows: S31, inputting the overlap degree in step S12 as the true value together with the training set into the overlap degree classification network for training to obtain an overlap degree label; S32, using the standard cross entropy loss function as the loss function, and using Adam to optimize the overlap classification network; The cross entropy L(u,v) loss function expression is as follows: Among them, u represents the true distribution of overlap labels, v represents the fitted distribution of overlap labels, and u i represents the true distribution of the i-th overlap label, v i The fitted distribution of the i-th overlap label; S33, performing back propagation training on the overlap classification network, and modifying the weight parameters of the overlap classification network, iterating the training until the classification accuracy is greater than or equal to a preset value, and then stopping the training to obtain a trained overlap classification network.
6. The livestock counting method based on frame filtering and target detection according to claim 5, characterized in that: The specific steps of step S4 are as follows: S41, inputting the video to be tested into the trained overlap classification network frame by frame to obtain the overlap level corresponding to each frame, wherein each frame of the video to be tested includes 3 channels; S42: Compare the overlap levels of the frames to obtain a preset number of frames with the lowest overlap in the video to be tested.
7. The livestock counting method based on frame filtering and target detection according to claim 6, characterized in that: The specific steps of step S5 are as follows: S51, use Faster RCNN as the initial target detection network, and use the region candidate network RPN instead of candidate region selection; S52, extracting image features of each preset number of frames by layer-by-layer learning of the network layer, and processing the image features by using the middle convolutional layer of the VGG16 network to obtain a feature map; S53, using the anchor to map the preset number of frames according to the size of the pooling layer, to obtain a bounding box of the same size and a category score of the corresponding target; The bounding box size f(w,h) is expressed as follows: f(w,h)=(w / r)×(h / r) Wherein, w represents the width of each preset number of frame bounding boxes, h represents the height of each preset number of frame bounding boxes, r represents the downsampling rate of the pooling layer, and r is 16; S54, using the RPN network, retaining and filtering bounding boxes and categories whose IOU is greater than a set ratio threshold, and obtaining proposals consisting of bounding boxes of a preset number of target categories; S55. Use the Roi pooling layer to combine the feature map and proposals and input them into the softmax classification and regression layer to determine the final position of the target, thereby completing the construction of the target detection network.
8. The livestock counting method based on frame filtering and target detection according to claim 7, characterized in that: The specific steps of step S6 are as follows: S61, inputting the X-axis coordinate, Y-axis coordinate, bounding box width and bounding box height in the five-tuple as true values together with the training set into the pre-trained model to train the RPN network, thereby obtaining a trained RPN network; S62. Use the trained RPN network to collect proposals and train the Fast RCNN network to obtain a trained model. S63, iteratively training the trained model using the validation set to obtain an optimal training model; S64. According to the optimal training model, a target detection network is trained using a training set plus a validation set. The accuracy of the target detection network is tested using a test set to achieve a preset accuracy. The target detection network training is completed to obtain a trained target detection network.
9. The livestock counting method based on frame filtering and target detection according to claim 8, characterized in that: The expression for weighted average count fusion of the number of livestock in each frame in step S7 is as follows: Among them, C f represents the number of livestock being measured, i represents the i-th frame, a i represents the weight of the i-th frame, c i represents the number of livestock obtained through target detection in the i-th frame, and m represents the value of the preset number.
Citation Information
Patent Citations
Face detection method and device, mobile terminal and storage medium
CN110688930A
Video SAR vehicle target detection method
CN111798490A