A pig detection method based on video front and back frame modeling

Abstract

The application discloses a pig detection method based on video front and rear frame modeling. First, the data in the data set is subjected to brightness transformation, local occlusion, PCA jitter and other enhancement operations, so that the robustness to various target detection environments is improved. Then, the transfer learning and the attention mechanism are combined with the end-to-end DETR model, so that the accuracy of target detection is effectively improved. Subsequently, the Yakal index reset confidence is designed to obtain the final target detection result, so that the probability of misrecognition of flying snow, spots and repeated identification of pigs is reduced. The application is helpful to intelligent pig farm construction and can provide a reference for subsequent pig behavior recognition research.

Description

TECHNICAL FIELD

[0001] The application relates to a pig detection method based on video front and rear frame modeling, which establishes a pig farm monitoring video as a digital image set containing a pig target, introduces a plug-and-play attention mechanism on the basis of a DETR model, designs a reset confidence algorithm and adds a man-machine interaction module, and belongs to the field of computer vision target detection. BACKGROUND

[0002] Digital farming is a method of cultivating and managing living organisms through digital technology. It can be applied in various fields such as agriculture, aquaculture, animal feeding, etc. Digital farming can achieve precise control of growth environment, feed, temperature, humidity, etc., thereby improving production efficiency and product quality. At the same time, digital farming can realize remote monitoring and management through sensors, the Internet, artificial intelligence, etc., thereby realizing intelligent production and management. The development of digital farming can bring more efficient, more sustainable, and more environmentally friendly production methods, which is of great significance to meet the increasing demand for food, aquatic products, and meat.

[0003] For pig farms, data acquisition and analysis technology is an important component of pig digital farming, which collects and analyzes information of pig farms. For digital image information of pig farms, target detection technology integrates data acquisition and analysis functions. This technology obtains the location information of all pigs in the pig farm at different time periods, which can help to judge the health and activity of pigs, and is also the basis for subsequent data collection such as behavior recognition, providing information support for managers to optimize breeding strategies and resource allocation.

[0004] The methods of target detection mainly include the following: target detection method based on traditional machine learning: including detection method based on sliding window classifier, target detection method based on deep learning, target detection method based on combination of deep learning and traditional machine learning, target detection method based on single-stage detector, target detection method based on multi-stage detector. Each has its advantages and disadvantages, and needs to be flexibly selected according to the application scenario.

[0005] However, the existing algorithms still have certain deficiencies. First, the existing algorithms have certain limitations in convolution operations, including difficulty in processing irregularly shaped targets, fixed size of receptive field, and inability to adapt to targets of different sizes. Second, the current algorithm has a large number of missed detection and false detection situations for a large number of target detection, which is difficult to apply in digital farming. SUMMARY

[0006] Technical problem: The purpose of the present application is to solve the problem of target detection in the case of large pig density, more overlapping pigs, and more interference conditions in the prior art, and to provide a pig detection method based on video front and rear frame modeling.

[0007] Technical scheme: The pig detection method based on video front and back frame modeling of the application successively comprises the following steps:

[0008] Step 1: Based on the group pig image of the real pig farm, 2000 image data sets are obtained by using data enhancement technology;

[0009] Step 2: Select the target detection model to obtain the preliminary target detection result;

[0010] Step 3: Design Jacard index to measure the target detection overlap index;

[0011] Step 4: Calculate the Euclidean distance to measure the target detection misidentification index;

[0012] Step 5: According to the results obtained above, the detection model based on video front and back frame is established, and the pig detection result of the input image is obtained.

[0013] Further, in step 1, based on the group pig image of the real pig farm, data enhancement is carried out by using brightness transformation, local shielding and PCA jitter technology, and 2000 image data sets are obtained.

[0014] Further, in step 2, the transfer learning and attention mechanism are combined with the end-to-end DETR model to train the data set and obtain the pig detection model.

[0015] Further, in step 3, Jacard index is designed to measure the target detection overlap index, and the Jacard index pc it The calculation formula is as follows.

[0016]

[0017] In the above formula, S t represents the area of the recognition frame of the tth detection result, S i represents the area of the recognition frame of the ith detection result, and S it represents the repeated area of the recognition frame of the ith and tth detection results, and the calculation formula is as follows.

[0018] S it =(max(x 1t ,x 1i )-min(x 2t ,x 2i ))(max(y it ,y ii )-min(y 2t ,y 2i ))

[0019] In the above formula, (x1t 1t 1i 1i (x 2t 2t (x 2i 2i (x

[0020] Further, in the step 4, the calculation formula of the target detection misrecognition index based on the Euclidean distance is as follows.

[0021]

[0022] In the above formula, D is the Euclidean distance set of all detection results, D i represents the Euclidean distance of the i-th object, and the calculation formula is as follows:

[0023]

[0024] In the above formula, x m m represents the center point pixel coordinates of any one detection result of the previous frame, x i i represents the center point pixel coordinates of the i-th result of the current frame.

[0025] Further, in the step 5, according to the overlap index, the misrecognition index and the pig detection model reset confidence, the video front and back frame detection model is established, and the calculation formula is as follows.

[0026]

[0027] In the above formula, p' i represents the confidence of the i-th identification result obtained by the video front and back frame detection model, p i represents the confidence of the i-th identification result obtained by the pig detection model, p t represents the confidence of the t-th identification result obtained by the pig detection model, and pw i represents the misrecognition index of the i-th identification result, pc it represents the overlap index of the i-th and t-th identification results, and T represents the overlap index threshold.

[0028] Beneficial effects:

[0029] 1. The application utilizes data enhancement technology to improve the adaptability and detection precision of the target detection model to various environments.

[0030] ​​​​​​​2. The target detection method based on video frame-by-frame modeling used in this invention can effectively detect individual pigs in scenes with significant occlusion, adhesion, and interference.

[0031] 3. The target detection model selected in this invention was compared with other target detection algorithms in experiments. The results show that the accuracy of recognition and the processing speed have been improved to a certain extent.

[0032] 4. This invention helps managers supervise the entire production process of pig farms, observe and monitor the production situation of pig farms, and minimize the number of people entering the farm to avoid cross-infection between humans and animals. Attached Figure Description

[0033] Figure 1 This is a general framework diagram of the present invention patent;

[0034] Figure 2 The data augmentation results for the dataset are shown in Figure 2.1, where 2.2 is the data augmentation figure for brightness transformation, 2.2 is the data augmentation figure for partial occlusion, and 2.3 is the data augmentation figure for PCA method.

[0035] Figure 3 This is a comparison chart showing the performance of the proposed pig target detection method based on consecutive video frames compared to detection using only the selected target detection model. Detailed Implementation

[0036] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments, but the scope of protection of the present invention is not limited to the embodiments described.

[0037] like Figure 1 As shown, the pig target detection method based on consecutive video frames in this embodiment includes the following steps:

[0038] Step 1: Based on images of pigs in a real pig farm, data augmentation techniques were used to obtain a dataset of 2000 images;

[0039] Step 2: Select an object detection model to obtain preliminary object detection results;

[0040] Step 3: Design the Jakarta index to measure the overlap index of target detection;

[0041] Step 4: Calculate the Euclidean distance to measure the false recognition index of object detection;

[0042] Step 5: Based on the results obtained above, establish a detection model based on the preceding and following frames of the video to obtain the detection results of pigs in the input image.

[0043] In the embodiment, in step 1, based on the group pig image of the real pig farm, data augmentation is performed by using brightness transformation, local occlusion and PCA jittering technology, and 2000 image data sets are obtained. Here, the data augmentation result used by the application is used to process the result of the input image as shown in Figure 2

[0044] In the embodiment, in step 2, the transfer learning and attention mechanism are combined with the end-to-end DETR model to train the data set and obtain a pig detection model.

[0045] Here, the performance of the YOLOV7, FasterRCNN, DETR and DETR+SA models based on the parameters for measuring the target detection accuracy is compared, as shown in Table 1.

[0046] Table 1: MAP parameter result table of each model

[0047]

[0048] Here, the model selected by the application is further compared with the current best target detection method in terms of model complexity, as shown in Table 2.

[0049] Table 2: Model complexity result table

[0050]

[0051] The results show that the model selected by the application has greatly improved the accuracy of recognition and the complexity of the model, greatly improving the processing efficiency.

[0052] The problem to be solved by video target detection is the correct recognition and positioning of each frame target in the video. Compared with image target detection, video is highly redundant and contains a large amount of temporal locality, i.e. the target detection results in adjacent time are probably similar, and spatial locality. Therefore, making full use of the temporal context information and time sequence relationship in the video detection process can solve the problem of a large amount of redundancy between consecutive frames, thereby improving the detection speed, improving the detection quality and improving the influence of motion blur, defocus deformation and other problems existing in video relative to image on target detection.

[0053] In the embodiment, in step 3, in order to reduce the probability of misidentification and partial identification of the target object under the condition of partial overlap, a Jaccard index is designed to measure the target detection overlap index, and the Jaccard index pc it The calculation formula is as follows.

[0054]

[0055] In the above formula, S t ​S i S it S

[0056] S it = (max(x 1t ,x 1i )-min(x 2t ,x 2i ))(max(y it ,y ii )-min(y 2t ,y 2i ))

[0057] In the above formula, (x 1t ,y 1t ), (x 1i ,y 1i ) represent the pixel coordinates of the upper left corner of the t, i identified object, and (x 2t ,y 2t ), (x 2i ,y 2i ) represent the pixel coordinates of the right lower corner of the t, i identified object.

[0058] In this embodiment, in step 4, in order to reduce the probability of misidentification of flying snow, spots, etc. in a frame, the calculation formula of the target detection misidentification index based on the Euclidean distance is as follows.

[0059]

[0060] In the above formula, D is the Euclidean distance set of all detection results, and D i represents the Euclidean distance of the i-th object, and the calculation formula is as follows.

[0061]

[0062] In the above formula, x m ,y m represent the pixel coordinates of the center point of any one detection result in the previous frame, and x i ,y i represent the pixel coordinates of the center point of the i-th result in the current frame.

[0063] In this embodiment, in step 5, according to the overlap index, the misidentification index, and the pig detection model reset confidence, the video front and back frame detection model is established, and the calculation formula is as follows.

[0064]

[0065] In the above formula, p′ i p represents the confidence level of the i-th recognition result obtained by the detection model of the preceding and following frames of the video. i p represents the confidence level of the i-th identification result obtained by the pig detection model. t pw represents the confidence level of the t-th identification result obtained by the pig detection model. i pc represents the misidentification index of the i-th identification result. it Let represent the overlap index of the i-th and t-th recognition results, and T represent the overlap index threshold.

[0066] Here, a comparison of the detection results of the pig target detection method based on consecutive video frames proposed in this invention with that using only the selected target detection model is shown in the figure below. Figure 3 As shown, the results indicate that using a pig target detection method based on previous and subsequent video frames can significantly reduce the probability of misidentification of objects such as flying snow and spots, as well as the probability of repeated identification in overlapping situations.

Claims

1. A method for pig detection based on video frame-by-frame modeling, characterized in that, Includes the following steps: Step 1: Based on images of pigs in a real pig farm, data augmentation techniques were used to obtain a dataset of 2000 images; Step 2: Select an object detection model to obtain preliminary object detection results; Step 3: Design the Jakarta index to measure the overlap index of target detection; Step 4: Calculate the Euclidean distance to measure the false recognition index of object detection; Step 5: Reset the confidence level to obtain the final pig detection result for the input image; In step 3, the Jacarbe index is designed to measure the overlap index of target detection. The Jacarbe index pc between the i-th and t-th objects is... it The calculation formula is as follows: In the above formula, S t S represents the area of ​​the bounding box for the t-th detection result. i S represents the area of ​​the bounding box for the i-th detection result. it The area of ​​the overlapping bounding box for the two detection results at the i-th and t-th digits is calculated using the following formula: S it <(max(x 1t ,x 1i )-min(x 2t ,x 2i ))(max(y 1t ,y 1i )-min(y 2t ,y 2i )) In the above formula, (x 1t ,y 1t ),(x 1i ,y 1i (x) represents the pixel coordinates of the top-left corner of the t,i-th identified object. 2t ,y 2t ),(x 2i ,y 2i () represents the pixel coordinates of the bottom right corner of the t,i-th identified object; In step 5, a video frame detection model is established based on the overlap index, misidentification index, and the reset confidence level of the pig detection model. The calculation formula is as follows: In the above formula, p′ i p represents the confidence level of the i-th recognition result obtained by the detection model of the preceding and following frames of the video. i p represents the confidence level of the i-th identification result obtained by the pig detection model. t pw represents the confidence level of the t-th identification result obtained by the pig detection model. i pc represents the misidentification index of the i-th identification result. it Let represent the overlap index of the i-th and t-th recognition results, and T represent the overlap index threshold.

2. The pig detection method based on video frame-by-frame modeling according to claim 1, characterized in that: In step 1, based on images of pigs in a real pig farm, data enhancement is performed using brightness transformation, local occlusion, and PCA dithering techniques to obtain a dataset of 2000 images.

3. The pig detection method based on video frame-by-frame modeling according to claim 1, characterized in that: In step 2, transfer learning and attention mechanisms are combined with an end-to-end DETR model to train the dataset and obtain a pig detection model.

4. The pig detection method based on video frame-by-frame modeling according to claim 1, characterized in that: In step 4, the formula for calculating the target detection misidentification index based on Euclidean distance is as follows: In the above formula, D is the set of Euclidean distances for all detection results. i The Euclidean distance of the i-th object is represented by the following formula: In the above formula, x m ,y m This represents the pixel coordinates of the center point of the detection result in the previous frame, x. i ,y i This represents the pixel coordinates of the center point of the i-th detection result in this frame.

Images