A method for analyzing the behavior of stored grain pests based on YOLOv5 and an improved SORT algorithm

Abstract

This invention relates to a method for analyzing the behavior of stored grain pests based on YOLOv5 and an improved SORT algorithm, belonging to the technical field of stored grain pest behavior analysis methods. This method utilizes a camera to collect and filter videos of stored grain pests crawling, and processes the videos frame by frame. First, each frame is preprocessed using OpenCV to detect the outline of the stored grain pests and extract target coordinate data. When target overlap or loss occurs in a frame, resulting in detection failures for certain stored grain pests, the frame is segmented and fed into the YOLOv5 target detection network for re-detection, optimizing the detection results. This method solves the problems of low accuracy, susceptibility to environmental factors, and poor detection performance when overlap occurs in existing stored grain pest behavior analysis methods, making it particularly suitable for the needs of stored grain pest behavior analysis.

Description

Technical Field

[0001] This invention relates to a method for analyzing the behavior of stored grain pests based on YOLOv5 and an improved SORT algorithm, belonging to the technical field of methods for analyzing the behavior of stored grain pests. Background Technology

[0002] In the grain storage sector, stored-grain pests cause serious damage to stored grains and result in significant economic losses. Traditional methods for controlling stored-grain pests, such as chemical control and light control, either pollute the environment or are ineffective. Therefore, it is necessary to analyze the behavioral habits of various stored-grain pests so that researchers can develop better control methods based on these habits.

[0003] Existing methods for analyzing the behavior of stored grain pests, such as the invention patent application CN113361429A which discloses a method and experimental apparatus for analyzing the movement behavior of stored grain pests, can meet the needs of analyzing the behavior of stored grain pests to a certain extent, but they have the following problems:

[0004] First, existing methods for analyzing the movement behavior of stored grain pests rely on traditional OpenCV and background subtraction to extract their coordinates. When multiple pests crawl to the same location, causing overlap, their outlines merge. The OpenCV algorithm may mistakenly identify these overlapping pests as a single insect, leading to errors in the existing detection method. Furthermore, background subtraction is only suitable for detecting moving targets against a relatively static background and is highly sensitive to changes in lighting and background. Therefore, this method has low accuracy in extracting pest coordinates and is easily affected by environmental factors.

[0005] Second: Existing methods for analyzing the movement behavior of stored grain pests use the K-nearest neighbor algorithm in the multi-target tracking and inter-frame stored grain pest target association matching stages. This algorithm is a relatively basic and simple classification algorithm. The prediction results are easily affected by noisy data, the computational load is large and the prediction speed is not very fast. Furthermore, when multiple stored grain pests crawl to the same position and multiple targets overlap or are very close, this algorithm cannot accurately match stored grain pest objects between frames, which can easily lead to tracking errors.

[0006] Therefore, it is necessary to develop a new analytical method to address the aforementioned problems with existing behavioral analysis methods for stored grain pests. Summary of the Invention

[0007] The purpose of this invention is to provide a method for analyzing the behavior of stored grain pests based on YOLOv5 and an improved SORT algorithm, in order to solve the problems of low accuracy, susceptibility to environmental factors, and poor detection performance when there is overlap in existing methods for analyzing the behavior of stored grain pests.

[0008] The technical solution of this invention is:

[0009] The method for analyzing the behavior of stored grain pests based on YOLOv5 and the improved SORT algorithm includes the following steps:

[0010] 1. Collect sample videos and preprocess them to remove redundant information and noise that may affect detection, and to highlight the outlines of stored grain pests. The specific preprocessing steps are as follows:

[0011] 1.1) Collect sample videos:

[0012] 1.2) Video frame cropping:

[0013] 1.3) Grayscale conversion:

[0014] 1.4) Binarization:

[0015] 1.5) Expansion:

[0016] 1.6) Corrosion:

[0017] 2. Coordinate Extraction

[0018] 2.1) OpenCV Target Extraction

[0019] 2.2) YOLOv5 target extraction

[0020] 2.2.1) Creating the dataset:

[0021] 2.2.2) Setting up a YOLOv5s network:

[0022] 2.2.3) Training the model:

[0023] 2.2.4) Filtering and detecting error frames:

[0024] 2.2.5) Sample frames that were incorrectly detected using the traditional OpenCV method are fed into the trained YOLOv5s network for prediction. Since the stored grain pests in the sample frames occupy a very small proportion of the entire image, this falls under the category of small target detection. To improve the accuracy of YOLOv5s detection, the sample frames are divided into four sub-images of the same size (top left, top right, bottom left, and bottom right) and fed into the trained YOLOv5s network in step 2.2.3) for prediction. The position information of the stored grain pests contained in the four sub-images is obtained. Then, using the relative positions of these four sub-images with respect to the original sample frame, the absolute positions of the stored grain pests detected in the four sub-images with respect to the original sample frame are calculated, and the center coordinates of each stored grain pest are calculated.

[0025] 3. Target tracking;

[0026] 4. Trajectory plotting and behavior analysis.

[0027] The advantages of this invention are:

[0028] This method for analyzing the behavior of stored grain pests, based on YOLOv5 and an improved SORT algorithm, utilizes a camera to capture and filter videos of stored grain pests crawling. The video is then processed frame by frame. First, each frame is preprocessed using OpenCV to detect the outline of the stored grain pests and extract their coordinate data. When overlapping or missing targets occur in a frame, causing some stored grain pests to fail to be detected, that frame is segmented and fed into the YOLOv5 target detection network for re-detection, optimizing the results. The improved SORT algorithm is then used for data association to track the movement trajectories of the stored grain pests. Finally, the movement trajectory of each stored grain pest is plotted, and the movement distance and real-time speed of each pest are calculated. This method solves the problems of low accuracy, susceptibility to environmental factors, and poor detection performance when overlapping targets exist in existing stored grain pest behavior analysis methods, making it particularly suitable for the needs of stored grain pest behavior analysis. Attached Figure Description

[0029] Figure 1 This is a flowchart of the analysis process for this application;

[0030] Figure 2 Extract target images of stored grain pests using OpenCV in this application;

[0031] Figure 3 Here is the flowchart for the SORT algorithm;

[0032] Figure 4 The improved SORT algorithm of this application is used to handle the IDswitch problem diagram;

[0033] Figure 5 This is a diagram illustrating the effect of tracking stored grain pests in this application.

[0034] Figure 6 An image illustrating the effect of detection failure caused by overlapping or missing targets;

[0035] Figure 7 The image shows the results of YOLOv5s network optimization detection.

[0036] Figure 8 A diagram illustrating how object detection failure leads to association errors in the SORT algorithm;

[0037] Figure 9 A schematic diagram illustrating how to improve the SORT algorithm to address the IDswitch phenomenon; Detailed Implementation

[0038] The method for analyzing the behavior of stored grain pests based on YOLOv5 and the improved SORT algorithm includes the following steps:

[0039] 1. Collect sample videos and preprocess them to remove redundant information and noise that may affect detection, and to highlight the outlines of stored grain pests. The specific preprocessing steps are as follows:

[0040] 1.1) Collect sample videos: Place one or more grain storage pests in an SPX250 biochemical incubator. The incubator does not provide a light source. The temperature inside the incubator is (30±1)℃ and the humidity is (45±5)%. Use a Sony IMX317 to film the crawling behavior of the grain storage pests. Use the built-in LED light of the camera to provide illumination. Select the group that did not exhibit flight behavior in the filmed video as the sample video. The video resolution is 1920×1080 and the frame rate is 24fps.

[0041] 1.2) Video frame cropping: Extract sample frames one by one from the collected sample video to generate sample frames, and use OpenCV methods to crop the sample frames to remove areas outside the activity range of stored grain pests.

[0042] 1.3) Grayscale conversion: Using OpenCV functions, the color space of the sample frame is converted from three dimensions to one dimension, and the RGB values ​​of each pixel are unified into the same value to form a grayscale image of the sample frame;

[0043] 1.4) Binarization: Set a pixel threshold of 75, change the value of pixels with a gray value lower than the threshold of 75 to 0, and set the value of pixels with a gray value higher than the threshold of 75 to 255, so that the gray image presents a clear 'black and white' effect, highlighting the outline of the grain storage pest in the gray image, and converting the gray image into a binarized image.

[0044] 1.5) Dilation: Dilate the white area in the binarized image to remove noise information and segment out the outline of each grain storage pest.

[0045] 1.6) Erosion: OpenCV is used to perform erosion on the dilated binarized image to expand the outline boundary of the black grain storage pests and enlarge the outline area, facilitating subsequent extraction. This transforms the sample frame into a preprocessed binarized image of the grain storage pests.

[0046] 2. Coordinate Extraction

[0047] 2.1) OpenCV Target Extraction

[0048] The binarized image of stored grain pests obtained in step 1 is used to extract contours using traditional OpenCV methods. Then, using a pre-defined threshold, contours with an area greater than 25 and less than 260 are selected. Contour detection is performed frame-by-frame on the binarized image to extract the position coordinates of all stored grain pests in the image. The center point of each pest is calculated, and the positions of the pests are drawn using bounding boxes (see the instruction manual appendix). Figure 2 );

[0049] 2.2) YOLOv5 target extraction

[0050] 2.2.1) Dataset Creation: First, extract images of stored grain pests from the sample videos captured in step 1.1). Ensure that the images contain as many movement scenarios of stored grain pests as possible. Then, use the labelimg data annotation tool to label the stored grain pests in the images with the corresponding VOC format tags. Next, use code to convert the VOC format tags to the YOLO format tags required by the YOLOv5s network, and divide the dataset into training and testing sets, with 70% used as the training set and 30% as the testing set.

[0051] 2.2.2) Setting up a YOLOv5s network: Configure the PyTorch environment, clone the YOLOv5s open source code from the official website, use command operations to update the dependency libraries in the environment to meet the requirements of YOLOv5s operation, and use yolov5s.pt as pre-trained weights;

[0052] 2.2.3) Training the model: Modify the parameters, set the dataset and pre-training weights, and train the YOLOv5s network using the prepared grain storage pest dataset;

[0053] 2.2.4) Screening and Detecting Error Frames: In step 2.1), during the extraction of stored grain pest targets using the traditional OpenCV method, a parameter is preset, which is the actual number of stored grain pests in the sample video. The system checks whether the number of stored grain pest targets in each sample frame after preprocessing and contour extraction matches the preset actual number. If they do not match, the sample frame is defined as an error frame.

[0054] 2.2.5) Sample frames that were incorrectly detected using the traditional OpenCV method are fed into the trained YOLOv5s network for prediction. Since the stored grain pests in the sample frames occupy a very small proportion of the entire image, this falls under the category of small target detection. To improve the accuracy of YOLOv5s detection, the sample frames are divided into four sub-images of the same size (top left, top right, bottom left, and bottom right) and fed into the trained YOLOv5s network in step 2.2.3) for prediction. The position information of the stored grain pests contained in the four sub-images is obtained. Then, using the relative positions of these four sub-images with respect to the original sample frame, the absolute positions of the stored grain pests detected in the four sub-images with respect to the original sample frame are calculated, and the center coordinates of each stored grain pest are calculated.

[0055] 3. Target tracking;

[0056] Step 2 involves using a combination of traditional OpenCV and YOLOv5s to detect stored grain pests frame by frame in the sample video. Then, the location coordinates of the stored grain pests are fed into the improved SORT algorithm frame by frame for inter-frame association matching, assigning a corresponding ID to each stored grain pest. The specific process is as follows:

[0057] SORT Algorithm: The SORT algorithm is a multi-target tracking algorithm. The two most important methods in the algorithm are the Hungarian algorithm and Kalman filtering (see the instruction manual appendix). Figure 3 ).

[0058] Frame 1: The location coordinates of stored grain pests detected by the target detector combining OpenCV and YOLOv5s are used as detection boxes. Since there are no tracks at this time, the information in the detection boxes is directly updated and initialized as tracking boxes using Kalman filtering in the first frame.

[0059] In frame N, where N>1: The predicted information in the tracks within frame N-1 is obtained using Kalman filtering. This prediction is then combined with the coordinates of stored grain pests from the target detector in frame N to form new detection boxes. The intersection-over-union ratio (IoU) between the predicted detection boxes and the track boxes is calculated as the cost matrix. The Hungarian algorithm is used to perform the association, and three matching scenarios may occur:

[0060] Tracking boxes that fail to match: Determine whether to delete them based on a set threshold;

[0061] For detection boxes that fail to match: Kalman filter initialization, assign a new ID to the box, and add its information to the tracking box;

[0062] The prediction results of the detection box and the tracking box are successfully matched: assign the corresponding ID to the successfully associated detection box, and then update the weights and parameters of its Kalman filter.

[0063] The improved SORT algorithm: Instead of setting a threshold and deleting failed tracking boxes, it records them as `miss_track`. When previously detected grain pests are re-detected in subsequent frames, their detection boxes are associated with the information in the `miss_track`. The Euclidean distances between the `m` failed detection boxes and the `n` failed tracking boxes in `miss_track` are calculated.

[0064] ;

[0065] In the formula: This represents the Euclidean distance between the i-th unmatched detection box and the j-th unmatched tracking box; , ),( , ) represent the horizontal and vertical coordinates of the center points of the failed detection box information of the i-th grain storage pest and the failed tracking box information in the j-th miss_track, respectively.

[0066] Based on the Euclidean distance, re-match the failed detection box information of the i-th grain storage pest with the Euclidean distance between it and the target. The smallest j-th miss_track is associated with the failed matching bounding box, thus preventing association errors due to detection failures of some frames (see the appendix of the specification). Figure 4 ).

[0067] 4. Trajectory Plotting and Behavior Analysis

[0068] Step 3: The coordinates of stored grain pests extracted frame by frame are fed into the improved SORT algorithm for inter-frame matching and association. This identifies the correspondence between multiple stored grain pests in two adjacent sample frames and assigns a corresponding ID to each stored grain pest. A dictionary is created to store the center point coordinates of each ID stored grain pest across all n frames. Then, based on the center point coordinates of the same ID stored grain pest in the previous (n-1)th frame and the current (n)th frame, the Euclidean distance between the center point coordinates of each stored grain pest in adjacent frames is calculated. As the distance traveled by each grain-storing pest in the current frame, in the formula This represents the distance traveled by the stored grain pest in the current nth frame. These represent the x and y coordinates of the center point of a stored grain pest with the same ID in frame n and frame (n-1), respectively. The total travel distance is... In the formula This represents the distance traveled by stored grain pests in the i-th frame. This means that the total walking distance of the stored grain pests is calculated by summing the distances they travel in frames 1, 2, 3, ..., up to frame n. The interval between each frame is then calculated based on the frame rate x. The reciprocal of the frame rate x is used as the interval between each frame. Current walking speed In the formula This represents the speed at which stored grain pests crawl in the nth frame. This represents the walking distance of the stored grain pest in the current nth frame, and t represents the time interval between each frame. Finally, based on the center point coordinates of each stored grain pest with a different ID in the dictionary across all n frames, different colored trajectories are drawn, and their real-time movement speed and movement distance are displayed respectively (see the instruction manual appendix). Figure 5 ).

[0069] This application utilizes the traditional OpenCV method to extract the location coordinates of stored grain pests, which has the advantage of very high processing speed. However, the traditional OpenCV method sometimes misses some stored grain pests in video frames. Furthermore, when multiple stored grain pests crawl to the same location, causing target overlap, the outlines of the overlapping pests will merge together. Using the OpenCV algorithm, multiple overlapping stored grain pests may be identified as the same pest, leading to detection errors, as shown in the attached specification. Figure 6 As shown. Therefore, when detection fails, it is necessary to use the YOLOv5 deep convolutional neural network algorithm to improve the detection accuracy and optimize the target detection results of stored grain pests.

[0070] Deep learning-based object detection methods can be broadly categorized into two types: two-stage and one-stage detection models. YOLOv5's one-stage detection algorithm is an end-to-end approach. After extracting features using a convolutional neural network, it directly regresses to obtain the object's class probability and location coordinates. While this algorithm has slightly lower detection accuracy than the two-stage algorithm, it offers a significant advantage in speed and is less prone to errors.

[0071] The YOLOv5 network architecture consists of four parts: input, backbone, neck, and prediction. This patent uses YOLOv5-5.0. Because stored grain pests are small targets, occupy a small proportion of the video, and contain few features, this patent uses the YOLOv5s network model to detect stored grain pests. YOLOv5s is the shallowest and fastest version of YOLOv5.

[0072] The use of deep convolutional neural networks significantly improves the success rate of detecting the location information of stored grain pests. It can also effectively detect overlapping stored grain pests. Furthermore, the deep learning-based object detection network extracts target features for detection, making it well-suited to various environments with minimal environmental influence. (See the instruction manual appendix.) Figure 7 However, YOLOv5s is significantly slower than traditional OpenCV. Therefore, it's necessary to incorporate conditional checks into the algorithm. Only when OpenCV detection fails will the video frame be passed to the YOLOv5s network for detection. This ensures both high detection accuracy and superior average processing speed. It also addresses the issue of low accuracy in extracting coordinates of stored grain pests, which is easily affected by environmental factors.

[0073] SORT, short for Simple Online And Realtime Tracking, is a multi-target tracking algorithm. Its main feature is its ability to quickly and in real-time correlate objects, with a speed of up to 260Hz. Since stored grain pests have limited feature information, SORT algorithm-based target tracking does not require any visual characteristics of the tracked target. It only needs the position and size information of the target detection box provided by any target detector to achieve motion estimation and data correlation.

[0074] The target detector using a combination of OpenCV and YOLOv5s in the target detection stage of this patent still fails to detect certain stored grain pests in a few sample frames. When these failed sample frames are fed into the SORT algorithm for association, matching boxes with incorrect values ​​appear. If the number of failed matching boxes exceeds a threshold, the boxes are deleted. Consequently, if a previously failed stored grain pest is detected again in a subsequent frame, it will be initialized and assigned a new ID because no matching box can be found, leading to tracking errors (see appendix to the specification). Figure 8 ).

[0075] The improved SORT algorithm does not delete the tracking boxes that failed to match. Instead, it saves their information. When the previously failed grain pest target is detected again in a subsequent frame, its detection box information is matched with the previously saved tracking box information to assign the correct ID to the grain pest, effectively solving the problem of ID assignment errors (see the instruction manual appendix). Figure 9 This patent improves the SORT algorithm, significantly enhancing the tracking accuracy of stored grain pests. It solves the problems of low tracking accuracy and tracking errors when multiple stored grain pests crawl to the same location and multiple targets overlap or are very close together.

[0076] This method for analyzing the behavior of stored grain pests, based on YOLOv5 and an improved SORT algorithm, utilizes a camera to capture and filter videos of stored grain pests crawling. The video is then processed frame by frame. First, each frame is preprocessed using OpenCV to detect the outline of the stored grain pests and extract their coordinate data. When overlapping or missing targets occur in a frame, causing some stored grain pests to fail to be detected, that frame is segmented and fed into the YOLOv5 target detection network for re-detection, optimizing the results. The improved SORT algorithm is then used for data association to track the movement trajectories of the stored grain pests. Finally, the movement trajectory of each stored grain pest is plotted, and the movement distance and real-time speed of each pest are calculated. This method solves the problems of low accuracy, susceptibility to environmental factors, and poor detection performance when overlapping targets exist in existing stored grain pest behavior analysis methods, making it particularly suitable for the needs of stored grain pest behavior analysis.

Claims

1. A grain storage pest behavior analysis method based on YOLOv5 and improved SORT algorithm, characterized by: Includes the following steps: Step 1: Collect sample videos. Preprocess the sample videos to remove redundant information and noise that affect detection, and highlight the outline of stored grain pests to obtain binarized images of stored grain pests. Step 2: Extract the coordinates of stored grain pests from the sample video frame by frame using a combination of traditional OpenCV and YOLOv5s. The steps are as follows: The binarized image from step 1 is used to extract the outline of stored grain pests using the OpenCV method. A threshold is set to extract the location information of stored grain pests in the image. The coordinates of the center point of the stored grain pests are calculated, and the location of the stored grain pests is drawn using a rectangle. A YOLOv5s network model was built and trained using the created dataset. The number of stored grain pests detected in each frame was judged to be consistent with the actual number. If they were inconsistent, the erroneous frame was divided into four sub-images: upper left, upper right, lower left, and lower right. These sub-images were then fed into the YOLOv5s network to obtain the location information of the stored grain pests in the sub-images. The absolute position of the stored grain pests in the sub-images relative to the original sample frames was then calculated using the relative position of the sub-images to the original sample frames, thus obtaining the center coordinates of the stored grain pests. Step 3: Target tracking; Step 2 uses a combination of OpenCV and YOLOv5s to detect stored grain pests in the sample video frame by frame. Then, the location coordinates of the stored grain pests are fed into the improved SORT algorithm frame by frame for inter-frame association matching, and corresponding IDs are assigned to the stored grain pests. Step 4: Trajectory plotting and behavior analysis; Step 3: The coordinates of stored grain pests extracted frame by frame are fed into the improved SORT algorithm for inter-frame matching and association. This identifies the correspondence between multiple stored grain pests in two adjacent sample frames and assigns corresponding IDs to the stored grain pests. A dictionary is created to store the center point coordinates of each ID of the stored grain pest across all n frames. Then, based on the center point coordinates of the stored grain pest with the same ID in the (n-1)th frame of the previous frame and the center point coordinates of the current frame's nth frame, the Euclidean distance between the center point coordinates of each stored grain pest in adjacent frames is calculated. As the distance traveled by the stored grain pest in the current frame, in the formula This represents the distance traveled by the stored grain pest in the current nth frame. , ),( , The coordinates (x and y) represent the center points of stored grain pests with the same ID in frames n and (n-1), respectively; the total walking distance is... In the formula This represents the distance traveled by stored grain pests in the i-th frame. This means that the travel distance of stored grain pests from frame 1 to frame n is added together to obtain the total travel distance; then the interval time of each frame is calculated based on the frame rate x. The reciprocal of the frame rate x is used as the interval between each frame; Current walking speed In the formula This represents the speed at which stored grain pests crawl in the nth frame. This represents the walking distance of the stored grain pest in the current nth frame, and t represents the time interval between each frame. Based on the center point coordinates of each stored grain pest with a different ID in the dictionary across all n frames, different colored trajectories are drawn, and their real-time movement speed and movement distance are displayed.

2. The method for analyzing the behavior of stored grain pests based on YOLOv5 and the improved SORT algorithm according to claim 1, characterized in that: The specific steps in step 1 are as follows: 1.1) Collect sample videos: Place one or more grain storage pests in an SPX250 biochemical incubator. The incubator does not provide a light source. The temperature inside the incubator is (30±1)℃ and the humidity is (45±5)%. Use a Sony IMX317 to film the crawling behavior of the grain storage pests. Use the built-in LED light of the camera to provide illumination. Select the group that did not exhibit flight behavior in the filmed video as the sample video. The video resolution is 1920×1080 and the frame rate is 24fps. 1.2) Video frame cropping: Extract sample frames one by one from the collected sample video to generate sample frames, and use OpenCV methods to crop the sample frames to remove areas outside the activity range of stored grain pests. 1.3) Grayscale conversion: Using OpenCV functions, the color space of the sample frame is converted from three dimensions to one dimension, and the RGB values ​​of each pixel are unified into the same value to form a grayscale image of the sample frame; 1.4) Binarization: Set a pixel threshold of 75, change the value of pixels with a gray value lower than the threshold of 75 to 0, and set the value of pixels with a gray value higher than the threshold of 75 to 255, so that the gray image presents a clear 'black and white' effect, highlighting the outline of the grain storage pest in the gray image, and converting the gray image into a binarized image. 1.5) Dilation: Dilate the white area in the binarized image to remove noise information and segment out the outline of each grain storage pest. 1.6) Erosion: OpenCV is used to perform erosion on the dilated binarized image to expand the outline boundary of the black grain storage pests and enlarge the outline area, which facilitates subsequent extraction; at this point, the sample frame can be transformed into a preprocessed binarized image of grain storage pests.

3. The method for analyzing the behavior of stored grain pests based on YOLOv5 and the improved SORT algorithm according to claim 1, characterized in that: Step 3, target tracking, involves assigning a corresponding ID to each stored grain pest as follows: SORT Algorithm: The SORT algorithm is a multi-target tracking algorithm. The two most important methods in the algorithm are the Hungarian algorithm and Kalman filtering. Frame 1: The location coordinates of stored grain pests detected by the target detector combining OpenCV and YOLOv5s are used as detection boxes. Since there are no tracks at this time, the information in the detection boxes is directly updated and initialized as tracking boxes using Kalman filtering in the first frame. Frame N: Where N>1: The prediction results of the tracking boxes in frame N-1 are obtained using Kalman filtering. These predictions are then combined with the coordinates of stored grain pests from the target detector in frame N to form new detection boxes. The intersection-over-union ratio (IoU) between the predicted detection boxes and tracking boxes is calculated as the cost matrix. The Hungarian algorithm is used to achieve association, and three matching scenarios may occur: Tracking boxes that fail to match: Determine whether to delete them based on a set threshold; For detection boxes that fail to match: Kalman filter initialization, assign a new ID to the box, and add its information to the tracking box; The prediction results of the detection box and the tracking box are successfully matched: assign the corresponding ID to the successfully associated detection box, and then update the weights and parameters of its Kalman filter; The improved SORT algorithm: Instead of setting a threshold and deleting failed tracking boxes, it records them as `miss_track`. When previously detected grain pests are re-detected in subsequent frames, their detection boxes are associated with the information in `miss_track`. The Euclidean distances between the m failed detection boxes and the n failed tracking boxes in `miss_track` are calculated. ； In the formula: This represents the Euclidean distance between the i-th unmatched detection box and the j-th unmatched tracking box; , ),( , ) represent the horizontal and vertical coordinates of the center points of the failed detection box information of the i-th grain storage pest and the failed tracking box information in the j-th miss_track, respectively; Based on the Euclidean distance, re-match the failed detection box information of the i-th grain storage pest with the Euclidean distance between it and the target. The smallest missing track is associated with the failed matching bounding box in the j-th miss_track, thus preventing the association error due to the failure of some frame detection.

Images