Vehicle multi-target detection tracking method based on YOLOv8 + DeepSORT + LSTM

Abstract

The invention discloses a vehicle multi-target detection tracking method based on YOLOv8 + DeepSORT + LSTM, and the method comprises the steps: carrying out the vehicle target detection of an input video frame or image through employing the YOLOv8, obtaining the position, type and other information of a vehicle, and inputting the target information detected by the YOLOv8 into a DeepSORT algorithm for preliminary tracking. According to the vehicle multi-target detection and tracking method based on YOLOv8 + DeepSORT + LSTM, the vehicle may reflect light under strong light and become fuzzy in details under weak light, the model can better adapt to illumination changes in actual scenes after being trained by simulating illumination changes, accurate detection and tracking are performed when similar illumination conditions are encountered, and the vehicle multi-target detection and tracking method based on YOLOv8 + DeepSORT + LSTM is more accurate. For the texture features of the vehicle, the visual expressions of textures under different illumination are different, and after illumination change training, the model can express the texture features of the vehicle in a more comprehensive and accurate manner, thereby improving the accuracy of vehicle detection and tracking.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM. Background Art

[0002] YOLOv8 is an advanced target detection algorithm with the characteristics of fast detection and high accuracy. DeepSORT is a tracking algorithm based on Kalman filtering and Hungarian algorithm, which can track targets online in real time. LSTM, as a recurrent neural network, is good at processing time series data. In target tracking scenarios, the historical information of the target can be used to improve the stability and accuracy of tracking. In actual traffic scenarios, lighting conditions are constantly changing. Different lighting conditions such as day, night, cloudy and sunny days will have a great impact on the quality of vehicle images, which may make it difficult for vision-based detection algorithms to accurately extract vehicle features. For example, under strong light, some parts of the vehicle may reflect light, affecting YOLOv8's accurate detection of the vehicle bounding box; at night or in dim light, the contrast of the image is reduced, the features of the vehicle become less obvious, and the difficulty of detection is also increased. Summary of the invention

[0003] 1. Technical issues to be solved

[0004] The purpose of the present invention is to provide a vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM to solve the problem that the lighting conditions mentioned in the above background technology are constantly changing. Different lighting conditions such as day, night, cloudy days, sunny days, etc. will have a great impact on the quality of vehicle images, which may cause vision-based detection algorithms to be difficult to accurately extract vehicle features. For example, under strong light, some parts of the vehicle may reflect light, affecting YOLOv8's accurate detection of the vehicle boundary box; at night or in dim light, the contrast of the image is reduced, the features of the vehicle become unclear, and the difficulty of detection is increased.

[0005] (II) Technical solution

[0006] To achieve the above object, the present invention provides the following technical solution: a vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM, comprising using YOLOv8 to detect vehicle targets on input video frames or images, obtaining information such as the location and category of the vehicle, then inputting the target information detected by YOLOv8 into the DeepSORT algorithm for preliminary tracking, and finally, optimizing the tracking results of DeepSORT through LSTM, and using the historical trajectory information of the vehicle in continuous frames to adjust and correct the tracking results, thereby achieving more accurate and stable vehicle multi-target detection and tracking, comprising the following steps:

[0007] Step 1: Use YOLOv8 to filter videos;

[0008] Step 2: Carry out vehicle target detection;

[0009] Step 3: Get vehicle information;

[0010] Step 4: Preliminary tracking through DeepSORT algorithm;

[0011] Step 5: Simulate lighting changes;

[0012] Step 6: Process the tracking results;

[0013] Step 7: Adjust and correct the tracking structure.

[0014] Preferably, make sure that PyTorch is installed, because YOLOv8 is implemented based on PyTorch. Select the appropriate PyTorch version for installation according to the operating system and hardware, and install other necessary libraries, such as numpy for numerical operations, opencv-python for video processing and image operations, etc. You can get the pre-trained models of YOLOv8 from the official code library. These pre-trained models have been trained on large-scale datasets and have good detection performance. If you need to optimize for specific scenarios, you can also prepare the dataset yourself and retrain the YOLOv8 model.

[0015] Preferably, basic information such as the frame rate, width, and height of the video is obtained. The frame rate determines the speed of video playback. This information is very important for subsequent processing and result display. It may be necessary to normalize the video frames and normalize the pixel values ​​to a range of 0-1 or -1 to 1 to meet the input requirements of the YOLOv8 model. The YOLOv8 model is loaded and the parameters of the model are set as needed, such as the detection threshold and the non-maximum suppression threshold. The detection threshold determines the minimum confidence that the model considers to be a target, and the non-maximum suppression threshold is used to remove overlapping detection frames.

[0016] Preferably, the vehicle target information is extracted from the detection results. The vehicle detection results of YOLOv8 usually classify the vehicles into categories such as 'car', 'truck', and 'bus'. The vehicle-related detection results are screened out according to the project requirements. The detection box coordinates can be further processed, such as converting the coordinates into a format more suitable for subsequent analysis or visualization. The vehicle detection box is drawn on the video frame using opencv-python. According to the bounding box coordinates in the detection results, the vehicle is drawn with a rectangular box on the original video frame, and the vehicle category and confidence are marked.

[0017] Preferably, the video frames are read in a loop until the video ends. For each frame, the vehicle detection result is input into the DeepSORT tracker. Assuming that detections is the vehicle detection result of the current frame, useful information is extracted from the tracking result. The information of each tracking target may include the tracking ID, the vehicle bounding box position of the current frame, etc. The tracking results can be stored in a data structure to facilitate subsequent analysis, such as statistics on the vehicle's trajectory, speed and other information.

[0018] Preferably, data augmentation technology is used in the training data set to simulate vehicle images under different lighting conditions. For example, vehicle samples under different lighting scenarios are generated by changing the brightness, contrast, saturation and hue of the image. Image processing libraries such as OpenCV can be used to implement these operations. The brightness of the original training image is increased or decreased to simulate strong light during the day and weak light environments on cloudy days; different light reflection conditions are simulated by adjusting the contrast; saturation and hue are changed to simulate light colors in different weather conditions. A generative adversarial network (GAN) is used to generate more realistic vehicle images under lighting change scenarios. The generator in the GAN can learn the distribution of real lighting changes. The lighting change images generated by the generator are used together with images of real vehicles under lighting change scenarios as training data, which can improve the robustness of the model to lighting changes.

[0019] Preferably, opencv-python is used to draw the tracking result on the video frame, and according to the vehicle boundary box and tracking ID in the tracking result, the vehicle is drawn with a rectangular box on the original video frame, and the tracking ID is marked.

[0020] Preferably, the historical trajectory information of the vehicle in the continuous frames is finally used to adjust and correct the tracking result, so as to achieve more accurate and stable vehicle multi-target detection and tracking.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: vehicles may reflect light under strong light, and details may become blurred under weak light. After being trained by simulating lighting changes, the model can better adapt to lighting changes in these actual scenes, and make accurate detection and tracking when encountering similar lighting conditions, thereby improving the robustness of the model in the real environment. After being trained by simulating lighting changes, the model can maintain good performance in such lighting transition scenarios and reduce dependence on specific lighting environments. For the texture features of the vehicle, the visual representation of the texture is different under different lighting. After being trained by lighting changes, the model can represent the texture features of the vehicle in a more comprehensive and accurate manner, thereby improving the accuracy of vehicle detection and tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the process structure of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 The present invention provides a technical solution: a vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM, including using YOLOv8 to detect vehicle targets on input video frames or images, obtaining information such as the location and category of the vehicle, and then inputting the target information detected by YOLOv8 into the DeepSORT algorithm for preliminary tracking. Finally, the tracking result of DeepSORT is optimized by LSTM, and the historical trajectory information of the vehicle in continuous frames is used to adjust and correct the tracking result, thereby achieving more accurate and stable vehicle multi-target detection and tracking: It is characterized in that it includes the following steps:

[0025] Step 1: Use YOLOv8 to filter videos;

[0026] Step 2: Carry out vehicle target detection;

[0027] Step 3: Get vehicle information;

[0028] Step 4: Preliminary tracking through DeepSORT algorithm;

[0029] Step 5: Simulate lighting changes;

[0030] Step 6: Process the tracking results;

[0031] Step 7: Adjust and correct the tracking structure.

[0032] Make sure PyTorch is installed, because YOLOv8 is implemented based on PyTorch. Select the appropriate PyTorch version to install according to the operating system and hardware. Install other necessary libraries, such as numpy for numerical operations, opencv-python for video processing and image operations, etc. You can get the pre-trained models of YOLOv8 from the official code library. These pre-trained models have been trained on large-scale datasets and have good detection performance. If you need to optimize for specific scenarios, you can also prepare the dataset yourself and retrain the YOLOv8 model. Read the video frames in a loop until the video ends. For each frame of the image, detect the vehicle. The result is input into the DeepSORT tracker. Assuming that detections is the vehicle detection result of the current frame, useful information is extracted from the tracking result. The information of each tracked target may include the tracking ID, the vehicle bounding box position of the current frame, etc., and basic information such as the frame rate, width, and height of the video is obtained. The frame rate determines the speed of video playback. This information is very important for subsequent processing and result display. It may be necessary to normalize the video frame and normalize the pixel value to the range of 0-1 or -1 to 1 to meet the input requirements of the YOLOv8 model. Load the YOLOv8 model and set the model parameters as needed, such as the detection threshold and non-maximum suppression threshold. The detection threshold determines the model The model is considered to be the lowest confidence of the target. The non-maximum suppression threshold is used to remove overlapping detection frames. The tracking results can be stored in a data structure to facilitate subsequent analysis, such as statistics on vehicle trajectories, speeds, and other information. Data enhancement technology is used in the training data set to simulate vehicle images under different lighting conditions. For example, by changing the brightness, contrast, saturation, and hue of the image to generate vehicle samples under different lighting scenes, image processing libraries such as OpenCV can be used to implement these operations. The brightness of the original training image is increased or decreased to simulate strong light during the day and weak light environments on cloudy days; the contrast is adjusted to simulate different light reflection conditions; the saturation and hue are changed to simulate different weather conditions. The light color under the scene is used, and the generative adversarial network (GAN) is used to generate more realistic vehicle images in scenes with changing lighting. The generator in GAN can learn the distribution of real lighting changes. The lighting change images generated by the generator and the images of real vehicles in scenes with changing lighting are used as training data, which can improve the robustness of the model to lighting changes. Opencv-python is used to draw the tracking results on the video frames. According to the vehicle bounding box and tracking ID in the tracking results, the vehicle is drawn with a rectangular box on the original video frame, and the tracking ID is marked. Finally, the historical trajectory information of the vehicle in consecutive frames is used to adjust and correct the tracking results, thereby achieving more accurate and stable vehicle multi-target detection and tracking.

[0033] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Simple modifications or equivalent substitutions of the technical solution of the present invention by ordinary technicians in this field do not deviate from the essence and scope of the technical solution of the present invention.

Claims

1. A vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM, including using YOLOv8 to detect vehicle targets on input video frames or images, obtaining information such as the location and category of the vehicle, then inputting the target information detected by YOLOv8 into the DeepSORT algorithm for preliminary tracking, and finally optimizing the tracking results of DeepSORT through LSTM, and using the historical trajectory information of the vehicle in continuous frames to adjust and correct the tracking results, thereby achieving more accurate and stable vehicle multi-target detection and tracking: It is characterized by: The steps include: Step 1: Use YOLOv8 to filter videos; Step 2: Carry out vehicle target detection; Step 3: Get vehicle information; Step 4: Preliminary tracking through DeepSORT algorithm; Step 5: Simulate lighting changes; Step 6: Process the tracking results; Step 7: Adjust and correct the tracking structure.

2. A vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM according to claim 1, characterized in that: Make sure PyTorch is installed, because YOLOv8 is implemented based on PyTorch. Select the appropriate PyTorch version to install according to the operating system and hardware. Install other necessary libraries, such as numpy for numerical operations, opencv-python for video processing and image operations, etc. You can get the pre-trained models of YOLOv8 from the official code library. These pre-trained models have been trained on large-scale datasets and have good detection performance. If you need to optimize for specific scenarios, you can also prepare the dataset yourself and retrain the YOLOv8 model.

3. A vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM according to claim 2, characterized in that: Get basic information about the video, such as the frame rate, width, and height. The frame rate determines the speed at which the video is played. This information is important for subsequent processing and result presentation. You may need to normalize the video frames and normalize the pixel values ​​to the range of 0-1 or -1 to 1 to meet the input requirements of the YOLOv8 model. Load the YOLOv8 model and set the model parameters as needed, such as the detection threshold and non-maximum suppression threshold. The detection threshold determines the minimum confidence that the model considers to be a target. The non-maximum suppression threshold is used to remove overlapping detection boxes.

4. A vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM according to claim 3, characterized in that: Extract vehicle target information from the detection results. YOLOv8's vehicle detection results usually classify vehicles into categories such as 'car', 'truck', and 'bus'. Filter out vehicle-related detection results according to project requirements. You can further process the detection box coordinates, such as converting the coordinates into a format more suitable for subsequent analysis or visualization. Use opencv-python to draw the vehicle detection box on the video frame. According to the bounding box coordinates in the detection results, draw the vehicle with a rectangular box on the original video frame, and mark the vehicle category and confidence.

5. A vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM according to claim 4, characterized in that: Read video frames in a loop until the video ends. For each frame, input the vehicle detection result into the DeepSORT tracker. Assume that detections is the vehicle detection result of the current frame. Extract useful information from the tracking result. The information of each tracked target may include the tracking ID, the vehicle bounding box position of the current frame, etc. The tracking results can be stored in a data structure for subsequent analysis, such as statistics on the vehicle's trajectory, speed, and other information.

6. A vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM according to claim 5, characterized in that: Data augmentation technology is used in the training data set to simulate vehicle images under different lighting conditions. For example, vehicle samples under different lighting scenarios are generated by changing the brightness, contrast, saturation and hue of the image. Image processing libraries such as OpenCV can be used to implement these operations. The brightness of the original training images is increased or decreased to simulate strong light during the day and weak light environments on cloudy days; different light reflection conditions are simulated by adjusting the contrast; saturation and hue are changed to simulate the light color in different weather conditions. Generative adversarial networks (GANs) are used to generate more realistic vehicle images under lighting change scenarios. The generator in GAN can learn the distribution of real lighting changes. The lighting change images generated by the generator are used together with the images of real vehicles under lighting change scenarios as training data, which can improve the robustness of the model to lighting changes.

7. A vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM according to claim 6, characterized in that: Use opencv-python to draw the tracking results on the video frame. According to the vehicle bounding box and tracking ID in the tracking result, draw the vehicle with a rectangular box on the original video frame and mark the tracking ID.

8. A vehicle multi-target detection and tracking method based on YOLOv8+DeepSORT+LSTM according to claim 7, characterized in that: Finally, the historical trajectory information of the vehicle in continuous frames is used to adjust and correct the tracking results, thereby achieving more accurate and stable vehicle multi-target detection and tracking.

Images