A target detection and tracking method combining a YOLOv8 detection algorithm and a KCF tracking algorithm

Abstract

The application relates to the field of target detection and tracking, and provides a target detection and tracking method combining a YOLOv8 detection algorithm and a KCF tracking algorithm. The method first acquires a video stream time sequence set of a tracking image of a target vehicle object, uses the YOLOv8 to perform target detection on a first frame to initialize a KCF tracker, then the KCF tracker processes subsequent frame images to perform target tracking, if tracking fails in the tth frame, the YOLOv8 is re-enabled to detect the frame to attempt to find back the target, if the target is successfully detected, a new target vehicle object detection image is generated and the KCF tracker is reinitialized, and if the target cannot be detected, a target vehicle object detection image of the tth frame is generated based on information of the previous t-1 frames. In this way, the continuity and accuracy of target vehicle object tracking can be effectively ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection and tracking, and more particularly, to a target detection and tracking method combining YOLOv8 detection algorithm and KCF tracking algorithm. BACKGROUND

[0002] In the fields of intelligent transportation systems, autonomous vehicles, and video surveillance, accurately and efficiently detecting and tracking moving targets such as vehicles, pedestrians, etc. is of great importance. Precise target detection and tracking not only enhances the overall performance of the system, but also ensures traffic safety, optimizes traffic flow, and promptly discovers abnormal events, providing reliable protection for intelligent monitoring and autonomous driving.

[0003] However, existing single target detection or tracking algorithms have obvious limitations. Specifically, traditional target detection algorithms such as the YOLO series can identify target vehicles in static images, but in video streams, the computational load is enormous as each frame of image needs to be comprehensively detected, making it difficult to meet real-time requirements. At the same time, while individual tracking algorithms such as KCF have higher computational efficiency during tracking, they are easily lost when the target vehicle changes appearance, is obscured, or moves quickly, leading to tracking failure. Moreover, KCF relies on high-quality detection results from the initial frame, and if the initialization is inaccurate, the subsequent tracking effect will be affected.

[0004] Therefore, an optimized target detection and tracking solution is desired. SUMMARY

[0005] The present application provides a target detection and tracking method combining YOLOv8 detection algorithm and KCF tracking algorithm to address the shortcomings of the prior art.

[0006] According to one aspect of the present application, a target detection and tracking method combining YOLOv8 detection algorithm and KCF tracking algorithm is provided, which includes:

[0007] Obtaining a video stream time sequence set of tracking images of target vehicle objects;

[0008] Using the YOLOv8 detection algorithm to perform target detection on a first frame tracking image in the video stream time sequence set of tracking images to obtain a target vehicle object detection image;

[0009] Initializing a KCF tracker based on the target vehicle object detection image, and using the initialized KCF tracker to perform target tracking on other frame tracking images in the video stream time sequence set of tracking images;

[0010] In response to the KCF tracker reporting that the target tracking fails to be displayed in a t-th frame tracking image in a video stream timing set of the tracking images, a YOLOv8 detection algorithm is used to re-detect the target in the t-th frame tracking image to obtain a target detection result;

[0011] In response to the target detection result being that the target vehicle object is re-detected, a t-th frame target vehicle object detection image is generated, and the KCF tracker is re-initialized based on the t-th frame target vehicle object detection image;

[0012] In response to the target detection result being that the target vehicle object is not detected, a t-th frame target vehicle object detection image is generated based on t-1 frames of tracking images before the t-th frame tracking image.

[0013] The application has the following technical solutions, which has the following technical effects:

[0014] The target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm provided by the application first acquires a video stream timing set of tracking images of a target vehicle object, and uses YOLOv8 to detect a first frame to initialize a KCF tracker, then the KCF tracker processes subsequent frames of images to track the target, if tracking fails in a t-th frame, YOLOv8 is re-enabled to detect the frame to try to find the target, if the target is successfully detected, a new target vehicle object detection image is generated and the KCF tracker is re-initialized, and if the target is not detected, a t-th frame target vehicle object detection image is generated based on information of the previous t-1 frames. In this way, the continuity and accuracy of the target vehicle object tracking can be effectively ensured. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application, taken in conjunction with the accompanying drawings. The drawings provided in the specification and the embodiments of the present application together serve to provide a further understanding that enables others skilled in the art to make or use the present application. The drawings provided are for illustrative purposes and are not intended to limit the present application in any way. In the drawings, like reference numerals refer to the same components throughout the several views.

[0016] Figure 1 A flowchart of the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to the embodiments of the present application.

[0017] Figure 2 A flowchart of step S6 in the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to the embodiments of the present application.

[0018] Figure 3 This is a flowchart of step S61 in the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to an embodiment of the present application.

[0019] Figure 4 This is a flowchart of step S64 in the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to an embodiment of the present application.

[0020] Figure 5 This is a flowchart of step S642 in the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0022] In areas such as intelligent transportation systems, self-driving cars, and video surveillance, accurate and efficient detection and tracking of moving targets such as vehicles and pedestrians is extremely critical. This not only improves system performance, but also ensures traffic safety, optimizes traffic flow, and promptly detects anomalies, providing a solid foundation for intelligent monitoring and autonomous driving. However, existing single algorithms have limitations: although traditional target detection algorithms (such as the YOLO series) can identify targets in static images, when processing video streams, they require comprehensive detection of each frame, which is computationally intensive and difficult to achieve real-time performance. While standalone tracking algorithms (such as KCF) have high tracking efficiency, they are prone to losing targets when their appearance changes, they are obscured, or they move rapidly. Furthermore, their effectiveness is highly dependent on high-quality detection results from the initial frame, and inaccurate initialization can significantly impact subsequent tracking results.

[0023] It should be understood that the YOLOv8 detection algorithm is a deep learning-based target detection algorithm, which belongs to a single-stage target detection algorithm. It continues the core idea of the YOLO series algorithm, that is, it predicts the category and bounding box position of the target through a neural network, and can complete the target detection task in one forward propagation process. YOLOv8 optimizes and improves the architecture design, adopts a more efficient backbone network, neck network and detection head structure, and has faster detection speed and higher detection accuracy. Since YOLOv8 can complete the detection of targets in images in a very short time, it is suitable for fast target positioning in real-time video streams. The KCF tracking algorithm is a target tracking algorithm based on a correlation filter. It realizes efficient single-target tracking by using a circulant matrix and fast Fourier transform (FFT). Moreover, the algorithm uses a kernel method to process nonlinear features, making it robust in complex scenes. YOLOv8 and KCF can be combined and applied in a target detection and tracking algorithm. Specifically, YOLOv8 is responsible for quickly and accurately detecting targets, and KCF uses the detection results for efficient and stable tracking.

[0024] Based on this, the present application proposes a target detection and tracking method combining YOLOv8 detection algorithm and KCF tracking algorithm, which combines the two algorithms by combining advanced technologies, solves the limitations of the prior art, and realizes accurate, stable and real-time detection and tracking of target vehicles. Figure 1 The flowchart of the target detection and tracking method combining YOLOv8 detection algorithm and KCF tracking algorithm according to the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the target detection and tracking method combining YOLOv8 detection algorithm and KCF tracking algorithm according to the embodiments of the present application includes the following steps: Figure 1As shown, according to the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to the embodiment of the present application, the method includes: S1, obtaining a video stream time sequence set of a tracking image of a target vehicle object; S2, using the YOLOv8 detection algorithm to perform target detection on the first frame tracking image in the video stream time sequence set of the tracking image to obtain a target vehicle object detection image; S3, initializing a KCF tracker based on the target vehicle object detection image, and using the initialized KCF tracker to perform target tracking on other frame tracking images in the video stream time sequence set of the tracking image; S4, in response to the KCF tracker report in the tracking image, The t-th frame tracking image in the video stream time series set of the image shows that target tracking fails, and the YOLOv8 detection algorithm is used to re-perform target detection on the t-th frame tracking image to obtain a target detection result; S5, in response to the target detection result that the target vehicle object is re-detected, the t-th frame target vehicle object detection image is generated, and the KCF tracker is reinitialized based on the t-th frame target vehicle object detection image; S6, in response to the target detection result that the target vehicle object is not detected, the target vehicle object is predicted based on the t-1 tracking images before the t-th frame tracking image to generate the t-th frame target vehicle object detection image.

[0025] In step S1, a time-series video stream of a tracking image of a target vehicle object is acquired. It should be understood that this time-series video stream of the tracking image of the target vehicle object provides the necessary input data for the subsequent target detection and tracking process. Specifically, the time-series video stream of the tracking image of the target vehicle object includes the target vehicle to be analyzed, detected, and tracked. By acquiring a continuous video stream, dynamic monitoring of the target vehicle can be performed, not just limited to information in a static image, but also capable of tracking changes in the target vehicle object across both temporal and spatial dimensions.

[0026] In step S2, the YOLOv8 detection algorithm is used to perform target detection on the first frame of the tracking image in the video stream time series set of the tracking image to obtain a target vehicle object detection image. Accordingly, considering that YOLOv8 is an advanced target detection algorithm, it has high accuracy and fast detection speed in target detection tasks. In the first frame image, the target vehicle may be in a complex scene and there may be other interference factors around it, such as other vehicles, pedestrians, obstacles, etc. YOLOv8 can use its powerful feature extraction and classification capabilities to accurately identify the target vehicle object and eliminate the influence of other interference factors, thereby providing reliable target information for subsequent tracking.

[0027] The following is a detailed description of a specific implementation process of "detecting the first frame tracking image in the video stream time sequence set using the YOLOv8 detection algorithm to obtain a target vehicle object detection image":

[0028] First, select a specific model. YOLOv8 has multiple versions, such as YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8 l, and YOLOv8x, each with different model sizes, speeds, and accuracies. Choose the appropriate version based on the specific application scenario and performance requirements. If real-time performance is important, choose the lightweight YOLOv8n, or if high accuracy is required, choose the larger YOLOv8x. Usually, pre-trained models on large-scale datasets such as COCO are loaded. Pre-trained models contain rich general target feature information, which can shorten the training time and improve the detection effect.

[0029] Next, perform image preprocessing operations. Extract the first frame tracking image from the video stream time sequence set and read it into a processable data format using an image processing library. To adapt to the input requirements of the YOLOv8 model, adjust the image size to the specified size. Uniform size can improve processing efficiency and detection stability. At the same time, normalize the image pixel values to the range of [0, 1] or [-1, 1], reducing the influence of factors such as light changes.

[0030] After that, enter the target detection inference stage. Input the preprocessed first frame image into the loaded YOLOv8 model. Its backbone network will extract features from the image, converting the input image into feature maps of different scales and abstraction levels through convolution, pooling, and other operations. These feature maps contain edge, texture, shape, and other feature information of the target. The feature maps are then processed by the neck network, which fuses and enhances feature maps of different scales through the PANet structure, effectively fusing high-level semantic features and low-level detail features, and improving the model's ability to detect targets of different sizes. Finally, the processed feature maps enter the detection head, which analyzes the feature maps, predicts the class, bounding box position, and confidence of the target in the image, and predicts multiple bounding boxes on each grid point of the feature map, assigning class probabilities and confidence scores to them.

[0031] After completing the inference, post-processing is required. Since the detection head will predict a large number of prediction boxes, many of which may overlap and point to the same target, non-maximum suppression is used to remove redundant prediction boxes. First, sort the prediction boxes by confidence, and select the highest confidence as the prediction box. Calculate the intersection over union of other prediction boxes with the prediction box, and filter out prediction boxes with an overlap greater than a threshold. At the same time, set a confidence threshold to filter out prediction boxes with a confidence lower than the threshold, removing possible false detections. Then, according to the detection target class, select the prediction boxes that belong to the vehicle class.

[0032] Finally, the target vehicle object detection image is generated. Since the target vehicle refers to a vehicle in this application, among the filtered vehicle bounding boxes, the prediction box with the highest confidence is selected as the detection result of the target vehicle object. According to the prediction box information of the filtered target vehicle object, a bounding box is drawn on the first frame tracking image to visually display the position of the target vehicle. A class label (such as "vehicle") and a confidence score of the target vehicle are added near the bounding box to facilitate the user's understanding of the accuracy of the detection result. Then, the first frame tracking image with the drawn bounding box and label is saved as the target vehicle object detection image.

[0033] In step S3, the KCF tracker is initialized based on the target vehicle object detection image, and the initialized KCF tracker is used to track the target in other frame tracking images in the video stream time sequence set of the tracking images. It should be understood that KCF is a high-efficiency tracking algorithm based on kernel correlation filtering, which can significantly reduce the demand for computing resources while maintaining high tracking accuracy. By using the initial target position information provided by the YOLOv8 detection algorithm, the KCF tracker can be quickly and accurately initialized. In a video stream, the target vehicle usually appears in multiple consecutive frames. Initializing the KCF tracker based on the target detection result of the first frame can ensure continuous and stable tracking of the target in subsequent frames, effectively dealing with slight deformation or occlusion of the target.

[0034] The following is a detailed description of a specific implementation process of "initializing the KCF tracker based on the target vehicle object detection image":

[0035] When initializing the KCF tracker based on the target vehicle object detection image, the target information needs to be accurately extracted from the target vehicle object detection image. The target vehicle in the target vehicle object detection image is labeled in the form of a bounding box. The left upper corner coordinates (x, y), width w, and height h of the bounding box are extracted, which accurately define the position and range of the target vehicle in the image. At the same time, features are extracted from the target region enclosed by the bounding box. KCF tracker commonly uses grayscale or color features. If grayscale features are used, the color image will be converted to a grayscale image to highlight shape information and reduce data volume; when color features are used, the target color characteristics are described according to different color spaces such as RGB and HSV, providing rich information for subsequent tracking.

[0036] Next, the extracted target features are used to construct a target template. The KCF tracker is based on kernel correlation filtering technology and needs to construct a template that can represent the target vehicle. In the initialization stage, the extracted features are used to construct an initial kernel correlation filtering model. This model learns the target appearance pattern by calculating the correlation between target features. Specifically, the target features are mapped to a high-dimensional space, and the similarity is calculated with the help of kernel functions such as Gaussian kernels to obtain the target's related filtering response. Subsequently, the extracted target features are used to train the correlation filter. During the training process, the filter parameters are continuously adjusted to minimize the error between the predicted output and the expected output. The expected output is generally a two-dimensional Gaussian distribution with a peak around the center of the target, which helps the filter learn the target's position information in the image.

[0037] Next, the initial state of the KCF tracker is set. The target vehicle's initial position in the first frame is determined based on the target bounding box information. This serves as the starting point for subsequent tracking. Key parameters for the KCF tracker are also set. The learning rate determines how quickly the tracker updates the target template. A higher learning rate allows the tracker to quickly adapt to changes in the target's appearance but is more susceptible to noise. A lower learning rate makes the tracker more stable but may not be able to keep up with rapid changes in the target. The search area size is also crucial, as it determines the range within which the tracker searches for the target in subsequent frames. An appropriate size improves tracking efficiency and accuracy.

[0038] Finally, verify the initialization results. After initialization, carefully examine the extracted target features and constructed target template. Visualize the target features or observe the relevant filter responses to determine the effectiveness of the initialization. If the target features are blurred or the filter responses are abnormal, reinitialize. Before actual tracking, use the initialized tracker to simulate tracking on the current frame to verify that the target vehicle can be accurately located. If the simulated tracking results are unsatisfactory, thoroughly review all aspects of the initialization process to identify any issues and make adjustments.

[0039] In step S4, in response to the KCF tracker reporting that the target tracking fails to be displayed in the t-th frame tracking image in the video stream timing set of the tracking images, a target detection result is obtained by re-performing target detection on the t-th frame tracking image using the YOLOv8 detection algorithm. It should be understood that although the KCF tracking algorithm has high tracking efficiency, it also has some limitations. For example, when the target vehicle is severely occluded, moves quickly, has a dramatic change in appearance (such as a large difference in appearance due to a change in viewing angle when the vehicle turns), or the lighting conditions change dramatically, the KCF tracker can lose the target, resulting in tracking failure. Once this happens, the KCF itself cannot continue to accurately track the target, and a more powerful target detection mechanism needs to be introduced. At this time, reusing the powerful target detection algorithm YOLOv8 can help the system reposition and recover the lost target. That is, YOLOv8 performs comprehensive feature extraction and target classification on the entire image at each detection, and does not rely on the tracking information of the previous frame. Therefore, when the KCF tracking fails, re-detecting the current frame using YOLOv8 can reposition the target vehicle from a global perspective, unaffected by the previous tracking failure.

[0040] In step S5, in response to the target detection result being that the target vehicle object is re-detected, a t-th frame target vehicle object detection image is generated, and the KCF tracker is reinitialized based on the t-th frame target vehicle object detection image. Accordingly, considering that the target model and position information stored therein can no longer accurately reflect the current state of the target vehicle when re-detection is performed. This can be due to the target being occluded, moving quickly, having a large change in appearance, or changes in lighting conditions, so that the target features and position information learned by the KCF tracker based on the previous frame are no longer applicable. Therefore, it is necessary to reinitialize it using accurate target information.

[0041] In step S6, in response to the target detection result being that the target vehicle object is not detected, a t-th frame target vehicle object detection image is generated based on t-1 tracking images before the t-th frame tracking image. It should be understood that in the target detection and tracking task, continuously tracking the target vehicle is a key requirement. The t-1 tracking images before contain rich historical information of the target vehicle, such as shape, color, etc. These information can reflect the motion law and trend of the target vehicle, and using these historical information for prediction can make up for the information loss caused by the detection failure of the current frame to some extent. For example, according to the shape and color of the target vehicle in the past few frames, its possible position in the t-th frame can be roughly guessed.

[0042] Based on this, the technical concept of this application is to use machine vision-based image analysis and feature extraction technology to perform HOG feature extraction and color histogram calculation on each of the first t-1 tracking images, and then perform semantic capture and multi-dimensional semantic fusion on the extracted directional gradient histograms of each tracking image and the color histograms of each tracking image, so as to automatically predict and obtain the target vehicle object detection image of the t-th frame based on the dynamic causal context walk representation between the multi-dimensional semantic fusion representation features of each fused tracking image. In this way, through HOG feature extraction and color histogram calculation, the shape and color information of the target vehicle can be fully described from different angles. At the same time, through the dynamic causal context walk representation, the dynamic changes of the target vehicle in time and its interaction with the environment can be captured, thereby improving the understanding of the target vehicle behavior and the prediction accuracy.

[0043] Specifically, Figure 2 Flowchart of step S6 in the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to an embodiment of the present application. Figure 2 As shown, the step S6 comprises: S61, performing HOG feature extraction and color histogram calculation on each tracking image in the t-1 tracking images to obtain a sequence of tracking image directional gradient histograms and a sequence of tracking image color histograms; S62, performing histogram feature extraction on the sequence of tracking image directional gradient histograms and the sequence of tracking image color histograms to obtain a sequence of tracking image directional gradient histogram semantic feature vectors and a sequence of tracking image color histogram semantic feature vectors; S63, performing histogram feature extraction on the sequence of tracking image directional gradient histograms and the sequence of tracking image color histograms to obtain a sequence of tracking image directional gradient histogram semantic feature vectors and a sequence of tracking image color histogram semantic feature vectors. Each group of corresponding tracking image directional gradient histogram semantic feature vectors and tracking image color histogram semantic feature vectors in the sequence of feature vectors and the sequence of tracking image color histogram semantic feature vectors are cascaded to obtain a sequence of tracking image multi-dimensional semantic fusion representation vectors; S64, causal modeling of tracking image dynamic wandering is performed on the sequence of tracking image multi-dimensional semantic fusion representation vectors to obtain a tracking image context dynamic wandering semantic coding vector; S65, based on the tracking image context dynamic wandering semantic coding vector, the target vehicle object detection image of the tth frame is obtained.

[0044] In step S61, HOG feature extraction and color histogram calculation are performed on each of the t-1 tracking images to obtain a sequence of tracking image directional gradient histograms and a sequence of tracking image color histograms. Specifically, Figure 3 Flowchart of step S61 in the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to an embodiment of the present application. Figure 3As shown, the step S61 includes: S611, performing HOG feature extraction on each tracking image in the t-1 tracking images to obtain a sequence of directional gradient histograms of the tracking images; S612, calculating the color histogram of each tracking image in the t-1 tracking images to obtain a sequence of color histograms of the tracking images.

[0045] In step S611, HOG feature extraction is performed on each of the t-1 tracking images to obtain a sequence of histograms of directional gradients of the tracking images. Accordingly, considering that the shape of the target vehicle is one of its important features, despite the target vehicle's position and posture changing, or being affected by factors such as lighting, its shape features remain relatively stable across different frames. Furthermore, considering that HOG features can effectively capture the shape and structural information of objects in an image, the technical solution of the present application performs HOG feature extraction on each of the t-1 tracking images to obtain a sequence of histograms of directional gradients of the tracking images. In other words, by performing HOG feature extraction on each frame, a description of the target vehicle's shape at different times can be obtained, providing a basis for subsequent analysis. For example, features such as the vehicle's body contour and wheel shape are well reflected in the HOG features. Even if the vehicle's angle in the image changes, its basic shape features can still be reflected by the HOG features. Furthermore, each histogram of directional gradients of the tracking image records the shape features of the target vehicle at that moment. The sequence of these histograms can reflect how the target vehicle's shape features change over time. For example, when a vehicle turns, the projection of its shape in different frames will change. This change trend can be observed through the HOG feature sequence, thus providing a basis for predicting the state of the target vehicle in the tth frame.

[0046] The following is a detailed description of a specific implementation process of "performing HOG feature extraction on each tracking image in the t-1 tracking images to obtain a sequence of directional gradient histograms of the tracking images":

[0047] First, image preprocessing is required. Since the HOG feature map primarily focuses on grayscale variations to highlight object shape, each tracking image must be converted from color to grayscale to reduce the amount of data. To eliminate the impact of light intensity differences between images on feature extraction, grayscale images must be normalized, mapping the image grayscale values ​​to a uniform range such as [0, 1] or [-1, 1]. Furthermore, since noise in the image can interfere with feature extraction, a Gaussian filter is used to convolve the image and perform Gaussian smoothing to smooth the image and reduce the impact of noise.

[0048] After image preprocessing is complete, image gradient calculation begins. Image gradient calculation is performed to determine the change in grayscale values ​​within the image, performed horizontally and vertically. This is typically accomplished using the Sobel operator, which convolves the image with the horizontal gradient Gx and the vertical gradient Gy. Based on these two gradient components, the gradient magnitude and direction are then calculated.

[0049] Next, the image is divided into multiple small cells. These cells, the basic unit for HOG feature calculation, are typically fixed in size, with 8x8 or 16x16 pixels being common. Each cell performs its own subsequent histogram of oriented gradients calculation. This division allows for detailed statistical analysis of gradient information within a local area, enabling more accurate capture of the object's local shape features.

[0050] After completing the cell division, start counting the directional gradient histogram of each cell. The specific approach is to divide the gradient direction into several intervals, such as dividing 0-180° into 9 intervals, each interval is 20°. In each cell, a directional gradient histogram is constructed based on the gradient direction and amplitude information of the pixel, and the sum of the gradient amplitudes in each interval is counted. When counting, the interval to which the gradient direction belongs is determined according to the angle of the gradient direction, and then the corresponding gradient amplitude is added to the statistical value of the interval. In this way, each cell can be represented by a vector containing multiple interval statistical values, which is a directional gradient histogram.

[0051] Finally, for each of the t-1 tracking images, HOG feature extraction is performed strictly according to the above steps from image preprocessing to counting the cell oriented gradient histograms, thus obtaining a directional gradient histogram for each image. These directional gradient histograms are arranged in chronological order of the images to form a sequence of directional gradient histograms for the tracking images.

[0052] In step S612, the color histogram of each tracking image in the t-1 tracking images is calculated separately to obtain a sequence of color histograms of the tracking images. Accordingly, it is considered that color is a very significant and easily distinguishable visual feature of the target vehicle. Vehicles of different brands and types often have unique colors. For example, fire trucks are usually red, and ambulances are mostly white with special blue or red logos. Even in a complex background environment, color information can help to quickly identify and locate the target vehicle. Based on this, the present application calculates the color histogram of each tracking image in the t-1 tracking images separately to obtain a sequence of color histograms of the tracking images. In this way, by calculating the color histogram for each frame of the image, a color histogram sequence is formed. This sequence records the color distribution characteristics of the target vehicle at different time points, reflects the changes in color characteristics over time, and provides a basis for subsequent predictions.

[0053] In step S62, histogram feature extraction is performed on the sequence of tracking image directional gradient histograms and the sequence of tracking image color histograms to obtain a sequence of tracking image directional gradient histogram semantic feature vectors and a sequence of tracking image color histogram semantic feature vectors. Specifically, in an embodiment of the present application, step S62 includes: passing the sequence of tracking image directional gradient histograms and the sequence of tracking image color histograms through a histogram feature extractor based on an FCN spatial model to obtain a sequence of tracking image directional gradient histogram semantic feature vectors and a sequence of tracking image color histogram semantic feature vectors. Accordingly, considering that the tracking image directional gradient histogram and color histogram contain basic information such as the shape and color distribution of the target vehicle, this information is relatively shallow and intuitive statistical results. Therefore, in order to further extract the deep semantic information contained in each histogram, the present application passes the sequence of the tracking image directional gradient histogram and the sequence of the tracking image color histogram through a histogram feature extractor based on the FCN spatial model to obtain a sequence of tracking image directional gradient histogram semantic feature vectors and a sequence of tracking image color histogram semantic feature vectors. It should be understood that the FCN spatial model has powerful convolution operations and feature mapping capabilities. It can gradually abstract and extract features from the input histogram through multiple layers of convolution layers, and convert the original histogram data into a more representative and semantic feature representation. For example, it can learn the association between the target vehicle shape and color features and the target category, state, etc., thereby digging out the semantic information hidden in the histogram.

[0054] In step S63, each corresponding set of tracking image directional gradient histogram semantic feature vectors and tracking image color histogram semantic feature vectors in the sequence of tracking image directional gradient histogram semantic feature vectors and the sequence of tracking image color histogram semantic feature vectors are concatenated to obtain a sequence of tracking image multidimensional semantic fusion representation vectors. It should be understood that the tracking image directional gradient histogram semantic feature vectors primarily reflect the shape and structural information of the target vehicle, while the tracking image color histogram semantic feature vectors focus on reflecting the color distribution characteristics of the target vehicle. These two features are highly complementary, and using either feature alone may not fully describe the characteristics of the target vehicle. Based on this, in the technical solution of the present application, each corresponding set of tracking image directional gradient histogram semantic feature vectors and tracking image color histogram semantic feature vectors in the sequence of tracking image directional gradient histogram semantic feature vectors and the sequence of tracking image color histogram semantic feature vectors are concatenated to integrate the different modal information they carry, forming a more comprehensive and richer feature representation, thereby obtaining a sequence of tracking image multidimensional semantic fusion representation vectors.

[0055] In step S64, the tracking image dynamic walk is performed on the sequence of the tracking image multi-dimensional semantic fusion representation vectors to obtain the tracking image context dynamic walk semantic encoding vector. Specifically, Figure 4 Flowchart of step S64 in the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to an embodiment of the present application. Figure 4 As shown, the step S64 includes: S641, performing implicit feature mining on each tracking image multidimensional semantic fusion representation vector in the sequence of the tracking image multidimensional semantic fusion representation vector to obtain a sequence of tracking image multidimensional semantic depth implicit feature coding vectors; S642, calculating the causal association topological features of the sequence of the tracking image multidimensional semantic depth implicit feature coding vectors to obtain a tracking image multidimensional semantic causal association topological feature matrix; S643, based on the tracking image multidimensional semantic causal association topological feature matrix, performing surface and hidden layer dynamic wandering coding on the sequence of the tracking image multidimensional semantic fusion representation vector and the sequence of the tracking image multidimensional semantic depth implicit feature coding vector respectively to obtain a tracking image multidimensional semantic surface context dynamic wandering coding vector and a tracking image multidimensional semantic hidden layer context dynamic wandering coding vector; S644, fusing the tracking image multidimensional semantic surface context dynamic wandering coding vector and the tracking image multidimensional semantic hidden layer context dynamic wandering coding vector to obtain the tracking image context dynamic wandering semantic coding vector.

[0056] It should be understandable that there is a dynamic relationship between the multi-dimensional semantic fusion representation vectors of each tracking image, such as how the shape, color and other features of the target vehicle at different moments influence and change each other over time. During the driving process of the target vehicle, its posture changes may cause changes in shape features, and changes in lighting conditions may also affect color features. In order to capture these complex dynamic connections, the present application dynamically walks the tracking image by causal modeling the sequence of the multi-dimensional semantic fusion representation vectors of the tracking image to obtain the contextual dynamic walk semantic encoding vector of the tracking image. In this way, the intrinsic connections between complex information such as target features in the image and key feature points can be more effectively excavated, so that it is not only limited to focusing on the local features of the target, but also fully captures the hidden global dependencies, while reducing the interference of noise, thereby helping to better understand the state and changes of the target.

[0057] Specifically, implicit feature mining is first performed on each tracking image multi-dimensional semantic fusion representation vector in the sequence of tracking image multi-dimensional semantic fusion representation vectors to obtain a sequence of tracking image multi-dimensional semantic depth implicit feature encoding vectors. The above process can be expressed as:

[0058] O={x1,x2,...,xi ,...,x n}

[0059] v i =Sigmoid[Conv 1×1 (x i )]

[0060] D={v1,v2,...,v i ,...,v n}

[0061] Wherein, O is the sequence of multi-dimensional semantic fusion representation vectors of the tracking image, x1, x2, x i and x n are respectively the 1st, 2nd, i-th and n-th tracking image multi-dimensional semantic fusion representation vectors in the sequence of the tracking image multi-dimensional semantic fusion representation vector, Conv 1×1 is point convolution coding, Sigmoid is the convolution coding activation function, v1, v2, v i , v j and v n are respectively the 1st, 2nd, i-th, j-th and n-th tracking image multi-dimensional semantic depth implicit feature coding vectors in the sequence of tracking image multi-dimensional semantic depth implicit feature coding vectors, and D is the sequence of the tracking image multi-dimensional semantic depth implicit feature coding vectors.

[0062] It's understandable that in actual target vehicle detection and tracking scenarios, the captured image data is inevitably subject to various noise artifacts, such as camera errors, unstable ambient lighting, and signal interference during image transmission. This noise is reflected in the multidimensional semantic fusion representation vector of the tracking image, affecting the subsequent accurate analysis of target features. Implicit feature mining can mitigate the effects of this noise to a certain extent, making the generated multidimensional semantic deep implicit feature encoding vector of the tracking image purer and more reliable. Implicit feature mining can also extract deep semantic features from the sequence of input multidimensional semantic fusion representation vectors of the tracking image. For example, taking the pose of a target vehicle as an example, the features in the original sequence may only describe the vehicle's general outline. However, implicit feature mining can more accurately capture the details and dynamic changes of the vehicle's pose by analyzing the changing relationships between the features of various vehicle components under different poses, thereby providing richer and more accurate feature information for target vehicle detection and tracking.

[0063] Specifically, Figure 5 Flowchart of step S642 in the target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to an embodiment of the present application. Figure 5As shown, the step S642 includes: S6421, calculating the semantic causal association factor between any two tracking image multidimensional semantic depth implicit feature coding vectors in the sequence of the tracking image multidimensional semantic depth implicit feature coding vectors to obtain a tracking image multidimensional semantic causal association topology matrix composed of multiple tracking image multidimensional semantic causal association factors; S6422, performing a causal gate activation function triggering on the tracking image multidimensional semantic causal association topology matrix to obtain the tracking image multidimensional semantic causal association topology feature matrix.

[0064] More specifically, in an embodiment of the present application, the S6421 includes: calculating the association matrix between any two tracking image multidimensional semantic depth implicit feature coding vectors in the sequence of the tracking image multidimensional semantic depth implicit feature coding vectors to obtain a sequence of tracking image multidimensional semantic association matrices; calculating the semantic causal association factors of each tracking image multidimensional semantic association matrix in the sequence of the tracking image multidimensional semantic association matrix to obtain the tracking image multidimensional semantic causal association topology matrix composed of multiple tracking image multidimensional semantic causal association factors, the tracking image multidimensional semantic causal association factor and its corresponding The mean, variance, and maximum value of the tracking image multidimensional semantic association matrix are related to the tracking image causal association bias value; wherein, in response to the variance of the tracking image multidimensional semantic association matrix being greater than or equal to a predetermined threshold, the weighted average of the distances between any two tracking image multidimensional semantic depth implicit feature coding vectors in the sequence of the tracking image multidimensional semantic depth implicit feature coding vectors is used as the tracking image causal association bias value; in response to the variance of the tracking image multidimensional semantic association matrix being less than the predetermined threshold, the weighted average of the tracking image multidimensional semantic association matrix is ​​used as the tracking image causal association bias value. The above process can be expressed by the formula:

[0065]

[0066] Among them, v i , v j are the i-th and j-th tracking image multi-dimensional semantic depth implicit feature encoding vectors in the sequence of tracking image multi-dimensional semantic depth implicit feature encoding vectors, is matrix multiplication, v j T It is v j The transposed vector, M i-j It is v i and v j The multidimensional semantic correlation matrix between the tracking images, σ 2 (M i-j ) is M i-j The variance of max(M i-j ) is to take M i-jThe maximum value in μ(M i-j ) is M i-j The mean of ,λ is the causal correlation bias value of the tracking image, t i-j It's M i-j The corresponding tracking image multi-dimensional semantic causal correlation factor, d(v i ,v j ) is v i and v j The distance between them, L is the number of vectors in D, ε is the predetermined threshold, α and β are weighted hyperparameters, t 1-1 , t 1-n , t n-1 and t n-n are respectively the tracking image multi-dimensional semantic causal association factors of each position in the tracking image multi-dimensional semantic causal association topology matrix, and T is the tracking image multi-dimensional semantic causal association topology matrix.

[0067] It should be understood that in practical target detection and tracking scenarios, the multidimensional semantic deep implicit feature encoding vectors of tracking images contain rich feature information about the target vehicle, and the relationships between these features are complex. Computing the semantic causal correlation factor between any two encoding vectors can reveal hidden causal connections within the implicit feature sequence. For example, a vehicle's color characteristics may change due to varying lighting conditions, which in turn may be related to the vehicle's environmental location. This deep-seated causal relationship is revealed through this step, contributing to a more comprehensive and in-depth understanding of the mechanisms by which target vehicle features vary. The multidimensional semantic causal correlation topology matrix of the tracking image, derived from multiple multidimensional semantic causal correlation factors of the tracking image, reveals the global causal structure between all features of the target vehicle. Analyzing this matrix reveals complex patterns and regularities within the feature data, helping to comprehensively understand the changing patterns of the target vehicle's features.

[0068] In particular, by viewing low-level causal associations in complex systems as molecular-level relationships inferred based on statistical correlations, we can further predict the intervention of the causal energy of tracking images based on the global fine-grained statistical association representation, so as to study the fine-grained structure of tracking image causal associations and their dynamic regulation based on the high-dimensional and heterogeneous representation of the causal omics of tracking images. Specifically, when the aggregate distribution representation of the causal graph is greater than a predetermined threshold, source data integration is biased based on the matrixed graph node effect representation of the semantic causal association factor. When the aggregate distribution representation of the causal graph is less than the predetermined threshold, agglomerative structural modeling can be directly performed through feature pattern integration and compression. In this way, not only can the causal energy of tracking images in the description system be encoded, but also the implicit causal intervention prediction results can be condensed, thereby obtaining a more efficient disclosure of key tracking image causal associations.

[0069] Next, a causal gate activation function is triggered on the multi-dimensional semantic causal association topology matrix of the tracking image to obtain the multi-dimensional semantic causal association topology feature matrix of the tracking image. The above process can be expressed as follows:

[0070]

[0071] Among them, t i-j It's M i-j The corresponding tracking image multi-dimensional semantic causal correlation factor, T is the tracking image multi-dimensional semantic causal correlation topology matrix, softmax is a nonlinear activation function, τ is the normalization threshold, f trigger (T) is the gated activation processing of T, and M is the multi-dimensional semantic causal association topological feature matrix of the tracking image.

[0072] It should be understood that by triggering the causal gating activation function on the multidimensional semantic causal topology matrix of the tracking image, more detailed feature extraction and modeling of the causal topological relationships in the original matrix can be achieved. Specifically, its core operations are a dynamic gating mechanism and a nonlinear activation function. The gating mechanism allows the model to identify key causal paths in a dynamic context, thereby strengthening important relationships and reducing noise interference. The activation function, on the other hand, enhances the model's expressiveness by introducing nonlinearity, capturing the higher-order regularities hidden in the complex causal structure. For example, as the input tracking image data changes over time, the characteristics and status of the target vehicle may change. The causal gating activation function triggering process can promptly respond to dynamic changes in this data and dynamically adjust the causal relationships. For example, when the target vehicle is occluded, accelerates, or decelerates, the model can reevaluate and adjust the causal relationships between features through the gating mechanism and activation function to ensure accurate tracking and detection of the target vehicle and adapt to dynamic data changes.

[0073] Specifically, in an embodiment of the present application, step S643 includes: performing graph convolution feature sequence dynamic walk coding on the sequence of the multi-dimensional semantic causal association topological feature matrix of the tracking image and the multi-dimensional semantic fusion representation vector of the tracking image to obtain the multi-dimensional semantic surface context dynamic walk coding vector of the tracking image. This process can be expressed by the formula:

[0074]

[0075] Among them, x i is the i-th tracking image multidimensional semantic fusion representation vector in the sequence of the tracking image multidimensional semantic fusion representation vector, M is the tracking image multidimensional semantic causal association topological feature matrix, GCN is graph convolution processing, H surface It is a dynamic wandering encoding vector for tracking the multi-dimensional semantic surface context of the image;

[0076] The graph convolution feature sequence dynamic walk coding is performed on the sequence of the tracking image multidimensional semantic causal association topological feature matrix and the tracking image multidimensional semantic depth implicit feature coding vector to obtain the tracking image multidimensional semantic hidden layer context dynamic walk coding vector. The process can be expressed by the formula:

[0077]

[0078] Among them, v i In particular, it is the i-th tracking image multi-dimensional semantic depth implicit feature encoding vector in the sequence of tracking image multi-dimensional semantic depth implicit feature encoding vector, M is the tracking image multi-dimensional semantic causal association topological feature matrix, GCM is graph convolution processing, GCN is graph convolution processing, H hidden It is the dynamic walking encoding vector of the multi-dimensional semantic hidden layer context of the tracking image.

[0079] It should be understood that the multi-dimensional semantic causal topological feature matrix of the tracking image reveals a complex causal structure between the target vehicle's features. To generate a surface context semantic representation, the dynamic walk mechanism simulates the feature propagation process within this topological structure. Just like information propagation in a network, features begin at local nodes and gradually diffuse globally. For example, for the target vehicle's appearance, the dynamic walk mechanism begins with the features of a local component (such as the headlights) and gradually propagates through the associations in the topological structure to the features of other components related to the headlights (such as the body color and overall shape), achieving feature aggregation from local to global. In other words, during the feature propagation process, the dynamic walk mechanism recursively aggregates the explicit semantics between nodes. This means that it continuously integrates the explicit semantic information carried by different feature nodes. In this way, the explicit semantic information scattered across different nodes is effectively aggregated to form a more comprehensive and representative surface context semantic representation, namely, the dynamic walk encoding vector of the multi-dimensional semantic surface context of the tracking image. This vector comprehensively reflects the target vehicle's distinct features in its current state and the relationships between them, providing a foundation for subsequent analysis. For dynamic semantic encoding of latent context, the dynamic walk mechanism conducts deeper semantic exploration of the latent feature encoding vector of the multi-dimensional semantic depth of the tracking image. During this latent semantic exploration, due to the latent embedding of features, it is important to focus on the multi-hop propagation of high-order information and the distributed decoupling of deep features. Multi-hop propagation means that feature information is not only transmitted between adjacent nodes but can also be transmitted over long distances through multiple intermediate nodes, thereby capturing deeper temporal dependencies. For example, the changes in a vehicle's driving trajectory over a period of time may involve feature information from multiple time points, and multi-hop propagation can effectively integrate this information. Furthermore, the distributed decoupling of deep features avoids the oversmoothing phenomenon caused by the propagation of deep topological features, preventing features from becoming too similar during propagation and losing their uniqueness. This approach results in a more discriminative and representative dynamic semantic encoding of the latent context, namely, the dynamic walk encoding vector of the multi-dimensional semantic latent context of the tracking image, which provides greater generalization capability for feature representation.

[0080] Finally, the tracking image multi-dimensional semantic surface context dynamic walk coding vector and the tracking image multi-dimensional semantic hidden context dynamic walk coding vector are fused to obtain the tracking image context dynamic walk semantic coding vector. The above process can be expressed as:

[0081] H final =γ·H surface +(1-γ)·H hidden

[0082] Among them, H surfaceis the dynamic walking encoding vector of the multi-dimensional semantic surface context of the tracking image, H hidden is the dynamic walking encoding vector of the multi-dimensional semantic hidden layer context of the tracking image, γ is the fusion weighting parameter, H final is the dynamic walk semantic encoding vector of the tracking image context.

[0083] It should be understood that the multi-dimensional semantic surface context dynamic wandering coding vector of the tracking image captures the explicit semantic information of the target vehicle, such as the vehicle's appearance features (color, shape, etc.); while the multi-dimensional semantic hidden context dynamic wandering coding vector of the tracking image mines implicit semantic information, such as the vehicle's driving intention, potential behavior patterns, and more abstract feature associations. These two types of semantic information are complementary, and by fusing these two vectors, explicit and implicit semantic information can be organically combined. In other words, the fused tracking image context dynamic wandering semantic coding vector integrates the semantic information of the surface and hidden layers, providing a more comprehensive feature description of the target vehicle. It is no longer limited to a single explicit or implicit feature, but covers multiple aspects of the target. This comprehensive feature description helps to more accurately identify and track the target vehicle. In the specific implementation of this application, a weighted summation strategy is adopted to fuse the two vectors, which enables the dynamic adjustment of the importance distribution of surface and hidden features during the fusion process. This ability to dynamically adjust feature importance allows the fused feature vector to better adapt to different situations and task requirements.

[0084] In step S65, the target vehicle object detection image of the t-th frame is obtained based on the dynamic wandering semantic coding vector of the tracking image context. Specifically, in an embodiment of the present application, step S65 includes: passing the dynamic wandering semantic coding vector of the tracking image context through a target vehicle object prediction generator based on AIGC to obtain the target vehicle object detection image of the t-th frame. That is, the dynamic wandering semantic coding vector of the tracking image context obtained by causal dynamic wandering using the sequence of the tracking image multi-dimensional semantic fusion representation vector is generated and processed, thereby automatically predicting and obtaining the target vehicle object detection image of the t-th frame. It is worth mentioning that the target vehicle object prediction generator based on AIGC (artificial intelligence generated content) technology has powerful generation capabilities. It can generate logical and expected image content based on the input dynamic wandering semantic coding vector of the tracking image context. In the target detection and tracking task, when the target vehicle is not detected in the t-th frame, this generation capability can be used to fill the information gap. By understanding and analyzing the information of the previous frame, a possible target vehicle object detection image of the t-th frame is generated. In this way, the predicted image replaces the actual detection image, thereby maintaining the continuity of target tracking.

[0085] The following is a detailed description of a specific implementation process of "passing the tracking image context dynamic walk semantic encoding vector through the target vehicle object prediction generator based on AI GC to obtain the target vehicle object detection image of the t-th frame":

[0086] First, during the preparation phase of the prediction generator, it's necessary to select an appropriate AI GC model based on the actual task requirements and the characteristics of the available data. Generative adversarial networks (GANs), variational autoencoders (VAEs), and diffusion models are common choices. GANs offer the advantage of generating high-resolution, realistic images. Through adversarial training between the generator and discriminator, the generator continuously optimizes the generated images, making them closer to real-world vehicle images. VAEs excel at learning the latent distribution of data. By mapping encoding vectors into the latent space, sampling, and then generating images using a decoder, they can produce diverse image samples. Diffusion models generate images by gradually removing noise, excelling at generating complex, high-quality images. After selecting a model, it should be pre-trained using a large, diverse dataset of vehicle images. These datasets should include images of vehicles from various angles and poses in a variety of scenarios. By learning from this data, the model develops a deep understanding of vehicle characteristics, appearance patterns, and structural information, laying a solid foundation for subsequent accurate image generation based on specific encoding vectors.

[0087] Once the prediction generator is ready, the semantic encoding vector of the dynamic walk context of the tracked image is fed into it. This encoding vector contains rich information about the target vehicle in the previous t-1 frame and serves as the key basis for generating the tth frame. After receiving the encoding vector, the model utilizes its internal neural network architecture to conduct in-depth analysis of the vector. For example, the convolutional layer identifies local features in the vector, such as the vehicle's outline and detailed components. The fully connected layer further integrates these local features, exploring hidden connections and underlying patterns between them, providing strong support for subsequent image generation.

[0088] Next, we move on to the core process of image generation. Taking a GAN as an example, the generator begins to generate an image of the target vehicle for object detection in frame t based on the parsed feature information. Initially, the generated image may differ significantly from the real vehicle image. However, the discriminator carefully distinguishes the generated image from the real image and provides feedback to the generator. Based on this feedback, the generator continuously adjusts its parameters to optimize the generated image. Over multiple iterations, the generator's generated image becomes increasingly similar to the real vehicle image until it can deceive the discriminator. If a VAE model is used, it maps the encoding vector to a latent space, where each point represents a unique combination of vehicle image features. The model samples from the latent space and uses the decoder to convert the sampled points into an image. In this process, the VAE learns the data distribution, generating a frame t image that is both consistent with the current context and exhibits a certain degree of diversity. The diffusion model, on the other hand, starts with a random noise image and, based on the information in the encoding vector, gradually predicts and removes the noise. In each iteration, the model updates the image based on the previous image state and encoding vector. After multiple iterations, the image gradually becomes clearer and shows the accurate features of the target vehicle, and finally generates the t-th frame target vehicle object detection image.

[0089] After image generation, optimization and post-processing are required. Since the generated images may contain some flaws or not meet the requirements of practical applications in some aspects, they need to be optimized. This can be achieved by adjusting the model's hyperparameters, such as the learning rate and number of iterations, or by increasing the training data. For example, in GANs, properly adjusting the training ratio of the generator and discriminator can make the generated images more realistic. At the same time, a series of post-processing operations are required, including image filtering to remove possible noise, enhancing contrast to make image details clearer, and adjusting brightness to suit different visual requirements. In addition, depending on the requirements of the subsequent object detection algorithm, it may also be necessary to crop, scale, and perform other operations on the image to ensure that the image size and format are compatible with the detection algorithm.

[0090] Finally, the generated target vehicle object detection image for frame t is evaluated and feedback is provided. Evaluation metrics such as peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are used to measure the similarity and quality difference between the generated image and the real image. If the generated image quality does not meet the expected standards, the evaluation results are fed back to the prediction generator for further training and adjustment of the model. By continuously optimizing the model, the quality of the generated images is improved to meet the accuracy and reliability requirements of target vehicle object detection images in practical applications.

[0091] In summary, step S6 is illustrated, which uses machine vision-based image analysis and feature extraction techniques to perform HOG feature extraction and color histogram calculation on each of the previous t-1 tracking images, and then performs semantic capture and multi-dimensional semantic fusion on the extracted directional gradient histogram of each tracking image and the color histogram of each tracking image, to automatically predict and obtain the t-th frame target vehicle object detection image according to the dynamic causal context walk representation between the fused multi-dimensional semantic fusion feature of each tracking image. In this way, through HOG feature extraction and color histogram calculation, the shape and color information of the target vehicle can be described comprehensively from different angles, and through the dynamic causal context walk representation, the dynamic changes of the target vehicle over time and its interaction with the environment can be captured, thereby improving the understanding and prediction accuracy of the target vehicle behavior.

[0092] In summary, the target detection and tracking method combining YOLOv8 detection algorithm and KCF tracking algorithm based on the embodiments of the present application is illustrated, which first acquires a video stream time sequence set of tracking images of the target vehicle object, and uses YOLOv8 to perform target detection on the first frame to initialize the KCF tracker, then the KCF tracker processes the subsequent frame images for target tracking, if tracking fails at the t-th frame, YOLOv8 is re-enabled to detect the frame to try to find the target, if the target is successfully detected, a new target vehicle object detection image is generated and the KCF tracker is reinitialized, if the target cannot be detected, the t-th frame target vehicle object detection image is predicted and generated based on the information of the previous t-1 frames. In this way, the continuity and accuracy of the target vehicle object tracking can be effectively ensured.

Claims

1. A target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm, characterized in that: include: Acquire a video stream time sequence set of tracking images of a target vehicle object; Performing target detection on the first frame tracking image in the video stream time series set of the tracking image using the YOLOv8 detection algorithm to obtain a target vehicle object detection image; Initializing a KCF tracker based on the target vehicle object detection image, and using the initialized KCF tracker to track the target on other frame tracking images in the video stream time series set of the tracking image; In response to the KCF tracker reporting that target tracking fails in the t-th frame tracking image in the video stream time series set of the tracking image, re-performing target detection on the t-th frame tracking image using a YOLOv8 detection algorithm to obtain a target detection result; In response to the target detection result indicating that the target vehicle object is re-detected, generating a t-th frame of target vehicle object detection image, and reinitializing the KCF tracker based on the t-th frame of target vehicle object detection image; In response to the target detection result indicating that the target vehicle object is not detected, performing target vehicle object prediction based on t-1 tracking images prior to the t-th frame tracking image to generate a t-th frame target vehicle object detection image, including: Performing HOG feature extraction and color histogram calculation on each tracking image in the t-1 tracking images to obtain a sequence of tracking image directional gradient histograms and a sequence of tracking image color histograms; Performing histogram feature extraction on the sequence of tracking image directional gradient histograms and the sequence of tracking image color histograms to obtain a sequence of tracking image directional gradient histogram semantic feature vectors and a sequence of tracking image color histogram semantic feature vectors; Cascading each corresponding set of tracking image directional gradient histogram semantic feature vectors and tracking image color histogram semantic feature vectors in the sequence of tracking image directional gradient histogram semantic feature vectors and the sequence of tracking image color histogram semantic feature vectors to obtain a sequence of tracking image multi-dimensional semantic fusion representation vectors; Performing causal modeling of the tracking image dynamic walk on the sequence of the tracking image multi-dimensional semantic fusion representation vector to obtain a tracking image context dynamic walk semantic encoding vector; Obtaining the target vehicle object detection image of the t-th frame based on the tracking image context dynamic wandering semantic encoding vector; The method of performing causal modeling on the sequence of the multi-dimensional semantic fusion representation vectors of the tracking image to dynamically walk the tracking image to obtain the contextual dynamic walk semantic encoding vector of the tracking image includes: Performing implicit feature mining on each tracking image multidimensional semantic fusion representation vector in the sequence of tracking image multidimensional semantic fusion representation vectors to obtain a sequence of tracking image multidimensional semantic depth implicit feature encoding vectors; Calculating the causal correlation topological features of the sequence of multi-dimensional semantic depth implicit feature encoding vectors of the tracking image to obtain a multi-dimensional semantic causal correlation topological feature matrix of the tracking image; Based on the multi-dimensional semantic causal association topological feature matrix of the tracking image, performing surface layer and hidden layer dynamic wandering coding on the sequence of the multi-dimensional semantic fusion representation vector of the tracking image and the sequence of the multi-dimensional semantic depth implicit feature coding vector of the tracking image, respectively, to obtain the multi-dimensional semantic surface layer context dynamic wandering coding vector of the tracking image and the multi-dimensional semantic hidden layer context dynamic wandering coding vector of the tracking image; Fusing the tracking image multi-dimensional semantic surface context dynamic wandering coding vector and the tracking image multi-dimensional semantic hidden context dynamic wandering coding vector to obtain the tracking image context dynamic wandering semantic coding vector; The causal correlation topological features of the sequence of the multi-dimensional semantic depth implicit feature encoding vectors of the tracking image are calculated to obtain the multi-dimensional semantic causal correlation topological feature matrix of the tracking image, including: Calculating a semantic causal correlation factor between any two tracking image multidimensional semantic depth implicit feature coding vectors in the sequence of the tracking image multidimensional semantic depth implicit feature coding vectors to obtain a tracking image multidimensional semantic causal correlation topology matrix composed of a plurality of tracking image multidimensional semantic causal correlation factors; A causal gating activation function is triggered on the multi-dimensional semantic causal association topology matrix of the tracking image to obtain the multi-dimensional semantic causal association topology feature matrix of the tracking image.

2. The target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to claim 1, characterized in that: Performing HOG feature extraction and color histogram calculation on each of the t-1 tracking images to obtain a sequence of tracking image directional gradient histograms and a sequence of tracking image color histograms, including: Performing HOG feature extraction on each tracking image in the t-1 tracking images to obtain a sequence of histograms of directional gradients of the tracking images; The color histogram of each tracking image in the t-1 tracking images is calculated respectively to obtain a sequence of the tracking image color histograms.

3. The target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to claim 2, characterized in that: Histogram feature extraction is performed on the sequence of tracking image directional gradient histograms and the sequence of tracking image color histograms to obtain a sequence of tracking image directional gradient histogram semantic feature vectors and a sequence of tracking image color histogram semantic feature vectors, including: passing the sequence of tracking image directional gradient histograms and the sequence of tracking image color histograms through a histogram feature extractor based on an FCN spatial model to obtain a sequence of tracking image directional gradient histogram semantic feature vectors and a sequence of tracking image color histogram semantic feature vectors.

4. The target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to claim 1, characterized in that Calculating the semantic causal correlation factor between any two tracking image multidimensional semantic depth implicit feature coding vectors in the sequence of the tracking image multidimensional semantic depth implicit feature coding vectors to obtain a tracking image multidimensional semantic causal correlation topology matrix composed of multiple tracking image multidimensional semantic causal correlation factors, including: Calculating the association matrix between any two tracking image multi-dimensional semantic depth implicit feature coding vectors in the sequence of tracking image multi-dimensional semantic depth implicit feature coding vectors to obtain a sequence of tracking image multi-dimensional semantic association matrices; Calculating the semantic causal association factor of each tracking image multidimensional semantic association matrix in the sequence of the tracking image multidimensional semantic association matrix to obtain the tracking image multidimensional semantic causal association topology matrix composed of multiple tracking image multidimensional semantic causal association factors, wherein the tracking image multidimensional semantic causal association factor is related to the mean, variance, maximum value and tracking image causal association bias value of the corresponding tracking image multidimensional semantic association matrix; In response to the variance of the tracking image multidimensional semantic association matrix being greater than or equal to a predetermined threshold, a weighted average of the distances between any two tracking image multidimensional semantic depth implicit feature coding vectors in the sequence of the tracking image multidimensional semantic depth implicit feature coding vectors is used as the tracking image causal association bias value; In response to the variance of the tracking image multidimensional semantic association matrix being smaller than the predetermined threshold, a weighted mean of the tracking image multidimensional semantic association matrix is ​​used as the tracking image causal association bias value.

5. The target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to claim 4, characterized in that: Based on the multi-dimensional semantic causal association topological feature matrix of the tracking image, surface layer and hidden layer dynamic walk coding are respectively performed on the sequence of the multi-dimensional semantic fusion representation vector of the tracking image and the sequence of the multi-dimensional semantic depth implicit feature coding vector of the tracking image to obtain the multi-dimensional semantic surface layer context dynamic walk coding vector of the tracking image and the multi-dimensional semantic hidden layer context dynamic walk coding vector of the tracking image, including: Performing graph convolution feature sequence dynamic walk coding on the sequence of the tracking image multidimensional semantic causal association topological feature matrix and the tracking image multidimensional semantic fusion representation vector to obtain the tracking image multidimensional semantic surface context dynamic walk coding vector; The graph convolution feature sequence dynamic walking coding is performed on the sequence of the tracking image multi-dimensional semantic causal association topological feature matrix and the tracking image multi-dimensional semantic depth implicit feature coding vector to obtain the tracking image multi-dimensional semantic hidden layer context dynamic walking coding vector.

6. The target detection and tracking method combining the YOLOv8 detection algorithm and the KCF tracking algorithm according to claim 5, characterized in that: Based on the tracking image context dynamic wandering semantic coding vector, the target vehicle object detection image of the t-th frame is obtained, including: passing the tracking image context dynamic wandering semantic coding vector through an AIGC-based target vehicle object prediction generator to obtain the target vehicle object detection image of the t-th frame.

Images