Tracking result generation method and device based on YOLO and MixFormer model, equipment and medium

Abstract

This application discloses a method, apparatus, device, and medium for generating tracking results based on YOLO and MixFormer models, relating to the field of computer vision technology. The method includes: acquiring raw video data; preprocessing the raw video data to obtain frame data; inputting the frame data into a YOLO model for target detection to obtain target video data; inputting the target video data into a preset model to obtain predicted position results; fusing the target video data and the predicted position results to obtain tracking results; and then superimposing the tracking results with the raw video data to generate video data with tracking results, which is then displayed in a visualized image format. By combining YOLO and MixFormer with raw video data and finally displaying it in a visualized image format, efficient and accurate multi-target tracking in complex video environments is achieved, improving the accuracy, robustness, and real-time performance of tracking.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, device, and medium for generating tracking results based on YOLO and MixFormer models. Background Technology

[0002] With the rapid development of computer vision technology, multi-object tracking technology has demonstrated enormous application potential in numerous fields such as intelligent monitoring, autonomous driving, human-computer interaction, and intelligent transportation. In intelligent monitoring, accurate multi-object tracking enables real-time monitoring of moving targets such as people and vehicles within the monitored area, providing crucial technical support for security protection and abnormal behavior detection. In autonomous driving scenarios, vehicles need to accurately perceive the position and movement status of surrounding pedestrians, other vehicles, and other targets to achieve safe path planning and decision-making. In human-computer interaction, multi-object tracking can be used for tasks such as gesture recognition and pose estimation, improving the naturalness and fluency of the interaction. Intelligent transportation leverages multi-object tracking to achieve functions such as traffic flow monitoring and violation recognition, optimizing traffic management. However, real-world video environments are often complex and varied. Factors such as occlusion between targets, differentiation of similar targets, rapid target movement, and background complexity pose significant challenges to multi-object tracking technology.

[0003] Currently, multi-target tracking technology mainly relies on two types of methods: detection-based tracking methods and Transformer-based tracking methods. Detection-based methods first use a target detector to detect targets in video frames, and then match the detection results with existing target trajectories using data association algorithms (such as the Hungarian algorithm and Kalman filtering). However, detection-based tracking methods are prone to tracking interruptions in complex scenes due to detection errors or occlusion. Transformer-based tracking methods, on the other hand, use a self-attention mechanism to capture global and local features of the target, making them suitable for target tracking in complex scenes. However, Transformers have high computational complexity and usually require manual target initialization, limiting their application in real-time scenarios. Furthermore, in complex scenes, traditional methods are prone to detection failures or mis-associations due to target occlusion, lighting changes, or background interference, thus affecting the continuity and accuracy of tracking. Therefore, there is an urgent need for a more efficient and accurate multi-target tracking method to achieve target tracking in complex and variable conditions. Summary of the Invention

[0004] The main purpose of this application is to provide a method, apparatus, device and medium for generating tracking results based on YOLO and MixFormer models, aiming to solve the technical problem of how to accurately and efficiently track multiple targets in video.

[0005] To achieve the above objectives, this application proposes a method for generating tracking results based on the YOLO and MixFormer model, including:

[0006] Obtain raw video data;

[0007] The original video data is preprocessed to obtain frame data;

[0008] The frame data is input into the YOLO model for target detection to obtain the target video data;

[0009] The predicted location result is obtained by inputting the target video data into a preset model.

[0010] The target video data and the predicted location result are fused to generate the target tracking result.

[0011] In one embodiment, the step of inputting the frame data into a YOLO model for target detection to obtain target video data includes:

[0012] The frame data is input into the YOLO model for target detection to obtain detection results, which include the target's location, size, category information, and confidence score.

[0013] The comparison result is obtained by comparing the confidence score in the detection result with a preset confidence threshold;

[0014] The target video data is obtained by filtering based on the comparison results.

[0015] In one embodiment, the step of filtering based on the comparison results to obtain target video data includes:

[0016] When the comparison result is that the confidence score is greater than the preset confidence threshold, the detection result corresponding to the confidence score is selected as the reserved detection result;

[0017] The category information of the reserved detection results is filtered according to the target category to obtain the target detection results;

[0018] Target video data is obtained based on the target detection results.

[0019] In one embodiment, the step of obtaining the predicted location result by inputting the target video data into a preset model includes:

[0020] The target video data is input into a preset model to obtain the detection result of the current frame. The preset model is a MixFormer model. The detection result of the current frame includes global features and local features. The global features are obtained through a global self-attention mechanism, and the local features are obtained through a window attention mechanism.

[0021] The detection results of the current frame are correlated using the multi-head attention mechanism of the MixFormer model to determine the correspondence between the detected target and known targets;

[0022] Based on the correspondence, the predicted position result is obtained according to the detection result of the current frame.

[0023] In one embodiment, the step of obtaining the predicted location result by inputting the target video data into a preset model further includes:

[0024] When the detected target does not have a corresponding relationship with a known target, the motion trajectory and appearance features of the detected target are obtained;

[0025] The position of the detected target is updated based on the motion trajectory and the appearance features until the detected target has a correspondence with a known target.

[0026] In one embodiment, the step of fusing the target video data and the predicted location result to generate a target tracking result includes:

[0027] The target video data and the predicted location result are fused using a weighted average method to obtain the fused target location information;

[0028] The fused target location information is smoothed using a Kalman filter algorithm to obtain the processed target location information;

[0029] The processed target location information is further processed to obtain target status information;

[0030] The target state information is summarized into target tracking results, which include the identifier, position, size, category, motion state, and confidence score of all tracked targets in each frame.

[0031] In one embodiment, the step of preprocessing the original video data to obtain frame data includes:

[0032] The original video data is read to obtain consecutive video frames;

[0033] The continuous video frames are resized to scale the image size to the size required by the YOLO model input, resulting in a resized image.

[0034] The resized image is then normalized to obtain a normalized image.

[0035] The normalized image is then subjected to channel adjustment to obtain the input image;

[0036] The input images are batch processed to obtain frame data.

[0037] Furthermore, to achieve the above objectives, this application also proposes a tracking result generation device based on the YOLO and MixFormer model, wherein the tracking result generation device based on the YOLO and MixFormer model includes:

[0038] The acquisition module is used to acquire raw video data;

[0039] The processing module is used to preprocess the raw video data to obtain frame data;

[0040] The extraction module is used to input the frame data into the YOLO model for target detection to obtain target video data;

[0041] The prediction module is used to obtain the predicted location result by inputting the target video data into a preset model;

[0042] The results module is used to fuse the target video data and the predicted location results to generate target tracking results.

[0043] In addition, to achieve the above objectives, this application also proposes a medium, which is a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the tracking result generation method based on the YOLO and MixFormer model described above.

[0044] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the tracking result generation method based on the YOLO and MixFormer model described above.

[0045] This application acquires raw video data, preprocesses it to obtain frame data, inputs the frame data into a YOLO model for target detection to obtain target video data, inputs the target video data into a preset model to obtain predicted position results, and fuses the target video data and predicted position results to generate target tracking results. By combining YOLO and MixFormer, the tracking results are obtained from the raw video data, and finally, the video data with tracking information is displayed in the form of a visual image, achieving efficient and accurate multi-target tracking in complex video environments, improving the accuracy, robustness, and real-time performance of tracking. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating the first embodiment of the tracking result generation method based on the YOLO and MixFormer model of this application;

[0048] Figure 2 This is a flowchart illustrating the second embodiment of the tracking result generation method based on the YOLO and MixFormer model in this application;

[0049] Figure 3 This is a flowchart illustrating the third embodiment of the tracking result generation method based on the YOLO and MixFormer model in this application;

[0050] Figure 4 This is a flowchart illustrating the fourth embodiment of the tracking result generation method based on the YOLO and MixFormer model in this application;

[0051] Figure 5 This is a schematic diagram of the module structure of the tracking result generation device based on the YOLO and MixFormer model, according to the first embodiment of the tracking result generation method based on the YOLO and MixFormer model of this application.

[0052] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the tracking result generation method based on YOLO and MixFormer models in the embodiments of this application.

[0053] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0055] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0056] With the rapid development of computer vision and artificial intelligence technologies, multi-object tracking (MOT) technology has been widely applied in video surveillance, autonomous driving, and intelligent transportation. However, complex and ever-changing video environments (such as high target density, drastic lighting changes, and frequent occlusion) pose significant challenges to MOT technology. Traditional MOT methods typically consist of two stages: target detection and target association. While these methods can achieve target tracking to a certain extent, their performance is prone to degradation in complex scenes due to target loss and misassociation. Therefore, a high-efficiency and robust MOT technology that can adapt to complex and ever-changing environments is needed.

[0057] Therefore, this application proposes an accurate and efficient method for tracking multiple targets in video by combining YOLO and MixFormer. The main solution of this application's embodiments is as follows: First, acquire raw video data; second, preprocess the raw video data to obtain frame data; third, input the frame data into the YOLO model for target detection to obtain target video data; fourth, input the target video data into a preset model to obtain predicted position results; and finally, fuse the target video data and the predicted position results to generate target tracking results.

[0058] Based on the above, this application also provides a method for generating tracking results based on the YOLO and MixFormer model, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the tracking result generation method based on the YOLO and MixFormer model of this application.

[0059] In this embodiment, the method for generating tracking results based on the YOLO and MixFormer models includes steps S10 to S60:

[0060] Step S10: Obtain the raw video data.

[0061] It should be noted that raw video data typically originates from various video capture devices, such as surveillance cameras, dashcams, or drones. These devices can capture large amounts of real-time video streams, providing abundant material for subsequent target detection and tracking. However, due to the diversity and unpredictability of environmental conditions, such as changes in lighting, weather conditions, occlusion, and dynamic backgrounds, data obtained directly from these sources often contains a significant amount of noise and interference.

[0062] Step S20: Preprocess the raw video data to obtain frame data.

[0063] It should be noted that this process includes, but is not limited to, adjusting frame rate, resolution conversion, noise reduction, and color correction. For example, videos shot in low-light environments may suffer from severe noise interference, requiring specialized image enhancement techniques to improve video quality; for image shakiness caused by camera movement, stabilization algorithms may be needed to correct it. Furthermore, considering the different specific needs for video data in various application scenarios, the video may need to be cropped or segmented according to actual requirements to extract the most suitable segments for analysis.

[0064] Further, step S20 also includes: reading the original video data to obtain continuous video frames. Specifically, the original video data is read from a video source (such as a camera, monitoring equipment, or video file) and decoded into a continuous sequence of image frames. This step is typically implemented using multimedia processing libraries such as OpenCV and FFmpeg to ensure that the video stream can be parsed frame by frame. During the reading process, the video resolution, frame rate, and color space (such as RGB or BGR) are checked to ensure compatibility with subsequent processing. Then, the continuous video frames are resized to scale the image size to the size required by the YOLO model input, resulting in a resized image. Specifically, since the input size of the YOLO model is usually fixed (such as 640×640), the original video frames need to be resized. The resizing methods include: proportional scaling (maintaining the aspect ratio), that is, while keeping the image aspect ratio unchanged, the shorter side is scaled to the target size, and the longer side is appropriately padded (such as zero padding or edge duplication) to avoid image distortion. Another approach is direct scaling (non-uniform scaling), where image distortion is permissible, and bilinear interpolation or nearest-neighbor interpolation can be used to scale the image to the target size. The resized image is then normalized to obtain a normalized image. This normalized image undergoes channel adjustment to obtain the input image, which is then batch-processed to obtain frame data. Specifically, the resized image is normalized. Normalization maps image pixel values ​​to a specific range, typically between [0, 1] or [-1, 1]. This ensures that images from different sources have similar scale and distribution, reducing differences caused by lighting conditions, camera characteristics, etc. This is particularly important for deep learning models, as it helps them converge faster and improves prediction accuracy. After normalization, the image's channels need to be adjusted to fit the input requirements of the YOLO model. In most cases, this involves converting the RGB image to BGR format (according to the default settings of libraries like OpenCV) and ensuring the channel order matches the model's expectations. Furthermore, some models may require the input image to have a specific number of channels; for example, grayscale images require a single channel, while color images require a three-channel input. The final step is to batch process the images to create frame data that can be used as input to the YOLO model. Batch processing primarily aims to improve computational efficiency; by processing multiple images at once instead of frame-by-frame, the parallel computing capabilities of the GPU can be fully utilized.

[0065] Step S30: Input the frame data into the YOLO model for target detection to obtain target video data.

[0066] It should be noted that the YOLO model is an efficient single-stage object detector, unique in that it can complete object detection and classification in a single network forward propagation.

[0067] It's important to understand that when frame data is fed into the YOLO model, the model first extracts image features through a series of convolutional layers. These convolutional layers capture various patterns and structural information in the image, from simple edges and textures to complex object shapes. YOLO then uses these features to predict bounding boxes and their corresponding class probabilities within each grid cell. Specifically, YOLO divides the input image into S×S grids, with each grid responsible for predicting a fixed number of bounding boxes and their corresponding confidence scores. The confidence score reflects the model's confidence in the presence of an object within that bounding box, as well as the overlap between the predicted box and the real object. After processing by the YOLO model, we obtain a set of data containing information about the target's location, size, and class—the target video data. Each piece of target information typically includes the target's location, its class (e.g., pedestrian, vehicle), and confidence score. Depending on the application scenario, we set a confidence threshold, retaining only targets with a confidence score higher than this threshold as valid detection results, thus filtering out low-quality or false positives.

[0068] Step S40: Input the target video data into the preset model to obtain the predicted location result.

[0069] It should be noted that after obtaining the target video data detected by the YOLO model, the next step is to input this data into a preset model to obtain the predicted position of the target in future frames. In this embodiment, the MixFormer model is adopted. As an advanced multi-target tracker, MixFormer combines the advantages of the Transformer architecture and classic tracking methods, and is good at handling target tracking problems in complex backgrounds.

[0070] Furthermore, based on the target location, category information, and confidence score output by the YOLO model, we need to further process each detected target to input its features into the MixFormer model. This step includes extracting the target's appearance features and its location information in the current frame. For each frame, the MixFormer model uses a global self-attention mechanism to capture the overall appearance features of the target, while extracting local detail features through deformable attention and window attention techniques. These features help distinguish similar targets and improve the tracking accuracy for occluded or fast-moving targets. High-resolution details of local features are particularly important for small targets or partially occluded targets. Next, MixFormer uses a multi-head attention mechanism for target association, that is, determining the correspondence between the detection results in the current frame and the target list in the previous frame. By calculating the similarity between each detection result and each target in the target list, it is possible to effectively determine which detection results belong to known targets and which may be new targets. This mechanism greatly enhances the model's robustness in complex environments. Finally, based on the updated feature representation, MixFormer predicts the position of each target in the next frame. This step takes into account the target's historical motion patterns and the detection results of the current frame, and uses the state vectors inside the model (such as the state estimation of Kalman filtering) to predict the target's future position, i.e., the predicted position result.

[0071] Step S50: The target video data and the predicted location results are fused to generate the target tracking result.

[0072] It's important to note that the target video data obtained from the YOLO model includes information such as the location, size, category, and confidence score of each detected target. This data provides a snapshot of the target's state in the current frame, but it lacks the ability to predict the target's position in future frames. In contrast, MixFormer, through in-depth analysis of target appearance features and motion patterns, can predict the target's position in the next frame, thus overcoming this deficiency.

[0073] It's important to understand that to effectively combine these two different types of information, methods such as weighted averaging or Kalman filtering are typically used for result fusion. For example, when using weighted averaging, weights can be assigned based on the confidence levels of the YOLO detection results and the MixFormer prediction results. If the detection confidence of a target is high, the target's position information in the final tracking result may rely more on the YOLO output; conversely, if the prediction confidence is higher, the MixFormer result is more likely to be used. This method effectively balances the relationship between immediate detection and long-term prediction, improving the accuracy of the tracking results.

[0074] On the other hand, Kalman filtering provides a probabilistic method for estimating the target's state. It updates the optimal estimate of the target's current position by combining the target state estimate from the previous moment with the current observation (i.e., the YOLO detection result). Simultaneously, Kalman filtering provides a quantitative indicator of estimation uncertainty, which is particularly useful for handling occlusion or fast-moving targets. When a target is temporarily occluded and cannot be directly observed, the Kalman filter can predict the approximate location of the target based on its previous state and adjust its estimate as the target reappears. Ultimately, the tracking result obtained after fusion processing includes not only the target's identity (ID), bounding box coordinates, and category information, but also the target's motion state (such as velocity and direction) and its historical trajectory. This comprehensive information provides strong support for subsequent video analysis, enabling effective tracking and improving the reliability of target tracking in complex situations, whether for security monitoring, traffic flow management, or other applications.

[0075] Furthermore, following step S50, the process includes overlaying the tracking results and the original video data to generate video data with target tracking results, which is then displayed in the form of a visual image. This step not only demonstrates the practical application effect of the technology but also provides users with an intuitive and visual analysis tool, enhancing usability and user experience. The tracking results contain detailed information for each tracked target, such as target ID, target location (bounding box coordinates), category label, confidence score, and motion state (including position, speed, and direction). To effectively display this information on the original video, visualization techniques are typically used to enhance the image's expressiveness. For example, different colored and styled bounding boxes can be used to distinguish different target categories, and the target's ID and category name can be displayed next to each bounding box to help observers quickly identify each target. Additionally, lines can be used to connect the positions of the same target in consecutive frames, forming trajectory lines that clearly show the target's movement path.

[0076] This embodiment acquires raw video data, preprocesses it to obtain frame data, inputs the frame data into a YOLO model for target detection to obtain target video data, inputs the target video data into a preset model to obtain predicted position results, fuses the target video data and predicted position results to generate target tracking results, and then overlays the tracking results with the raw video data to generate video data with target tracking results, which is then displayed in the form of a visual image. By combining raw video data with YOLO and MixFormer, and finally displaying the video data with tracking information in the form of a visual image, efficient and accurate multi-target tracking in complex video environments is achieved, improving the accuracy, robustness, and real-time performance of tracking.

[0077] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 The method for generating tracking results based on the YOLO and MixFormer models, step S30, further includes steps S201 to S203:

[0078] Step S201: Input the frame data into the YOLO model for target detection and obtain the detection results.

[0079] It should be noted that the detection results mentioned above include the target's location, size, category information, and confidence score. When frame data is fed into the YOLO model, it first passes through a series of convolutional layers, pooling layers, and feature pyramid networks to extract multi-level feature representations of the image. These feature maps contain various visual information, from simple to complex, such as edges, textures, and even the overall shape and structure of objects. Subsequently, YOLO uses these feature maps to predict the bounding box within each grid cell and its corresponding category probability and confidence score. Specifically, YOLO divides the input image into S×S grids, with each grid responsible for predicting several bounding boxes and their corresponding confidence scores. The confidence score reflects the probability of a target existing within that bounding box and the degree of overlap (IoU) between the bounding box and the real target. For each detected target, YOLO outputs a set of detailed information, including the target's location (usually given in the form of a bounding box, such as size (i.e., the width and height of the bounding box), category information (e.g., pedestrian, vehicle), and confidence score.

[0080] This process relies not only on the carefully designed network architecture within the model but also benefits from a large amount of training data and optimization algorithms. By training on large-scale datasets, the YOLO model learns how to effectively distinguish different objects and accurately estimate their location and size.

[0081] Step S202: Compare the confidence score in the detection result with the preset confidence threshold to obtain the comparison result.

[0082] It's important to note that each detection result includes a confidence score, reflecting the YOLO model's confidence in the presence of a target within a specific bounding box. This score is typically a floating-point number between 0 and 1, with higher values ​​indicating greater model confidence in the detection. However, not all confidence scores above zero are considered valid detections. To filter out truly meaningful targets, a pre-set confidence threshold needs to be set. This threshold is adjusted based on the specific needs of the application scenario. For example, in high-precision security monitoring scenarios, a higher threshold (e.g., 0.9) is required to reduce false alarms; while in scenarios where real-time performance is more critical, the threshold can be appropriately lowered (e.g., 0.5) to ensure more potential targets are detected.

[0083] During the comparison process, for each detection result, its confidence score is compared with a preset threshold. If the confidence score of a detection result is higher than the set threshold, the result is considered reliable and can be used for subsequent processing steps; conversely, if the score is lower than the threshold, the detection result is considered invalid and ignored. This strategy effectively balances recall and precision, avoiding a large number of false positives and false negatives.

[0084] It is worth noting that in some complex application scenarios, the confidence threshold is not fixed but dynamically adjusted according to environmental conditions. For example, the formula for automatically adjusting the detection confidence threshold based on scene complexity (such as target density and light intensity) is as follows:

[0085]

[0086] in, These are learning parameters. For average density, Indicates light intensity.

[0087] Furthermore, by combining historical data and information from the current frame, the selection of the confidence threshold can be further optimized. For example, a Kalman filter can be used to predict the target's position in the next frame, and the confidence threshold can be dynamically adjusted based on the prediction error, thereby improving overall accuracy while ensuring tracking continuity.

[0088] Step S203: Filter according to the comparison results to obtain the target video data.

[0089] It should be noted that this process ensures that only detection results that the model considers highly likely to contain the target will proceed to subsequent processing steps, thereby effectively reducing false detections and improving the reliability of tracking.

[0090] Further, step S203 also includes: when the comparison result shows that the confidence score is greater than a preset confidence threshold, the detection results corresponding to the confidence score are selected as reserved detection results; the category information of the reserved detection results is filtered according to the target category to obtain the target detection results; and the target video data is obtained based on the target detection results. Specifically, when the comparison result shows that the confidence score is greater than the preset confidence threshold, these high-confidence detection results will be selected as reserved detection results. After obtaining the reserved detection results, the next step is to further filter the category information in these results. Different application scenarios may be interested in specific target categories. For example, in traffic monitoring, the focus is on pedestrians and vehicles; while in wildlife conservation projects, the focus is on specific species of animals. Therefore, a target category list is set according to application requirements, and the reserved detection results are filtered to retain the detection results that match the target category as the target detection results.

[0091] Based on the object detection results, the final target video data can be constructed. This process includes not only organizing the target's location (bounding box coordinates), size, category information, and confidence score, but also assigning a unique identifier (ID) to each detected target. This is done to ensure that the same target can be continuously identified and tracked throughout the video sequence, thus providing consistency across frames.

[0092] This embodiment uses the YOLO model to detect targets in frame data, filters high-quality detection results based on confidence scores, and generates target video data containing location, size, category, and ID, providing accurate input for subsequent tracking.

[0093] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 The step S40, which generates tracking results based on the YOLO and MixFormer models, further includes steps S301 to S303:

[0094] Step S301: Input the target video data into the preset model to obtain the detection result of the current frame.

[0095] It should be noted that the preset model mentioned above is the MixFormer model. The detection results of the current frame include global features and local features. The global features are obtained through a global self-attention mechanism, and the local features are obtained through a window attention mechanism. As an advanced tracker, MixFormer combines the advantages of the Transformer architecture with classic tracking methods, and is particularly adept at handling target tracking problems in complex backgrounds.

[0096] First, the MixFormer model extracts features from each frame, a process that includes acquiring global and local features. Global features are captured through a global self-attention mechanism, which performs global attention calculations on all spatial locations of the input feature map, thereby identifying the overall appearance of the target and scene context information. These global features help understand the target's position within the scene and its relationship to other objects, which is particularly important when it's necessary to correlate target trajectories across frames. Furthermore, global features can adaptively adjust attention weights based on environmental complexity (such as illumination intensity and target density), prioritizing high-confidence regions and enhancing the inference ability for occluded targets.

[0097] Meanwhile, local features are extracted using a window attention mechanism. This mechanism divides the feature map into fixed-size windows, calculating local attention independently within each window. This not only reduces computational cost but also allows the model to focus on detailed features within local regions, such as edges and textures. Furthermore, local features can also be acquired through a deformable attention mechanism. This mechanism allows the model to dynamically learn the offsets of local regions, focusing on key parts of the target (such as a pedestrian's head or a vehicle's tires), enhancing its ability to capture details of deformed and occluded targets.

[0098] These two feature extraction methods complement each other: global features provide semantic-level contextual information, which helps in understanding scene layout and group movement trends; local features provide pixel-level detailed information, which is particularly crucial for small target tracking, occlusion recovery, and fast motion compensation. By combining these two types of features, MixFormer can more accurately locate and track multiple targets in complex video environments.

[0099] Step S302: The detection results of the current frame are correlated through the multi-head attention mechanism of the MixFormer model to determine the correspondence between the detected target and the known target.

[0100] It should be noted that multi-head attention mechanisms allow models to focus on information from different parts simultaneously, thereby more effectively capturing the complex relationships between targets.

[0101] Furthermore, leveraging a multi-head attention mechanism, MixFormer can calculate the similarity between each detection and known targets in previous frames. Each multi-head attention module contains multiple parallel attention heads, each focusing on a different subspace of the input, allowing the model to capture different types of correlations or patterns. For example, one attention head might focus on visual features such as color and texture, while another focuses on position and motion trajectory. To determine the correspondence between detected targets and known targets, MixFormer calculates a similarity score based on these features. Specifically, for each detection, the model compares its features with the historical features of all known targets in previous frames and calculates a similarity matrix between them. This process considers not only the appearance features of the target but also dynamic information such as its historical motion trajectory. Based on this similarity matrix, MixFormer can find the best-matching known target for each detection, maintaining high-precision association even in the presence of occlusion, rapid movement, or background interference. Once this association is established, MixFormer can use the target state information accumulated in previous frames to predict the target's position in the next frame. Furthermore, by maintaining a target list that records all currently tracked targets and their unique identifiers, the status of each target can be continuously updated in subsequent frames, ensuring consistency and continuity across frames.

[0102] Furthermore, step S302 also includes: when the detected target does not have a correspondence with a known target, acquiring the motion trajectory and appearance features of the detected target, and updating the position of the detected target based on the motion trajectory and appearance features until a correspondence exists between the detected target and a known target. Specifically, when the detected target fails to find a correspondence with a known target in the current frame, it means that the target is a newly appearing target or has been temporarily lost due to occlusion, rapid movement, or other reasons. To solve this problem and ensure continuous and accurate tracking, the MixFormer model takes a series of measures to update the position of the detected target until the association with the known target can be re-established. First, for newly appearing detected targets, it is necessary to acquire their initial motion trajectory and appearance features. The motion trajectory can be obtained in various ways, such as using a Kalman filter to estimate the predicted position based on the previous few frames, or using optical flow to analyze the pixel displacement between adjacent frames to infer the target's movement direction and speed. Appearance features are usually extracted from the target's image region by a deep learning model (such as a ReID network), including information such as color distribution, texture pattern, and shape. These features not only help distinguish different targets but can also be used in the subsequent re-identification process. Once the initial motion trajectory and appearance features of the detected target are obtained, MixFormer predicts and updates the target's position based on this information. Specifically, the model may combine historical data and visual cues from the current frame, using the motion model to predict the target's possible position in the next frame and adjust its bounding box coordinates. Simultaneously, it continuously monitors changes in the target's appearance to adapt to changes in lighting conditions or partial occlusion. In each frame, if a correspondence with a known target cannot be found, MixFormer continues the above process, updating the motion trajectory and appearance features and attempting to re-associate. During this process, long-term trajectory recovery techniques can be employed, using a cache pool and global retrieval method to search for potentially matching targets within a certain range. This method is particularly suitable for handling target re-identification problems after prolonged occlusion.

[0103] Furthermore, to further improve the success rate of re-identification, additional contextual information can be introduced, such as the behavioral patterns of other targets in the scene and the changing trends of the environmental background. By comprehensively considering multiple factors, MixFormer can more accurately locate targets that have temporarily lost contact and ultimately reinstate them into the tracking list, ensuring the stability and robustness of the entire system.

[0104] Step S303: Based on the correspondence and the detection results of the current frame, obtain the predicted position result.

[0105] It's important to note that once the correspondence is established, MixFormer updates the target's state using historical information. First, it uses a Kalman filter or a similar motion prediction model to predict the target's approximate position in the next frame based on parameters such as position and velocity from previous frames. Simultaneously, it refines the prediction results by incorporating the target's appearance features (such as embedding vectors extracted by the ReID network). This method, which integrates kinematic and visual features, can effectively address issues such as occlusion and deformation in complex environments.

[0106] Furthermore, MixFormer utilizes a multi-head attention mechanism to capture complex relationships between targets, ensuring accurate prediction of each target's position even in high-density scenes. For each associated target, the model not only predicts its position but also updates its size, orientation, and other motion state information to adapt to the dynamic changes in the target.

[0107] This embodiment utilizes the MixFormer model to obtain global and local features, achieves accurate target association through a multi-head attention mechanism, and predicts target positions based on historical information, ensuring the continuity and accuracy of tracking, and enhancing the robustness and accuracy of multi-target tracking in complex environments.

[0108] Based on the first embodiment of this application, in the fourth embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 The method for generating tracking results based on the YOLO and MixFormer models, step S50, further includes steps S401 to S404:

[0109] Step S401: The target video data and the predicted location results are fused using a weighted average method to obtain the fused target location information.

[0110] It's important to note that weights need to be assigned to the YOLO detection results and the MixFormer prediction results. These weights can be determined based on the confidence or reliability of their respective results. For example, if the YOLO detection confidence in a frame is very high, it can be assigned a larger weight; conversely, if the detection quality in the current frame is low (e.g., due to occlusion), then the system relies more on the MixFormer predictions based on information from previous frames and is given a higher weight. This dynamic weight adjustment effectively balances the relationship between immediate detection and long-term prediction, improving the overall robustness of the system.

[0111] Next, a weighted average formula is used to fuse the location information from the two sources. This method not only considers the latest detection information in the current frame but also incorporates trend analysis based on historical data, thereby generating more stable and reliable tracking results. Ultimately, the fused target location information not only contains accurate target coordinates but also reflects the system's best estimate of the target's future motion trend.

[0112] Step S402: The fused target location information is smoothed using the Kalman filter algorithm to obtain the processed target location information.

[0113] It should be noted that Kalman filtering is a recursive optimal estimation method, which is particularly suitable for state estimation problems of linear dynamic systems and can provide an effective solution even in the presence of noise and uncertainty.

[0114] First, the Kalman filter updates the target's state through two main stages: prediction and update. In the prediction stage, based on the target's state (including position, velocity, etc.) from the previous frame, the Kalman filter predicts the target's state in the current frame. This step utilizes the target's historical motion patterns to provide a preliminary estimate of its future position. For example, if a target has been moving to the right for the past few frames, the Kalman filter assumes it will continue moving in the same direction and predicts its position in the current frame accordingly.

[0115] Next, in the update phase, the fused target location information is combined with the prediction results to correct the prediction error. Specifically, the Kalman filter calculates the difference between the current measurement (i.e., the fused target location information) and the predicted value, and adjusts the target state estimate based on this difference. The key here is the calculation of the Kalman gain, which determines the importance of the measurement relative to the prediction. When the measurement is very reliable, a high Kalman gain means more confidence is given to the new measurement data; conversely, if the measurement is unreliable, more reliance is placed on the model's prediction.

[0116] Furthermore, the Kalman filter provides a quantitative indicator of estimation uncertainty, typically represented as a covariance matrix. This matrix reflects the confidence level of the target state estimate, helping to manage the target more accurately in subsequent processing. For example, in data association, the matching strategy can be adjusted based on the uncertainty of the target state, prioritizing targets with lower uncertainty.

[0117] After smoothing by the Kalman filter, the obtained target position information is not only more accurate, but also has higher continuity and stability.

[0118] Step S403: Process the processed target location information to obtain target status information.

[0119] It should be noted that this process includes not only updating position and velocity, but also maintaining target appearance features, motion history and other contextual information to ensure that the system can provide continuous and accurate tracking results.

[0120] Furthermore, based on the position and velocity estimates output by the Kalman filter, we can calculate the target's basic state information, such as its current position, velocity, and predicted position for the next frame. These parameters are crucial for maintaining continuous target tracking, especially in the face of occlusion or rapid movement, providing important predictions of future position. In addition, the Kalman filter provides a measure of uncertainty about these estimates (typically represented by a covariance matrix), which is very useful for subsequent data association steps, as it helps determine which detections are most likely to match known targets. Besides the basic state information, the target's appearance features also need to be updated. This step typically involves updating the ReID embedding vector, which is extracted from the target image and used to identify the same target across frames. Updating these features through moving averages or other strategies can improve the model's adaptability to changes in target appearance, such as variations in lighting conditions or re-identification under partial occlusion.

[0121] Step S404: Summarize the target status information into target tracking results.

[0122] It's important to note that for each tracked target, the tracking results typically contain information across multiple dimensions. The target tracking results mentioned above include the identifiers, positions, sizes, categories, motion states, and confidence scores of all tracked targets in each frame. The most fundamental element is the target's identity (ID), a unique identifier used to continuously identify the target throughout the video sequence. Next is the bounding box information, which defines the target's position and size in the current frame. The bounding box can be represented using absolute coordinates or relative parameters, depending on the application requirements. For example, in some cases, absolute coordinates like [x_min, y_min, x_max, y_max] are more intuitive, while in other scenarios, relative parameters like [x_center, y_center, width, height] may be more helpful for further calculations. In addition to positional information, the tracking results also include a confidence score, which is a metric for measuring the reliability of the detection results. A high confidence score means the model is very confident in the existence and location of the target. Furthermore, motion state is also an important component of the tracking results, including dynamic parameters such as velocity and acceleration. This information not only helps predict the target's position in future frames but also helps understand the target's behavioral patterns, such as whether it is accelerating or decelerating. Appearance features are equally important, especially the visual features captured through ReID embedding vectors. These features help the system re-identify targets after they have temporarily disappeared, especially in the presence of occlusion. Maintaining a trajectory cache pool to store historical features can significantly improve the re-identification success rate after long-term occlusion. Contextual information is also part of the tracking results, including the target's historical trajectory, class label, timestamp, and lifespan. The historical trajectory records the target's positional changes over several frames, helping to analyze its movement patterns; the class label clarifies the target's type, such as pedestrian or vehicle; and the timestamp and lifespan manage the target's lifecycle, ensuring the system can promptly update and remove inactive targets.

[0123] Finally, all this information needs to be aggregated in a structured manner and generated into an easy-to-understand and use format. For example, bounding boxes and labels can be overlaid on each frame to visually display the status of each target; or the tracking results can be exported into a standardized data format for easier subsequent analysis and processing.

[0124] This embodiment uses a weighted average to fuse target video data and predicted location, and then uses Kalman filtering to smooth the data to obtain accurate target state information, which is then summarized into tracking results, thus improving the accuracy, continuity, and robustness of multi-target tracking.

[0125] Based on the first embodiment of this application, this application also provides a tracking result generation device based on the YOLO and MixFormer model. Please refer to... Figure 5 The device includes:

[0126] Acquisition module 10 is used to acquire raw video data.

[0127] The processing module 20 is used to preprocess the raw video data to obtain frame data.

[0128] The extraction module 30 is used to input frame data into the YOLO model for target detection to obtain target video data.

[0129] The prediction module 40 is used to obtain the predicted position result by inputting the target video data into the preset model.

[0130] The results module 50 is used to fuse the target video data and the predicted location results to generate target tracking results.

[0131] The tracking result generation device based on the YOLO and MixFormer model provided in this application, employing the tracking result generation method based on the YOLO and MixFormer model in the above embodiments, can solve the technical problem of how to accurately and efficiently track multiple targets in video. Compared with the prior art, the beneficial effects of the tracking result generation device based on the YOLO and MixFormer model provided in this application are the same as those of the tracking result generation method based on the YOLO and MixFormer model provided in the above embodiments, and other technical features in the tracking result generation device based on the YOLO and MixFormer model are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0132] In one embodiment, the extraction module 30 is further configured to input frame data into the YOLO model for target detection to obtain detection results, the detection results including the target's location, size, category information and confidence score; compare the confidence score in the detection results with a preset confidence threshold to obtain comparison results; and filter according to the comparison results to obtain target video data.

[0133] In one embodiment, the extraction module 30 is further configured to: filter out the detection results corresponding to the confidence score as reserved detection results when the comparison result is that the confidence score is greater than the preset confidence threshold; filter the category information of the reserved detection results according to the target category to obtain the target detection results; and obtain the target video data based on the target detection results.

[0134] In one embodiment, the prediction module 40 is further configured to input target video data into a preset model to obtain the detection result of the current frame. The preset model is a MixFormer model. The detection result of the current frame includes global features and local features. The global features are obtained through a global self-attention mechanism, and the local features are obtained through a window attention mechanism. The detection result of the current frame is associated with the multi-head attention mechanism of the MixFormer model to determine the correspondence between the detected target and the known target. Based on the correspondence, the predicted position result is obtained according to the detection result of the current frame.

[0135] In one embodiment, the prediction module 40 is further configured to acquire the motion trajectory and appearance features of the detected target when there is no correspondence between the detected target and a known target; and update the position of the detected target according to the motion trajectory and appearance features until there is a correspondence between the detected target and a known target.

[0136] In one embodiment, the result module 50 is further configured to fuse the target video data and the predicted position result using a weighted average method to obtain fused target position information; to smooth the fused target position information using a Kalman filter algorithm to obtain processed target position information; to process the processed target position information to obtain target state information; and to summarize the target state information into a target tracking result, which includes the identifier, position, size, category, motion state, and confidence score of all tracked targets in each frame.

[0137] In one embodiment, the processing module 20 is further configured to read the original video data to obtain continuous video frames; resize the continuous video frames to scale the image size to the size required for the YOLO model input, to obtain a resized image; normalize the resized image to obtain a normalized image; adjust the channels of the normalized image to obtain an input image; and batch process the input image to obtain frame data.

[0138] This application provides a tracking result generation device based on the YOLO and MixFormer model. The tracking result generation device based on the YOLO and MixFormer model includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the tracking result generation method based on the YOLO and MixFormer model in the above embodiment 1.

[0139] The following is for reference. Figure 6This document illustrates a schematic diagram of a tracking result generation device based on the YOLO and MixFormer model suitable for implementing embodiments of this application. The tracking result generation device based on the YOLO and MixFormer model in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The tracking result generation device based on the YOLO and MixFormer model shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0140] like Figure 6 As shown, the YOLO and MixFormer-based tracing result generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the YOLO and MixFormer-based tracing result generation device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the YOLO and MixFormer model-based tracking result generation device to wirelessly or wiredly communicate with other devices to exchange data. Although various YOLO and MixFormer model-based tracking result generation devices are shown in the figures, it should be understood that implementation or possession of all of them is not required. More or fewer devices may be implemented alternatively.

[0141] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0142] The tracking result generation device based on the YOLO and MixFormer model provided in this application, employing the tracking result generation method based on the YOLO and MixFormer model in the above embodiments, can solve the technical problem of how to accurately and efficiently track multiple targets in video. Compared with the prior art, the beneficial effects of the tracking result generation device based on the YOLO and MixFormer model provided in this application are the same as those of the tracking result generation method based on the YOLO and MixFormer model provided in the above embodiments, and other technical features in this tracking result generation device based on the YOLO and MixFormer model are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0143] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0144] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0145] This application provides a computer-readable medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the tracking result generation method based on the YOLO and MixFormer model in the above embodiments.

[0146] The computer-readable medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable medium may be any tangible medium containing or storing a program that can be executed by instructions, used by a device, or used in conjunction with it. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0147] The aforementioned computer-readable medium may be included in a tracking result generation device based on the YOLO and MixFormer model; or it may exist independently and not assembled into a tracking result generation device based on the YOLO and MixFormer model.

[0148] The aforementioned computer-readable medium carries one or more programs that, when executed by a YOLO and MixFormer-based tracing result generation device, enable the YOLO and MixFormer-based tracing result generation device to write computer program code for performing the operations of this application in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using dedicated hardware-based implementations that perform the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0150] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0151] The readable medium provided in this application is a computer-readable medium, which stores computer-readable program instructions (i.e., a computer program) for executing the above-described tracking result generation method based on the YOLO and MixFormer model, and can solve the technical problem of how to accurately and efficiently track multiple targets in video. Compared with the prior art, the beneficial effects of the computer-readable medium provided in this application are the same as those of the tracking result generation method based on the YOLO and MixFormer model provided in the above embodiments, and will not be repeated here.

[0152] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the tracking result generation method based on the YOLO and MixFormer model described above.

[0153] The computer program product provided in this application can solve the technical problem of how to accurately and efficiently track multiple targets in video. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the tracking result generation method based on YOLO and MixFormer models provided in the above embodiments, and will not be repeated here.

[0154] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for generating tracking results based on the YOLO and MixFormer models, characterized in that, The method includes: Obtain raw video data; The original video data is preprocessed to obtain frame data; The frame data is input into the YOLO model for target detection to obtain the target video data; The predicted location result is obtained by inputting the target video data into a preset model. The target video data and the predicted location result are fused to generate the target tracking result; The step of obtaining the predicted location result by inputting the target video data into a preset model includes: The target video data is input into a preset model to obtain the detection result of the current frame. The preset model is a MixFormer model. The detection result of the current frame includes global features and local features. The global features are obtained through a global self-attention mechanism, and the local features are obtained through a window attention mechanism. The detection results of the current frame are correlated using the multi-head attention mechanism of the MixFormer model to determine the correspondence between the detected target and known targets; Based on the correspondence, the predicted position result is obtained according to the detection result of the current frame.

2. The method as described in claim 1, characterized in that, The step of inputting the frame data into the YOLO model for target detection to obtain target video data includes: The frame data is input into the YOLO model for target detection to obtain detection results, which include the target's location, size, category information, and confidence score. The comparison result is obtained by comparing the confidence score in the detection result with a preset confidence threshold; The target video data is obtained by filtering based on the comparison results.

3. The method as described in claim 2, characterized in that, The step of filtering based on the comparison results to obtain the target video data includes: When the comparison result is that the confidence score is greater than the preset confidence threshold, the detection result corresponding to the confidence score is selected as the reserved detection result; The category information of the reserved detection results is filtered according to the target category to obtain the target detection results; Target video data is obtained based on the target detection results.

4. The method as described in claim 1, characterized in that, The step of obtaining the predicted location result by inputting the target video data into the preset model further includes: When the detected target does not have a corresponding relationship with a known target, the motion trajectory and appearance features of the detected target are obtained; The position of the detected target is updated based on the motion trajectory and the appearance features until the detected target has a correspondence with a known target.

5. The method as described in claim 1, characterized in that, The step of fusing the target video data and the predicted location result to generate the target tracking result includes: The target video data and the predicted location result are fused using a weighted average method to obtain the fused target location information; The fused target location information is smoothed using a Kalman filter algorithm to obtain the processed target location information; The processed target location information is further processed to obtain target status information; The target state information is summarized into target tracking results, which include the identifier, position, size, category, motion state, and confidence score of all tracked targets in each frame.

6. The method as described in claim 1, characterized in that, The step of preprocessing the original video data to obtain frame data includes: The original video data is read to obtain consecutive video frames; The continuous video frames are resized to scale the image size to the size required by the YOLO model input, resulting in a resized image. The resized image is then normalized to obtain a normalized image. The normalized image is then subjected to channel adjustment to obtain the input image; The input images are batch processed to obtain frame data.

7. A tracking result generation device based on the YOLO and MixFormer model, characterized in that, The device includes: The acquisition module is used to acquire raw video data; The processing module is used to preprocess the raw video data to obtain frame data; The extraction module is used to input the frame data into the YOLO model for target detection to obtain target video data; The prediction module is used to input the target video data into a preset model to obtain a predicted location result; it is also used to input the target video data into the preset model to obtain the detection result of the current frame, wherein the preset model is a MixFormer model, and the detection result of the current frame includes global features and local features, wherein the global features are obtained through a global self-attention mechanism, and the local features are obtained through a window attention mechanism; the detection result of the current frame is associated through the multi-head attention mechanism of the MixFormer model to determine the correspondence between the detected target and known targets; and based on the correspondence, the predicted location result is obtained according to the detection result of the current frame. The results module is used to fuse the target video data and the predicted location results to generate target tracking results.

8. A tracking result generation device based on the YOLO and MixFormer models, characterized in that, The device includes: a memory, a processor, and a tracking result generation program based on the YOLO and MixFormer model stored in the memory and running on the processor, the tracking result generation program based on the YOLO and MixFormer model being configured to implement the steps of the tracking result generation method based on the YOLO and MixFormer model as described in any one of claims 1-6.

9. A storage medium, characterized in that, The storage medium stores a tracking result generation program based on the YOLO and MixFormer model. When the YOLO and MixFormer model-based tracking result generation program is executed by the processor, it implements the steps of the tracking result generation method based on the YOLO and MixFormer model as described in any one of claims 1-6.

Images