Target detection tracking method and system based on automatic detection

Abstract

The invention discloses a target detection tracking method and system based on automatic detection, and relates to the technical field of target detection tracking, and the method comprises the following steps: carrying out the target position and category labeling of an image through a labeling tool based on an image data set of a typical target, and obtaining a training data set with a label, establishing a target detection model according to a YOLO architecture, and introducing a dynamic weighted loss function; according to the training data set, a training batch is generated by using a hierarchical sampling strategy, an optimized training sample is obtained through an online difficult case mining mechanism, and iterative training is performed on the target detection model based on the generated training batch and the optimized sample; and performing target detection on the real-time video stream by using the trained target detection model to obtain the tracking position and category information of each frame. According to the method, the problem of class imbalance can be effectively solved, the detection capability of rare targets is improved, and meanwhile, the overall performance of the model is improved through difficult case mining.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and tracking, and in particular to a target detection and tracking method and system based on automatic detection. Background Art

[0002] Target detection and tracking is one of the core requirements in the security field, especially in flight guidance systems, where its performance directly affects the success or failure of the mission. Traditional target detection and tracking mainly rely on manual operation, which is not only inefficient but also susceptible to human error, especially in the face of complex and changing environments. With the rapid development of computer vision and artificial intelligence technology, target detection and tracking algorithms based on deep learning have provided new possibilities for solving this problem. These technologies can automatically identify and locate targets in complex backgrounds, and track the target's motion trajectory in continuous image sequences, greatly improving the accuracy and real-time performance of target recognition.

[0003] At present, in the field of target detection and tracking, methods based on deep learning, especially single-stage target detection algorithms represented by YOLO (You Only LookOnce), have attracted widespread attention due to their high efficiency and real-time performance. The YOLO algorithm achieves rapid positioning and recognition of targets by dividing images into grids and predicting the bounding box and category of the target in each grid; this method overcomes the shortcomings of traditional target detection algorithms, such as complex calculations and slow speed, and achieves real-time detection while ensuring accuracy, providing strong technical support for flight guidance systems, and can quickly identify and track typical targets such as armored vehicles in complex environments, providing accurate target information for subsequent guidance decisions.

[0004] However, existing target detection and tracking technologies still face some challenges when applied to flight guidance systems. First, in actual application environments, the distribution of target categories is often uneven, and some types of armored vehicles may appear less frequently, which makes it difficult for traditional training methods to effectively identify these rare categories. In addition, the existing loss function design usually cannot fully consider the difference in category frequency, resulting in the model's emphasis on learning common categories and insufficient recognition of rare categories. Second, the commonly used random sampling strategy cannot guarantee that each training batch contains samples of all categories, which may lead to insufficient or discontinuous learning of certain categories during the training process. In addition, traditional training methods often ignore the special treatment of difficult-to-classify samples. Although these samples are small in number, they are crucial to improving the generalization and robustness of the model. These problems lead to the existing technology facing complex and changeable actual environments. Problems such as low detection accuracy for rare categories, unbalanced model learning, and insufficient ability to handle difficult examples may affect the reliability and effectiveness of the flight guidance system.

[0005] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0006] In response to the problems in the related art, the present invention proposes a target detection and tracking method and system based on automatic detection, which has the advantages of effectively solving the problem of category imbalance, improving the detection ability of rare targets, and improving the overall performance of the model through difficult example mining, thereby solving the problems of category imbalance, improper sample sampling and insufficient difficult example processing in the prior art.

[0007] To this end, the specific technical solution adopted by the present invention is as follows: According to one aspect of the present invention, a target detection and tracking method based on automatic detection is provided, and the target detection and tracking method based on automatic detection comprises the following steps: S1. Based on the image dataset of typical targets, use annotation tools to annotate the target position and category of the image to obtain a labeled training dataset, and establish a target detection model based on the YOLO architecture, introducing a dynamic weighted loss function; S2. Generate training batches using a stratified sampling strategy based on the training data set, and obtain optimized training samples through an online hard example mining mechanism. Perform iterative training on the target detection model based on the generated training batches and optimized samples. S3. Use the trained target detection model to perform target detection on the real-time video stream to obtain the tracking position and category information of each frame, and predict the target's motion trajectory through the Kalman filter algorithm based on the detection results of consecutive frames.

[0008] Furthermore, based on the image dataset of typical targets, the target position and category of the image are annotated using annotation tools to obtain a labeled training dataset, and a target detection model is established according to the YOLO architecture. The introduction of a dynamic weighted loss function includes the following steps: S11, using annotation tools to annotate the target position and category of the images in the typical target image dataset, obtaining a training dataset with bounding boxes and category labels, and performing data cleaning and verification; S12. According to the YOLO architecture, build the basic network structure of the target detection model and set the network parameters. The basic network structure includes the backbone network, feature pyramid network and detection head; S13. Based on the target detection model, an initial total loss function is established, and a dynamic weighted loss function is introduced to optimize the weight distribution of samples of different categories.

[0009] Furthermore, based on the target detection model, an initial total loss function is established, and a dynamic weighted loss function is introduced to optimize the weight distribution of samples of different categories, including the following steps: S131. According to the target detection model, the bounding box regression loss, target confidence loss and category prediction loss are calculated respectively, an initial total loss function is established, and the improvement direction is determined by analyzing the contribution of each part of the loss; S132. Based on the improvement direction, design a corresponding dynamic weighted loss function, and determine the initial weight value by analyzing the category distribution of the training data set; S133. According to the initial total loss function and the dynamic weighted loss function, a loss function of the target detection model is established, and the weight distribution of samples of different categories is optimized by adjusting parameters.

[0010] Furthermore, the bounding box regression loss is calculated as:

[0011] In the formula, L box represents the bounding box regression loss; IoU represents the intersection-over-union ratio of the predicted bounding box coordinates and the true bounding box coordinates; OS Represents the Euclidean distance between the center point of the predicted box and the true box; c Indicates the diagonal length of the circumscribed rectangle of the predicted box and the true box; v represents the adjustment parameter; CK Indicates the consistency of the bounding box aspect ratio; The calculation expression of target confidence loss is:

[0012] In the formula, L obj represents the bounding box regression loss; y represents the true label; p Represents the confidence of the target detection model prediction; The calculation expression of category prediction loss is:

[0013] In the formula, L cls represents the category prediction loss; i Represents the category label; E Represents the total number of category labels; y i One-hot encoding representing the true category; p i Represents the class probability distribution predicted by the object detection model.

[0014] Furthermore, based on the improvement direction, the corresponding dynamic weighted loss function is designed, and by analyzing the category distribution of the training data set, the initial weight value is determined, including the following steps: S1321. Establish a corresponding dynamic weighted loss function according to the improvement direction; S1322. Calculate the dynamic weight coefficient based on the dynamic weighted loss function, and analyze the distribution of each category in the training data set; S1323. Set initial hyperparameters and calculate initial weight values ​​for each category.

[0015] Furthermore, according to the training data set, a stratified sampling strategy is used to generate training batches, and an optimized training sample is obtained through an online difficult example mining mechanism. Based on the generated training batches and optimized samples, the target detection model is iteratively trained, including the following steps: S21. Based on the training data set, a stratified sampling strategy is used to generate training batches to obtain balanced batches containing samples of all categories, and the frequency of occurrence of rare categories is increased by setting the sampling probability; S22. According to the samples in each training batch, the loss value is calculated by forward propagation to obtain the difficulty of each sample, and the difficult samples are screened out for key learning by setting the selection threshold; S23. Based on the screened difficult samples and other samples, the back propagation algorithm is used to update the model parameters to obtain the optimized model, and the training process is adjusted through the learning rate scheduling strategy.

[0016] Furthermore, based on the training data set, a stratified sampling strategy is used to generate training batches to obtain balanced batches containing samples of all categories, and the occurrence frequency of rare categories is increased by setting the sampling probability, including the following steps: S211. Based on the training data set, the samples are stratified using the category information to obtain a sample subset of each category, and the sample quantity distribution of each category is obtained through statistical analysis; S212, according to the sample quantity distribution of each category, the sampling probability is designed by using the inverse proportional function to obtain the sampling weight of each category, and the final sampling probability is obtained by normalization processing; S213. Based on the final sampling probability, a weighted random sampling algorithm is used to generate training batches to obtain balanced batches containing samples of all categories, and the sampling results are recorded to verify whether the occurrence frequency of rare categories is increased.

[0017] Furthermore, according to the samples in each training batch, the loss value is calculated by forward propagation to obtain the difficulty of each sample, and the difficult samples are screened out for key learning by setting the selection threshold, including the following steps: S221. Based on the generated training batch, use the target detection model to perform forward propagation to obtain the prediction result of each sample, and calculate the loss value of each sample by comparing the prediction result with the true label; S222, sorting the samples by difficulty using a sorting algorithm according to the calculated loss value, obtaining a sample list arranged in descending order of difficulty, and determining a selection range of difficult samples by setting a percentage threshold; S223. Based on the selection range of difficult samples, a secondary screening is performed using a category balance strategy to obtain the final difficult sample set, and the selection threshold of each category is adjusted to ensure that rare category samples have a higher probability of being selected.

[0018] Furthermore, the trained target detection model is used to detect targets in real-time video streams to obtain the tracking position and category information of each frame, and the motion trajectory of the target is predicted through the Kalman filter algorithm based on the detection results of consecutive frames, including the following steps: S31. Based on the trained target detection model, use the real-time video stream to perform target detection, obtain the location and category information of typical targets in each frame, and filter out the final detection result through the non-maximum suppression algorithm; S32, according to the detection results of the continuous frames, the Hungarian algorithm is used to perform target association, the ID and trajectory of the typical target are obtained, and the situation where the typical target disappears temporarily is handled by setting a time window; S33. Based on the target association results, the Kalman filter algorithm is used to establish the motion trajectory of each typical target, obtain the state estimation and prediction of the typical target, and optimize the prediction accuracy by adjusting the process noise and observation noise.

[0019] According to another aspect of the present invention, there is also provided a target detection and tracking system based on automatic detection, the target detection and tracking system based on automatic detection comprising: The target detection model building module is used to annotate the target position and category of the image based on the image dataset of typical targets using annotation tools to obtain a labeled training dataset, and to build a target detection model based on the YOLO architecture, introducing a dynamic weighted loss function; The stratified sampling optimization training module is used to generate training batches based on the training data set using the stratified sampling strategy, and obtain optimized training samples through the online difficult example mining mechanism. Based on the generated training batches and optimized samples, the target detection model is iteratively trained; The target detection trajectory tracking module is used to use the trained target detection model to perform target detection on the real-time video stream, obtain the tracking position and category information of each frame, and predict the target's motion trajectory based on the detection results of consecutive frames through the Kalman filter algorithm.

[0020] The beneficial effects of the present invention are: (1) The present invention effectively solves the problem of class imbalance by introducing a dynamic weighted loss function, and significantly improves the target detection model's ability to identify rare categories. Traditional target detection methods often perform poorly when processing unbalanced data sets, resulting in low detection accuracy for rare categories. The dynamic weighted loss function proposed in the present invention dynamically adjusts the weights of samples of different categories according to the distribution of samples of each category in the training data set. By designing a reasonable weight calculation formula, rare category samples obtain higher weights in the loss function, thereby increasing the model's learning emphasis on these categories. This method not only improves the model's detection accuracy for rare categories, but also maintains good performance for common categories, achieving a balanced improvement in overall performance. At the same time, the introduction of a dynamic weighting mechanism enables the model to better adapt to changes in the data set, thereby enhancing the model's generalization ability and robustness.

[0021] (2) The stratified sampling strategy and online hard example mining mechanism adopted by the present invention effectively solve the problems of improper sample sampling and insufficient processing of hard examples, and significantly improve the learning efficiency and detection accuracy of the model; the stratified sampling strategy ensures that each training batch contains samples of all categories, and increases the frequency of occurrence of rare category samples by adjusting the sampling probability; this method not only balances the contribution of samples of different categories in the training process, but also improves the model's learning opportunities for rare categories; at the same time, the online hard example mining mechanism calculates the loss value of each sample in real time, and screens out samples that are difficult to classify for key learning; by setting a dynamic selection threshold, it ensures that rare category samples have a higher probability of being selected as hard examples; the combination of these two mechanisms not only improves the learning efficiency of the model, but also enhances the model's ability to handle complex scenes and difficult-to-identify targets; in addition, this training strategy also helps to reduce the risk of overfitting and improve the generalization performance of the model in practical applications.

[0022] (3) The present invention realizes accurate detection and stable tracking of typical targets in real-time video streams by integrating an optimized target detection model with an efficient target tracking algorithm. Based on the trained target detection model, the present method can quickly and accurately locate and identify typical targets in real-time video streams. By introducing the Hungarian algorithm for target association, the ID switching and target loss problems in multi-target tracking are effectively solved. At the same time, the Kalman filter algorithm is used to establish a motion model for each target, which can not only accurately estimate the current state of the target, but also predict its future motion trajectory. This integrated method greatly improves the target detection and tracking performance of the system in complex scenarios, especially when dealing with situations such as short-term occlusion, rapid movement, or sudden appearance / disappearance of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 is a flow chart of a target detection and tracking method based on automatic detection according to an embodiment of the present invention; Figure 2 It is a principle block diagram of a target detection and tracking system based on automatic detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0026] According to an embodiment of the present invention, a method and system for target detection and tracking based on automatic detection are provided.

[0027] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to one embodiment of the present invention, a target detection and tracking method based on automatic detection is provided, and the target detection and tracking method based on automatic detection includes the following steps: S1. Based on the image dataset of typical targets, use annotation tools to annotate the target position and category of the image to obtain a labeled training dataset, and establish a target detection model based on the YOLO architecture, introducing a dynamic weighted loss function; S2. Generate training batches using a stratified sampling strategy based on the training data set, and obtain optimized training samples through an online hard example mining mechanism. Perform iterative training on the target detection model based on the generated training batches and optimized samples. S3. Use the trained target detection model to perform target detection on the real-time video stream to obtain the tracking position and category information of each frame, and predict the target's motion trajectory through the Kalman filter algorithm based on the detection results of consecutive frames.

[0028] In one embodiment, based on an image dataset of typical targets, an annotation tool is used to annotate the target position and category of the image to obtain a labeled training dataset, and a target detection model is established according to the YOLO architecture. The introduction of a dynamic weighted loss function includes the following steps: S11, using annotation tools to annotate the target position and category of the images in the collected typical target image dataset, obtaining a training dataset with bounding boxes and category labels, and performing data cleaning and verification; Specifically, the typical target image data set in the present invention includes the collected image data of various types of ground vehicles, including the following categories: armored vehicles, including various types of armored vehicles; heavy vehicles, including large trucks and engineering vehicles; light vehicles, including off-road vehicles and pickup trucks; special vehicles, including fire trucks and ambulances. In this embodiment, the typical target is an armored vehicle, and the corresponding typical target image data set includes 5 types of armored vehicles (Type A heavy armored vehicle, Type B light armored vehicle, Type C armored transport vehicle, Type D armored support vehicle, Type E armored reconnaissance vehicle), and a total of 10,000 high-definition images are collected with a resolution of 3840x2160 pixels.

[0029] Specifically, use a labeling tool (LabelImg tool is used in this embodiment) for labeling, and the specific steps are as follows: ① After opening the image, draw a rectangular bounding box for each armored vehicle target; ② Select the corresponding category label (A, B, C, D or E); ③ Save the labeling information as a txt file in YOLO format, and the format of each line is: [category index / center x / center y / width / height], for example: 3 / 0.347 / 0.589 / 0.126 / 0.235 represents a type C armored transport vehicle, with center coordinates (0.347, 0.589), width 0.126, and height 0.235.

[0030] Specifically, the data cleaning and verification process includes: ① using Python scripts to check the format correctness of all annotation files to ensure that each line has 5 values ​​and the coordinate values ​​are in the range of 0-1; ② randomly selecting 10% of the images for review to check the accuracy of the bounding box position and category label; ③ counting the number of each type of target to ensure a reasonable category distribution (in this embodiment, 3000 images of type A, 2500 images of type B, 2000 images of type C, 1500 images of type D, and 1000 images of type E); ④ eliminating target annotations that are severely occluded (more than 50% of the area) or unclear.

[0031] Specifically, after data cleaning and verification, we finally obtained 9,800 valid annotated images, including 12,500 valid object annotations (each image may contain multiple objects). The annotation data is saved in the form of image files and corresponding txt annotation files for subsequent training.

[0032] S12. According to the YOLO architecture, a basic network structure of the target detection model is constructed, and network parameters are set. The basic network structure includes a backbone network, a feature pyramid network, and a detection head. Specifically, in this embodiment, YOLOv5s is selected as the basic architecture, the backbone network uses CSPDarknet53, the feature pyramid network uses PANet, and the detection head contains 3 different scales. Use the yaml file to configure the network parameters, set the input size to 640x640, and the anchor frame size is obtained according to the training set statistics to obtain the small-scale detection layer [(10,13), (16,30), (33,23)], the medium-scale detection layer [(30,61), (62,45), (59,119)], and the large-scale detection layer [(116,90), (156,198), (373,326)].

[0033] It should be noted that the numbers in the anchor box size represent the size of the anchor boxes used in the YOLO model, and each pair of numbers represents the width and height of an anchor box. In the YOLOv5s used in the above embodiment, three detection layers are used, each detection layer has three anchor boxes, and a total of nine anchor boxes.

[0034] S13. Based on the target detection model, an initial total loss function is established, and a dynamic weighted loss function is introduced to optimize the weight distribution of samples of different categories.

[0035] In one embodiment, based on the target detection model, an initial total loss function is established, and a dynamic weighted loss function is introduced to optimize the weight distribution of samples of different categories, including the following steps: S131. According to the target detection model, the bounding box regression loss, target confidence loss and category prediction loss are calculated respectively, an initial total loss function is established, and the improvement direction is determined by analyzing the contribution of each part of the loss; Specifically, based on the bounding box regression loss, target confidence loss and category prediction loss, an initial total loss function is established: L 0 , the expression is: L 0 = l 1× L box + l 2× L obj + l 3× L cls ; In the formula, l 1. l 2. l3 are weight coefficients corresponding to the bounding box regression loss, target confidence loss, and category prediction loss, which are set to 1.0, 1.0, and 0.5 respectively in this embodiment.

[0036] Specifically, experiments were conducted on the validation set: a) using a validation set containing 5,000 images, including various types of armored vehicles; b) calculating the above three losses for each sample and statistically averaging them by category; c) analyzing the distribution of each loss among different categories.

[0037] Specifically, according to the experimental results, the contribution of each part of the loss is analyzed. In the above embodiment, the bounding box regression loss is calculated as follows: L box The average values ​​of each category are similar, ranging from 0.15 to 0.22; target confidence loss L obj The rare class (E type) is slightly higher, with an average of 0.18, while the common class (A type) is 0.12; while the class prediction loss L cls The average loss of the rare class (E type) is 0.42, and the average loss of the common class (A type) is 0.28; in the initial total loss, the E type samples have L cls It accounts for only 8%, while type A accounts for 35%.

[0038] Specifically, based on the analysis results, the improvement direction is determined. Through analysis, it can be found that in the above embodiment, although the rare category (E type) samples have a low category prediction loss L cls The single sample loss value in is high, but due to its small number of samples, its proportion in the total loss is the lowest, which may cause the object detection model to ignore the learning of rare categories during training. Therefore, it is finally determined that the category prediction loss L cls Dynamic weighted loss is partially introduced to increase the importance of rare category samples in the training process, thereby improving the balanced learning ability of the object detection model for all categories.

[0039] S132. Based on the improvement direction, design a corresponding dynamic weighted loss function, and determine the initial weight value by analyzing the category distribution of the training data set; S133. According to the dynamic weighted loss function, a loss function of the target detection model is established, and the weight distribution of samples of different categories is optimized by adjusting parameters.

[0040] Specifically, according to the dynamic weighted loss function DW Loss , establish the final loss function of the target detection model L In the above embodiment, the expression of the final loss function is:L = l 1× L box + l 2× L obj + l 3× α 3× L cls ; In the formula, l 1=1.0, l 2=1.0, l 3=0.5 (consistent with the initial total loss function in step S131); α 3 is the dynamic weight coefficient of the category prediction loss, which is calculated by the formula in step S132.

[0041] Specifically, by adjusting the parameters, the weight distribution of samples of different categories is optimized. ① The parameter that needs to be adjusted is the first adjustable hyperparameter c and a second adjustable hyperparameter d , set the parameter search range. In this embodiment, c ∈[1.5,2.5], step size 0.1; d ∈[0.3,0.7], step size 0.05; ②For each group ( c , d ) parameter combination performs the following steps: a) Use the current ( c , d ) Calculate the dynamic weight coefficient α i ; b) α i Substitute the dynamic weighted loss function; c) Use the loss function to train the target detection model (for example, 50 cycles); d) Evaluate the model performance on the validation set and calculate the average precision (AP) of each category and the overall average precision (mAP); It should be noted that calculating the average precision (AP) of each category and the overall average precision (mAP) is an existing technology in the field of target detection and a standard method for evaluating the performance of target detection models. These indicators are widely used in the field of computer vision.

[0042] ③ Select the parameter combination with the best comprehensive performance in all categories. In this embodiment, the final c =2.2, d =0.45; Using the optimized parameters, calculate the final dynamic weight coefficient: α 1 (Type A) ≈ 0.61;α 2 (Type B) ≈ 0.73; α 3 (Type C) ≈ 0.89; α 4 (D type) ≈ 1.13; α 5 (E type) ≈ 1.98; ④Verify the improvement effect and compare the performance of the original loss function and the dynamic weighted loss function on the validation set: In the original method, the mAP is 88.6%, and the AP of each category is 95% for type A, 93% for type B, 90,%, for type C, 87% for type D, and 78% for type E; In the improved method, the mAP is 91.8%, and the AP of each category is 94% for type A, 93% for type B, 92% for type C, 90% for type D, and 90% for type E; From the above results, it can be seen that the dynamic weighted loss function significantly improves the overall performance of the target detection model, especially the detection accuracy of rare categories (type E) has been significantly improved from 78% to 90%; at the same time, the performance of common categories has also been maintained or slightly improved.

[0043] In one embodiment, the bounding box regression loss is calculated as:

[0044] In the formula, L box represents the bounding box regression loss; IoU represents the intersection-over-union ratio of the predicted bounding box coordinates and the true bounding box coordinates; OS Represents the Euclidean distance between the center point of the predicted box and the true box; c Indicates the diagonal length of the circumscribed rectangle of the predicted box and the true box; v represents the adjustment parameter; CK Indicates the consistency of the bounding box aspect ratio; The calculation expression of the target confidence loss is:

[0045] In the formula, L obj represents the bounding box regression loss; y represents the true label; p Represents the confidence of the target detection model prediction; The calculation expression of the category prediction loss is:

[0046] In the formula, L cls represents the category prediction loss; i Represents the category label; E Represents the total number of category labels; y i One-hot encoding representing the true category;p i Represents the class probability distribution predicted by the object detection model.

[0047] In one embodiment, based on the improvement direction, designing a corresponding dynamic weighted loss function and determining the initial weight value by analyzing the category distribution of the training data set includes the following steps: S1321. Establish a corresponding dynamic weighted loss function according to the improvement direction; Specifically, a corresponding dynamic weighted loss function is established. Considering that dynamic weighting may be applied to any one or more of bounding box regression loss, target confidence loss or category prediction loss, the present invention designs a general dynamic weighted loss function DW Loss , the expression is:

[0048] In the formula, l 1. l 2. l 3 are the weight coefficients corresponding to the bounding box regression loss, target confidence loss and category prediction loss respectively; m 1. m 2. m 3 are weighted coefficients corresponding to bounding box regression loss, target confidence loss, and category prediction loss, respectively; in this embodiment, when bounding box regression loss, target confidence loss, or category prediction loss is selected as the improvement direction, its corresponding weighted coefficient is equal to the dynamic weight coefficient α i , when it is not selected as the improvement direction, its corresponding weighting coefficient is equal to 1 to adjust the importance of samples of different categories in each loss part.

[0049] S1322. Calculate the dynamic weight coefficient based on the dynamic weighted loss function, and analyze the distribution of each category in the training data set; Specifically, based on the dynamic weighted loss function, the calculation formula of the dynamic weight coefficient is designed:

[0050] In the formula, α i For the i Dynamic weight coefficient of class samples; i Indicates the category index (in this embodiment i =1,2,3,4,5 correspond to A,B,C,D,E armored vehicles respectively); β i For the i The frequency of class samples in the training dataset; N is the total number of samples (9800 in this embodiment); n i For thei Number of class samples; c and d are the first adjustable hyperparameter and the second adjustable hyperparameter respectively; Specifically, the distribution of each category in the training data set is analyzed. In the above embodiment: Type A (heavy armored vehicle) is 3000 samples; Type B (light armored vehicle) is 2500 samples; Type C (armored transport vehicle) is 2000 samples; Type D (armored support vehicle) is 1500 samples; Type E (armored reconnaissance vehicle) is 800 samples.

[0051] S1323. Set initial hyperparameters and calculate initial weight values ​​for each category.

[0052] Specifically, set the initial hyperparameters and calculate the initial weight values ​​of each category. Considering the analysis results in step S131, we mainly focus on the category prediction loss L cls The dynamic weighting of α 3. Perform calculations: ①Set the first adjustable hyperparameter c =2, the second adjustable hyperparameter d =0.5; ② Calculate the initial dynamic weight value of each category in the category prediction loss: Type A: α 1=(1-3000 / 9800)^2*(9800 / 3000)^0.5≈0.58; Type B: α 2=(1-2500 / 9800)^2*(9800 / 2500)^0.5≈0.69; Type C: α3=(1-2000 / 9800)^2*(9800 / 2000)^0.5≈0.85; Type D: α 4=(1-1500 / 9800)^2*(9800 / 1500)^0.5≈1.08; Type E: α 5=(1-800 / 9800)^2*(9800 / 800)^0.5≈1.94; Therefore, in the dynamic weighted loss function: α 3= α i ( i =1,2,3,4,5; corresponding to each category); α 1= α 2=1 (in this embodiment, the bounding box regression loss and the object confidence loss are not dynamically weighted).

[0053] In one embodiment, according to the training data set, a training batch is generated by using a stratified sampling strategy, and an optimized training sample is obtained by an online hard example mining mechanism. Based on the generated training batch and the optimized sample, iterative training of the target detection model includes the following steps: S21. Based on the training data set, a stratified sampling strategy is used to generate training batches to obtain balanced batches containing samples of all categories, and the frequency of occurrence of rare categories is increased by setting the sampling probability; S22. According to the samples in each training batch, the loss value is calculated by forward propagation to obtain the difficulty of each sample, and the difficult samples are screened out for key learning by setting the selection threshold; S23. Based on the screened difficult samples and other samples, the back propagation algorithm is used to update the model parameters to obtain the optimized model, and the training process is adjusted through the learning rate scheduling strategy.

[0054] It should be noted that the Adam optimizer is a commonly used parameter optimization algorithm in deep learning and is widely used in various deep learning tasks, such as image recognition and natural language processing. When using the Adam optimizer, you only need to specify the optimizer type as Adam in the training code and set the initial learning rate. As an existing technology, this optimizer has been implemented in most deep learning frameworks and can be called directly.

[0055] Specifically, ① combine the 12 hard examples selected in S22 with the other 52 examples in the original batch to form a new training batch; ② use the dynamic weighted loss function defined in step S1 to calculate the total loss: L = l 1× L box + l 2× L obj + l 3× α 3× L cls ; ③ Give 1.2 times the weight to the loss of difficult samples to increase its impact in back propagation: For difficult samples, L 难例 =1.2× L ; ④ Use the Adam optimizer for back propagation to update the model parameters. The initial learning rate is set to 0.001; ⑤ Use a simple learning rate decay strategy: every 50 cycles, reduce the learning rate to 90% of the original; ⑥ Repeat the above ①-⑤ until the preset training rounds are completed (set to 100 cycles in this embodiment).

[0056] Specifically, through step S23, in conjunction with the stratified sampling strategy of S21 and the online difficult example mining mechanism of S22, the goal of S2 is achieved: generating balanced training batches, identifying and focusing on learning difficult examples, and optimizing model parameters through back propagation. This method not only maintains balanced learning of samples of all categories, but also strengthens the learning of difficult examples (especially rare categories), thereby improving the overall performance of the target detection model and the ability to detect rare targets.

[0057] In one embodiment, based on the training data set, a stratified sampling strategy is used to generate training batches to obtain balanced batches containing samples of all categories, and the occurrence frequency of rare categories is increased by setting the sampling probability, including the following steps: S211. Based on the training data set, the samples are stratified using the category information to obtain a sample subset of each category, and the sample quantity distribution of each category is obtained through statistical analysis; Specifically, according to the category information of 9,800 annotated images, the dataset is divided into 5 subsets: 3,000 images of Type A (heavy armored vehicles), 2,500 images of Type B (light armored vehicles), 2,000 images of Type C (armored transport vehicles), 1,500 images of Type D (armored support vehicles), and 800 images of Type E (armored reconnaissance vehicles). The sample ratio of each category is calculated: Type A 30.6%, Type B 25.5%, Type C 20.4%, Type D 15.3%, and Type E 8.2%.

[0058] S212, according to the sample quantity distribution of each category, the sampling probability is designed by using the inverse proportional function to obtain the sampling weight of each category, and the final sampling probability is obtained by normalization processing; Specifically, the dynamic weight coefficient determined in step S1 is used α i ,in c =2.2, d =0.45, and the sampling probability is calculated.

[0059] Specifically, the calculation formula for sampling probability is: P ( i )= α i / Σ( α i );where, P ( i ) indicates the i The sampling probability of class samples; α i Indicates i Dynamic weight coefficient of class samples; Σ( α i ) represents the sum of dynamic weight coefficients of all categories.

[0060] Specifically, the final sampling probabilities are calculated as follows: P(A)=0.11, P(B)=0.13, P(C)=0.16, P(D)=0.22, and P(E)=0.38.

[0061] S213. Based on the final sampling probability, a weighted random sampling algorithm is used to generate training batches to obtain balanced batches containing samples of all categories, and the sampling results are recorded to verify whether the occurrence frequency of rare categories is increased.

[0062] Specifically, the batch size is set to 64, and samples are selected from each category subset according to the calculated sampling probability. After generating 100 batches, the number of occurrences of each category is counted: 704 for type A, 832 for type B, 1024 for type C, 1408 for type D, and 2432 for type E. It is verified that the occurrence frequency of type E (a rare category) has increased from the original 8.2% to 38%, achieving the expected effect.

[0063] In one embodiment, according to the samples in each training batch, the loss value is calculated by forward propagation to obtain the difficulty of each sample, and the difficult samples are screened out for focused learning by setting a selection threshold, including the following steps: S221. Based on the generated training batch, use the target detection model to perform forward propagation to obtain the prediction result of each sample, and calculate the loss value of each sample by comparing the prediction result with the true label; Specifically, forward propagation is performed on 64 samples. Using the trained object detection model, the input image size is 640x640. The total loss of each sample is calculated, including bounding box regression loss, object confidence loss, and category prediction loss. For example, the total loss value of samples in a batch ranges from 0.85 to 4.32, where higher loss values ​​correspond to difficult samples.

[0064] S222, sorting the samples by difficulty using a sorting algorithm according to the calculated loss value, obtaining a sample list arranged in descending order of difficulty, and determining a selection range of difficult samples by setting a percentage threshold; Specifically, the 64 samples are sorted in descending order of loss value using a sorting algorithm (sorted function is used in this embodiment). The hard case selection threshold is set to the top 25%. In the above embodiment, the 16 samples with the highest loss values ​​are selected as candidate hard cases.

[0065] S223. Based on the selection range of difficult samples, a secondary screening is performed using a category balance strategy to obtain the final difficult sample set, and the selection threshold of each category is adjusted to ensure that rare category samples have a higher probability of being selected.

[0066] Specifically, among the 16 candidate difficult examples, at least one sample is selected for each category. For type E (armored reconnaissance vehicle, a rare category), the selection threshold is increased to the top 40%. Finally, 12 difficult examples are selected, including: 2 for type A, 2 for type B, 2 for type C, 3 for type D, and 3 for type E. In this way, it is ensured that the proportion of type E samples in the difficult examples (25%) is higher than its proportion in the entire dataset (8.2%).

[0067] In one embodiment, the trained target detection model is used to detect the target in the real-time video stream to obtain the tracking position and category information of each frame, and the motion trajectory of the target is predicted by the Kalman filter algorithm based on the detection results of consecutive frames, including the following steps: S31. Based on the trained target detection model, use the real-time video stream to perform target detection, obtain the location and category information of typical targets in each frame, and filter out the final detection result through the non-maximum suppression algorithm; It should be noted that the non-maximum suppression (NMS) algorithm is a common post-processing technique in target detection and is widely used in various target detection tasks, such as face detection, vehicle detection, etc. When using the NMS algorithm, it only needs to be applied in the output processing stage of the target detection model by setting the IoU threshold and confidence threshold. Most computer vision libraries have built-in implementations of the NMS algorithm, allowing developers to easily integrate it into the target detection process without having to write complex code from scratch.

[0068] Specifically, ① use the object detection model trained in S2 to process the real-time video stream. The video resolution is 1920x1080 and the frame rate is 30fps; ② preprocess each frame, resize it to 640x640, and normalize it; ③ input the preprocessed image into the model to obtain the original detection results, including the bounding box coordinates, confidence, and category prediction; ④ apply the non-maximum suppression algorithm, set the intersection over union (IoU) threshold to 0.5, the confidence threshold to 0.3, and filter out the final detection results; ⑤ map the detection results back to the original image size, and record the center coordinates (x, y), bounding box size (w, h), category, and confidence of each typical target.

[0069] S32, according to the detection results of the continuous frames, the Hungarian algorithm is used to perform target association, the ID and trajectory of the typical target are obtained, and the situation where the typical target disappears temporarily is handled by setting a time window; It should be noted that the Hungarian algorithm is a classic combinatorial optimization algorithm that is widely used to solve data association problems in multi-target tracking. When using the Hungarian algorithm, you only need to input the cost matrix between the detection result and the tracked target, and the algorithm will automatically calculate the optimal match. As an existing technology, this algorithm only requires developers to provide the cost matrix as input to obtain the optimal match result.

[0070] Specifically, ① set a time window of size 5 to handle the situation where the typical target disappears briefly; ② for the detection result of each frame, calculate the IoU matrix with the detection result of the previous frame; ③ use the Hungarian algorithm to associate targets based on the IoU matrix, and set the threshold to 0.3; ④ assign a unique ID to the newly appeared typical target, and keep the ID of the successfully matched typical target unchanged; ⑤ for the unmatched typical target, retain its ID and the location information of the last appearance within the time window; if the typical target has not reappeared after the end of the time window, it will be removed from the tracking list.

[0071] S33. Based on the target association results, the Kalman filter algorithm is used to establish the motion trajectory of each typical target, obtain the state estimation and prediction of the typical target, and optimize the prediction accuracy by adjusting the process noise and observation noise.

[0072] It should be noted that the Kalman filter algorithm, as a classic state estimation method, is widely used in target tracking, navigation systems, signal processing and other fields. It makes the best estimate of the system state by combining the prediction model and measurement data. At present, most signal processing and control system libraries provide well-encapsulated interfaces. When using them, developers only need to focus on how to define the system model and adjust the filter parameters without going into the underlying implementation details of the algorithm.

[0073] Specifically, ① initialize a Kalman filter for each typical target to be tracked, and the state vector includes [x, y, w, h, v x ,v y ], where (x, y) is the target center coordinate, (w, h) is the bounding box size, (v x ,v y) is the speed; ② Set the initial state covariance matrix P and the process noise covariance matrix Q: P=diag([10,10,10,10,20,20]), Q=diag([25,25,10,10,1,1]); ③ Set the observation noise covariance matrix R, R=diag([50,50,50,50]); ④ For each frame: a. Prediction step, use the Kalman filter to predict the new state of each typical target; b. Update step, use the detection result as the observation value to update the state estimate of the Kalman filter; ⑤ If a typical target is not detected in the current frame, only the prediction step is executed without updating; ⑥ According to the prediction results of the Kalman filter, the predicted trajectory of each typical target is drawn to predict the position of the next 5 frames.

[0074] Specifically, S3 implements real-time target detection and tracking based on the target detection model trained by S1 and S2; S31 uses the optimized model for detection, S32 solves the target association problem, and S33 implements target motion prediction through Kalman filtering. This method not only ensures the accuracy of detection, but also realizes stable tracking and trajectory prediction of armored vehicle targets.

[0075] like Figure 2 As shown, according to another embodiment of the present invention, a target detection and tracking system based on automatic detection is also provided, and the target detection and tracking system based on automatic detection includes: The target detection model building module 1 is used to annotate the target position and category of the image based on the image dataset of typical targets using annotation tools to obtain a labeled training dataset, and to establish a target detection model based on the YOLO architecture, introducing a dynamic weighted loss function; The stratified sampling optimization training module 2 is used to generate training batches according to the training data set using the stratified sampling strategy, and obtain optimized training samples through the online difficult example mining mechanism, and iteratively train the target detection model based on the generated training batches and optimized samples; The target detection trajectory tracking module 3 is used to use the trained target detection model to perform target detection on the real-time video stream, obtain the tracking position and category information of each frame, and predict the target's motion trajectory through the Kalman filter algorithm based on the detection results of consecutive frames.

[0076] In summary, with the help of the above technical solution of the present invention, by introducing a dynamic weighted loss function, the category imbalance problem is effectively solved, and the recognition ability of the target detection model for rare categories is significantly improved. Traditional target detection methods often perform poorly when processing unbalanced data sets, resulting in low detection accuracy for rare categories. The dynamic weighted loss function proposed in the present invention dynamically adjusts the weights of samples of different categories according to the distribution of samples of each category in the training data set. By designing a reasonable weight calculation formula, rare category samples obtain higher weights in the loss function, thereby increasing the model's learning emphasis on these categories. At the same time, the introduction of the dynamic weighting mechanism enables the model to better adapt to changes in the data set and enhances the generalization ability and robustness of the model. The stratified sampling strategy and online difficult example mining mechanism adopted by the present invention effectively solve the problems of improper sample sampling and insufficient processing of difficult examples, and significantly improve the learning efficiency and detection accuracy of the model. The stratified sampling strategy ensures that each training batch contains samples of all categories, and increases the probability of sampling by adjusting the sampling probability. The occurrence frequency of rare category samples is increased. At the same time, the online difficult example mining mechanism calculates the loss value of each sample in real time, screens out difficult-to-classify samples for key learning, and sets a dynamic selection threshold to ensure that rare category samples have a higher probability of being selected as difficult examples. The combination of these two mechanisms not only improves the learning efficiency of the model, but also enhances the model's processing capabilities for complex scenes and difficult-to-identify targets. The present invention integrates the optimized target detection model with an efficient target tracking algorithm to achieve accurate detection and stable tracking of typical targets in real-time video streams. Based on the trained target detection model, the method can quickly and accurately locate and identify typical targets in real-time video streams. By introducing the Hungarian algorithm for target association, the ID switching and target loss problems in multi-target tracking are effectively solved. At the same time, the Kalman filter algorithm is used to establish a motion model for each target. This integrated method greatly improves the target detection and tracking performance in complex scenes, especially when dealing with situations such as short-term occlusion, rapid movement, or sudden appearance / disappearance of targets.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A target detection and tracking method based on automatic detection, characterized in that: The target detection and tracking method based on automatic detection includes the following steps: S1. Based on the image dataset of typical targets, use annotation tools to annotate the target position and category of the image to obtain a labeled training dataset, and establish a target detection model based on the YOLO architecture, introducing a dynamic weighted loss function; S2. Generate training batches using a stratified sampling strategy based on the training data set, and obtain optimized training samples through an online hard example mining mechanism. Perform iterative training on the target detection model based on the generated training batches and optimized samples. S3. Use the trained target detection model to perform target detection on the real-time video stream to obtain the tracking position and category information of each frame, and predict the target's motion trajectory through the Kalman filter algorithm based on the detection results of consecutive frames.

2. The target detection and tracking method based on automatic detection according to claim 1, characterized in that: The image dataset based on typical targets uses annotation tools to annotate the target position and category of the image to obtain a labeled training dataset, and establishes a target detection model according to the YOLO architecture. The introduction of a dynamic weighted loss function includes the following steps: S11, using annotation tools to annotate the target position and category of the images in the typical target image dataset, obtaining a training dataset with bounding boxes and category labels, and performing data cleaning and verification; S12. According to the YOLO architecture, a basic network structure of the target detection model is constructed, and network parameters are set. The basic network structure includes a backbone network, a feature pyramid network, and a detection head. S13. Based on the target detection model, an initial total loss function is established, and a dynamic weighted loss function is introduced to optimize the weight distribution of samples of different categories.

3. The target detection and tracking method based on automatic detection according to claim 2, characterized in that: The target detection model is based on which an initial total loss function is established, and a dynamic weighted loss function is introduced to optimize the weight distribution of samples of different categories, including the following steps: S131. According to the target detection model, the bounding box regression loss, target confidence loss and category prediction loss are calculated respectively, an initial total loss function is established, and the improvement direction is determined by analyzing the contribution of each part of the loss; S132. Based on the improvement direction, design a corresponding dynamic weighted loss function, and determine the initial weight value by analyzing the category distribution of the training data set; S133. According to the initial total loss function and the dynamic weighted loss function, a loss function of the target detection model is established, and the weight distribution of samples of different categories is optimized by adjusting parameters.

4. The target detection and tracking method based on automatic detection according to claim 3 is characterized in that: The calculation expression of the bounding box regression loss is: ; In the formula, L box represents the bounding box regression loss; IoU represents the intersection-over-union ratio of the predicted bounding box coordinates and the true bounding box coordinates; OS Represents the Euclidean distance between the center point of the predicted box and the true box; c Indicates the diagonal length of the circumscribed rectangle of the predicted box and the true box; v represents the adjustment parameter; CK Indicates the consistency of the bounding box aspect ratio; The calculation expression of the target confidence loss is: ; In the formula, L obj represents the bounding box regression loss; y represents the true label; p Represents the confidence of the target detection model prediction; The calculation expression of the category prediction loss is: ; In the formula, L cls represents the category prediction loss; i Represents the category label; E Represents the total number of category labels; y i One-hot encoding representing the true category; p i Represents the class probability distribution predicted by the object detection model.

5. The target detection and tracking method based on automatic detection according to claim 3, characterized in that: The method of designing a corresponding dynamic weighted loss function based on the improvement direction and determining the initial weight value by analyzing the category distribution of the training data set includes the following steps: S1321. Establish a corresponding dynamic weighted loss function according to the improvement direction; S1322. Calculate the dynamic weight coefficient based on the dynamic weighted loss function, and analyze the distribution of each category in the training data set; S1323. Set initial hyperparameters and calculate initial weight values ​​for each category.

6. The target detection and tracking method based on automatic detection according to claim 1, characterized in that: The method of generating training batches by using a stratified sampling strategy based on the training data set and obtaining optimized training samples by an online difficult example mining mechanism, and iteratively training the target detection model based on the generated training batches and optimized samples includes the following steps: S21. Based on the training data set, a stratified sampling strategy is used to generate training batches to obtain balanced batches containing samples of all categories, and the frequency of occurrence of rare categories is increased by setting the sampling probability; S22. According to the samples in each training batch, the loss value is calculated by forward propagation to obtain the difficulty of each sample, and the difficult samples are screened out for key learning by setting the selection threshold; S23. Based on the screened difficult samples and other samples, the back propagation algorithm is used to update the model parameters to obtain the optimized model, and the training process is adjusted through the learning rate scheduling strategy.

7. The target detection and tracking method based on automatic detection according to claim 6, characterized in that: The method of generating training batches based on the training data set using a stratified sampling strategy to obtain a balanced batch containing samples of all categories, and increasing the occurrence frequency of rare categories by setting the sampling probability includes the following steps: S211. Based on the training data set, the samples are stratified using the category information to obtain a sample subset of each category, and the sample quantity distribution of each category is obtained through statistical analysis; S212, according to the sample quantity distribution of each category, the sampling probability is designed by using the inverse proportional function to obtain the sampling weight of each category, and the final sampling probability is obtained by normalization processing; S213. Based on the final sampling probability, a weighted random sampling algorithm is used to generate training batches to obtain balanced batches containing samples of all categories, and the sampling results are recorded to verify whether the frequency of occurrence of rare categories is increased.

8. The target detection and tracking method based on automatic detection according to claim 6, characterized in that: The method of calculating the loss value by forward propagation based on the samples in each training batch to obtain the difficulty of each sample and selecting the difficult samples for focused learning by setting a selection threshold comprises the following steps: S221. Based on the generated training batch, use the target detection model to perform forward propagation to obtain the prediction result of each sample, and calculate the loss value of each sample by comparing the prediction result with the true label; S222, sorting the samples by difficulty using a sorting algorithm according to the calculated loss value, obtaining a sample list arranged in descending order of difficulty, and determining a selection range of difficult samples by setting a percentage threshold; S223. Based on the selection range of difficult samples, a secondary screening is performed using a category balance strategy to obtain the final difficult sample set, and the selection threshold of each category is adjusted to ensure that rare category samples have a higher probability of being selected.

9. The target detection and tracking method based on automatic detection according to claim 1, characterized in that: The method of using the trained target detection model to detect targets in real-time video streams to obtain tracking position and category information of each frame, and predicting the target's motion trajectory through the Kalman filter algorithm based on the detection results of consecutive frames includes the following steps: S31. Based on the trained target detection model, use the real-time video stream to perform target detection, obtain the location and category information of typical targets in each frame, and filter out the final detection result through the non-maximum suppression algorithm; S32, according to the detection results of the continuous frames, the Hungarian algorithm is used to perform target association, the ID and trajectory of the typical target are obtained, and the situation where the typical target disappears temporarily is handled by setting a time window; S33. Based on the target association results, the Kalman filter algorithm is used to establish the motion trajectory of each typical target, obtain the state estimation and prediction of the typical target, and optimize the prediction accuracy by adjusting the process noise and observation noise.

10. A target detection and tracking system based on automatic detection, used to implement the target detection and tracking method based on automatic detection according to any one of claims 1 to 9, characterized in that: The automatic detection-based target detection and tracking system includes: The target detection model building module is used to annotate the target location and category of the image based on the image dataset of typical targets using annotation tools to obtain a labeled training dataset, and to build a target detection model based on the YOLO architecture, introducing a dynamic weighted loss function; The stratified sampling optimization training module is used to generate training batches based on the training data set using the stratified sampling strategy, and obtain optimized training samples through the online difficult example mining mechanism. Based on the generated training batches and optimized samples, the target detection model is iteratively trained; The target detection trajectory tracking module is used to use the trained target detection model to perform target detection on the real-time video stream, obtain the tracking position and category information of each frame, and predict the target's motion trajectory based on the detection results of consecutive frames through the Kalman filter algorithm.

Images