A target tracking method, system, electronic device and storage medium

Abstract

The application discloses a target tracking method and system, electronic equipment and a storage medium. In the target tracking process, the detector is used to correct the target, which can prevent the problem of tracking frame deviation caused by long-time tracking. In addition, the improvement of the yolov7 target detection framework makes it have a lightweight 16-layer backbone network, so that the model can be more lightweight as a whole while ensuring the richness of extracted features and the detection accuracy, and real-time target tracking can be performed under the condition of low device performance and low device computing power. The modeling based on the target color can reduce the possibility of following the wrong target in the target tracking process. The embodiments of the application can be widely applied to the field of computer technology.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a target tracking method, system, electronic device, and storage medium. Background Technology

[0002] In recent years, thanks to the rapid development of computer vision, object tracking, as one of the core areas of computer vision, has received increasing attention. However, current object tracking algorithms often perform poorly in scenes where the target is occluded and in situations involving scale changes and cumulative offsets caused by long-term tracking.

[0003] To address these issues, the academic community has proposed tracking methods based on deep learning. Early target tracking algorithms primarily relied on target modeling or tracking target features. Target modeling methods require building a model of the target's appearance and then locating the target in subsequent frame sequences; a representative algorithm for this is optical flow. Target feature-based tracking methods utilize feature matching, first extracting target features and then finding the most similar features in subsequent frames for localization.

[0004] Subsequently, a multi-instance learning algorithm (MIL) was proposed, which develops a discriminative model by packaging all ambiguous negative and positive samples. Later, correlation filtering-based tracking algorithms were proposed for integration with the communications field. Among these algorithms, KCF and MOSSE are known for their high real-time performance, while CSRT demonstrates high accuracy. Meanwhile, researchers decomposed the final tracking task into Tracking, Learning, and Detection (TLD) subtasks, where tracking and detection mutually reinforce each other, thus proposing the TLD algorithm.

[0005] With its integration with the field of deep learning, target tracking algorithms have made another leap forward. The SiamRPN series of tracking algorithms has significantly increased their tracking accuracy through the design of Siamese networks.

[0006] However, in practical applications, although the above algorithms can achieve maximum power point tracking, they also have their own shortcomings, mainly:

[0007] (1) Optical flow and feature matching methods: These algorithms are prone to tracking failures due to the lack of consideration for background information, such as target occlusion, lighting changes, and motion blur. At the same time, the algorithms are slow and cannot meet the requirements of real-time operation.

[0008] (2) The MIL, KCF, TLD and MOSSE algorithms perform poorly in cases with target occlusion and scale transformation and cumulative offset due to long-term tracking, while the accuracy and power consumption of the CSRT algorithm are highly dependent on the target selection and have low robustness.

[0009] (3) Pure deep learning algorithms, represented by SiamRPN, have high requirements for the running device. They occupy too much CPU and memory in low-performance devices, such as mobile phones, tablets or some low-priced chips, making them difficult to run and unable to be well applied in industrial applications.

[0010] These methods place high demands on computing power, resulting in high actual costs. Furthermore, for devices like drones and robots that use lightweight computing equipment, the limitations of their device performance make it difficult to apply deep learning methods for real-time target tracking. Summary of the Invention

[0011] In view of this, embodiments of the present invention provide a target tracking method, system, electronic device, and storage medium that are highly accurate, have strong real-time performance, and have low equipment performance requirements.

[0012] This invention provides a target tracking method, comprising: constructing a detector based on an improved YOLOv7 target detection framework; wherein the improved YOLOv7 target detection framework has a 16-layer backbone network; constructing a tracker based on target color modeling and a CSRT target tracking algorithm; acquiring a video stream and an initial frame of the video stream; initializing the tracker according to the initial frame; configuring tracking time conditions; wherein the tracking time conditions include a tracking timer and a search time limit; performing target tracking on the target object using the tracker according to the video stream and the tracking time conditions; and correcting the image region where the target object is located using the detector during the target tracking process.

[0013] Optionally, the construction of the detector based on the improved YOLOv7 object detection framework includes: constructing an improved YOLOv7 model using the improved YOLOv7 object detection framework; wherein the improved YOLOv7 model has a 16-layer backbone network; training the improved YOLOv7 model to obtain an object detection model; and constructing the detector by combining a dynamic learning mechanism with the object detection model.

[0014] Optionally, configuring the tracker includes: capturing an initial frame of the video stream; determining a target region where the target object is located; determining the outline of the target object using the detector based on the initial frame and the target region; and initializing the tracker based on the initial frame and the target object outline.

[0015] Optionally, the step of tracking the target object using the tracker based on the video stream and the tracking time condition includes at least one of the following: when the target object does not exist, the target model in the last image frame containing the target object is saved, a search timeout is started, and the target is detected in the image frames within the search timeout using the detector until the target object is detected or the search timeout has expired; when the target object exists and the tracking timer ends, the target is corrected in the image frame using the target correction function of the detector and the tracking timer is reset, and the target object is tracked by the tracker after the tracking timer is reset; when the target object exists and the tracking timer has not ended, the target object is continuously tracked by the tracker.

[0016] Optionally, the step of performing target detection on image frames within the search time limit using the detector until the target object is detected or the search time limit has expired includes: when the target object is detected, stopping the search timer and continuing to track the target object using the tracker; when the search time limit has expired, issuing an abnormal alarm and stopping target tracking.

[0017] Optionally, the method further includes: correcting the target model based on changes in the target area being tracked.

[0018] Optionally, the target tracking of the target object by the tracker further includes: acquiring target region movement information between image frames; and predicting the position of the target region in the next image frame based on the target region movement information.

[0019] Embodiments of the present invention also provide a target tracking system, characterized in that it includes: a first module, the first module being used to construct a detector based on an improved YOLOv7 target detection framework; wherein the improved YOLOv7 target detection framework has a 16-layer backbone network, and the detector is used to correct the region where the target object is located; a second module, the second module being used to construct a tracker based on target color modeling and the CSRT target tracking algorithm; a third module, the third module being used to acquire a video stream and an initial frame of the video stream; a fourth module, the fourth module being used to initialize the tracker according to the initial frame; a fifth module, the fifth module being used to configure tracking time conditions; wherein the tracking time conditions include a tracking timer and a search time limit; a sixth module, the sixth module being used to perform target tracking on the target object through the tracker according to the video stream and the tracking time conditions; and a seventh module, the seventh module being used to correct the image region where the target object is located through the detector during the target tracking process.

[0020] Embodiments of the present invention also provide an electronic device, including a processor and a memory; the memory is used to store the program; the processor executes the program to implement the method described above.

[0021] Embodiments of the present invention also provide a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.

[0022] The embodiments of the present invention have the following beneficial effects: by using a detector to correct the target during the target tracking process, the problem of tracking box offset caused by long-term tracking can be prevented. Furthermore, the improvement of the YOLOv7 target detection framework results in a lightweight 16-layer backbone network, enabling the model to be more lightweight overall while maintaining rich feature extraction and detection accuracy, allowing for real-time target tracking even with low device performance and computing power; target color-based modeling reduces the possibility of following incorrect targets during target tracking. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the method steps provided in the embodiments of the present invention;

[0025] Figure 2 This is a neck framework structure diagram of the yolov7-shuffle target detection framework and the yolov7-mobile target detection framework according to embodiments of the present invention;

[0026] Figure 3 This is a diagram of the backbone framework structure of the yolov7-shuffle target detection framework provided in this embodiment of the invention;

[0027] Figure 4 This is a backbone structure diagram of the yolov7-mobile target detection framework provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0029] To address the issue of high performance requirements for computer equipment in existing technologies, embodiments of the present invention provide a target tracking method, referring to... Figure 1 , Figure 1 This is a flowchart of the method steps provided in an embodiment of the present invention, including: constructing a detector based on an improved YOLOv7 target detection framework; wherein the improved YOLOv7 target detection framework has a 16-layer backbone network, and the detector is used to correct the region where the target object is located; constructing a tracker based on target color modeling and the CSRT target tracking algorithm; acquiring a video stream; configuring the tracker and tracking time conditions according to the video stream; wherein the tracking time conditions include at least one of a tracking timer and a search time limit; and performing target tracking on the target object using the tracker according to the video stream and the tracking time conditions.

[0030] It should be noted that the previous frame and next frame described below refer to consecutive image frames in the video stream.

[0031] Specifically, embodiments of the present invention construct a detector based on an improved YOLOv7 object detection framework.

[0032] An improved YOLOv7 model is constructed using an improved YOLOv7 object detection framework.

[0033] Specifically, the YOLOv7 model is a one-stage detection model consisting of a backbone network and a detection head. The YOLOv7 model uses the backbone network to extract features from the input image, while the detection head performs detection and classification tasks. The improved YOLOv7 model in this embodiment has a 16-layer backbone network, which makes the model more lightweight while maintaining the richness of the extracted features, making it suitable for real-time detection on low-performance devices.

[0034] In embodiments of the present invention, the improved YOLOv7 target detection framework may be an improved YOLOv7-Shuffle target detection framework configured with SiLU activation function and LeakyReLU activation function, which is suitable for building detectors for low-performance chips; or it may be an improved YOLOv7-mobile target detection framework configured with H-swish activation function, LeakyReLU activation function and dynamic learning mechanism, which is suitable for building detectors for mobile devices.

[0035] The neck and head of both the Yolov7-shuffle and Yolov7-mobile object detection frameworks use Yolov7 models; among them, the reference... Figure 2 , Figure 2This is a diagram of the neck framework structure of the yolov7-shuffle and yolov7-mobile object detection frameworks according to embodiments of the present invention. In the neck, high-level features, mid-level features, and shallow features undergo convolution (Conv), upsampling (upSample), and concatenation (Concat) operations, and the generated data is then passed to the head for further processing. PyTorch is an open-source deep learning framework that provides various tensor operations and can automatically calculate gradients through automatic differentiation, facilitating the construction of various dynamic neural networks. It supports GPU / TPU acceleration for computation. Figure 2 The specific implementation process of each step can be implemented in the PyTorch framework, and will not be elaborated here.

[0036] Both the Yolov7-shuffle and Yolov7-mobile object detection frameworks have 16 layers in their backbones, which ensures richness of extracted features while making the models more lightweight and suitable for real-time detection on low-performance devices. (See reference...) Figure 3 , Figure 3 This is a diagram of the backbone framework of the YOLOv7-shuffle object detection framework provided in this embodiment of the invention. In the backbone framework of the YOLOv7-shuffle object detection framework, channel splitting is performed first. A portion of the image channels is processed through a 1×1 convolutional layer, a 3×3 DW convolutional layer, and a 1×1 convolutional layer. The resulting data is concatenated with the data from the other portion of the channels, and then channel shuffling is performed. The DW convolutional layer is a depthwise separable convolution, which can further reduce computational cost and the parameter size of the large kernel. (Refer to...) Figure 4 , Figure 4 This is a structural diagram of the backbone framework of the YOLOv7-Mobile object detection framework provided in this embodiment of the invention. In the backbone framework of the YOLOv7-Mobile object detection framework, the image data is subjected to operations such as convolution, batch normalization (BN), dynamic learning, and drop block regularization through activation functions to obtain the output image.

[0037] In this embodiment of the invention, the dynamic learning mechanism of the YOLOv7-Mobile object detection framework is to multiply each channel by a weight coefficient. During each training, the weight ratio is adjusted according to the error between the real bounding box and the predicted bounding box. After multiple iterations, the weight coefficient can obtain the most suitable parameters.

[0038] The SILU activation function in this embodiment of the invention is:

[0039] SiLU(x) = x × simoid(x)

[0040] Where SiLU represents the SiLU activation function; x represents the input tensor; and simoid represents the simoid activation function.

[0041] The LeakyReLU activation function in this embodiment of the invention is:

[0042]

[0043] Where LeakyReLU represents the LeakyReLU activation function; x represents the input tensor; and α represents a custom coefficient.

[0044] The swish activation function in this embodiment of the invention is:

[0045] swish(x) = x × simoid(βx)

[0046] Where swish represents the swish activation function; x represents the input tensor; β represents a custom coefficient; and simoid represents the simoid activation function.

[0047] The h-swish activation function in this embodiment of the invention is:

[0048] h-swish(x) = x × ReLU6(x+3) / 6

[0049] Where h-swish represents the h-swish activation function; x represents the input tensor; and ReLU represents the ReLU activation function.

[0050] A dynamic learning mechanism is added to the yolov7-mobile object detection model. Specifically, a Squeeze operation is performed on a 1×1 pooling layer output by global average pooling. The Squeeze operation is a data dimension compression operation. Then, a learning activation operation is performed by combining a fully connected layer + LeakyReLU activation function + a fully connected layer + h_sigmoid activation function.

[0051] The embodiments of the present invention employ AdamW as an optimizer, which can seek a smoother descent mode during training, thereby accelerating the training speed and reducing the possibility of getting stuck in a local optimum.

[0052] S120. Train the improved YOLOv7 object detection model using the training set to obtain the object detection model.

[0053] This object detection framework is used to correct target objects during the tracking process.

[0054] Specifically, the model training process is as follows: images are read from the dataset, input into the model for reverse mapping, the weight coefficients are automatically modified according to the set loss function, and finally, the end-to-end output is the image with completed detection and labeling. In this embodiment of the invention, the loss function used during model training is the CIOU function, the expression of which is:

[0055]

[0056] Where CIOU represents the loss function, IOU represents the maximum intersection-union ratio, and d o d represents the Euclidean distance between the center points of the target box and the predicted box. c Represents the diagonal distance of the target box; v is a parameter that measures the consistency of aspect ratio.

[0057] The formula for calculating v is:

[0058]

[0059] Among them, w gt h represents the width of the actual bounding box. gt Represents the height of the actual bounding box; w p h represents the width of the prediction box. p This represents the width of the prediction box.

[0060] The formula for calculating IOU is:

[0061]

[0062] Where P represents the predicted bounding box and GT represents the ground truth bounding box.

[0063] Build a tracker based on target color modeling and the CSRT target tracking algorithm.

[0064] Specifically, the workflow of the target-based CSRT target tracking algorithm consists of three processes: initialization, scale estimation, and tracking update.

[0065] Based on the input image, the target region Pt in the next frame is located. The location with the highest correlation between the filter Ht-1 of the previous frame and the image patch feature f extracted from the target object at position Pt-1 in the previous frame is calculated and weighted using channel reliability score. OpenCV is a lightweight and efficient open-source computer vision and machine learning software library that implements many general algorithms in image processing and computer vision. The initialization and scale estimation implementation process of this embodiment is based on the above-mentioned OpenCV function library, and will not be described in detail here.

[0066] The scale St of the target object is estimated by the target region Pt in the next frame. It should be noted that the scale in this embodiment of the invention refers to the length, width and rotation angle of the target.

[0067] During the tracker update process:

[0068] Extract the foreground and background histograms of the image, update the foreground and background histograms, estimate the reliability map m, estimate the filter h for the next frame based on the reliability map m, estimate the pass reliability w based on the filter h for the next frame, update the tracker filter Ht based on the filter h, and update the channel reliability Wt.

[0069] It should be noted that the tracker update process is also based on the OpenCV function library mentioned above, and will not be elaborated further.

[0070] In the tracking process of this invention embodiment, motion prediction is performed on each frame of image while the object is moving, which can reduce the amount of computation, speed up the tracking speed, and effectively track the target without losing it even when the target is moving fast.

[0071] The specific process of motion prediction is as follows:

[0072] (1) Obtain the three past frames of images and store them in array V;

[0073] (2) Read the images in the data V in sequence and obtain the coordinates (x, y) of the upper left corner of the prediction box of the image;

[0074] (3) Calculate the offset between every two consecutive frames. The formula for calculating the offset is:

[0075] x off =x1-x2

[0076] y off =y1-y2

[0077] Where, x off y off This represents the offset between two frames; (x1, y1) is the top-left corner coordinate of the previous frame; (x2, y2) is the top-left corner coordinate of the current frame.

[0078] (4) Calculate the weighted average of the three frames with weights of 0.5, 0.3, and 0.2 respectively to obtain the total offset;

[0079] (5) Obtain the size of the target box in the previous frame and the coordinates of its upper left corner, and add the offset to predict the coordinates of the target box in the next frame.

[0080] Get the video stream and its initial frame.

[0081] Specifically, the acquired video stream can be real-time video or local video, and the first frame is extracted from the video stream as the initial frame.

[0082] The tracker is initialized based on the initial frame.

[0083] Specifically, the target region where the target object is located is determined in the initial frame. In this embodiment of the invention, determining the target region can be either obtaining a square region of the target object framed in the initial frame or obtaining a rectangular region of the target object framed in the initial frame; based on the initial frame and the target region, target detection is performed on the target object using a detector to determine the target object's outline; and the tracker is initialized based on the initial frame and the target object's outline.

[0084] In embodiments of the present invention, in order to adapt to devices with different performance and achieve corresponding target tracking effects in different scenarios (such as scale changes, target occlusion, lighting changes, fast movement, etc.), the tracker parameters can be adjusted. The tracker parameters to be adjusted may include whether to use channel weights, whether to use image segmentation, the type of image to be read (grayscale or color), the window function used for data truncation, various parameters of the optimization algorithm, the size of image scaling, etc.

[0085] The following describes the window function of the tracker:

[0086] The window function used in this embodiment of the invention is the Kaiser window function. The Kaiser window function is an optimization window whose optimization criterion is: for a given signal energy, a signal with a finite time bandwidth is required to maximize its energy within that bandwidth. Therefore, the Kaiser window function has strong adaptability to small objects in the UAV view. The expression for the Kaiser window function is:

[0087]

[0088]

[0089] Where α represents the user-defined input parameter; β represents the parameter determined based on the value of α; n represents the length of the input window function, and the range of n is [0, N-1]; I0(β) represents the first-order modified zero-order Bessel function.

[0090] Configure tracking time conditions, which include the tracking timer and search time limit.

[0091] If the target object does not exist, the last image frame containing the target object is saved, the search time limit is started, and the target is detected by the detector on the image frames within the search time limit until the target object is detected or the search time limit has expired.

[0092] Specifically, when the target object is not found in the currently processed image frame, the target model Target from the last frame in which the target object was found is saved, a search timer is started, and the detector performs target detection on the image frames transmitted within the search time limit to determine if there is a target object matching Target. If the target object is detected, the tracker is updated, and new image frames are processed. If the search time limit has expired but the target object has not been detected, an exception is issued, and tracking stops.

[0093] When the target object exists and the tracking timer ends, the image frame is corrected by the target correction function of the detector and the tracking timer is reset. After the tracking timer is reset, the tracker continues to track the target object.

[0094] If the target object exists and the tracking timer has not expired, the target object will be continuously tracked by the tracker.

[0095] Specifically, the tracking timer can automatically correct the target after tracking for a period of time, and can periodically detect the target. The detected target box is compared with the current tracking box of the tracker. When the comparison error reaches the error threshold, the target position obtained by the target detection replaces the current tracking box to achieve target correction, thereby preventing the tracking box offset problem caused by long-term tracking.

[0096] During the tracking process, the color of the target object is modeled to obtain the target object color model. The color of the tracking box area of ​​the tracker is compared with the target object color model to determine whether the object selected in the tracking box is the target object. If the comparison is successful, the comparison stops and the tracking box details are returned. If the comparison fails, the matching is repeated to find the next similar target until the video stream ends. This can reduce the possibility of following the wrong target due to misjudgment during the target tracking process.

[0097] Embodiments of the present invention also provide a target tracking system, comprising: a first module for constructing a detector based on an improved YOLOv7 target detection framework; wherein the improved YOLOv7 target detection framework has a 16-layer backbone network, and the detector is used to correct the region where the target object is located; a second module for constructing a tracker based on target color modeling and the CSRT target tracking algorithm; a third module for acquiring a video stream and an initial frame of the video stream; a fourth module for initializing the tracker according to the initial frame; a fifth module for configuring tracking time conditions; wherein the tracking time conditions include a tracking timer and a search time limit; and a sixth module for performing target tracking on the target object using the tracker according to the video stream and the tracking time conditions.

[0098] Embodiments of the present invention also provide an electronic device, including a processor and a memory; the memory is used to store a program; the processor executes the program to implement the method described above.

[0099] Embodiments of the present invention also provide a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.

[0100] The embodiments of the present invention have the following beneficial effects:

[0101] 1. By incorporating a lightweight target detection model into the target tracking process, target detection can be performed on low-performance, low-computing-power devices.

[0102] 2. It can compare the detection bounding box of the target with the current tracking bounding box of the tracker to correct the target and prevent the tracking bounding box from shifting due to long-term tracking.

[0103] 3. By comparing the color of the tracking bounding box area with the color model of the target object, the possibility of following the wrong target due to misjudgment during the target tracking process can be reduced.

[0104] The following is an application scenario provided by an embodiment of the present invention:

[0105] In applications involving target tracking on mobile devices, a detector based on an improved YOLOv7 target detection framework is constructed. This improved YOLOv7 framework has a 16-layer backbone network. A tracker based on target color modeling and the CSRT target tracking algorithm is built. The video stream and its initial frames are acquired. The tracker is initialized based on the initial frames. Tracking time conditions are configured, including a tracking timer and a search time limit. Based on the video stream and the tracking time conditions, the tracker performs target tracking. During target tracking, the detector corrects the image region containing the target object.

[0106] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0107] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0108] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0110] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0111] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0112] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0113] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0114] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A target tracking method, characterized in that, include: A detector based on an improved YOLOv7 object detection framework is constructed; the improved YOLOv7 object detection framework has a 16-layer backbone network. Construct a tracker based on target color modeling and the CSRT target tracking algorithm; Acquire the video stream and the initial frame of the video stream; The tracker is initialized based on the initial frame; Configure tracking time conditions; wherein, the tracking time conditions include a tracking timer and a search time limit; Based on the video stream and the tracking time conditions, the tracker performs target tracking on the target object. During target tracking, the detector corrects the image region where the target object is located; The improved YOLOv7 object detection framework includes an improved YOLOv7-Mobile object detection framework configured with a dynamic learning mechanism; The dynamic learning mechanism is as follows: Output via global average pooling The pooling layer performs a Squeeze operation; Learning activation computation is performed through a combination of fully connected layers, LeakyReLU activation function, and h_sigmoid activation function.

2. The target tracking method according to claim 1, characterized in that, The detector constructed based on the improved YOLOv7 object detection framework includes: An improved YOLOv7 model is constructed using an improved YOLOv7 object detection framework; wherein, the improved YOLOv7 model has a 16-layer backbone network; The improved YOLOv7 model is trained to obtain an object detection model; A detector is constructed by combining a dynamic learning mechanism with the object detection model.

3. The target tracking method according to claim 1, characterized in that, The configuration of the tracker includes: Extract the initial frame of the video stream; Determine the target area where the target object is located; The target object outline is determined by the detector based on the initial frame and the target region; The tracker is initialized based on the initial frame and the target object outline.

4. The target tracking method according to claim 1, characterized in that, The step of tracking the target object using the tracker based on the video stream and the tracking time conditions includes at least one of the following: If the target object does not exist, the target model in the last image frame containing the target object is saved, the search time limit is started, and the target is detected in the image frames within the search time limit by the detector until the target object is detected or the search time limit has expired. When the target object exists and the tracking timer ends, the image frame is corrected by the target correction function of the detector and the tracking timer is reset. After the tracking timer is reset, the tracker continues to track the target object. If the target object exists and the tracking timer has not ended, the target object will be continuously tracked by the tracker.

5. The target tracking method according to claim 4, characterized in that, The step of performing target detection on image frames within the search time limit using the detector until the target object is detected or the search time limit has expired includes: When the target object is detected, the search timer stops, and the tracker continues to track the target object. When the search time limit has expired, an abnormal alarm will be issued and target tracking will be stopped.

6. The target tracking method according to claim 4, characterized in that, The method further includes: The target model is corrected based on changes in the target area being tracked.

7. The target tracking method according to claim 1, characterized in that, The target tracking of the target object using a tracker further includes: Acquire target region movement information between image frames; The position of the target region in the next image frame is predicted by the movement information of the target region.

8. A target tracking system, characterized in that, include: The first module is used to construct a detector based on an improved YOLOv7 target detection framework; wherein the improved YOLOv7 target detection framework has a 16-layer backbone network, and the detector is used to correct the region where the target object is located. The second module is used to build a tracker based on target color modeling and the CSRT target tracking algorithm; The third module is used to acquire the video stream and the initial frame of the video stream; The fourth module is used to initialize the tracker based on the initial frame; The fifth module is used to configure tracking time conditions; wherein, the tracking time conditions include a tracking timer and a search time limit; The sixth module is used to perform target tracking on the target object using the tracker based on the video stream and the tracking time conditions; The seventh module is used to correct the image region where the target object is located through the detector during the target tracking process; The improved YOLOv7 object detection framework includes an improved YOLOv7-Mobile object detection framework configured with a dynamic learning mechanism; The dynamic learning mechanism is as follows: Squeeze operation is performed on the pooling layer output by global average pooling; Learning activation computation is performed through a combination of fully connected layers, LeakyReLU activation function, and h_sigmoid activation function.

9. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 7.

Images