Target tracking method, device, electronic device and readable storage medium

Abstract

The present application discloses a target tracking method, device, electronic device and readable storage medium. The method includes obtaining the detection frame of each target to be tracked in the video to be processed, and using the local texture features of the previous frame image as the supplementary information of each detection frame of the current frame image. Based on each detection frame carrying the supplementary information, a corresponding tracking frame is generated, and the multi-scale local features of the detection frame and the tracking frame of each target to be tracked are calculated; based on the multi-scale local features, the image matching algorithm is used to perform multiple cyclic matching on each detection frame and each tracking frame, and multi-target tracking is performed based on the successfully matched detection frame and tracking frame, which effectively improves the accuracy of multi-target tracking and can meet the actual business needs of high-precision target tracking.

Description

Technical Field

[0001] The present application relates to the technical field of visual image processing, and in particular to a target tracking method, device, electronic device and readable storage medium. Background Art

[0002] As we all know, when the weather conditions are good, the visible light images collected by ordinary visible light equipment can meet the needs of most image processing fields. However, in rainy days, foggy days or at night, the imaging effect of visible light equipment is poor, and the obtained visible light images are blurred, which is not conducive to back-end image processing. Long-wave infrared uses its passive imaging principle to make up for the situation that visible light cannot obtain high-quality image effects in bad weather. Since infrared images and visible light images have their own advantages and disadvantages, many fields such as autonomous driving, intelligent security, remote sensing and industrial monitoring will combine infrared images and visible light images for image analysis at the same time, and dual-light equipment that can collect visible light images and infrared images at the same time is born. In view of the need for all-weather uninterrupted detection, more and more dual-light equipment is deployed, and the demand for visual image processing technology is also gradually increasing. In the application field of real-time monitoring using dual-light equipment, target detection and tracking is one of its important machine vision business needs.

[0003] In some application scenarios, such as security monitoring and autonomous driving, it is often necessary to track many targets at the same time, that is, multi-target tracking, which cannot be achieved using only target detection algorithms or single target tracking algorithms. In these application scenarios, monitoring the surrounding environment through infrared equipment and tracking surrounding targets is an important means of achieving night monitoring. Due to the characteristics of infrared images such as low signal-to-noise ratio, spatial discontinuity, and susceptibility to ambient temperature, they lack more information compared to general visible light images, increasing the difficulty of multi-target tracking on infrared images. In the current multi-target tracking task, the more accurate identification and tracking of multiple targets in infrared images, such as people and vehicles, is a key technical problem to be solved.

[0004] In the problem of multi-target tracking, the existing target trajectory is usually matched according to the detection results of the target in each frame of the image; for newly appearing targets, new targets need to be generated; for targets that have left the field of view of the image acquisition device, the trajectory tracking needs to be terminated. In this process, the matching of targets and detections can be regarded as target re-identification. The multi-target tracking algorithm in the related art is overly dependent on the results of target detection. If the target detection results are inaccurate or occlusion occurs, the tracking information will be inaccurate or lost, which makes it difficult to meet the business needs of actual application scenarios. Summary of the invention

[0005] The present application provides a target tracking method, device, electronic device and readable storage medium, which effectively solves the current situation of inaccurate target tracking results or target tracking loss due to occlusion, improves the accuracy of target tracking, and can meet the actual business needs of high-precision target tracking.

[0006] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0007] An embodiment of the present invention provides a target tracking method, including:

[0008] Obtain the detection frame of each target to be tracked in the video to be processed, and use the local texture features of the previous frame image as supplementary information for each detection frame of the current frame image;

[0009] Generate corresponding tracking frames based on each detection frame carrying supplementary information, and calculate multi-scale local features of the detection frame and tracking frame of each target to be tracked;

[0010] The image matching algorithm is used to perform multiple cycles of repeated matching of each detection frame and each tracking frame to obtain successfully matched detection frames and tracking frames.

[0011] Optionally, the image matching algorithm is a Hungarian algorithm, and after repeatedly matching each detection frame and each tracking frame using the image matching algorithm for multiple times, and before obtaining the successfully matched detection frame and tracking frame, the method further includes:

[0012] For the unmatched detection frames and tracking frames to be matched, the optimized intersection-and-union ratio relationship is called to calculate the optimized intersection-and-union ratio between each detection frame to be matched and each tracking frame to be matched;

[0013] Based on the optimized value of the intersection-over-union ratio between each detection frame to be matched and each tracking frame to be matched, the Hungarian algorithm is used to perform repeated matching again for multiple cycles;

[0014] The optimized IoU relationship is determined based on the IoU of the current detection frame to be matched and the current tracking frame to be matched, the distance between the center points of the current tracking frame to be matched and the current detection frame to be matched, and the frame size information of the current detection frame to be matched and the current tracking frame to be matched.

[0015] Optionally, the performing multiple cycles of repeated matching of each detection frame and each tracking frame using an image matching algorithm includes:

[0016] According to the order of the trajectory loss time of the target to be tracked from small to large, each detection frame and each tracking frame are matched repeatedly for multiple cycles until the current number of iterations reaches the maximum number of iterations.

[0017] Optionally, after calculating the multi-scale local features of the detection frame and the tracking frame of each target to be tracked, and before using the image matching algorithm to perform multiple cycles of repeated matching on each detection frame and each tracking frame, the method further includes:

[0018] Normalizing the multi-scale local features of each detection frame and each tracking frame of each target to be tracked to obtain a normalized multi-scale local feature vector;

[0019] For each detection frame, the inner product of the normalized multi-scale local feature vector of the current detection frame and the normalized multi-scale local feature vector of each tracking frame is calculated respectively, so as to construct a weight coefficient matrix of the Hungarian algorithm based on the inner product result.

[0020] Optionally, generating a corresponding tracking frame based on each detection frame carrying the supplementary information includes:

[0021] Calculate the position information of each detection frame in the current frame image according to the motion state parameters of each detection frame;

[0022] Calculate the Kalman filter gain by presetting the observation matrix;

[0023] The position information of the corresponding tracking frame is obtained according to the position information of each detection frame and the Kalman filter gain calculation;

[0024] Correspondingly, after obtaining the successfully matched detection frame and tracking frame, the method further includes:

[0025] The parameters of the Kalman filter are updated using the successfully matched detection frame and tracking frame.

[0026] Optionally, the using the local texture features of the previous frame image as supplementary information of each detection frame of the current frame image includes:

[0027] Calculating the position information of each detection frame in the current frame image according to the motion state parameters of each detection frame, so as to determine the image area where each detection frame is located in the current frame image;

[0028] For each detection frame, calculate the gradient information of each pixel in the target image area corresponding to the current detection frame; calculate the gradient histogram information based on the gradient information of each pixel; perform block normalization processing based on the target pixel information used in the gradient histogram information calculation process to obtain block feature information; construct the local texture feature of the current detection frame based on all block feature information in the target image area;

[0029] Constructing multiple target frames with the same width and height within a range where the distance from the center point of the detection frame of the previous frame is a first preset value, and calculating local texture features within the range of each target frame respectively;

[0030] The local texture features of each target frame are subjected to an inner product operation with the local texture features of the detection frame corresponding to the current frame image, and the target frame with the largest inner product value is used as supplementary information.

[0031] Optionally, calculating the position information of each detection frame in the current frame image according to the motion state parameters of each detection frame includes:

[0032] The motion state information of each target to be tracked is represented by using the center point position of the detection frame, the frame size information, and the speed information corresponding to the center point position and the frame size information in the image coordinate system;

[0033] Based on the motion state information of each target to be tracked and the fact that each target to be tracked moves at a uniform speed, a state transfer matrix for describing the target motion relationship between two adjacent frames of the video to be processed is constructed;

[0034] The motion state parameters at the previous moment are used to predict the operating state parameters at the current moment according to the state transfer matrix and the motion process noise, so as to determine the position coordinates of each detection frame in the current frame image based on the motion state parameters at the current moment.

[0035] Another aspect of the present invention provides a target tracking device, including:

[0036] The detection information acquisition module is used to obtain the detection frame of each target to be tracked in the video to be processed, and use the local texture features of the previous frame image as the supplementary information of each detection frame of the current frame image;

[0037] A tracking frame calculation module, used to generate a corresponding tracking frame based on each detection frame carrying supplementary information;

[0038] A multi-scale feature calculation module is used to calculate the multi-scale local features of the detection frame and tracking frame of each target to be tracked;

[0039] The matching module is used to use the image matching algorithm to perform multiple cycles of repeated matching of each detection frame and each tracking frame to obtain a successfully matched detection frame and tracking frame.

[0040] An embodiment of the present invention further provides an electronic device, comprising a processor, wherein the processor is configured to implement the steps of the target tracking method as described in any of the preceding items when executing a computer program stored in a memory.

[0041] Finally, an embodiment of the present invention further provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the target tracking method as described in any of the preceding items are implemented.

[0042] The advantage of the technical solution provided by the present application is that the local texture features of the image of the previous frame are added as additional supplementary information in the generation stage of the detection frame of the current frame, which is a supplement to the previous target detection task. Since the tracking frame is generated by the detection frame, this can solve the problem that the target detection model does not detect the target in the current frame to a certain extent, and can also solve the problem that the tracking frame is lost in some frames, which can avoid the phenomenon that the detection frame is missed and the target to be tracked is lost to the greatest extent, and effectively improve the tracking accuracy of the target. Further, the matching between the detection frame and the tracking frame is performed based on multi-scale local features. The multi-scale local features pay more attention to the local features of the image, are insensitive to rotation, scale scaling, and brightness changes, and maintain a certain degree of stability to changes in perspective, affine transformation, and noise. Multiple matching screening within a certain time range can retrieve the occluded target, reduce the number of ID changes that occur when the target is occluded and then appears, so as to solve the problem of the target being occluded for a long time, effectively solve the current situation that the target tracking result is inaccurate or the target tracking is lost due to occlusion, improve the accuracy of target tracking, and meet the actual business needs of high-precision target tracking.

[0043] In addition, the embodiments of the present invention also provide corresponding implementation devices, electronic devices and readable storage media for the target tracking method, which further make the method more practical, and the devices, electronic devices and readable storage media have corresponding advantages.

[0044] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A flowchart of a target tracking method provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a Gaussian pyramid and a Gaussian difference pyramid provided in an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of a multi-scale local feature construction process provided by an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of a gradient histogram calculation process provided by an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of block collection provided by an embodiment of the present invention;

[0051] Figure 6 A schematic diagram of a HOG feature supplement detection frame provided in an embodiment of the present invention;

[0052] Figure 7 A schematic diagram of a tracking result without adding HOG feature supplementation provided in an embodiment of the present invention;

[0053] Figure 8 A schematic diagram of a tracking result with HOG feature supplement provided in an embodiment of the present invention;

[0054] Fig. 9 A schematic diagram of a flow chart of another target tracking method provided by an embodiment of the present invention;

[0055] Fig.10 A schematic diagram of tracking results in an illustrative example provided in an embodiment of the present invention;

[0056] Fig.11 A schematic diagram of tracking results in another illustrative example provided by an embodiment of the present invention;

[0057] Fig.12 A structural diagram of a specific implementation of a target tracking device provided by an embodiment of the present invention;

[0058] Fig.13 A structural diagram of a specific implementation of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] The terms "first", "second", "third", "fourth", etc. in the specification and claims of this application and the above drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0061] After introducing the technical solutions of the embodiments of the present invention, various non-limiting implementation methods of the present application are described in detail below.

[0062] See first Figure 1 , Figure 1 A flow chart of a target tracking method provided by an embodiment of the present invention is provided. In this embodiment, multi-target tracking using a feature supplementation and retention method can realize re-tracking of blocked targets such as people and vehicles in video images such as infrared video images, and can meet the real-time and accuracy requirements in practical applications. The embodiment of the present invention may include the following contents:

[0063] S101: Acquire detection frames of each target to be tracked in the video to be processed, and use local texture features of the previous frame image as supplementary information of each detection frame of the current frame image.

[0064] The video to be processed in this embodiment can be a visible light video, an infrared video, or a fusion video of visible light and infrared light, which does not affect the implementation of this application. The target to be tracked is the target that needs to be tracked in the current application scenario. The target to be tracked can be multiple or one. The target to be tracked can be multiple different types of targets or multiple targets of the same type. For example, a person and a car can be tracked at the same time, or multiple people can be tracked, or multiple cars can be tracked. Before executing this step, a target detection model can be constructed based on any existing target detection method, such as a target detection model constructed based on a convolutional neural network model, yolov3, etc. The construction process of the target detection model can refer to the content recorded in the relevant technology of the adopted target detection algorithm, and will not be repeated here. The trained or constructed target detection model is used to identify the targets in each frame image in the video to be processed, and the corresponding detection frame is output. Taking into account the accuracy of the target detection model, there may be missed detections or the target may not be detected due to occlusion. In order to improve the target tracking accuracy, this step can also use the local texture features of the previous frame image of the current frame image as supplementary information for each detection frame of the current frame image. That is to say, when tracking the current frame image, it is based on the local texture features of the current frame image and the previous frame image. In this way, even if the target detection model is inaccurate or the target is lost, the lost target can be found again based on the local texture features of the previous frame image, thereby improving the overall target tracking accuracy.

[0065] S102: Generate a corresponding tracking frame based on each detection frame carrying supplementary information, and calculate multi-scale local features of the detection frame and tracking frame of each target to be tracked.

[0066] After determining the detection frames of the targets to be tracked in the current frame image in the previous step, any method can be used to calculate the tracking frame of each detection frame, that is, the tracking frame is generated based on the detection frame. The calculation of the multi-scale local features of the tracking frame and the detection frame is divided into four steps: scale space establishment, detection of poles, feature point direction assignment, and calculation of feature descriptors. Figure 2 and Figure 3 The calculation process of each step is introduced separately:

[0067] A1: Establish a scale space. The purpose of constructing a scale space is to find a position that is invariant in scale changes. Continuous scale changes can be used, that is, to find stable feature points in all possible scale changes in the scale space. The poles found in this way can ensure invariance in image scaling and rotation changes. The scale space L(x,y,σ) of an image can be defined as L(x,y,σ)=G(x,y,σ)*I(x,y), where I(x,y) is the original image, x is the xth row of the image matrix, y is the yth column of the image matrix, G(x,y,σ) is a Gaussian function with variable scale space, * represents the convolution operation, σ represents the size of the scale space, the larger σ is, the more blurred it is, which can be used to represent the general appearance of the image; the smaller σ is, the clearer it is, which can be used to represent the details of the image.

[0068] A2: Detect extreme points. This step uses Gaussian difference to find stable extreme points in the scale space. The Gaussian difference function D(x,y,σ) can be expressed as D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y,σ), where kσ and σ are the smooth scales of two consecutive images. The obtained difference image is in the Gaussian difference pyramid, as shown in Figure 2 As shown. The Gaussian difference pyramid is obtained by subtracting two adjacent layers of the Gaussian pyramid in sequence to obtain one layer of the Gaussian difference pyramid. The smoothing scale of each layer in each group of the pyramid is different. The first layer of the next group is obtained by downsampling the feature image of the previous layer at alternate points. The purpose of this is to make the Gaussian difference pyramid meet scale continuity. Compare the current point with the values ​​of the 24 points around it. If it is the maximum or minimum value, then the point is an extreme point, otherwise it is not. This comparison method has a relatively small amount of calculation.

[0069] A3: Assignment of feature point direction. For each key point extracted above, select a window around the point. The gradient directions of the sampling points in the window form a direction histogram. The direction of the key point is determined according to the peak value of the histogram. The scale of the key point is used to select the Gaussian filter image and the size of the window involved in the calculation. In order to ensure that the directions of the same key point at different scales contain the same amount of information, the size of the window must be different. For the same original image, the larger the scale, the larger the window should be; conversely, if the window size remains unchanged, the larger the scale of the image, the less information in the window.

[0070] A4: Calculate feature descriptors. Multi-scale local feature descriptors are a representation of the statistical results of Gaussian image gradients in the key point area. By dividing the image area around the key point into blocks and calculating the gradient histogram within the block, a unique vector is generated. This vector is an abstraction of the image information in the area and is unique. The feature descriptor uses the gradient information in 8 directions calculated in a 4×4 window in the key point scale space, with a total of 4×4×8=128-dimensional vector representation. The overall process of constructing multi-scale local features is as follows: Figure 3 shown.

[0071] S103: using an image matching algorithm to perform multiple cycles of repeated matching of each detection frame and each tracking frame to obtain a successfully matched detection frame and tracking frame.

[0072] The image matching algorithm of this step can adopt any grayscale matching method or feature matching method, which does not affect the implementation of this application. Since the technical problem to be solved by this application is the problem of low tracking accuracy caused by occlusion, it is understandable that the occluded target will not be occluded all the time, that is, the target to be tracked will often be occluded for a certain period of time, and these occluded targets will reappear in the field of vision at other times. Therefore, a maximum number of iterations or a maximum matching duration can be set according to the actual application scenario, that is, the surrounding environment of the video to be processed, and the matching process of S103 can be set as a loop process. For example, if the maximum number of iterations is set to 50, the image matching algorithm is used to repeatedly match each detection frame and each tracking frame 50 times. If the maximum matching duration is set to 10 minutes, each detection frame and each tracking frame are repeatedly matched within 10 minutes. For example, for the convenience of subsequent use, the detection frame and tracking frame that can be successfully matched in this step and the detection frame and tracking frame that are not successfully matched can be saved in a list respectively.

[0073] In order to further improve the matching accuracy, a restriction condition can also be set in the repeated matching process, that is, each detection frame and each tracking frame can be repeatedly matched in a loop in the order of the time when the trajectory of the target to be tracked is lost from small to large, until the current number of iterations reaches the maximum number of iterations. For example, the matching process of S103 can be set to a loop with a maximum number of iterations of 50 times, and the trajectory tracking frame and the detection frame from the 0th to the 50th are matched. The trajectory that has not been lost is matched first, and the trajectory that has been lost for a long time is matched later. Through this part of the processing, the occluded target can be retrieved again, and the number of ID changes that occur when the target is occluded and then appears is reduced. After the cyclic matching process is completed, the successfully matched detection frame and tracking frame can be obtained, and the motion trajectory of each target to be tracked can be obtained based on the successfully matched detection frame and tracking frame, and the multi-target tracking is completed based on the determined motion trajectory.

[0074] In the technical solution provided by the embodiment of the present invention, the local texture features of the image of the previous frame are added as additional supplementary information in the generation stage of the detection frame of the current frame, which is a supplement to the previous target detection task. Since the tracking frame is generated by the detection frame, this can solve the problem that the target detection model does not detect the target in the current frame to a certain extent, and can also solve the problem that the tracking frame is lost in some frames, which can avoid the phenomenon that the detection frame is missed and the target to be tracked is lost to the greatest extent, and effectively improve the tracking accuracy of the target. Further, the matching between the detection frame and the tracking frame is performed based on multi-scale local features. The multi-scale local features pay more attention to the local features of the image, are insensitive to rotation, scale scaling, and brightness changes, and maintain a certain degree of stability to perspective changes, affine transformations, and noise. Multiple matching screening within a certain time range can retrieve the occluded target, reduce the number of ID changes that occur when the target is occluded and then appears, so as to solve the problem of the target being occluded for a long time, effectively solve the current situation that the target tracking result is inaccurate or the target tracking is lost due to occlusion, improve the accuracy of target tracking, and meet the actual business needs of high-precision target tracking.

[0075] In the above embodiment, there is no limitation on how to perform step S103. In this embodiment, a matching implementation method of each detection frame and each tracking frame is provided, which may include the following steps:

[0076] In this embodiment, the Hungarian algorithm can be used as an image matching algorithm. The Hungarian matching is based on a theorem: if each element in a row or a column of the weight coefficient matrix C=c_ij is added or subtracted with the same number to obtain a new matrix B=b_ij, then the assignment problem with C or B as the coefficient matrix has the same optimal assignment. The weight coefficient matrix can be determined based on the multi-scale local features of each detection frame and each tracking frame. As an optional implementation, after calculating the multi-scale local features of each detection frame and each tracking frame of each target to be tracked, the multi-scale local features of each detection frame and each tracking frame of each target to be tracked can also be normalized to obtain a normalized multi-scale local feature vector; for each detection frame, the inner product of the normalized multi-scale local feature vector of the current detection frame and the normalized multi-scale local feature vector of each tracking frame is calculated respectively, so that the inner product value is distributed in the range of [0, 1], and the weight coefficient matrix of the Hungarian algorithm is constructed based on the inner product result. For example, the target to be tracked has three detection frames ABC and three tracking frames abc. The detection frame A performs inner products with the tracking frames a, b, and c respectively, the detection frame B performs inner products with the tracking frames a, b, and c respectively, and the detection frame C performs inner products with the tracking frames a, b, and c respectively. Finally, a weight coefficient matrix is ​​constructed based on 9 inner product values ​​distributed in the range [0, 1]. The Hungarian matching algorithm is divided into the following four steps:

[0077] B1: Subtract the smallest number in each row of the weight coefficient matrix, and then subtract the smallest number in each column of the weight coefficient matrix.

[0078] B2: The matrix obtained in step B1 must contain 0 elements, and then use the least horizontal and vertical lines to cover all the 0 elements.

[0079] B3: Determine whether the total number of lines in step B2 is less than the dimension of the matrix. If so, execute B4.

[0080] B4: Find the minimum value among the numbers not covered, subtract the minimum value from each row not covered by the line, add the minimum value to each column covered, and then jump to B2. This process continues until the minimum number of line segments covering all 0s is equal to the dimension of the matrix. Take the value of the 0 element in each row and find the corresponding number in the original matrix. This is the optimal allocation. The optimal allocation is the first group found.

[0081] The detection frame and the tracking frame correspond one to one. The Hungarian algorithm matching is to find n numbers in the weight matrix formed by the inner product, such that the sum of these n numbers is the smallest. These n numbers cannot be in the same row or column. Using these four steps, a set of solutions can be found. Then, target tracking is to find the n numbers with the largest inner product, that is, the strongest correlation, so the inner product value is added with a negative sign to enter the Hungarian matching to get the solution. In other words, since the maximum distribution is found in target tracking, all the calculated inner product values ​​can be changed to negative numbers and sent to the Hungarian matching algorithm.

[0082] It should be noted that if the image matching algorithm adopts the Hungarian algorithm, the tracking frame ID change problem will be caused by the multiple solution problem of the Hungarian algorithm. In view of this, in order to solve this problem, after using the image matching algorithm to repeatedly match each detection frame and each tracking frame for multiple times, before obtaining the successfully matched detection frame and tracking frame, the following steps may also be included:

[0083] For the unmatched detection frames and tracking frames to be matched, the optimized intersection-and-union relationship is called to calculate the optimized intersection-and-union ratio between each detection frame to be matched and each tracking frame to be matched; based on the optimized intersection-and-union ratio between each detection frame to be matched and each tracking frame to be matched, the Hungarian algorithm is used to perform repeated matching again. The optimized intersection-and-union relationship is determined based on the intersection-and-union ratio of the current detection frame to be matched and the current tracking frame to be matched, the distance between the center point of the current tracking frame to be matched and the current detection frame to be matched, and the frame size information of the current detection frame to be matched and the current tracking frame to be matched. The frame size information includes but is not limited to the width and height of the detection frame and the tracking frame, the width and height of the maximum circumscribed rectangle, etc.

[0084] After the image matching process is performed in S103 of the above embodiment, a successfully matched detection frame and tracking frame are obtained. In this embodiment, the detection frame and tracking frame that are not successfully matched can be optimized. After the optimization process, an image matching algorithm such as the Hungarian algorithm is used again for matching process. As an optional implementation, in order to enable the tracking frame and detection frame that are close to each other to maintain the IOU feature, while the tracking frame and detection frame that are far away have their IOU features reduced, the above-mentioned optimized intersection-over-union ratio relationship can be expressed as:

[0085]

[0086] In the formula, is the optimization value of the intersection-over-union ratio of the current detection frame to be matched and the current tracking frame to be matched, IOU is the intersection-over-union ratio of the current detection frame to be matched and the current tracking frame to be matched, a is the current tracking frame to be matched, b is the current detection frame to be matched, c is the diagonal distance between the current tracking frame to be matched and the maximum circumscribed rectangle of the current detection frame to be matched, and w a is the width of the current tracking box to be matched, wb is the width of the current detection box to be matched, w c h is the width of the maximum bounding rectangle of the current tracking frame to be matched and the current detection frame to be matched, a is the height of the current tracking frame to be matched, h b is the height of the current detection frame to be matched, h c is the height of the maximum circumscribed rectangle between the current tracking frame to be matched and the current detection frame to be matched, ρ(a,b) is the distance between the center point of the current tracking frame to be matched and the center point of the current detection frame to be matched, and ρ(w a ,w b ) is the width difference between the current tracking frame to be matched and the current detection frame to be matched, ρ(h a ,h b ) is the height difference between the current tracking frame to be matched and the current detection frame to be matched.

[0087] As can be seen from the above, the embodiment of the present invention introduces the center point distance feature and the width and height difference feature when calculating the IOU between the detection frame and the tracking frame. This method can be referred to as EIOU matching, which can make the similar detection frame and tracking frame better matched, and the detection frame and tracking frame that are far away can be filtered out, thereby increasing the correspondence of the data in the weight matrix and solving the tracking frame ID transformation problem caused by the multiple solution problem of Hungarian matching.

[0088] The above embodiment does not limit the method for generating the supplementary information of each detection frame. This embodiment also provides a method for generating the supplementary information of the detection frame, which may include:

[0089] As an optional implementation of this embodiment, the motion state information of each target to be tracked can be represented by the center point position of the detection frame, the frame size information, and the speed information corresponding to the center point position and the frame size information in the image coordinate system. The frame size information includes but is not limited to the width and height of the frame. The motion state information of each target to be tracked in this embodiment can be expressed by the target motion state parameter, which can be used to describe the target motion state. The target motion state parameter can be expressed as m=[x, y, α, h, v x , v y , v α , v h ], x, y represent the position coordinates of the center point of the detection frame m, α is the aspect ratio of the detection frame m, h is the height of the detection frame m, and v x , v y , v α , v hThe four variables x, y, α, and h represent the corresponding speed information in the image coordinate system. Since the time interval between two frames in the video to be processed is very short, the offset of the target motion process is relatively small, so the target motion can be approximated as uniform motion, and the interval between two frames is replaced by the normalized time, that is, x t =x t-1 +v x ,y t =y t-1 +v y , α t =α t-1 +v α ,h t =h t-1 +v h . Add motion noise w t As the motion relationship between the two frames, a state transfer matrix is ​​constructed to describe the target motion relationship between two adjacent frames of the video to be processed. The operation process is converted into a matrix form:

[0090]

[0091] Using the motion state parameters of the previous moment, the state transfer matrix and the motion process noise are used to predict the running state parameters of the current moment, so as to determine the position coordinates of each detection frame in the current frame image based on the motion state parameters of the current moment. That is, the state transfer matrix is ​​set to F, and the state at the current moment t is predicted from the state at the previous moment t-1, and we can get in Refers to the target motion state vector m at time t. Then, by solving the covariance of the motion state parameters and the motion process noise, the prediction formula P of the error covariance matrix is ​​obtained: t - =FP t-1 F T +Q. Q=cov(w t ,w t ), cov is the calculated covariance. In this way, the new error can be predicted by the error covariance of the previous moment and the process noise.

[0092] The position information of each detection frame in the current frame image is calculated according to the motion state parameters of each detection frame shown in the above embodiment, so as to determine the image area where each detection frame is located in the current frame image.

[0093] For each detection frame, calculate the gradient information of each pixel in the target image area corresponding to the current detection frame; calculate the gradient histogram information based on the gradient information of each pixel; and perform block normalization processing based on the target pixel information used in the gradient histogram information calculation process to obtain block feature information; construct the local texture features of the current detection frame based on all block feature information in the target image area;

[0094] Constructing multiple target frames with the same width and height within a range where the distance from the center point of the detection frame of the previous frame is a first preset value, and calculating local texture features within the range of each target frame respectively;

[0095] The local texture features of each target frame are subjected to an inner product operation with the local texture features of the detection frame corresponding to the current frame image, and the target frame with the largest inner product value is used as supplementary information.

[0096] In this embodiment, the whole process is described by taking the local texture feature HOG (Histogram of Oriented Gradient) as an example:

[0097] After obtaining the motion state information of the detection frame of the target to be tracked in the previous frame, that is, the motion state vector m, the position of the detection frame on the image can be found. The HOG features in the found area are calculated, including four steps: gradient calculation, gradient histogram calculation, block normalization and HOG feature statistics:

[0098] C1: Gradient calculation. The horizontal gradient g of a single pixel x is the difference between the pixel values ​​of its left and right adjacent pixels, and the vertical gradient g y It is the difference between the pixel values ​​of the upper and lower adjacent pixels, and then the total gradient intensity of the pixel can be obtained And the gradient direction The gradient direction needs to be taken as the absolute value.

[0099] C2: Gradient histogram calculation. In step C1, the gradient strength and gradient direction of each pixel point have been obtained. 9 pixel points can be taken, and then 0~180° is evenly divided into 9 parts with an interval of 20°. The gradient strength of the corresponding position is placed according to the distance between the gradient direction and each angle point. The smaller the distance, the corresponding gradient strength is placed at the corresponding angle position. If the distance is the same, the gradient strength is averaged into two parts and placed at the corresponding angle position, such as Figure 4 As shown in the figure, through the form of gradient histogram, the 18 feature information of the original 9 pixels are put into an array of size 9, the feature information is reduced by half, and the complexity of subsequent calculations is greatly reduced.

[0100] C3: Block normalization. The 9 pixels above are taken as a cell, and the 4 cells are taken as a block. In this way, a block contains 36 feature information. Then, these 36 feature information need to be normalized. In this embodiment, the L2 norm normalization is used, that is, in is the normalized feature information of the ith element, x i is the i-th feature information. The advantage of doing this is that each feature information is normalized to the range of [0, 1], which is convenient for subsequent comparison.

[0101] C4: HOG feature statistics. In step C3, the feature information of a block can be calculated. Then, we only need to collect all the block feature information within the target range and put them together to construct the HOG feature within the target area. The block collection needs to move a cell distance each time, such as Figure 5 shown.

[0102] Construct 8 boxes with the same width and height within the range of 1 of the center point of the detection box of the previous frame, which are 8 directions of up, down, left, right, and diagonal. Then calculate the HOG features within the range of these 8 boxes respectively, and perform inner product operation with the HOG of the detection box to obtain the box with the largest value as the supplement of the detection box of the next frame, such as Figure 6 shown.

[0103] Since the tracking frame is generated by the detection frame, using HOG features to supplement the detection frame can solve the tracking loss problem caused by missed detection by the detection model. For experimental results, please refer to Figure 7 and Figure 8 As shown in the figure, after comparing the two figures, it can be seen that after supplementing the detection frame, the target tracking accuracy can indeed be effectively improved.

[0104] The above embodiment does not limit the method for generating the tracking frame. This embodiment also provides a method for generating the tracking frame, which may include:

[0105] The position information of each detection frame in the current frame image is calculated according to the motion state parameters of each detection frame; the Kalman filter gain is calculated through the preset observation matrix; and the position information of the corresponding tracking frame is obtained according to the position information of each detection frame and the Kalman filter gain.

[0106] This embodiment uses the Kalman filter method to generate the tracking frame of each detection frame. First, it is necessary to use the observation matrix based on K t =P t - H T ·((HP t - H T)+R) -1 The Kalman filter gain is calculated, and the first 4-dimensional features can be extracted by multiplying the observation matrix and the motion state vector. The observation matrix is ​​as follows:

[0107]

[0108] In the formula, K t is the Kalman filter gain at time t, P t - is the prior estimated covariance at time t, H T is the transposed matrix of the observation matrix, and R is the measurement noise covariance.

[0109] After the Kalman filter gain is calculated, the specific coordinates of each detection frame are calculated based on the above embodiment, and the relationship is used A more accurate tracking frame can be obtained. is the posterior state estimate at time t, is the prior state estimate at time t, z is [x, y, α, h] in the target motion state parameter m, and H is the observation matrix. In this way, the specific position coordinates of each tracking frame can be obtained.

[0110] Correspondingly, after obtaining the successfully matched detection frame and tracking frame, the method may further include: using the successfully matched detection frame and tracking frame to update the parameters of the Kalman filter, such as Fig. 9 If EIOU matching is performed after S103, after the EIOU matching, the parameters of the Kalman filter are updated using the successfully matched detection frame and tracking frame, and then the updated Kalman filter method is used to perform Kalman filtering again on the various motion parameters in the new detection frame, that is, the target motion state parameter m in the above embodiment, to complete the cyclic tracking of the entire method.

[0111] In order to verify the effectiveness of this embodiment, this application also uses infrared video as an example to perform a multi-target tracking test according to the multi-target tracking method. The multi-target tracking method is: first, use the HOG feature information of the detection target and its adjacent position in the previous frame to generate position coordinates as a supplement to the detection target in the next frame, and then use the Kalman filter to predict the motion trajectory of the object, and then use the Hungarian algorithm to match the predicted tracking frame and the detection frame in the current frame through the designed multi-scale local features, and retain the features of the obtained results, and then use the Hungarian algorithm again to match the tracking frame and the detection frame through the improved IOU (Intersection over Union) information, that is, the EIOU matching method described in the above embodiment, and finally get the matching result to update the Kalman filter to complete the target tracking. Test results please refer to Fig.10 and Fig.11As shown in the two figures, it can be seen that this application solves the problems of tracking loss, target occlusion and frequent ID exchange in the current multi-target tracking algorithm by constructing a tracking algorithm that supplements and retains features and using an improved IOU strategy. This method has high tracking accuracy and operation speed and can achieve real-time tracking.

[0112] It should be noted that there is no strict order of execution between the steps in this application. As long as they comply with the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 1-Figure 9 This is just a schematic and does not mean that this is the only execution order.

[0113] The embodiment of the present invention also provides a corresponding device for the target tracking method, which further makes the method more practical. The device can be described from the perspective of functional modules and hardware. The target tracking device provided by the embodiment of the present invention is introduced below. The target tracking device described below and the target tracking method described above can be referred to each other.

[0114] From the perspective of functional modules, see Fig.12 , Fig.12 A structural diagram of a target tracking device provided by an embodiment of the present invention in a specific implementation manner, the device may include:

[0115] The detection information acquisition module 121 is used to obtain the detection frame of each target to be tracked in the video to be processed, and use the local texture features of the previous frame image as the supplementary information of each detection frame of the current frame image;

[0116] A tracking frame calculation module 122, configured to generate a corresponding tracking frame based on each detection frame carrying supplementary information;

[0117] A multi-scale feature calculation module 123, used to calculate multi-scale local features of the detection frame and tracking frame of each target to be tracked;

[0118] The matching module 124 uses an image matching algorithm to perform multiple cycles of repeated matching on each detection frame and each tracking frame to obtain a successfully matched detection frame and tracking frame.

[0119] Optionally, in some implementations of the present embodiment, the above-mentioned device may further include an EIOU matching module, in which the image matching algorithm is the Hungarian algorithm, and for the unsuccessfully matched detection frame to be matched and the tracking frame to be matched, the optimized intersection-and-union ratio relationship is called to respectively calculate the intersection-and-union ratio tuning value between each detection frame to be matched and each tracking frame to be matched; based on the intersection-and-union ratio tuning value between each detection frame to be matched and each tracking frame to be matched, the Hungarian algorithm is used to perform repeated matching again for multiple times; wherein the optimized intersection-and-union ratio relationship is jointly determined according to the intersection-and-union ratio of the current detection frame to be matched and the current tracking frame to be matched, the distance between the center points of the current tracking frame to be matched and the current detection frame to be matched, and the frame size information of the current detection frame to be matched and the current tracking frame to be matched.

[0120] As an optional implementation of this embodiment, the above-mentioned EIOU matching module can be further used to perform multiple cycles of repeated matching on each detection frame and each tracking frame in the order of the trajectory loss time of the target to be tracked from small to large until the current number of iterations reaches the maximum number of iterations.

[0121] As another optional implementation of this embodiment, the above-mentioned device may also include a normalization processing module, for example, for normalizing the multi-scale local features of each detection frame and each tracking frame of each target to be tracked, so as to obtain a normalized multi-scale local feature vector; for each detection frame, the inner product of the normalized multi-scale local feature vector of the current detection frame and the normalized multi-scale local feature vector of each tracking frame is calculated, so as to construct a weight coefficient matrix of the Hungarian algorithm based on the inner product result.

[0122] Optionally, in some other implementations of the present embodiment, the above-mentioned tracking frame calculation module 122 can be further used to: calculate the position information of each detection frame in the current frame image according to the motion state parameters of each detection frame; calculate the Kalman filter gain through a preset observation matrix; obtain the position information of the corresponding tracking frame according to the position information of each detection frame and the Kalman filter gain calculation; the corresponding above-mentioned device may also include a parameter updating module for updating the parameters of the Kalman filter using the successfully matched detection frame and tracking frame.

[0123] Optionally, in some other implementations of the present embodiment, the detection information acquisition module 121 may include a detection information generation unit, which is used to: calculate the position information of each detection frame in the current frame image according to the motion state parameters of each detection frame, so as to determine the image area where each detection frame is located in the current frame image; for each detection frame, calculate the gradient information of each pixel in the target image area corresponding to the current detection frame; obtain gradient histogram information based on the gradient information of each pixel; and perform block normalization processing based on the target pixel information used in the calculation process of the gradient histogram information to obtain block feature information; construct local texture features of the current detection frame based on all block feature information in the target image area; construct multiple target frames with the same width and height within the range where the distance from the center point of the detection frame of the previous frame is a first preset value, and calculate the local texture features within the range of each target frame respectively; perform inner product operation on the local texture features of each target frame and the local texture features of the detection frame corresponding to it in the current frame image, and use the target frame with the largest inner product value as supplementary information.

[0124] As an optional implementation of this embodiment, the above-mentioned device may also include a state calculation module, which is used to use the center point position, frame size information, and speed information corresponding to the center point position and frame size information of the detection frame in the image coordinate system to represent the motion state information of each target to be tracked; based on the motion state information of each target to be tracked and the uniform motion of each target to be tracked, a state transfer matrix is ​​constructed to describe the target motion relationship between two adjacent frames of the video to be processed; using the motion state parameters of the previous moment, the operating state parameters of the current moment are predicted according to the state transfer matrix and the motion process noise, so as to determine the position coordinates of each detection frame in the current frame image based on the motion state parameters of the current moment.

[0125] The functions of the functional modules of the target tracking device in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0126] It can be seen from the above that the embodiments of the present invention effectively solve the current situation where target tracking results are inaccurate or target tracking is lost due to occlusion, improve the accuracy of target tracking, and can meet the actual business needs of high-precision target tracking.

[0127] The target tracking device mentioned above is described from the perspective of functional modules. Furthermore, the present application also provides an electronic device, which is described from the perspective of hardware. Fig.13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application in one implementation manner. Fig.13As shown, the electronic device includes a memory 130 for storing a computer program; and a processor 131 for implementing the steps of the target tracking method mentioned in any of the above embodiments when executing the computer program.

[0128] Among them, the processor 131 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 131 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 131 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 131 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 131 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0129] The memory 130 may include one or more computer-readable storage media, which may be non-transitory. The memory 130 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 130 is at least used to store the following computer program 1301, wherein the computer program can implement the relevant steps of the target tracking method disclosed in any of the aforementioned embodiments after being loaded and executed by the processor 131. In addition, the resources stored in the memory 130 may also include an operating system 1302 and data 1303, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 1302 may include Windows, Unix, Linux, etc. Data 1303 may include, but is not limited to, data corresponding to the target tracking results, etc.

[0130] In some embodiments, the electronic device may further include a display screen 132, an input / output interface 133, a communication interface 134 or a network interface, a power supply 135 and a communication bus 136. Among them, the display screen 132 and the input / output interface 133, such as a keyboard, belong to the user interface, and the optional user interface may also include a standard wired interface, a wireless interface, etc. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 134 may optionally include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, etc., which is usually used to establish a communication connection between the electronic device and other electronic devices. The communication bus 136 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.13 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0131] Those skilled in the art will understand that Fig.13 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, for example, it may also include a sensor 137 to implement various functions.

[0132] The functions of the functional modules of the electronic device described in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0133] It can be seen from the above that the embodiments of the present invention effectively solve the current situation where target tracking results are inaccurate or target tracking is lost due to occlusion, improve the accuracy of target tracking, and can meet the actual business needs of high-precision target tracking.

[0134] It is understandable that if the target tracking method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, disk or optical disk and other media that can store program code.

[0135] Based on this, an embodiment of the present invention further provides a readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the target tracking method described in any of the above embodiments are performed.

[0136] The functions of the functional modules of the readable storage medium described in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0137] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the hardware disclosed in the embodiments, including devices and electronic devices, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0138] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0139] The above is a detailed introduction to a target tracking method, device, electronic device and readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A target tracking method, characterized in that: include: Obtain the detection frame of each target to be tracked in the video to be processed, and use the local texture features of the previous frame image as supplementary information for each detection frame of the current frame image; Generate corresponding tracking frames based on each detection frame carrying supplementary information, and calculate multi-scale local features of the detection frame and tracking frame of each target to be tracked; Normalizing the multi-scale local features of each detection frame and each tracking frame of each target to be tracked to obtain a normalized multi-scale local feature vector; For each detection frame, the inner product of the normalized multi-scale local feature vector of the current detection frame and the normalized multi-scale local feature vector of each tracking frame is calculated respectively, so as to construct the weight coefficient matrix of the Hungarian algorithm based on the inner product result; An image matching algorithm is used to perform multiple cycles of repeated matching of each detection frame and each tracking frame to obtain a successfully matched detection frame and tracking frame; the image matching algorithm is the Hungarian algorithm.

2. The target tracking method according to claim 1, characterized in that: The image matching algorithm is the Hungarian algorithm. After the image matching algorithm is used to repeatedly match each detection frame and each tracking frame, and before the successfully matched detection frame and tracking frame are obtained, the method further includes: For the unmatched detection frames and tracking frames to be matched, the optimized intersection-and-union ratio relationship is called to calculate the optimized intersection-and-union ratio between each detection frame to be matched and each tracking frame to be matched; Based on the optimized value of the intersection-over-union ratio between each detection frame to be matched and each tracking frame to be matched, the Hungarian algorithm is used to perform repeated matching again for multiple cycles; The optimized IoU relationship is determined based on the IoU of the current detection frame to be matched and the current tracking frame to be matched, the distance between the center points of the current tracking frame to be matched and the current detection frame to be matched, and the frame size information of the current detection frame to be matched and the current tracking frame to be matched.

3. The target tracking method according to claim 2, characterized in that: The method of using the image matching algorithm to repeatedly match each detection frame with each tracking frame includes: According to the order of the trajectory loss time of the target to be tracked from small to large, each detection frame and each tracking frame are matched repeatedly for multiple cycles until the current number of iterations reaches the maximum number of iterations.

4. The target tracking method according to claim 1, characterized in that: The generating a corresponding tracking frame based on each detection frame carrying the supplementary information includes: Calculate the position information of each detection frame in the current frame image according to the motion state parameters of each detection frame; Calculate the Kalman filter gain by presetting the observation matrix; The position information of the corresponding tracking frame is obtained according to the position information of each detection frame and the Kalman filter gain calculation; Correspondingly, after obtaining the successfully matched detection frame and tracking frame, the method further includes: The parameters of the Kalman filter are updated using the successfully matched detection frame and tracking frame.

5. The target tracking method according to any one of claims 1 to 4, characterized in that: The method of using the local texture features of the previous frame image as supplementary information of each detection frame of the current frame image includes: Calculating the position information of each detection frame in the current frame image according to the motion state parameters of each detection frame, so as to determine the image area where each detection frame is located in the current frame image; For each detection frame, calculate the gradient information of each pixel in the target image area corresponding to the current detection frame; calculate the gradient histogram information based on the gradient information of each pixel; perform block normalization processing based on the target pixel information used in the gradient histogram information calculation process to obtain block feature information; construct the local texture feature of the current detection frame based on all block feature information in the target image area; Constructing multiple target frames with the same width and height within a range where the distance from the center point of the detection frame of the previous frame is a first preset value, and calculating local texture features within the range of each target frame respectively; The local texture features of each target frame are subjected to an inner product operation with the local texture features of the detection frame corresponding to the current frame image, and the target frame with the largest inner product value is used as supplementary information.

6. The target tracking method according to claim 5, characterized in that: The calculating the position information of each detection frame in the current frame image according to the motion state parameters of each detection frame includes: Using the center point position of the detection frame, the frame size information, and the speed information corresponding to the center point position and the frame size information in the image coordinate system to represent the motion state information of each target to be tracked; Based on the motion state information of each target to be tracked and the fact that each target to be tracked moves at a uniform speed, a state transfer matrix for describing the target motion relationship between two adjacent frames of the video to be processed is constructed; The motion state parameters at the previous moment are used to predict the operating state parameters at the current moment according to the state transfer matrix and the motion process noise, so as to determine the position coordinates of each detection frame in the current frame image based on the motion state parameters at the current moment.

7. A target tracking device, characterized in that: include: The detection information acquisition module is used to obtain the detection frame of each target to be tracked in the video to be processed, and use the local texture features of the previous frame image as the supplementary information of each detection frame of the current frame image; A tracking frame calculation module, used to generate a corresponding tracking frame based on each detection frame carrying supplementary information; A multi-scale feature calculation module is used to calculate the multi-scale local features of the detection frame and tracking frame of each target to be tracked; The normalization processing module is used to normalize the multi-scale local features of each detection frame and each tracking frame of each target to be tracked, so as to obtain a normalized multi-scale local feature vector; for each detection frame, the inner product of the normalized multi-scale local feature vector of the current detection frame and the normalized multi-scale local feature vector of each tracking frame is calculated, so as to construct a weight coefficient matrix of the Hungarian algorithm based on the inner product result; the matching module is used to use the image matching algorithm to perform multiple cycles of repeated matching on each detection frame and each tracking frame, so as to obtain a successfully matched detection frame and tracking frame; the image matching algorithm is the Hungarian algorithm.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to implement the steps of the target tracking method according to any one of claims 1 to 6 when executing a computer program stored in the memory.

9. A readable storage medium, characterized in that: The readable storage medium stores a target tracking computer program, and when the target tracking computer program is executed by a processor, the steps of the target tracking method according to any one of claims 1 to 6 are implemented.

Images