A multi-scale adaptive correlation filtering target tracking method, device and medium

Abstract

The present invention discloses a multi-scale adaptive correlation filtering target tracking method, system, device and medium, wherein the method comprises: obtaining an image sequence, determining a target to be tracked in a first frame image; obtaining an appearance model of the target according to the position of the target; obtaining a new image, obtaining a new position of the target, and completing a rough positioning of the position; based on the new position, obtaining the target scale in the new image, and completing a rough positioning of the scale; based on the position and scale determined by the rough positioning, sampling the target frame according to a frame regression filter, extracting a sample feature set, calculating a response map of the target, and obtaining a fine regression target position; using the fine regression target position as the final target position of the new image, and updating the filter according to the final target position. The present invention fits a frame that is closest to the real target through a frame regression filter, obtains a more accurate target position on a continuous domain, and can be widely used in the field of target tracking technology.

Description

Technical Field

[0001] The present invention relates to the field of target tracking technology, and in particular to a multi-scale adaptive correlation filtering target tracking method, system, device and medium. Background Art

[0002] In recent years, with the rapid development of computer vision technology, many tedious tasks can be completed efficiently by computers. Visual target tracking is one of the basic problems in the field of computer vision research. Its task is to predict the position and size of the moving target in each frame of the subsequent image sequence based on the known position of the moving target. It plays an important role in human-computer interaction, new media, intelligent monitoring, smart transportation, medical care, national defense and military and other fields. Since target tracking is often used in video streams, a faster algorithm processing speed is required to ensure the real-time performance of the algorithm. Secondly, the scenes used by the target tracking algorithm are often more complex, and it is necessary to face challenges such as illumination changes, scale changes, appearance changes, and occlusion to ensure the accuracy of tracking. For the scale change problem, DSST is fast, and its implementation idea is used by a large number of target tracking algorithms. DSST regards the position determination problem and the scale determination problem as two independent problems, and solves the position filter and the scale filter respectively. After determining the position first, the scale search is performed with the predicted position as the center. There will be two obvious defects here:

[0003] (1) The position and size of tracking are inseparable issues that affect each other. Treating the position and scale as two independent issues will lead to the accumulation of errors. Since the original size of the target is used to solve the position filter, when the size and aspect ratio of the target changes, the position predicted by the position filter may be slightly different from the center of the real target. However, the size filter performs multi-scale sampling based on the position predicted by the position filter as the center point, and cannot correct the deviation of the position filter. The deviation accumulates continuously, which easily causes tracking drift.

[0004] (2) The size filter of DSST uses a variable scale fixed aspect ratio method to sample the size. Therefore, when the aspect ratio of the target changes due to posture change or deformation, the overlap rate between the predicted target box and the actual target box decreases, which may introduce a large amount of background information to pollute the appearance model, resulting in inaccurate tracking or even drift. Summary of the invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the present invention aims to provide a multi-scale adaptive correlation filtering target tracking method, system, device and medium.

[0006] The technical solution adopted by the present invention is:

[0007] A multi-scale adaptive correlation filtering target tracking method comprises the following steps:

[0008] Acquire an image sequence, acquire a first frame of image, and determine a target to be tracked in the first frame of image;

[0009] Obtain the appearance model of the target according to its position, and initialize the position filter, scale filter, and border regression filter;

[0010] Get a new image, get the new position of the target according to the position filter, and complete the rough positioning;

[0011] Based on the new position, the target scale in the new image is obtained according to the scale filter to complete the rough positioning of the scale;

[0012] Based on the position and scale determined by coarse positioning, sample the target border according to the border regression filter, extract the sample feature set, calculate the target response map, and obtain the target position of fine regression;

[0013] The target position of the refined regression is used as the final target position of the new image, and the position filter, scale filter and border regression filter are updated according to the final target position.

[0014] Furthermore, the frame regression filter ω anchor The samples were extracted by:

[0015] Consider the sampling factor ratio of different aspect ratios = P w / P h , where P h is the width of the border P, P w is the height of the border P;

[0016] The aspect ratio is extended by sampling the center of the border P, where the extended range covers the maximum sampling factor ratio max Ratio to the minimum sampling factor min All aspect ratios between 2 and 5, and border widths between 2 and 5 min *P w , ratio max *P w ], the border height is in [ratio min *P h , ratio max *P h ] interval;

[0017] Fits a translation factor to the bounding box.

[0018] Furthermore, in the process of fitting the translation factor of the border, the step size step adopted is determined by the following method:

[0019]

[0020] Among them, γ is the proportional coefficient, step w is the moving step length in the border height direction, step h is the moving step length in the border width direction.

[0021] Furthermore, the design and solution method of the frame regression filter is as follows:

[0022] Considering bounding box regression as a ridge regression problem, the optimization problem to be solved is:

[0023]

[0024] Among them, ω is the filter to be learned, x i is the extracted training sample, λ is the regularization coefficient, y i is the label of the training sample;

[0025] This problem has a closed solution. By matrixing the problem and taking the derivative of the filter ω, we can get:

[0026] ω=(X H X+λI) -1 X H y

[0027] Where X is the sample x=[x1 x2 x3 … x n ] T The circulant matrix obtained by cyclic expansion is I, which is the identity matrix; X H is the conjugate transpose of X, and y represents the label of the sample. X can be displayed as:

[0028]

[0029] Furthermore, the training sample label and bounding box regression filter structure are designed as follows:

[0030] According to the mapping relationship in the two-dimensional space, the candidate bounding box P = (P x , P y , P w , P h ), and the actual bounding box coordinates G = (G x , G y , G w , G h ) x , d y and scale factor dw , d h satisfy:

[0031] G x =P w d x +P x

[0032] G y =P h d y +P y

[0033]

[0034]

[0035] In the formula, x and y are the coordinate values ​​of the center of the border, h represents the height of the border, and w represents the width of the border;

[0036] According to formula (1), the displacement factor solved by regression is and scale factor Should be consistent with the actual displacement factor d x , d y and scale factor d w , d h equal, thus making the predicted bounding box As close to the real border as possible (G x , G y , G w , G h );

[0037] The label of each sampling bounding box can be obtained by the following formula:

[0038]

[0039]

[0040]

[0041]

[0042] Design the labels y of the training samples i Including displacement factor d x , d y , scale factor d w , d h And the score of the sample;

[0043] y i =(d xi d yi d wi dhi score i ) T

[0044] The score calculation formula of the sample is as follows:

[0045]

[0046] Among them, d i represents the displacement factor d of the i-th sample x , d y and scale factor d w , d h .

[0047] Furthermore, extracting the sample feature set, calculating the target response map, and obtaining the target position of the precise regression includes:

[0048] Extract sample feature set x new , after Fourier transformation, we get Calculate Response

[0049]

[0050] in, For the trained scale filter, the target response map can be obtained by inverse Fourier transform:

[0051]

[0052] The response F is an N×5 matrix, where N is the number of samples extracted. For one of the samples, F=[F dx , F dy , F dw , F dh , F score ]; among them, F dx is the response value of the displacement factor dx, F dy is the response value of the displacement factor dy, F dw is the response value of the scale factor dw, F dh is the response value of the scale factor dh, F score is the score of the response value;

[0053] Remember F score The largest sample is F * , get its translation factor and scale factor F * dx , F * dy , F * dw , F *dh , the bounding box after regression is That is, the predicted target position after regression, and the border is corrected using the penalty learning rate. The corrected new border is:

[0054]

[0055]

[0056] in, is the border size after regression, P is the border size before regression, lr p is the penalty learning rate.

[0057] Furthermore, the calculation process of the penalty learning rate is as follows:

[0058] Since the scale factor changes rapidly, directly fitting the border scale will lead to divergence. In order to prevent divergence, it is necessary to set a penalty learning rate for the fitted border to penalize the aspect ratio change and scale change. The aspect ratio change penalty factor is:

[0059]

[0060] The scale change penalty factor is:

[0061]

[0062] Among them, P h , P w are the height and width of the target before bounding box regression, and are the predicted target height and width after bounding box regression.

[0063] Furthermore, considering that when the aspect ratio or scale becomes larger, the aspect ratio penalty factor and the scale penalty factor will be less than 1, affecting the penalty effect, it is necessary to process the aspect ratio penalty factor and the scale penalty factor to ensure that they are greater than 1;

[0064]

[0065]

[0066] The comprehensive penalty factor is:

[0067]

[0068] The final penalty learning rate used is:

[0069] 1. p =p×scores×K

[0070] Among them, K is the penalty coefficient.

[0071] Furthermore, the filter update method is as follows:

[0072] The update of the filter includes the update of the appearance model and the update of the filter model. The linear update strategy is used to perform weighted updates on the target appearance and the filter:

[0073] x=(1-β)x last +βx new

[0074] ω=(1-β)ω last +βω new

[0075] Among them, β is the learning rate, x last For the old appearance model, x new is the appearance model of the new target position, ω last is the old filter model, ω new is the newly obtained filter model; x represents the appearance model after linear weighted update, ω represents the filter after linear weighted update, which will be used to calculate the next frame response.

[0076] Another technical solution adopted by the present invention is:

[0077] A multi-scale adaptive correlation filtering target tracking device, comprising:

[0078] at least one processor;

[0079] at least one memory for storing at least one program;

[0080] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0081] Another technical solution adopted by the present invention is:

[0082] A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to execute the method described above when executed by the processor.

[0083] The beneficial effect of the present invention is that the present invention fits a frame that is closest to the real target through a frame regression filter, breaks through the limitations of traditional discrete samples and discrete sample interpolation, obtains a more accurate target position in the continuous domain, and improves the accuracy of detection or tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0085] Figure 1 is a flowchart of the steps of a multi-scale adaptive correlation filtering target tracking method in an embodiment of the present invention;

[0086] Figure 2 It is a schematic diagram of a flow chart of a multi-scale adaptive correlation filtering target tracking algorithm based on frame regression in an embodiment of the present invention;

[0087] Figure 3 It is a schematic diagram of the border filter sampling method provided by the present invention. DETAILED DESCRIPTION

[0088] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0089] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0090] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0091] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0092] As shown in the figure, this embodiment provides a multi-scale adaptive correlation filtering target tracking method, including the following steps:

[0093] S101, acquiring an image sequence, acquiring a first frame of image, and determining a target to be tracked in the first frame of image.

[0094] S102, acquiring an appearance model of the target according to the position of the target, and initializing a position filter, a scale filter, and a bounding box regression filter.

[0095] Extract the target and sample appearance model based on the known position and image, and initialize the position filter ω trans , scaling filter ω scale and the bounding box regression filter ω anchor .

[0096] S103, acquiring a new image, acquiring a new position of the target according to a position filter, and completing a rough positioning of the position.

[0097] Extract image samples with the previous frame image position as the center, calculate the confidence map according to the position filter, and the highest position with the highest response is the new position to complete the rough positioning of the position. Extract samples at different scales with the new position as the center, increase the number of scale pools through bilinear interpolation, obtain the response confidence map according to the scale filter calculation, and complete the rough positioning of the scale by using the maximum response position as the scale through inverse Fourier transform.

[0098] S104: Based on the new position, the target scale in the new image is obtained according to the scale filter to complete the rough positioning of the scale.

[0099] S105, based on the position and scale determined by the coarse positioning, sample the target border according to the border regression filter, extract the sample feature set, calculate the target response map, and obtain the target position of the fine regression.

[0100] Based on the position and scale determined by coarse positioning, sample sampling is performed, sample feature sets are extracted, and the target response map is calculated. score The maximum sample response is F * , calculate to get the regression frame, and use the penalty learning rate to correct the regression frame to get the target position of the precise regression.

[0101] S106: Taking the target position of the refined regression as the final target position of the new image, and updating the position filter, the scale filter, and the frame regression filter according to the final target position.

[0102] The refined regression result is used as the target position of this frame, and samples are taken around this position to solve and update the position filter ω trans , scaling filter ω scale and the bounding box regression filter ω anchor .

[0103] The above method is explained in detail below with reference to the accompanying drawings and specific embodiments.

[0104] See also Figure 2 This embodiment provides a multi-scale adaptive correlation filtering target tracking method based on frame regression, and the method at least includes the following steps:

[0105] S1. Image acquisition.

[0106] Acquire an image sequence, obtain the first frame image f1 and the target position P1 (x, y) and the target scale S1 (w, h), and determine the target to be tracked.

[0107] S2: Initialization phase.

[0108] S2.1. Sample sampling.

[0109] Different sampling methods are used for three different filters. The displacement filter searches in the range of (1+padding)×(w, h) based on the original position (x, y). The scale filter extracts samples based on the original target size and the position (x, y) using N different scale factors σ.

[0110] Specifically, the border regression filter sampling method is as follows:

[0111] Since the aspect ratio of the scale may change during tracking, a sampling factor ratio = P with different aspect ratios is required. w / P h , where P h is the width of the border P, P w is the height of the border P.

[0112] The extended sampling of the aspect ratio is performed at the center of the border P. This sampling method covers the aspect ratio min ratio min All aspect ratios and border widths between [ratio min *P w , ratio max *Pw ] interval, the border height is in [ratio min *P h , ratio max *P h ] The scaling of the interval can fit the scale of the sample more accurately through the smooth fitting of the scale factor.

[0113] In addition to scaling, the sample also needs to fit the translation factor. If the sample is the center, step is the step length, and N translations are performed in the horizontal and vertical directions of the image. step If the sample search is performed M=N times, ratio ×N step 2 Sampling times. step When it increases, the number of samples will increase rapidly, which will seriously affect the training time of the filter. Therefore, in order to reduce the number of samples and improve the operation speed, consider using displacement filters and scale filters for rough positioning first, and then sampling based on the surrounding range of the rough positioning position, so as to regress a more accurate target border. With step as the sampling step, sampling is performed up, down, left, and right respectively. In this way, the number of samples is reduced to M=N. ratio ×N step ×2+1; Sampling method see Figure 3 .

[0114] In order to ensure that the confidence level of images of different scales is too small to be trained, the sampling step is related to the width and height scales, so the step is considered to be:

[0115]

[0116] Among them, γ is the proportional coefficient. The smaller γ is, the larger the step size is, and the better the effect is when tracking fast-moving objects. The larger γ is, the more accurate the tracking is, and it is generally appropriate to be in the range of 10 to 20.

[0117] S2.2. Sample feature processing.

[0118] The sampling method is used to extract the HOG features of the three filter sample sets. The sample feature x = [x1x2 x3 … x n ] T Obtain the circulant matrix X through the basis transformation matrix:

[0119]

[0120] According to the Fourier diagonalization property of the circulant matrix X:

[0121]

[0122] in, represents the Fourier transform of x, diag(·) represents the diagonal matrix, and F is the Fourier matrix.

[0123] S2.3. Filter initialization.

[0124] The target tracking problem can be regarded as a classic ridge regression problem. By adding a regularization term, the ill-posed problem that is difficult to solve by the least squares method is transformed into a well-posed problem. The objective optimization function is:

[0125]

[0126] Where λ is the regularization term coefficient. The above formula is written in matrix form and the derivative is obtained:

[0127] ω=(X T X+λI) -1 X T y

[0128] Substituting the circulant matrix X extracted from the sample set into the above formula, we can get the filter ω as:

[0129]

[0130] Since F is the Fourier transform matrix, the Fourier transform of the filter is:

[0131]

[0132] Perform Fourier transform on the extracted sample set features to initialize the position filter ω trans and the scaling filter ω scale .

[0133] Specifically, for the frame regression filter, according to the mapping relationship in the two-dimensional space, the candidate frame P is sampled = (P x , P y , P w , P h ), and the actual bounding box coordinates G = (G x , G y , G w , G h ) has a displacement factor d x , d y and scale factor d w , d h satisfy:

[0134] G x =P w d x +P x

[0135] G y =Ph d y +P y

[0136]

[0137]

[0138] According to formula (1), the displacement factor solved by regression is and scale factor The displacement factor d x , d y and scale factor d w , d h equal, thus making the predicted bounding box As close to the real border as possible (G x , G y , G w , G h ).

[0139] The label of each sampling bounding box can be obtained by the following formula:

[0140]

[0141]

[0142]

[0143]

[0144] The label y of the training sample i Including displacement factor d x , d y , scale factor d w , d h And the score of the sample.

[0145] y i =(d xi d yi d wi d hi score i ) T

[0146] The score calculation formula for the sample is as follows

[0147]

[0148] Among them, d i represents the displacement factor d of the i-th sample x , d y and scale factor dw , d h . The same calculation is performed for the filter, where ω=[ω dx ,ω dy ,ω dw ,ω dh ,ω score ].

[0149] S3: Rough positioning process.

[0150] Get a new frame of image, extract image samples with the previous frame of image as the center, calculate the confidence map according to the position filter, and the position with the highest response is the new position, completing the coarse positioning of the position.

[0151] With the new position as the center, samples are extracted at different scales, the number of scale pools is increased through bilinear interpolation, the response confidence map is obtained according to the scale filter calculation, and the coarse positioning of the scale is completed through inverse Fourier transform with the maximum response position as the scale.

[0152] S4, refined regression process.

[0153] Based on the position and scale determined by coarse positioning, sample sampling is performed to extract the sample feature set x new , after Fourier transformation, we get Calculate Response

[0154]

[0155] in, Hook the trained scale filter. The target response map can be obtained by inverse Fourier transform.

[0156]

[0157] The response F is an N×5 matrix, where N is the number of samples extracted. For one of the samples, F=[F dx , F dy , F dw , F dh , F score ].

[0158] Remember F score The largest sample is F * , get its translation factor and scale factor F * dx , F * dy , F * dw , F * dh , calculated using formula (1), the regressed bounding box is That is, the predicted target position after regression, and the border needs to be added with a penalty learning rate. The corrected new border is:

[0159]

[0160]

[0161] The above penalty learning rate calculation process is as follows:

[0162] Since the scale factor changes rapidly, directly fitting the border scale will lead to divergence. In order to prevent divergence, it is necessary to set a penalty learning rate for the fitted border to penalize the aspect ratio change and scale change. The aspect ratio change penalty factor is:

[0163]

[0164] The scale change penalty factor is

[0165]

[0166] Among them, P h , P w are the height and width of the target before bounding box regression, and are the predicted target height and width after bounding box regression.

[0167] As an optional implementation, when the aspect ratio or the scale becomes larger, the aspect ratio penalty factor and the scale penalty factor will be less than 1, which affects the penalty effect. Therefore, the aspect ratio penalty factor and the scale penalty factor need to be processed to ensure that they are greater than 1.

[0168]

[0169]

[0170] The comprehensive penalty factor is

[0171]

[0172] Among them, K is the penalty coefficient.

[0173] The final penalty learning rate used is

[0174] 1. p =p×scores×K

[0175] S5: filter update process.

[0176] The update of the filter includes the update of the appearance model and the update of the filter model. A linear update strategy is used to perform weighted updates on the target appearance and the filter.

[0177] x=(1-β)x last +βx new

[0178] ω=(1-β)ω last +βω new

[0179] Among them, β is the learning rate, x last For the old appearance model, x new is the appearance model of the new target position, ω last is the old filter model, ω new is the newly obtained filter model.

[0180] S6. Determine whether the sequence is finished. If it is finished, the tracking is finished. If it is not finished, return to step S3.

[0181] In summary, the method of this embodiment has the following advantages and beneficial effects compared with the prior art:

[0182] (1) The method of this embodiment designs a frame regression filter to fit a frame that is closest to the real target. This breaks through the limitations of traditional discrete samples and discrete sample interpolation, and obtains a more accurate target position in the continuous domain to improve the accuracy of detection or tracking.

[0183] (2) The bounding box regression filter models the translation scale factor between the target box and the sample box, and uses L2 norm regularization for least squares linear regression. The Fourier diagonalization property of the circulant matrix is ​​used to transfer the calculation to the Fourier domain to solve the ridge regression problem, thereby improving the running speed.

[0184] (3) The idea of ​​coarse positioning and fine regression is adopted to reduce the number of samples and ensure the real-time performance of the tracker. The error accumulation problem caused by the fixed aspect ratio and the independence of the position scale filters in coarse positioning is solved.

[0185] (4) The bounding box regression filter does not need to be trained in advance. Instead, the filter is trained online during the tracking process. It does not require GPU computing power support and can run in real time on a device equipped with only a CPU.

[0186] This embodiment also provides a multi-scale adaptive correlation filtering target tracking device, including:

[0187] at least one processor;

[0188] at least one memory for storing at least one program;

[0189] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 The method shown.

[0190] A multi-scale adaptive correlation filtering target tracking device of this embodiment can execute a multi-scale adaptive correlation filtering target tracking method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0191] The present application also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 1 The method shown.

[0192] This embodiment also provides a storage medium, which stores instructions or programs that can execute a multi-scale adaptive correlation filtering target tracking method provided by an embodiment of the method of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.

[0193] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0194] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0195] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0196] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0197] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0198] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0199] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0200] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0201] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A multi-scale adaptive correlation filtering target tracking method, It is characterized in that The following steps are involved: Acquire an image sequence, acquire a first frame of image, and determine a target to be tracked in the first frame of image; Obtain the appearance model of the target according to its position, and initialize the position filter, scale filter, and border regression filter; Get a new image, get the new position of the target according to the position filter, and complete the rough positioning; Based on the new position, the target scale in the new image is obtained according to the scale filter to complete the rough positioning of the scale; Based on the position and scale determined by coarse positioning, sample the target border according to the border regression filter, extract the sample feature set, calculate the target response map, and obtain the target position of fine regression; The target position of the refined regression is used as the final target position of the new image, and the position filter, scale filter and border regression filter are updated according to the final target position; The design and solution method of the border regression filter are as follows: Considering bounding box regression as a ridge regression problem, the optimization problem to be solved is: Among them, ω is the filter to be learned, x i is the extracted training sample, λ is the regularization coefficient, y i is the label of the training sample; This problem has a closed solution. By matrixing the problem and taking the derivative of the filter ω, we can get: ω=(X H X+λI) -1 X H y 1 Where X is the sample [x 1 x 2 x 3 …x n ] T The circulant matrix obtained by cyclic expansion, I is the identity matrix; The training sample label and bounding box regression filter structure is designed as follows: According to the mapping relationship in the two-dimensional space, the candidate bounding box P = (P x , P y , P w , P h ), and the actual bounding box coordinates G = (G x , G y , G w , G h ) x , d y and scale factor d w , d h satisfy: G x =P w d x +P x G y =P h d y +P y In the formula, x and y are the coordinate values ​​of the center of the border, h represents the height of the border, and w represents the width of the border; According to formula (1), the displacement factor solved by regression is and scale factor Should be consistent with the actual displacement factor d x , d y and scale factor d w , d h equal, thus making the predicted bounding box As close to the real border as possible (G x , G y , G w , G h ); The label of each sampling bounding box can be obtained by the following formula: Design the labels y of the training samples i Including displacement factor d x , d y , scale factor d w , d h And the score of the sample; y i =(d xi d yi d wi d hi score i ) T The score calculation formula of the sample is as follows: Among them, d i represents the displacement factor d of the i-th sample x , d y and scale factor d w , d h .

2. A multi-scale adaptive correlation filtering target tracking method according to claim 1, It is characterized in that The bounding box regression filter ω anchor Sample extraction was performed by: Consider the sampling factor ratio of different aspect ratios = P w / P h , where P h is the width of the border P, P w is the height of the border P; the extended sampling of the aspect ratio is performed at the center of the border P, where the extended range covers the maximum sampling factor ratio max Ratio to the minimum sampling factor min All aspect ratios between 2 and 5, and border widths between 2 and 5 min *P w , ratio max *P w ], the border height is in [ratio min *P h , ratio max *P h ] interval; fit the translation factor of the border.

3. A multi-scale adaptive correlation filtering target tracking method according to claim 2, It is characterized in that In the process of fitting the translation factor of the border, the step size step is determined by the following method: Among them, γ is the proportional coefficient, step w is the moving step length in the border height direction, step h is the moving step length in the border width direction.

4. The multi-scale adaptive correlation filtering target tracking method according to claim 1, It is characterized in that The extracting of the sample feature set, calculating the target response map, and obtaining the target position of the precise regression include: Extract sample feature set x new , after Fourier transformation, we get Calculate the response in, For the trained scale filter, the target response map can be obtained by inverse Fourier transform: Among them, the response F is an N×5 matrix, where N is the number of samples extracted. For one of the samples, in, is the displacement factor d x The response value of is the displacement factor d y The response value of is the scale factor d w The response value of is the scale factor d h The response value, F score is the score of the response value; F score The largest sample is F * , get its translation factor and scale factor The bounding box after regression is That is, the predicted target position after regression, and the border is corrected using the penalty learning rate. The corrected new border is: in, is the border size after regression, P is the border size before regression, lr p is the penalty learning rate.

5. A multi-scale adaptive correlation filtering target tracking method according to claim 4, It is characterized in that The calculation process of the penalty learning rate is as follows: Since the scale factor changes rapidly, directly fitting the border scale will lead to divergence. In order to prevent divergence, it is necessary to set a penalty learning rate for the fitted border to penalize the aspect ratio change and scale change. The aspect ratio change penalty factor is: The scale change penalty factor is: Among them, P h , P w are the height and width of the target before bounding box regression, and The predicted target height and width after bounding box regression; Considering that when the aspect ratio or scale becomes larger, the aspect ratio penalty factor and the scale penalty factor will be less than 1, affecting the penalty effect, so it is necessary to process the aspect ratio penalty factor and the scale penalty factor to ensure that they are greater than 1; The comprehensive penalty factor is: The final penalty learning rate used is: lr p =p×scores×K Among them, K is the penalty coefficient.

6. A multi-scale adaptive correlation filtering target tracking method according to claim 1, It is characterized in that The filter update method is as follows: The update of the filter includes the update of the appearance model and the update of the filter model. The linear update strategy is used to perform weighted updates on the target appearance and the filter: x 1 =(1-β)x last +βx new ω=(1-β)ω last +vo new Among them, β is the learning rate, x last For the old appearance model, x new is the appearance model of the new target position, ω last is the old filter model, ω new is the newly obtained filter model; x 1 represents the appearance model after linear weighted update, and ω represents the filter after linear weighted update.

7. A multi-scale adaptive correlation filtering target tracking device, It is characterized in that include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a processor-executable program stored therein, It is characterized in that The processor-executable program is used to perform the method according to any one of claims 1 to 6 when executed by the processor.

Images