Target multi-scale tracking and re-detection method and device based on trajectory prediction, and medium

Abstract

A target multi-scale tracking re-detection method, device, and medium based on trajectory prediction are disclosed. The method includes: acquiring tracking images; for the first frame image, setting the initial target position and scale factor; training and initializing the position tracker template and scale tracker template; for non-first frame images, adjusting the target region of the current frame according to the scale factor of the previous frame, calculating the peak-to-sidelobe ratio and peak value of preset relevant evaluation indicators between the previous and next frames, and calculating the dynamic level of the indicators to determine whether occlusion occurs; when there is no occlusion, calculating the latest target position and scale factor, and generating trajectory prediction information; adjusting the target region according to the latest scale factor and updating the scale tracking filter template; updating the target position and scale factor; adjusting the target region and updating the position tracking filter template, and storing it as a re-detection template; when there is occlusion, stopping the template update, predicting the target position in the next frame and determining the search region; extracting features within the search region and matching them with the re-detection template to determine whether the target reappears.

Description

Technical Field

[0001] This invention relates to the field of image motion analysis technology, and in particular to a target multi-scale tracking re-detection method, device and medium based on trajectory prediction. Background Technology

[0002] The Fast Discriminative Scale Space Tracking (FDSST) algorithm is a visual tracking method that uses target image features to match and correlate frames before and after the target. It has wide applications in scenarios such as photoelectric surveillance.

[0003] In existing technologies, the FDSST algorithm extracts the target's gradient orientation histogram (HOG) features and constructs discriminative correlation filters in both the position and scale domains to achieve joint tracking of the target's position and scale. Its advantage lies in the ability to perform target scale tracking simultaneously with position tracking, addressing size changes caused by variations in target distance, and adaptively adjusting the target tracking area to make target image features more accurate, thus significantly improving tracking continuity.

[0004] However, in visual tracking tasks, targets are often lost due to occlusion in complex backgrounds. When the target reappears, it is difficult to quickly re-detect and resume tracking. When the target is completely occluded, the position tracking filter and scale tracking filter of the existing FDSST algorithm are continuously updated during the occlusion process, gradually introducing occlusion features, causing the target feature template to be "contaminated" by occlusion features. When the target reappears in the field of view, the tracking box is often locked on the occluder, making it impossible to resume matching and tracking the original target.

[0005] To achieve rapid re-detection and recovery tracking, some improved techniques include: when occlusion occurs, searching for the target by performing local and global searches sequentially, and using GPU devices for algorithm acceleration. However, local and global search strategies require traversal matching over a large image area, resulting in high computational overhead, and rely on parallel computing devices such as GPUs to meet real-time requirements. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention provides a target multi-scale tracking and re-detection method device and medium based on trajectory prediction, employing the following technical solution: Firstly, a target multi-scale tracking re-detection method based on trajectory prediction is provided. include: S1: Initialize the parameters of the multi-scale tracker, including the position tracker parameters and the scale tracker parameters; acquire the continuously tracked image; S2: For the first frame image, set the initial target position and initial scale factor to determine the target region; S3: Calculate the fhog feature matrix of the target region, generate the position tracking response confidence map, and train the initialization template for the position tracker; S4: Calculate the multi-scale fhog feature matrix of the target region, generate the scale tracking response confidence map, and train the initial scale tracker template; S5: For images that are not the first frame, adjust the target region of the current frame according to the scale factor of the previous frame; extract the fhog feature matrix from the adjusted target region and perform correlation matching with the fhog feature matrix of the previous frame to generate a confidence map of the current position tracking response. S6: Based on the current position tracking response confidence map, calculate the peak sidelobe ratio (PSR) and peak PV of the preset relevant evaluation indicators in the previous and next frames, and calculate the dynamic level of the indicators to determine whether the target is occluded. S7: When it is determined that the target is not occluded, calculate the latest position of the target based on the current position tracking response confidence map; and generate trajectory prediction information based on the position information. S8: Based on the latest target position, calculate the multi-scale fhog feature matrix of the target region, and perform relevant response calculation based on the scale tracker template of the previous frame to obtain the latest target scale factor; S9: Adjust the target region based on the latest target scale factor and calculate the multi-scale fhog feature matrix of the adjusted target region; update the scale tracking filter template; update the target position and scale factor based on the updated scale tracking filter template. S10: Based on the updated target location and scale factor, adjust the target region and calculate the fhog feature matrix of the adjusted target region, update the position tracking filter template, and store the position tracking filter template of at least one historical frame as a re-detection template. S11: When it is determined that the target is occluded, stop updating the position tracking filter template and the scale tracking filter template, predict the target's position in the next frame based on the trajectory motion information, and determine the search area based on the predicted position and the latest scale factor when the target is not occluded. S12: During the occlusion period, extract the image fhog feature matrix within the search area and match it with the re-detection template. When the matching index meets the preset conditions, it is determined that the target has reappeared, and return to step S5 to resume continuous tracking. Otherwise, return to step S11 to continue re-detection.

[0007] 2. The target multi-scale tracking re-detection method based on trajectory prediction according to claim 1, characterized in that, To determine whether a target is occluded, the occlusion conditions are: PV is greater than the first threshold and PSR is greater than the second threshold of the current dynamic PSR level.

[0008] In a second aspect, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, performs the steps of the method described in the first aspect.

[0009] Thirdly, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] The target multi-scale tracking re-detection method based on trajectory prediction in this application has the following beneficial effects: This application uses a trajectory prediction-based method to update target location information for re-detection, which has a shorter matching time compared with large-scale search matching and is suitable for embedded platforms with high real-time requirements.

[0011] The electronic device and readable storage medium corresponding to the target multi-scale tracking re-detection method based on trajectory prediction in this application can achieve the same technical effect, and will not be described in detail here to avoid duplication. Attached Figure Description

[0012] Figure 1 A schematic flowchart illustrating a target multi-scale tracking re-detection method based on trajectory prediction provided in an embodiment of this application; Figure 2 This is a first effect diagram of an example of a target multi-scale tracking re-detection method based on trajectory prediction provided in an embodiment of this application; Figure 3 This is a second effect diagram of an example of a target multi-scale tracking re-detection method based on trajectory prediction provided in an embodiment of this application; Detailed Implementation

[0013] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the technical solutions in the embodiments of this application are clearly described. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] The steps described in this application and the flowcharts in the accompanying drawings are not necessarily strictly executed according to the step numbers; the execution order of the steps can be changed. Furthermore, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.

[0016] This specification provides a target multi-scale tracking and re-detection method based on trajectory prediction, a computer device, and a computer-readable storage medium, which are described in detail below with reference to the accompanying drawings and preferred embodiments.

[0017] Please see Figure 1 This application provides a target multi-scale tracking re-detection method based on trajectory prediction, including: Step S1: Initialize the parameters of the multi-scale tracker, including the position tracker parameters and the scale tracker parameters; acquire the continuously tracked image.

[0018] In this step, parameters such as multi-scale tracking template size, template update rate, and number of scales are initialized.

[0019] Step S2: For the first frame image, set the initial target position and initial scale factor to determine the target region.

[0020] In this step, the first frame image frame_1 sets the initial position roi of the target, including the coordinates (x, y) of the top left vertex of the target, and the initial scale factor; the target region is the width and height dimensions (width, height).

[0021] Step S3: Calculate the fhog feature matrix of the target region, generate the position tracking response confidence map, and train the initial position tracker template.

[0022] In this step, the target fhog feature matrix tmpl_1 is calculated, the position tracking response confidence map prob is generated, and the position tracker template alphaf is initialized.

[0023] Step S4: Calculate the multi-scale fhog feature matrix of the target region, generate the scale tracking response confidence map, and train the initial scale tracker template.

[0024] In this step, the target multi-scale fhog feature matrix xsf is calculated, the scale tracking response confidence map ysf is generated, and the initial scale tracker template sf is trained.

[0025] Step S5: For non-first frame images, adjust the target region of the current frame according to the scale factor of the previous frame; extract the fhog feature matrix from the adjusted target region and perform correlation matching with the fhog feature matrix of the previous frame to generate a current position tracking response confidence map.

[0026] In this step, for non-initial frame_n, the target region size is updated according to the scale factor_n-1 of the previous frame, the target fhog feature matrix tmpl_n is calculated, and then the feature matrix is ​​correlated and matched with the target fhog feature matrix _tmpl_n-1 of the previous frame in the frequency domain to obtain the current position tracking response confidence map res.

[0027] Step S6: Based on the current position tracking response confidence map, calculate the peak sidelobe ratio (PSR) and peak PV of the preset relevant evaluation indicators in the previous and next frames, and calculate the dynamic level of the indicators to determine whether the target is occluded.

[0028] In this step, based on the current position tracking response confidence map res, the peak sidelobe ratio (PSR) and peak PV of the correlation evaluation indicators between the previous and next frames are obtained, and the average PSR value within n frames, PSR_aver, is calculated as the target correlation dynamic level.

[0029] Further, it is determined whether the target is occluded, where the occlusion condition is: PV is greater than the first threshold and PSR is greater than the second threshold of the current PSR dynamic level.

[0030] In other words, when both PSR and PV meet the conditions, the match is considered good and the target is not occluded. Otherwise, a decrease in the matching index indicates that the target is occluded and the target is determined to be occluded.

[0031] Step S7: When it is determined that the target is not occluded, calculate the latest position of the target based on the current position tracking response confidence map; and generate trajectory prediction information based on the position information.

[0032] In this step, the latest target position is obtained based on the current position tracking response confidence map res. The miss distances of the target in the x and y directions in the nearby multiple frames are accumulated and calculated. Kalman filtering is used to predict the target's moving speed in real time. The two are weighted together as the target's predicted speed information to generate the target's trajectory prediction information.

[0033] Step S8: Based on the latest target position, calculate the multi-scale fhog feature matrix of the target region, and perform relevant response calculation based on the scale tracker template of the previous frame to obtain the latest target scale factor.

[0034] In this step, the target multi-scale fhog feature matrix xsf is extracted based on the latest target position, and the correlation response is calculated with the previous frame multi-scale tracking filter template to obtain the latest target scale factor.

[0035] Step S9: Adjust the target region according to the latest target scale factor and calculate the multi-scale fhog feature matrix of the adjusted target region, and update the scale tracking filter template; update the target position and scale factor according to the updated scale tracking filter template.

[0036] In this step, the target region image is extracted based on the latest target scale factor, the target multi-scale fhog feature matrix is ​​calculated, the denominator and numerator (sf_den, sf_num) of the scale tracking filter template are trained, and the scale tracking filter template sf is updated by weighting the template update rate. Finally, the target position and scale are updated.

[0037] Step S10: Based on the updated target location and scale factor, adjust the target region and calculate the fhog feature matrix of the adjusted target region, update the position tracking filter template, and store the position tracking filter template of at least one historical frame as a re-detection template.

[0038] In this step, the target fhog feature matrix is ​​extracted based on the latest target position and scale factor, the position tracking filter template is trained and updated, and the templates alphaf_5 and alphaf_10 of the next n-5 and n-10 frames are stored as the target's historical real templates for use during re-detection. The position tracking filter template is also updated by weighting the template update rate.

[0039] Step S11: When it is determined that the target is occluded, stop updating the position tracking filter template and the scale tracking filter template, predict the target's position in the next frame based on the trajectory motion information, and determine the search area based on the predicted position and the latest scale factor when the target is not occluded.

[0040] In this step, a decrease in the matching metric indicates that the target is occluded. The target miss distances and Kalman prediction velocities in neighboring frames are linearly weighted, and the target position in the next frame is predicted based on the target position before the matching metric decreased. The target width and height are calculated using the scale factor before the matching metric decreased.

[0041] Step S12: During the occlusion period, extract the image fhog feature matrix within the search area and match it with the re-detection template. When the matching index meets the preset conditions, it is determined that the target has reappeared, and return to step S5 to resume continuous tracking. Otherwise, return to step S11 to continue re-detection.

[0042] In this step, an image of a specified region is extracted based on the target's predicted location and the search area size. This region image is used as the target to calculate its fhog feature. This feature is then correlated with the target's historical target position tracking filter template stored before the matching index decreased. The maximum value is used to determine the occlusion condition. When the matching index is met, continuous tracking resumes. When the matching index is not met, re-detection continues. In some possible implementations, when the matching of consecutive frames does not meet the preset condition, the target search size can be increased to compensate for the prediction position error and improve the matching probability after the target appears.

[0043] This application employs a trajectory prediction-based method to update target location information for re-detection. Compared to large-scale search matching, it has a shorter matching time and is suitable for embedded platforms with high real-time requirements. First, this application adds a logical judgment to quantitatively calculate the occurrence of target occlusion. The quantitative indicator combines dynamic statistical values ​​from the current target tracking process with a fixed threshold, reflecting both the general characteristics of the tracking and the matching characteristics of the target at different stages, providing a fast and accurate judgment when occlusion occurs. Second, it adopts a trajectory prediction-based target re-detection strategy. This trajectory prediction utilizes Kalman prediction information from the normal target matching process and the miss distance information before occlusion, making the target trajectory prediction more accurate. Furthermore, considering the multi-scale tracking characteristics of the target, if the target is not detected for several consecutive frames, the search size is increased, improving the search probability without affecting the position tracking-related calculations. This invention updates the search area using target trajectory prediction information during target re-detection, without employing a large-scale search cyclic matching method, thus not increasing the computational load of FHOG features. Therefore, the algorithm's real-time performance is guaranteed without relying on external parallel computing devices, making it applicable to embedded platforms.

[0044] More specifically, the multi-scale target tracking algorithm incorporates a detection mechanism to assess the quality of target matching. Occlusion is quantitatively determined by real-time calculation of two indicators: the dynamic level of the peak-to-sidelobe ratio and the correlation peak. Dynamic matching of neighboring frames accurately identifies anomalies, making the detection method more adaptable to the current tracking scenario, rather than relying on a fixed threshold throughout the process. During periods of good matching, a backup template for the target position tracker is continuously stored and updated. Upon occlusion, training and updating of both the position tracker and scale tracker templates are immediately halted, and the stored backup template from neighboring frames is used for target re-detection. A trajectory prediction-based target re-detection strategy is employed after target occlusion. Kalman prediction is used to calculate the target prediction velocity and miss distance in real-time. After occlusion, the target search position is updated with the trajectory prediction value, avoiding distortion of position tracker calculations during occlusion. When the target reappears, its position can be accurately predicted, improving the re-detection success rate. When target tracking is good, on the one hand, the peak-to-sidelobe ratio dynamic level and the correlation peak value calculated by the target position tracker are calculated in real time to determine whether occlusion has occurred. When one of the indicators drops significantly, it indicates a tracking anomaly, and there is a significant difference between the target tracking area and the target template. On the other hand, the target Kalman prediction velocity and miss distance are calculated in real time to understand its motion pattern, and the target position tracking filter template is stored as a backup matching template when the target is occluded. When target tracking is abnormal, the template training and update of the target position tracker and scale tracker are stopped first. The target position vertex coordinates and width and height of the next frame are updated using the current scale factor and the target trajectory prediction value. This target area image is matched with the first stored backup template of the position tracker. If the match fails multiple times, the target search size is increased under the current scale factor. Secondly, during the re-detection process, the template training and update of the position tracker and scale tracker, the calculation of the scale tracker's related response, the update of the occlusion judgment quantitative indicators, and the calculation and update of the target trajectory prediction information are continuously stopped. The target template, scale factor, matching degree index, and target prediction velocity are kept to be the calculation results before the occlusion occurred.

[0045] The following provides a specific example of a target multi-scale tracking re-detection method based on trajectory prediction.

[0046] Step 1: Initialize the position tracker and scale tracker parameters. The main parameter settings are as follows: Target search expansion parameter: padding = 2.5; Location tracking model update rate: space_lr = 0.012; Hog feature: cell_size = 4; PCA dimension: dim = 13; Target feature template size: tmpl_size = 96; Number of scales: scale_num = 33; Scale step size: scale_step = 1.02; The maximum scale tracking region is scale_max = 512; Scale tracking model update rate: scale_lr = 0.025; The scale model outputs the label parameter scale_sigma_factor = 0.0625; Step 2: Input the first frame image frame_1, and set the coordinates of the top left vertex of the target (x=510, y=455) and the width and height dimensions (width=130, height=120); Step 3: Based on the target input size and search expansion parameters, adjust the target image to z according to the feature template size, and calculate the target fhog feature matrix tmpl; tmpl = fhog(z, cell_size); Generate the initial position tracking response confidence map prob; prob = createGaussianPeak(size_patch[0], size_patch[1]); Among them, size_patch[0] = z.rows / cell_size, size_patch[1] = z.cols / cell_size; Initialize the position tracker template matrix alphaf; alphaf = Mat(size_patch[0], size_patch[1]); Step 4: Generate the scale tracking response confidence map ysf; ysf = fft(exp(-0.5 × (ss^2) / scale_sigma^2)); Wherein, the scale parameter scale_sigma = scale_num / sqrt(33) × scale_sigma_factor; Discrete independent variable vector ss = (1:scale_num) - ceil(scale_num / 2); Calculate the target multi-scale fhog feature matrix xsf, and train the initial scale tracker template sf; sf = sf_num / sf_den; The denominator and numerator are obtained by performing the following operations in the frequency domain: sf_num = mulSpectrums(ysf, xsf) sf_den = mulSpectrums(xsf, xsf) The fifth step is to perform PCA dimensionality reduction on the target fhog feature matrix tmpl to obtain the feature matrix X, and then train the position tracking filter template. k = gaussianCorrelation(X, X); alphaf = complexDivision(prob, (fftd(k) + 0.0001)); Step 6: For non-initial frame_n, update the target region size according to the scale factor updated in the previous frame, extract the target image and adjust it to the fhog feature template size of 96×96, and calculate the target fhog feature matrix tmpl_n. Then, perform correlation matching between this feature matrix and the target fhog feature matrix tmpl_n-1 in the frequency domain to obtain the position tracking response confidence map res. res = (real(fftd(complexMultiplication(alphaf, fftd(k)), true))); Among them, k = gaussianCorrelation(tmpl_n, tmpl_n-1); Step 7: Based on the position-related response confidence map res, obtain the peak sidelobe ratio (PSR) and peak PV of the correlation evaluation metrics between consecutive frames. Simultaneously, calculate the average PSR_aver over 10 frames as the dynamic level of the target feature correlation. The calculation process is as follows: PV = minMaxLoc(res); mean, std =meanStdDev(res); PSR = (PV - mean[0]) / (std[0] + 0.00001); PSR_aver = PSR_sum / 10; Wherein, PSR_sum is the cumulative sum of PSR values ​​over the past 10 frames; Step 8: When PV > 0.4 and PSR > PSR_aver×0.75 are both satisfied, the match is considered good, and proceed to step 9; otherwise, if the matching index decreases, it means that the target is occluded, and proceed to step 13. Step 9: Obtain the latest target position based on the response confidence graph res from step 7: roi.x = cx - roi.width / 2 + res.x × cell_size × scale ×scalefactor; roi.y = cy - roi.height / 2 + res.y × cell_size × scale ×scalefactor; Among them, scale = (roi.width × padding) / template_size; The target miss distance delta_aver in the horizontal and vertical directions of the next four frames is accumulated and calculated. The target moving speed Speed_est is predicted in real time using Kalman filtering as the target trajectory prediction information. The Kalman prediction calculation process is as follows, taking the horizontal direction as an example: Position_cal = roi.x; Position_pre =Position_est + Speed_est × deltaT; Speed_pre = Speed_est; P11_pre = P11 + P21 × deltaT + (P12 + P22 × deltaT) × deltaT + Q11; P12_pre = P12 + P22 × deltaT; P21_pre = P21 + P22 × deltaT; P22_pre = P22 + Q22; K1 = P11_pre / (P11_pre + R); K2 = P21_pre / (P11_pre + R); Position_est = Position_pre + (Position_cal - Position_pre)×K1; Speed_est = Speed_pre + (Position_cal - Position_pre)×K2; P11 = (1 - K1) × P11_pre; P12 = (1 - K1) × P12_pre; P21 = -K2 × P11_pre + P21_pre; P22 = -K2 × P12_pre + P22_pre; Step 10: Extract the target multi-scale fhog feature matrix xsf based on the latest target position, and perform correlation response calculation with the previous frame multi-scale tracking filter template sf to obtain the latest target scale factor. scalefactor = scalefactor × interp_scaleFactors[scale_pi.x]; Where scale_pi.x represents the coordinates of the maximum value of the scale tracking-related response; Step 11: Based on the latest target scale factor, extract the target region image, calculate the target multi-scale fhog feature matrix, train the scale tracking filter templates new_sf_den and new_sf_num, and update the denominator and numerator of the scale tracking filter templates by update rate weighting. Then update the target width and height. Finally, update the target top left vertex coordinates in the next frame based on the latest target width and height. sf_den = (1 - scale_lr) × sf_den + scale_lr × new_sf_den; sf_num = (1 - scale_lr) × sf_num + scale_lr × new_sf_num; roi.width = base_width × scalefactor; roi.height = base_height × scalefactor; roi.x = cx - roi.width / 2; roi.y = cy - roi.height / 2; Where base_width and base_height are the initial width and height dimensions of the target bounding box, respectively, and cx and cy are the coordinates of the target center point; Step 12: Based on the latest target position and scale factor, extract the target fhog feature matrix tmpl, perform PCA dimensionality reduction to obtain the feature matrix X, train the position tracking filter template new_alphaf, and update the position tracking filter template alphaf by weighting the update rate. k = gaussianCorrelation(X, X); new_alphaf = complexDivision(prob, (fftd(k) + 0.0001)); alphaf = (1 - space_lr) × alphaf + (space_lr) × new_alphaf; Simultaneously, store the adjacent n-5 and n-10 frame templates alphaf_5 and alphaf_10 as target historical real templates for use during re-detection; Step 13: When the position tracker's matching index decreases in step 8, indicating occlusion, linearly weight the mean target miss distance of neighboring multi-frame targets and the Kalman prediction velocity from step 9: roi.x = roi.x + 0.2× Speed_est × 0.02 + 0.8×failcnt×deltax_3_aver; roi.y = roi.y + 0.2× Speed_est × 0.02 + 0.8×failcnt×deltay_3_aver; Where failcnt is the number of consecutive failed matches, and the weight here tends to be the average off-target amount of the target before occlusion; The target position (ROI) in the next frame is predicted based on the target position before the matching index drops. The target width and height are calculated using the scale factor before the matching index drops. When the matching fails for 10 consecutive frames, the target search size is appropriately increased to compensate for the prediction position error and improve the matching probability after the target appears. roi.width = (1+ failcnt ×0.015) × originwidth; roi.height = (1+ failcnt ×0.015) × originheight; Among them, originwidth and originheight are the target width and height records during normal matching; Step 14: Extract the image of the specified region based on the target's predicted location and search size. Calculate the fhog feature of this region image as the target. Perform correlation calculations between this feature and the multi-frame position tracking filter templates (alphaf_5, alphaf_10) stored before the matching index decreases. Take the maximum value to determine the occlusion condition. If the matching index is satisfied, proceed to Step 9; otherwise, proceed to Step 13.

[0047] See the example for the effect. Figure 2-3When the black vehicle enters under the bridge, the tracking box changes from green to blue, indicating that a tracking anomaly has been detected. In subsequent frames, the blue tracking box automatically updates its position, and the tracking box increases slightly starting from (d). Starting from (f), the target gradually moves away from the occlusion. In (g), the tracking box successfully appears at the target's location, indicating that the trajectory prediction is effective. At this point, the tracking box is still blue because the target template uses a good template before the tracking occlusion, and the matching condition has not yet been met. In (h), after the target completely moves away, the matching is successful. At the same time, the target tracking box in (h) returns to its size before the occlusion, indicating that expanding the search area during the re-detection process did not affect subsequent scale tracking.

[0048] Corresponding to the above embodiments of the target multi-scale tracking re-detection method based on trajectory prediction, this application provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps and processes of the above embodiments of the target multi-scale tracking re-detection method based on trajectory prediction, and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0049] Memory can be used to store software programs and various data. Memory can primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area can store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, memory can include volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus RAM (DRRAM). The memory in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0050] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly handles operations related to the operating system, user interface, and applications, while the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor.

[0051] Corresponding to the above embodiments of the target multi-scale tracking re-detection method based on trajectory prediction, this application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the steps and processes of the above embodiments of the target multi-scale tracking re-detection method based on trajectory prediction, and achieve the same technical effect. To avoid repetition, these will not be described again here.

[0052] The processor is the processor in the electronic device described in the above embodiments of this application. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0053] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0054] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0055] It is understood that the embodiments of this application have been described above in conjunction with the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. As those skilled in the art will know, various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, those skilled in the art, under the guidance or instruction of this application, can modify these features and embodiments to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, this invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of this invention.

Claims

1. A target multi-scale tracking re-detection method based on trajectory prediction, characterized in that, include: S1: Initialize the parameters of the multi-scale tracker, including the position tracker parameters and the scale tracker parameters; acquire the continuously tracked image; S2: For the first frame image, set the initial target position and initial scale factor to determine the target region; S3: Calculate the fhog feature matrix of the target region, generate the position tracking response confidence map, and train the initialization template for the position tracker; S4: Calculate the multi-scale fhog feature matrix of the target region, generate the scale tracking response confidence map, and train the initial scale tracker template; S5: For images that are not the first frame, adjust the target region of the current frame according to the scale factor of the previous frame; extract the fhog feature matrix from the adjusted target region and perform correlation matching with the fhog feature matrix of the previous frame to generate a confidence map of the current position tracking response. S6: Based on the current position tracking response confidence map, calculate the peak sidelobe ratio (PSR) and peak PV of the preset relevant evaluation indicators in the previous and next frames, and calculate the dynamic level of the indicators to determine whether the target is occluded. S7: When it is determined that the target is not occluded, calculate the latest position of the target based on the current position tracking response confidence map; And generate trajectory prediction information based on location information; S8: Based on the latest target position, calculate the multi-scale fhog feature matrix of the target region, and perform relevant response calculation based on the scale tracker template of the previous frame to obtain the latest target scale factor; S9: Adjust the target region based on the latest target scale factor and calculate the multi-scale fhog feature matrix of the adjusted target region; update the scale tracking filter template; update the target position and scale factor based on the updated scale tracking filter template. S10: Based on the updated target location and scale factor, adjust the target region and calculate the fhog feature matrix of the adjusted target region, update the position tracking filter template, and store the position tracking filter template of at least one historical frame as a re-detection template. S11: When it is determined that the target is occluded, stop updating the position tracking filter template and the scale tracking filter template, predict the target's position in the next frame based on the trajectory motion information, and determine the search area based on the predicted position and the latest scale factor when the target is not occluded. S12: During the occlusion period, extract the image fhog feature matrix within the search area and match it with the re-detection template. When the matching index meets the preset conditions, it is determined that the target has reappeared, and return to step S5 to resume continuous tracking. Otherwise, return to step S11 to continue re-detection.

2. The target multi-scale tracking re-detection method based on trajectory prediction according to claim 1, characterized in that, To determine whether a target is occluded, the occlusion conditions are: PV is greater than the first threshold and PSR is greater than the second threshold of the current dynamic PSR level.

3. The target multi-scale tracking re-detection method based on trajectory prediction according to claim 2, characterized in that, The dynamic level of the indicator is the average value of PSR over multiple frames, calculated on a rolling basis.

4. The target multi-scale tracking re-detection method based on trajectory prediction according to claim 1, characterized in that, The parameters for initializing the multi-scale tracker also include the template update rate; The updating of the scale tracking filter template specifically involves updating the scale tracking filter template using a template update rate weighting method. Update the position tracking filter template by weighting the template update rate.

5. The target multi-scale tracking re-detection method based on trajectory prediction according to claim 1, characterized in that, In step S12, when the matching of consecutive set frames does not meet the preset conditions, the target search size of the set value is increased.

6. The target multi-scale tracking re-detection method based on trajectory prediction according to claim 1, characterized in that, The generation of trajectory prediction information based on location information includes: The miss distances of the target in the x and y directions of the nearby multiple frames are accumulated and calculated. Kalman filtering is used to predict the target's moving speed in real time. The two are weighted together as the target's predicted speed information to generate the target's trajectory prediction information.

7. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the target multi-scale tracking re-detection method based on trajectory prediction as described in any one of claims 1 to 6.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the target multi-scale tracking re-detection method based on trajectory prediction as described in any one of claims 1 to 6.

Images