A target tracking method and system for image target simulator

Abstract

The present application relates to the field of computer vision technology, and specifically to a target tracking method and system for an image target simulator, the method comprising: obtaining an initial bounding box and an initial search area; constructing a scale change factor for each frame; constructing a pixel difference index for each frame, and constructing a scale factor for each frame in combination with the scale change factor; obtaining moving pixels and stationary pixels for each frame and its previous frame; constructing an occlusion influence index for each frame, constructing a scale adjustment coefficient for each frame and determining a dynamic search area for each frame; constructing a position factor for each frame, and determining an adaptive search area for each frame to track the target. The present application improves the accuracy of target tracking by determining an adaptive search area.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target tracking method and system for an image target simulator. Background Art

[0002] An image seeker refers to a missile seeker that uses visible light or infrared image sensors to lock and track targets. It is usually installed on the head of guided weapons such as missiles, and measures the relative motion information between the missile and the target through visible light or infrared information. When performing performance testing on the image seeker, it is generally necessary to detect dynamic tracking, and the dynamic tracking test can be performed using an image target simulator.

[0003] When testing the image seeker to track the target on the image target simulator, the target tracking algorithm often used is the KCF algorithm. The existing KCF algorithm represents the target as a high-dimensional feature vector and calculates and updates the relevant filter in each frame to adapt to the changes in the target's appearance. However, the image seeker installed on the missile is in a high-speed motion state, so the scale of the target changes very obviously, and it is necessary to adaptively adjust the search area to ensure the accuracy of target tracking. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of this application is to provide a target tracking method and system for an image target simulator. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present application provides a target tracking method for an image target simulator, the method comprising the following steps:

[0006] Obtain key frames in the motion video of the image target to form a video sequence;

[0007] According to the distribution difference of corresponding pixels between each frame and the next frame in the video sequence, the moving pixels and the stationary pixels of each frame are obtained, the minimum enclosing rectangle of all moving pixels in the first frame is used as the initial bounding box of each frame in the video sequence, and the initial search area of ​​each frame is obtained;

[0008] The minimum bounding rectangle of all moving pixels in each frame except the first frame is used as the target bounding box of each frame;

[0009] Constructing a scale change factor for each frame according to the difference in the number of pixels between the target bounding box of each frame and the target bounding box of the previous frame; constructing a pixel difference index for each frame according to the distribution of pixel values ​​in the area between the initial search area and the target bounding box of each frame, and constructing a scale factor for each frame in combination with the scale change factor;

[0010] Constructing an occlusion impact index for each frame according to the difference in the number of moving pixels and the difference in the number of stationary pixels in the target bounding box of each frame and its previous frame, constructing a scale adjustment coefficient for each frame in combination with the scale factor and the initial search area, and determining a dynamic search area for each frame;

[0011] A position factor of each frame is constructed according to the offset of the target bounding box of each frame in the horizontal direction and the vertical direction, an adaptive search area of ​​each frame is determined in combination with the dynamic search area, and a target tracking algorithm is used to track the target.

[0012] Preferably, the method for acquiring the moving pixel points and the stationary pixel points of each frame is:

[0013] The optical flow algorithm is used between each frame and the next frame in the video sequence to obtain the velocity vector of each pixel in each frame; the modulus value of the velocity vector of each pixel in each frame is analyzed;

[0014] The average velocity vector of all pixels in each frame is formed by the average of the components of the velocity vector of all pixels in each frame in the horizontal direction and the average of the components of the velocity vector of all pixels in each frame, and the modulus of the average velocity vector is used as the motion threshold of all pixels in each frame;

[0015] The pixel points whose velocity vector modulus in each frame is greater than or equal to the motion threshold are the motion pixel points of the first frame; the pixel points whose velocity vector modulus in each frame is less than the motion threshold are the stationary pixel points of each frame.

[0016] Preferably, the method for acquiring the initial search area of ​​each frame is:

[0017] The two ends of the diagonal of the initial bounding box are increased by a preset ratio respectively, and the area surrounded by the bounding box after the diagonal increase is recorded as the initial search area of ​​each frame image in the video sequence.

[0018] Preferably, the scale change factor of each frame is the difference between the number of pixels in the target bounding box of each frame and the number of pixels in the target bounding box of the previous frame.

[0019] Preferably, the process of constructing the pixel difference index of each frame is:

[0020] Determine the mean of all pixel values ​​in the area between the initial search area and the target bounding box in each frame, recorded as the first mean of each frame; analyze the difference between each pixel value in the area between the initial search area and the target bounding box in each frame and the first mean, recorded as the pixel difference of each pixel point in the area between the initial search area and the target bounding box in each frame;

[0021] A pixel difference index of each frame is determined based on the pixel difference, where the pixel difference index of each frame is the product of the pixel differences of all pixel points in the area between the search area and the target bounding box in each frame.

[0022] Preferably, the scale factor of each frame is a result of fusing the scale change factor of each frame with a pixel difference index.

[0023] Preferably, the process of constructing the occlusion impact index of each frame is:

[0024] The ratio of the number of moving pixels to the number of stationary pixels in the target bounding box of each frame is analyzed and recorded as the occlusion impact ratio of each frame; the occlusion impact index of each frame is determined based on the occlusion impact ratio of each frame and its previous frame, and the occlusion impact index of each frame is the difference between the occlusion impact ratio of each frame and its previous frame.

[0025] Preferably, constructing the scale adjustment coefficient of each frame and determining the dynamic search area of ​​each frame includes:

[0026] Determining a scale adjustment coefficient for each frame based on the scale factor and the occlusion impact index, wherein the scale adjustment coefficient for each frame is a normalized result of a product of the scale factor and the occlusion impact index for each frame;

[0027] The length of the dynamic search area = (1 + scale adjustment coefficient) × the length of the initial search area;

[0028] The width of the dynamic search area = (1 + scale adjustment coefficient) × the width of the initial search area.

[0029] Preferably, the step of constructing the position factor of each frame and determining the adaptive search area of ​​each frame in combination with the dynamic search area includes:

[0030] Determine the distance between the left boundary of the target bounding box of each frame and the left boundary of the initial search area, recorded as the left distance of each frame; obtain the right distance of each frame according to the left distance determination method; take the direction of the minimum value between the left distance and the right distance of each frame as the offset direction of each frame; take half of the sum of the left distance and the right distance as the offset distance of each frame, and obtain the horizontal displacement of each frame in combination with the offset direction of each frame;

[0031] According to the method for obtaining the horizontal displacement of each frame, the vertical displacement of each frame is obtained based on the distance between the left boundary of the target bounding box of each frame and the upper and lower boundaries of the initial search area;

[0032] The vector sum of the horizontal displacement and the vertical displacement of each frame is used as the position factor of each frame;

[0033] The center of the dynamic search area of ​​each frame is moved according to the position factor to obtain the adaptive search area of ​​each frame.

[0034] In a second aspect, an embodiment of the present application also provides a target tracking system for an image target simulator, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0035] This application has at least the following beneficial effects:

[0036] The present application obtains the initial bounding box and initial search area of ​​each frame in the video sequence based on the distribution difference of corresponding pixel points between the first frame and the next frame in the video sequence; constructs the pixel difference index of each frame based on the distribution of pixel values ​​in the area between the initial search area and the target bounding box of each frame, and constructs the scale factor of each frame in combination with the scale change factor; constructs the occlusion influence index of each frame in combination with the difference of moving pixel points and static pixel points in the target bounding box of each frame and the previous frame to determine the dynamic search area of ​​each frame; constructs the position factor of each frame based on the horizontal and vertical offset of the target bounding box of each frame, and determines the adaptive search area of ​​each frame in combination with the dynamic search area to track the target. The present application improves the accuracy of target tracking by determining the adaptive search area. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A flowchart of the steps of a target tracking method for an image target simulator;

[0039] Figure 2 Extraction diagram for scale adjustment coefficients;

[0040] Figure 3 Schematic diagram of the target tracking process of the image target simulator. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, describes in detail a target tracking method and system for an image target simulator proposed in the present application, its specific implementation, structure, features and effects. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0043] The following is a detailed description of a target tracking method and system for an image target simulator provided by the present application with reference to the accompanying drawings.

[0044] See also Figure 1 , which shows a flowchart of a target tracking method for an image target simulator provided by an embodiment of the present application, the method comprising the following steps:

[0045] Step S1: Use an image target simulator to obtain a motion video of a simulated target and perform preprocessing.

[0046] The performance of the image seeker is tested by using an image target simulator to simulate the motion video of the aircraft and test the tracking effect of the image seeker on the target. It should be noted that the video simulating the motion of the aircraft needs to simulate the real flight situation, including the rapid change of position and the cloud cover during the flight. The key frames of the motion video of the image target are extracted to form a video sequence.

[0047] At this point, the preprocessed motion video of the simulated target is obtained.

[0048] Step S2: obtaining the initial bounding box and the initial search area of ​​each frame in the video sequence according to the distribution difference of corresponding pixel points between the first frame and the next frame in the video sequence; constructing the pixel difference index of each frame according to the pixel value distribution of the area between the initial search area and the target bounding box of each frame, and constructing the scale factor of each frame in combination with the scale change factor.

[0049] The aircraft motion video simulated by the image target simulator is used as the target for image seeker performance testing. The position of the aircraft in the video is tracked, and the accuracy of the tracking result is used as an indicator for evaluating the performance of the image seeker.

[0050] The KCF (Kernelized Correlation Filters) target tracking algorithm is a target tracking algorithm based on correlation filtering. It uses circulant matrices and fast Fourier transforms to efficiently calculate correlation filters. It has fast detection speed and high tracking accuracy and is widely used in practical applications.

[0051] However, the KCF algorithm has some defects when tracking targets in aircraft motion videos simulated using an image target simulator. Because the relative motion speed of the target aircraft is fast, the scale of the target will change greatly, and drift will occur during the tracking process. To solve this problem, the initial bounding box size can be determined according to the motion of the target, and the size of the search area can be adaptively adjusted.

[0052] The size of the initial bounding box determines the amount of contextual information when the algorithm starts. A larger bounding box can provide more contextual information and help deal with occlusion of the target and partial appearance changes, but a larger bounding box requires more calculations. On the contrary, a smaller bounding box can speed up calculations but lose some information.

[0053] The size of the search area determines the range in which the KCF algorithm searches for the target in each frame. If the search area is too large, it may lead to the tracking of erroneous background objects, while if it is too small, it may not cover the actual position of the target. Therefore, it needs to be adjusted according to the size of the initial bounding box and the situation around the bounding box.

[0054] The performance of the image seeker is tested by using an adaptive search area to track the target in the target motion video simulated by the target simulator.

[0055] Specifically, the initial bounding box is obtained based on the features of the target in the first frame of the video, and the initial search area is obtained from the initial bounding box and the background features around the target. The scale factor is calculated based on the scale change of the target, and the occlusion impact index is calculated considering the occlusion during the target tracking process. The position factor is calculated based on the position change of the target. The adaptive search area is obtained based on the scale factor, occlusion impact index and position factor. Finally, the KCF algorithm is applied to the search area of ​​each frame of the video to achieve target tracking.

[0056] S201: acquiring moving pixels and stationary pixels of a first frame according to distribution differences of corresponding pixels between each frame in a video sequence and its next frame, respectively constituting a moving region and a stationary region of the first frame, so as to obtain an initial bounding box of each frame in the video sequence.

[0057] Determining the initial bounding box of the target is crucial to the performance of the tracking algorithm. An accurate initial bounding box can improve the stability and accuracy of the tracking algorithm, especially when the target's appearance or scale changes greatly, as well as in scenes with occlusion or complex lighting changes.

[0058] In a video sequence, if the target is distinguished from the background due to motion, the optical flow method can be used to analyze the changes in pixel intensity to estimate the initial search box. Optical flow is a method for measuring the motion of objects in an image. It assumes that the surface and texture structure of the object are stable over a short period of time. The steps to estimate the initial bounding box using the optical flow method are as follows:

[0059] First, an optical flow algorithm is used between each frame of the video sequence and its next consecutive frame to obtain the velocity vector of each pixel in the first frame.

[0060] It should be understood that there are many commonly used optical flow algorithms. This embodiment uses the Lucas-Kanade algorithm. The implementer may also use other optical flow algorithms such as Horn-Schunck to calculate the movement of pixels between each frame and the next frame. This embodiment does not impose any special restrictions.

[0061] The velocity vector of each pixel in each frame can be represented by (u, v), where u and v are the distances moved by each pixel in each frame in the horizontal direction and the vertical direction compared to the next frame.

[0062] The average velocity vector of all pixels in each frame is formed by the average of the components of the velocity vector of all pixels in each frame in the horizontal direction and the average of the components of the velocity vector of all pixels in each frame, and the modulus of the average velocity vector is used as the motion threshold of all pixels in each frame;

[0063] The pixel points whose modulus of the velocity vector in each frame is greater than or equal to the motion threshold are the motion pixel points of each frame; the pixel points whose modulus of the velocity vector in each frame is less than the motion threshold are the stationary pixel points of each frame.

[0064] Analyze the modulus of the velocity vector of each pixel in each frame; the pixels whose velocity modulus is greater than or equal to the motion threshold in each frame are moving pixels; the pixels whose velocity modulus is less than the motion threshold in each frame are stationary pixels.

[0065] It should be noted that, for the last frame in the video sequence, the moving pixels and the stationary pixels are no longer obtained.

[0066] The minimum bounding rectangle of all moving pixels in the first frame is used as the initial bounding box of each frame in the video sequence.

[0067] At this point, the initial bounding box of each frame in the video sequence is obtained.

[0068] S202: Obtain an initial search area based on the initial bounding box of each frame.

[0069] Taking the center of each frame in the video sequence as the center of the initial search area, expanding the initial bounding box, and using the expanded range as the initial search area of ​​each frame in the video sequence, it can be ensured that the initial search area contains not only the target but also the area where the target may move, which is used to provide sufficient context information and help the tracking algorithm to accurately locate and track the target in subsequent frames. The method for determining the initial search area is specifically as follows:

[0070] The intersection of the diagonals of the initial bounding box of each frame in the video sequence is used as the center of the initial search area of ​​each frame in the video sequence; both ends of the diagonal of the initial bounding box are increased by a preset ratio, and the area enclosed by the bounding box after the diagonal increase is recorded as the initial search area of ​​each frame in the video sequence.

[0071] It should be noted that the value of the preset ratio is artificially set. In this embodiment, the value of the preset ratio is 20%. The implementer can set it according to the specific situation, and this embodiment does not impose any special restrictions.

[0072] At this point, the initial bounding box and initial search area of ​​each frame in the video sequence are obtained through step S201.

[0073] Further, when the target starts to move, the target bounding box of each frame is obtained according to the method of obtaining the initial search area box of each frame in the video sequence in step S201; further, an adaptive search area is obtained according to the change of pixel values ​​between the target bounding boxes of each frame.

[0074] S203: constructing a scale change factor for each frame according to the difference in the number of pixels between the target bounding box of each frame and the target bounding box of the previous frame.

[0075] When the target moves faster than the tracker, the target will move away from the tracker. If the search area remains unchanged, it may contain too many background pixels, thus interfering with the tracking results. On the contrary, when the target moves slower than the tracker, the target is moving closer to the tracker. If the search area is too small, the tracker may not be able to cover the new position of the target, resulting in tracking loss. Therefore, the search area needs to be expanded to ensure that the tracker can include the contextual information of the target and its surroundings.

[0076] By properly adjusting the scale factor, the tracker can locate the target more accurately, thereby improving the tracking accuracy. The specific process of determining the scale factor is:

[0077] (1) Construct a scale change factor for each frame based on the difference in the number of pixels between the target bounding box of each frame and the target bounding box of the previous frame.

[0078] In the process from the previous frame to each frame, the target's flight direction may be consistent with the seeker's flight direction. However, due to the difference between the target's moving speed and the seeker's moving speed, the target moves away from the seeker or approaches the seeker, which is manifested in the video as follows: when the target's moving speed is greater than the seeker's moving speed, the target accelerates away and the target bounding box in each frame becomes smaller; when the target's moving speed is less than the seeker's moving speed, the target accelerates towards the seeker and the target bounding box in each frame becomes larger.

[0079] Therefore, the faster the target moves, the larger the change range of the search area should be, so as to update the information in time and increase the accuracy of target tracking; on the contrary, the change range should be smaller to avoid excessive differences with the target bounding box, which will cause unnecessary calculations. To this end, the search area is dynamically adjusted by analyzing the scale change factor of each frame.

[0080] Scale change factor for each frame: The scale change factor for each frame is constructed based on the difference in the number of pixels between the target bounding box of each frame and the target bounding box of the previous frame. Specifically, it is:

[0081] The difference between the number of pixels in the target bounding box of each frame and the number of pixels in the target bounding box of the previous frame is recorded as the scale change factor of each frame.

[0082] It should be noted that there are many methods for measuring differences. In this embodiment, the scale change factor of each frame is the difference between the number of pixels in the initial search area of ​​each frame and the number of pixels in the target bounding box of the previous frame as a method of measuring the difference. The implementer may also use a ratio or other methods of measuring the difference to determine the difference between each frame and the previous frame according to the specific situation.

[0083] When the scale change factor is negative, it means that the number of pixels in the target bounding box of the previous frame is less than the number of pixels in the target bounding box of each frame, which further indicates that the target is getting bigger. At this time, the search area should be increased to ensure that the target bounding box can be completely covered. On the contrary, when the scale change factor is positive, the search area should be reduced to reduce the context information and speed up the calculation.

[0084] (2) Construct a pixel difference index for each frame based on the pixel value distribution of the area between the initial search area and the target bounding box of each frame.

[0085] In object tracking, contextual information is very important to keep tracking. If the search area is too large, it may contain too many background pixels, which will interfere with the tracking results. Similarly, if the search area is too small, part of the target may be missed, resulting in tracking loss.

[0086] Analyzing the pixel difference index can help the KCF algorithm avoid such mismatching of contextual information and ensure the accuracy of the tracking results.

[0087] Pixel difference index of each frame: The pixel difference index of each frame is calculated according to the scale change factor and the characteristics of the pixels between the initial search area and the target bounding box (i.e., the pixels providing context information), specifically:

[0088] Determine the mean of all pixel values ​​in the area between the initial search area and the target bounding box in each frame, and record it as the first mean of each frame; analyze the difference between each pixel value in the area between the initial search area and the target bounding box in each frame and the first mean, and record it as the pixel difference of each pixel point in the area between the initial search area and the target bounding box in each frame;

[0089] It should be noted that there are many ways to measure differences, and the methods for measuring differences are not limited to methods such as taking the absolute value of the difference and the ratio. In this embodiment, the pixel difference is obtained by taking the absolute value of the difference between each pixel value in the area between the initial search area and the target boundary box in each frame and the first mean. The implementer can select an appropriate difference measurement method based on the specific application scenario, data characteristics and tracking target requirements.

[0090] A pixel difference index of each frame is determined based on the pixel difference, where the pixel difference index of each frame is the product of the pixel differences of all pixel points in the area between the search area and the target bounding box in each frame.

[0091] It should be noted that since the video is not grayscaled, each frame is an RGB image, and the calculation of pixel values ​​requires calculating the pixel values ​​of the red, blue, and green channels separately.

[0092] There are three cases where the search area needs to be increased. The first case is that the initial search area in each frame is already smaller than the target bounding box, and most of the pixels between them are target pixels. It can be understood that the larger the pixel difference index is, the smaller the pixel ratio of the target is.

[0093] The second case is that the search area in each frame is larger than the target bounding box. At this time, most of the pixels between them are background pixels. It can be understood that the larger the pixel difference index is, the larger the pixel ratio of the target is.

[0094] The third case is that the initial search area overlaps with the target bounding box. In this case, the search area is adjusted in the same manner as that of obtaining the initial search area in step S201.

[0095] When the search area needs to be reduced, the search area must be larger than the target bounding box, because the search area cannot be reduced further when it is smaller than the target bounding box. At this time, most of the pixels between them are background pixels. The larger the pixel difference index, the larger the pixel ratio of the target.

[0096] Based on the above analysis, the pixel difference index can characterize the difference in pixel values ​​between the initial search area and the target bounding box. The larger the pixel difference index, the greater the difference. The search area should be adjusted to reduce the difference and avoid having too many pixels belonging to the target among the pixels representing context information, which would interfere with the tracking results.

[0097] (3) Determine a scale factor for each frame based on the scale change factor and the pixel difference index.

[0098] Scale factor of each frame: The scale factor of each frame is determined based on the scale change factor and the pixel difference index of each frame. The scale factor of each frame is the result of the fusion of the scale change factor and the pixel difference index.

[0099] It should be understood that fusion refers to combining two or more indicators by adding or multiplying them together in order to obtain a comprehensive indicator, thereby more comprehensively and accurately evaluating a phenomenon or problem. This fusion method is not limited to simple arithmetic operations, but can also include more complex statistical models and analysis methods. The implementer can choose according to the specific situation, and this embodiment does not impose any special restrictions.

[0100] Preferably, as an embodiment of the present application, the scale factor of each frame is the product of the scale change factor of each frame and the pixel difference index.

[0101] Preferably, as another embodiment of the present application, the scale factor of each frame is an exponential function value with a natural constant as the base and a sum of the scale change factor of each frame and a pixel difference index as the independent variable.

[0102] The larger the scale factor, the faster the target moves, the greater the difference in pixel values ​​between the initial search area and the target bounding box, and the larger the range of the search area should be to facilitate timely updating of information; conversely, the larger the scale factor, the slower the target moves, the smaller the difference in pixel values ​​between the initial search area and the target bounding box, and the search area should be reduced to avoid excessive differences with the target bounding box, resulting in unnecessary calculations.

[0103] S204: Construct an occlusion impact index for each frame based on the difference between the moving pixels and the stationary pixels in the target bounding box of each frame and the previous frame. The target may be occluded during its movement, for example, an aircraft may be occluded by clouds during flight. At this time, the seeker may lose the target to be tracked, and it is necessary to expand the search area and add more context information to help the algorithm recover from interference and relocate the target.

[0104] Occlusion impact index of each frame: According to the step of obtaining the initial bounding box in step S201, if the target is occluded, the proportion of moving pixels in the initial bounding box will decrease, that is, the static pixels in the initial bounding box will increase. The occlusion impact index is calculated based on the proportion, specifically:

[0105] Determine the ratio of the number of moving pixels to the number of stationary pixels in the target bounding box of each frame, and record it as the occlusion impact ratio of each frame;

[0106] The occlusion impact ratio of each frame is determined according to the occlusion impact ratio of each frame to the previous frame; the occlusion impact index of each frame is determined based on the occlusion impact ratio of each frame to the previous frame, and the occlusion impact index of each frame is the difference between the occlusion impact ratio of each frame and the occlusion impact ratio of the previous frame.

[0107] It should be noted that there are many ways to measure differences, and the methods of measuring differences are not limited to taking absolute values ​​of differences, ratios, etc. Implementers can choose appropriate difference measurement methods based on specific application scenarios, data characteristics, and tracking target requirements.

[0108] Preferably, as an embodiment of the present application, the occlusion impact index of each frame is the absolute value of the difference between the occlusion impact ratio of each frame and the occlusion impact ratio of the previous frame.

[0109] The larger the occlusion impact index, the more serious the target is occluded, and the larger the search area should be to provide more context information; the smaller the occlusion impact index, the less the target is occluded, and the search area can be narrowed.

[0110] S205: Determine a scale adjustment coefficient for each frame based on the scale factor and the occlusion impact index.

[0111] Scale adjustment coefficient of each frame: the scale adjustment coefficient of each frame is determined based on the scale factor and the occlusion influence index, and the scale adjustment coefficient of each frame is the normalized result of the product of the scale factor and the occlusion influence index of each frame.

[0112] The larger the scale adjustment coefficient of each frame, the faster the target moves, the more severely the target is blocked, and the larger the search area should be; conversely, the smaller the scale adjustment coefficient of each frame, the slower the target moves, the less blocked the target is, and the smaller the search area can be. Based on the scale adjustment coefficient, the size of the dynamic search area of ​​each frame is determined in combination with the initial search area, specifically:

[0113] The length of the dynamic search area = (1 + scale adjustment coefficient) × the length of the initial search area;

[0114] The width of the dynamic search area = (1 + scale adjustment coefficient) × the width of the initial search area;

[0115] It should be noted that the center of the search area, ie, the intersection of the diagonals of the original search area, is not changed during the adjustment process.

[0116] Preferably, as an embodiment of the present application, the scale adjustment coefficient extraction schematic diagram is as follows: Figure 2 shown.

[0117] Step S3: constructing a position factor for each frame according to the offset of the target bounding box of each frame in the horizontal direction and the vertical direction, and determining an adaptive search area for each frame in combination with the dynamic search area.

[0118] The target will move in the video sequence. In order for the tracker to accurately track the new position of the target, the position of the search area needs to be dynamically adjusted to ensure that the search area always contains the target and reduces interference with the background. If the search area cannot be adjusted in time to adapt to the movement of the target, it may cause tracking loss. Therefore, in order to adjust the search area in time and avoid tracking loss, the position of the search area is adjusted by analyzing the position factor.

[0119] Position factor of each frame: The position factor of each frame is constructed according to the horizontal and vertical offsets of the target bounding box of each frame:

[0120] Determine the distance between the left boundary of the target bounding box and the left boundary of the initial search area for each frame, record it as the left distance of each frame, determine the distance between the right boundary of the target bounding box and the right boundary of the initial search area, record it as the right distance of each frame. Take the direction of the minimum value between the left distance and the right distance of each frame as the offset direction of each frame.

[0121] Taking half of the sum of the left distance and the right distance as the offset distance of each frame, and combining the offset direction of each frame to obtain the horizontal displacement of each frame;

[0122] According to the method for obtaining the horizontal displacement of each frame, the vertical displacement of each frame is obtained;

[0123] The vector sum of the horizontal displacement and the vertical displacement of each frame is used as the position factor of each frame.

[0124] Furthermore, the search area is moved according to the position factor, and the center of the dynamic search area of ​​each frame is moved according to the position factor. The moving direction is the direction of the position factor, and the moving distance is the size of the position factor. The dynamic search area after the move is obtained and recorded as the adaptive search area of ​​each frame.

[0125] Step S4: Apply the KCF algorithm to the search area of ​​each frame of the video sequence to track the target.

[0126] According to steps S2 and S3, the adaptive search area of ​​each frame in the video sequence is obtained, the moving pixels and the stationary pixels are distinguished by the optical flow method, the bounding box of the target is obtained, and then the dynamic search area of ​​each frame is obtained by the change of the bounding box of the moving target, and the video sequence and the adaptive search area are used as the input of the KCF algorithm, and the response map and tracking status of the target are output. The KCF algorithm belongs to the well-known technology of target tracking, and the specific process is not repeated here.

[0127] At this point, the tracking of the moving target is completed based on the adaptive adjustment of the search area.

[0128] Preferably, as an embodiment of the present application, the target tracking process diagram of the image target simulator is as follows: Figure 3 shown.

[0129] Based on the same inventive concept as the above method, an embodiment of the present application also provides a target tracking system for an image target simulator, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the steps of any one of the above-mentioned target tracking methods for an image target simulator are implemented.

[0130] It should be noted that the above sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0131] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0132] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application should be included in the protection scope of the present application.

Claims

1. A target tracking method for an image target simulator, characterized in that: The method comprises the following steps: Obtain key frames in the motion video of the image target to form a video sequence; According to the distribution difference of corresponding pixels between each frame and the next frame in the video sequence, the moving pixels and the stationary pixels of each frame are obtained, the minimum enclosing rectangle of all moving pixels in the first frame is used as the initial bounding box of each frame in the video sequence, and the initial search area of ​​each frame is obtained; The minimum bounding rectangle of all moving pixels in each frame except the first frame is used as the target bounding box of each frame; Constructing a scale change factor for each frame according to the difference in the number of pixels between the target bounding box of each frame and the target bounding box of the previous frame; constructing a pixel difference index for each frame according to the distribution of pixel values ​​in the area between the initial search area and the target bounding box of each frame, and constructing a scale factor for each frame in combination with the scale change factor; Constructing an occlusion impact index for each frame according to a difference in the number of moving pixels and a difference in the number of stationary pixels in a target bounding box between each frame and a previous frame, constructing a scale adjustment coefficient for each frame in combination with the scale factor, and determining a dynamic search area for each frame based on the scale adjustment coefficient and the initial search area; Constructing a position factor of each frame according to the offset of the target bounding box of each frame in the horizontal direction and the vertical direction, determining an adaptive search area of ​​each frame in combination with the dynamic search area, and tracking the target using a target tracking algorithm; The scale change factor of each frame is the difference between the number of pixels in the target bounding box of each frame and the number of pixels in the target bounding box of the previous frame; The process of constructing the occlusion impact index of each frame is as follows: The ratio of the number of moving pixels to the number of stationary pixels in the target bounding box of each frame is analyzed and recorded as the occlusion impact ratio of each frame. The occlusion impact index of each frame is determined based on the occlusion impact ratio of each frame and its previous frame. The occlusion impact index of each frame is the difference between the occlusion impact ratio of each frame and its previous frame.

2. A target tracking method for an image target simulator as claimed in claim 1, characterized in that: The method for acquiring the moving pixels and the stationary pixels of each frame is as follows: The optical flow algorithm is used between each frame and the next frame in the video sequence to obtain the velocity vector of each pixel in each frame; the modulus value of the velocity vector of each pixel in each frame is analyzed; The average velocity vector of all pixels in each frame is formed by the average of the components of the velocity vector of all pixels in each frame in the horizontal direction and the average of the components of the velocity vector of all pixels in each frame, and the modulus of the average velocity vector is used as the motion threshold of all pixels in each frame; The pixel points whose velocity vector modulus in each frame is greater than or equal to the motion threshold are the motion pixel points of the first frame; the pixel points whose velocity vector modulus in each frame is less than the motion threshold are the stationary pixel points of each frame.

3. A target tracking method for an image target simulator as claimed in claim 1, characterized in that: The method for obtaining the initial search area of ​​each frame is: The two ends of the diagonal of the initial bounding box are increased by a preset ratio respectively, and the area surrounded by the bounding box after the diagonal increase is recorded as the initial search area of ​​each frame image in the video sequence.

4. A target tracking method for an image target simulator as claimed in claim 1, characterized in that: The construction process of the pixel difference index of each frame is as follows: Determine the mean of all pixel values ​​in the area between the initial search area and the target bounding box in each frame, recorded as the first mean of each frame; analyze the difference between each pixel value in the area between the initial search area and the target bounding box in each frame and the first mean, recorded as the pixel difference of each pixel point in the area between the initial search area and the target bounding box in each frame; A pixel difference index of each frame is determined based on the pixel difference, where the pixel difference index of each frame is the product of the pixel differences of all pixel points in the area between the search area and the target bounding box in each frame.

5. A target tracking method for an image target simulator as claimed in claim 1, characterized in that: The scale factor of each frame is a fusion result of the scale change factor of each frame and the pixel difference index.

6. A target tracking method for an image target simulator as claimed in claim 1, characterized in that: The step of constructing a scale adjustment coefficient for each frame and determining a dynamic search area for each frame based on the scale adjustment coefficient and the initial search area includes: Determining a scale adjustment coefficient for each frame based on the scale factor and the occlusion impact index, wherein the scale adjustment coefficient for each frame is a normalized result of a product of the scale factor and the occlusion impact index for each frame; The length of the dynamic search area = (1 + scale adjustment coefficient) × the length of the initial search area; The width of the dynamic search area = (1 + scale adjustment coefficient) × the width of the initial search area.

7. A target tracking method for an image target simulator as claimed in claim 1, characterized in that: The step of constructing the position factor of each frame and determining the adaptive search area of ​​each frame in combination with the dynamic search area includes: Determine the distance between the left boundary of the target bounding box of each frame and the left boundary of the initial search area, recorded as the left distance of each frame; obtain the right distance of each frame according to the left distance determination method; take the direction of the minimum value between the left distance and the right distance of each frame as the offset direction of each frame; take half of the sum of the left distance and the right distance as the offset distance of each frame, and obtain the horizontal displacement of each frame in combination with the offset direction of each frame; According to the method for obtaining the horizontal displacement of each frame, the vertical displacement of each frame is obtained based on the distance between the left boundary of the target bounding box of each frame and the upper and lower boundaries of the initial search area; The vector sum of the horizontal displacement and the vertical displacement of each frame is used as the position factor of each frame; The center of the dynamic search area of ​​each frame is moved according to the position factor to obtain the adaptive search area of ​​each frame.

8. A target tracking system for an image target simulator, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images