A real-time judgment method and device for visual tracking accuracy

Abstract

The present invention provides a method and device for real-time determination of visual tracking accuracy, the method comprising obtaining an image I obtained by a visual tracking algorithm i , calculate image I i The clarity of the image I i Image brightness; Get image I i The signal-to-noise ratio and clutter intensity of image I i Global object saliency of I; Recognize image I i The tracking target in the image is obtained, the area and rotation angle of the tracking target are obtained, and the deformation degree of the tracking target is calculated; the position change degree of the tracking target is determined; and the confidence of the tracking target identified by the visual tracking algorithm is determined in an information fusion manner. The method can obtain the confidence of the tracking algorithm accuracy through online real-time judgment of the tracking accuracy under the premise that the actual position and scale of the target are unknown.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to a real-time judgment method and device for visual tracking accuracy. Background Art

[0002] Visual tracking is to find the target of interest from the image sequence obtained by the camera or video, and mark the position and size of the target. In some application fields, the position of the marked target is needed to guide other motion mechanisms to achieve the function of pointing to or following the target. In these applications, the accuracy of tracking the target position, that is, tracking accuracy, is particularly important.

[0003] Improving tracking accuracy has always been the goal of various image processing methods. However, no method can currently adapt to various application scenarios, and there is a general problem of low adaptability. The more serious problem is that the visual tracking method itself does not know that its accuracy is not accurate enough or has deteriorated, causing the motion mechanism to deviate from the actual target.

[0004] The existing methods for judging the accuracy of visual tracking all give a number of image sequences, mark the position and scale of the target in each image, take the target position and scale of the first image in the image sequence as known quantities, inform the visual tracking algorithm, and then compare the target position and scale obtained by the visual tracking algorithm in each subsequent image with the marked target position and scale, so as to calculate the accuracy of the visual tracking algorithm or other indicators. This judgment method is only applicable to the case where the target position and scale are known, and it is impossible to judge the accuracy of visual tracking in real time while acquiring images. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a real-time judgment method and device for visual tracking accuracy. The method and device are used to solve the technical problem that the accuracy judgment of the visual tracking algorithm in the prior art is only applicable to the situation where the target position and scale are known, and the accuracy of visual tracking cannot be judged in real time while acquiring the image.

[0006] According to a first aspect of the present invention, a method for real-time determination of visual tracking accuracy is provided, the method comprising the following steps:

[0007] Step S101: Obtaining an image I obtained by a visual tracking algorithm i , calculate image I i clarity;

[0008] Step S102: Obtain image I i Image brightness;

[0009] Step S103: Obtaining image I iThe signal-to-noise ratio and clutter intensity of image I i The global target saliency of

[0010] Step S104: Identify image I i The tracking target in the tracking object is obtained, and the area and rotation angle of the tracking target are obtained, and the deformation degree of the tracking target is calculated;

[0011] Step S105: determining the degree of position change of the tracking target;

[0012] Step S106: Determine the confidence level of the tracking target identified by the visual tracking algorithm in an information fusion manner.

[0013] According to a second aspect of the present invention, a real-time judgment device for visual tracking accuracy is provided, the device comprising:

[0014] Image acquisition module: configured to acquire images acquired by the visual tracking algorithm I i , calculate image I i clarity;

[0015] Brightness acquisition module: configured to acquire image I i Image brightness;

[0016] Saliency acquisition module: configured to acquire image I i The signal-to-noise ratio and clutter intensity of image I i The global target saliency of

[0017] Deformation degree calculation module: configured to recognize image I i The tracking target in the tracking object is obtained, and the area and rotation angle of the tracking target are obtained, and the deformation degree of the tracking target is calculated;

[0018] A position change determination module: configured to determine a position change degree of the tracking target;

[0019] Fusion module: configured to determine the confidence of the tracking target identified by the visual tracking algorithm in an information fusion manner.

[0020] According to a third aspect of the present invention, there is provided a real-time judgment system for visual tracking accuracy, comprising:

[0021] A processor, which is used to execute multiple instructions;

[0022] A memory for storing a plurality of instructions;

[0023] The plurality of instructions are used to be stored by the memory, and loaded and executed by the processor to implement the real-time judgment method of visual tracking accuracy as described above.

[0024] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the real-time judgment method for visual tracking accuracy as described above.

[0025] According to the above scheme of the present invention, under the premise that the actual position and scale of the target are unknown, the confidence level of the tracking algorithm accuracy can be obtained through online real-time judgment of the tracking accuracy, and then it can be decided how to use the target position and scale obtained by the visual tracking algorithm to avoid the motion mechanism being guided by the wrong target position and deviating from the actual target.

[0026] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention, and the present invention is described by providing the following accompanying drawings. In the accompanying drawings:

[0028] Figure 1 A flow chart of a method for real-time determination of visual tracking accuracy according to an embodiment of the present invention;

[0029] Figure 2 A structural block diagram of a real-time judgment device for visual tracking accuracy according to an embodiment of the present invention; DETAILED DESCRIPTION

[0030] First combine Figure 1 A method for real-time determination of visual tracking accuracy according to an embodiment of the present invention is described, and the method comprises the following steps:

[0031] Step S101: Obtaining an image I obtained by a visual tracking algorithm i , calculate image I i clarity;

[0032] Step S102: Obtain image I i Image brightness;

[0033] Step S103: Obtaining image I i The signal-to-noise ratio and clutter intensity of image I i The global target saliency of

[0034] Step S104: Identify image I i The tracking target in the tracking object is obtained, and the area and rotation angle of the tracking target are obtained, and the deformation degree of the tracking target is calculated;

[0035] Step S105: determining the degree of position change of the tracking target;

[0036] Step S106: Determine the confidence level of the tracking target identified by the visual tracking algorithm in an information fusion manner.

[0037] Step S101: Obtaining an image I acquired by a visual tracking algorithm i , calculate image I i clarity, including:

[0038] The image I i The clarity of D i (f) is:

[0039]

[0040] Among them, N i For image I i The number of pixels, f i (x,y) is image I i The gray value of the pixel (x, y), D i (f) is image I i clarity.

[0041] In this embodiment, there are many ways to calculate image clarity, and the commonly used ones include the Tenengrad gradient method and the Laplacian gradient method.

[0042] In this embodiment, the image clarity is calculated, and the factor considered is to determine whether the current image is blurred due to camera focus adjustment or platform movement, which affects the tracking stability. The higher the clarity, the higher the tracking reliability.

[0043] Step S102: Acquiring image I i Image brightness, including:

[0044] Calculate the pixel average value and brightness change of the image Ii, the pixel average value B(f i )for

[0045]

[0046] Image I i Brightness change ΔB i for

[0047]

[0048] Among them, B(f i-1 ) is the pixel average value of the previous frame image of image Ii.

[0049] In this embodiment, by calculating the brightness of the image, the change trend of the image brightness can be obtained. When the entire field of view is blocked or the illumination changes significantly, the brightness changes suddenly, thereby reducing the tracking confidence.

[0050] Step S103: Acquiring image I i The signal-to-noise ratio and clutter intensity of image I i The global target saliency of

[0051] Image I i Signal-to-noise ratio SNR i for:

[0052] SNR i =I i-max / σ Bi

[0053] Among them, I i-max For image I i The maximum background gray value of the tracking target, σ Bi For image I i Standard deviation of the background grayscale;

[0054] The clutter intensity is given by image I i The clutter intensity is determined by the number of signals similar to the target signal in the background and the degree of similarity. In this embodiment, the edge probability scale is used to characterize the clutter intensity:

[0055]

[0056] Where T is the preset human eye observation threshold. i The division is M i units, is the number of edge points in the jth unit whose absolute value of the difference between the grayscale value and the target grayscale mean is less than the human eye observation threshold T. i The larger the value, the better the image I i The more serious the noise.

[0057] The image I i The global target saliency S i for:

[0058]

[0059] In this embodiment, the global target saliency is used to determine the similarity between the entire image and the tracking target. The more salient the tracking target is in the image, the higher the tracking confidence.

[0060] Step S104: Identify image I iThe tracking target in , obtains the area and rotation angle of the tracking target, and calculates the deformation degree of the tracking target, wherein:

[0061] Obtaining the length and width of the target determined by the target tracking algorithm currently performing target tracking, wherein the area of ​​the tracked target is the product of the length and the width of the target;

[0062] Get Image I i The tracking target is relative to image I i The previous frame image I i-1 The rotation angle θ of the tracking target described in;

[0063] The deformation degree T of the tracking target i for:

[0064]

[0065] Among them, A i For image I i The area of ​​the target being tracked, A i-1 For image I i The previous frame image I i-1 The area of ​​the target being tracked.

[0066] The implementation method of this embodiment is: using a multi-angle template matching method to estimate the rotation angle of the tracking target, specifically including:

[0067] Image I i The previous frame image I i-1 The angle of the target to be tracked is 0 degrees, a 0 degree template is established, and the 0 degree template is rotated at intervals of 10 degrees to obtain 35 angle templates of 10 degrees, 20 degrees, 30 degrees, ... 350 degrees, and the 0 degree template and the 35 templates obtained by the above rotation are sequentially compared with the image I i The tracked target is matched and 36 matching degrees are obtained. The template with the highest matching degree corresponds to the rotation angle.

[0068] In this embodiment, the changes in the tracking target area and the rotation angle are calculated to characterize the deformation degree of the tracking target. The greater the deformation, the lower the tracking confidence.

[0069] The step S105: determining the position change degree of the tracking target, wherein:

[0070] Obtain the angle change Δ of the tracked target in the horizontal direction X and the vertical direction Y relative to the previous frame image xi and Δ yi , and obtain the position change degree R of the tracking target i :

[0071]

[0072] In this embodiment, when the camera is stationary relative to the ground, the target position obtained by the tracking algorithm can be used directly; when the camera moves relative to the ground, it is necessary to know the camera's motion information and use it together with the target position obtained by the tracking algorithm to represent the target position. In this embodiment, the angle change Δ can also be obtained by the tracking algorithm and the motion mechanism. xi and Δ yi .

[0073] The step S106: determining the confidence of the tracking target identified by the visual tracking algorithm in an information fusion manner, comprises:

[0074] The weight is determined by the tracking algorithm in a weighted summation manner, and the confidence C of the tracking target is i for:

[0075] C i =a·D i (f)+b·(1-ΔB i )+c·S i +d·(1-T i )+e·(1-R i )

[0076] Among them, a, b, c, d, and e are all weights.

[0077] In this embodiment, a weighted summation method is adopted, and the weight is adjusted according to the characteristics of the tracking algorithm. For example, if the algorithm can adapt to changes in illumination but is greatly affected by target deformation, the weight of brightness change is reduced and the weight of target deformation is increased. Through the above information fusion method, the tracking deviation confidence of the tracked target is obtained.

[0078] Furthermore, the present invention also includes step S107: determining whether to guide the motion mechanism based on the confidence of the tracking target.

[0079] The motion mechanism is a servo mechanism that corrects the pointing angle of an imaging device (such as an infrared thermal imager) according to the tracking deviation, so that the imaging device always points to the target direction, including but not limited to an optoelectronic turntable and an airborne pod.

[0080] In this embodiment, the confidence level C of the tracking target can be i , C i Multiply by the guide amount as the control system input of the motion mechanism. A predetermined threshold can also be set. i When the value is less than the threshold value, the tracking algorithm is no longer used to guide the motion mechanism, so as to prevent the motion mechanism from following the wrong target.

[0081] Under the premise that the actual position and scale of the target are unknown, this embodiment can obtain the confidence level of the accuracy of the tracking algorithm through online real-time judgment of the tracking accuracy, and then decide how to use the target position and scale obtained by the visual tracking algorithm to avoid the motion mechanism being guided by the wrong target position and deviating from the actual target.

[0082] The embodiment of the present invention further provides a real-time judgment device for visual tracking accuracy, such as Figure 2 As shown, the device comprises:

[0083] Image acquisition module: configured to acquire images acquired by the visual tracking algorithm I i , calculate image I i clarity;

[0084] Brightness acquisition module: configured to acquire image I i Image brightness;

[0085] Saliency acquisition module: configured to acquire image I i The signal-to-noise ratio and clutter intensity of image I i The global target saliency of

[0086] Deformation degree calculation module: configured to recognize image I i The tracking target in the tracking object is obtained, and the area and rotation angle of the tracking target are obtained, and the deformation degree of the tracking target is calculated;

[0087] A position change determination module: configured to determine a position change degree of the tracking target;

[0088] Fusion module: configured to determine the confidence of the tracking target identified by the visual tracking algorithm in an information fusion manner.

[0089] The embodiment of the present invention further provides a real-time judgment system for visual tracking accuracy, including:

[0090] A processor, which is used to execute multiple instructions;

[0091] A memory for storing a plurality of instructions;

[0092] The plurality of instructions are used to be stored by the memory, and loaded and executed by the processor to implement the real-time judgment method of visual tracking accuracy as described above.

[0093] The embodiment of the present invention further provides a computer-readable storage medium, in which a plurality of instructions are stored; the plurality of instructions are used for a processor to load and execute the real-time judgment method for visual tracking accuracy as described above.

[0094] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0095] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0096] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0097] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0098] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a physical server, or a network cloud server, etc., which needs to be installed with Windows or Windows Server operating system) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (RandomAccess Memory, RAM), disk or optical disk and other media that can store program codes.

[0099] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A real-time judgment method for visual tracking accuracy, characterized in that: The method comprises the following steps: Step S101: Obtaining an image I obtained by a visual tracking algorithm i , calculate image I i clarity; Step S102: Obtain image I i image brightness; Step S103: Obtaining image I i The signal-to-noise ratio and clutter intensity of image I i The global target saliency of Step S104: Identify image I i The tracking target in the tracking object is obtained, and the area and rotation angle of the tracking target are obtained, and the deformation degree of the tracking target is calculated; Step S105: determining the degree of position change of the tracking target; Step S106: determining the confidence of the tracking target identified by the visual tracking algorithm in an information fusion manner; Step S101: Obtaining an image I acquired by a visual tracking algorithm i , calculate image I i Clarity, including: The image I i Clarity D i (f) is: Among them, N i For image I i The number of pixels, f i (x,y) is image I i The gray value of the pixel (x, y), D i (f) is image I i clarity; Step S102: Acquiring image I i Image brightness, including: Calculate the image I i The pixel average value and brightness change of image Ii, the pixel average value B(f i )for Image I i Brightness change ΔB i for Among them, B(f i-1 ) is image I i The average pixel value of the previous frame image; Step S103: Acquiring image I i The signal-to-noise ratio and clutter intensity of image I i The global target saliency of Image I i Signal-to-noise ratio SNR i for: SNR i =I i-max / σ Bi Among them, I i-max For image I i The maximum background gray value of the tracking target, σ Bi For image I i Standard deviation of the background grayscale; Clutter intensity Where T is the preset human eye observation threshold. i Split into M i units, is the number of edge points in the jth unit whose absolute value of the difference between the grayscale value and the target grayscale mean is less than the human eye observation threshold T; the image I i The global target saliency S i for: Step S104: Identify image I i The tracking target in , obtains the area and rotation angle of the tracking target, and calculates the deformation degree of the tracking target, wherein: Obtaining the length and width of the target determined by the target tracking algorithm currently performing target tracking, wherein the area of ​​the tracked target is the product of the length and the width of the target; Get Image I i The tracking target is relative to image I i The previous frame image I i-1 The rotation angle θ of the tracking target described in; The deformation degree T of the tracking target i for: Among them, A i For image I i The area of ​​the target being tracked, A i-1 For image I i The previous frame image I i-1 The area of ​​the target being tracked.

2. The real-time determination method of visual tracking accuracy according to claim 1, characterized in that: The step S105: determining the position change degree of the tracking target, wherein: Obtain the angle change Δ of the tracking target in the horizontal direction X and the vertical direction Y xi and Δ yi , and obtain the position change degree R of the tracking target i :

3. The real-time determination method of visual tracking accuracy as claimed in claim 2, characterized in that: The step S106: determining the confidence of the tracking target identified by the visual tracking algorithm in an information fusion manner, wherein: The confidence C of the tracked target i For: C i =a·D i (f)+b·(1-ΔB i )+c·S i +d·(1-T i )+e·(1-R i ) Among them, a, b, c, d, and e are all weights.

4. A real-time judgment device for visual tracking accuracy, characterized in that: The device comprises: Image acquisition module: configured to acquire images acquired by the visual tracking algorithm I i , calculate image I i clarity; Brightness acquisition module: configured to acquire image I i Image brightness; Saliency acquisition module: configured to acquire image I i The signal-to-noise ratio and clutter intensity of image I i The global target saliency of Deformation degree calculation module: configured to recognize image I i The tracking target in the tracking object is obtained, and the area and rotation angle of the tracking target are obtained, and the deformation degree of the tracking target is calculated; A position change determination module: configured to determine a position change degree of the tracking target; A fusion module: configured to determine the confidence of the tracking target identified by the visual tracking algorithm in an information fusion manner; The image acquisition module: acquires the image I acquired by the visual tracking algorithm i , calculate image I i Clarity, including: The image I i Clarity D i (f) is: Among them, N i For image I i The number of pixels, f i (x,y) is image I i The gray value of the pixel (x, y), D i (f) is image I i clarity; The brightness acquisition module: acquires image I i Image brightness, including: Calculate the image I i The pixel average value and brightness change of image Ii, the pixel average value B(f i )for Image I i Brightness change ΔB i for Among them, B(f i-1 ) is image I i The average pixel value of the previous frame image; The saliency acquisition module: acquires image I i The signal-to-noise ratio and clutter intensity of image I i The global target saliency of Image I i Signal-to-noise ratio SNR i for: SNR i =I i-max / σ Bi Among them, I i-max For image I i The maximum background gray value of the tracking target, σ Bi For image I i Standard deviation of the background grayscale; Clutter intensity Where T is the preset human eye observation threshold. i Split into M i units, is the number of edge points in the jth unit whose absolute value of the difference between the grayscale value and the target grayscale mean is less than the human eye observation threshold T; the image I i The global target saliency S i for: The deformation degree calculation module: recognizes image I i The tracking target in , obtains the area and rotation angle of the tracking target, and calculates the deformation degree of the tracking target, wherein: Obtaining the length and width of the target determined by the target tracking algorithm currently performing target tracking, wherein the area of ​​the tracked target is the product of the length and the width of the target; Get Image I i The tracking target is relative to image I i The previous frame image I i-1 The rotation angle θ of the tracking target described in; The deformation degree T of the tracking target i for: Among them, A i For image I i The area of ​​the target being tracked, A i-1 For image I i The previous frame image I i-1 The area of ​​the target being tracked.

5. A computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the real-time judgment method for visual tracking accuracy as described in any one of claims 1 to 3.

Images