Method, device, equipment and medium for tracking small infrared targets under complex background

Abstract

The present application belongs to the field of target tracking technology, and relates to methods, devices, equipment and media for tracking infrared small targets under complex backgrounds. The method includes: obtaining the infrared image of the current frame, performing feature extraction, and obtaining a feature extraction image; calculating the fusion features of the grayscale image and the directional gradient histogram, and calculating the fusion feature response value; adding spatiotemporal information and calculating the confidence index; when the confidence index meets the preset range, updating the model and position of the infrared small target; when the confidence index does not meet the preset range, obtaining a significant difference image based on the infrared images of the current frame and the previous frame; selecting suspected targets and performing target screening. When the number of suspected targets in the screening result is not zero, updating the model and position of the infrared small target. The present application can accurately locate and track infrared small targets.

Description

Technical Field

[0001] The present application relates to the field of target tracking technology, and in particular to methods, devices, equipment and media for tracking infrared dim targets under complex backgrounds. Background Art

[0002] Target search and tracking, as a key aspect of machine vision, are widely used in many fields. Infrared images, due to their unique position among optical imagery, have attracted considerable attention. While infrared target search and tracking are becoming increasingly valuable, challenges remain, such as complex background interference, target scale variations, target submersion, and camera self-motion. In practical applications, infrared targets appear smaller and weaker at longer observation distances, making tracking prone to failure due to external noise. Building on classic visual tracking algorithms, researchers have developed numerous useful infrared target tracking methods.

[0003] In existing technologies, object tracking must be continuously localized in each frame based on the contextual information of the video or sequence. From the perspective of model construction, there are generally two types: generative models and discriminative models.

[0004] Generative models mean that a similar target model needs to be established in advance, but accurate modeling is a difficult problem. Common algorithms include: Kalman filtering, particle filtering, Mean shift algorithm, Cam shift algorithm, optical flow method, etc.

[0005] The discriminant model comprehensively utilizes the features of the target and background to train and update the classifier. Commonly used algorithms include: Struck, Tracking-Learning-Detection (TLD), Support Vector Machine, Random Forest, Correlation Filtering, etc.

[0006] However, existing methods are unable to demonstrate their advantages for weak infrared targets and are prone to losing the tracking of weak infrared targets in complex environments. Summary of the Invention

[0007] Based on this, it is necessary to provide an infrared dim small target tracking method, device, equipment and medium to address the above technical problems, which can accurately locate and track fast-moving, featureless infrared dim small targets.

[0008] The infrared dim target tracking method under complex background includes:

[0009] Obtain the current frame infrared image, perform feature extraction, and obtain a feature-extracted image;

[0010] Extracting the image based on the features, calculating the fusion features of the grayscale image and the directional gradient histogram, and calculating the fusion feature response value based on the fusion features;

[0011] According to the fused feature response value, adding spatiotemporal information, and calculating the confidence index;

[0012] When the confidence index meets the preset range, the infrared weak target in the image is extracted according to the feature, and the model and position are updated to achieve target tracking.

[0013] In one embodiment, when the confidence index does not meet a preset range, a significant difference image is obtained based on the current frame infrared image and the previous frame infrared image;

[0014] Selecting suspected targets based on the significant difference image and performing target screening to obtain screening results;

[0015] The target screening includes at least one of screening based on target movement distance, screening based on target image similarity, and screening based on target movement patterns;

[0016] When the number of suspected targets in the screening results is not zero, the suspected targets in the screening results are regarded as infrared weak targets, and the model and position are updated to achieve target tracking.

[0017] In one embodiment, screening based on target movement distance includes:

[0018]

[0019] in:

[0020]

[0021] Υ1=[x t -2d offset :x t +2d offset ,y t -2d offset :y t +2d offset ]

[0022]

[0023] In the formula, (x best ,y best ) is the screening result, I r (x, y) is the coordinate of the suspected target, d offset is the global environment moving distance, is the average movement distance of the target, Y1 and Y2 are two different movement intervals, I i (x i ,y i ) is the pixel grayscale value of the i-th frame infrared image at coordinate (x, y), For the i-1 frame, select the pixel value to be differentiated at the coordinate (x, y), (x t ,y t ) is the position of the suspected target in the previous frame of infrared image.

[0024] In one embodiment, screening based on target image similarity includes:

[0025] Calculate the target area similarity based on the current frame infrared image and the previous frame infrared image:

[0026] S=min(BC,NC)

[0027]

[0028]

[0029] Where S is the target region similarity, BC is the Bhattacharyya coefficient, NC is the normalized cross-correlation coefficient, and u is the number of blocks, ranging from 1 to n. is the local binary pattern histogram of the suspected target in the current frame infrared image, is the local binary pattern histogram of the suspected target in the previous frame of infrared image, corr is the cross-correlation operation, is the area where the suspected target is located in the infrared image of the i-th frame, σ i is the standard deviation of the object area in the i-th frame;

[0030] A screening result is obtained according to the target image similarity.

[0031] In one embodiment, screening based on target motion patterns includes:

[0032]

[0033] Where θ i is the target motion direction angle of the i-th frame, P i is the target position in the i-th frame, and δP is the relative movement distance of the suspected target in the previous infrared image frame.

[0034] In one embodiment, when the number of suspected targets in the screening result is zero and the type of target screening is not one, feature coding is introduced based on the preliminary screening result of one target screening to calculate the local background feature vector:

[0035] T b =[c1,c2,…,c 10 ]

[0036]

[0037]

[0038]

[0039] Where, T b is the local background feature vector, B j is the grayscale average value of the jth grid in the current frame infrared image, n is the number of background grids, I p is the grid where the suspected target is located, For I p The grayscale average value, m l For I p The lth gray value in the grayscale order from largest to smallest;

[0040] Based on the local background feature vector, the suspected target in the preliminary screening results is judged whether it meets the preset conditions:

[0041]

[0042] Where, is the feature vector of the local background around the true target in the i-th frame infrared image, is the feature vector of the true target in the i-th frame, is the reference background feature vector, k is the number of suspected targets, j indicates that the jth suspected target is the true target, λ is the update rate, ζ i-1 is the background feature difference measure, is the feature vector of the true target in the i-1th frame;

[0043] When the preset conditions are met, the suspected target in the preliminary screening results is regarded as a weak infrared target, and the model and position are updated to achieve target tracking;

[0044] When the preset conditions are not met, or the number of suspected targets in the screening result is zero and the type of target screening is one, target tracking for the next frame is performed.

[0045] In one embodiment, obtaining a current frame infrared image and performing feature extraction to obtain a feature-extracted image includes:

[0046]

[0047] Where G c (x, y) is the feature extraction image, n is the number of background grids, i(x, y) is the gray value of the center pixel in the current frame infrared image, B i is the grayscale average of the i-th grid in the current frame infrared image.

[0048] Infrared weak target tracking device under complex background, including:

[0049] An acquisition module is used to acquire the infrared image of the current frame, perform feature extraction, and obtain a feature-extracted image;

[0050] A fusion module is used to extract an image according to the feature, calculate a fusion feature of the grayscale image and the directional gradient histogram, and calculate a fusion feature response value based on the fusion feature;

[0051] A calculation module, configured to calculate a confidence index based on the fused feature response value and adding spatiotemporal information;

[0052] The tracking module is used to extract the infrared weak target in the image according to the feature, update the model and position, and realize target tracking when the confidence index meets the preset range.

[0053] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0054] Obtain the current frame infrared image, perform feature extraction, and obtain a feature-extracted image;

[0055] Extracting the image based on the features, calculating the fusion features of the grayscale image and the directional gradient histogram, and calculating the fusion feature response value based on the fusion features;

[0056] According to the fused feature response value, adding spatiotemporal information, and calculating the confidence index;

[0057] When the confidence index meets the preset range, the infrared weak target in the image is extracted according to the feature, and the model and position are updated to achieve target tracking.

[0058] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0059] Obtain the current frame infrared image, perform feature extraction, and obtain a feature-extracted image;

[0060] Extracting the image based on the features, calculating the fusion features of the grayscale image and the directional gradient histogram, and calculating the fusion feature response value based on the fusion features;

[0061] According to the fused feature response value, adding spatiotemporal information, and calculating the confidence index;

[0062] When the confidence index meets the preset range, the infrared weak target in the image is extracted according to the feature, and the model and position are updated to achieve target tracking.

[0063] The above-mentioned infrared weak target tracking method, device, equipment and medium under complex background is a method for precise tracking under complex background. It adopts the fusion feature of grayscale and directional gradient histogram to improve the representativeness of features and effectively locate weak targets. When updating the model, the target grayscale feature vector is introduced and the update process is adjusted according to the actual situation. It can avoid the accumulation of errors caused by tracking drift, thereby avoiding tracking failure caused by error accumulation and reducing time consumption. The present application is more practical for weak targets, has a high reference value for improving the accuracy of infrared imaging guidance, and plays a good auxiliary role in future military offensive and defensive confrontations. At the same time, it can also provide assistance in private industries such as security monitoring and maritime navigation, and be more effectively applied to related tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a diagram showing an application scenario of a method for tracking small infrared targets in a complex background according to an embodiment;

[0065] Figure 2 1. A flowchart of a method for tracking a small infrared target in a complex background according to an embodiment;

[0066] Figure 3 A schematic diagram of local image segmentation in one embodiment;

[0067] Figure 4 Schematic diagram of contrast feature extraction in one embodiment, where (a) is the original image and (b) is the contrast feature map;

[0068] Figure 5 Schematic diagram of the fusion of confidence indicators in one embodiment, wherein (a) is a schematic diagram before the fusion of confidence indicators, and (b) is a schematic diagram after the fusion of confidence indicators;

[0069] Figure 6 is a schematic diagram of a selective inter-frame difference method in one embodiment;

[0070] Figure 7 A flowchart of a method for tracking small infrared targets in a complex background according to an embodiment;

[0071] Figure 8 1 is a comparative experimental result diagram of the infrared dim small target tracking method in scene 1 in one embodiment, wherein (a) is the first sequence frame, (b) is the second sequence frame, (c) is the third sequence frame, (d) is the fourth sequence frame, and (e) is the fifth sequence frame;

[0072] Figure 91 is a graph showing comparative experimental results of an infrared dim small target tracking method in scene 2 in one embodiment, wherein (a) is the first sequence frame, (b) is the second sequence frame, (c) is the third sequence frame, (d) is the fourth sequence frame, and (e) is the fifth sequence frame;

[0073] Figure 10 1 is a comparative experimental result diagram of the infrared dim small target tracking method in scene 3 in one embodiment, wherein (a) is the first sequence frame, (b) is the second sequence frame, (c) is the third sequence frame, (d) is the fourth sequence frame, and (e) is the fifth sequence frame;

[0074] Figure 11 1 is a comparative experimental result diagram of the infrared dim small target tracking method in scene 4 in one embodiment, wherein (a) is the first sequence frame, (b) is the second sequence frame, (c) is the third sequence frame, (d) is the fourth sequence frame, and (e) is the fifth sequence frame;

[0075] Figure 12 1 is a graph showing comparative experimental results of an infrared dim small target tracking method in scene 5 in one embodiment, wherein (a) is the first sequence frame, (b) is the second sequence frame, (c) is the third sequence frame, (d) is the fourth sequence frame, and (e) is the fifth sequence frame;

[0076] Figure 13 1 is a graph showing comparative experimental results of an infrared dim small target tracking method in scene 6 in one embodiment, wherein (a) is the first sequence frame, (b) is the second sequence frame, (c) is the third sequence frame, (d) is the fourth sequence frame, and (e) is the fifth sequence frame;

[0077] Figure 14 2. It is a structural block diagram of an infrared small target tracking device under complex background in one embodiment;

[0078] Figure 15 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for explaining this application and are not intended to limit this application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application without creative work are within the scope of protection of this application.

[0080] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0081] In addition, the terms "first," "second," and so on, used in this application are for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "multiple groups" means at least two groups, such as two groups, three groups, and so on, unless otherwise specifically defined.

[0082] In this application, unless otherwise specified or limited, the terms "connect," "fix," etc. should be understood in a broad sense. For example, "fix" can mean a fixed connection, a detachable connection, or an integral connection; it can mean a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean internal communication between two elements or an interaction between two elements, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0083] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0084] The method provided in this application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The terminal 102 may include but is not limited to various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server 104 may be a server corresponding to various portal websites or a backend of a work system.

[0085] This application provides a method for tracking infrared dim targets in complex backgrounds, such as Figure 2 As shown, in one embodiment, the method is applied to Figure 1 The following example illustrates the terminal in the example, including:

[0086] Step 202: Acquire the infrared image of the current frame, perform feature extraction, and obtain a feature-extracted image.

[0087] Specifically:

[0088]

[0089] Where G c (x, y) is the local pixel contrast measurement result, that is, the feature extraction image, n is the number of background grids, i(x, y) is the gray value of the center pixel in the current frame infrared image, B i is the grayscale average of the i-th grid in the current frame infrared image.

[0090] In this step, if Figure 3 As shown in the figure, the infrared image is divided into blocks, where 0 is the middle block and 1 to 8 are the peripheral blocks, and the local contrast measurement method (LCM, such as Figure 4 The feature extraction of the area within the tracking frame can enhance the weak infrared target.

[0091] Step 204 : extracting the image based on the features, calculating the fusion features of the grayscale image and the histogram of oriented gradients, and calculating the fusion feature response value based on the fusion features.

[0092] Specifically:

[0093] response = (1-β)response gray +βresponse hog (2)

[0094]

[0095] In the formula, response is the fusion feature response value, response gray is the grayscale feature response value, response hog is the characteristic response value of the directional gradient histogram (ie: hog characteristic response value), β is the weight coefficient, G c The number of pixels in the image that are greater than the set threshold, T size It is the estimated size of the infrared weak target in the current frame infrared image.

[0096] It should be noted that calculating the fusion features of the grayscale image and the directional gradient histogram belongs to the existing technology and will not be described in detail here.

[0097] Step 206 : Calculate the confidence index based on the fused feature response value and adding spatiotemporal information.

[0098] Specifically:

[0099]

[0100]

[0101] Where Th(i) is the confidence index, α1 and α2 are two different weighting coefficients, PSR(i) is the peak sidelobe ratio, which represents the peak response degree of the correlation filter at the i-th frame, and T s (i) is the target feature descriptor under the i-th frame, max response (i) is the maximum value of the fusion feature response value, mean response (i) is the average value of the fusion feature response value, σ response (i) is the standard deviation of the fusion feature response value.

[0102] In this step, the confidence index integrates the spatiotemporal information and can better reflect the response difference of the current frame, such as Figure 5 As shown, the horizontal axis represents the frame number, and the vertical axis represents the response amplitude; Figure 5 (a) is the response diagram calculated only by the peak-to-sidelobe ratio. Figure 5 (b) is the response diagram calculated after introducing spatiotemporal information. It can be seen that Figure 5 (b) Zoomed in on the response differences during anomaly tracking.

[0103] Step 208: When the confidence index satisfies a preset range, the infrared dim small target in the image is extracted according to the feature, and the model and position are updated to achieve target tracking.

[0104] When the confidence index does not meet the preset range, the current frame infrared image and the previous frame infrared image are used to select the frame difference method (such as Figure 6 As shown in FIG, a significant difference image is obtained by subtracting the maximum pixel grayscale value in the shadow area S in the previous frame from the point P in the current frame (the selection of the S area is determined by the angle range 2θ and the distance range R obtained after image matching calculation); based on the significant difference image, a suspected target is selected, and target screening is performed to obtain a screening result; when the number of suspected targets in the screening result is not zero, the suspected target in the screening result is regarded as a weak infrared target, and the model and position are updated to achieve target tracking.

[0105] When the number of suspected targets in the screening results is zero and the type of target screening is not one (that is, no suspected target meets all the screening requirements, and the current frame has a large interference and discontinuous motion occurs), based on the preliminary screening results of a target screening (that is, the screening results obtained after the first target screening), feature coding is introduced and the local background feature vector is calculated: based on the local background feature vector, the suspected targets in the preliminary screening results are judged whether the preset conditions are met; when the preset conditions are met, the suspected targets in the preliminary screening results are regarded as infrared weak targets, and the model and position updates are performed to achieve target tracking.

[0106] When the preset conditions are not met, or the number of suspected targets in the screening results is zero and the target screening type is one, the target is considered lost, the target tracking of the current frame is stopped, and the target tracking of the next frame is carried out.

[0107] The specific process architecture is as follows Figure 7 shown.

[0108] In this step, target screening includes: screening based on target movement distance, screening based on target image similarity, and screening based on target movement rules. That is to say, among these three target screening methods, you can select only one, or you can select two or three. Each time you select one, you will perform a screening, and the next screening will be performed again based on the results of the previous screening. When the type of target screening is not one, there is no restriction on the order of screening.

[0109] Specifically, the method for screening each target is as follows:

[0110] (1) Screening based on target movement distance includes:

[0111]

[0112] in:

[0113]

[0114] Υ1=[x t -2d offset :x t +2d offset ,y t -2d offset :y t +2d offset ]

[0115]

[0116] In the formula, (x best ,y best ) is the screening result (one of the coordinates of the suspected target, that is, the object that meets the conditions after the target movement distance screening is not necessarily unique), I r (x, y) is the coordinate of the suspected target (that is, the pixel in the selective inter-frame difference map, that is, the saliency map T A pixels in ), d offset is the global environment moving distance, is the average moving distance of the target in the previous few frames (the specific number of frames can be determined according to the actual situation, such as 5 frames), Y1 and Y2 are two different moving intervals, I i (x i ,y i) is the pixel grayscale value of the i-th frame infrared image at coordinate (x, y), The pixel value to be selected for the difference at the coordinate (x, y) of the i-1th frame (the i-1th frame is the current frame and the i-1th frame is the previous frame) is Figure 6 The middle shaded area ( Figure 6 The middle shaded area is the S area, which is selected based on the maximum pixel grayscale value of the angle range 2θ and the distance range R obtained after image matching calculation, (x t ,y t ) is the position of the suspected target in the previous frame of infrared image.

[0117] (2) Screening based on target image similarity includes: calculating the target area similarity based on the current frame infrared image and the previous frame infrared image:

[0118] S=min(BC,NC) (8)

[0119]

[0120]

[0121] Where S is the target area similarity. The larger the value, the more similar the two samples (the samples in this application refer to infrared images). BC is the Bhattacharyya coefficient, which measures the overlap between two statistical samples. NC is the normalized cross-correlation coefficient. u is the number of blocks, ranging from 1 to n. is the local binary pattern histogram of the suspected target in the current frame infrared image, is the local binary pattern histogram of the suspected target in the previous frame of infrared image, corr is the cross-correlation operation, is the area where the suspected target is located in the infrared image of the i-th frame, σ i is the standard deviation of the object area in the i-th frame;

[0122] A screening result is obtained according to the target image similarity.

[0123] (3) Screening based on target movement patterns includes:

[0124]

[0125] Where θ i is the target motion direction angle of the i-th frame, P i is the target position in the i-th frame, and δP is the relative movement distance of the suspected target in the previous infrared image frame.

[0126] In this step, the local background feature vector is calculated:

[0127] T b =[c1,c2,…,c 10]

[0128]

[0129]

[0130]

[0131] Where, T b is the local background feature vector, B j is the grayscale average value of the jth grid in the current frame infrared image, n is the number of background grids (the number of surrounding blocks except the center block), I p is the grid where the suspected target is located, For I p The grayscale average value, m l For I p The lth gray value in the grayscale order from largest to smallest;

[0132] Determine whether the suspected targets in the preliminary screening results meet the preset conditions:

[0133]

[0134] Where, is the feature vector of the local background around the real target (such as aircraft target) in the i-th frame infrared image, is the feature vector of the true target in the i-th frame, is the reference background feature vector, which is continuously updated (how to update is a prior art), k is the number of suspected targets, j indicates that the jth suspected target is the true target, λ is the update rate, ζ i-1 is the background feature difference measure, is the feature vector of the true target in the i-1th frame.

[0135] In this step, the model update and position update include: constructing a target grayscale feature vector based on the obtained infrared dim target, and updating the sample feature model to achieve target tracking.

[0136] Specifically, construct the target grayscale feature vector:

[0137] T s =[s1,s2,s3,s4,s5]

[0138]

[0139]

[0140]

[0141]

[0142] Where, T s is the target grayscale feature vector, For I p The maximum value of pixel grayscale in the block image, For I p The average value of pixel grayscale in the block image, σ p is the block image standard deviation, for The number of pixels with values ​​greater than 0, for The maximum value of pixel grayscale in the block image, where i is 1, 2, or 3. and Only I p The pixel value is greater than and less than Pixels, c is the normalization parameter;

[0143] Update the sample feature model:

[0144]

[0145]

[0146]

[0147] Where, is the sample feature model, ξ i is the target grayscale feature difference metric, is the target grayscale feature vector in the i-th frame infrared image, is the reference target feature vector, express The values ​​of , f and λ are two different update rates.

[0148] It should be noted that the sample feature model determines whether to update based on whether the execution index meets the conditions. In other words, whether to update the model is determined based on whether formula (15) is satisfied. The model is updated only if it is satisfied. If not, it means that the model is lost.

[0149] This embodiment also includes: scale update; specifically, using the proportional pyramid method in the prior art, there are 7 different scales, and the estimation process is performed every few frames (for example, every 30 frames). This saves time while meeting the tracking accuracy requirements (because the size of small targets changes slowly) and does not require constant adjustment of the tracking window size.

[0150] The above-mentioned infrared weak target tracking method under complex background is a method for precise tracking under complex background. It adopts the fusion feature of grayscale and directional gradient histogram to improve the representativeness of features and effectively locate weak targets. When updating the model, more parameters are taken into account, and the target grayscale feature vector is introduced. The update process is adjusted according to the actual situation, which can avoid error accumulation caused by tracking drift, thereby avoiding tracking failure caused by error accumulation and reducing time consumption. The present application is more practical for weak targets, has a high reference value for improving the accuracy of infrared imaging guidance, and plays a good auxiliary role in future military offensive and defensive confrontations. At the same time, it can also provide assistance in private industries such as security monitoring and maritime navigation, and be more effectively applied to related tasks.

[0151] Preferably, a response threshold is judged based on a confidence index. When the confidence index does not meet a preset range (i.e., the confidence index fluctuates significantly at the moment), re-search is initiated (i.e., at least one target screening is performed based on a significant difference image). During the re-search process: screening is performed based on the target motion distance, so that most unnecessary targets can be screened out; screening is performed based on target image similarity, and the target position can be further verified by comparing the suspected target area in the previous frame with the current frame using image correlation operations and local binary pattern histograms; screening is performed based on the target motion law, so that the precise position of the infrared weak target can be obtained; after multiple screenings, the re-search can search for targets lost due to various abnormal disturbances and obtain the required targets, and can maintain a high anti-interference ability under a variety of undesirable objective conditions.

[0152] Further preferably, when target screening fails to obtain the desired target, target local feature encoding is introduced, and a feature vector is used to represent the state of the target in each frame, thereby obtaining the precise position of the target.

[0153] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0154] In a specific embodiment, the method of the present application (solid line rectangle) and the method in the prior art are used to conduct simulation experiments, and the results under different scenarios are compared. Figures 8 to 13 As shown, in the prior art method, the dotted rectangular box represents DSST, the arc box represents SAMF, the dotted rectangular box represents DSARCF, the dotted rectangular box represents RPT, and the elliptical box represents LADCF. Figure 8 (a) to Figure 8 (e) The tracking stability of each algorithm at different frames in the scene sequence, Figure 9 (a) to Figure 9 (e) The tracking stability of each algorithm in different frames in the scene 2 sequence, Figure 10 (a) to Figure 10 (e) The tracking stability of each algorithm in different frames in the three sequences of the scene, Figure 11 (a) to Figure 11 (e) The tracking stability of each algorithm in different frames in the four sequences of the scene, Figure 12 (a) to Figure 12 (e) The tracking stability of each algorithm in different frames in the five sequences of the scene, Figure 13 (a) to Figure 13 (e) The tracking stability of each algorithm in different frames of the six scene sequences. It can be seen that the proposed method can achieve stable tracking, and its advantages gradually emerge when the environment background becomes complex.

[0155] This application also provides an infrared weak target tracking device under complex background, such as Figure 14 As shown, in one embodiment, it includes: an acquisition module 1402, a fusion module 1404, a calculation module 1406 and a tracking module 1408, wherein:

[0156] The acquisition module 1402 is used to acquire the infrared image of the current frame, perform feature extraction, and obtain a feature-extracted image;

[0157] A fusion module 1404 is configured to extract an image based on the features, calculate fusion features of the grayscale image and the oriented gradient histogram, and calculate a fusion feature response value based on the fusion features;

[0158] A calculation module 1406 is configured to calculate a confidence index based on the fused feature response value and adding spatiotemporal information;

[0159] The tracking module 1408 is used to extract the infrared small target in the image according to the feature, perform model update and position update, and realize target tracking when the confidence index meets the preset range.

[0160] Regarding the specific definition of the infrared small target tracking device under complex background, please refer to the definition of the infrared small target tracking method under complex background above, and will not be repeated here. Each module in the above-mentioned device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0161] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 15 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for tracking infrared weak targets in a complex background is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0162] Those skilled in the art will understand that Figure 15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0163] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0164] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0165] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0166] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. The infrared small target tracking method under complex background is characterized by: include: Obtain the current frame infrared image, perform feature extraction, and obtain a feature-extracted image; Extracting the image based on the features, calculating the fusion features of the grayscale image and the directional gradient histogram, and calculating the fusion feature response value based on the fusion features; According to the fused feature response value, adding spatiotemporal information, and calculating the confidence index; When the confidence index meets the preset range, the infrared weak target in the image is extracted according to the feature, and the model and position are updated to achieve target tracking; Model updating and position updating include: constructing the target grayscale feature vector based on the obtained infrared dim target, and updating the sample feature model to achieve target tracking; The target grayscale feature vector is constructed as follows: Where, is the target grayscale feature vector, for The maximum value of pixel grayscale in the block image, for The average value of pixel grayscale in the block image, is the block image standard deviation, for The number of pixels with values ​​greater than 0, for The maximum value of pixel grayscale in the block image, middle i Take 1, 2, 3, 、 and Only keep The pixel value is greater than , and less than pixels, is the normalization parameter; The updated sample feature model is: Where, is the sample feature model, is the target grayscale feature difference metric, For the The target grayscale feature vector in the frame infrared image, is the reference target feature vector, express The value of and For two different update rates.

2. The method for tracking small infrared targets under complex background according to claim 1, characterized in that: When the confidence index does not meet the preset range, a significant difference image is obtained based on the current frame infrared image and the previous frame infrared image; Selecting suspected targets based on the significant difference image and performing target screening to obtain screening results; The target screening includes at least one of screening based on target movement distance, screening based on target image similarity, and screening based on target movement patterns; When the number of suspected targets in the screening results is not zero, the suspected targets in the screening results are regarded as infrared weak targets, and the model and position are updated to achieve target tracking.

3. The infrared small target tracking method under complex background according to claim 2 is characterized in that: Filtering based on target distance includes: in: Where, To filter the results, are the coordinates of the suspected target, is the global environment movement distance, is the average movement distance of the target, and There are two different motion zones. For the Frame infrared image at coordinates The pixel gray value, For the Frame in coordinates Select the pixel value to be subtracted. is the position of the suspected target in the previous frame of infrared image.

4. The method for tracking small infrared targets under complex background according to claim 2, characterized in that: Filtering based on target image similarity includes: Calculate the target area similarity based on the current infrared image and the previous infrared image: Where, is the target region similarity, is the Bhattacharyya coefficient, is the normalized cross-correlation coefficient, is the number of blocks, ranging from 1 to n , is the local binary pattern histogram of the suspected target in the current frame infrared image, is the local binary pattern histogram of the suspected target in the previous frame of infrared image, is the cross-correlation operation, For the The area block where the suspected target is located in the frame infrared image, For the The standard deviation of the object area in the frame; A screening result is obtained according to the target image similarity.

5. The method for tracking small infrared targets under complex background according to claim 2, characterized in that: Screening based on target motion patterns includes: Where, For the The target motion direction angle of the frame, For the The target location of the frame, is the relative moving distance of the suspected target in the previous frame of infrared image.

6. The method for tracking small infrared targets under complex background according to any one of claims 2 to 5, characterized in that: When the number of suspected targets in the screening result is zero and the type of target screening is not one, feature coding is introduced according to the preliminary screening result of one target screening to calculate the local background feature vector: Where, is the local background feature vector, The first j The average grayscale value of the grid, is the number of background grids, is the grid where the suspected target is located, for The average grayscale value of for The gray values ​​are arranged from large to small. Grayscale values; Based on the local background feature vector, the suspected target in the preliminary screening results is judged whether it meets the preset conditions: Where, For the The feature vector of the local background around the real target in the frame infrared image, The true target is The feature vector in the frame, is the reference background feature vector, is the number of suspected targets, Indicates the suspected targets are real targets, is the update rate, is the background feature difference measure, The true target is Feature vectors in the frame; When the preset conditions are met, the suspected target in the preliminary screening results is regarded as a weak infrared target, and the model and position are updated to achieve target tracking; When the preset conditions are not met, or the number of suspected targets in the screening result is zero and the type of target screening is one, target tracking for the next frame is performed.

7. The method for tracking small infrared targets under complex background according to any one of claims 1 to 5, characterized in that: Get the current frame infrared image and perform feature extraction to obtain the feature-extracted image including: Where, For feature extraction images, is the number of background grids, is the grayscale value of the center pixel in the current frame infrared image, The first The average grayscale value of the grid.

8. Infrared small target tracking device under complex background, characterized by: The method according to any one of claims 1 to 7, comprising: An acquisition module is used to acquire the infrared image of the current frame, perform feature extraction, and obtain a feature-extracted image; A fusion module is used to extract an image according to the feature, calculate a fusion feature of the grayscale image and the directional gradient histogram, and calculate a fusion feature response value based on the fusion feature; A calculation module, configured to calculate a confidence index based on the fused feature response value and adding spatiotemporal information; The tracking module is used to extract the infrared weak target in the image according to the feature, update the model and position, and realize target tracking when the confidence index meets the preset range.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images