Ghost image identification method and system in infrared image plane small target tracking

Abstract

The invention provides a ghost image recognition method and system in infrared image plane small target tracking, and the method comprises the steps: S1, carrying out the statistical analysis of the gray ratio and geometric relative position of a real target and an accompanying ghost image on an image plane through an infrared image plane imaging test, and determining recognition feature parameters; s2, loading recognition feature parameters, then detecting small targets by adopting a deep learning method, and extracting tracking trajectories of the plurality of small targets through data association and Kalman filtering; and S3, traversing the tracking trajectory which is confirmed to be the moving target for the first time, extracting gray scale ratio characteristics and geometric relative position characteristics of the tracking trajectory and other moving tracking trajectories, and comparing the gray scale ratio characteristics and geometric relative position characteristics with threshold value characteristic parameters determined in the test to judge whether the moving target is a ghost image or not. The method has the advantages of high ghost image recognition accuracy, recognition modularization, high operation speed and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared small target tracking and surveillance, and specifically, to a method and system for ghost image recognition in infrared focal plane small target tracking. Background Technique

[0002] With the wide application of long-distance infrared search and tracking systems, the technology of infrared focal plane small target detection and tracking has been continuously developed. When the detected target has a strong radiation intensity and brightness, ghost images often appear on the focal plane. After the strong radiation signal enters the infrared optical system, it is reflected on the surface of optical elements such as lenses or lenses. These reflections are reflected multiple times inside the system and finally reach the detector, forming false images and ghost images. The ghost images appear along with the strong radiation target, with weak energy and often having a fixed imaging geometric relationship with the strong radiation target. It is necessary to study the technology of suppressing and identifying ghost images to eliminate the false alarms of the system caused by ghost images.

[0003] The suppression of ghost images is generally achieved by improving the design of the optical system itself, such as lens coating [1], optical system design [2], stray light suppression [3], etc. However, due to the limitations of the design, process, and assembly of the optical system itself at the source, the ghost images cannot be completely eliminated in the final imaging effect. Therefore, how to identify ghost images from infrared imaging data has important application value for suppressing false alarms in infrared search and tracking systems.

[0004] Identifying using the size and contour features of ghost images on the focal plane is one type of method [4, 5]. Another method is to distinguish by the intensity difference between ghost images and the background [6]. However, in the infrared small target detection and tracking, the ghost images do not have obvious size and contour features, so the methods in [4, 5] are not applicable, and the target has faint features similar to ghost images, so the method in [6] cannot be used either. Therefore, it is necessary to design a new method to identify and process ghost images in the tracking of infrared moving small targets.

[0005] The references for the above technologies are as follows:

[0006] [1]. Yu Tianyan, Jiang Lin, Qin Yang, etc. An efficient infrared wide-spectrum antireflection film suitable for space environment [P]. CN115201941B, September 12, 2023.

[0007] [2]. Shen Jian, Lan Shun, Chen Yuan. An opto-mechanical structure for improving ghost images, its manufacturing method, and a display waveguide device [P]. CN11599725A, April 21, 2023.

[0008] [3]. Hu Xiongchao, Mao Xiaonan, Du Weifeng, etc. A ghost image suppression structure for space cameras, a ground verification system, and a method [P]. CN116471488A, July 21, 2023.

[0009] [4]. Yu Menglu, Wu Rui, Ma Weifei, etc. Ghost image detection method, device, equipment and readable storage medium [P]. CN115147413B, December 13, 2022.

[0010] [5]. Ran Chengrong, Sun Jie, Jiang Kunjun, etc. Ghost image detection method and its system, electronic equipment and ghost image detection platform [P]. CN113112444B, May 31, 2022.

[0011] [6]. Zhang Xing, Dai Peng, Wang Wei, etc. Moving target detection method, device, equipment and medium [P]. CN117576147A, February 20, 2024. Summary of the Invention

[0012] Aiming at the defects in the prior art, the purpose of the present invention is to provide a ghost image recognition method and system in infrared image plane small target tracking.

[0013] The ghost image recognition method in infrared image plane small target tracking provided by the present invention includes:

[0014] Step S1: Through infrared image plane imaging experiments, statistically analyze the gray ratio and geometric relative position between the real target and the accompanying ghost image on the image plane to determine the recognition feature parameters;

[0015] Step S2: Load the recognition feature parameters, then use the deep learning method to detect small targets, and then extract the tracking trajectories of multiple small targets through data association and Kalman filtering;

[0016] Step S3: Traverse the tracking trajectories initially confirmed as moving targets, extract the gray ratio features and geometric relative position features between them and other active tracking trajectories, and determine whether they are ghost images by comparing with the threshold feature parameters determined in the experiment.

[0017] Preferably, the step S1 includes:

[0018] In the laboratory, set a target target with strong infrared radiation to make an accompanying ghost image appear on the infrared image plane and collect data; or in the infrared surveillance imaging of the actual scene, collect data of randomly appearing strong radiation targets and accompanying ghost images;

[0019] Analyze the infrared surveillance imaging data of the actual scene, manually discriminate and collect data of randomly appearing strong radiation targets and accompanying ghost images, extract the gray values and coordinate values of the two on the image plane, and calculate the gray ratio of the two and the differences in the x and y directions of the coordinates of the two respectively;

[0020] Form 7 types of recognition features through statistics, namely the gray ratio threshold Thgray The mean threshold range Th of the differences in the x and y directions of the position coordinates x,min Th x,max Th y,min Th y,max , and the threshold Th of the standard deviation x,syd Th y,std ; As the continuous accumulation of strong target and accompanying ghost image data is collected, the recognition threshold features are iteratively updated.

[0021] Preferably, the specific method for detecting small targets by using the deep learning method in step S2 is as follows: By building a deep convolutional neural network and training, a probability heat map of each pixel in the output image belonging to the target is obtained. The probability heat map is subjected to threshold processing and four-neighborhood connected clustering to obtain the centroid position and pixel distribution map corresponding to each target. The background is estimated and removed by median filtering the original image, and further a background-removed residual map is obtained, and then a grayscale map corresponding to each target is obtained, and the sum of the grayscale distribution is taken as the grayscale value of the target.

[0022] Preferably, the specific method for extracting the tracking trajectories of multiple small targets by data association and Kalman filtering in step S2 is as follows: Adopting typical multi-target tracking techniques, including Kalman filtering for single-target state estimation and two-dimensional assignment data association for Kalman filter prediction and detection results, combined with tracking start, maintenance, and termination determination; When performing tracking start, in addition to the trajectory being continuously associated with the detection results for multiple frames, the trajectory length also needs to exceed the set value; Trajectories that meet the tracking start conditions and have not terminated are marked as active tracking trajectories, and the target trajectory that first meets the tracking start conditions is called the tracking trajectory of the first confirmed moving target.

[0023] Preferably, step S3 includes:

[0024] If there is only one currently active tracking trajectory, it is judged as a non-ghost image and the recognition process is terminated; otherwise, it transfers to traverse other active trajectories, which are recorded as comparison trajectories;

[0025] If the starting time of the current trajectory is earlier than that of the comparison trajectory, it transfers to traverse the next comparison trajectory; otherwise, it further discriminates based on the gray-scale contrast feature; Calculate the gray-scale ratio sequence formed by the gray-scale ratio of each moment of the current trajectory to the corresponding moment of the comparison trajectory, calculate the mean of this sequence. If the mean is less than the preset threshold, it transfers to traverse the next comparison trajectory; otherwise, it discriminates based on the geometric relative position feature; Calculate the differences in the x and y directions of the coordinates of each moment of the current trajectory and the corresponding moment of the comparison trajectory to form two sequences, calculate the mean and standard deviation of the two sequences respectively. If the means and standard deviations of the two sequences are both within the preset threshold ranges, it is determined as a ghost image, and it transfers to traverse the ghost image recognition process of the next tracking trajectory of the first confirmed moving target; otherwise, it transfers to traverse the next comparison trajectory;

[0026] When all the comparison trajectories are traversed and the current trajectory is not confirmed as a ghost image, it is determined that the current trajectory is not a ghost image, and the ghost image recognition process for the next tracking trajectory first confirmed as a moving target is entered; when the ghost image recognition of all the tracking trajectories first confirmed as moving targets is completed, the process ends.

[0027] Embed the ghost image recognition process into the large process of infrared image plane small target detection and tracking, and trigger the execution at the moment when a moving target is first determined.

[0028] According to the ghost image recognition system in the infrared image plane small target tracking provided by the present invention, it includes:

[0029] Module M1: Through infrared image plane imaging experiments, statistically analyze the gray-scale ratio and geometric relative position between the real target and the accompanying ghost image on the image plane, and determine the recognition feature parameters.

[0030] Module M2: Load the recognition feature parameters, then use the deep learning method to detect small targets, and then extract the tracking trajectories of multiple small targets through data association and Kalman filtering.

[0031] Module M3: Traverse the tracking trajectories first confirmed as moving targets, extract the gray-scale ratio features and geometric relative position features between them and other active tracking trajectories, and determine whether it is a ghost image by comparing with the threshold feature parameters determined in the experiment.

[0032] Preferably, the module M1 includes:

[0033] In the laboratory, by setting a target with strong infrared radiation, make the accompanying ghost image appear on the infrared image plane and collect data; or in the infrared surveillance imaging of the actual scene, collect the data of randomly appearing strong radiation targets and the accompanying ghost images.

[0034] Analyze the infrared surveillance imaging data of the actual scene, collect the data of randomly appearing strong radiation targets and the accompanying ghost images through manual discrimination, extract the gray-scale values and coordinate values of the two on the image plane, and calculate the gray-scale ratio of the two respectively, as well as the differences between the coordinates of the two in the x and y directions.

[0035] Form 7 types of recognition features through statistics, which are the gray-scale ratio threshold Th gray , the mean threshold range Th x,min of the differences in the x and y directions of the position coordinates, Th x,max , Th y,min , Th y,max , and the standard deviation threshold Th x,std , Th y,std ; as the data of strong targets and accompanying ghost images continues to accumulate, the recognition threshold features are iteratively updated.

[0036] Preferably, the detection of small targets by using deep learning methods in the module M2 is specifically as follows: By building a deep convolutional neural network and training it, a probability heat map of each pixel belonging to the target in the output image is obtained. The probability heat map is subjected to threshold crossing processing and four-neighbor connected component clustering to obtain the centroid position and pixel distribution map corresponding to each target. The background is estimated by median filtering the original image and removed, and further a background-removed residual map is obtained. Then, a grayscale map corresponding to each target is obtained, and the sum of the grayscale distribution is taken as the grayscale value of the target.

[0037] Preferably, the extraction of the tracking trajectories of multiple small targets by data association and Kalman filtering in the module M2 is specifically as follows: The typical multi-target tracking technology is adopted, including Kalman filtering for single-target state estimation and two-dimensional assignment data association for Kalman filter prediction and detection results, combined with tracking start, maintenance, and termination determination; When performing tracking start, in addition to the trajectory being associated with the detection results for multiple consecutive frames, the trajectory length also needs to exceed a set value; For the trajectories that meet the tracking start conditions and have not terminated, they are marked as active tracking trajectories, and for the target trajectories that first meet the tracking start conditions, they are called the tracking trajectories of the first confirmed moving targets.

[0038] Preferably, the module M3 includes:

[0039] If there is only one currently active tracking trajectory, it is judged as non-ghost image and the recognition process is terminated; Otherwise, it transfers to traverse other active trajectories, denoted as comparison trajectories;

[0040] If the starting time of the current trajectory is earlier than that of the comparison trajectory, it transfers to traverse the next comparison trajectory; Otherwise, it is further judged based on the grayscale comparison feature; Calculate the grayscale ratio sequence by forming the grayscale ratio of each moment of the current trajectory to the corresponding moment of the comparison trajectory, and calculate the mean value of this sequence. If the mean value is less than the preset threshold, it transfers to traverse the next comparison trajectory, otherwise it is judged based on the geometric relative position feature; Calculate the differences in the x and y directions of the coordinates of each moment of the current trajectory and the corresponding moment of the comparison trajectory to form two sequences, and calculate the mean value and standard deviation of the two sequences respectively. If the mean values and standard deviations of the two sequences are respectively within the preset threshold ranges, it is judged as a ghost image, and it transfers to traverse the ghost image recognition process of the next tracking trajectory of the first confirmed moving target, otherwise it transfers to traverse the next comparison trajectory;

[0041] When all comparison trajectories have been traversed and the current trajectory has not been confirmed as a ghost image, it is judged that the current trajectory is not a ghost image, and it transfers to traverse the ghost image recognition process of the next tracking trajectory of the first confirmed moving target; When the ghost image recognition of all tracking trajectories of the first confirmed moving targets has been traversed, the process ends;

[0042] Embed the ghost image recognition process into the large process of infrared focal plane small target detection and tracking, and trigger the execution at the moment when a moving target is first determined.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] Through infrared focal plane imaging experiments, the present invention statistically determines the ghost image recognition feature parameters. On the basis of the detection and tracking of small targets in the infrared focal plane, it traverses the tracking trajectory of the first confirmed moving target, and identifies the ghost image trajectory through the threshold determination of the gray ratio and the geometric relationship of the coordinate positions, having the advantages of high correct rate of ghost image trajectory recognition, recognition modularization, fast operation speed, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Other features, objects and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0046] Figure 1 It is a flowchart of the recognition method of the present invention;

[0047] Figure 2 It is a specific flowchart of step S3 in the recognition method process of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several changes and improvements can still be made. These all belong to the protection scope of the present invention.

[0049] Embodiment 1

[0050] As Figure 1 shown, the present invention provides a ghost image recognition method in infrared focal plane small target tracking, including the following steps:

[0051] Step S1, through infrared focal plane imaging experiments, statistically analyze the gray ratio and geometric relative position between the real target and the accompanying ghost image on the focal plane, and determine the recognition feature parameters;

[0052] There are two technical approaches for infrared focal plane imaging experiments: one is to set a strong infrared radiation target in the laboratory to make an accompanying ghost image appear on the infrared focal plane and collect data; the other is to collect data of randomly appearing strong radiation targets and accompanying ghost images in the infrared surveillance imaging of the actual scene. From the feasibility, economy and practicality of the test implementation, the second test approach is adopted in this embodiment;

[0053] First, perform infrared surveillance imaging on the actual scene, analyze the sequence imaging data, manually identify and collect the data of randomly occurring strong radiation targets and accompanying ghost images, and extract the gray values and coordinate values of both on the image plane. The gray values and coordinate values of the target are automatically obtained through the detection and tracking process in subsequent step S2, and then the main target and the accompanying ghost image are manually identified;

[0054] Secondly, after obtaining the gray values and position coordinates of a certain number of strong targets and accompanying ghost images, divide the data according to different spatial geometric relationships. If there is only one spatial geometric relationship, there is no need to divide. Through manual confirmation, assume that there are N different spatial geometric relationships between the strong target and the accompanying ghost image. Taking the i-th spatial geometric relationship as an example, there are N i groups of sequence data of strong targets and accompanying ghost images. The sequence data of the strong target can be recorded as:

[0055]

[0056] The sequence data of the accompanying ghost image is recorded as,

[0057]

[0058] In the above two formulas, x ijk , y ijk are the values of the target position coordinates in the x-direction and y-direction respectively, I ijk is the gray value of the target, K j is the length of the j-th group of data sequences; the sequence data of the accompanying ghost image is distinguished from the strong target by the superscript g.

[0059] Finally, count the gray recognition features and spatial geometric position features to determine the threshold range.

[0060] For the gray recognition feature, calculate the gray ratio of all accompanying ghost images and strong targets as follows:

[0061]

[0062] Then calculate the mean (denoted as mean_r ) and standard deviation (denoted as std_r gray ) of all gray and set the threshold Th gray of the gray ratio according to the following formula:

[0063]

[0064] The coefficient k gray takes values between 10 and 500 and can be selected according to the actual situation. In this embodiment, the value is 100.

[0065] The spatial geometric features are determined respectively according to different geometric relationships. First, calculate the differences between the coordinates of the adjoint ghost image and the strong target in the x-direction and y-direction as follows:

[0066]

[0067] Calculate the mean and standard deviation of the coordinate position differences in the two directions respectively, and denote them as:

[0068] mean_Δx i , i = 1, 2, …, N (7)

[0069] std_Δx i , i = 1, 2, …, N (8)

[0070] mean_Δy i , i = 1, 2, …, N (9)

[0071] std_Δy i , i = 1, 2, …, N (10)

[0072] Determine the spatial geometric feature thresholds, including the mean thresholds (Th ix,min 、Th ix,max 、Th iy,min 、Th iy,max ) and standard deviation thresholds (Th ix,std 、Th iy,std ) of the differences in the x and y directions of the position coordinates, as follows:

[0073]

[0074] Coefficient k ix 、k iy take values between 6 and 18, and take the value of 12 in this embodiment; coefficient k ix,std 、k iy,std take values between 3 and 9, and take the value of 3 in this embodiment.

[0075] Finally, the ghost image recognition feature parameters are as follows:

[0076] {Th gray ,{Th ix,min ,Th ix,max ,Th iy,min ,Th iy,max ,Th ix,std ,Th iy,std}i = 1, 2, …, N} (17)

[0077] Store the above (6×N + 1) feature parameters. As the data of strong targets and adjoint ghost images collected continues to accumulate, the recognition threshold features can be iteratively updated;

[0078] Step S2: In practical applications, first load the feature parameters; then, use the deep learning method to detect small targets, and then extract multiple small target tracking trajectories through data association and Kalman filtering;

[0079] First, load the feature parameters and the feature parameters obtained in step S1 (such as Equation 17).

[0080] Secondly, for the infrared sequence images, use the image segmentation deep learning method to detect small targets. By building and training a deep convolutional neural network, output the probability heat map of each pixel in the image belonging to the target. Perform threshold processing and four-neighborhood clustering on the probability heat map to obtain the centroid position and pixel distribution map corresponding to each target. Obtain the background estimate through median filtering of the original image and further obtain the background-subtracted residual map, and then obtain the grayscale image corresponding to each target. Take the sum of the grayscales as the grayscale value of the target; the deep learning method can use existing methods. For the multi-frame processing method, it can detect faint infrared targets including ghost images;

[0081] Finally, extract multiple small target tracking trajectories through data association and Kalman filtering. A typical multi-target tracking technology is adopted, including Kalman filtering for single-target state estimation and two-dimensional assignment data association for Kalman filter prediction and detection results. Combined with the start, maintenance, and termination determination of tracking, this embodiment adopts the data association multi-target filtering method in deep sort. Since the features of infrared small targets are not obvious in morphology, only the predicted motion state is used for association in the data association calculation. The first confirmation of a moving target is determined by the number of trajectory points of the tracking trajectory being no less than 5 and the moving distance on the image plane exceeding 2 pixels. For each tracking moment, at the end of multi-target tracking, it is necessary to determine whether there is a first-confirmed moving target trajectory. All trajectories confirmed as moving targets and not terminated become active trajectories. The target trajectory information includes the target coordinate position, grayscale value, and attribute information (whether it is a ghost image) at each tracking moment.

[0082] Step S3: Traverse the tracking trajectories that are first confirmed as moving targets, extract the grayscale ratio features and geometric relative position features between them and other active tracking trajectories, and determine whether they are ghost images by comparing with the threshold feature parameters determined in the experiment;

[0083] The process of traversing the tracking trajectories that are first confirmed as moving targets in step S3 to determine whether they are ghost images is as follows Figure 2 and is described as follows:

[0084] If there is only one currently active tracking trajectory, it is determined as a non-ghost image and the recognition process is terminated. Otherwise, transfer to traverse other active tracking trajectories (denoted as comparison trajectories);

[0085] If the starting time of the current trajectory is earlier than that of the comparison trajectory, then proceed to traverse the next comparison trajectory; otherwise, further discriminate based on the gray-scale comparison feature.

[0086] Calculate the gray-scale ratio sequence by forming the gray-scale ratio of each moment of the current trajectory to the corresponding moment of the comparison trajectory, and calculate the mean of this sequence, denoted as Th. gray,c , if this mean is greater than the specified threshold, the expression is:

[0087] Th gray,c > Th gray (18)

[0088] Then proceed to traverse the next comparison trajectory; otherwise, discriminate based on the geometric relative position feature.

[0089] Calculate the differences in the x and y directions of the coordinates of each moment of the current trajectory and the corresponding moment of the comparison trajectory to form two sequences, and calculate the mean and standard deviation of the two sequences respectively, denoted as Th cx , Th cy and Th cx,std , Th cy,std , if the means and standard deviations of the two sequences are respectively within the specified threshold ranges;

[0090] That is, there exists i ≤ N such that the following formula holds;

[0091] Th ix,min ≤ Th cx ≤ Th ix,max (19)

[0092] Th iy,min ≤ Th cy ≤ Th iy,max (20)

[0093] Th cx,std ≤ Th ix,std (21)

[0094] Th cy,std ≤ Th iy,std (22)

[0095] Then it is determined as a ghost image, and proceed to traverse the ghost image recognition process of the next tracking trajectory that is first confirmed as a moving target; otherwise, proceed to traverse the next comparison trajectory.

[0096] When all comparison trajectories have been traversed and the current trajectory has not been confirmed as a ghost image, then it is determined that the current trajectory is not a ghost image, and proceed to traverse the recognition of the next tracking trajectory that is first confirmed as a moving target.

[0097] When the recognition of all tracking trajectories that are first confirmed as moving targets has been completed, the process ends.

[0098] Furthermore, this method will be integrated into the large process of detecting and tracking weak and small moving multi-targets in the infrared image plane for identifying the trajectory of ghost targets; in the large process, it is triggered and executed at the moment when the multi-target tracking first determines a moving target.

[0099] Embodiment 2

[0100] The present invention also provides a ghost image recognition system in infrared image plane small target tracking. The ghost image recognition system in infrared image plane small target tracking can be implemented by executing the process steps of the ghost image recognition method in infrared image plane small target tracking. That is, those skilled in the art can understand the ghost image recognition method in infrared image plane small target tracking as the preferred implementation manner of the ghost image recognition system in infrared image plane small target tracking.

[0101] According to the ghost image recognition system in infrared image plane small target tracking provided by the present invention, it includes:

[0102] Module M1: Through infrared image plane imaging experiments, statistically analyze the gray-scale ratio and geometric relative position between the real target and the accompanying ghost image on the image plane to determine the recognition feature parameters;

[0103] Module M2: Load the recognition feature parameters, then use the deep learning method to detect small targets, and then extract the tracking trajectories of multiple small targets through data association and Kalman filtering;

[0104] Module M3: Traverse the tracking trajectories that are first confirmed as moving targets, extract the gray-scale ratio features and geometric relative position features between them and other active tracking trajectories, and determine whether they are ghost images by comparing with the threshold feature parameters determined in the experiment.

[0105] The said Module M1 includes:

[0106] In the laboratory, by setting a target with strong infrared radiation, make an accompanying ghost image appear on the infrared image plane and collect data; or in the infrared surveillance imaging of the actual scene, collect data of randomly appearing strong radiation targets and accompanying ghost images;

[0107] Analyze the infrared surveillance imaging data of the actual scene, through manual discrimination and collect data of randomly appearing strong radiation targets and accompanying ghost images, extract the gray-scale values and coordinate values of both on the image plane, and calculate the gray-scale ratio of both respectively, as well as the differences between the coordinates of both in the x and y directions;

[0108] Form 7 types of recognition features through statistics, which are respectively the gray-scale ratio threshold Th gray and the mean threshold range Th x,min of the differences in the x and y directions of the position coordinates, Th x,max Th y,min, Th y,max , and the threshold Th of the standard deviation x,std , Th y,std ; As the continuous accumulation of strong target and accompanying ghost image data collected, the recognition threshold features are iteratively updated.

[0109] The specific method of detecting small targets using deep learning methods in the module M2 is as follows: By building a deep convolutional neural network and training, output the probability heat map of each pixel in the image belonging to the target, perform threshold processing on the probability heat map, and perform four-neighborhood connected clustering to obtain the centroid position and pixel distribution map corresponding to each target. Obtain the background estimate by median filtering the original image and remove it, further obtain the background-removed residual map, and then obtain the grayscale map corresponding to each target. Take the sum of the grayscale distribution as the grayscale value of the target.

[0110] The specific method of extracting the tracking trajectories of multiple small targets through data association and Kalman filtering in the module M2 is as follows: Adopt typical multi-target tracking technology, including Kalman filtering for single-target state estimation and two-dimensional assignment data association for Kalman filter prediction and detection results, combined with tracking start, maintenance, and termination determination; When performing tracking start, in addition to the trajectory being continuously associated with the detection result for multiple frames, the trajectory length also needs to exceed the set value; Trajectories that meet the tracking start conditions and have not terminated are marked as active tracking trajectories, and the target trajectory that first meets the tracking start conditions is called the tracking trajectory of the first confirmed moving target.

[0111] The module M3 includes:

[0112] If there is only one currently active tracking trajectory, it is judged as non-ghost and the recognition process is terminated; otherwise, transfer to traverse other active trajectories, denoted as comparison trajectories;

[0113] If the starting time of the current trajectory is earlier than the starting time of the comparison trajectory, transfer to traverse the next comparison trajectory; otherwise, further judge based on the gray-scale contrast feature; Calculate the gray-scale ratio sequence of each moment of the current trajectory and the corresponding moment of the comparison trajectory to form a gray-scale ratio sequence, calculate the mean value of this sequence, if the mean value is less than the preset threshold, transfer to traverse the next comparison trajectory, otherwise judge based on the geometric relative position feature; Calculate the differences in the x and y directions of the coordinates of each moment of the current trajectory and the corresponding moment of the comparison trajectory to form two sequences, calculate the mean values and standard deviations of the two sequences respectively, if the mean values and standard deviations of the two sequences are both within the preset threshold range, it is judged as a ghost, transfer to traverse the ghost recognition process of the next tracking trajectory of the first confirmed moving target, otherwise transfer to traverse the next comparison trajectory;

[0114] When all comparison trajectories have been traversed and the current trajectory has not been confirmed as a ghost image, it is determined that the current trajectory is not a ghost image, and the ghost image recognition process for the next tracking trajectory that is first confirmed as a moving target is entered; when the ghost image recognition of all tracking trajectories that are first confirmed as moving targets is completed, the process ends;

[0115] Embed the ghost image recognition process into the large process of infrared image plane small target detection and tracking, and trigger its execution at the moment when a moving target is first determined.

[0116] Those skilled in the art know that in addition to implementing the systems, devices, and their various modules provided by the present invention in the form of pure computer-readable program codes, the method steps can be logically programmed to enable the systems, devices, and their various modules provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc., to achieve the same program. Therefore, the systems, devices, and their various modules provided by the present invention can be regarded as a kind of hardware component, and the modules included therein for implementing various programs can also be regarded as the structures within the hardware component; the modules for implementing various functions can also be regarded as either software programs for implementing the methods or the structures within the hardware component.

[0117] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for identifying ghost images in small target tracking in an infrared image plane, characterized in that: include: Step S1: Through infrared image plane imaging test, the grayscale ratio and geometric relative position between the real target and the accompanying ghost image on the image plane are statistically analyzed to determine the recognition feature parameters; Step S2: Load the recognition feature parameters, then use the deep learning method to detect small targets, and then extract the tracking trajectories of multiple small targets through data association and Kalman filtering; Step S3: traverse the tracking trajectory that is first confirmed as a moving target, extract its grayscale ratio features and geometric relative position features with other active tracking trajectories, and determine whether it is a ghost image by comparing it with the threshold feature parameters determined in the experiment.

2. The method for identifying ghost images in small target tracking in infrared image plane according to claim 1, characterized in that: The step S1 comprises: In the laboratory, by setting up a target with strong infrared radiation, ghost images will appear on the infrared image plane and data will be collected; or in the infrared surveillance imaging of actual scenes, data of randomly appearing strong radiation targets and ghost images will be collected; Analyze the infrared surveillance imaging data of the actual scene, manually identify and collect the data of the randomly appearing strong radiation targets and the accompanying ghost images, extract the grayscale value and coordinate value of the two on the image plane, and calculate the grayscale ratio of the two, as well as the difference between the two coordinates in the x and y directions respectively; Through statistics, 7 types of recognition features are formed, namely grayscale ratio threshold Th gray , the mean threshold range Th of the difference between the position coordinates in the x and y directions x,min ,Th x,max ,Th y,min ,Th y,max , and the threshold value of the standard deviation Th x,std ,Th y,std ; With the continuous accumulation of collected strong target and accompanying ghost image data, the recognition threshold features are iteratively updated.

3. The method for identifying ghost images in small target tracking in infrared image plane according to claim 1, characterized in that: The deep learning method used in step S2 to detect small targets is specifically as follows: by building and training a deep convolutional neural network, a probability heat map of each pixel in the output image belonging to the target is output, the probability heat map is thresholded and four-neighborhood connected clustering is performed to obtain the centroid position and pixel distribution map corresponding to each target, the background is estimated and removed by median filtering the original image, and a background removal residual map is further obtained, thereby obtaining a grayscale image corresponding to each target, and the sum of the grayscale distributions is taken as the grayscale value of the target.

4. The method for identifying ghost images in small target tracking in infrared image plane according to claim 1, characterized in that: The tracking trajectories of multiple small targets extracted by data association and Kalman filtering in step S2 are specifically as follows: a typical multi-target tracking technology is adopted, including Kalman filtering for single target state estimation and two-dimensional distribution data association for Kalman filtering prediction and detection results, and then combined with tracking start, maintenance and termination judgment; when executing tracking start, in addition to satisfying the detection results of multiple consecutive frames of association of the trajectory, the trajectory length must also exceed a set value; the trajectory that meets the tracking start condition and has not been terminated is marked as an active tracking trajectory, and the target trajectory that meets the tracking start condition for the first time is called the tracking trajectory confirmed as a moving target for the first time.

5. The method for identifying ghost images in small target tracking in infrared image plane according to claim 1, characterized in that: The step S3 comprises: If there is only one tracking track in the current activity, it is judged as a non-ghost image and the recognition process is terminated; otherwise, it turns to traversing other active tracks and records them as comparison tracks; If the starting time of the current trajectory is earlier than the starting time of the comparison trajectory, then proceed to traverse the next comparison trajectory; otherwise, further distinguish based on the grayscale contrast feature; calculate the grayscale ratio of each moment of the current trajectory to the corresponding moment of the comparison trajectory to form a grayscale ratio sequence, calculate the mean of the sequence, if the mean is less than the preset threshold, then proceed to traverse the next comparison trajectory, otherwise distinguish based on the geometric relative position feature; calculate the difference in the x and y directions of the coordinates of each moment of the current trajectory and the corresponding moment of the comparison trajectory to form two sequences, calculate the mean and standard deviation of the two sequences respectively, if the mean and standard deviation of the two sequences are both within the preset threshold range, it is determined to be a ghost image, and proceed to traverse the ghost image recognition process of the next tracking trajectory that is first confirmed as a moving target, otherwise proceed to traverse the next comparison trajectory; When all comparison tracks are traversed and the current track is not confirmed as a ghost image, it is determined that the current track is not a ghost image, and the ghost image recognition process of the next tracking track that is confirmed as a moving target for the first time is turned into traversal; when the ghost image recognition traversal of all tracking tracks that are confirmed as moving targets for the first time is completed, the process ends; The ghost image recognition process is embedded into the infrared image plane small target detection and tracking process, and is triggered when a moving target is first identified.

6. A ghost image recognition system for small target tracking in infrared image plane, characterized in that: include: Module M1: Through infrared image plane imaging test, the grayscale ratio and geometric relative position between the real target and the accompanying ghost image on the image plane are statistically analyzed to determine the recognition feature parameters; Module M2: Load the recognition feature parameters, then use the deep learning method to detect small targets, and then extract the tracking trajectories of multiple small targets through data association and Kalman filtering; Module M3: traverse the tracking trajectory that is first confirmed as a moving target, extract its grayscale ratio features and geometric relative position features with other active tracking trajectories, and determine whether it is a ghost image by comparing it with the threshold feature parameters determined in the experiment.

7. The ghost image recognition system in small target tracking in infrared image plane according to claim 6, characterized in that: The module M1 comprises: In the laboratory, by setting up a target with strong infrared radiation, ghost images will appear on the infrared image plane and data will be collected; or in the infrared surveillance imaging of actual scenes, data of randomly appearing strong radiation targets and ghost images will be collected; Analyze the infrared surveillance imaging data of the actual scene, manually identify and collect the data of the randomly appearing strong radiation targets and the accompanying ghost images, extract the grayscale value and coordinate value of the two on the image plane, and calculate the grayscale ratio of the two, as well as the difference between the two coordinates in the x and y directions respectively; Through statistics, 7 types of recognition features are formed, namely grayscale ratio threshold Th gray , the mean threshold range Th of the difference between the position coordinates in the x and y directions x,min ,Th x,max ,Th y,min ,Th y,max , and the threshold value of the standard deviation Th x,std ,Th y,std ; With the continuous accumulation of collected strong target and accompanying ghost image data, the recognition threshold features are iteratively updated.

8. The ghost image recognition system in small target tracking in infrared image plane according to claim 6, characterized in that: The module M2 uses a deep learning method to detect small targets as follows: a deep convolutional neural network is built and trained to output a probability heat map that each pixel in the image belongs to the target, the probability heat map is thresholded and four-neighborhood connected clustering is performed to obtain the centroid position and pixel distribution map corresponding to each target, the background is estimated and removed by median filtering the original image, and a background-removed residual map is further obtained, thereby obtaining a grayscale image corresponding to each target, and the sum of the grayscale distributions is taken as the grayscale value of the target.

9. The ghost image recognition system in small target tracking in infrared image plane according to claim 6, characterized in that: The module M2 specifically extracts the tracking trajectory of multiple small targets through data association and Kalman filtering as follows: adopts typical multi-target tracking technology, including Kalman filtering for single target state estimation and two-dimensional distribution data association for Kalman filtering prediction and detection results, and then combines tracking start, maintenance and termination judgment; when executing tracking start, in addition to satisfying the detection results of multiple consecutive frames of association of the trajectory, the trajectory length must also exceed the set value; the trajectory that meets the tracking start condition and has not been terminated is marked as an active tracking trajectory, and the target trajectory that meets the tracking start condition for the first time is called the tracking trajectory confirmed as a moving target for the first time.

10. The ghost image recognition system in small target tracking in infrared image plane according to claim 6, characterized in that: The module M3 comprises: If there is only one tracking track in the current activity, it is judged as a non-ghost image and the recognition process is terminated; otherwise, it turns to traversing other active tracks and records them as comparison tracks; If the starting time of the current trajectory is earlier than the starting time of the comparison trajectory, then proceed to traverse the next comparison trajectory; otherwise, further distinguish based on the grayscale contrast feature; calculate the grayscale ratio of each moment of the current trajectory to the corresponding moment of the comparison trajectory to form a grayscale ratio sequence, calculate the mean of the sequence, if the mean is less than the preset threshold, then proceed to traverse the next comparison trajectory, otherwise distinguish based on the geometric relative position feature; calculate the difference in the x and y directions of the coordinates of each moment of the current trajectory and the corresponding moment of the comparison trajectory to form two sequences, calculate the mean and standard deviation of the two sequences respectively, if the mean and standard deviation of the two sequences are both within the preset threshold range, it is determined to be a ghost image, and proceed to traverse the ghost image recognition process of the next tracking trajectory that is first confirmed as a moving target, otherwise proceed to traverse the next comparison trajectory; When all comparison tracks are traversed and the current track is not confirmed as a ghost image, it is determined that the current track is not a ghost image, and the ghost image recognition process of the next tracking track that is confirmed as a moving target for the first time is turned into traversal; when the ghost image recognition traversal of all tracking tracks that are confirmed as moving targets for the first time is completed, the process ends; The ghost image recognition process is embedded into the infrared image plane small target detection and tracking process, and is triggered when a moving target is first identified.

Images