Infrared moving target detection method and device

Abstract

The present invention provides an infrared moving target detection method, device, equipment and medium, comprising: obtaining an infrared sequence image to be detected containing an infrared moving target, extracting the motion information of each frame in the infrared sequence image to be detected, and forming a motion information image; screening the target trajectory segments in the motion information image; predicting and associating the discrete trajectory segments in the target trajectory segments based on the radiation characteristics and motion trajectory characteristics of the infrared moving target, and constructing a target continuous trajectory; inverting the target continuous trajectory to the infrared sequence image to be detected, and locating the infrared moving target according to the inverted trajectory information.

Description

Technical Field

[0001] The present invention relates to the field of infrared image processing and target detection, and in particular to an infrared moving target detection method, device, equipment and storage medium. Background Art

[0002] Infrared search and tracking systems have been widely used in security, monitoring, remote sensing and other fields due to their all-day, high flexibility and anti-interference characteristics. Among them, the crucial target detection, especially the discovery and tracking of moving targets, and the continuous observation of trajectories, play a key role. At present, there are two main ideas for infrared target detection: one is detection first and then tracking (DBT), and the other is tracking first and then detection (TBD). Although DBT methods can achieve high detection accuracy, they are prone to false alarms and missed detections for complex scene problems. At this time, TBD methods can better and continuously detect targets by using motion characteristics and timing characteristics. However, the energy accumulation-based method in the TBD method is more suitable for simple and uniform backgrounds, and cannot distinguish interference points with strong radiation energy. It has low scene adaptability and high computational complexity. The other method based on spatiotemporal matching needs to analyze and model the spatiotemporal distribution characteristics of the target and background. Therefore, the more observation data it obtains, the more accurate the parameter estimation and the more accurate the modeling will be. This leads to a large amount of calculation and low algorithm efficiency when using longer sequence images for processing. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] In view of the above problems, the present disclosure provides an infrared moving target detection method, device, equipment and storage medium to at least partially solve the technical problems of missed detection, false alarm interference and large amount of calculation of infrared moving targets.

[0005] (II) Technical solution

[0006] The present disclosure provides an infrared moving target detection method, comprising: acquiring infrared sequence images to be detected containing infrared moving targets, extracting motion information of each frame of the infrared sequence images to be detected, and forming a motion information image; screening target trajectory segments in the motion information image; predicting and associating discrete trajectory segments in the target trajectory segments based on radiation characteristics and motion trajectory characteristics of the infrared moving targets, and constructing a target continuous trajectory; inverting the target continuous trajectory to the infrared sequence images to be detected, and locating the infrared moving target according to the inverted trajectory information.

[0007] According to an embodiment of the present disclosure, the method of acquiring an infrared sequence image to be detected of an infrared moving target, extracting motion information of each frame in the infrared sequence image to be detected, and forming a motion information image includes: subtracting images of consecutive frames in the infrared sequence image to be detected to obtain a plurality of frame difference images; based on a preset preliminary screening threshold, screening each of the frame difference images for candidate points to obtain a plurality of candidate point images containing the motion information; and spatiotemporally superimposing all of the candidate point images to obtain a motion information image containing a plurality of connected areas.

[0008] According to an embodiment of the present disclosure, the screening of target trajectory segments in the motion information image includes: performing attribute analysis on different connected areas of the motion information image and calculating geometric features of the connected areas; and screening all target trajectory segments in the motion information image based on the geometric features of the connected areas.

[0009] According to an embodiment of the present disclosure, based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, discrete trajectory segments in the target trajectory segments are predicted and associated to construct a target continuous trajectory, including: based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, a search area of ​​the infrared motion target is determined, and the search area is adaptively sampled to increase search points within the search area; the future trajectory of the infrared motion target is predicted according to the search points, and a predicted trajectory is generated; the curvature of the predicted trajectory is limited, and the direction change of the search points of the predicted trajectory is constrained to be less than or equal to a preset search range, so as to select search points to form a target predicted trajectory.

[0010] According to an embodiment of the present disclosure, based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, the discrete trajectory segments in the target trajectory segments are predicted and associated, and the construction of the target continuous trajectory also includes: smoothing the target predicted trajectory; selecting a search point in the target predicted trajectory that makes the value of a predefined target gradient potential energy function smaller, and selecting a search point in the target predicted trajectory that makes the cost value of a predefined target function the lowest, to construct the target continuous trajectory.

[0011] According to an embodiment of the present disclosure, the method further includes: judging whether they are the same target continuous trajectory according to the spatial distance between different target trajectory segments; and when they are the same target continuous trajectory, associating the target trajectory segments to obtain the target continuous trajectory.

[0012] According to an embodiment of the present disclosure, inverting the target continuous trajectory to the infrared sequence image to be detected, and locating the infrared moving target according to the inverted trajectory information includes: generating a trajectory area image according to the target continuous trajectory, and performing a point-by-point multiplication operation on the candidate point image corresponding to each frame of the infrared sequence image to be detected and the trajectory area image; analyzing the image obtained after the point-by-point multiplication, and extracting the position information of the infrared moving target in each frame of the image.

[0013] The second aspect of the present disclosure provides an infrared motion target detection device, comprising: an extraction module, used to obtain an infrared sequence image to be detected containing an infrared motion target, extract motion information of each frame of the infrared sequence image to be detected, and form a motion information image; a screening module, used to screen target trajectory segments in the motion information image; a construction module, used to predict and associate discrete trajectory segments in the target trajectory segments based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, and construct a target continuous trajectory; a positioning module, used to invert the target continuous trajectory to the infrared sequence image to be detected, and locate the infrared motion target according to the inverted trajectory information.

[0014] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more computer programs, which, when executed by the one or more processors, enables the one or more processors to execute the infrared motion target detection method.

[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the infrared moving target detection method.

[0016] (III) Beneficial effects

[0017] The infrared moving target detection method, device, equipment and storage medium provided by the present disclosure utilize the strong radiation characteristics and motion trajectory characteristics of the infrared moving target to establish a complete motion trajectory of the infrared moving target, thereby solving the problem of missing targets in certain frames and improving the detection rate of the target. At the same time, by superimposing the candidate point image and filtering the motion information image, the prediction and association of the target trajectory fragments are achieved at one time, avoiding the prediction of each frame image and saving the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] For a more complete understanding of the present disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 The flowchart of the infrared moving target detection method provided by the embodiment of the present disclosure is schematically shown;

[0020] Figure 2 A schematic diagram of a single-frame candidate point image and a motion information image provided by an embodiment of the present disclosure is schematically shown;

[0021] Figure 3 A schematic diagram of connected region analysis provided by an embodiment of the present disclosure is schematically shown;

[0022] Figure 4 A schematic diagram of a prediction trajectory growth search area provided by an embodiment of the present disclosure is schematically shown;

[0023] Figure 5 The schematic diagram of predicted trajectory growth provided by the embodiment of the present disclosure is schematically shown;

[0024] Figure 6 A schematic diagram of the predicted trajectory growth result provided by an embodiment of the present disclosure is schematically shown;

[0025] Figure 7 A schematic diagram of target accurate detection results provided by an embodiment of the present disclosure is schematically shown;

[0026] Figure 8 The structure block diagram of the infrared moving target detection device provided according to the embodiment of the present disclosure is schematically shown;

[0027] Fig. 9 The hardware structure diagram of an electronic device is schematically shown. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0032] Therefore, the technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable medium storing instructions, which can be used by an instruction execution system or in combination with an instruction execution system. In the context of the present disclosure, a computer-readable medium can be any medium that can contain, store, transmit, propagate, or transmit instructions. For example, a computer-readable medium can include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, device, or propagation medium. Specific examples of computer-readable media include: magnetic storage devices, such as magnetic tape or hard disk (HDD); optical storage devices, such as compact disk (CD-ROM); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0033] The inventor has found through research that in the relevant technology, infrared search and tracking systems have a wide range of applications in many fields. For example, by using infrared sensors to monitor vehicles, drones and other targets at night, traffic conditions and safety conditions can be grasped in real time. Therefore, infrared search and tracking systems have played a huge role in our lives, and the key target detection, especially the discovery and tracking of moving targets, and the continuous observation of trajectories, have played a key role. At present, there are two main ideas for infrared target detection: one is detection first and then tracking (DBT), and the other is tracking first and then detection (TBD).

[0034] DBT methods need to detect each frame of the image and then associate the detection results of the single frame. There are three main types of existing DBT methods: (1) detection methods based on background consistency assumptions; (2) detection methods based on visual saliency characteristics; and (3) detection methods based on low-rank sparse decomposition. Among them, the first type of method mainly assumes that the target and the background have obvious radiation differences, that is, grayscale differences, and then uses threshold analysis, modeling and other methods to extract the target from the background. However, this method that uses spatial grayscale differences is very sensitive to complex backgrounds and it is difficult to obtain an accurate segmentation threshold, so the false alarm and missed detection are high. The second type of method mainly relies on the principles of the human visual system. By extracting information entropy, contrast, spectrum and other features from the target, the target is effectively enhanced, thereby standing out from the background. However, this type of method requires pixel-by-pixel or local feature extraction of the target, which is computationally intensive and difficult to distinguish interference such as high radiation noise with the same characteristics as the target. The third type of method models the target and the background, uses low rank and sparsity to perform component decomposition through optimization methods, and thus detects the target. However, due to the complex solution process and constraints, its computational complexity is high and its practicality is poor. Although DBT methods can achieve high detection accuracy, they are prone to false alarms and missed detections for complex scene problems. Therefore, considering the limitations of DBT methods, many scholars have begun to study TBD methods, hoping to detect targets using the target's motion, trajectory, and time domain characteristics.

[0035] TBD methods detect targets after accumulating a certain amount of data. They mainly rely on the accumulation of time-series observation information, that is, after accumulating a certain amount of sequential images, they search for the trajectory of the moving target, and then distinguish the trajectory according to the characteristics that the target trajectory should meet, thereby obtaining the position of the target. This type of method mainly includes methods based on energy accumulation, methods based on spatiotemporal matching, and various filtering methods. Among them, the method based on energy accumulation mainly obtains the position of the target by finding the maximum value of energy accumulation at each moment. The method based on spatiotemporal matching mainly separates the target from the background according to the grayscale distribution characteristics of the target and the background in the spatial and temporal domains. Specifically, there are three-dimensional matching filtering, generalized likelihood ratio test and other methods. Compared with DBT methods, TBD methods can improve the reliability and robustness of target detection through the accumulation of multi-frame information, and realize more complete and continuous discovery of the target's motion position. However, the energy accumulation-based method generally assumes that the target moves in a straight line at a uniform speed, and needs to perform a frame-by-frame, traversal search in the area that meets the assumption. This makes this type of method more suitable for simple, uniform backgrounds, unable to distinguish interference points with strong radiation energy, low scene adaptability, and high computational complexity. The method based on spatiotemporal matching needs to analyze and model the spatiotemporal distribution characteristics of the target and background, so the more observation data it obtains, the more accurate the parameter estimation, and the more precise the modeling. This results in a large amount of calculation and low algorithm efficiency when using longer sequence images for processing.

[0036] The embodiments of the present disclosure overcome the deficiencies of the prior art and provide an infrared moving target detection method based on radiation characteristics and motion trajectory characteristics to solve the problems of missed detection, false alarm interference, and large amount of calculation of infrared moving targets.

[0037] like Figure 1 As shown, the flowchart of the infrared moving target detection method includes S1~S4.

[0038] In operation S1, a sequence of infrared images to be detected containing infrared moving targets is acquired, and motion information of each frame of the sequence of infrared images to be detected is extracted to form a motion information image.

[0039] In operation S2, target track segments in the motion information image are screened.

[0040] In operation S3, based on the radiation characteristics and motion trajectory characteristics of the infrared moving target, the discrete trajectory segments in the target trajectory segments are predicted and associated to construct a target continuous trajectory.

[0041] In operation S4, the target continuous trajectory is inverted into the infrared sequence image to be detected, and the infrared moving target is located according to the inverted trajectory information.

[0042] Specifically, for the infrared sequence images to be detected containing infrared moving targets, the frame difference method is first used to obtain the motion information in each frame of the image, and then the continuous target trajectory is extracted using the superposition screening method. Then, for the discrete trajectory segments, the trajectory segments are predicted and associated according to the radiation characteristics and motion trajectory characteristics of the infrared moving target, thereby completing the extraction of the target continuous trajectory. Finally, the associated target continuous trajectory is inverted to the sequence image to achieve the detection of the target in each frame of the image.

[0043] It can be understood that the infrared motion target detection method constructed in the embodiment of the present disclosure can effectively improve the problems of missed detection and false alarm of the target. At the same time, it can realize the prediction and association of the continuous trajectory of the target in one go, solving the problem of large amount of calculation caused by traversal and search of each frame of image.

[0044] Furthermore, an infrared sequence image to be detected of an infrared moving target is obtained, and the motion information of each frame image in the infrared sequence image to be detected is extracted to form a motion information image, including: subtracting images of consecutive frames in the infrared sequence image to be detected to obtain a plurality of frame difference images; based on a preset preliminary screening threshold, screening each of the frame difference images for candidate points to obtain a plurality of candidate point images containing the motion information; and spatiotemporally superimposing all the candidate point images to obtain a motion information image containing a plurality of connected areas.

[0045] Specifically, for the infrared sequence images to be detected, the images of consecutive frames are first subtracted to obtain frame difference images, so as to capture motion information by comparing the differences between adjacent frames. Then, considering that the target generally has strong infrared radiation characteristics, the frame difference images are screened for candidate points according to the set preliminary screening threshold to obtain a fast screened target candidate point image, such as Figure 2 As shown on the left. Then all candidate point images are superimposed in time and space to obtain a motion information image containing multiple connected areas, as shown in Figure 2 As shown on the right, these connected regions not only reflect the spatial position of the moving target, but also imply its continuity in time, thereby more accurately describing the motion trajectory and morphology of the moving target.

[0046] It should be noted that other spatial-domain-based rapid target detection methods can also be used when performing rough screening of candidate points, such as background subtraction, optical flow, and deep learning-based detection methods.

[0047] It is understandable that through the application of the frame difference method and the preliminary screening threshold, a large amount of background noise and irrelevant information can be quickly eliminated, the amount of calculation for subsequent processing can be reduced, and the detection efficiency can be significantly improved. At the same time, the method of spatiotemporal superposition effectively utilizes the temporal continuity and spatial position change information of the moving target in continuous frames, which can more accurately identify and locate the moving target and reduce false detection and missed detection.

[0048] Furthermore, the screening of the target trajectory segments in the motion information image includes: performing attribute analysis on different connected areas of the motion information image and calculating geometric features of the connected areas; and screening all the target trajectory segments in the motion information image based on the geometric features of the connected areas.

[0049] Specifically, since the target is in continuous motion, while the high radiation noise and background are relatively still, the target will appear as a connected strip in the motion information image, while other backgrounds and noise will appear as discrete points or connected surfaces. According to the above rules, the embodiment of the present disclosure performs attribute analysis on different connected regions and calculates the geometric features of the connected regions, thereby obtaining all target trajectory segments in the sequence by analyzing all regions in the motion information image.

[0050] For example, the lengths of the major and minor axes of the ellipse with the same normalized second-order central moment as the region are calculated, and then the major-minor axis ratio (geometric feature) is calculated, thereby obtaining the judgment feature of whether the connected region is strip-shaped, such as Figure 3 When the ratio of the major axis to the minor axis exceeds 5, the region is considered to be a strip-shaped target trajectory segment.

[0051] It is understandable that by calculating geometric features and applying screening criteria, it is possible to more accurately identify and extract target track segments in motion information images, which helps reduce false detections and missed detections and improves the accuracy and reliability of track segments. At the same time, the calculation of geometric features does not depend on specific target features or background models, so it can cope with complex and changing infrared image environments, allowing the system to maintain high detection performance under conditions of changing lighting or complex backgrounds, thereby enhancing the robustness of the system.

[0052] Furthermore, based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, discrete trajectory segments in the target trajectory segments are predicted and associated to construct a target continuous trajectory, including: based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, a search area of ​​the infrared motion target is determined, and the search area is adaptively sampled to increase search points within the search area; the future trajectory of the infrared motion target is predicted according to the search points, and a predicted trajectory is generated; the curvature of the predicted trajectory is limited, and the direction change of the search points of the predicted trajectory is constrained to be less than or equal to a preset search range, so as to select search points to form a target predicted trajectory.

[0053] After obtaining multiple target trajectory segments, since the target may not meet the preliminary screening threshold in some frames, it may cause the discontinuity of the trajectory, the target is lost in the middle frame and other problems, so it is necessary to predict and associate the trajectory segments to obtain the complete motion trajectory of the target. However, when accurately predicting and associating multiple trajectories in an image, it may happen that the coordinate positions of multiple trajectories are very close, which makes the multi-target trajectory association problem very complicated. Therefore, the embodiment of the present disclosure proposes a trajectory growth technology based on motion characteristics and radiation characteristics to achieve accurate prediction of multi-target trajectory segments and reduce the difficulty of subsequent association.

[0054] In some exemplary embodiments, according to the infrared target motion and radiation model, a reasonable target continuous trajectory should have an optimal objective function. Therefore, the disclosed embodiment intends to use dynamic programming (DP) to find the trajectory path and curve with the lowest comprehensive cost after each trajectory is predicted to grow. At the same time, considering that the target should meet the characteristics of strong radiation and smooth trajectory, the potential field value of the target trajectory should be composed of the radiation energy function and the curve elastic energy function, that is:

[0055]

[0056] in, represents the radiation energy function, and represents the trajectory curve morphology function, represents the modulation parameter. In addition, since the target has strong radiation, its gradient potential field in the image should also be large, and the trajectory of the target in a short time generally does not change greatly, and its trajectory shape is also smooth. Based on this assumption, the optimal trajectory energy function is found through optimization method, so as to reasonably predict the trajectory of multiple targets.

[0057] Specifically, to grow the trajectory of the target, the search point area should be established first and the calculation range of dynamic programming should be determined. Then the search area is sampled adaptively. In this process, considering the strong radiation characteristics of the target, the alternative point sampling method based on the gradient potential field is relatively easier to implement, more intuitive and faster. Therefore, with the help of the infrared radiation characteristics of the target, the search points in the area are increased through the image upsampling method. At the same time, considering that the target's track should move forward in a certain direction, the search point should be in the direction of the trajectory.

[0058] Furthermore, considering the turning and action state of the target, its trajectory cannot suddenly turn or fly backwards, so the curvature of the predicted trajectory is restricted. Based on the last time point of the trajectory The heading can be calculated ,definition is the center of the tangent circle in the positive direction, The center of the tangent circle in the negative direction is used to calculate the positions of the centers of the two circles with a radius of r:

[0059]

[0060]

[0061] Therefore, the maximum curvature of the predicted trajectory can be limited to , and according to these restrictions, it can be obtained that the target point P on the predicted trajectory should satisfy the following constraints:

[0062]

[0063]

[0064]

[0065]

[0066] in, Indicates the maximum search range. In order to ensure that the trajectory does not make a sudden turn or U-turn, It should not exceed 180°. The above formulas ensure that the subsequent search points are in the direction of the target prediction trajectory. Figure 4 As shown, the dotted parts arranged in an array are the predicted trajectory growth search areas after screening.

[0067] It should be noted that when performing frame difference, forward frame difference or backward frame difference can be used. Forward frame difference refers to calculating the pixel difference between the current frame and the previous frame. This method only needs to compare two adjacent frames, and the amount of calculation is relatively small. At the same time, due to the small amount of calculation, it is suitable for real-time video processing systems, and is insensitive to changes in illumination, and can resist slight changes in illumination to a certain extent. Backward frame difference is to calculate the pixel difference between the current frame and the next frame. In some cases, it can better capture the complete outline of the moving object, reduce the appearance of internal holes, and is suitable for predicting changes in future frames in specific video processing tasks.

[0068] Furthermore, based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, the discrete trajectory segments in the target trajectory segments are predicted and associated, and the target continuous trajectory construction also includes: smoothing the target predicted trajectory; selecting a search point in the target predicted trajectory that makes the value of a predefined target gradient potential energy function smaller, and selecting a search point in the target predicted trajectory that makes the cost value of a predefined target function the lowest, to construct the target continuous trajectory.

[0069] In some exemplary embodiments, the degree of trajectory bending or elastic energy may be defined according to the trajectory morphology, that is, In order to obtain a smooth trajectory, the trajectory curve should not have too much curvature, that is, the elastic energy function of the trajectory should be relatively small. Therefore, according to the trajectory consisting of a series of discrete points Composition, its shape energy can be defined as the direction of trajectory The relationship function with the distance d between two points:

[0070]

[0071] Among them, the distance . By constraining the elastic energy of the trajectory curve, a relatively smooth trajectory curve is established.

[0072] Furthermore, in addition to the trajectory shape, considering that the target has a strong radiation characteristic, its gradient potential energy should be large, and the growing search points should also have this characteristic. Therefore, the gradient potential energy function of the target is defined to ensure that the predicted points have strong radiation characteristics:

[0073]

[0074] It should be noted that if the search point is the background area, its gradient will be smaller, resulting in a larger value of the gradient potential energy function. When the search point is the target, its gradient will be larger, which will make the gradient potential energy function smaller and the energy optimal. Therefore, combining the target radiation characteristics and the trajectory curve morphological energy, the objective function of trajectory growth can be defined as:

[0075]

[0076] After the adaptive sampling area, the search area is searched based on the dynamic programming algorithm, hoping to link the sampling points and obtain the curve with the lowest function cost. The trajectory growth problem is converted into the problem of selecting trajectory points with the lowest cost value of the trajectory objective function from the alternative search points, so as to achieve the trajectory search with the lowest cost function and realize the reasonable prediction growth of the fracture trajectory, such as Figure 5 shown.

[0077] It can be understood that the combination of the adaptive sampling area and the dynamic programming algorithm significantly improves the precision and accuracy of trajectory construction, making the constructed continuous trajectory closer to the actual motion trajectory of the target.

[0078] In an embodiment of the present disclosure, the method further includes: judging whether they are the same target continuous trajectory according to the spatial distances between different target trajectory segments; and when they are the same target continuous trajectory, associating the target trajectory segments to obtain the target continuous trajectory.

[0079] After growing the trajectories, there may still be some unassociated situations. Therefore, in the embodiment of the present disclosure, a trajectory association strategy based on spatial distance is proposed. By judging whether the spatial distances between different trajectories are the same target trajectory, the target trajectories are associated, and then a complete target trajectory is obtained. Figure 6 shown.

[0080] It can be understood that judging whether a continuous trajectory belongs to the same target based on spatial distance can more accurately judge the continuity between trajectory segments, thereby avoiding incorrectly associating the trajectories of different targets together and effectively reducing trajectory misjudgment and missed judgment caused by target occlusion, detection error and other reasons.

[0081] Furthermore, inverting the target continuous trajectory to the infrared sequence image to be detected and locating the infrared moving target according to the inverted trajectory information includes: generating a trajectory area image according to the target continuous trajectory, and performing a point-by-point multiplication operation on the candidate point image corresponding to each frame of the infrared sequence image to be detected and the trajectory area image; analyzing the image obtained after the point-by-point multiplication, and extracting the position information of the infrared moving target in each frame of the image.

[0082] Specifically, after extracting the continuous trajectory of the target in the infrared sequence image to be detected and constructing a complete trajectory area image, the coarse-screened candidate point results of each frame of the image are used to perform Hadamard product with the trajectory area image. Specifically, for the points in the trajectory area, their corresponding Hadamard product results will be retained or enhanced; while for the points in the non-trajectory area, their product results will be close to zero or significantly reduced, and thus effectively suppressed. After the Hadamard product operation, a set of candidate points filtered by the trajectory area in each frame of the image is obtained. Compared with the original coarse-screened candidate points, these point sets have removed most of the non-target interference, making target detection more accurate.

[0083] like Figure 7 As shown in the figure, the left side is the result of the rough screening of candidate points. It can be seen that there are a large number of noise points and interference points around the target trajectory; the right side is the precise detection result based on the trajectory. After the Hadamard product operation, the interference points in the non-target trajectory area are effectively suppressed, and the target points are clearly presented.

[0084] It can be understood that the candidate point image of each frame in the infrared sequence image to be detected is multiplied point by point with the trajectory area image, and the trajectory area image is used as a mask to suppress other interference in the non-target trajectory area, further reduce false alarms, and complete accurate detection of the target in each frame.

[0085] Compared with DBT methods, which mainly rely on the grayscale difference between the target and the background in the space-time domain, but do not make full use of the motion characteristics of the target, resulting in missed detection when the difference between the target and the background characteristics is small, or false alarm when there is strong radiation interference in the background, the infrared moving target detection method provided by the disclosed embodiment uses the strong radiation characteristics and motion characteristics of the target to establish a complete motion trajectory of the target, thereby solving the problem of missing targets in certain frames and improving the detection rate of the target. In addition, the traditional energy accumulation-based method generally assumes that the target moves in a straight line at a uniform speed, and needs to perform a frame-by-frame, traversal search in the area that meets the assumption, resulting in this type of method being more suitable for simple and uniform backgrounds, and being unable to distinguish interference points with the same strong radiation energy, having low scene adaptability, and having high computational complexity. In the disclosed embodiment, a dynamic search is directly performed on the trajectory image of multiple frames superimposed to link the broken trajectory, avoiding the large amount of calculation required to predict each frame of the image.

[0086] Based on the above infrared moving target detection method, the present disclosure also provides an infrared moving target detection device. Figure 8 The device is described in detail.

[0087] like Figure 8 As shown, the infrared moving target detection device 800 of this embodiment includes an extraction module 801, a screening module 802, a construction module 803 and a positioning module 804.

[0088] The extraction module 801 is used to obtain a sequence of infrared images to be detected containing infrared moving targets, extract motion information of each frame of the sequence of infrared images to be detected, and form a motion information image.

[0089] The screening module 802 is used to screen the target track segments in the motion information image.

[0090] The construction module 803 is used to predict and associate the discrete trajectory segments in the target trajectory segments based on the radiation characteristics and motion trajectory characteristics of the infrared moving target, so as to construct a target continuous trajectory.

[0091] The positioning module 804 is used to invert the target continuous trajectory into the infrared sequence image to be detected, and locate the infrared moving target according to the inverted trajectory information.

[0092] It is understandable that the extraction module 801, the screening module 802, the construction module 803 and the positioning module 804 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the extraction module 801, the screening module 802, the construction module 803 and the positioning module 804 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or in a proper combination of software, hardware and firmware. Alternatively, at least one of the extraction module 801, the screening module 802, the construction module 803 and the positioning module 804 can be at least partially implemented as a computer program module, and when the program is run by a computer, the function of the corresponding module can be executed.

[0093] Fig. 9 The structural block diagram of an electronic device provided by an embodiment of the present disclosure is schematically shown.

[0094] like Fig. 9 As shown, the electronic device described in this embodiment includes: the electronic device 900 includes a processor 910 and a computer readable storage medium 920. The electronic device 900 can execute the above reference Figure 1 Describes the method to achieve detection of specific operations.

[0095] Specifically, the processor 910 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 910 may also include an onboard memory for cache purposes. The processor 910 may be a processor for executing reference Figure 1 The described method flow according to the embodiment of the present disclosure may be a single processing unit or multiple processing units for different actions.

[0096] Computer readable storage medium 920, for example, can be any medium capable of containing, storing, conveying, propagating or transmitting instructions. For example, readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices, devices or propagation media. Specific examples of readable storage media include: magnetic storage devices, such as magnetic tapes or hard disks (HDDs); optical storage devices, such as compact disks (CD-ROMs); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0097] The computer readable storage medium 920 may include a computer program 921, which may include code / computer executable instructions, which when executed by the processor 910 causes the processor 910 to perform, for example, the above-mentioned combination Figure 1 The described method sequences and any variations thereof.

[0098] The computer program 921 may be configured to have, for example, computer program codes including computer program modules. For example, in an exemplary embodiment, the codes in the computer program 921 may include one or more program modules, for example, 921A, module 921B, ... It should be noted that the division method and number of modules are not fixed, and those skilled in the art may use appropriate program modules or program module combinations according to actual conditions. When these program module combinations are executed by the processor 910, the processor 910 may execute, for example, the above combination of Figure 1 to Figure 7 The described method sequences and any variations thereof.

[0099] According to an embodiment of the present invention, at least one of the extraction module 801, the screening module 802, the construction module 803 and the positioning module 804 can be implemented as a reference Fig. 9 The computer program modules described herein, when executed by the processor 910, can implement the corresponding operations described above.

[0100] The present disclosure also provides a computer-readable medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0101] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0102] Although the present disclosure has been shown and described with reference to specific exemplary embodiments of the present disclosure, it should be understood by those skilled in the art that various changes in form and details may be made to the present disclosure without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents. Therefore, the scope of the present disclosure should not be limited to the above-mentioned embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A method for detecting infrared moving targets, characterized in that: include: Acquire a sequence of infrared images to be detected containing infrared moving targets, extract motion information of each frame of the sequence of infrared images to be detected, and form a motion information image; Filtering target track segments in the motion information image; Based on the radiation characteristics and motion trajectory characteristics of the infrared moving target, the discrete trajectory segments in the target trajectory segments are predicted and associated to construct a target continuous trajectory; Specifically, the method includes: determining a search area of ​​the infrared moving target based on the radiation characteristics and the motion trajectory characteristics of the infrared moving target, and adaptively sampling the search area to increase the search points in the search area; predicting the future trajectory of the infrared moving target according to the search points, and generating a predicted trajectory; limiting the curvature of the predicted trajectory, and constraining the direction change of the search points of the predicted trajectory to be less than or equal to a preset search range, so as to select search points to form a target predicted trajectory; The step of predicting and associating discrete trajectory segments in the target trajectory segments based on the radiation characteristics and motion trajectory characteristics of the infrared moving target to construct the target continuous trajectory further includes: smoothing the target predicted trajectory; selecting a search point in the target predicted trajectory that makes the value of a predefined target gradient potential energy function smaller, and selecting a search point in the target predicted trajectory that makes the cost value of a predefined target function the lowest, to construct the target continuous trajectory; The target continuous trajectory is inverted into the infrared sequence image to be detected, and the infrared moving target is located according to the inverted trajectory information.

2. The infrared moving target detection method according to claim 1, characterized in that: The step of acquiring a sequence of infrared images to be detected containing infrared moving targets, extracting motion information of each frame of the sequence of infrared images to be detected, and forming a motion information image comprises: Subtracting images of consecutive frames in the infrared sequence image to be detected to obtain a plurality of frame difference images; Based on a preset preliminary screening threshold, screening candidate points for each of the frame difference images to obtain a plurality of candidate point images containing the motion information; All the candidate point images are temporally and spatially superimposed to obtain a motion information image containing multiple connected areas.

3. The infrared moving target detection method according to claim 1, characterized in that: The screening of the target track segments in the motion information image comprises: Performing attribute analysis on different connected areas of the motion information image to calculate geometric features of the connected areas; Based on the geometric features of the connected area, all target track segments in the motion information image are screened.

4. The infrared moving target detection method according to claim 1, characterized in that: The method further comprises: Determining whether they are continuous trajectories of the same target based on the spatial distances between different target trajectory segments; When the target continuous trajectory is the same, the target trajectory segments are associated to obtain the target continuous trajectory.

5. The infrared moving target detection method according to claim 1, characterized in that: The step of inverting the target continuous trajectory into the infrared sequence image to be detected, and locating the infrared moving target according to the inverted trajectory information comprises: Generate a track area image according to the target continuous track, and perform a point-by-point multiplication operation on the candidate point image corresponding to each frame of the infrared sequence image to be detected and the track area image; The image obtained after point-by-point multiplication is analyzed to extract the position information of the infrared moving target in each frame of the image.

6. An infrared moving target detection device, characterized in that: The device comprises: An extraction module is used to obtain a sequence of infrared images to be detected containing infrared moving targets, extract motion information of each frame of the sequence of infrared images to be detected, and form a motion information image; A screening module, used for screening target track segments in the motion information image; A construction module is used to predict and associate discrete trajectory segments in the target trajectory segment based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, and construct a target continuous trajectory; specifically including: determining the search area of ​​the infrared motion target based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, and adaptively sampling the search area to increase the search points in the search area; predicting the future trajectory of the infrared motion target according to the search points, and generating a predicted trajectory; limiting the curvature of the predicted trajectory, and constraining the direction change of the search points of the predicted trajectory to be less than or equal to a preset search range, so as to select search points to form a target predicted trajectory; based on the radiation characteristics and motion trajectory characteristics of the infrared motion target, predicting and associating discrete trajectory segments in the target trajectory segment, constructing a target continuous trajectory also includes: smoothing the target predicted trajectory; selecting a search point in the target predicted trajectory that makes the value of a predefined target gradient potential energy function smaller, and selecting a search point in the target predicted trajectory that makes the cost value of a predefined target function the lowest, to construct the target continuous trajectory; The positioning module is used to invert the target continuous trajectory into the infrared sequence image to be detected, and locate the infrared moving target according to the inverted trajectory information.

7. An electronic device comprising: one or more processors; a storage device for storing one or more computer programs, It is characterized in that when the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having executable instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor executes the method according to any one of claims 1 to 5.

Images