A high-precision detection method for infrared sea surface targets based on detection-tracking-detection in time and space

Abstract

The present invention provides a high-precision detection method for infrared sea surface targets in a spatiotemporal joint based on detection-tracking-detection, comprising: inputting an infrared sea surface image sequence, using a multi-scale differential local peak detection method to detect each image in the sequence, and obtaining a single-frame detection result, wherein there are certain missed detections and false alarms; using pipeline filtering to filter the single-frame detection results to reduce the false alarm rate; if the filtered target is lost in the current image, using its trajectory in the previous sequence to predict the position in the current image to reduce the missed detection rate; taking the local image block where the predicted position is located and performing local target detection, judging the validity of the predicted position, and obtaining the final detection result. The present invention can effectively reduce the false alarm rate and missed alarm rate of target detection under a complex sea surface background, and solve the problems of weak targets and excessive background interference. The structure of parallel processing of each link can realize real-time processing of 640×512 pixel images.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a high-precision detection method for infrared sea surface targets based on detection-tracking-detection and spatiotemporal joint detection. Background Art

[0002] Infrared imaging technology has been widely used in the fields of ocean monitoring, military reconnaissance, and medical treatment. At present, in the ocean field, infrared is an important means of detecting sea surface targets. Accurate detection of targets in infrared images is a key prerequisite for target tracking and rescue and salvage, and is also an important topic that has received widespread attention. In addition, maritime accidents often occur in severe sea conditions, so the detection of low-significance targets in severe weather and complex backgrounds is still a valuable and challenging problem.

[0003] In recent years, there have been many studies on target detection algorithms. Most existing detection systems use two approaches: pre-tracking detection and post-tracking detection. Among them, pre-tracking detection first obtains potential targets through a single-frame detection method, and then uses a multi-frame method to remove false targets. This type of method can effectively suppress false alarms, but targets that are accidentally missed in a single frame may be completely eliminated during the multi-frame processing process, and the single-frame results cannot be corrected and supplemented, resulting in unnecessary missed detections. In addition, the method of tracking first and then detecting uses multi-frame information to determine possible target trajectories, and detects and judges based on the position of the trajectory in the current frame. This leads to a high complexity of the algorithm, making it difficult to achieve real-time detection. Summary of the invention

[0004] According to the above-mentioned technical problem of the difficulty of target detection under the background of strong sea waves, the present invention combines the characteristics of the two systems of pre-tracking detection and post-tracking detection, and proposes a spatiotemporal joint infrared sea surface target high-precision detection system based on "detection-tracking-detection". The main contents of the present invention include: providing a spatiotemporal joint infrared sea surface target high-precision detection system based on "detection-tracking-detection". Input an infrared sea surface image sequence, first, use a multi-scale differential local peak detection method to detect each image in the sequence, and obtain a single-frame detection result, in which there are certain missed detections and false alarms; then, use pipeline filtering to filter the single-frame detection results to reduce the false alarm rate; then, if the filtered target is lost in the current image, use its trajectory in the previous sequence to predict the position in the current image to reduce the missed detection rate; finally, take the local image block where the predicted position is located and perform local target detection, judge the validity of the predicted position, and obtain the final detection result. The present invention can effectively reduce the false alarm rate and missed alarm rate of target detection under complex sea surface background, and solve the problems of weak targets and strong background interference. By adopting a structure in which each link is processed in parallel, the present invention can realize real-time processing of a 640×512 pixel image.

[0005] The technical means adopted by the present invention are as follows:

[0006] A high-precision detection method for infrared sea surface targets based on detection-tracking-detection in time and space, comprising the following steps:

[0007] S1. Based on the differential Gaussian local peak detection method, single-frame target detection is performed on the input sequence image to obtain a single-frame detection result;

[0008] S2, based on the pipeline filtering method, multi-frame screening is performed on the single-frame detection result sequence to remove most of the false targets;

[0009] S3, performing inter-frame matching on the remaining targets after multi-frame screening. If a target is lost in the current frame, the trajectory of the lost target is predicted based on the matching result between the previous frames to obtain the predicted position;

[0010] S4. Take the neighborhood where the predicted position is located and perform local target detection to obtain the final detection result.

[0011] Furthermore, the specific implementation process of step S1 is as follows:

[0012] S11, use Gaussian filtering to obtain a smoother image background;

[0013] S12, subtract the original image from the filtered image to obtain a residual image containing the target and other high-frequency components, as shown in the following formula:

[0014] I res =I in -F(I in ) (1)

[0015] Among them, I res Represents the residual graph, I in represents the original input image, and F represents Gaussian filtering;

[0016] S13, the image is divided into 40×40 image blocks, and a salient area is found in each image block as a potential target; specifically, the portion of the image block that is higher than the block mean is retained, and then adaptive threshold segmentation is used to obtain the peak area in the current block, as shown in the formula:

[0017] I res = {P k},k=1,2,3... (2)

[0018] P m (i,j)=TH(p k (i,j)-mean(p k )) (3)

[0019] thk =aμ k +δ k (4)

[0020] Among them, p k is the segmented image block, and I res A subset of m is the image block that only retains the peak area, (i, j) is the pixel position, mean and TH are the averaging and threshold segmentation operations respectively; th k is the segmentation threshold of each image block, μ k and δ k are the mean and variance of the current image block, respectively. a is usually selected from [3,5];

[0021] S14, the obtained P m Splice them together and perform threshold segmentation again. The threshold this time is shown in formula (5):

[0022] th2=bμ+δ (5)

[0023] Among them, μ and δ are the mean and variance of the spliced ​​image respectively, and b is usually selected from [5,10]. The segmented image is the single-frame detection result R1.

[0024] Furthermore, the specific implementation process of step S2 is as follows:

[0025] Pipeline filtering retains targets that reach the pipeline length in the single-frame detection results, removes others, and allows the target position to jitter within a certain range.

[0026] Furthermore, the specific implementation process of step S3 is as follows:

[0027] S31, matching all targets that pass through pipeline filtering according to the principle of closest geometric distance and assigning unique IDs;

[0028] S32: In the current frame, if the marked target cannot be matched in the adjacent area, it is considered that the target is missed and trajectory prediction is required;

[0029] S33, according to the matched target positions before and after the missed target, the pixel position (m, n) of the suspected missed target in the current frame is calculated by using the least square method;

[0030] S34. The size of the target usually does not change significantly between adjacent frames. The size s1×s2 of the current suspected missed target is estimated based on the matched targets with the same ID.

[0031] Furthermore, the specific implementation process of step S4 is as follows:

[0032] Make a local decision on the pixel position (m,n) of the suspected missed target. in In the example, an image block with (m, n) as the center and 3s1×3s2 as the side length is taken, and the image block is used as an image block in step S1. The differential Gaussian local peak detection method in step S1 is used to perform single-frame target detection.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1. The spatiotemporal joint infrared sea surface target high-precision detection method based on detection-tracking-detection provided by the present invention combines the advantages of both pre-tracking detection and post-tracking detection methods, and improves the detection accuracy while controlling the computational complexity.

[0035] 2. The high-precision detection method for infrared sea surface targets based on detection-tracking-detection and spatiotemporal joint detection provided by the present invention can realize real-time detection and can be applied to engineering practice.

[0036] Based on the above reasons, the present invention can be widely promoted in the fields of image processing and the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 The figure is a flow chart of the method of the present invention.

[0039] Figure 2 A test image sequence provided by an embodiment of the present invention.

[0040] Figure 3 This is a single-frame detection binarization result provided by an embodiment of the present invention.

[0041] Figure 4 The multi-frame screening binarization result provided by the embodiment of the present invention.

[0042] Figure 5 The trajectory prediction local area and the judgment result provided by the embodiment of the present invention.

[0043] Figure 6 This is the final detection binarization result provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, the numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be clear that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The technology, methods and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, methods and equipment should be regarded as part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0048] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the device or element referred to must have a specific direction or be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the scope of protection of the present invention: the directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.

[0049] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below their position devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0050] In addition, it should be noted that the use of terms such as "first" and "second" to limit components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above terms have no special meaning and therefore cannot be understood as limiting the scope of protection of the present invention.

[0051] like Figure 1 As shown, the present invention provides a high-precision detection method for infrared sea surface targets based on detection-tracking-detection in time and space, comprising the following steps:

[0052] S1, based on the differential Gaussian local peak detection method, for input such as Figure 2 The sequence images shown in the figure (the framed part is the real target) are used for single-frame target detection to obtain the single-frame detection results, as shown in Figure 3 White area shown;

[0053] S2, based on the pipeline filtering method, multi-frame screening is performed on the single-frame detection result sequence to remove most of the false targets, and the screening results are as follows Figure 4 White area shown;

[0054] S3, after multi-frame screening, the remaining targets are matched between frames. If a target is lost in the current frame, the trajectory of the lost target is predicted based on the matching result between the previous frames to obtain the predicted position. The neighborhood of the predicted position is as follows: Figure 5 shown.

[0055] S4. Take the neighborhood where the predicted position is located and perform local target detection. The local detection results are shown in the figure Figure 5 As shown, the final detection result is obtained, such as Figure 6 The white area is shown.

[0056] In specific implementation, as a preferred embodiment of the present invention, the specific implementation process of step S1 is as follows:

[0057] S11. The target is usually in the high-frequency part of the image, so Gaussian filtering is used to obtain a smoother image background;

[0058] S12, subtract the original image from the filtered image to obtain a residual image containing the target and other high-frequency components, as shown in the following formula:

[0059] I res =I in -F(I in ) (1)

[0060] Among them, I res Represents the residual graph, I in represents the original input image, and F represents Gaussian filtering;

[0061] S13. In order to extract the real target from the residual image, according to the characteristic that the target is more significant in the local area, first, the image is divided into 40×40 image blocks, and the significant area is found in each image block as a potential target; specifically, the part of the image block that is higher than the block mean is retained, and then the adaptive threshold segmentation is used to obtain the peak area in the current block, as shown in the formula:

[0062] I res = {P k},k=1,2,3... (2)

[0063] P m (i,j)=TH(p k (i,j)-mean(p k )) (3)

[0064] th k =aμ k +δ k (4)

[0065] Among them, p k is the segmented image block, and I resA subset of m is the image block that only retains the peak area, (i, j) is the pixel position, mean and TH are the averaging and threshold segmentation operations respectively; th k is the segmentation threshold of each image block, μ k and δ k are the mean and variance of the current image block, respectively. a is usually selected from [3,5];

[0066] S14, the obtained P m Splice them together and perform threshold segmentation again. The threshold this time is shown in formula (5):

[0067] th2=bμ+δ (5)

[0068] Among them, μ and δ are the mean and variance of the spliced ​​image respectively, and b is usually selected from [5,10]. The segmented image is the single-frame detection result R1.

[0069] In specific implementation, as a preferred embodiment of the present invention, the specific implementation process of step S2 is as follows:

[0070] Pipeline filtering retains the targets that reach the length of the pipeline in the single-frame detection results, removes the others, and allows the target position to jitter within a certain range. Pipeline filtering retains the targets that reach the length of the pipeline in the single-frame detection results, removes the others, and allows the target position to jitter within a certain range. Therefore, the real targets are retained and the false targets are removed, and the multi-frame screening result R2 is obtained. However, the missed targets are filtered out at certain positions in the sequence images due to the occasional missed detection in the single-frame detection results, resulting in missed detection.

[0071] In specific implementation, as a preferred embodiment of the present invention, the specific implementation process of step S3 is as follows:

[0072] S31. All targets that pass through pipeline filtering are matched according to the principle of closest geometric distance and are assigned unique IDs.

[0073] S32: In the current frame, if the marked target cannot be matched in the adjacent area, it is considered that the target is missed and trajectory prediction is required.

[0074] S33, according to the matched target positions before and after the missed target, the pixel position (m, n) of the suspected missed target in the current frame is calculated by using the least square method;

[0075] S34. The size of the target usually does not change significantly between adjacent frames. The size s1×s2 of the current suspected missed target is estimated based on the matched targets with the same ID.

[0076] In specific implementation, as a preferred embodiment of the present invention, the specific implementation process of step S4 is as follows:

[0077] Make a local decision on the pixel position (m,n) of the suspected missed target. in In the example, an image block with (m, n) as the center and 3s1×3s2 as the side length is taken, and the image block is used as an image block in step S1. The differential Gaussian local peak detection method in step S1 is used to perform single-frame target detection.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-precision detection method for infrared sea surface targets based on detection-tracking-detection in time and space, characterized in that: The steps include: S1. Based on the differential Gaussian local peak detection method, single-frame target detection is performed on the input sequence image to obtain a single-frame detection result; The specific implementation process of step S1 is as follows: S11, use Gaussian filtering to obtain a smoother image background; S12, subtract the original image from the filtered image to obtain a residual image containing the target and other high-frequency components, as shown in the following formula: I res =I in -F(I in ) (1) Among them, I res Represents the residual graph, I in represents the original input image, and F represents Gaussian filtering; S13, the image is divided into 40×40 image blocks, and a salient area is found in each image block as a potential target; specifically, the portion of the image block that is higher than the block mean is retained, and then adaptive threshold segmentation is used to obtain the peak area in the current block, as shown in the formula: I res ={P k },k=1,2,3... (2) P m (i,j)=TH(p k (i,j)-mean(p k )) (3) th k =aμ k +δ k (4) Among them, p k is the segmented image block, and I res A subset of m is the image block that only retains the peak area, (i, j) is the pixel position, mean and TH are the averaging and threshold segmentation operations respectively; th k is the segmentation threshold of each image block, μ k and δ k are the mean and variance of the current image block, respectively. a is usually selected from [3,5]; S14, the obtained P m Splice them together and perform threshold segmentation again. The threshold this time is shown in formula (5): th2=bμ+δ (5) Among them, μ and δ are the mean and variance of the spliced ​​image, respectively, and b is usually selected from [5,10]. The segmented image is the single-frame detection result R1. S2, based on the pipeline filtering method, multi-frame screening is performed on the single-frame detection result sequence to remove most of the false targets; S3, performing inter-frame matching on the remaining targets after multi-frame screening. If a target is lost in the current frame, the trajectory of the lost target is predicted based on the matching result between the previous frames to obtain the predicted position; S4. Take the neighborhood where the predicted position is located and perform local target detection to obtain the final detection result.

2. The high-precision detection method for infrared sea surface targets based on detection-tracking-detection in time and space according to claim 1 is characterized in that: The specific implementation process of step S2 is as follows: Pipeline filtering retains targets that reach the pipeline length in the single-frame detection results, removes others, and allows the target position to jitter within a certain range.

3. The high-precision detection method for infrared sea surface targets based on detection-tracking-detection in time and space according to claim 1 is characterized in that: The specific implementation process of step S3 is as follows: S31, matching all targets that pass through pipeline filtering according to the principle of closest geometric distance and assigning unique IDs; S32: In the current frame, if the marked target cannot be matched in the adjacent area, it is considered that the target is missed and trajectory prediction is required; S33, according to the matched target positions before and after the missed target, the pixel position (m, n) of the suspected missed target in the current frame is calculated by using the least square method; S34. The size of the target usually does not change significantly between adjacent frames. The size s1×s2 of the current suspected missed target is estimated based on the matched targets with the same ID.

4. The high-precision detection method for infrared sea surface targets based on detection-tracking-detection in time and space according to claim 1 is characterized in that: The specific implementation process of step S4 is as follows: Make a local decision on the pixel position (m,n) of the suspected missed target. in In the example, an image block with (m, n) as the center and 3s1×3s2 as the side length is taken, and the image block is used as an image block in step S1. The differential Gaussian local peak detection method in step S1 is used to perform single-frame target detection.

Images