An infrared image saliency extraction method for ship detection

Abstract

The patent discloses a kind of infrared image saliency extraction method for ship detection.The specific implementation steps are as follows:1.based on minimum spanning tree superpixel segmentation;2.based on the generation of infrared image background seed region of ship shape feature;3.infrared image global saliency calculation;4.superpixel adjacency matrix calculation;5.based on sea, ship texture characteristics saliency correction;6.infrared image saliency region extraction.This method uses minimum spanning tree algorithm to accelerate superpixel segmentation, considers the imaging characteristics of infrared image, generates background seed region according to infrared image characteristics and ship shape feature, carries out nonlinear mapping to superpixel gray distance and geometric distance, calculates image global saliency, calculates the local texture contrast of each superpixel by superpixel neighborhood relationship and sea surface, ship texture difference, corrects infrared image saliency, realizes the extraction of final saliency region, and lays foundation for subsequent image feature extraction and target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of saliency extraction of two-dimensional images, in particular to a saliency extraction method for ships in infrared images. BACKGROUND

[0002] Infrared thermal imaging is a passive detection imaging technology based on the thermal radiation received in the camera field of view, and its imaging effect has no requirement on the lighting conditions. Objects with a temperature higher than absolute zero will autonomously emit radiation in the infrared spectrum band. Infrared imaging detection devices can achieve continuous passive detection day and night, and have strong anti-low-altitude attack, anti-electromagnetic interference and anti-stealth capabilities. Compared with visible light, infrared has strong smoke penetration capability and can work all day; compared with synthetic aperture radar, infrared imaging passively receives radiation, has good concealment and stronger safety. Therefore, infrared imaging has excellent applicability in complex sea conditions.

[0003] When detecting and identifying ships through infrared sensors, there are often many background interferences, and the existence of sea waves, clouds and other clutter further increases the difficulty of infrared ship detection and identification. Effective extraction of the region of interest can improve such problems, but direct use of simple threshold segmentation methods often fails to obtain reliable results, and use of complex graph model algorithms, mean shift algorithms and fuzzy mean algorithms will lead to a sharp increase in computational complexity. Therefore, the present application accelerates processing through superpixel segmentation based on a minimum spanning tree, considers the imaging characteristics of infrared images, generates a background seed region according to the gray level characteristics of the infrared image and the ship shape features, performs nonlinear mapping on the superpixel gray level distance and geometric distance, calculates the global saliency of the image, calculates the local texture contrast of each superpixel through the neighborhood relationship of the superpixels and the texture difference between the sea surface and the ship, and corrects the saliency of the image to realize extraction of the final saliency region. SUMMARY

[0004] The present application aims to face ships in infrared images and provide a fast saliency calculation method, the key of which is to generate a reasonable background seed region and select a suitable saliency calculation method and correction strategy.

[0005] The present application follows the following technical solution: an infrared image saliency extraction method for ship detection, characterized by comprising the following steps:

[0006] Step 1: superpixel segmentation based on a minimum spanning tree;

[0007] Step 2: extraction of a background seed region by comprehensively utilizing the characteristics of infrared images and the geometric features of ships;

[0008] The darker superpixel region in the image is taken as a basic background seed region, and then the superpixel region similar to the ship in shape is excluded according to the length-width ratio of the minimum enclosing rectangle of the superpixel region, to obtain the background seed region; the composition of the background seed region A is shown in formula (1),

[0009]

[0010] Wherein: A gray is a basic background seed region set, A shape is a shape constraint seed region set, a gray , a shape is a superpixel element constituting the set; mean(a) is the average gray value of the superpixel region a, TH is the best segmentation threshold obtained by OTSU segmentation of the infrared image, l(a) is the length-width ratio of the minimum enclosing rectangle of the superpixel region a, l1 and l2 are the prior range of the length-width ratio of the ship, which is related to the specific type of the ship, l1 is 1.8-2.5, and l2 is 5-12; A gray and the intersection of A shape constitute the background seed region A;

[0011] Step 3: Global saliency calculation of the infrared image;

[0012] Step 4: Superpixel adjacency matrix calculation;

[0013] Step 5: Saliency correction based on sea and ship texture characteristics; the gradient is used to define the superpixel texture richness, and the difference in texture richness between each superpixel and the neighborhood is used to correct the global saliency of the image, highlighting the ship target;

[0014] Step 6: Infrared image saliency region extraction.

[0015] Further, in step 5: first, calculate the gradient of each pixel in 0°, 45°, 90° and 135° directions, and then calculate the mean value of the four direction gradients in each superpixel region, and take the maximum value as the superpixel texture richness, denoted as μ.

[0016] The saliency correction is shown in formula (2),

[0017] Sa' = H x Sa (4)

[0018] Wherein Sa, Sa' are the superpixel saliency vectors before and after correction, and each element in the vector represents the saliency of each superpixel region; H is a superpixel local contrast vector, which is calculated as shown in formula (3), and each element in the vector represents the texture correction coefficient of the corresponding superpixel region, wherein μ ai , μ NB(ai) are the texture richness and the superpixel a i , respectively.i The texture richness of the neighborhood, where σ is the standard deviation of the texture richness of all superpixels in the image; when the texture richness of a superpixel differs significantly from that of its surrounding superpixels, the saliency of the superpixel will be enhanced.

[0019]

[0020] Furthermore, in step 6: after obtaining the significance result of the local correction, normalization is performed, and the result is divided into L intervals with successively increasing significance for statistical analysis, where the number of superpixels n satisfies

[0021] n = n0 + n1 + ... + n L-1 , where n i Let Sa be the number of superpixels in the i-th interval, and let Sa be the probability that a superpixel in Sa is salient in the i-th interval. If a threshold T(k) is used to divide the first k intervals of a superpixel into non-significant regions B, and the remaining intervals into significant regions F, then the probability that a superpixel is divided into non-significant region B is P. B (k), The probability of being classified as a salient region is P. F (k), P F (k)=1-P B (k); the average saliency m of the superpixels assigned to the non-saliency region B and the saliency region F. B m F for:

[0022]

[0023] Where m k The average significance of the number of superpixels relative to the total number of superpixels in the first k intervals, m G The average significance of all superpixels, and the inter-class variance of the non-significant region B and the significant region F are:

[0024]

[0025] Make the inter-class variance χ 2 The largest T(k) is the optimal significance threshold.

[0026] The advantages of this invention compared to the prior art are:

[0027] (1) The present invention uses a superpixel segmentation algorithm based on minimum spanning tree to segment infrared images. This method can speed up the calculation without destroying the overall shape of the target.

[0028] (2) The background seed region is generated based on the characteristics of infrared images and the geometric features of ships. This method is simple and easy to implement, and can enhance the stability of the algorithm without involving complex logic.

[0029] (3) The global saliency results are corrected by the difference between sea and ship textures and the superpixel nearest neighbor relationship. This method can effectively improve the saliency region extraction effect in the sea background. Attached image description:

[0030] Figure 1 : Flowchart of method implementation.

[0031] Figure 2 : Contains infrared images of the sea surface containing ships.

[0032] Figure 3 : Results of local significance correction.

[0033] Figure 4 : Results of salient region extraction. Detailed implementation method:

[0034] The following is in conjunction with the appendix Figure 1 The technical solution of the present invention will be described in detail below.

[0035] like Figure 1 As shown, this embodiment provides a method for saliency extraction of infrared images for ship detection, specifically including the following steps:

[0036] Step 1: Perform superpixel segmentation based on the minimum spanning tree.

[0037] If we consider the image I as a standard four-connected undirected graph G, and each pixel as a node v of the graph, and use the absolute value of the gray difference between nodes as the weight w of the edge e between two nodes, and let V and E be the set of v and e, then the undirected graph is represented as shown in equation (3), where (x,y) is the position of node v in the image.

[0038]

[0039] The Boruvka algorithm is used to generate the minimum spanning tree corresponding to the undirected graph G. The undirected graph G is considered as a forest with n subtrees, where the root of each subtree corresponds to a pixel node, and the distance between subtrees C1 and C2 is...

[0040]

[0041] Treat each subtree as graph G a node v a The distance between nodes is used as the edge e between nodes. a weight w a A depth-first search method is used to search G. aThe nodes in the subtree are merged until k nodes remain. At this point, the k subtrees correspond to the k superpixels obtained from the segmentation.

[0042] Step 2: Generation of background seed regions from infrared images based on ship shape features

[0043] First, the darker superpixel areas in the image are used as the basic background seed regions. Then, by utilizing the geometric characteristics of the ship, and based on the aspect ratio of the minimum bounding rectangle of the superpixels, superpixel areas with shapes similar to the ship are excluded to obtain the background seed regions.

[0044] The background seed region A is constructed as shown in equation (5), where A gray A is a set of basic background seed regions. shape For the set of shape-constrained seed regions, a gray a shape Let A be the superpixel element that makes up the set. mean(a) is the average gray value of superpixel region a, TH is the optimal segmentation threshold obtained by OTSU segmentation of the infrared image, l(a) is the aspect ratio of the minimum bounding rectangle of superpixel region a, and l1 and l2 are the prior ranges of the ship's aspect ratio, which are related to the specific ship type; typically, l1 takes values ​​of 1.8-2.5, and l2 takes values ​​of 5-12. gray With A shape The intersection of these regions forms the background seed region.

[0045]

[0046] Step 3: Calculation of global saliency of infrared image

[0047] After obtaining the background seed region A, the saliency calculation method for each superpixel region is shown in Equation (6).

[0048]

[0049] Where: Sa is the superpixel saliency vector of the image, and p is the number of background seed regions. For superpixel region a i a j The center coordinates of , || ||2 is the 2-norm, σ1 and σ2 are significance constant factors, satisfying σ1>0, σ2>0.

[0050] Step 4: Complete the superpixel adjacency matrix calculation

[0051] For a superpixel region a, let its center coordinates be r. a The neighborhood NB(a) of region a consists of other superpixel regions that share a common boundary with region a and whose center-to-center distance is less than the distance R.

[0052] NB(a)={b|||ra -r b ||2 <R} (7)

[0053] Where b is the superpixel region sharing a common boundary with a, r a r b Let be the center coordinates of superpixel regions a and b. Based on this, the superpixel adjacency matrix F is shown in equation (8), where a i a j For the i-th and j-th superpixel regions.

[0054]

[0055] Step 5: Saliency Correction Based on Infrared Ship and Sea Texture Characteristics

[0056] This step utilizes gradients along different directions to reflect differences in image texture. Ships typically have more complex textures than the sea surface. By correcting the global saliency of the image based on the gradient differences between superpixels and their neighbors, the ship target is highlighted.

[0057] First, calculate the gradient of each pixel in the four directions of 0°, 45°, 90°, and 135°. Then, calculate the average gradient of the four directions in each superpixel region and use the maximum value to represent the texture richness of the superpixel, denoted as μ.

[0058] The saliency correction is shown in equation (7). Where Sa and Sa' are the superpixel saliency vectors before and after correction, and each element in the vector represents the saliency of each superpixel region.

[0059] Sa'=H×Sa (9)

[0060] H is the local contrast vector of the superpixel, which is calculated as shown in equation (8). Each element in the vector represents the texture correction coefficient of the corresponding superpixel region, where... Superpixel a i Texture richness and superpixel a i The texture richness of the neighborhood, σ is the standard deviation of the texture richness of all superpixels in the image. When the texture richness of a superpixel differs significantly from that of its surrounding superpixels, the saliency of that superpixel will be enhanced. Since the texture richness of ships is higher than that of the sea surface, the ship target can be highlighted through the correction in equation (7).

[0061]

[0062] Step 6: Complete the extraction of salient regions

[0063] After obtaining the significance results of the local correction, normalization is performed, and the results are divided into L intervals with increasing significance for numerical statistics. Then, the number of superpixels n satisfies n = n0 + n1 + ... + nL-1 , where n i Let Sa be the number of superpixels in the i-th interval, and let Sa be the probability that a superpixel in Sa is salient in the i-th interval. If a threshold T(k) is used to divide the first k intervals of a superpixel into a non-significant region B and a significant region F, then the probability that a superpixel is classified into the non-significant region B is P. B (k), The probability of being classified as a salient region is P. F (k), P F (k)=1-P B (k); the average saliency m of the superpixels assigned to the non-saliency region B and the saliency region F. B m F for:

[0064]

[0065] Where m k The average significance of superpixels within the first k intervals, m G Given the average significance of all superpixels, the inter-class variance of the non-significant region B and the significant region F is:

[0066]

[0067] Make the inter-class variance χ 2 The largest T(k) is the optimal significance threshold.

[0068] To verify the effectiveness of infrared image saliency extraction for ship detection, a saliency extraction experiment based on infrared images of ships on the sea surface was conducted. Figure 2 This is an infrared image of the sea surface. Besides the high-grayscale area where the ship is located, there are also numerous bright spots caused by sea surface reflection. After global saliency calculation and local saliency correction, the final saliency result of the image is as follows: Figure 3 As shown, Figure 4 The final salient region extraction result is consistent with the region where the ship is located, which meets the experimental expectations.

[0069] In summary, this invention accelerates superpixel segmentation using the minimum spanning tree algorithm, takes into account the imaging characteristics of infrared images, generates background seed regions based on infrared image characteristics and ship shape features, performs nonlinear mapping on superpixel grayscale distance and geometric distance, calculates global image saliency, and corrects image saliency through superpixel neighborhood relationships and differences in sea surface and ship textures, thereby achieving the extraction of the final salient region. Experimental results also verify the good effect of the method proposed in this invention.

[0070] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A method for saliency extraction from infrared images for ship detection, characterized in that... Includes the following steps: Step 1: Perform superpixel segmentation based on the minimum spanning tree; Step 2: Extract background seed regions by comprehensively utilizing infrared image characteristics and ship geometric features; The darker superpixel regions in the image are used as the basic background seed regions. Then, based on the geometric characteristics of the ship, and according to the aspect ratio of the minimum bounding rectangle of the superpixels, superpixel regions with similar shapes to the ship are excluded to obtain the background seed regions. The composition of the background seed region A is shown in Equation (1). Among them: A gray A is a set of basic background seed regions. shape For the set of shape-constrained seed regions, a gray a shape Let A be the superpixel element that makes up the set; mean(a) is the average gray value of superpixel region a; TH is the optimal segmentation threshold obtained by OTSU segmentation of the infrared image; l(a) is the aspect ratio of the minimum bounding rectangle of superpixel region a; l1 and l2 are the prior ranges of the ship's aspect ratio, which are related to the specific ship type, with l1 ranging from 1.8 to 2.5 and l2 ranging from 5 to 12; gray With A shape The intersection of these regions constitutes the background seed region A; Step 3: Calculate the global saliency of the infrared image; Step 4: Calculate the superpixel adjacency matrix; Step 5: Saliency correction based on sea and ship texture characteristics; use gradient to define superpixel texture richness, and use the difference in texture richness between each superpixel and its neighborhood to correct the global saliency of the image and highlight the ship target. Step 6: Extraction of salient regions from infrared images.

2. The infrared image saliency extraction method for ship detection as described in claim 1, characterized in that: In step 5: First, calculate the gradient of each pixel in four directions: 0°, 45°, 90°, and 135°. Then, calculate the mean of the gradients in these four directions within each superpixel region, and use the maximum value to characterize the superpixel texture richness, denoted as μ. The significance correction is shown in equation (2). Sa'=H×Sa (2) Where Sa and Sa' are the superpixel saliency vectors before and after correction, and each element in the vector represents the saliency of each superpixel region; H is the superpixel local contrast vector, which is calculated as shown in equation (3), and each element in the vector represents the texture correction coefficient of the corresponding superpixel region, where μ ai ,μ NB(ai) Superpixel a i Texture richness and superpixel a i The texture richness of the neighborhood, where σ is the standard deviation of the texture richness of all superpixels in the image; when the texture richness of a superpixel differs significantly from that of its surrounding superpixels, the saliency of the superpixel will be enhanced.

3. The infrared image saliency extraction method for ship detection as described in claim 1, characterized in that: In step 6: After obtaining the significance results of the local correction, normalization is performed, and the results are divided into L intervals with increasing significance for statistical analysis. The number of superpixels n satisfies n = n0 + n1 + ... + n L-1 , where n i Let Sa be the number of superpixels in the i-th interval, and let Sa be the probability that a superpixel in Sa is salient in the i-th interval. If a threshold T(k) is used to divide the first k intervals of a superpixel into non-significant regions B, and the remaining intervals into significant regions F, then the probability that a superpixel is divided into non-significant region B is P. B (k), The probability of being classified as a salient region is P. F (k), P F (k)=1-P B (k); the average saliency m of the superpixels assigned to the non-saliency region B and the saliency region F. B m F for: Where m k The average significance of the number of superpixels relative to the total number of superpixels in the first k intervals, m G The average significance of all superpixels, and the inter-class variance of the non-significant region B and the significant region F are: Make the inter-class variance χ 2 The largest T(k) is the optimal significance threshold.

Images