Water surface dynamic weak target tracking method based on dynamic and static combination attention

Abstract

The invention discloses a water surface dynamic weak target tracking method based on dynamic and static combination attention, and relates to the technical field of computer vision, and the method comprises the steps: firstly obtaining a clear image which is successfully defogged based on a prior defogging algorithm of transmittance indirect estimation, and reducing the interference influence of complex foggy weather on the sea on optical target recognition; a spatial self-attention feature graph is obtained based on a spatial self-attention feature extraction algorithm; and finally, tracking and positioning the dynamic weak target based on a dynamic and static combination attention target tracking algorithm. According to the water surface dynamic weak target tracking method based on dynamic and static combination attention provided by the invention, the real-time performance and accuracy of dynamic weak target tracking can be improved, and the target type and the detection range of sea surface early warning investigation are expanded.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for tracking a dynamic weak target on a water surface based on combining static and dynamic attention. Background Art

[0002] In terms of weak target detection on the water surface, the light collection is greatly attenuated due to the refraction and scattering of sunlight, and the clarity of water surface imaging is significantly reduced. Before target tracking and recognition, we should focus on the image optimization processing of the water surface in foggy weather. The existing defogging methods mainly include three categories: image enhancement, image restoration, and deep learning. The above traditional algorithms are not effective in distinguishing the boundaries between foggy water surface and foggy land. Due to the lack of public water surface image datasets, it takes a lot of computing time to separate water surface targets and interfering backgrounds, and the detection rate and recognition accuracy are poor.

[0003] In the field of dynamic weak target tracking and recognition research, currently widely used algorithms include deep learning, Kalman filtering, YOLO network, multimodal collaborative detection, etc. The basic idea is to obtain target feature information through sample learning training, and then predict the target positioning information based on the temporal causal relationship between the previous and next frames of the video stream. Although the existing algorithms have made certain progress, there are two main problems: first, in complex interference environments, the dynamic weak target tracking performance is unstable and discontinuous, and the target positioning accuracy needs to be improved; second, due to the large number of target categories, traditional recognition methods are more dependent on the training effect of previous algorithms, and the amount of calculation is large, and the recognition effect for targets of unknown type and irregular shape is poor. Therefore, there is an urgent need for a surface dynamic weak target tracking method based on the combination of static and dynamic attention to solve the shortcomings of the above-mentioned existing technologies. Summary of the invention

[0004] The purpose of the present invention is to provide a method for tracking dynamic weak targets on the water surface based on combined dynamic and static attention, so as to solve the problems of unstable and discontinuous dynamic weak target tracking performance, poor target positioning accuracy and large amount of calculation in the prior art.

[0005] To achieve the above object, the present invention provides a method for tracking a dynamic weak target on a water surface based on combining static and dynamic attention, comprising the following steps: Step 1: Obtain a clear image with successful defogging based on a priori defogging algorithm based on indirect estimation of transmittance. On the premise of obtaining prior knowledge of defogging, firstly construct a physical model of foggy imaging, then indirectly estimate the transmittance through the atmospheric light dissipation function, and finally obtain a clear image with successful defogging. Step 2: Obtain a spatial self-attention feature map based on a spatial self-attention feature extraction algorithm; adaptively adjust the static image frame through autonomous learning, use global maximum pooling and global average pooling to calculate the depth value of the correlation between features, and obtain a spatial self-attention feature map; Step 3: Track and locate dynamic weak targets based on the static and dynamic combined attention target tracking algorithm; obtain the depth feature values ​​of static and dynamic image frames respectively through the spatial self-attention feature extraction algorithm, and then analyze the correlation between the two, and finally dynamically update the estimation according to the target response scale.

[0006] Preferably, the process of obtaining a clear image with successful defogging by using the priori defogging algorithm based on indirect transmittance estimation in step 1 is as follows: S11. Construct a physical model for foggy imaging. The expression is as follows: (1) In the formula, is the original image, For a clear image after dehazing, is the global atmospheric light coefficient, is the medium transmittance; S12. Calculate transmittance estimation value based on transmittance indirect estimation algorithm ; S13, the transmittance estimation value obtained by S12 Substituting into formula (1), we can obtain the dehazed image, which is expressed as follows: (2).

[0007] Preferably, in S12, the transmittance estimation value is calculated based on the transmittance indirect estimation algorithm. The process is as follows: S121. Constructing atmospheric dissipation function , the expression is as follows: (3) In the formula, is the global atmospheric light coefficient, is the medium transmittance, for any pixel point, Always holds true; S122, original image Find the minimum value of the three color channels to obtain a grayscale image of the same size as the original image , the expression is as follows: (4) In formula (4), the minimum color channel component of the original image is Always holds true; S123. Combining equations (3) and (4), the following formula is derived: (5); S124, yes Adaptive surface blur processing is performed to obtain images of the sky area and non-sky area. The expressions are as follows: (6) In the formula, is the grayscale image after blurring; S125, by and The transformation trend is consistent, and the transmittance estimate is obtained , the expression is as follows: (7) In the formula, To adjust the parameters.

[0008] Preferably, the process of obtaining the spatial self-attention feature map based on the spatial self-attention feature extraction algorithm in step 2 is as follows: S21. Based on the static image frame as the input tensor x, set the pooling windows (1, W) and (H, 1) respectively, and calculate the row and column feature means of the input tensor x in the horizontal and vertical directions. , , thereby obtaining spatial information to suppress background noise interference, the expression is as follows: (8) (9) In the formula, the input tensor is , the first-order tensor in the horizontal direction is W is the width, and the first-order tensor in the vertical direction is , H is the height, is the second-order gradient tensor; S22. Construct an attention information fusion network, introduce the number of channels C in the attention information fusion network, and decompose the input tensor into , respectively obtain the decomposed first-order tensor in the horizontal direction and the first-order tensor in the vertical direction , use the add method to the decomposed and For feature fusion, the expression is as follows: ) (10) In the formula, Indicates fusion features; S23. Fusion features Perform batch normalization and activation processing to obtain the output value z, which is expressed as follows: (11) S24, perform global maximum pooling and global average pooling along the channel axis to capture the correlation between input features and obtain a spatial self-attention feature map containing color, shape, and texture salient areas , the expression is as follows: (12) In the formula, is the global maximum pooling operation, is the global average pooling operation.

[0009] Preferably, the process of tracking and locating the dynamic weak target based on the dynamic and static combined attention target tracking algorithm in step 3 is as follows: S31, cropping the target area of ​​the static image frame as a reference template; setting the length of the target O to w and the length to h, taking the first frame of the video sequence as an example, cropping a square area with the target point O as the center and the side length L, the area can contain all the information of the target O, the expression is as follows: (13) Where S is the clipping area of ​​the target area; S32, continuously tracking the target according to the dynamic image frame, when the target moves or is blocked by the surrounding environment, selecting the image frame with the highest confidence as the dynamic template, completing the automatic update of the dynamic template through the self-learning update strategy, and obtaining the best confidence H of the current frame; S33, real-time tracking of the target, calculation of the similarity between the static / dynamic and static dual template sub-sequences and the search sub-sequences , , and then perform weighted fusion to obtain a more accurate target positioning result. The expression is as follows: (14) (15) (16) In the formula, is the static template self-learning target change value, is the target change value of dynamic and static dual template self-learning, is the spatial self-attention feature map of the static template, is the spatial self-attention feature map of the dynamic template, is the convolution operation, Learn features for the background suppression module, is the spatial self-attention feature map of the search area, is the equilibrium parameter of the dynamic and static dual templates, It is the response map after weighted fusion of the current frame, and its best confidence is the target prediction positioning; S34. Dynamically update the estimate according to the target response scale, and the expression is as follows: (17) In the formula, is the maximum response scale of the current frame image target, is the maximum response scale of the target in the previous frame, and v is the scale change speed between two frames.

[0010] Preferably, the dynamic template is automatically updated to meet the current frame's best confidence H and response graph peak value All of them are greater than the historical average level, and the expression is as follows: (18) (19) In the formula, is the historical average, is the maximum mean, , Update frequency of dynamic templates.

[0011] Therefore, the present invention adopts the above-mentioned water surface dynamic weak target tracking method based on the combination of static and dynamic attention, which has the following beneficial effects: (1) A physical model for foggy imaging was constructed, and the transmittance was indirectly estimated through the atmospheric light dissipation function. Finally, a clear image with successful defogging was obtained. This successfully overcame the problem of poor optical imaging clarity in complex foggy weather at sea, created basic conditions for the subsequent continuous tracking and positioning of dynamic weak targets on the water surface, and further reduced the impact of environmental constraints on surface target recognition. (2) The real-time and accuracy of dynamic weak target tracking are improved: First, a convolution channel is added to the attention information fusion network, and feature depth value association operations are performed through adaptive maximum pooling and average pooling operations to obtain the spatial self-attention feature maps of dynamic and static image frames; then, the dynamic and static combined attention target tracking algorithm is used to calculate the association relationship between dynamic and static image frames, and the self-learning update algorithm is used to keep the target detection in the best state at all times, avoiding the loss of dynamic weak targets due to motion or occlusion, and further improving the real-time and continuity of target tracking throughout the entire process.

[0012] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is an overall flow chart of a method for tracking a dynamic weak target on a water surface based on combining static and dynamic attention according to the present invention; Figure 2 This is a principle block diagram of a spatial self-attention feature extraction algorithm according to an embodiment of the present invention; Figure 3 This is a principle block diagram of a dynamic and static combined attention target tracking algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] See also Figure 1-Figure 3 , a method for tracking dynamic weak targets on the water surface based on dynamic and static combined attention, comprising the following steps: Step 1: Obtain a clear image with successful defogging based on the priori defogging algorithm indirectly estimated by transmittance. On the premise of obtaining the priori knowledge of defogging, firstly construct a physical model of foggy imaging, then indirectly estimate the transmittance through the atmospheric light dissipation function, and finally obtain a clear image with successful defogging. The process of obtaining a clear image with successful defogging based on the priori defogging algorithm indirectly estimated by transmittance is as follows: S11. Construct a physical model for foggy imaging. The expression is as follows: (1) In the formula, is the original image, For a clear image after dehazing, is the global atmospheric light coefficient, is the medium transmittance; S12. Calculate transmittance estimation value based on transmittance indirect estimation algorithm ; The specific calculation process is as follows: S121. Constructing atmospheric dissipation function , the expression is as follows: (3) In the formula, is the global atmospheric light coefficient, is the medium transmittance, for any pixel point, Always holds true; S122, original image Find the minimum value of the three color channels to obtain a grayscale image of the same size as the original image , the expression is as follows: (4) In formula (4), the minimum color channel component of the original image is Always holds true; S123. Combining equations (3) and (4), the following formula is derived: (5); S124, yes Adaptive surface blur processing is performed to obtain images of the sky area and non-sky area. The expressions are as follows: (6) In the formula, is the grayscale image after blurring; S125, by and The transformation trend is consistent, and the transmittance estimate is obtained , the expression is as follows: (7) In the formula, To adjust the parameters.

[0016] S13, the transmittance estimation value obtained by S12 Substituting into formula (1), we can obtain the dehazed image, which is expressed as follows: (2).

[0017] Step 2: Obtain a spatial self-attention feature map based on a spatial self-attention feature extraction algorithm; adaptively adjust the static image frame through autonomous learning, use global maximum pooling and global average pooling to calculate the depth value of the correlation between features, and obtain the spatial self-attention feature map; wherein, the process of obtaining the spatial self-attention feature map based on the spatial self-attention feature extraction algorithm is as follows: S21. Based on the static image frame as the input tensor x, set the pooling windows (1, W) and (H, 1) respectively, and calculate the row and column feature means of the input tensor x in the horizontal and vertical directions. , thereby obtaining spatial information to suppress background noise interference, the expression is as follows: (8) (9) In the formula, the input tensor is , the first-order tensor in the horizontal direction is W is the width, and the first-order tensor in the vertical direction is , H is the height, is the second-order gradient tensor; S22. Construct an attention information fusion network, introduce the number of channels C in the attention information fusion network, and decompose the input tensor into , respectively obtain the decomposed first-order tensor in the horizontal direction and the first-order tensor in the vertical direction , use the add method to the decomposed and For feature fusion, the expression is as follows: (10) In the formula, Indicates fusion features; S23. Fusion features Perform batch normalization and activation processing to obtain the output value z, which is expressed as follows: (11) S24, perform global maximum pooling and global average pooling along the channel axis to capture the correlation between input features and obtain a spatial self-attention feature map containing color, shape, and texture salient areas , the expression is as follows: (12) In the formula, is the global maximum pooling operation, is the global average pooling operation.

[0018] Step 3: Track and locate the dynamic weak target based on the static and dynamic combined attention target tracking algorithm; obtain the depth feature values ​​of the static and dynamic image frames respectively through the spatial self-attention feature extraction algorithm, then analyze the correlation between the two, and finally dynamically update the estimation according to the target response scale; wherein, the process of tracking and locating the dynamic weak target based on the static and dynamic combined attention target tracking algorithm is as follows: S31, cropping the target area of ​​the static image frame as a reference template; setting the length of the target O to w and the length to h, taking the first frame of the video sequence as an example, cropping a square area with the target point O as the center and the side length L, the area can contain all the information of the target O, the expression is as follows: (13) Where S is the clipping area of ​​the target area; S32, continuously track the target according to the dynamic image frame. When the target moves or is blocked by the surrounding environment, select the image frame with the highest confidence as the dynamic template, and complete the automatic update of the dynamic template through the self-learning update strategy to obtain the best confidence H of the current frame; the automatic update of the dynamic template must meet the best confidence H of the current frame and the peak value of the response graph All of them are greater than the historical average level, and the expression is as follows: (18) (19) In the formula, is the historical average, is the maximum mean, , The update frequency of dynamic templates; S33, real-time tracking of the target, calculation of the similarity between the static / dynamic and static dual template sub-sequences and the search sub-sequences , , and then perform weighted fusion to obtain a more accurate target positioning result. The expression is as follows: (14) (15) (16) In the formula, is the static template self-learning target change value, is the target change value of dynamic and static dual template self-learning, is the spatial self-attention feature map of the static template, is the spatial self-attention feature map of the dynamic template, is the convolution operation, Learn features for the background suppression module, is the spatial self-attention feature map of the search area, is the equilibrium parameter of the dynamic and static dual templates, It is the response map after weighted fusion of the current frame, and its best confidence is the target prediction positioning; S34. Dynamically update the estimate according to the target response scale, and the expression is as follows: (17) In the formula, is the maximum response scale of the current frame image target, is the maximum response scale of the target in the previous frame, and v is the scale change speed between two frames.

[0019] Therefore, the present invention adopts the above-mentioned surface dynamic weak target tracking method based on combined dynamic and static attention. First, a clear image with successful defogging is obtained based on a priori defogging algorithm with indirect transmittance estimation, thereby reducing the interference effect of complex foggy weather at sea on optical target recognition; then, a spatial self-attention feature map is obtained based on a spatial self-attention feature extraction algorithm; finally, a dynamic weak target is tracked and located based on a dynamic and static combined attention target tracking algorithm, thereby improving the real-time and accuracy of dynamic weak target tracking.

[0020] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for tracking dynamic weak targets on the water surface based on dynamic and static combined attention, characterized in that: The following steps are involved: Step 1: Obtain a clear image with successful defogging based on a priori defogging algorithm based on indirect estimation of transmittance. On the premise of obtaining prior knowledge of defogging, firstly construct a physical model of foggy imaging, then indirectly estimate the transmittance through the atmospheric light dissipation function, and finally obtain a clear image with successful defogging. Step 2: Obtain a spatial self-attention feature map based on a spatial self-attention feature extraction algorithm; adaptively adjust the static image frame through autonomous learning, use global maximum pooling and global average pooling to calculate the depth value of the correlation between features, and obtain a spatial self-attention feature map; Step 3: Track and locate dynamic weak targets based on the static and dynamic combined attention target tracking algorithm; obtain the depth feature values ​​of static and dynamic image frames respectively through the spatial self-attention feature extraction algorithm, and then analyze the correlation between the two, and finally dynamically update the estimation according to the target response scale.

2. According to claim 1, a method for tracking dynamic weak targets on the water surface based on dynamic and static combined attention is characterized in that: The process of obtaining a clear image with successful defogging using the prior defogging algorithm based on indirect transmittance estimation in step 1 is as follows: S11. Construct a physical model for foggy imaging. The expression is as follows: (1) In the formula, is the original image, For a clear image after dehazing, is the global atmospheric light coefficient, is the medium transmittance; S12. Calculate transmittance estimation value based on transmittance indirect estimation algorithm ; S13, the transmittance estimation value obtained by S12 Substituting into formula (1), we can obtain the dehazed image, which is expressed as follows: (2)。 3. According to claim 2, a method for tracking dynamic weak targets on the water surface based on dynamic and static combined attention is characterized in that: S12 calculates the transmittance estimation value based on the transmittance indirect estimation algorithm The process is as follows: S121. Constructing atmospheric dissipation function , the expression is as follows: (3) In the formula, is the global atmospheric light coefficient, is the medium transmittance, for any pixel point, Always holds true; S122, original image Find the minimum value of the three color channels to obtain a grayscale image of the same size as the original image , the expression is as follows: (4) In formula (4), the minimum color channel component of the original image is Always holds true; S123. Combining equations (3) and (4), the following formula is derived: (5)； S124, yes Adaptive surface blur processing is performed to obtain images of the sky area and non-sky area. The expressions are as follows: (6) In the formula, is the grayscale image after blurring; S125, by and The transformation trend is consistent, and the transmittance estimate is obtained , the expression is as follows: (7) In the formula, To adjust the parameters.

4. According to claim 3, a method for tracking dynamic weak targets on the water surface based on dynamic and static combined attention is characterized in that: The process of obtaining the spatial self-attention feature map based on the spatial self-attention feature extraction algorithm in step 2 is as follows: S21. Based on the static image frame as the input tensor x, set the pooling windows (1, W) and (H, 1) respectively, and calculate the row and column feature means of the input tensor x in the horizontal and vertical directions. , , the expression is as follows: (8) (9) In the formula, the input tensor is , the first-order tensor in the horizontal direction is W is the width, and the first-order tensor in the vertical direction is , H is the height, is the second-order gradient tensor; S22. Construct an attention information fusion network, introduce the number of channels C in the attention information fusion network, and decompose the input tensor into , respectively obtain the decomposed first-order tensor in the horizontal direction and the first-order tensor in the vertical direction , use the add method to the decomposed and For feature fusion, the expression is as follows: (10) In the formula, Indicates fusion features; S23. Fusion features Perform batch normalization and activation processing to obtain the output value z, which is expressed as follows: (11) S24, perform global maximum pooling and global average pooling along the channel axis to capture the correlation between input features and obtain a spatial self-attention feature map containing color, shape, and texture salient areas , the expression is as follows: (12) In the formula, is the global maximum pooling operation, is the global average pooling operation.

5. According to claim 4, a method for tracking a dynamic weak target on a water surface based on combining static and dynamic attention, characterized in that: The process of tracking and locating dynamic weak targets based on the dynamic and static combined attention target tracking algorithm in step 3 is as follows: S31, cropping the target area of ​​the static image frame as a reference template; setting the length of the target O to w and the length to h, taking the first frame of the video sequence as an example, cropping a square area with the target point O as the center and the side length L, the area can contain all the information of the target O, the expression is as follows: (13) Where S is the clipping area of ​​the target area; S32, continuously tracking the target according to the dynamic image frame, when the target moves or is blocked by the surrounding environment, selecting the image frame with the highest confidence as the dynamic template, completing the automatic update of the dynamic template through the self-learning update strategy, and obtaining the best confidence H of the current frame; S33, real-time tracking of the target, calculation of the similarity between the static / dynamic and static dual template sub-sequences and the search sub-sequences , , and then perform weighted fusion to obtain the target positioning result, the expression is as follows: (14) (15) (16) In the formula, is the static template self-learning target change value, is the dynamic and static dual template self-learning target change value, is the spatial self-attention feature map of the static template, is the spatial self-attention feature map of the dynamic template, is the convolution operation, Learn features for the background suppression module, is the spatial self-attention feature map of the search area, is the equilibrium parameter of the dynamic and static dual templates, It is the response map after weighted fusion of the current frame; S34. Dynamically update the estimate according to the target response scale, and the expression is as follows: (17) In the formula, is the maximum response scale of the current frame image target, is the maximum response scale of the target in the previous frame, and v is the scale change speed between two frames.

6. The method for tracking a dynamic weak target on a water surface based on combining static and dynamic attention according to claim 5, characterized in that: The automatic update of dynamic templates must meet the current frame's best confidence H and response graph peak All of them are greater than the historical average level, and the expression is as follows: (18) (19) In the formula, is the historical average, is the maximum mean, , Update frequency of dynamic templates.

Images