An underwater target re-identification method combining local perception and high-order feature reconstruction

By combining local perception and high-order feature reconstruction methods, the identification accuracy problem caused by robot navigation changes in underwater target recognition is solved, and more efficient underwater target object re-identification is achieved, improving the recognition accuracy and robustness sex.

CN114283326BActive Publication Date: 2024-11-05DALIAN MARITIME UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111582065.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-22
Publication Date
2024-11-05
Estimated Expiration
2041-12-22

AI Technical Summary

Technical Problem

The existing underwater target recognition methods cause the target object to deform due to changes in the navigation angle, speed and trajectory of the underwater robot, which affects the recognition accuracy and cannot effectively re-identify the same target object.

Method used

Using a method combining local perception and higher-order feature reconstruction, the feature map of underwater targets is extracted and reconstructed through YOLOv4 object detection algorithm, Mish activation function, spatial pyramid pooling, path aggregation network, prediction network and residual network. Generate fine-grained context features and perform re-identification processing.

Benefits of technology

It improves the robustness and accuracy of underwater target recognition, can more accurately identify target objects of different angles and scales, and enhances feature extraction performance and image reconstruction capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114283326B_ABST
    Figure CN114283326B_ABST
Patent Text Reader

Abstract

The present invention discloses an underwater target re-identification method combining local perception and high-order feature reconstruction, which includes using a cross-stage network of a target detection algorithm model to process underwater target features on a collected image to obtain an underwater target feature map; the underwater target feature map is mapped and scaled pooled to obtain a feature map with a feature matrix, a path aggregation network is input and a target object is predicted and generated, the target object is sampled and image features are extracted through a residual network to obtain a three-dimensional tensor and a vertical bar, the vertical bar is averagely pooled to generate a predicted image, the predicted image is tensor reconstructed to obtain context information fragments, synthesis and element-level product and weighted average are used to obtain context features, and the context features are averaged and pooled and the cross entropy loss is minimized to obtain a prediction result. Tensor reconstruction extracts a feature map with higher-order feature information, which is used to predict a re-identification prediction result with stronger robustness and higher accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of underwater target recognition, and in particular to an underwater target re-recognition method combining local perception and high-order feature reconstruction. Background Art

[0002] Existing underwater target recognition methods are mainly based on relevant target recognition algorithms to extract underwater target features for recognition and detection tasks. However, due to changes in the underwater robot's navigation angle, navigation speed and navigation trajectory, the captured target will be affected by deformation, angle and contrast. This brings many challenges to the underwater target re-identification task based on visual guidance of underwater robots to recognize the same target at different angles. The current target re-identification task is mainly based on the recognition of unified targets under different cameras on land. The existing underwater target recognition algorithm cannot perceive the current position and cannot repeatedly recognize the same target due to changes in the underwater robot's shape and route. Therefore, the change in the trajectory during the navigation of the underwater robot will affect the size of the collected image, resulting in the final recognition affecting the accuracy. Summary of the invention

[0003] The present invention provides an underwater target re-identification method combining local perception and high-order feature reconstruction to overcome technical problems such as the inaccurate results of existing underwater target recognition methods.

[0004] In order to achieve the above object, the technical solution of the present invention is:

[0005] An underwater target re-identification method combining local perception and high-order feature reconstruction includes the following steps:

[0006] Step 1: Collect images of the area where the underwater robot passes;

[0007] Step 2: Use the cross-stage network of the YOLOv4 target detection algorithm model to process the underwater target features of the collected images to obtain an underwater target feature map;

[0008] Step 3: Use the Mish activation function to map the underwater target feature map to obtain the target feature result map;

[0009] Step 4: Perform a spatial pyramid pooling operation on the target feature result map through the target detection algorithm, and connect the pooling results by string, so as to separate the feature map with the feature matrix;

[0010] Step 5: Input the feature map with the feature matrix into the path aggregation network for feature fusion to obtain a fused feature map;

[0011] Step 6: Input the fused feature map into the prediction network, set the anchor frame, and use the clustering algorithm to predict the anchor frame to generate a prediction frame, where the prediction frame includes the network output image, which is used to detect objects of different scales;

[0012] Step 7: After sampling the network output image in batches, extract the features of the network output image through the residual network to obtain a three-dimensional tensor T.

[0013] Step 8: Perform average pooling on the three-dimensional tensor T so that the three-dimensional tensor T is divided into p vertical strips, i.e., block re-recognition processing, and use the matrix channel axis of the three-dimensional tensor T to obtain the activation vectors of the p vertical strips and define them as column vectors;

[0014] Step 9: Input the column vector into a classifier for prediction processing to obtain a predicted image;

[0015] Step 10: Use the residual network to perform tensor reconstruction and re-recognition processing on the predicted image, and extract the matrix features of the predicted image after the tensor reconstruction and re-recognition processing, and use the matrix features to generate features from three angles of channel, width and height to obtain context information fragments;

[0016] Step 11: Repeat step 10 to synthesize the obtained context information fragments to obtain a context attention map representing three-dimensional context features; activate and aggregate the context attention map using element-wise product and weighted average to obtain fine-grained context features in spatial and channel dimensions;

[0017] Step 12: Perform global average pooling on the fine-grained context feature map through a global average pooling layer to obtain a target feature map; connect the target feature map with the column vector and optimize the connection result by minimizing the cross entropy loss to obtain a joint optimized feature map; perform prediction processing on the joint optimized feature map to obtain a prediction result.

[0018] Furthermore, in step 5, the feature pyramid structure in the path aggregation network is used to laterally connect the top-down path and the bottom-up path to segment the feature map with the feature matrix, and an adaptive pooling operation is used to perform a pooling extraction operation on each feature map with the feature matrix to obtain the target feature, and the extracted target feature is subjected to a fully connected fusion operation to obtain a fused feature map.

[0019] Furthermore, in step 6, the global intersection-over-union ratio is used as a prediction frame regression function to predict the anchor frame and generate a prediction frame, wherein the specific formula of the global intersection-over-union ratio is:

[0020]

[0021]

[0022]

[0023] where ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the predicted box and the true box, c represents the diagonal length of the minimum circumscribed moment between the center points of the predicted box and the center points of the true box, IOU represents the intersection-over-union ratio of the predicted box and the true box, α represents the influence factor corresponding to the parameter v, v represents the parameter for measuring the consistency of the aspect ratio, w represents the width of the corresponding box, h represents the height of the corresponding box, gt represents the true box, and CIOU represents the global intersection-over-union ratio.

[0024] Furthermore, step 8 also includes modifying the residual network and then extracting the deep features of the network output image to obtain a three-dimensional tensor T. Specifically, the modification of the residual network includes eliminating the global average pooling layer, the fully connected layer and the output layer of the convolutional neural network.

[0025] Furthermore, in step 11, the context attention map is activated and aggregated using element-wise multiplication and weighted average to obtain fine-grained context features in spatial and channel dimensions:

[0026] Step 11.1, repeat step 10 to obtain context information fragments in different directions;

[0027] Step 11.2: Reconstruct different context information fragments into tensors to obtain sub-attention maps A in different directions. i ;

[0028] Step 11.3: Sub-attention maps A in different directions i Synthesize the context attention map;

[0029] Step 11.4: Activate and aggregate the contextual attention map using element-wise multiplication and weighted averaging to obtain fine-grained contextual features in spatial and channel dimensions.

[0030] Furthermore, the specific method of step 11.4 is:

[0031]

[0032] S={s1,s2,...,s CHW}

[0033] A={a1,a2,...,a CHW}

[0034] Among them, S represents the input feature matrix, A represents the contextual attention map, Y represents the fine-grained contextual feature, i represents the i-th feature, CHW represents the total number of features, S represents the input feature, and A represents the contextual attention map.

[0035] Beneficial effects: The present invention uses a target detection network to obtain the marking information of underwater targets for constructing underwater target feature maps. A local perception branch network is constructed to perform block re-identification processing on underwater targets to improve feature extraction performance. At the same time, a high-order reconstruction branch network is constructed to reconstruct the high-order discriminant features of image features using tensor reconstruction, and further extract feature maps with higher-order feature information. By using a cross-stage network to extract feature maps with more detailed features, a high-order reconstruction branch network extracts feature maps with higher-order feature information, and the two feature maps are connected for prediction, thereby obtaining a more robust and accurate re-identification prediction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 It is a schematic diagram of the structure of the bus abnormality processing device of the present invention;

[0038] Figure 2 This is a structural diagram of the target detection algorithm model in the present invention;

[0039] Figure 3 This is a cross-stage network structure diagram in the present invention;

[0040] Figure 4 This is a diagram of the dual-branch re-identification network structure in the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] This embodiment provides an underwater target re-identification method combining local perception and high-order feature reconstruction, such as Figure 1-4 , including the following steps:

[0043] Step 1: Collect images of the area where the underwater robot passes;

[0044] Step 2: Use the cross-stage network of the YOLOv4 target detection algorithm model to process the underwater target features of the collected images. Split the stacked residual blocks into two parts, and then combine them through a cross-stage hierarchical structure to obtain a more robust underwater target feature map F. g ;

[0045] The cross-stage network includes multiple standard 3×3 convolutions for feature extraction, and a Mish activation function for preventing overfitting of the training network and obtaining more accurate results. The formula is as follows:

[0046] Mish=X×tan h(ln(1+e x ))

[0047] Specifically, feature blocks are extracted from seafood targets in the image of the visible area of ​​the underwater robot.

[0048] First, the cross-stage network (CSPDarknet53) consists of multiple standard 3×3 convolutions, Mish functions, and CSPNet. The purpose of multiple standard 3×3 convolutions is to better extract the features of the target. After extracting the features, CSPNet is used to first divide the feature map of the base layer into two parts, and then merge them through the cross-stage hierarchy to extract a robust target feature map F. g The global features are then used as input, training, and final optimization for the feature pyramid network.

[0049] Step 3: Use the Mish activation function to map the underwater target feature map F g Perform mapping processing to obtain a more accurate and stable target feature result map;

[0050] Step 4: Use the target detection algorithm to perform a spatial pyramid pooling operation on the target feature result map, and concatenate the pooling results to separate the feature map F with the feature matrix. f ;

[0051] Four different sizes of pooling kernels are used to perform maximum pooling on the feature maps, and the pooling results are F m1 , F m2 , F m3 , F m4 Then the above four pooling results are connected to improve the discriminability and comprehensiveness of cross-stage network features and significantly separate more important context features. The connection part is shown in the following formula:

[0052] F f =concat(F m1 ,F m2 ,F m3 ,F m4 )

[0053] Step 5: The feature map F with the feature matrix f The input path aggregation network is used for feature fusion. Specifically, the feature map with the feature matrix is ​​segmented, extracted, and fused, the weights of the features with more channel information in the feature map are enhanced, and the weights of unimportant features are reduced to obtain a fused feature map.

[0054] Step 6: Input the fused feature map into the prediction network, set the anchor frame, and use the clustering algorithm to predict the anchor frame to generate a prediction frame, wherein the prediction frame includes a network output image, which is used to detect objects of different scales; specifically, the clustering algorithm sets the selection of anchor points, and by setting three anchor points of different sizes, corresponding anchor frames of different sizes are generated for prediction, and finally the result of the prediction network is three network output images of different sizes;

[0055] Step 7: After sampling the network output image in batches, extract the features of the network output image through the residual network to obtain a three-dimensional tensor T;

[0056] Step 8: Perform average pooling on the three-dimensional tensor T so that the three-dimensional tensor T is divided into p vertical strips, i.e., block re-recognition processing, and use the matrix channel axis of the three-dimensional tensor T to obtain the activation vectors of the p vertical strips and define them as column vectors;

[0057] Step 9: Perform average pooling on each of the p vertical strips to generate a single part-level column vector with more detailed features, and input the column vector into the classifier for prediction processing to obtain a predicted image; in order to dynamically classify all column vectors e in the three-dimensional tensor T, the classifier is composed of an FC layer and a Softmax function, and the Softmax activation is used as a part classifier as shown below:

[0058]

[0059] Where P(P i |e) is e belongs to P i The predicted probability of the part, p is the number of partition blocks defined previously. W is a trainable weight matrix;

[0060] Step 10: Use a convolutional neural network to perform tensor reconstruction and re-recognition processing on the predicted image, and extract the matrix features of the predicted image after the tensor reconstruction and re-recognition processing, and use the matrix features to generate features from three angles of channel, width and height to obtain context information fragments in the channel direction, width direction and height direction;

[0061] Step 11, repeat step 10 to obtain context information fragments in different directions, synthesize the context information fragments in different directions, and obtain a context attention map representing three-dimensional context features; repeat continuously, reconstruct other context fragments, and use element-level multiplication and weighted average to activate and aggregate the context attention map, so as to obtain fine-grained context features in spatial and channel dimensions.

[0062] Step 12: Perform global average pooling on the fine-grained context feature map through a global average pooling layer to obtain a higher-order target feature map; connect the target feature map with the column vector and use the minimized cross entropy loss to optimize the connection result to obtain a joint optimized feature map, and perform prediction processing on the joint optimized feature map to obtain a prediction result.

[0063] In a specific embodiment, in step 5, the feature pyramid structure in the path aggregation network is used to horizontally connect the top-down path and the bottom-up path to segment the feature map with the feature matrix, and the adaptive pooling operation is used to perform a pooling extraction operation on each feature map with the feature matrix to obtain the target feature, and the extracted target feature is subjected to a fully connected fusion operation to obtain a fused feature map. In order to better generate the target mask, adaptive pooling is used to perform a pooling extraction operation on each representative feature map in the path to obtain the target feature, and the extracted target feature is subjected to a fully connected fusion operation to obtain a fused feature map.

[0064] In a specific embodiment, step 6 uses the global intersection-over-union ratio as a prediction box regression function to predict the anchor box and generate a prediction box, wherein the specific formula of the global intersection-over-union ratio is:

[0065]

[0066]

[0067]

[0068] where ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the predicted box and the true box, c represents the diagonal length of the minimum circumscribed moment between the center points of the predicted box and the center points of the true box, IOU represents the intersection-over-union ratio of the predicted box and the true box, α represents the influence factor corresponding to the parameter v, v represents the parameter for measuring the consistency of the aspect ratio, w represents the width of the corresponding box, h represents the height of the corresponding box, gt represents the true box, and CIOU represents the global intersection-over-union ratio.

[0069] In a specific embodiment, step 8 also includes modifying the residual network and then extracting the deep features of the network output image to obtain a three-dimensional tensor T. Specifically, modifying the residual network includes eliminating the global average pooling layer, the fully connected layer and the output layer of the convolutional neural network.

[0070] In a specific embodiment, in step 11, the context attention map is activated and aggregated using element-wise multiplication and weighted average to obtain fine-grained context features in spatial and channel dimensions:

[0071] Step 11.1, repeat step 10 to obtain context information fragments in different directions;

[0072] Step 11.2: Reconstruct different context information fragments into tensors to obtain sub-attention maps A in different directions. i ;

[0073] Step 11.3: Sub-attention maps A in different directions i Synthesize the context attention map;

[0074] Step 11.4: Activate and aggregate the contextual attention map using element-wise multiplication and weighted averaging to obtain fine-grained contextual features in spatial and channel dimensions.

[0075] In a specific embodiment, the specific method of step 11.4 is:

[0076]

[0077] S={s1,s2,...,s CHW}

[0078] A={a1,a2,...,a CHW}

[0079] Among them, S represents the input feature matrix, A represents the contextual attention map, Y represents the fine-grained contextual feature, i represents the i-th feature, CHW represents the total number of features, S represents the input feature, and A represents the contextual attention map.

[0080] Specifically, the dual-branch re-identification network proposed in the present invention is as follows: Figure 4 As shown, it includes a local perception branch network and a high-order reconstruction branch network;

[0081] (1) The local perception branch network is used to perform average pooling on the three-dimensional tensor T as described in step 8 so that the three-dimensional tensor T is divided into p vertical strips, that is, block re-identification processing, and the activation vectors of the p vertical strips are obtained using the matrix channel axis of the three-dimensional tensor T and defined as column vectors.

[0082] Specifically, the three-dimensional tensor T is divided into five sub-feature tensors of equal size in the horizontal direction, which can be described as: T(i) = [T(i,1), T(i,2), ... T(i,p)]sti = 1, 2 ... N, where N is the number of samples in the data set, and p represents the number of blocks in the first branch. Then, global average pooling is applied to the sub-regions of each partition to obtain the column vector corresponding to each sub-feature tensor, which can effectively obtain the detailed features of the image. Then, for each column vector, each block-based segmentation sub-vector is obtained through the convolution layer and represented as: H(i) = [H(i,1), H(i,2), ... H(i,p)]sti = 1, 2 ... N. This method can model the feature distribution of each local area to obtain local perception features to improve the feature extraction performance of the network. During the training process, M is the number of classes in the training set. Finally, the identity of the predicted input is obtained by inputting the column vector into the classifier for prediction processing. The classifier consists of an FC layer and a Softmax function, and the Softmax activation is a partial classifier as shown below:

[0083]

[0084] Where p is the number of previously defined partitions, M is the number of categories in the training set, and L softmax is the local perception branch loss, H(i,p) is the i-th sample obtained by convolution of the p-th column vector, and H(j,p) is the j-th sample obtained by convolution of the p-th column vector. The prediction probability of the local refined features of the underwater target is obtained by predicting each feature sub-vector to increase the local perception performance of the network.

[0085] (2) The high-order reconstruction branch network uses the residual network in step 10 to perform tensor reconstruction and re-recognition processing on the predicted image, and extracts the matrix features of the predicted image after the tensor reconstruction and re-recognition processing, and uses the matrix features to generate features from three angles of channel, width and height to obtain context information fragments in three directions.

[0086] (3) The tensor reconstruction part in the high-order reconstruction branch network mainly generates features from the tensor T obtained from the cross-stage network from the three perspectives of channel, width and height to obtain context information fragments in three directions. The context features generated in the three directions are processed to obtain a sub-attention map representing a part of the three-dimensional context features. Other context fragments are continuously reconstructed iteratively, and the sub-maps are activated and aggregated through element-level multiplication and weighted averaging to obtain fine-grained context features in the spatial and channel dimensions.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater target re-identification method combining local perception and high-order feature reconstruction, characterized in that: The following steps are involved: Step 1: Collect images of the area where the underwater robot passes; Step 2: Use the cross-stage network of the YOLOv4 target detection algorithm model to process the underwater target features of the collected images to obtain an underwater target feature map; Step 3: Use the Mish activation function to map the underwater target feature map to obtain the target feature result map; Step 4: Perform a spatial pyramid pooling operation on the target feature result map through the target detection algorithm, and connect the pooling results by string, so as to separate the feature map with the feature matrix; Step 5: Input the feature map with the feature matrix into the path aggregation network for feature fusion to obtain a fused feature map; Step 6: Input the fused feature map into the prediction network, set the anchor frame, and use the clustering algorithm to predict the anchor frame to generate a prediction frame, where the prediction frame includes the network output image, which is used to detect objects of different scales; Step 7: After sampling the network output image in batches, extract the features of the network output image through the residual network to obtain a three-dimensional tensor T. Step 8: Perform average pooling on the three-dimensional tensor T so that the three-dimensional tensor T is divided into p vertical strips, i.e., block re-recognition processing, and use the matrix channel axis of the three-dimensional tensor T to obtain the activation vectors of the p vertical strips and define them as column vectors; Step 9: Input the column vector into a classifier for prediction processing to obtain a predicted image; Step 10: Use the residual network to perform tensor reconstruction and re-recognition processing on the predicted image, and extract the matrix features of the predicted image after the tensor reconstruction and re-recognition processing, and use the matrix features to generate features from three angles of channel, width and height to obtain context information fragments; Step 11: Repeat step 10 to synthesize the obtained context information fragments to obtain a context attention map representing three-dimensional context features; activate and aggregate the context attention map using element-wise product and weighted average to obtain fine-grained context features in spatial and channel dimensions; Step 12: Perform global average pooling on the fine-grained context feature map through a global average pooling layer to obtain a target feature map; connect the target feature map with the column vector and optimize the connection result by minimizing the cross entropy loss to obtain a joint optimized feature map; perform prediction processing on the joint optimized feature map to obtain a prediction result.

2. The underwater target re-identification method combining local perception and high-order feature reconstruction as claimed in claim 1, characterized in that: In step 5, the feature pyramid structure in the path aggregation network is used to horizontally connect the top-down path and the bottom-up path to segment the feature map with the feature matrix, and an adaptive pooling operation is used to perform a pooling extraction operation on each feature map with the feature matrix to obtain the target feature, and the extracted target feature is subjected to a fully connected fusion operation to obtain a fused feature map.

3. The underwater target re-identification method combining local perception and high-order feature reconstruction as claimed in claim 2, characterized in that: In step 6, the global intersection-over-union ratio is used as a prediction frame regression function to predict the anchor frame and generate a prediction frame, wherein the specific formula of the global intersection-over-union ratio is: where ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the predicted box and the true box, c represents the diagonal length of the minimum circumscribed moment between the center points of the predicted box and the center points of the true box, IOU represents the intersection-over-union ratio of the predicted box and the true box, α represents the influence factor corresponding to the parameter v, v represents the parameter for measuring the consistency of the aspect ratio, w represents the width of the corresponding box, h represents the height of the corresponding box, gt represents the true box, and CIOU represents the global intersection-over-union ratio.

4. The underwater target re-identification method combining local perception and high-order feature reconstruction as claimed in claim 3, characterized in that: Step 8 also includes modifying the residual network and then extracting the deep features of the network output image to obtain a three-dimensional tensor T. Specifically, the modification of the residual network includes eliminating the global average pooling layer, the fully connected layer and the output layer of the convolutional neural network.

5. The underwater target re-identification method combining local perception and high-order feature reconstruction as claimed in claim 4, characterized in that: In step 11, the context attention map is activated and aggregated using element-wise multiplication and weighted average to obtain fine-grained context features in spatial and channel dimensions: Step 11.1, repeat step 10 to obtain context information fragments in different directions; Step 11.2: Reconstruct different context information fragments into tensors to obtain sub-attention maps A in different directions. i ; Step 11.3: Sub-attention maps A in different directions i Synthesize the context attention map; Step 11.4: Activate and aggregate the contextual attention map using element-wise multiplication and weighted averaging to obtain fine-grained contextual features in spatial and channel dimensions.

6. The underwater target re-identification method combining local perception and high-order feature reconstruction as claimed in claim 5, characterized in that: The specific method of step 11.4 is: S={s1,s2,...,s CHW } <h2 style=";text-align:left;direction:ltr">A = {a1,a2,...,a<h2 style=";text-align:left;direction:ltr"> CHW <h2 style=";text-align:left;direction:ltr">} Among them, S represents the input feature matrix, A represents the contextual attention map, Y represents the fine-grained contextual feature, i represents the i-th feature, CHW represents the total number of features, S represents the input feature, and A represents the contextual attention map.