Nearshore ship target detection method based on anchor frame guided attention refinement network

Abstract

This application relates to a method for detecting nearshore ship targets based on an anchor-frame guided attention refinement network. The method comprises: obtaining a dataset of nearshore ship targets from surveillance videos; constructing a target detection model based on an anchor-frame guided attention refinement network; the target detection model primarily comprising an attention feature screening module and an anchor-frame guided alignment detection module; constructing an overall loss function for the target detection model; training the target detection model with the constructed overall loss function using the nearshore ship target dataset to obtain a trained target detection model; and, based on the trained target detection model, inputting an image of a nearshore ship target to be detected into the target detection model, outputting the category and location information of the nearshore ship target. This method can improve the detection accuracy of nearshore ship targets.

Description

Technical Field

[0001] The present invention mainly relates to the field of target detection technology, in particular to a nearshore ship target detection method based on an anchor frame guided attention refinement network. Background Art

[0002] Accurately and rapidly detecting nearshore vessels in surveillance video can effectively improve waterway utilization and port management efficiency. In recent years, surveillance video systems have been widely used in fields such as public security and transportation due to their advantages, such as real-time imaging, ease of deployment, and low cost. Unlike detecting remotely sensed vessel images captured by satellites, detecting nearshore vessels in surveillance video can provide more precise location and classification information, enhancing intelligent port and maritime dispatch capabilities.

[0003] Currently, approaches for detecting nearshore vessel targets in surveillance videos can be categorized into traditional methods and deep learning-based methods. Traditional methods detect nearshore vessel targets by modeling and distinguishing background and foreground regions. Leveraging domain knowledge of ocean images, these methods successfully identify most vessel targets in surveillance videos. However, due to limited generalization, their detection performance is significantly affected by complex parameters and typically degrades significantly under complex and changing background conditions. Compared to traditional methods, deep learning-based convolutional neural network (CNN) methods can automatically extract salient and robust features from input images and demonstrate strong performance in remote sensing. While these CNN-based methods and techniques have improved the detection performance of nearshore vessel targets in surveillance videos, many challenges remain. First, it is difficult to accurately capture the detailed features of multi-scale nearshore vessel targets. While the multi-level feature maps in convolutional neural networks alleviate this problem to some extent, they still present challenges in processing nearshore vessel targets with large aspect ratios. Second, the environments in which nearshore vessel targets exist are complex and changing. Due to the varying observation angles and deployment positions of surveillance cameras, the diverse backgrounds of these targets pose significant interference to the detectors. Furthermore, different ship targets in parallel waterways can easily block each other, causing serious confusion between different ship targets such as cabins, masts and other components, resulting in low accuracy in ship target detection. Summary of the Invention

[0004] Based on this, it is necessary to provide a nearshore ship target detection method, device, computer equipment and storage medium based on an anchor frame guided attention refinement network that can improve the accuracy of ship target detection in response to the above technical problems.

[0005] A method for detecting nearshore ship targets based on an anchor frame-guided attention refinement network, the method comprising:

[0006] Obtain a dataset of nearshore ship targets in surveillance videos;

[0007] Construct an object detection model based on an anchor-guided attention refinement network; the object detection model mainly includes an attention feature screening module and an anchor-guided alignment detection module;

[0008] The overall loss function of the target detection model is constructed; the target detection model with the constructed overall loss function is trained using the nearshore ship target dataset to obtain a trained target detection model;

[0009] According to the trained target detection model, the image of the nearshore ship target to be detected is input into the target detection model, and the category and location information of the nearshore ship target is obtained as output.

[0010] In one embodiment, the target detection model also includes an improved VGG-16 network, a transport connection block, and an ARM module in the RefineDet network model; the improved VGG-16 network includes multiple base layers and newly added layers; and the ARM module is located above the base layer and the newly added layer.

[0011] In one embodiment, based on the trained target detection model, an image of a nearshore ship target to be detected is input into the target detection model, and the category and location information of the nearshore ship target is output, including:

[0012] The improved VGG-16 network is used to extract features from the nearshore ship target image to be detected, and the basic feature map of the nearshore ship target is obtained;

[0013] The basic feature map of the nearshore ship target is transmitted to the transmission connection block using the newly added layer to generate the top-level feature map;

[0014] According to the attention feature screening module, the fine-grained feature map of the target is obtained from the top-level feature map. The anchor frame is used to guide the alignment detection module to detect the fine-grained feature map and obtain the category and location information of the nearshore ship target.

[0015] In one embodiment, the process of obtaining a fine-grained feature map of a target from a top-level feature map according to the attention feature screening module includes:

[0016] The top-level feature map is input into the network convolution layer, and the top-level feature map is processed using the Sigmoid function to obtain the attention feature map;

[0017] Interpolate and upsample the attention feature map to obtain the attention refinement feature map;

[0018] Multiply the basic feature map of the nearshore ship target with the attention refined feature map, and perform element-wise superposition with the basic feature map to obtain the refined feature map;

[0019] Perform two upsampling operations on the top feature map to obtain a sampled feature map;

[0020] The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target.

[0021] In one embodiment, the sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target, including:

[0022] The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain the fine-grained feature map of the nearshore ship target:

[0023]

[0024] in, represents the attention refinement feature map, represents the sampling feature map, represents the refined feature map, F represents the basic feature map of the nearshore ship target, and P i+1 Represents the top-level feature map, σ and δ represent the Sigmoid activation function and the ReLU activation function respectively, Up and Conv represent two upsampling and convolution operations respectively, and i represents the sequence number of the fine-grained feature map.

[0025] In one embodiment, the target detection module further includes a deformable convolution layer; and the alignment detection module is used to detect the fine-grained feature map using an anchor frame to obtain the category and location information of the nearshore ship target, including:

[0026] Generate a parameterized representation of the target position based on the ARM module, and use the parameterized representation to adjust the position of the predefined anchor box to obtain a refined anchor box;

[0027] Multiple original sampling points are randomly generated in the fine anchor frame, and the aligned sampling points of the original sampling points are found from the fine-grained feature map. The original sampling points and the corresponding aligned sampling points are calculated according to the deformable convolution to obtain the two-dimensional offset;

[0028] The two-dimensional offset and fine-grained feature map are input into the deformable convolution layer for regression and feature extraction to obtain the location information and category of nearshore ship targets.

[0029] In one embodiment, the original sampling point is

[0030]

[0031] Among them, i,j∈{0,1,k-1}, k=3 corresponds to the convolution operation with a convolution kernel size of 3×3.

[0032] In one embodiment, the alignment sampling points are

[0033]

[0034] Among them, (x1, y1, x2, y2) represents the position coordinates of the fine anchor box, and S represents the original sampling point.

[0035] In one embodiment, the two-dimensional offset is obtained by calculating based on the original sampling points and the corresponding aligned sampling points, including:

[0036] Calculate based on the original sampling points and the corresponding aligned sampling points, and get the two-dimensional offset:

[0037]

[0038] In one embodiment, the overall loss function is

[0039]

[0040] Among them, N ARM Indicates ARM loss, N AADM Represents the loss of the anchor box guided alignment detection module, N refers to the number of positive anchor boxes whose overlap with the true value of the label in all anchor boxes meets the predefined threshold, i represents the sequence number of the sampling point in the anchor box, represents the binary cross entropy loss and represents the binary cross entropy loss, p i and c i Represents the i th The anchor box contains the target and belongs to the cth th The probability of the class target, l i Indicates the i th The ground truth labels of the anchor boxes, in l i The output is 1 if ≥1, otherwise it is 0. i and t i represents the parameterized target position predicted by the network, represents the parameterized true value label.

[0041] A device for detecting nearshore ship targets based on an anchor frame-guided attention refinement network, the device comprising:

[0042] A detection model construction module is used to obtain a dataset of nearshore ship targets in surveillance videos; a target detection model based on an anchor-guided attention refinement network is constructed; the target detection model includes an attention feature screening module and an anchor-guided alignment detection module;

[0043] The detection model training module is used to construct the overall loss function of the target detection model; the target detection model with the constructed overall loss function is trained using the nearshore ship target dataset to obtain a trained target detection model;

[0044] The image detection module is used to detect the nearshore ship target image to be detected based on the trained target detection model to obtain the category and location information of the nearshore ship target.

[0045] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0046] Obtain a dataset of nearshore ship targets in surveillance videos;

[0047] Build an object detection model based on the anchor-guided attention refinement network; the object detection model includes an attention feature screening module and an anchor-guided alignment detection module;

[0048] The overall loss function of the target detection model is constructed; the target detection model with the constructed overall loss function is trained using the nearshore ship target dataset to obtain a trained target detection model;

[0049] According to the trained target detection model, the image of the nearshore ship target to be detected is input into the target detection model, and the category and location information of the nearshore ship target is obtained as output.

[0050] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0051] Obtain a dataset of nearshore ship targets in surveillance videos;

[0052] Construct an object detection model based on an anchor-guided attention refinement network; the object detection model mainly includes an attention feature screening module and an anchor-guided alignment detection module;

[0053] The overall loss function of the target detection model is constructed; the target detection model with the constructed overall loss function is trained using the nearshore ship target dataset to obtain a trained target detection model;

[0054] According to the trained target detection model, the image of the nearshore ship target to be detected is input into the target detection model, and the category and location information of the nearshore ship target is obtained as output.

[0055] The above-mentioned method, device, computer equipment and storage medium for detecting nearshore ship targets based on the anchor frame guided attention refinement network first construct a target detection model based on the anchor frame guided attention refinement network; the target detection model mainly includes an attention feature screening module and an anchor frame guided alignment detection module; the overall loss function of the target detection model is constructed; the target detection model with the constructed overall loss function is trained using the nearshore ship target data set to obtain a trained target detection model; the nearshore ship target image to be detected is input into the target detection model according to the trained target detection model, and the category and position information of the nearshore ship target is obtained as output. This application uses the attention feature screening module as an attention mechanism, embedded in the top of the bottom-up path of the horizontally connected feature pyramid network within the target detection model. It obtains the attention feature map from the high-level semantic feature map through several convolutional layers and Sigmoid activation functions, and filters out the key features of the nearshore ship target contained in the corresponding shallow feature map from the attention feature map, thereby suppressing the background interference on the shallow feature map and highlighting the important information of the nearshore ship target, thereby improving the accuracy of nearshore ship target detection; the anchor frame guided alignment detection module is applied to the output layer, and the feature calibration of the nearshore ship target is performed by refining the anchor frame and directly calculating the two-dimensional coordinate offset of the target position, thereby achieving accurate positioning of the nearshore ship target. By predicting and regressing the final position and category of the target on multiple feature maps with the anchor frame guided alignment detection module, the positioning accuracy of multi-scale nearshore ship targets can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of an embodiment of a method for detecting nearshore ship targets based on an anchor frame-guided attention refinement network;

[0057] Figure 2 A schematic diagram of the overall structure of an anchor-guided attention refinement network in one embodiment;

[0058] Figure 3 Schematic diagram of the structure of an attention feature screening module in one embodiment;

[0059] Figure 4 is a schematic structural diagram of an anchor frame guided alignment detection module in another embodiment;

[0060] Figure 5 1 is a structural block diagram of a nearshore ship target detection device based on an anchor frame guided attention refinement network in one embodiment;

[0061] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0063] In one embodiment, Figure 1 As shown, a method for detecting nearshore ship targets based on an anchor frame guided attention refinement network is provided, comprising the following steps:

[0064] Step 102: Obtain a dataset of nearshore ship targets in surveillance videos; construct a target detection model based on an anchor-frame guided attention refinement network; the target detection model mainly includes an attention feature screening module and an anchor-frame guided alignment detection module.

[0065] The target detection model based on the anchor box guided attention refinement network includes an attention feature screening module, an anchor box guided alignment detection module, an improved VGG-16 network, a transmission connection block, a deformable convolution layer and an ARM module. The ARM module is an existing module obtained from the RefineDet network and will not be described in detail in this application.

[0066] like Figure 2 The figure shows the structure of the target detection model (AARN). The target detection model uses the VGG-16 network as the basic backbone network to extract the hierarchical semantic information of the nearshore ship target from the input image. Conv4_3, Conv5_3, Conv7 and the newly added layer (Extas) in the intermediate feature layer of VGG-16 are selected to extract the basic features of multi-scale ships. In order to solve the problem of feature imbalance between shallow layers (such as Conv4_3, Conv5_4) and deep layers (such as Conv7, Extras), in the top-down network path, a feature pyramid is iteratively constructed with high-level features and current-level features. When constructing the refined feature pyramid, the semantic information in the high-level feature map is used to constrain the shallow feature map, thereby improving the algorithm's ability to represent the key details of the target in the shallow feature map. Among them, the Extras layer directly sends the forward-transmitted features to the transmission connection block ( Figure 2 The target detection model (AARN) integrates the attention feature filtering module (AFFM) ( Figure 2.1) is horizontally connected to obtain refined target feature maps P2, P3, and P4. To obtain the accurate location and category of nearshore ship targets, several ARM modules are placed on top of several basic feature layers (Conv4_3, Conv5_3, Conv7, and Extas) to perform a rough position regression of potential ship targets. At the same time, redundant pre-defined anchor frames are filtered out to generate refined anchor frames. A deformable convolutional network (DCN) is also used to accurately regress the target position by guiding the alignment detection module through the anchor frames and applying them to the P2, P3, P4, and P5 feature maps respectively. AFFM and AADM have powerful feature refinement and alignment capabilities, which can achieve accurate positioning of nearshore ship targets in complex backgrounds.

[0067] Step 104: construct an overall loss function for the target detection model; and train the target detection model with the constructed overall loss function using the nearshore ship target dataset to obtain a trained target detection model.

[0068] During the training phase, two-class and multi-class cross entropy losses are used to optimize the classification branches of the ARM module and anchor-guided alignment detection module (AADM), respectively, and Smooth L1 loss is used to optimize the regression branches of ARM and AADM to obtain the trained object detection model.

[0069] Step 106: Based on the trained target detection model, the image of the nearshore ship target to be detected is input into the target detection model, and the category and location information of the nearshore ship target is obtained as output.

[0070] In the above-mentioned method for detecting nearshore ship targets based on the anchor frame guided attention refinement network, a target detection model based on the anchor frame guided attention refinement network is first constructed; the target detection model mainly includes an attention feature screening module and an anchor frame guided alignment detection module; the overall loss function of the target detection model is constructed; the target detection model with the constructed overall loss function is trained using the nearshore ship target dataset to obtain a trained target detection model; according to the trained target detection model, the nearshore ship target image to be detected is input into the target detection model, and the category and position information of the nearshore ship target is obtained as output. This application uses the attention feature screening module as an attention mechanism, embedded in the top of the bottom-up path of the horizontally connected feature pyramid network within the target detection model. It obtains the attention feature map from the high-level semantic feature map through several convolutional layers and Sigmoid activation functions, and filters out the key features of the nearshore ship target contained in the corresponding shallow feature map from the attention feature map, thereby suppressing the background interference on the shallow feature map and highlighting the important information of the nearshore ship target, thereby improving the accuracy of nearshore ship target detection; the anchor frame guided alignment detection module is applied to the output layer, and the feature calibration of the nearshore ship target is performed by refining the anchor frame and directly calculating the two-dimensional coordinate offset of the target position, thereby achieving accurate positioning of the nearshore ship target. By predicting and regressing the final position and category of the target on multiple feature maps with the anchor frame guided alignment detection module, the positioning accuracy of the nearshore ship target can be improved.

[0071] In one embodiment, the target detection model also includes an improved VGG-16 network, a transmission connection block and an ARM module; the improved VGG-16 network includes multiple base layers and newly added layers; the ARM module is located above the base layer and the newly added layer.

[0072] In one embodiment, based on the trained target detection model, an image of a nearshore ship target to be detected is input into the target detection model, and the category and location information of the nearshore ship target is output, including:

[0073] The improved VGG-16 network is used to extract features from the nearshore ship target image to be detected, and the basic feature map of the nearshore ship target is obtained;

[0074] The basic feature map of the nearshore ship target is transmitted to the transmission connection block using the newly added layer to generate the top-level feature map;

[0075] According to the attention feature screening module, the fine-grained feature map of the target is obtained from the top-level feature map. The anchor frame is used to guide the alignment detection module to detect the fine-grained feature map and obtain the category and location information of the nearshore ship target.

[0076] In one embodiment, the process of obtaining a fine-grained feature map of a target from a top-level feature map according to the attention feature screening module includes:

[0077] The top-level feature map is input into the network convolution layer, and the top-level feature map is processed using the Sigmoid function to obtain the attention feature map;

[0078] Performing interpolation calculation and upsampling on the attention feature map to obtain a refined attention feature map;

[0079] Multiply the basic feature map of the nearshore ship target with the attention refined feature map, and perform element-wise superposition with the basic feature map to obtain the refined feature map;

[0080] Perform two upsampling operations on the top feature map to obtain a sampled feature map;

[0081] The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target.

[0082] In one embodiment, the sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target, including:

[0083] The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain the fine-grained feature map of the nearshore ship target:

[0084]

[0085] in, represents the attention refinement feature map, represents the sampling feature map, represents the refined feature map, F represents the basic feature map of the nearshore ship target, and P i+1 Represents the top-level feature map, σ and δ represent the Sigmoid activation function and the ReLU activation function respectively, Up and Conv represent two upsampling and convolution operations respectively, and i represents the sequence number of the fine-grained feature map.

[0086] In a specific embodiment, Figure 3 As shown, AFFM is a "plug and play" module used to generate P i,i∈2,3,4 Feature map. In AAFM, first the previous feature map P i+1 Send it to the network convolution layer, and then generate the attention feature map A through the Sigmoid function i+1 , the number of channels is the same as P i+1The number of channels is the same and its value range is 0.0~1.0. Then i+1 Perform interpolation operations and generate the attention refinement feature map A after two upsampling operations i+1 UP The basic feature map F of multi-scale ships is combined with the feature map A i+1 UP Multiply and superimpose elements with F to obtain the refined feature map F A This operation can use high-level abstract information to filter background interference information and enhance the important features of the target. In order to further retain the high-level semantic features of the nearshore ship target, in P i+1 Two upsampling operations are also performed on the sampled feature map P i+1 UP and the refined feature map F A Finally, after a 3×3 convolution layer and ReLU activation function, the fine-grained feature map P of the nearshore ship target is obtained. i Compared with the currently common attention mechanisms such as SE and CBAM, where attention information is only generated by the same input layer, AFFM can utilize supervision extracted from high-level feature layers to refine the information of low-level feature maps.

[0087] In one embodiment, the target detection module further includes a deformable convolution layer; and the alignment detection module is used to detect the fine-grained feature map using an anchor frame to obtain the category and location information of the nearshore ship target, including:

[0088] Generate a parameterized representation of the target position based on the ARM module, and use the parameterized representation to adjust the position of the predefined anchor box to obtain a refined anchor box;

[0089] Multiple original sampling points are randomly generated in the fine anchor frame, and the aligned sampling points of the original sampling points are found from the fine-grained feature map. The original sampling points and the corresponding aligned sampling points are calculated according to the deformable convolution to obtain the two-dimensional offset;

[0090] The two-dimensional offset and fine-grained feature map are input into the deformable convolution layer for regression and feature extraction to obtain the location information and category of nearshore ship targets.

[0091] In a specific embodiment, Figure 3 As shown, first, according to the c generated by the ARM module x , c y, w, h and other parameterized representations of the target position, adjust the position of the predefined anchor box (the thin black rectangular box in the figure) to obtain the refined anchor box (the gray rectangular box in the figure). It can be clearly seen from the figure that compared with the predefined anchor box, the overlap rate between the refined anchor box and the target true value label (the thick black rectangular box in the figure) is higher, so the target features contained in the refined anchor box are more representative than the target features contained in the predefined anchor box. Therefore, the newly generated uniformly distributed sampling points in the refined anchor box (9 gray dots) are aligned with the refined anchor frame to a certain extent, so the target features captured by them are consistent with the target features captured by the refined anchor frame to a certain extent. Then, the 9 original sampling points S are calculated by deformable convolution i,j (X,Y) (black dots in the figure) and 9 alignment sampling points The two-dimensional offset O between i,j (X, Y), and input it into several deformable convolutional layers for final regression to obtain the position information of the nearshore ship target, and use the offset to represent the position;

[0092] The fine-grained feature map is then input into the deformable convolutional layer for feature extraction, and near-shore ship target detection is performed based on the features to obtain the ship classification.

[0093] In one embodiment, the original sampling point is

[0094]

[0095] Among them, i,j∈{0,1,k-1}, k=3 corresponds to the convolution operation with a convolution kernel size of 3×3.

[0096] In one embodiment, the alignment sampling points are

[0097]

[0098] Among them, (x1, y1, x2, y2) represents the position coordinates of the fine anchor box, and S represents the original sampling point.

[0099] In one embodiment, the two-dimensional offset is obtained by calculating based on the original sampling points and the corresponding aligned sampling points, including:

[0100] Calculate based on the original sampling points and the corresponding aligned sampling points, and get the two-dimensional offset:

[0101]

[0102] In one embodiment, the overall loss function is

[0103]

[0104] Among them, N ARM Indicates ARM loss, N AADM Represents the loss of the anchor box guided alignment detection module, N refers to the number of positive anchor boxes whose overlap with the true value of the label in all anchor boxes meets the predefined threshold, i represents the sequence number of the sampling point in the anchor box, represents the binary cross entropy loss and represents the binary cross entropy loss, p i and c i Represents the i th The anchor box contains the target and belongs to the cth th The probability of the class target, l i Indicates the i th The ground truth labels of the anchor boxes, in l i The output is 1 if ≥1, otherwise it is 0. i and t i represents the parameterized target position predicted by the network, represents the parameterized true value label.

[0105] In a specific embodiment, two-class and multi-class cross entropy losses are used to optimize the classification branches of ARM and AADM respectively, and Smooth L1 loss is used to optimize the regression branches of ARM and AADM.

[0106] In one embodiment, to verify the technical effects of this application, the performance of the proposed target detection model was explored using the Seaships7000 public dataset. The nearshore ship target images contained in the Seaships7000 dataset were collected from visible light surveillance video systems deployed at multiple locations around Hengqin Island, Zhuhai, totaling 7,000 visible light images with a resolution of 1920 pixels × 1080 pixels. This dataset has relatively complete and diverse nearshore ship target data. The target types contained in the dataset are rich, and the scenes include multiple nearshore backgrounds such as buildings, mountains, and jungles. The images are collected under different lighting conditions in different seasons, which can fully verify the effectiveness of the algorithm in detecting and recognizing nearshore ship targets. The six types of ship targets contained in the dataset include ore carriers (OC), bulk carriers (BCC), general cargo carriers (GCC), container ships (CS), fishing ships (FS), and passenger ships (PS).

[0107] In order to fully verify the detection ability of the target detection model for near-shore ships, the average precision (AP) of MS COCO (Microsoft Common Objects in Context) was selected to evaluate different algorithms, and the AP values ​​and precision-recall (PR) curves of different algorithms for detection and identification of different categories of ships were statistically analyzed and plotted when the predicted results and the true values ​​were at a specific intersection-over-union (IoU) threshold. In addition, the frame rate (FPS) was used to evaluate the running speed of the algorithm. As shown in Table 1, Table 1 shows the comparison of the detection results of this application with other methods based on convolutional neural networks. It can be seen that the AARN proposed in this application is more accurate in predicting near-shore ships than other algorithms, and its AP value is 0.710. In addition, due to the more effective feature refinement and alignment strategy introduced in AARN, its AP 0.5 Value and AP 0.75 The values ​​of AARN are 0.965 and 0.853 respectively, which verifies the accuracy of AARN in detecting and positioning near-shore ship targets. When predicting medium-sized ship targets, AARN’s AP m Value and AP l The values ​​are 3.2% and 2.8% higher than RefineDet, further validating AARN's effectiveness in detecting medium- and large-sized nearshore vessels. Furthermore, the last column of Table 1 evaluates the runtime of various convolutional neural network-based object detection and recognition methods. Because AARN's backbone network utilizes the lightweight V-GG16 architecture with a 3×3 convolution kernel size, and its internal modules AFFM and AADM have very few parameters, AARN's computational speed is significantly superior to all other two-stage detectors and specific single-stage detectors (such as RPDet).

[0108] Table 1

[0109]

[0110] To fully validate the algorithm's detection and recognition capabilities for different types of ship targets, Table 2 shows the average detection accuracy of AARN and other convolutional neural network-based algorithms for six types of ship targets. It is clear that the algorithm achieves significant advantages over the two-stage and most single-stage algorithms for all six types of ship targets. However, while the RPDet algorithm achieves higher prediction accuracy for general cargo ships (GCCs) and ore carriers (OCs) than the proposed algorithm, its detection and recognition capabilities for other targets still lag significantly behind those of the proposed algorithm.

[0111] Table 2

[0112]

[0113] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0114] In one embodiment, Figure 5 As shown, a nearshore ship target detection device based on an anchor frame guided attention refinement network is provided, comprising: a detection model construction module 502, a detection model training module 504 and an image detection module 506, wherein:

[0115] The detection model construction module 502 is used to obtain a dataset of nearshore ship targets in the surveillance video; construct a target detection model based on an anchor-guided attention refinement network; the target detection model mainly includes an attention feature screening module and an anchor-guided alignment detection module;

[0116] The detection model training module 504 is used to construct an overall loss function of the target detection model; the target detection model with the constructed overall loss function is trained using the nearshore ship target dataset to obtain a trained target detection model;

[0117] The image detection module 506 is used to input the image of the nearshore ship target to be detected into the target detection model based on the trained target detection model, and output the category and location information of the nearshore ship target.

[0118] In one embodiment, the target detection model also includes an improved VGG-16 network, a transmission connection block and an ARM module; the improved VGG-16 network includes multiple base layers and newly added layers; the ARM module is located above the base layer and the newly added layer.

[0119] In one embodiment, the image detection module 506 is further configured to input an image of a nearshore vessel target to be detected into the target detection model based on the trained target detection model, and output the category and location information of the nearshore vessel target, including:

[0120] The improved VGG-16 network is used to extract features from the nearshore ship target image to be detected, and the basic feature map of the nearshore ship target is obtained;

[0121] The basic feature map of the nearshore ship target is transmitted to the transmission connection block using the newly added layer to generate the top-level feature map;

[0122] According to the attention feature screening module, the fine-grained feature map of the target is obtained from the top-level feature map. The anchor frame is used to guide the alignment detection module to detect the fine-grained feature map and obtain the category and location information of the nearshore ship target.

[0123] In one embodiment, the image detection module 506 is further configured to obtain a fine-grained feature map of the target from the top-level feature map according to the attention feature screening module, including:

[0124] The top-level feature map is input into the network convolution layer, and the top-level feature map is processed using the Sigmoid function to obtain the attention feature map;

[0125] Interpolate and upsample the attention feature map to obtain the attention refinement feature map;

[0126] Multiply the basic feature map of the nearshore ship target with the attention refined feature map, and perform element-wise superposition with the basic feature map to obtain the refined feature map;

[0127] Perform two upsampling operations on the top feature map to obtain a sampled feature map;

[0128] The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target.

[0129] In one embodiment, the image detection module 506 is further configured to perform element-wise superposition of the sampled feature map and the refined feature map and pass the superposition through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target, including:

[0130] The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain the fine-grained feature map of the nearshore ship target:

[0131]

[0132] in, represents the attention refinement feature map, represents the sampling feature map, represents the refined feature map, F represents the basic feature map of the nearshore ship target, and P i+1 Represents the top-level feature map, σ and δ represent the Sigmoid activation function and the ReLU activation function respectively, Up and Conv represent two upsampling and convolution operations respectively, and i represents the sequence number of the fine-grained feature map.

[0133] In one embodiment, the target detection module further includes a deformable convolution layer; and the image detection module 506 is further configured to use the anchor frame to guide the alignment detection module to detect the fine-grained feature map, thereby obtaining the category and location information of the nearshore ship target, including:

[0134] Generate a parameterized representation of the target position based on the ARM module, and use the parameterized representation to adjust the position of the predefined anchor box to obtain a refined anchor box;

[0135] Multiple original sampling points are randomly generated in the fine anchor frame, and the aligned sampling points of the original sampling points are found from the fine-grained feature map. The original sampling points and the corresponding aligned sampling points are calculated according to the deformable convolution to obtain the two-dimensional offset;

[0136] The two-dimensional offset and fine-grained feature map are input into the deformable convolution layer for regression and feature extraction to obtain the location information and category of nearshore ship targets.

[0137] In one embodiment, the original sampling point is

[0138]

[0139] Among them, i,j∈{0,1,k-1}, k=3 corresponds to the convolution operation with a convolution kernel size of 3×3.

[0140] In one embodiment, the alignment sampling points are

[0141]

[0142] Among them, (x1, y1, x2, y2) represents the position coordinates of the fine anchor box, and S represents the original sampling point.

[0143] In one embodiment, the two-dimensional offset is obtained by calculating based on the original sampling points and the corresponding aligned sampling points, including:

[0144] Calculate based on the original sampling points and the corresponding aligned sampling points, and get the two-dimensional offset:

[0145]

[0146] In one embodiment, the overall loss function is

[0147]

[0148] Among them, N ARM Indicates ARM loss, N AADM Represents the loss of the anchor box guided alignment detection module, N refers to the number of positive anchor boxes whose overlap with the true value of the label in all anchor boxes meets the predefined threshold, i represents the sequence number of the sampling point in the anchor box, represents the binary cross entropy loss and represents the binary cross entropy loss, p i and c i Represents the i th The anchor box contains the target and belongs to the cth th The probability of the class target, l i Indicates the i th The ground truth labels of the anchor boxes, in l i The output is 1 if ≥1, otherwise it is 0. i and t i represents the parameterized target position predicted by the network, represents the parameterized true value label.

[0149] Regarding the specific definition of a nearshore vessel target detection device based on an anchor frame guided attention refinement network, please refer to the definition of a nearshore vessel target detection method based on an anchor frame guided attention refinement network above, and will not be repeated here. The various modules in the above-mentioned nearshore vessel target detection device based on an anchor frame guided attention refinement network can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of the above-mentioned modules.

[0150] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting near-shore ship targets based on an anchor frame-guided attention refinement network is implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0151] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0152] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0153] In one embodiment, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0154] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0155] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for detecting nearshore ship targets based on an anchor frame guided attention refinement network, characterized in that: The method comprises: Obtain a dataset of nearshore ship targets in surveillance videos; Constructing an object detection model based on an anchor-guided attention refinement network; the object detection model mainly includes an attention feature screening module and an anchor-guided alignment detection module; Constructing an overall loss function of the target detection model; training the target detection model with the constructed overall loss function using a nearshore ship target dataset to obtain a trained target detection model; According to the trained target detection model, the image of the nearshore ship target to be detected is input into the target detection model, and the category and location information of the nearshore ship target is obtained as output; The target detection model also includes an improved VGG-16 network, a transport connection block, and an ARM module in the RefineDet network model; the improved VGG-16 network includes multiple base layers and newly added layers; the ARM module is located above the base layers and the newly added layers; The overall loss function is Among them, N ARM Indicates ARM loss, N AADM Represents the loss of the anchor box guided alignment detection module, N refers to the number of positive anchor boxes whose overlap with the true value of the label in all anchor boxes meets the predefined threshold, i represents the sequence number of the sampling point in the anchor box, represents the binary cross entropy loss and represents the binary cross entropy loss, p i and c i Represents the i th The anchor box contains the target and belongs to the cth th The probability of the class target, l i Indicates the i th The ground truth labels of the anchor boxes, in l i The output is 1 if ≥1, otherwise it is 0. i and t i represents the parameterized target position predicted by the network, represents the parameterized true value label.

2. The method according to claim 1, characterized in that According to the trained target detection model, the image of the nearshore ship target to be detected is input into the target detection model, and the category and location information of the nearshore ship target is output, including: The improved VGG-16 network is used to extract features from the nearshore ship target image to be detected, and the basic feature map of the nearshore ship target is obtained; Using the newly added layer, the basic feature map of the nearshore ship target is transmitted to the transmission connection block to generate a top-level feature map; According to the attention feature screening module, a fine-grained feature map of the target is obtained from the top-level feature map, and the anchor frame is used to guide the alignment detection module to detect the fine-grained feature map to obtain the category and location information of the nearshore ship target.

3. The method according to claim 2, characterized in that The process of obtaining a fine-grained feature map of a target from the top-level feature map according to the attention feature screening module includes: Input the top-level feature map into the network convolution layer, and process the top-level feature map using the Sigmoid function to obtain an attention feature map; Performing interpolation calculation and upsampling on the attention feature map to obtain a refined attention feature map; Multiplying the basic feature map of the nearshore ship target with the attention refined feature map, and performing element-wise superposition with the basic feature map to obtain a refined feature map; Performing two upsampling operations on the top-most feature map to obtain a sampled feature map; The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target.

4. The method according to claim 3, characterized in that The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target, including: The sampled feature map and the refined feature map are element-wise superimposed and passed through a 3×3 convolution layer and a ReLU activation function to obtain a fine-grained feature map of the nearshore ship target: in, represents the attention refinement feature map, represents the sampling feature map, represents the refined feature map, F represents the basic feature map of the ship target, and P i+1 Represents the top-level feature map, σ and δ represent the Sigmoid activation function and the ReLU activation function respectively, Up and Conv represent two upsampling and convolution operations respectively, and i represents the sequence number of the fine-grained feature map.

5. The method according to claim 4, characterized in that The target detection module further includes a deformable convolution layer; the anchor frame is used to guide the alignment detection module to detect the fine-grained feature map to obtain the category and location information of the nearshore ship target, including: Generate a parameterized representation of the target position according to the ARM module, and use the parameterized representation to adjust the position of the predefined anchor box to obtain a refined anchor box; A plurality of original sampling points randomly generated in the fine anchor frame, finding the aligned sampling points of the original sampling points from the fine-grained feature map, and calculating the original sampling points and the corresponding aligned sampling points according to the deformable convolution to obtain a two-dimensional offset; The two-dimensional offset and fine-grained feature map are input into the deformable convolution layer for regression and feature extraction to obtain the location information and category of the nearshore ship target.

6. The method according to claim 5, characterized in that The original sampling point is Among them, i,j∈{0,1,k-1}, k=3 corresponds to the convolution operation with a convolution kernel size of 3×3.

7. The method according to claim 6, characterized in that The alignment sampling points are Among them, (x1, y1, x2, y2) represents the position coordinates of the fine anchor box, and S represents the original sampling point.

8. The method according to claim 7, characterized in that Calculation is performed based on the original sampling points and the corresponding aligned sampling points to obtain a two-dimensional offset, including: Calculation is performed based on the original sampling points and the corresponding aligned sampling points to obtain the two-dimensional offset:

Images