A ship target detection method in SAR images based on improved YOLOV7 network
By improving the YOLOV7 network, combining the pyramid pooling attention module and the fast fully connected space pyramid pooling module, combined with the irrelevant information removal module and PANet, the problem of insufficient ship detection accuracy in SAR images is solved, and higher detection accuracy is achieved and real-time.
Patent Information
- Application Number
- CN202411047347.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-01
AI Technical Summary
The existing SAR image ship detection methods have insufficient accuracy in background information, noise interference and multi-scale ship detection, making it difficult to achieve real-time and high detection accuracy.
Adopting the improved YOLOV7 network, introducing the pyramid pooling attention module, the fast fully connected space pyramid pooling module, and combining the irrelevant information removal module and PANet to build a ship detection model, and construct a priori box through clustering to reduce false detection and Missing inspection improves detection accuracy.
Under the condition of real-time performance, the accuracy of ship target detection is significantly improved, false detection and missed detection is reduced, detection accuracy is improved, and multi-scale detection performance is shown.
Smart Images

Figure CN118570482B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a SAR image ship target detection method based on an improved YOLOV7 network, and belongs to the technical field of SAR image target detection. Background Art
[0002] Synthetic Aperture Radar (SAR) is an active microwave imaging sensor that can detect targets at any time and in any weather. It has strong penetration capabilities for soil, vegetation, and clouds, and is widely used in military and civilian fields, such as environmental monitoring, oil spill detection, maritime rescue, and maritime transportation. Therefore, it is of great significance to be able to detect ship targets in real time and accurately.
[0003] One of the traditional SAR ship detection methods is to use a constant false alarm rate (CFAR) detector. CFAR needs to set the detection threshold based on a pre-established clutter statistical model, so the detection accuracy is limited.
[0004] In recent years, deep learning has made significant breakthroughs in many fields such as computer vision and natural language processing. The advantage of deep learning lies in its powerful feature extraction and feature learning capabilities. Compared with traditional manual feature extraction methods, it reduces human intervention and assumptions. In the study of ship detection in SAR images based on deep learning, there are often problems such as background information, noise interference, and multi-scale ships. Therefore, how to propose a multi-scale ship detection method with strong real-time performance and high detection accuracy is a technical problem that needs to be solved urgently. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a SAR image ship target detection method based on an improved YOLOV7 network, introduce a pyramid pooling attention module, a fast fully connected spatial pyramid pooling module, and combine an irrelevant information elimination module and PANet to build a training ship detection model, reduce false detections and missed detections, and improve the detection accuracy of ship target detection while meeting real-time requirements.
[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions: the present invention designs a SAR image ship target detection method based on an improved YOLOV7 network, performs the following steps A to C, obtains a ship detection model, and then applies the ship detection model to identify the position of each ship object in the SAR image to be analyzed:
[0007] Step A. Based on a preset number of sample SAR images containing ship objects and with known positions of the ship objects, construct a priori boxes corresponding to the ship objects of large, medium and small classification scales in a clustering manner, and then proceed to step B;
[0008] Step B. Construct a network to be trained including a backbone network Backbone, a polymerization network PANet, and a head network Head, and used to output large, medium, and small scale ship object detection, wherein the input end of the backbone network Backbone constitutes the input end of the network to be trained, and each output end of the backbone network Backbone is connected to each input end of the polymerization network PANet through a first irrelevant information elimination module and a second irrelevant information elimination module, and the three output ends of the polymerization network PANet are respectively connected to the input ends of the sub-head networks of the head network Head for large, medium, and small scale ship objects, and the output ends of each sub-head network in the head network Head constitute each output end of the network to be trained, which is respectively used to output large, medium, and small scale ship object detection, and then enter step C;
[0009] Step C. Based on each sample SAR image, according to the prior frames corresponding to the large, medium and small scale ship objects, the sample SAR image is used as input, and the positions of the large, medium and small scale ship objects in the sample SAR image are used as output. The network to be trained is trained to obtain a ship detection model.
[0010] As a preferred technical solution of the present invention: the backbone network Backbone includes, from its input to its output direction, a first convolution normalization activation module, a second convolution normalization activation module, a third convolution normalization activation module, a first pyramid pooling attention module, a first multiplication module, a fourth convolution normalization activation module, a first multidimensional stacking module, a second pyramid pooling attention module, a second multiplication module, a first transition module, a second multidimensional stacking module, a third pyramid pooling attention module, a third multiplication module, a second transition module, a third multidimensional stacking module, a fourth pyramid pooling attention module, a fourth multiplication module, a third transition module, a fourth multidimensional stacking module, a fifth pyramid pooling attention module, a fifth multiplication module, and a fast fully connected spatial pyramid pooling module;
[0011] Among them, the output end of the third convolution normalization activation module is simultaneously connected to the input end of the first multiplication module. For the feature map output by the third convolution normalization activation module, the feature map is updated by multiplying the feature map with the weight output by the first pyramid pooling attention module about the feature map, and then the feature map is transmitted to the input end of the fourth convolution normalization activation module; the output end of the first multidimensional stacking module is simultaneously connected to the input end of the second multiplication module. For the feature map output by the first multidimensional stacking module, the feature map is updated by multiplying the feature map with the weight output by the second pyramid pooling attention module about the feature map, and then the feature map is transmitted to the input end of the first transition module; the output end of the second multidimensional stacking module is simultaneously connected to the input end of the third multiplication module. For the feature map output by the second multidimensional stacking module, the feature map is updated by multiplying the feature map with the weight output by the second pyramid pooling attention module about the feature map, and then the feature map is transmitted to the input end of the first transition module. The feature map is multiplied by the weight of the feature map output by the third pyramid pooling attention module to update the feature map, and then the feature map is transmitted to the input end of the second transition module; the output end of the third multidimensional stacking module is simultaneously connected to the input end of the fourth multiplication module, and for the feature map output by the third multidimensional stacking module, the feature map is updated by multiplying the weight of the feature map output by the fourth pyramid pooling attention module, and then the feature map is transmitted to the input end of the third transition module; the output end of the fourth multidimensional stacking module is simultaneously connected to the input end of the fifth multiplication module, and for the feature map output by the fourth multidimensional stacking module, the feature map is updated by multiplying the weight of the feature map output by the fifth pyramid pooling attention module, and then the feature map is transmitted to the input end of the fast fully connected spatial pyramid pooling module;
[0012] The input end of the first convolution normalization activation module constitutes the input end of the backbone network Backbone, and the output end of the third multiplication module, the output end of the fourth multiplication module, the output end of the fifth multiplication module, and the output end of the fast fully connected spatial pyramid pooling module constitute the output ends of the backbone network Backbone;
[0013] The output end of the third multiplication module and the output end of the fourth multiplication module in the backbone network Backbone are respectively connected to the two input ends of the second irrelevant information elimination module; the output end of the fourth multiplication module and the output end of the fifth multiplication module in the backbone network Backbone are respectively connected to the two input ends of the first irrelevant information elimination module; the output end of the second irrelevant information elimination module, the output end of the first irrelevant information elimination module, and the output end of the fast fully connected spatial pyramid pooling module are respectively connected to the input ends of the aggregation network PANet.
[0014] As a preferred technical solution of the present invention: in the structure of the aggregation network PANet, the input end of the first convolution module constitutes the first input end of the aggregation network PANet connected to the output end of the fast fully connected spatial pyramid pooling module in the backbone network Backbone, and the output end of the first convolution module is sequentially connected in series with the first upsampling module, the first stacking and multi-dimensional stacking module, the third convolution module, the second upsampling module, the second stacking and multi-dimensional stacking module, the first downsampling module, the fourth transition module, the third stacking and multi-dimensional stacking module, the second downsampling module, the fifth transition module, the fourth stacking and multi-dimensional stacking module;
[0015] The input ends of the fourth stacking and multi-dimensional stacking modules simultaneously constitute the second input end of the aggregation network PANet and are connected to the output end of the fast fully connected spatial pyramid pooling module in the backbone network Backbone, and the output ends of the fourth stacking and multi-dimensional stacking modules constitute the first output end of the aggregation network PANet;
[0016] The input end of the second convolution module constitutes the third input end of the aggregation network PANet and is connected to the output end of the first irrelevant information elimination module. The input ends of the first stacking and multi-dimensional stacking modules are simultaneously connected to the output end of the second convolution module. The output ends of the first stacking and multi-dimensional stacking modules are simultaneously connected to the input ends of the third stacking and multi-dimensional stacking modules. The output ends of the third stacking and multi-dimensional stacking modules constitute the second output end of the aggregation network PANet.
[0017] The input end of the fourth convolution module constitutes the fourth input end of the aggregation network PANet and is connected to the output end of the second irrelevant information elimination module. The input ends of the second stacking and multi-dimensional stacking modules are simultaneously connected to the output ends of the fourth convolution module. The output ends of the second stacking and multi-dimensional stacking modules constitute the third output end of the aggregation network PANet.
[0018] The first stacking and multi-dimensional stacking module, the second stacking and multi-dimensional stacking module, the third stacking and multi-dimensional stacking module, and the fourth stacking and multi-dimensional stacking module are respectively stacking modules and multi-dimensional stacking modules connected in series from input to output.
[0019] As a preferred technical solution of the present invention: the structures of the first transition module, the second transition module, the third transition module, the fourth transition module, and the fifth transition module are the same. In each transition module structure, one branch is formed by a 2×2 convolution kernel global maximum pooling module and a convolution normalization activation module connected in series from the input to the output direction, and the other branch is formed by two convolution normalization activation modules connected in series from the input to the output direction. The input ends of the two branches are connected to form the input end of the transition module structure, and the output ends of the two branches are connected to the input end of the stacking module. The output end of the stacking module constitutes the output end of the transition module structure, and the stacking module connects the outputs of the two branches received.
[0020] As a preferred technical solution of the present invention: the sub-head network for large-scale ship objects in the head network Head includes a first parameterized convolution module and a first detection head module connected in series, the input end of the first parameterized convolution module constitutes the first input end of the head network Head connected to the third output end of the aggregation network PANet, and the output end of the first detection head module constitutes the first output end of the head network Head; the sub-head network for medium-scale ship objects includes a second parameterized convolution module and a second detection head module connected in series, the input end of the second parameterized convolution module constitutes the second input end of the head network Head connected to the second output end of the aggregation network PANet, and the output end of the second detection head module constitutes the second output end of the head network Head; the sub-head network for small-scale ship objects includes a third parameterized convolution module and a third detection head module connected in series, the input end of the third parameterized convolution module constitutes the third input end of the head network Head connected to the first output end of the aggregation network PANet, and the output end of the third detection head module constitutes the third output end of the head network Head.
[0021] As a preferred technical solution of the present invention: the structures of the first pyramid pooling attention module, the second pyramid pooling attention module, the third pyramid pooling attention module, the fourth pyramid pooling attention module, and the fifth pyramid pooling attention module are identical to each other. In each pyramid pooling attention module structure, the input end of the 5×5 convolution kernel global maximum pooling module, the input end of the 9×9 convolution kernel global maximum pooling module, and the input end of a global average pooling module are connected to form the input end of the pyramid pooling attention module structure, which is used to receive the feature map. The output end of the 5×5 convolution kernel global maximum pooling module and the output end of the 9×9 convolution kernel global maximum pooling module are respectively connected to the input end of the other two global average pooling modules in a one-to-one correspondence. At the end, three global average pooling modules respectively reduce the dimensions of the feature maps they receive, and the output ends of the three global average pooling modules are respectively connected to the input ends of the three 1×1 convolution kernels, and the weights of the feature maps received by each 1×1 convolution kernel are obtained respectively. The output ends of the three 1×1 convolution kernels are connected to the input end of the superposition activation module, and the output end of the superposition activation module constitutes the output end of the pyramid pooling attention module structure. The superposition activation module is sequentially connected in series with the superposition module and the Sigmod function from the input to the output direction. The superposition activation module first applies the superposition module to perform a weight superposition operation on the weights output by the three 1×1 convolution kernels, and then obtains the weight of the feature map received at the input end of the pyramid pooling attention module structure through the Sigmod function.
[0022] As a preferred technical solution of the present invention: the structures of the first multidimensional stacking module, the second multidimensional stacking module, the third multidimensional stacking module, and the fourth multidimensional stacking module are the same. In each multidimensional stacking module structure, one branch is composed of a convolutional normalization activation module, and another branch is composed of five convolutional normalization activation modules connected in series from input to output. The input ends of the two branches are connected to form the input end of the multidimensional stacking module structure, the output end of the branch composed of a single convolutional normalization activation module, and the output end of the first convolutional normalization activation module in sequence in another branch. , the output end of the third convolution normalization activation module, and the output end of the fifth convolution normalization activation module are connected to the input end of the stacking and convolution normalization activation module. The output ends of the stacking and convolution normalization activation modules constitute the output end of the multi-dimensional stacking module structure. The stacking modules and convolution normalization activation modules are sequentially connected in series from input to output. The stacking and convolution normalization activation modules first apply the stacking modules to connect each received feature map, and then input it into a convolution normalization activation module to perform convolution, normalization, and activation operations in sequence.
[0023] As a preferred technical solution of the present invention: in the structure of the fast fully connected spatial pyramid pooling module in the backbone network Backbone, four convolution normalization activation modules, a 1×1 convolution kernel global maximum pooling module, and nine 3×3 convolution kernel global maximum pooling modules are sequentially connected in series from input to output, wherein the input end of the first convolution normalization activation module in sequence constitutes the input end of the fast fully connected spatial pyramid pooling module, the output end of the fourth convolution normalization activation module in sequence, the output end of the 1×1 convolution kernel global maximum pooling module, the output end of the first sequential 3×3 convolution kernel global maximum pooling module, the output end of the third sequential 3×3 convolution kernel global maximum pooling module, the output end of the fifth sequential 3×3 convolution kernel global maximum pooling module, and the output end of the seventh sequential 3×3 convolution kernel global maximum pooling module , the output end of the ninth sequential 3×3 convolution kernel global maximum pooling module is connected to the input end of the stacking module, and the stacking module is used to connect each received feature map. The output end of the stacking module is connected in series with three convolution normalization activation modules in sequence. The input end of the fast fully connected spatial pyramid pooling module is also connected to the input end of another eighth convolution normalization activation module. The output end of the eighth convolution normalization activation module and the output end of the third convolution normalization activation module connected in series with the stacking module are connected to the input end of the stacking and convolution normalization activation module. The output ends of the stacking and convolution normalization activation modules constitute the output end of the fast fully connected spatial pyramid pooling module. The stacking and convolution normalization activation modules first apply the stacking module to connect the two received feature maps, and then input them into a convolution normalization activation module to perform convolution, normalization, and activation operations in sequence.
[0024] As a preferred technical solution of the present invention: the structure of the first irrelevant information elimination module is the same as that of the second irrelevant information elimination module, and each irrelevant information elimination module includes a first branch, a second branch, and a subtraction module. In the structure of each irrelevant information elimination module, the first branch is composed of a dot product module and an addition module connected in series in sequence, the input end of the dot product module constitutes the input end of the first branch, the output end of the addition module constitutes the output end of the first branch, and the input end of the addition module is simultaneously connected to the input end of the dot product module; the second branch is composed of an upsampling module, a sub-parallel structure, a dot product module, and a sigmiod function connected in series in sequence from input to output, the sub-parallel structure includes a global average pooling module and a global maximum pooling module connected in parallel to each other, and the input end of the upsampling module constitutes the second branch. The input end of the second branch and the output end of the Sigmiod function constitute the output end of the second branch; the two input ends of the subtraction module are respectively connected to the output end of the second branch and the feature map whose eigenvalues of each channel are 1, and the input end of the point multiplication module in the first branch is also connected to the output end of the subtraction module; the input end of the first branch constitutes one of the input ends of the irrelevant information elimination module, which is used to access the output end of the first multiplication module in the sequence of two multiplication modules in the backbone network Backbone connected to the irrelevant information elimination module, the input end of the second branch constitutes the other input end of the irrelevant information elimination module, which is used to access the output end of the second multiplication module in the sequence of two multiplication modules in the backbone network Backbone connected to the irrelevant information elimination module, and the output end of the first branch constitutes the output end of the irrelevant information elimination module.
[0025] As a preferred technical solution of the present invention: in the training process of the network to be trained, the step C uses CIoU as the loss function, and the calculation formula is as follows:
[0026]
[0027]
[0028]
[0029] In the formula, is the intersection-over-union ratio between the predicted box and the real box of the ship object in the SAR image, Represents the diagonal distance of the minimum closure area of the predicted box and the real box that contain the ship object at the same time, , Represent the center points of the predicted box and the real box of the ship object respectively, and Represents the Euclidean distance between the center point of the predicted box of the ship object and the center point of the real box, represents the weight function, Used to measure the similarity of aspect ratios, and are the width and height of the real frame of the ship object, and are the width and height of the prediction box of the ship object respectively.
[0030] The SAR image ship target detection method based on the improved YOLOV7 network described in the present invention adopts the above technical solution compared with the prior art, and has the following technical effects:
[0031] The present invention designs a SAR image ship target detection method based on an improved YOLOV7 network. Based on the backbone network Backbone, a pyramid pooling attention module and a fast fully connected spatial pyramid pooling module are introduced, and an irrelevant information removal module and PANet are combined to construct a network to be trained. Sample SAR images with known ship object positions are used to train the network to be trained to obtain a ship detection model. Then, for the SAR image to be analyzed, the ship detection model is applied to identify the position of each ship object in the SAR image to be analyzed. The obtained ship detection model is designed to reduce false detections and missed detections in applications while meeting real-time requirements, and improve the detection accuracy of ship target detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a flow chart of the SAR image ship target detection method based on the improved YOLOV7 network of the present invention;
[0033] Figure 2 It is a schematic diagram of the structure of the network to be trained in the design of the present invention;
[0034] Figure 3 It is a network structure diagram of the pyramid pooling attention module in the design of the present invention;
[0035] Figure 4 It is a network structure diagram of the multi-dimensional stacking module in the design of the present invention;
[0036] Figure 5 It is a network structure diagram of the transition module in the design of the present invention;
[0037] Figure 6 It is a network structure diagram of the fast fully connected spatial pyramid pooling module in the design of the present invention;
[0038] Figure 7 A network structure diagram of an irrelevant information elimination module in the design of the present invention;
[0039] Figure 8 This is the detection result diagram of the original YOLOV7;
[0040] Fig. 9 This is the detection result diagram of the improved YOLOV7 of the present invention. DETAILED DESCRIPTION
[0041] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings.
[0042] The present invention designs a SAR image ship target detection method based on an improved YOLOV7 network. In practical application design, Figure 1 As shown, execute the following steps A to C to obtain a ship detection model.
[0043] Step A. Based on a preset number of sample SAR images containing ship objects and with known positions of the ship objects, construct a priori boxes corresponding to ship objects of large, medium and small classification scales in a clustering manner, and then proceed to step B.
[0044] In practical applications, the following data processing is performed on each sample SAR image here.
[0045] (1) Convert the collected sample SAR images into a dataset format suitable for YOLOV7 training.
[0046] (2) The k-means algorithm is used to cluster the real frames of the ship objects in each sample SAR image into three categories: large, medium, and small, and the prior frames corresponding to the ship objects of large, medium, and small scales are obtained.
[0047] (3) Adjust the width and height of each sample SAR image to a uniform size.
[0048] Step B. Construct a network to be trained including a backbone network Backbone, a polymerization network PANet, and a head network Head, and used to output large, medium, and small scale ship object detection, wherein the input end of the backbone network Backbone constitutes the input end of the network to be trained, and each output end of the backbone network Backbone is connected to each input end of the polymerization network PANet through a first irrelevant information elimination module (Irrelevant information elimination, IIE) and a second irrelevant information elimination module (Irrelevant information elimination, IIE), and the three output ends of the polymerization network PANet are respectively connected to the input ends of the sub-head network of the head network Head about the large, medium, and small scale ship objects, and the output ends of each sub-head network in the head network Head constitute the output ends of the network to be trained, which are respectively used to output large, medium, and small scale ship object detection, and then enter step C.
[0049] In the actual construction design of the network to be trained, the above step B is as follows: Figure 2As shown, the backbone network Backbone is specifically designed to include the first convolution normalization activation module (Conv2D_BN_SiLU), the second convolution normalization activation module (Conv2D_BN_SiLU), the third convolution normalization activation module (Conv2D_BN_SiLU), the first pyramid pooling attention module (PPAM), the first multiplication module, the fourth convolution normalization activation module (Conv2D_BN_SiLU), the first multi-dimensional stacking module (Multi_Concat), the second pyramid pooling attention module (PPAM), the second multiplication module, the first transition module (Transiti on_Block), the second multi-dimensional stacking module (Multi_Concat), the third pyramid pooling attention module (PPAM), the third multiplication module, the second transition module (Transition_Block), the third multi-dimensional stacking module (Multi_Concat), the fourth pyramid pooling attention module (PPAM), the fourth multiplication module, the third transition module (Transition_Block), the fourth multi-dimensional stacking module (Multi_Concat), the fifth pyramid pooling attention module (PPAM), the fifth multiplication module, the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC).
[0050] Among them, the output end of the third convolution normalization activation module (Conv2D_BN_SiLU) is simultaneously connected to the input end of the first multiplication module. For the feature map output by the third convolution normalization activation module (Conv2D_BN_SiLU), the feature map is updated by multiplying the feature map with the weight output by the first pyramid pooling attention module (PPAM) about the feature map, and then the feature map is transmitted to the input end of the fourth convolution normalization activation module (Conv2D_BN_SiLU); the output end of the first multi-dimensional stacking module (Multi_Concat) is connected to the input end of the first multi-dimensional stacking module (Multi_Concat). The output end of the second multiplication module is connected to the input end of the second multiplication module. For the feature map output by the first multi-dimensional stacking module (Multi_Concat), the feature map is updated by multiplying the feature map with the weight output by the second pyramid pooling attention module (PPAM) about the feature map, and then the feature map is transmitted to the input end of the first transition module (Transition_Block); the output end of the second multi-dimensional stacking module (Multi_Concat) is connected to the input end of the third multiplication module at the same time. For the feature map output by the second multi-dimensional stacking module (Multi_Concat), , the feature map is updated by multiplying the weight of the feature map output by the third pyramid pooling attention module (PPAM), and then the feature map is transmitted to the input end of the second transition module (Transition_Block); the output end of the third multi-dimensional stacking module (Multi_Concat) is simultaneously connected to the input end of the fourth multiplication module, and for the feature map output by the third multi-dimensional stacking module (Multi_Concat), the feature map is updated by multiplying the weight of the feature map output by the fourth pyramid pooling attention module (PPAM), and then the feature map is transmitted to the input end of the third transition module (Transition_Block); the output end of the fourth multi-dimensional stacking module (Multi_Concat) is simultaneously connected to the input end of the fifth multiplication module, and for the feature map output by the fourth multi-dimensional stacking module (Multi_Concat), the feature map is updated by multiplying the weight of the feature map output by the fifth pyramid pooling attention module (PPAM), and then the feature map is transmitted to the input end of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC).
[0051] The input end of the first convolution normalization activation module (Conv2D_BN_SiLU) constitutes the input end of the backbone network Backbone, and the output end of the third multiplication module, the output end of the fourth multiplication module, the output end of the fifth multiplication module, and the output end of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC) constitute the output ends of the backbone network Backbone.
[0052] In order to improve the detection accuracy, a top-down approach is usually adopted, that is, the feature information of different scales is fused, so that the shallow feature map is responsible for detecting small targets, while the deep feature map is responsible for detecting large targets. However, for the shallow feature map where large and small targets exist at the same time, the information of large targets is more significant. After the feature fusion through the feature pyramid, the non-significant information such as small targets may be lost. Therefore, it becomes very difficult to detect small targets in the shallow feature map. In order to deal with the above problems, the design adds an irrelevant information elimination module (IIE) in front of the aggregation network PANet. The irrelevant information elimination module (IIE) is used to purify the target information of the shallow and medium feature maps, and extract pure feature maps containing only small and medium-sized ship target information respectively; then, the multi-scale feature maps of these pure information are fused by the aggregation network PANet. In this case, the salient features of the shallow feature map only contain small targets, which reduces the problem of missed detection of small and medium-sized ship targets caused by feature confusion before passing them to the aggregation network PANet.
[0053] like Figure 2 As shown, the output end of the third multiplication module and the output end of the fourth multiplication module in the backbone network Backbone are specifically designed to be connected to the two input ends of the second irrelevant information elimination module (Irrelevant information elimination, IIE) respectively, and the second irrelevant information elimination module (Irrelevant information elimination, IIE) performs irrelevant information elimination on the feature map; the output end of the fourth multiplication module and the output end of the fifth multiplication module in the backbone network Backbone are respectively connected to the two input ends of the first irrelevant information elimination module (Irrelevant information elimination, IIE); the output end of the second irrelevant information elimination module (Irrelevant information elimination, IIE), the output end of the first irrelevant information elimination module (Irrelevant information elimination, IIE), and the output end of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC) are respectively connected to the input ends of the aggregation network PANet.
[0054] Regarding the structure of the first irrelevant information elimination module (Irrelevant information elimination, IIE) and the structure of the second irrelevant information elimination module (Irrelevant information elimination, IIE), in practical applications, the designs are the same as each other, such as Figure 7 As shown, each irrelevant information elimination module (IIE) includes a first branch, a second branch, and a subtraction module. In the structure of IIE, a first branch is composed of a dot product module and an addition module connected in series in sequence, the input end of the dot product module constitutes the input end of the first branch, the output end of the addition module constitutes the output end of the first branch, and the input end of the addition module is simultaneously connected to the input end of the dot product module; a second branch is composed of an up-sampling module (Up_Sampling), a sub-parallel structure, a dot product module, and a Sigmiod function connected in series in sequence from input to output, the sub-parallel structure includes a global average pooling module (AvgPool2d) and a global maximum pooling module (MaxPool2d) connected in parallel to each other, the input end of the up-sampling module (Up_Sampling) constitutes the input end of the second branch, and the output end of the Sigmiod function constitutes the output end of the second branch; the two input ends of the subtraction module are respectively connected to the output end of the second branch and the feature map T1 whose eigenvalues of each channel are 1, and the input end of the dot product module in the first branch is simultaneously connected to the output end of the subtraction module; the input end of the first branch constitutes an irrelevant information elimination module (Irrelevant information elimination module One of the input ends of the irrelevant information elimination module (Irrelevant information elimination, IIE) is used to access the output end of the first multiplication module in the sequence of two multiplication modules in the backbone network Backbone connected to the irrelevant information elimination module (Irrelevant information elimination, IIE), and the input end of the second branch constitutes the other input end of the irrelevant information elimination module (Irrelevant information elimination, IIE), which is used to access the output end of the second multiplication module in the sequence of two multiplication modules in the backbone network Backbone connected to the irrelevant information elimination module (Irrelevant information elimination, IIE), and the output end of the first branch constitutes the output end of the irrelevant information elimination module (Irrelevant information elimination, IIE).
[0055] In the application, the design of the irrelevant information elimination (IIE) module structure is as follows:
[0056] (1) First, for the deeper feature map P S+1 Up-sampling is performed, and then average pooling and maximum pooling are performed respectively. Then, more significant information in space and channels is obtained by multiplying the corresponding elements. Then, the Sigmoid function is used to scale the result to between 0 and 1. Since the deep feature map obtains the information characteristics of larger ship targets, we introduce a feature map T1 with the same number of channels and the characteristic value of each channel is 1, and use it to subtract the result processed by the Sigmoid function, which is the feature information of the smaller ship target.
[0057] (2) Then design the feature map P of the current layer S With P S+1 The processed results correspond to element multiplication, and then P S Adding it to the result of multiplying the two, the feature is amplified to enhance the characteristic information of smaller ship targets.
[0058] In the backbone network Backbone, a pyramid pooling attention module (PPAM) is connected in series behind each multi-dimensional stacking module (Multi_Concat). The pooling structure is used to reduce the interference of noise and background information on weight generation, ensuring the correct distribution of feature map weights and suppressing noise and background information. In addition, the SPPCSPC in the original YOLOV7 network is replaced with a fast fully connected spatial pyramid pooling module (SPPF_t-CSPC), using different pooling kernels of the same size and combining with the multi-dimensional stacking module (Multi_Concat) to capture more small ship target features. Finally, the irrelevant information elimination module (Irrelevant information elimination, IIE) is used to suppress the ship features in the backbone network Backbone that are inconsistent with the detection scale of the current layer feature map, and then send them to the aggregation network PANet for fusion.
[0059] Regarding the structure of the aggregation network PANet, Figure 2As shown, the specific design is that the input end of the first convolution module (Conv) constitutes the first input end of the aggregation network PANet connected to the output end of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC) in the backbone network Backbone, and the output end of the first convolution module (Conv) is connected in series with the first up-sampling module (Up_Sampling), the first stacking and multi-dimensional stacking module (Concat+Multi_Concat), the third convolution module (Conv), the second up-sampling module (Up_Sampling), the second stacking and multi-dimensional stacking module (Concat+Multi_Concat), the first down-sampling module (Down_Sampling), the fourth transition module (Transition_Block), the third stacking and multi-dimensional stacking module (Concat+Multi_Concat), the second down-sampling module (Down_Sampling), the fifth transition module (Transition_Block), the fourth stacking and multi-dimensional stacking module (Concat+Multi_Concat).
[0060] The input ends of the fourth stacking and multi-dimensional stacking modules (Concat+Multi_Concat) simultaneously constitute the second input end of the aggregation network PANet and are connected to the output end of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC) in the backbone network Backbone, and the output ends of the fourth stacking and multi-dimensional stacking modules (Concat+Multi_Concat) constitute the first output end of the aggregation network PANet;
[0061] The input end of the second convolution module (Conv) constitutes the third input end of the aggregation network PANet and is connected to the output end of the first irrelevant information elimination module (IIE). The input ends of the first stacking and multi-dimensional stacking modules (Concat+Multi_Concat) are simultaneously connected to the output end of the second convolution module (Conv). The output ends of the first stacking and multi-dimensional stacking modules (Concat+Multi_Concat) are simultaneously connected to the input ends of the third stacking and multi-dimensional stacking modules (Concat+Multi_Concat). The output ends of the third stacking and multi-dimensional stacking modules (Concat+Multi_Concat) constitute the second output end of the aggregation network PANet.
[0062] The input end of the fourth convolution module (Conv) constitutes the fourth input end of the aggregation network PANet and is connected to the output end of the second irrelevant information elimination module (IIE). The input end of the second stacking and multi-dimensional stacking module (Concat+Multi_Concat) is simultaneously connected to the output end of the fourth convolution module (Conv). The output end of the second stacking and multi-dimensional stacking module (Concat+Multi_Concat) constitutes the third output end of the aggregation network PANet.
[0063] The first stacking and multi-dimensional stacking module (Concat+Multi_Concat), the second stacking and multi-dimensional stacking module (Concat+Multi_Concat), the third stacking and multi-dimensional stacking module (Concat+Multi_Concat), and the fourth stacking and multi-dimensional stacking module (Concat+Multi_Concat) are respectively stacking modules (Concat) and multi-dimensional stacking modules (Multi_Concat) that are connected in series from input to output.
[0064] like Figure 5 As shown, in the application, the structures of the first transition module (Transition_Block), the second transition module (Transition_Block), the third transition module (Transition_Block), the fourth transition module (Transition_Block), and the fifth transition module (Transition_Block) are the same. In the structure of each transition module (Transition_Block), a 2×2 convolution kernel global maximum pooling module (MaxPool2d) and a convolution normalization activation module (Conv2D_BN_SiLU) are connected in series from the input to the output direction to form one branch, and two convolution normalization activation modules (Conv2D_BN_SiLU) are connected in series from the input to the output direction to form another branch. The input ends of the two branches are connected to form the input end of the transition module (Transition_Block) structure, and the output ends of the two branches are connected to the input end of the stacking module (Concat). The output end of the stacking module (Concat) constitutes the output end of the transition module (Transition_Block) structure, and the stacking module (Concat) connects the outputs of the two branches received to compress the size of the feature map.
[0065] Regarding the head network Head, such as Figure 2As shown, the head network Head is specifically designed to include sub-head networks for large, medium and small classification scale ship objects, wherein the sub-head network for large classification scale ship objects includes a first parameterized convolution module (ReConv) and a first detection head module (Head) connected in series, the input end of the first parameterized convolution module (ReConv) constitutes the first input end of the head network Head connected to the third output end of the aggregation network PANet, and the output end of the first detection head module (Head) constitutes the first output end of the head network Head; the sub-head network for medium classification scale ship objects includes a second parameterized convolution module (ReConv) and a second detection head module (Head) connected in series ead), the input end of the second parameterized convolution module (ReConv) constitutes the second input end of the head network Head connected to the second output end of the aggregation network PANet, and the output end of the second detection head module (Head) constitutes the second output end of the head network Head; the sub-head network for small classification scale ship objects includes a third parameterized convolution module (ReConv) and a third detection head module (Head) connected in series, the input end of the third parameterized convolution module (ReConv) constitutes the third input end of the head network Head connected to the first output end of the aggregation network PANet, and the output end of the third detection head module (Head) constitutes the third output end of the head network Head.
[0066] Regarding the design of the above-mentioned network structure to be trained, the structures of the first pyramid pooling attention module (PPAM), the second pyramid pooling attention module (PPAM), the third pyramid pooling attention module (PPAM), the fourth pyramid pooling attention module (PPAM), and the fifth pyramid pooling attention module (PPAM) are the same as each other. Figure 3As shown in the figure, in the specific design of each pyramid pooling attention module (PPAM) structure, the input end of the 5×5 convolution kernel global maximum pooling module (MaxPool2d), the input end of the 9×9 convolution kernel global maximum pooling module (MaxPool2d), and the input end of a global average pooling module (GAP) are connected to form the input end of the pyramid pooling attention module (PPAM) structure, which is used to receive the feature map. Here, the 5×5 convolution kernel global maximum pooling module (MaxPool2d) and the 9×9 convolution kernel global maximum pooling module (MaxPool2d) increase the contrast of multi-scale ship targets and background information for the received feature map; the output end of the 5×5 convolution kernel global maximum pooling module (MaxPool2d) and the output end of the 9×9 convolution kernel global maximum pooling module (MaxPool2d) are connected to the other two global average pooling modules (GAP) one by one. The input end of the stacking activation module (Add+s) is connected to the input end of the three 1×1 convolution kernels, and the output end of the stacking activation module (Add+s) constitutes the output end of the pyramid pooling attention module (PPAM) structure. The stacking activation module (Add+s) is connected in series with the stacking module (Add) and the Sigmod function from the input to the output direction. The stacking activation module (Add+s) first applies the stacking module (Add) to the weights of the three 1×1 convolution kernel outputs for weight stacking operation, and then obtains the weight of the feature map received by the input end of the pyramid pooling attention module (PPAM) structure through the Sigmod function.
[0067] In the application, the structures of the first multi-dimensional stacking module (Multi_Concat), the second multi-dimensional stacking module (Multi_Concat), the third multi-dimensional stacking module (Multi_Concat), and the fourth multi-dimensional stacking module (Multi_Concat) are designed to be identical to each other, such as Figure 4As shown in the figure, in the specific design of each multi-dimensional stacking module (Multi_Concat) structure, one branch is composed of a convolutional normalized activation module (Conv2D_BN_SiLU), and another branch is composed of five convolutional normalized activation modules (Conv2D_BN_SiLU) connected in series from input to output. The input ends of the two branches are connected to form the input end of the multi-dimensional stacking module (Multi_Concat) structure, and the output end of the branch composed of a single convolutional normalized activation module (Conv2D_BN_SiLU), and the output end of the first convolutional normalized activation module (Conv2D_BN_SiLU), the output end of the third convolutional normalized activation module (Conv2D_BN_SiLU), and the output end of the fifth convolutional normalized activation module (Conv2D_BN_SiLU) in the other branch are connected to the stack and the convolutional normalized activation module. The input end of the block (Concat+Conv2D_BN_SiLU) and the output end of the stacking and convolution normalization activation module (Concat+Conv2D_BN_SiLU) constitute the output end of the multi-dimensional stacking module (Multi_Concat) structure. The stacking and convolution normalization activation module (Concat+Conv2D_BN_SiLU) connects the Concat module and the convolution normalization activation module (Conv2D_BN_SiLU) in series from input to output. The stacking and convolution normalization activation module (Concat+Conv2D_BN_SiLU) first applies the stacking module (Concat) to connect each received feature map, and then inputs it into a convolution normalization activation module (Conv2D_BN_SiLU) to perform convolution, normalization, and activation operations in sequence to alleviate the gradient disappearance caused by the increase in the network.
[0068] The fast fully connected spatial pyramid pooling module (SPPF_t-CSPC) in the backbone network Backbone is used in practical applications, such as Figure 6As shown in the figure, in the structure of the specific design of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC), four convolution normalization activation modules (Conv2D_BN_SiLU), a 1×1 convolution kernel global maximum pooling module (MaxPool2d), and nine 3×3 convolution kernel global maximum pooling modules (MaxPool2d) are connected in series from the input to the output direction, among which the input end of the first convolution normalization activation module (Conv2D_BN_SiLU) in sequence constitutes the input end of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC), and the output end of the fourth convolution normalization activation module (Conv2D_BN_SiLU) in sequence , the output of the 1×1 convolution kernel global maximum pooling module (MaxPool2d), the output of the first sequential 3×3 convolution kernel global maximum pooling module (MaxPool2d), the output of the third sequential 3×3 convolution kernel global maximum pooling module (MaxPool2d), the output of the fifth sequential 3×3 convolution kernel global maximum pooling module (MaxPool2d), the output of the seventh sequential 3×3 convolution kernel global maximum pooling module (MaxPool2d), and the output of the ninth sequential 3×3 convolution kernel global maximum pooling module (MaxPool2d) are connected to the input of the stacking module (Concat), and the stacking module (Concat) performs a stacking operation on the connected Collect each feature map for connection. The output of the stacking module (Concat) is connected in series with three convolutional normalization activation modules (Conv2D_BN_SiLU) in sequence. The input of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC) is also connected to the input of the eighth convolutional normalization activation module (Conv2D_BN_SiLU). The output of the eighth convolutional normalization activation module (Conv2D_BN_SiLU) and the output of the third convolutional normalization activation module (Conv2D_BN_SiLU) connected in series with the stacking module (Concat) are connected to the stacking and convolutional normalization activation module (Concat+Conv2D_B The input end of the stacking and convolution normalization activation module (Concat+Conv2D_BN_SiLU) and the output end of the stacking and convolution normalization activation module (Concat+Conv2D_BN_SiLU) constitute the output end of the fast fully connected spatial pyramid pooling module (SPPF_t-CSPC). The stacking and convolution normalization activation module (Concat+Conv2D_BN_SiLU) first applies the stacking module (Concat) to connect the two received feature maps, and then inputs them into a convolution normalization activation module (Conv2D_BN_SiLU) to perform convolution, normalization, and activation operations in sequence. This design refines each sub-region, obtains more detailed features and refined sub-regions, and realizes the capture of more small ship target features.
[0069] Step C. Based on each sample SAR image, according to the prior frames corresponding to the large, medium and small scale ship objects, the sample SAR image is used as input, and the positions of the large, medium and small scale ship objects in the sample SAR image are used as output. The network to be trained is trained to obtain a ship detection model.
[0070] In the above training process for the network to be trained, relevant parameters are set, the ship data set is divided into a training data set and a test data set, with the number of data being 928 and 232 respectively, and the training data set is passed into the network to be trained, the network to be trained is trained until the network converges, and the network training weights are saved, as follows.
[0071] (1) Set the starting learning rate, maximum learning rate, number of training batches, and number of training rounds.
[0072] (2) During the training process, data augmentation (rotation, translation, scaling, flipping) and L2 regularization are used to prevent the network from overfitting.
[0073] (3) The Adam optimizer with better convergence effect and stronger optimization ability is used instead of the SGD optimizer. CIoU is used as the loss function in the optimization process. The calculation formula is as follows:
[0074]
[0075]
[0076]
[0077] In the formula, is the intersection-over-union ratio between the predicted box and the real box of the ship object in the SAR image, Represents the diagonal distance of the minimum closure area of the predicted box and the real box that contain the ship object at the same time, , Represent the center points of the predicted box and the real box of the ship object respectively, and Represents the Euclidean distance between the center point of the predicted box of the ship object and the center point of the real box, represents the weight function, Used to measure the similarity of aspect ratios, and are the width and height of the real frame of the ship object, and are the width and height of the prediction box of the ship object respectively.
[0078] (4) Repeat steps (2) and (3) until the network converges. The final model and weights for the ship detection network are obtained and saved, and the ship detection model is obtained.
[0079] Based on the ship detection model obtained by the above design, in the application, for the SAR image to be analyzed, the ship detection model is applied to identify the position of each ship object in the SAR image to be analyzed.
[0080] In the actual application of the ship detection model, the design evaluates the application results of the ship detection model. Fraction, The value is used to evaluate the accuracy of ship target detection. Evaluate the speed of inference. A higher score means that the model performs better in the detection task, can more accurately predict positive examples, and can better avoid misjudgments. The higher the value, the higher the accuracy and recall rate, that is, the lower the false alarm rate while ensuring that as few targets as possible are missed. The calculation formula is as follows:
[0081]
[0082]
[0083]
[0084]
[0085] In the formula, It indicates positive samples, i.e. the number of ships detected; represents the false positive samples, that is, the number of misjudged ships; represents the false samples, i.e. the number of ships missed; represents the precision rate, that is, the proportion of samples predicted to be positive; Represents the recall rate, that is, the proportion of all positive examples that are correctly detected; represents the number of categories; Indicates The precision of each category; Represents recall rate.
[0086] The above design is applied in practice, and the application of the original YOLOV7 network is analyzed. The above training is performed to obtain each trained model, and the models are applied in actual detection. The detection results of MAP, F1, recall rate and precision rate of the original YOLOV7 network and the present invention are shown in Table 1. It can be concluded from Table 1 that compared with the original YOLOV7 network, the MAP value of the present invention is increased by 1.55%, the F1 score is increased by 0.01, the recall rate Recall is increased by 1.16%, and the precision rate Precision is increased by 1.39%.
[0087] Table 1
[0088]
[0089] The original YOLOV7 detection results are as follows Figure 8 As shown, the detection results of the trained model designed by the present invention are as follows Fig. 9 shown; through Figure 8 , Fig. 9 It can be seen from the detection results of large, medium and small ship targets on SAR images that the present invention has better detection effect. Experimental results show that the detection effect of the present invention on SAR data sets for multi-scale and complex backgrounds is more outstanding, and MAP, F1, Recall and Precision are all improved. Compared with algorithms such as Faster R-CNN, SSD, and YOLOV5, all detection indicators show better performance. It is a ship target detection method with fast convergence speed, strong real-time performance, strong robustness and high accuracy.
[0090] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.
Claims
1. A SAR image ship target detection method based on an improved YOLOV7 network, characterized by: Perform the following steps A to C to obtain a ship detection model, and then apply the ship detection model to the SAR image to be analyzed to identify the position of each ship object in the SAR image to be analyzed: Step A. Based on a preset number of sample SAR images containing ship objects and with known positions of the ship objects, construct a priori boxes corresponding to ship objects of large, medium and small classification scales in a clustering manner, and then proceed to step B; Step B. Construct a network to be trained including a backbone network Backbone, a polymerization network PANet, and a head network Head, and used to output large, medium, and small three-scale ship object detection, wherein the input end of the backbone network Backbone constitutes the input end of the network to be trained, and each output end of the backbone network Backbone is connected to each input end of the polymerization network PANet through a first irrelevant information elimination module and a second irrelevant information elimination module. The three output ends of the polymerization network PANet are respectively connected to the input ends of the sub-head networks of the head network Head for large, medium, and small three-scale ship objects, and the output ends of each sub-head network in the head network Head constitute the output ends of the network to be trained, which are respectively used to output large, medium, and small three-scale ship object detection, and then enter step C; Step C. Based on each sample SAR image, according to the prior frames corresponding to the large, medium and small scale ship objects, the sample SAR image is used as input, and the positions of the large, medium and small scale ship objects in the sample SAR image are used as output, and the network to be trained is trained to obtain a ship detection model; The backbone network Backbone includes, from its input to its output direction, a first convolution normalized activation module, a second convolution normalized activation module, a third convolution normalized activation module, a first pyramid pooling attention module, a first multiplication module, a fourth convolution normalized activation module, a first multidimensional stacking module, a second pyramid pooling attention module, a second multiplication module, a first transition module, a second multidimensional stacking module, a third pyramid pooling attention module, a third multiplication module, a second transition module, a third multidimensional stacking module, a fourth pyramid pooling attention module, a fourth multiplication module, a third transition module, a fourth multidimensional stacking module, a fifth pyramid pooling attention module, a fifth multiplication module, and a fast fully connected spatial pyramid pooling module; Among them, the output end of the third convolution normalization activation module is simultaneously connected to the input end of the first multiplication module. For the feature map output by the third convolution normalization activation module, the feature map is updated by multiplying the feature map with the weight output by the first pyramid pooling attention module about the feature map, and then the feature map is transmitted to the input end of the fourth convolution normalization activation module; the output end of the first multidimensional stacking module is simultaneously connected to the input end of the second multiplication module. For the feature map output by the first multidimensional stacking module, the feature map is updated by multiplying the feature map with the weight output by the second pyramid pooling attention module about the feature map, and then the feature map is transmitted to the input end of the first transition module; the output end of the second multidimensional stacking module is simultaneously connected to the input end of the third multiplication module. For the feature map output by the second multidimensional stacking module, the feature map is updated by multiplying the feature map with the weight output by the second pyramid pooling attention module about the feature map, and then the feature map is transmitted to the input end of the first transition module. The feature map is multiplied by the weight of the feature map output by the third pyramid pooling attention module to update the feature map, and then the feature map is transmitted to the input end of the second transition module; the output end of the third multidimensional stacking module is simultaneously connected to the input end of the fourth multiplication module, and for the feature map output by the third multidimensional stacking module, the feature map is updated by multiplying the weight of the feature map output by the fourth pyramid pooling attention module, and then the feature map is transmitted to the input end of the third transition module; the output end of the fourth multidimensional stacking module is simultaneously connected to the input end of the fifth multiplication module, and for the feature map output by the fourth multidimensional stacking module, the feature map is updated by multiplying the weight of the feature map output by the fifth pyramid pooling attention module, and then the feature map is transmitted to the input end of the fast fully connected spatial pyramid pooling module; The input end of the first convolution normalization activation module constitutes the input end of the backbone network Backbone, and the output end of the third multiplication module, the output end of the fourth multiplication module, the output end of the fifth multiplication module, and the output end of the fast fully connected spatial pyramid pooling module constitute the output ends of the backbone network Backbone; The output end of the third multiplication module and the output end of the fourth multiplication module in the backbone network Backbone are respectively connected to the two input ends of the second irrelevant information elimination module; the output end of the fourth multiplication module and the output end of the fifth multiplication module in the backbone network Backbone are respectively connected to the two input ends of the first irrelevant information elimination module; the output end of the second irrelevant information elimination module, the output end of the first irrelevant information elimination module, and the output end of the fast fully connected spatial pyramid pooling module are respectively connected to the input ends of the aggregation network PANet; In the structure of the aggregation network PANet, the input end of the first convolution module constitutes the first input end of the aggregation network PANet connected to the output end of the fast fully connected spatial pyramid pooling module in the backbone network Backbone, and the output end of the first convolution module is connected in series with the first upsampling module, the first stacking and multi-dimensional stacking module, the third convolution module, the second upsampling module, the second stacking and multi-dimensional stacking module, the first downsampling module, the fourth transition module, the third stacking and multi-dimensional stacking module, the second downsampling module, the fifth transition module, the fourth stacking and multi-dimensional stacking module; The input ends of the fourth stacking and multi-dimensional stacking modules simultaneously constitute the second input end of the aggregation network PANet and are connected to the output end of the fast fully connected spatial pyramid pooling module in the backbone network Backbone, and the output ends of the fourth stacking and multi-dimensional stacking modules constitute the first output end of the aggregation network PANet; The input end of the second convolution module constitutes the third input end of the aggregation network PANet and is connected to the output end of the first irrelevant information elimination module. The input ends of the first stacking and multi-dimensional stacking modules are simultaneously connected to the output end of the second convolution module. The output ends of the first stacking and multi-dimensional stacking modules are simultaneously connected to the input ends of the third stacking and multi-dimensional stacking modules. The output ends of the third stacking and multi-dimensional stacking modules constitute the second output end of the aggregation network PANet. The input end of the fourth convolution module constitutes the fourth input end of the aggregation network PANet and is connected to the output end of the second irrelevant information elimination module. The input ends of the second stacking and multi-dimensional stacking modules are simultaneously connected to the output ends of the fourth convolution module. The output ends of the second stacking and multi-dimensional stacking modules constitute the third output end of the aggregation network PANet. The first stacking and multi-dimensional stacking module, the second stacking and multi-dimensional stacking module, the third stacking and multi-dimensional stacking module, and the fourth stacking and multi-dimensional stacking module are respectively stacking modules and multi-dimensional stacking modules connected in series from input to output.
2. According to claim 1, a SAR image ship target detection method based on an improved YOLOV7 network is characterized in that: The structures of the first transition module, the second transition module, the third transition module, the fourth transition module and the fifth transition module are the same. In each transition module structure, one branch is formed by a 2×2 convolution kernel global maximum pooling module and a convolution normalization activation module connected in series from the input to the output direction, and the other branch is formed by two convolution normalization activation modules connected in series from the input to the output direction. The input ends of the two branches are connected to form the input end of the transition module structure, and the output ends of the two branches are connected to the input end of the stacking module. The output end of the stacking module constitutes the output end of the transition module structure, and the stacking module connects the outputs of the two branches received.
3. According to claim 1, a SAR image ship target detection method based on an improved YOLOV7 network is characterized in that: The sub-head network for large-scale ship objects in the head network Head includes a first parameterized convolution module and a first detection head module connected in series, the input end of the first parameterized convolution module constitutes the first input end of the head network Head connected to the third output end of the aggregation network PANet, and the output end of the first detection head module constitutes the first output end of the head network Head; the sub-head network for medium-scale ship objects includes a second parameterized convolution module and a second detection head module connected in series, the input end of the second parameterized convolution module constitutes the second input end of the head network Head connected to the second output end of the aggregation network PANet, and the output end of the second detection head module constitutes the second output end of the head network Head; the sub-head network for small-scale ship objects includes a third parameterized convolution module and a third detection head module connected in series, the input end of the third parameterized convolution module constitutes the third input end of the head network Head connected to the first output end of the aggregation network PANet, and the output end of the third detection head module constitutes the third output end of the head network Head.
4. The method for detecting ship targets in SAR images based on an improved YOLOV7 network according to claim 2 or 3, characterized in that: The structures of the first pyramid pooling attention module, the second pyramid pooling attention module, the third pyramid pooling attention module, the fourth pyramid pooling attention module and the fifth pyramid pooling attention module are identical to each other. In the structure of each pyramid pooling attention module, the input end of the 5×5 convolution kernel global maximum pooling module, the input end of the 9×9 convolution kernel global maximum pooling module and the input end of a global average pooling module are connected to form the input end of the pyramid pooling attention module structure for receiving the feature map. The output end of the 5×5 convolution kernel global maximum pooling module and the output end of the 9×9 convolution kernel global maximum pooling module are respectively connected to the input ends of the other two global average pooling modules in a one-to-one correspondence. The average pooling modules reduce the dimensions of the feature maps they receive respectively. The output ends of the three global average pooling modules are connected to the input ends of the three 1×1 convolution kernels respectively, and the weights of the feature maps they receive are obtained by each 1×1 convolution kernel. The output ends of the three 1×1 convolution kernels are connected to the input end of the superposition activation module. The output end of the superposition activation module constitutes the output end of the pyramid pooling attention module structure. The superposition activation module is a superposition module and a Sigmod function connected in series from the input to the output direction. The superposition activation module first applies the superposition module to perform weight superposition operation on the weights of the three 1×1 convolution kernel outputs, and then obtains the weight of the feature map received at the input end of the pyramid pooling attention module structure through the Sigmod function.
5. The method for detecting ship targets in SAR images based on an improved YOLOV7 network according to claim 2 or 3, characterized in that: The structures of the first multi-dimensional stacking module, the second multi-dimensional stacking module, the third multi-dimensional stacking module, and the fourth multi-dimensional stacking module are the same. In each multi-dimensional stacking module structure, one branch is composed of a convolutional normalization activation module, and another branch is composed of five convolutional normalization activation modules connected in series from input to output. The input ends of the two branches are connected to form the input end of the multi-dimensional stacking module structure, and the output end of the branch composed of a single convolutional normalization activation module, and the output end of the first convolutional normalization activation module in the other branch, the output end of the third convolutional normalization activation module in sequence, and the output end of the branch composed of the first convolutional normalization activation module in sequence, and the output end of the third convolutional normalization activation module in sequence are connected to form the input end of the multi-dimensional stacking module structure. The output end of the stacking and convolution normalization activation module and the output end of the fifth convolution normalization activation module are connected to the input end of the stacking and convolution normalization activation module. The output end of the stacking and convolution normalization activation module constitutes the output end of the multi-dimensional stacking module structure. The stacking modules and the convolution normalization activation modules are sequentially connected in series from input to output. The stacking and convolution normalization activation modules first apply the stacking modules to connect each received feature map, and then input it into a convolution normalization activation module to perform convolution, normalization, and activation operations in sequence.
6. The method for detecting ship targets in SAR images based on an improved YOLOV7 network according to claim 2 or 3, characterized in that: In the structure of the fast fully connected spatial pyramid pooling module in the backbone network Backbone, four convolution normalization activation modules, a 1×1 convolution kernel global maximum pooling module, and nine 3×3 convolution kernel global maximum pooling modules are sequentially connected in series from input to output, wherein the input end of the first convolution normalization activation module in sequence constitutes the input end of the fast fully connected spatial pyramid pooling module, the output end of the fourth convolution normalization activation module in sequence, the output end of the 1×1 convolution kernel global maximum pooling module, the output end of the first sequential 3×3 convolution kernel global maximum pooling module, the output end of the third sequential 3×3 convolution kernel global maximum pooling module, the output end of the fifth sequential 3×3 convolution kernel global maximum pooling module, the output end of the seventh sequential 3×3 convolution kernel global maximum pooling module, and the output end of the ninth sequential 3×3 convolution kernel global maximum pooling module constitute the input end of the fast fully connected spatial pyramid pooling module. The output end of the 3-convolution kernel global maximum pooling module is connected to the input end of the stacking module, and the stacking module is used to connect each received feature map. The output end of the stacking module is sequentially connected to three convolution normalization activation modules in series, and the input end of the fast fully connected spatial pyramid pooling module is simultaneously connected to the input end of another eighth convolution normalization activation module. The output end of the eighth convolution normalization activation module and the output end of the third convolution normalization activation module connected in series with the stacking module are connected to the input end of the stacking and convolution normalization activation module. The output end of the stacking and convolution normalization activation module constitutes the output end of the fast fully connected spatial pyramid pooling module. The stacking and convolution normalization activation module first applies the stacking module to connect the two received feature maps, and then inputs them into a convolution normalization activation module to perform convolution, normalization, and activation operations in sequence.
7. The method for detecting ship targets in SAR images based on an improved YOLOV7 network according to any one of claims 1 to 3, characterized in that: The structure of the first irrelevant information elimination module is the same as that of the second irrelevant information elimination module. Each irrelevant information elimination module includes a first branch, a second branch, and a subtraction module. In the structure of each irrelevant information elimination module, the first branch is composed of a dot product module and an addition module connected in series in sequence, the input end of the dot product module constitutes the input end of the first branch, the output end of the addition module constitutes the output end of the first branch, and the input end of the addition module is simultaneously connected to the input end of the dot product module; the second branch is composed of an upsampling module, a sub-parallel structure, a dot product module, and a sigmiod function connected in series in sequence from input to output, the sub-parallel structure includes a global average pooling module and a global maximum pooling module connected in parallel to each other, and the input end of the upsampling module constitutes the input end of the second branch. , the output end of the Sigmiod function constitutes the output end of the second branch; the two input ends of the subtraction module are respectively connected to the output end of the second branch and the feature map whose eigenvalues of each channel are 1, and the input end of the point multiplication module in the first branch is also connected to the output end of the subtraction module; the input end of the first branch constitutes one of the input ends of the irrelevant information elimination module, which is used to access the output end of the first multiplication module in the sequence of two multiplication modules in the backbone network Backbone connected to the irrelevant information elimination module, the input end of the second branch constitutes the other input end of the irrelevant information elimination module, which is used to access the output end of the second multiplication module in the sequence of two multiplication modules in the backbone network Backbone connected to the irrelevant information elimination module, and the output end of the first branch constitutes the output end of the irrelevant information elimination module.
8. The method for detecting ship targets in SAR images based on an improved YOLOV7 network according to any one of claims 1 to 3, characterized in that: In the step C, CIoU is used as the loss function during the training process of the network to be trained, and the calculation formula is as follows: Where IoU is the intersection over union (IoU) between the predicted box and the true box of the ship object in the SAR image, c represents the diagonal distance of the minimum closure area that contains both the predicted box and the true box of the ship object, b and b' represent the center points of the predicted box and the true box of the ship object, respectively, and ρ represents the Euclidean distance between the center points of the predicted box and the true box of the ship object, α represents the weight function, v is used to measure the similarity of the aspect ratio, ω' and h' are the width and height of the true box of the ship object, respectively, and w and h are the width and height of the predicted box of the ship object, respectively.
Citation Information
Patent Citations
Remote sensing image multi-scale ship target detection method based on improved YOLOV5 network
CN117218550A
Pedestrian detection method based on YOLOv7
CN117542082A
Multi-scale remote sensing image landslide target detection method and system
CN118015456A
SAR ship target detection method and system based on context multiple features
CN118172569A
Contextual visual-based SAR target detection method and apparatus, and storage medium
US20230184927A1
