High-resolution visible light ship detection method and system guided by saliency information

By using augmented training set and feature fusion technology in high-resolution visible ship images, the problem of low detection accuracy is solved and higher detection accuracy and generalization capabilities are achieved.

CN117876884BActive Publication Date: 2024-11-05INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410034858.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-09
Publication Date
2024-11-05
Estimated Expiration
2044-01-09

AI Technical Summary

Technical Problem

The prior art has low detection accuracy in high-resolution visible ship images, especially the difficulty in target positioning caused by complex backgrounds and diversified ship scales.

Method used

Adopting a ship detection network based on the training set, including encoder module, semantic module, transition module, decoder module and predictor module, through the dual matching mechanism and cross-level cross-channel feature fusion unit, significant information is introduced to improve Detection accuracy.

Benefits of technology

It significantly improves the detection accuracy of high-resolution visible ship images, can position and classify ships more accurately, and enhances the generalization ability of the network.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117876884B_ABST
    Figure CN117876884B_ABST
Patent Text Reader

Abstract

The present invention provides a high-resolution visible light ship detection method and system guided by saliency information, which relates to the field of remote sensing image recognition, and includes: obtaining a trained ship detection network; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-built, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module, and a predictor module; the training set is obtained by augmenting a high-resolution visible light ship image; obtaining a high-resolution visible light ship image to be detected; inputting the high-resolution visible light ship image to be detected into the trained ship detection network, and obtaining the position information of the target ship, the saliency mask of the target ship, and the classification confidence of the target ship. The present invention can provide more accurate detection results for high-resolution visible light ship images by using the trained ship detection network.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image recognition, and in particular to a high-resolution visible light ship detection method and system guided by saliency information. Background Art

[0002] With the improvement of satellite imaging technology, the spatial resolution of visible light remote sensing images has been increasing. Using high-resolution visible light images for ship detection is a key focus, and it has played a pivotal role in environmental protection, resource management, and military surveys. However, in high-resolution visible light ship images, the complex background, the diversity of ship scales, and the high differentiation of ship categories bring more challenges to the acquisition of ship representation and the positioning of ship targets. In order to solve the above difficulties, methods based on ship detection can be divided into two categories: traditional methods and methods based on deep neural networks. Traditional methods usually divide ship detection into two steps: acquisition of candidate regions and positioning of targets. In the first step, features such as texture, saliency information, or boundaries are generally used to segment candidate regions. After that, for accurate positioning, scale-invariant features, Gabor filters, or gradient histograms are used to distinguish ships. However, the above methods have the characteristics of low robustness and low generalization, and the detection performance tends to be saturated. It is difficult to achieve higher breakthroughs in the short term, and the detection accuracy is low. In recent years, with the rapid development of deep neural network technology, target detection technology based on deep networks has received more attention. For example, the R-CNN (Region-based Convolutional Neural Network) series of methods, YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector) methods. Since these methods are specially designed for images of natural scenes, they are not effective when applied to high-resolution visible light ship images, resulting in low detection accuracy. Summary of the invention

[0003] The present invention provides a high-resolution visible light ship detection method and system guided by saliency information, so as to solve the defect of low detection accuracy when the existing image detection algorithm is applied to high-resolution visible light ship images, and improve the detection accuracy of high-resolution visible light ship images.

[0004] The present invention provides a high-resolution visible light ship detection method guided by saliency information, comprising:

[0005] Acquire a trained ship detection network; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-constructed, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module, and a predictor module; the training set is obtained by augmenting a high-resolution visible light ship image;

[0006] Acquire high-resolution visible light images of ships to be detected;

[0007] The high-resolution visible light ship image to be detected is input into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship and the classification confidence of the target ship.

[0008] Optionally, the training set is obtained in the following manner:

[0009] Acquire high-resolution visible light ship images;

[0010] The high-resolution visible light ship image and the positioning labels in the image are randomly flipped, randomly cropped and fixed in size;

[0011] The saliency mask labels of the high-resolution visible light ship image are randomly flipped, randomly cropped and fixed in size.

[0012] Optionally, the backbone network of the encoder module is ResNet50, and the encoder module includes a multi-layer first network structure constructed according to the backbone network, the multi-layer first network structure adopts a feature pyramid structure, and the feature dimensions of outputs of different layers in the multi-layer first network structure are different.

[0013] Optionally, the semantic module includes a ReLU activation layer and a first convolutional layer.

[0014] Optionally, the transition module includes a multi-layer second network structure, the multi-layer second network structure adopts a top-down fusion method, and the feature dimensions of outputs of different layers in the multi-layer second network structure are the same.

[0015] Optionally, the decoder module includes a multi-layer third network structure, and the decoder module adopts a double matching mechanism to introduce saliency information, the double matching mechanism includes a prediction branch of structure matching and a saliency label supervision branch of size matching, and the sizes of saliency labels of different layers in the multi-layer third network structure are different; the number of layers of the third network structure is the same as the number of layers of the network structure of the encoder module.

[0016] Optionally, the predictor module includes a classification branch prediction structure and a regression branch prediction structure, the classification branch prediction structure includes multiple second convolutional layers and a third convolutional layer with an output channel of A; the regression branch prediction structure includes multiple fourth convolutional layers and a fifth convolutional layer with an output channel of 4A, where A represents the number of anchor boxes, and the number of the second convolutional layers is the same as the number of the fourth convolutional layers.

[0017] Optionally, the decoder module also includes a cross-level and cross-channel feature fusion unit, which includes a first average pooling layer, a second average pooling layer and a sixth convolutional layer. The first average pooling layer is used to extract the first channel-level features of the encoder module, the second average pooling layer is used to extract the second channel-level features of the decoder module, and the sixth convolutional layer is used to fuse the first channel-level features and the second channel-level features to achieve cross-channel information interaction.

[0018] The present invention also provides a high-resolution visible light ship detection system guided by saliency information, comprising:

[0019] a ship detection network acquisition module, used to acquire a trained ship detection network; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-constructed, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module, and a predictor module; the training set is obtained by augmenting a high-resolution visible light ship image;

[0020] An image acquisition module to be detected, used to acquire a high-resolution visible light ship image to be detected;

[0021] The image detection module is used to input the high-resolution visible light ship image to be detected into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship and the classification confidence of the target ship.

[0022] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the high-resolution visible light ship detection method guided by the significance information is implemented.

[0023] The present invention provides a saliency information-guided high-resolution visible light ship detection method and system, which obtains a trained ship detection network; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-constructed, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module, and a predictor module; the training set is obtained by augmenting a high-resolution visible light ship image; a high-resolution visible light ship image to be detected is obtained; the high-resolution visible light ship image to be detected is input into the trained ship detection network, and the position information of the target ship, the saliency mask of the target ship, and the classification confidence of the target ship are obtained. The present invention can provide more accurate detection results for high-resolution visible light ship images through the trained ship detection network. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a flow chart of the high-resolution visible light ship detection method guided by significant information provided by the present invention;

[0026] Figure 2 It is a schematic diagram of the ship detection network structure provided by the present invention;

[0027] Figure 3 It is a schematic diagram of the principle of the double matching mechanism provided by the present invention;

[0028] Figure 4 It is a schematic diagram of the principle of the cross-level and cross-channel feature fusion unit provided by the present invention;

[0029] Figure 5 is a schematic diagram of high-resolution visible light ship detection results provided by the present invention;

[0030] Figure 6 is a schematic diagram of the significant information detection result provided by the present invention;

[0031] Figure 7 This is a module diagram of a high-resolution visible light ship detection system guided by significant information provided by the present invention;

[0032] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Combine the following Figure 1-Figure 8 The present invention describes a high-resolution visible light ship detection method and system guided by saliency information.

[0035] Figure 1 is a flow chart of the high-resolution visible light ship detection method guided by saliency information provided by the present invention, such as Figure 1 As shown, a high-resolution visible light ship detection method guided by saliency information includes:

[0036] Step 101: Obtain a trained ship detection network; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-constructed, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module and a predictor module; the training set is obtained by augmenting high-resolution visible light ship images.

[0037] In a specific embodiment, the training set is obtained by:

[0038] Acquire high-resolution visible light ship images.

[0039] The high-resolution visible light ship image and the positioning labels in the image are randomly flipped, randomly cropped and fixed in size.

[0040] Specifically, random cropping is to randomly make the random area of ​​any area equal to 0. In addition, fixed size means resizing the image or the positioning label of the image to 600×600×3. It should be noted that the corresponding transformation of the image or the positioning label of the image is aimed at adjusting the space and scale of the target after the image transformation.

[0041] The salient mask labels of the high-resolution visible light ship image are randomly flipped, randomly cropped and fixed in size

[0042] Specifically, the saliency mask labels of the high-resolution visible light ship image are fixed in size in order to resize them to 600×600×1.

[0043] By augmenting high-resolution visible light ship images, the present invention can expand the number and form of training data sets and improve the generalization ability of the training model.

[0044] In a specific embodiment, the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module and a predictor module. The specific structure of the ship detection network is as follows: Figure 2 shown.

[0045] In a specific embodiment, the backbone network of the encoder module is ResNet50, and the encoder module includes a multi-layer first network structure constructed according to the backbone network, wherein the multi-layer first network structure adopts a feature pyramid structure, and the feature dimensions of different layers in the multi-layer first network structure are different. In the present invention, the encoder module is mainly used to encode image detail features.

[0046] Specifically, the encoder module is used to extract the first level (initial or low level) feature pyramid. The encoder module uses ResNet50 as the backbone network, builds a four-layer network structure, and fuses the features in a bottom-up manner to form pyramid features. The first to third layers are denoted as {E 1 ,E 2 ,E 3}, corresponding to the three building blocks at the top of the ResNet50 network. The output feature stream of the third layer is connected to a 3×3 convolutional layer as the fourth layer E of the encoder. 4 The feature dimensions of the output of the four-layer first network structure are 256, 512, 1024 and 2048 from low to high.

[0047] In a specific embodiment, the present invention constructs a semantic module based on the encoder module, and the semantic module includes a ReLU activation layer and a first convolutional layer for extracting semantic representations of primary features.

[0048] Specifically, the semantic module is used to extract the semantic features of the first-level features by replacing the top layer E 4 The features are input to the semantic module and encoded into features containing semantic information. The semantic module includes a ReLU activation layer and a 3×3 convolution (the first convolution layer). The input features of the semantic module are the output features of the fourth layer of the encoder module.

[0049] In a specific embodiment, a transition module is constructed based on the encoder module. The transition module includes a multi-layer second network structure, the multi-layer second network structure adopts a top-down fusion method, and the feature dimensions of different layers in the multi-layer second network structure are the same.

[0050] Specifically, the transition module includes a three-layer second network structure, denoted as {M 1 ,M 2 ,M 3}, corresponding to the first, second and third layers of the encoder module, namely E 1 ~E 3 , the transition module uses the same size of convolution kernel in the same layer of the corresponding encoder module. The transition module itself adopts a top-down fusion mode. Each layer of the structure is composed of a 3×3 convolution layer, and the interpolation algorithm is used between layers to upsample the upper layer features. When connecting the same layer of encoder modules, a 3×3 convolution layer is used for feature dimensionality reduction, so that the number of feature channels of each layer of the transition layer module remains consistent, which is 256 dimensions.

[0051] In a specific embodiment, a decoder module is constructed based on a transition module, a semantic module and an encoder module. The decoder module includes a multi-layer third network structure, and the number of layers of the decoder module is consistent with that of the encoder module. The decoder module adopts a double matching mechanism to introduce saliency information, and the double matching mechanism includes a prediction branch of structural matching and a saliency label supervision branch of size matching, and the sizes of saliency labels of different layers in the multi-layer third network structure are different.

[0052] Specifically, the decoder module structure also adopts a feature pyramid structure, which includes a total of four layers of the third network structure, denoted as {D 1 ,D 2 ,D 3 ,D 4 Each layer contains a 3×3 convolution, and uses a top-down approach to fuse features of different scales. Interpolation algorithms are used between layers to upsample the upper layer features. For the highest layer network (i.e., the fourth layer D 4 ) is connected to the output of the semantic module to provide semantic guidance for the features of the lower layers, that is, the output features of the semantic module and the highest layer E of the encoder module 4 The output features of are used as input features for fusion. The input features of the remaining three layers are two, namely the output features of the adjacent higher layer network at the same level and the output features of the same layer network corresponding to the transition module M. The decoder module mainly obtains saliency-related features by introducing more saliency information to enhance the discrimination ability of network features.

[0053] Furthermore, the input of each layer of the decoder module consists of two parts: one is the more semantic features generated by the high-level module, and the other is the more detailed features generated by the corresponding layer of the previous module. 4 Fusion Encoder Module E 4 , the residual decoder module {D 1 ,D2 ,D 3} respectively correspond to the fusion transition module {M 1 ,M 2 ,M 3}.

[0054] In the present invention, a dual matching mechanism is used to map the fused features of each layer of the network in the decoder module into a saliency prediction map, and a saliency mask label is used for supervision during the training process. Due to the different resolutions of each layer of the feature pyramid, the dual matching mechanism uses a structurally matched prediction mapping structure and a size-matched saliency label mechanism to introduce more saliency information into the entire model and more efficiently enhance the supervision of saliency information.

[0055] It should be noted that the dual matching mechanism includes two types of matching, namely, the structural matching mechanism and the scale matching mechanism. For the structural matching mechanism, the number of stacked deconvolutions from high to low layers in the present invention is 0, 1, 2, and 3, respectively. Secondly, the present invention adopts a supervision form of size matching, that is, the sizes of saliency labels in different layers are different, from high to low, respectively, 75×75, 150×150, 300×300, and 600×600.

[0056] Further, Figure 3 The proposed dual matching mechanism structure is shown. The encoder adopts a feature pyramid structure, so the feature resolution of different layers is different. The resolution of high-level features is lower than that of low-level features. Therefore, we use the feature pyramid structure to obtain the predicted saliency map SP. i In the embodiment, the prediction branch of structural matching is adopted. In the embodiment, deconvolution represents a 4×4 deconvolution operation, which is used to increase the resolution of the feature while decreasing the feature channel; in the embodiment, convolution represents a 3×3 convolution kernel, which is used to ensure more efficient information fusion. During the training process, the main function of the high-level mapping structure is to introduce more saliency information to the network, so the mapping structure level is relatively simple. The low-level features are rich in semantic information due to the top-down fusion structure, and the details-rich features in the previous level are connected through the side branches. Therefore, the low-level features can be used to map to produce more accurate saliency maps. Therefore, the low-level mapping structure adopts a larger number of deconvolution layers to achieve the protection of high-resolution information. In addition, the saliency supervision labels in the training process select size-matched saliency labels to achieve multi-layer supervision. This processing method will not bring pressure to the high-level mapping structure, so that more saliency information can be introduced and optimized.

[0057] Furthermore, during the network training process, since the decoder module adopts a four-layer feature pyramid structure, the feature resolution of each layer is different: the higher the level, the lower the resolution. In order to introduce salient information with discriminative characteristics to the entire ship detection network, the present invention chooses an end-to-end training method. Therefore, each layer will use the salient label of the input image for supervision, and different layers select salient labels of different sizes for supervision and use a binary cross entropy function to calculate the loss. The formula is as follows:

[0058]

[0059] Among them, N i refers to the number of pixels in the saliency label of the i-th layer, and They represent the predicted saliency value and the true saliency value of the i-th layer at pixel position n, respectively.

[0060] In the mapping process, it is necessary to map the multi-dimensional small-size feature map into a single-channel large-size mapping map. In order to reduce the information loss in the mapping process, the decoder module of the present invention selects a feature decreasing structure, that is, deconvolution is selected to increase the size of the feature while the feature dimension decreases. Different numbers of deconvolutions are stacked at different layers of the decoder module to achieve the effect of structural matching. Specifically, the feature resolution of the high-level is low and is not suitable for generating high-resolution prediction maps. Therefore, the prediction structure selects fewer deconvolution layers to achieve mapping. Relatively speaking, the feature resolution of the low-level is higher, and because it is a top-down feature fusion structure, it contains semantic information. The low-level features are connected by side branches, thereby containing detailed information. Therefore, the low-level features are more suitable for generating fine saliency prediction maps, so the number of deconvolutions in the corresponding mapping structure is more.

[0061] It is worth mentioning that in the training phase, the present invention uses the binary cross entropy loss function as the training loss function, and adds the losses of each layer as the overall saliency loss function of the decoder module. Due to the different resolutions of the feature pyramid layers, the dual matching mechanism uses a structure-matched prediction mapping structure and a size-matched saliency label mechanism to introduce more saliency information into the entire model and more effectively optimize the supervision information of saliency. For the specific structure, please see Figure 3 .

[0062] In a specific embodiment, the predictor module includes a classification branch prediction structure and a regression branch prediction structure, the classification branch prediction structure includes multiple second convolutional layers and a third convolutional layer with an output channel of A; the regression branch prediction structure includes multiple fourth convolutional layers and a fifth convolutional layer with an output channel of 4A, where A represents the number of anchor boxes, and the number of the second convolutional layers is the same as the number of the fourth convolutional layers.

[0063] Furthermore, before constructing the predictor module in the present invention, it is necessary to perform cross-level and cross-channel feature fusion of the features fused by the decoder module with the output features of the encoder at the same layer. Since the feature channel dimensions of each layer of the encoder module are different, the features of the encoder are reduced in dimension by 3×3 convolution.

[0064] Then, ship classification and bounding box regression are performed based on the cross-level and cross-channel fused features. The predictor module adopts a five-layer pyramid structure, denoted as {P 1 ,P 2 ,P 3 ,P 4 ,P 5}, using top-down connections. The top layer connects the output of the semantic module, and the remaining four layers are provided by cross-level and cross-channel feature fusion modules. Each layer of the predictor module needs to process classification and regression tasks respectively. Therefore, the present invention adopts a two-branch full convolution prediction structure. Specifically, the classification branch adopts four second convolutional layers and a third convolutional layer with an output channel of A; correspondingly, the regression branch adopts four fourth convolutional layers and a fifth convolutional layer with an output channel of 4A, where A represents the number of anchor boxes.

[0065] Further, Figure 4 This is a schematic diagram of the cross-level and cross-channel feature fusion unit structure in the present invention. For the cross-level and cross-channel fusion structure, the unit uses the first mean pooling layer and the second mean pooling layer to extract the first channel-level features and the second channel-level features of the encoder module and the decoder module respectively, and fuses the two features through the 3×3 sixth convolutional layer to achieve cross-channel information interaction. The formula is as follows:

[0066]

[0067] Among them, S c Encodes the information interaction between the two levels of features. Refers to the feature vector after the encoder is reduced in dimension and then averaged and pooled. avg It refers to the feature vector of the decoder after dimensionality reduction and mean pooling, C[·] refers to the serial operation mode, and Conv refers to the 3×3 convolution operation.

[0068] When the information interaction coding S of the two-level features is obtained c Afterwards, the present invention adopts the following method to fuse the encoder and decoder features of each layer to obtain the feature P of the predictor module: i :

[0069]

[0070] Among them, T is the output feature of the semantic module, represents the channel-level multiplication operation, represents the addition of element-level features at corresponding positions, Represents the features after dimensionality reduction, D i represents the output of the decoder module, and i represents the number of network layers.

[0071] In addition, the process of training the ship detection network in the present invention adopts an end-to-end multi-task learning framework: the decoder side module is trained for the saliency detection task, and the predictor side module is trained for the classification task and the positioning task.

[0072] Furthermore, during the training process, the classification branch and regression losses adopt the corresponding losses in “Focal Loss for DenseObject Detection”, while for the saliency loss in the decoder module, the binary cross entropy loss is selected for supervision.

[0073] In a specific embodiment, after the present invention trains the ship detection network through the training set, the trained ship detection network can also be tested using the test set data, and after reaching the preset test accuracy, it can be further applied.

[0074] Specifically, during the test process, non-maximum likelihood suppression (NMS) can be used to remove redundancy from the generated detection box, and the threshold is set to 0.5.

[0075] Step 102: Acquire a high-resolution visible light ship image to be detected.

[0076] Step 103: Input the high-resolution visible light ship image to be detected into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship, and the classification confidence of the target ship.

[0077] Specifically, the present invention outputs the saliency mask of the target ship through the decoder module, and outputs the classification confidence (classification score) and positioning information of the target ship through the predictor module. The position information of the target ship includes two pairs of coordinate values ​​of the upper left and lower right of the detection box.

[0078] Specifically, according to the above solution of the present invention, the high-resolution visible light ship image is detected, and the result is as follows: Figure 5 and Figure 6 shown.

[0079] In summary, the present invention proposes a high-resolution visible light ship detection method guided by saliency information, which improves the positioning accuracy of low-discriminability targets and finds more small-sized ships by efficiently utilizing saliency information.

[0080] The present invention also proposes a saliency-guided ship detection network, which includes a double matching mechanism and a cross-level and cross-channel feature fusion unit; the double matching mechanism can be used to introduce and optimize saliency features, and the cross-level and cross-channel feature fusion unit enhances the feature representation of ships by fusing low-level detail features and high-level saliency-related features.

[0081] The present invention also proposes a multi-task end-to-end training framework for joint ship saliency detection, ship classification and positioning, which can enhance the capabilities of the three tasks and improve the effect of ship detection.

[0082] The high-resolution visible light ship detection system guided by significant information provided by the present invention is described below. The high-resolution visible light ship detection system guided by significant information described below and the high-resolution visible light ship detection method guided by significant information described above can be referenced to each other.

[0083] like Figure 7 As shown, a high-resolution visible light ship detection system guided by saliency information comprises:

[0084] The ship detection network acquisition module 701 is used to obtain a trained ship detection network; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-constructed, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module and a predictor module; the training set is obtained by augmenting high-resolution visible light ship images.

[0085] In a specific embodiment, the training set is obtained by:

[0086] Acquire high-resolution visible light ship images;

[0087] The high-resolution visible light ship image and the positioning labels in the image are randomly flipped, randomly cropped and fixed in size;

[0088] The saliency mask labels of the high-resolution visible light ship image are randomly flipped, randomly cropped and fixed in size.

[0089] In a specific embodiment, the backbone network of the encoder module is ResNet50, and the encoder module includes a multi-layer first network structure constructed according to the backbone network. The multi-layer first network structure adopts a feature pyramid structure, and the feature dimensions of outputs of different layers in the multi-layer first network structure are different.

[0090] In a specific embodiment, the semantic module includes a ReLU activation layer and a first convolutional layer.

[0091] In a specific embodiment, the transition module includes a multi-layer second network structure, the multi-layer second network structure adopts a top-down fusion method, and the feature dimensions of outputs of different layers in the multi-layer second network structure are the same.

[0092] In a specific embodiment, the decoder module includes a multi-layer third network structure, and the decoder module adopts a double matching mechanism to introduce saliency information. The double matching mechanism includes a prediction branch of structure matching and a saliency label supervision branch of size matching. The sizes of saliency labels of different layers in the multi-layer third network structure are different; the number of layers of the third network structure is the same as the number of layers of the first network structure.

[0093] In a specific embodiment, the predictor module includes a classification branch prediction structure and a regression branch prediction structure, the classification branch prediction structure includes multiple second convolutional layers and a third convolutional layer with an output channel of A; the regression branch prediction structure includes multiple fourth convolutional layers and a fifth convolutional layer with an output channel of 4A, where A represents the number of anchor boxes, and the number of the second convolutional layers is the same as the number of the fourth convolutional layers.

[0094] The image acquisition module 702 to be detected is used to acquire a high-resolution visible light ship image to be detected.

[0095] The image detection module 703 is used to input the high-resolution visible light ship image to be detected into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship and the classification confidence of the target ship.

[0096] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the high-resolution visible light ship detection method guided by saliency information, and the method includes:

[0097] A trained ship detection network is obtained; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-constructed, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module, and a predictor module; the training set is obtained by augmenting a high-resolution visible light ship image.

[0098] Acquire high-resolution visible light images of ships to be detected.

[0099] The high-resolution visible light ship image to be detected is input into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship and the classification confidence of the target ship.

[0100] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0101] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can perform a high-resolution visible light ship detection method guided by saliency information, the method comprising:

[0102] A trained ship detection network is obtained; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-constructed, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module, and a predictor module; the training set is obtained by augmenting a high-resolution visible light ship image.

[0103] Acquire high-resolution visible light images of ships to be detected.

[0104] The high-resolution visible light ship image to be detected is input into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship and the classification confidence of the target ship.

[0105] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, a high-resolution visible light ship detection method guided by saliency information is implemented, the method comprising:

[0106] A trained ship detection network is obtained; the trained ship detection network is obtained by training the ship detection network based on a training set, the ship detection network is pre-constructed, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module, and a predictor module; the training set is obtained by augmenting a high-resolution visible light ship image.

[0107] Acquire high-resolution visible light images of ships to be detected.

[0108] The high-resolution visible light ship image to be detected is input into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship and the classification confidence of the target ship.

[0109] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-resolution visible light ship detection method guided by saliency information, characterized in that: include: Get the trained ship detection network; The trained ship detection network is obtained by training the ship detection network based on the training set, the ship detection network is pre-built, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module and a predictor module; the training set is obtained by augmenting the high-resolution visible light ship image; Acquire high-resolution visible light images of ships to be detected; Inputting the high-resolution visible light ship image to be detected into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship, and the classification confidence of the target ship; The encoder module uses ResNet50 as the backbone network, including a four-layer first network structure, and fuses features in a bottom-up manner to form pyramid features; Inputting the top-level features in the encoder module into the semantic module and encoding them into features containing semantic information; The transition module comprises three layers of second network structure, which correspond to the first layer, the second layer and the third layer connected to the encoder module respectively; The decoder module includes a multi-layer third network structure, the decoder module adopts a double matching mechanism to introduce saliency information, the double matching mechanism includes a prediction branch of structure matching and a saliency label supervision branch of size matching, and the sizes of saliency labels of different layers in the multi-layer third network structure are different; the number of layers of the third network structure is the same as the number of layers of the network structure of the encoder module; The predictor module adopts a five-layer pyramid structure, the top layer is connected to the output of the semantic module, and the remaining four layers are provided by cross-level and cross-channel feature fusion modules. Each layer of the predictor module needs to process classification and regression tasks respectively.

2. The high-resolution visible light ship detection method guided by saliency information according to claim 1 is characterized in that: The training set is obtained in the following way: Acquire high-resolution visible light ship images; The high-resolution visible light ship image and the positioning labels in the image are randomly flipped, randomly cropped and fixed in size; The saliency mask labels of the high-resolution visible light ship image are randomly flipped, randomly cropped and fixed in size.

3. The high-resolution visible light ship detection method guided by saliency information according to claim 1 or 2, characterized in that: The backbone network of the encoder module is ResNet50, and the encoder module includes a multi-layer first network structure constructed according to the backbone network. The multi-layer first network structure adopts a feature pyramid structure, and the feature dimensions of outputs of different layers in the multi-layer first network structure are different.

4. The high-resolution visible light ship detection method guided by saliency information according to claim 1 or 2, characterized in that: The semantic module includes a ReLU activation layer and a first convolutional layer.

5. The high-resolution visible light ship detection method guided by saliency information according to claim 1 or 2, characterized in that: The transition module includes a multi-layer second network structure, which adopts a top-down fusion method, and the feature dimensions of outputs of different layers in the multi-layer second network structure are the same.

6. The high-resolution visible light ship detection method guided by saliency information according to claim 1 or 2, characterized in that: The predictor module includes a classification branch prediction structure and a regression branch prediction structure, the classification branch prediction structure includes multiple second convolutional layers and a third convolutional layer with an output channel of A; the regression branch prediction structure includes multiple fourth convolutional layers and a fifth convolutional layer with an output channel of 4A, where A represents the number of anchor boxes, and the number of the second convolutional layers is the same as the number of the fourth convolutional layers.

7. The high-resolution visible light ship detection method guided by saliency information according to claim 1 or 2, characterized in that: The decoder module also includes a cross-level and cross-channel feature fusion unit, which includes a first average pooling layer, a second average pooling layer and a sixth convolutional layer. The first average pooling layer is used to extract the first channel-level features of the encoder module, the second average pooling layer is used to extract the second channel-level features of the decoder module, and the sixth convolutional layer is used to fuse the first channel-level features and the second channel-level features to achieve cross-channel information interaction.

8. A high-resolution visible light ship detection system guided by saliency information, characterized in that: include: A ship detection network acquisition module is used to obtain the trained ship detection network; The trained ship detection network is obtained by training the ship detection network based on the training set, the ship detection network is pre-built, and the ship detection network includes an encoder module, a semantic module, a transition module, a decoder module and a predictor module; the training set is obtained by augmenting the high-resolution visible light ship image; An image acquisition module to be detected, used to acquire a high-resolution visible light ship image to be detected; An image detection module, used for inputting the high-resolution visible light ship image to be detected into the trained ship detection network to obtain the position information of the target ship, the saliency mask of the target ship and the classification confidence of the target ship; The encoder module uses ResNet50 as the backbone network, including a four-layer first network structure, and fuses features in a bottom-up manner to form pyramid features; Inputting the top-level features in the encoder module into the semantic module and encoding them into features containing semantic information; The transition module comprises three layers of second network structure, which correspond to the first layer, the second layer and the third layer connected to the encoder module respectively; The decoder module includes a multi-layer third network structure, the decoder module adopts a double matching mechanism to introduce saliency information, the double matching mechanism includes a prediction branch of structure matching and a saliency label supervision branch of size matching, and the sizes of saliency labels of different layers in the multi-layer third network structure are different; the number of layers of the third network structure is the same as the number of layers of the network structure of the encoder module; The predictor module adopts a five-layer pyramid structure, the top layer is connected to the output of the semantic module, and the remaining four layers are provided by cross-level and cross-channel feature fusion modules. Each layer of the predictor module needs to process classification and regression tasks respectively.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the high-resolution visible light ship detection method guided by the saliency information as described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Optical remote sensing image ship detection method based on feature fusion convolutional network

    CN108921066A

  • Infrared remote sensing image ship detection method

    CN111723748A

  • Optical remote sensing image ship detection and segmentation method based on deep learning

    CN112507777A

  • Ship target detection method based on visible light and infrared image fusion

    CN115909085A

  • Lightweight SAR ship detection method based on significant features

    CN116524354A