Ship detection method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention relates to the technical field of remote sensing image detection, and discloses a ship detection method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: carrying out the image processing of a ship image of a target ship, so as to obtain an initial candidate box set corresponding to the target ship, the initial candidate frame set comprises a plurality of initial candidate frames corresponding to the target ship, and the initial candidate frames are used for positioning the boundary of the target ship; and determining a target detection frame corresponding to the target ship according to the plurality of confidence scores corresponding to the plurality of initial candidate frames and the plurality of areas corresponding to the plurality of initial candidate frames. According to the embodiment, the target detection card corresponding to the target ship is determined through the confidence score and the area of the initial candidate box, so that the detection accuracy of the target detection box and the target boundary integrity are improved.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image inspection technology, and in particular to a ship inspection method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Ship detection technology using remote sensing imagery is gradually becoming one of the key foundational technologies in fields such as marine monitoring, port security, maritime supervision, and maritime rescue. Currently, ship detection mainly targets two types of remote sensing data sources: optical remote sensing imagery and synthetic aperture radar (SAR) imagery. Optical remote sensing imagery has advantages such as high resolution and rich color and texture, but it is easily affected by imaging conditions such as lighting and fog, and is therefore uncontrollable. SAR imagery has the ability to penetrate fog and image day and night, making it suitable for harsh environments, but it has higher image noise, unclear object outlines, and is prone to speckle artifacts and speckle effects.

[0003] At the level of detection algorithms, the mainstream methods can be divided into two categories: one is the traditional method based on artificial features, such as ship detection methods based on edge detection, morphological features, and other low-level features combined with classifiers; the other is the target detection method based on deep learning, especially represented by CNN (convolutional neural network), which is widely used in remote sensing ship target detection and has a good balance between detection accuracy and speed.

[0004] However, due to the large size of remote sensing images (e.g., 10000×10000 pixels or more), existing ship detection methods require the use of sliding window or region segmentation strategies for block prediction, which introduces window overlap problems. For example, the same ship may be detected multiple times in multiple overlapping sliding window block images, generating multiple prediction boxes and affecting the accuracy of the detection results. Summary of the Invention

[0005] The purpose of this invention is to provide at least one ship inspection method, apparatus, electronic device, and computer-readable storage medium, which can at least solve the above-mentioned problems.

[0006] At least one embodiment of the present invention provides a ship detection method, comprising: performing image recognition on a ship image of a target ship to obtain an initial candidate box set corresponding to the target ship, wherein the initial candidate box set includes multiple initial candidate boxes corresponding to the target ship, the initial candidate boxes being used to locate the boundary of the target ship; and determining a target detection box corresponding to the target ship based on multiple confidence scores corresponding to the multiple initial candidate boxes and multiple areas corresponding to the multiple initial candidate boxes.

[0007] Furthermore, before performing image recognition on the ship image of the target ship, the method further includes: acquiring optical remote sensing images and radar images of the target ship; preprocessing the optical remote sensing images and radar images to obtain joint input data, wherein the joint input data includes multi-channel data composed of the optical remote sensing images and radar images.

[0008] Further, the image processing of the target ship's image to obtain an initial candidate box set corresponding to the target ship includes: inputting the joint input data, the optical remote sensing image, and the radar image into a pre-trained ship detection neural network; obtaining a first image feature corresponding to the optical remote sensing image, a second image feature corresponding to the radar image, and a third image feature corresponding to the joint input data through the feature extraction module in the ship detection neural network; obtaining the fused features of the target ship based on the first image feature, the second image feature, and the third image feature through the feature fusion module in the ship detection neural network; and obtaining the initial candidate box set based on the fused features.

[0009] Further, the initial candidate boxes include multiple vertex coordinates of the initial candidate boxes and the confidence scores. The step of determining the target detection box corresponding to the target ship based on the multiple confidence scores and areas corresponding to the multiple initial candidate boxes includes: determining the area corresponding to each initial candidate box based on the multiple vertex coordinates of each initial candidate box in the initial candidate box set; sorting the multiple initial candidate boxes according to the confidence scores and areas of each initial candidate box in the initial candidate box set to obtain an initial candidate box list; and determining the initial candidate boxes in the initial candidate box list that meet preset conditions as the target detection boxes.

[0010] Furthermore, the preset conditions include at least one of the following: the aspect ratio of the initial candidate box is within a preset ratio range; the area of ​​the initial candidate box is within a preset area range; the initial candidate box is located within the water area in the ship image.

[0011] Further, before determining the target detection box corresponding to the target ship based on the multiple confidence scores corresponding to the multiple initial candidate boxes and the multiple areas corresponding to the multiple initial candidate boxes, the method further includes: determining the normalized difference water index corresponding to the optical remote sensing image based on the green band reflectance of the water body in the green band and the infrared band reflectance of the water body in the near-infrared band; determining the land-water segmentation mask in the ship image based on the normalized difference water index corresponding to the optical remote sensing image; or determining the land-water segmentation mask based on the normalized difference water index corresponding to the radar image; or determining the land-water segmentation mask in the ship image based on the normalized difference water index corresponding to the optical remote sensing image and the normalized difference water index corresponding to the radar image.

[0012] At least one embodiment of the present invention provides a ship detection device, comprising: an image recognition module, configured to perform image recognition on a ship image of a target ship to obtain an initial candidate box set corresponding to the target ship, wherein the initial candidate box set includes multiple initial candidate boxes corresponding to the target ship, the initial candidate boxes being used to locate the boundary of the target ship; and a determination module, configured to determine a target detection box corresponding to the target ship based on multiple confidence scores corresponding to the multiple initial candidate boxes and multiple areas corresponding to the multiple initial candidate boxes.

[0013] Further, the initial candidate box includes multiple vertex coordinates of the initial candidate box and the confidence score, wherein the determining module includes: a first determining submodule, used to determine the area corresponding to each initial candidate box according to the multiple vertex coordinates corresponding to each initial candidate box in the initial candidate box set; a sorting submodule, used to sort the multiple initial candidate boxes according to the confidence score and the area corresponding to each initial candidate box in the initial candidate box set to obtain an initial candidate box list; and a second determining submodule, used to determine the initial candidate boxes in the initial candidate box list that meet preset conditions as the target detection boxes.

[0014] At least one embodiment of the present invention provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the ship detection method as described above.

[0015] At least one embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the ship detection method as described above.

[0016] The ship detection method proposed in this invention performs image processing on the ship image of the target ship to obtain an initial candidate box set corresponding to the target ship. This initial candidate box set includes multiple initial candidate boxes corresponding to the target ship, which are used to locate the boundary of the target ship. Based on multiple confidence scores and areas corresponding to the multiple initial candidate boxes, the target detection box corresponding to the target ship is determined. In this embodiment, the target detection box corresponding to the target ship is determined by the confidence score and area of ​​the initial candidate boxes, avoiding the problem in traditional algorithms where complete ship detection boxes are mistakenly deleted due to high confidence in local ship detection boxes, thus improving the detection accuracy and target boundary integrity. Attached Figure Description

[0017] The accompanying drawings, which form part of this specification, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0018] Figure 1 This is a flowchart illustrating an optional ship detection method according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the frame of an optional ship detection device according to an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of the frame of an optional electronic device according to an embodiment of the present invention. Detailed Implementation

[0021] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0022] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this invention is for describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0023] Before introducing the ship detection method in the embodiments of this invention, the background technology of this invention should first be explained: In existing ship detection methods, especially those based on deep learning, due to the huge size of remote sensing images (e.g., 10000×10000 pixels or more), existing detection methods need to use sliding window or region segmentation strategies for block prediction. This process introduces the problem of window overlap. For example, the same ship may be detected multiple times in multiple overlapping sliding window block images, generating multiple predicted boxes. Such problems are often solved by using the non-maximum suppression (NMS) algorithm for post-processing, that is, for a set of overlapping boxes, only the one with the highest prediction confidence is retained, and other redundant boxes are removed. However, this algorithm mechanism relies on confidence ranking. When a local ship region is mistakenly assigned a high confidence level while the complete ship box has a low confidence level, the true complete target box will be incorrectly deleted, affecting the detection accuracy.

[0024] To address the aforementioned problems, this embodiment provides a ship detection method, such as... Figure 1 As shown, the method may specifically include the following steps:

[0025] S102, perform image recognition on the ship image of the target ship to obtain an initial candidate box set corresponding to the target ship. The initial candidate box set includes multiple initial candidate boxes corresponding to the target ship. The initial candidate boxes are used to locate the boundary of the target ship.

[0026] In this embodiment, the ship images of the target ship include, but are not limited to, optical images, optical remote sensing images, radar images, and infrared images. Image recognition of the ship images includes, but is not limited to: extracting local or global features of the ship images and then using a classifier such as a support vector machine for recognition and classification; constructing and training a convolutional neural network (CNN) to extract image features from the ship images and perform ship recognition.

[0027] It should also be noted that the image recognition of ship images in this embodiment can be performed on single-modal ship images or on multi-modal ship images. Multi-modal ship images include, but are not limited to, ship images acquired through multiple sensors or acquisition methods. For example, ship images acquired through optical sensors, multi-aperture radar, far-infrared sensors, etc.

[0028] Image recognition is performed on ship images to obtain an initial set of candidate bounding boxes for the target ship; that is, multiple initial candidate bounding boxes for the target ship are obtained. In practical applications, the initial candidate bounding boxes are polygons used to locate the boundaries of the target ship. The initial candidate bounding boxes also include information such as the target ship's orientation, shape, color, and area.

[0029] In this embodiment, the initial candidate bounding boxes are the detection boxes of the target ship, used to define the precise position of the target ship in the ship image, providing spatial anchor points for subsequent tracking and behavior analysis. The initial candidate bounding boxes in this embodiment include, but are not limited to, horizontal rectangles, rotated rectangles, and 3D bounding boxes.

[0030] S104. Based on the multiple confidence scores corresponding to multiple initial candidate boxes and the multiple areas corresponding to multiple initial candidate boxes, determine the target detection box corresponding to the target ship.

[0031] In this embodiment, each initial candidate box corresponds to a confidence score. The confidence score reflects the probability that the detection box contains the target ship. The higher the score, the higher the probability that the image recognition algorithm has identified the target ship's location in the detection box; conversely, the lower the score, the lower the probability that the detection box has identified the target ship's location.

[0032] In this embodiment, the initial candidate boxes in the initial candidate box set are sorted and filtered according to the confidence score and area of ​​the initial detection box to determine the target detection box of the target ship.

[0033] In one example, the area of ​​the target ship on the horizontal plane is obtained in advance. Based on the ratio of the ship image to the target ship image, the area of ​​the detection box of the target ship in the ship image is determined. In the initial candidate box set, the initial candidate box with the same area as the detection box area and the highest confidence score is selected as the target detection box.

[0034] In this embodiment, image processing is performed on the target ship's image to obtain an initial candidate box set corresponding to the target ship. This initial candidate box set includes multiple initial candidate boxes corresponding to the target ship, used to locate the target ship's boundaries. Based on multiple confidence scores and areas corresponding to these initial candidate boxes, a target detection box corresponding to the target ship is determined. This embodiment determines the target detection box corresponding to the target ship by using the confidence scores and areas of the initial candidate boxes, avoiding the problem in traditional algorithms where high confidence scores of local ship detection boxes lead to the mistaken deletion of complete ship detection boxes. This embodiment considers both the confidence scores and areas of the initial candidate boxes simultaneously, significantly improving the detection accuracy and target boundary integrity.

[0035] Optionally, in this embodiment, before performing image recognition on the target ship's image, the process includes, but is not limited to, acquiring optical remote sensing images and radar images of the target ship. In this embodiment, ship images of the target ship are acquired using different sensors. Optical remote sensing images and radar images are obtained through satellite imaging, wherein the radar images are acquired using synthetic aperture radar (SAR).

[0036] In this embodiment, optical remote sensing images and radar images are preprocessed to obtain joint input data, wherein the joint input data includes multi-channel data composed of optical remote sensing images and radar images.

[0037] In one example, optical remote sensing images of the same area at the same time were acquired from a satellite platform. opt With radar image I sar For optical remote sensing images I opt With radar image I sar Preprocessing is performed, including but not limited to radiometric calibration, image cropping, and normalization. Then, registration is used to spatially align the two modalities to form joint input data I. multi :

[0038] I multi ={I opt ,I sar}

[0039] In this embodiment, the input data I is combined multi The data consists of 4 channels, including 3 channels of optical remote sensing imagery and 1 channel of radar imagery.

[0040] In addition, when only a single modality exists (such as only optical remote sensing imagery or only radar imagery), a missing modality filling strategy (radar imagery is the missing modality when only optical remote sensing imagery exists; and optical remote sensing imagery is the missing modality when only radar imagery exists) or an automatic modality recognition mechanism can be used to maintain the consistency of the input structure.

[0041] In one example, feature vectors from available modalities (such as optical-only remote sensing imagery or radar-only imagery) are used to directly predict the embedding representation of the missing modality via a lightweight generative network (such as a deep learning architecture like Transformer or a multilayer perceptron MLP). For example, prompt-tuning is used to guide the feature predictor to generate embeddings aligned with downstream tasks.

[0042] In another example, generative adversarial networks (GANs), diffusion models, and other techniques are used to generate raw data for missing modalities (e.g., generating corresponding radar image data from optical remote sensing imagery).

[0043] In this embodiment, by performing image recognition on multimodal data (optical remote sensing images and radar images), an initial detection box set of the target ship is obtained, which effectively enhances the robustness under various weather, lighting and imaging conditions.

[0044] Optionally, in this embodiment, step S102 includes, but is not limited to: inputting the joint input data, optical remote sensing image, and radar image into a pre-trained ship detection neural network; obtaining the first image feature corresponding to the optical remote sensing image, the second image feature corresponding to the radar image, and the third image feature corresponding to the joint input data through the feature extraction module in the ship detection neural network; obtaining the fused features of the target ship based on the first image feature, the second image feature, and the third image feature through the feature fusion module in the ship detection neural network; and obtaining an initial candidate box set based on the fused features.

[0045] In one example, optical remote sensing image I opt Radar Image I sar and joint input data I multi The data is input into a ship detection neural network. The main structure of this network includes a feature extraction module, a feature fusion module, and a feature processing module. The input to the ship detection neural network can be optical remote sensing images, radar images, and combined input data, or a combination of both. The output of the ship detection neural network is the fused features of the target ship.

[0046] (a) Feature extraction of ship images

[0047] In this embodiment, the optical remote sensing image I is processed by a feature extraction module. opt Radar Image I sar and joint input data I multi I perform data feature extraction:

[0048] F opt =f opt (I opt );F sar =f sar (I sar );F multi =f multi (I multi )

[0049] Among them, f opt It is a network structure used to extract features from optical remote sensing images, f sar It is a network structure used to extract features from radar images, f multi These are network structures used to extract features from joint input data. In practical applications, convolutional neural networks can be selected from these three network structures; F opt Optical remote sensing image I opt Corresponding extracted image features; F sar It is radar image I sar Corresponding extracted image features; F multi It is the combined input data Imulti The corresponding extracted image features.

[0050] (b) Feature fusion of ship images

[0051] In this embodiment, an attention-guided feature fusion module is used to fuse the image features extracted in step (a) above:

[0052] F fusion =A(F opt ,F sar ,F multi )

[0053] Where A represents the feature fusion algorithm, such as channel attention module (SE, Squeeze-and-Excitation), weighted addition fusion, or cross-attention mechanism. F fusion This refers to the fused features of the final output.

[0054] (c) Feature processing

[0055] In one example, the YOLO algorithm (You Only Look Once, a single-stage object detection algorithm) is used to process the fused features F. fusion Process the data to generate an initial set of candidate boxes.

[0056] Through the above embodiments, the ship detection neural network performs image recognition on multimodal data, including optical remote sensing images, radar images, and joint input data. Based on the fusion features output by the model, an initial candidate box set is obtained, which effectively enhances the robustness of the model under various weather, lighting, and imaging conditions.

[0057] Optionally, in this embodiment, the initial candidate box includes multiple vertex coordinates and confidence scores of the initial candidate box, wherein step S104 includes, but is not limited to:

[0058] The area corresponding to each initial candidate box is determined based on the coordinates of multiple vertices corresponding to each initial candidate box in the initial candidate box set.

[0059] In one example, the initial candidate box set D is defined as:

[0060]

[0061] Where, d i This is the i-th initial candidate bounding box (polygon detection box) of the target ship identified by the model. The first 8 elements are the coordinates of the eight vertices of the initial candidate bounding box (arranged in order), and the 9th initial bounding box is the ith initial candidate bounding box. i is the confidence score of the initial candidate box, and N is the index of the target ship.

[0062] For each polygon detection box d i Calculate the area A of its enclosed polygon. i Specifically:

[0063]

[0064] In this embodiment, multiple initial candidate boxes are sorted according to the confidence score and area of ​​each initial candidate box in the initial candidate box set to obtain an initial candidate box list;

[0065] In this embodiment, all initial candidate boxes are jointly sorted according to the rule of "confidence score first, area second". That is, the following sorting priority is constructed:

[0066] Order = lexsort(-s i ,-A i )

[0067] Here, lexsort is a function that sorts by keywords (keywords are "confidence score" and "area"). In this embodiment, confidence score is given priority in descending order, and then area is given priority in descending order.

[0068] In this embodiment, initial candidate boxes that meet preset conditions in the initial candidate box list are identified as target detection boxes. In specific application scenarios, an initial candidate box is identified as a target detection box after at least one of the preset conditions—confidence score, area, type, aspect ratio, and color—is satisfied.

[0069] In one example, after sorting all the initial candidate boxes in the initial candidate box set, it is necessary to filter out extreme values ​​to avoid errors. This can include the following steps:

[0070] Let the initial candidate boxes be sorted as {i1, i2, ..., i...} N The specific filtering process may include the following steps:

[0071] S11, Initialize the reserved list K;

[0072] S12, sequentially take the first index i from the sorted list obtained in step 3 and place the corresponding box d i Join K;

[0073] S13, for all remaining candidate boxes d j Calculate its relationship with d i Polygon IoU:

[0074]

[0075] S14, if IoU ij≤θ, keep d j Otherwise, discard. Here, θ is the set threshold, ranging from 0 to 1.

[0076] Repeat the above steps until all candidate boxes have been traversed, then return the final set of initial candidate boxes:

[0077] B final ={d k |k∈K}

[0078] Among them, B final This is the target detection bounding box corresponding to the target ship.

[0079] The above embodiments sort the initial candidate boxes based on their confidence scores and areas, replacing the traditional NMS algorithm that relies solely on confidence scores. This overcomes the problem of misleading local high-confidence results caused by window overlap and scale variations. This embodiment improves the completeness retention rate of ship target boundaries and reduces duplicate and missed detections of ships.

[0080] Optionally, in this embodiment, the preset conditions include at least one of the following:

[0081] (1) The aspect ratio of the initial candidate box is within the preset ratio range;

[0082] Specifically, the detection frame is usually a standard rectangle, and its tilt direction is consistent with the ship's orientation. Let the length and width of the detection frame be h and w, respectively, then its aspect ratio is:

[0083]

[0084] Set [r] min ,r max [This represents a preset ratio range; if...] If the initial candidate box is determined to have an abnormal structure, it is discarded. Where r min ,r max It can be set based on practical experience.

[0085] (2) The area of ​​the initial candidate box is within the preset area range;

[0086] The initial candidate box area is A = w * h. If the area A of the initial candidate box is not within the preset area range [A... min A max Within the specified interval, the initial candidate box is considered a non-ship target. Among them, A... min A max It can be set based on practical experience.

[0087] (3) The initial candidate box is located within the water area in the ship image.

[0088] Determine the water-land boundary in the ship image. If the initial candidate box is located in a non-water body (e.g., land), the initial candidate box is considered a non-ship target.

[0089] In this embodiment, prior knowledge of the actual size distribution of the target ship (known knowledge or pre-set based on actual experience) is combined to filter the initial candidate boxes with multiple structural constraints such as aspect ratio and area, eliminating obviously non-ship detection results such as abnormally small boxes, flat boxes and oversized boxes, thereby significantly reducing the false detection rate and improving the model's adaptability to the actual ship shape.

[0090] Optionally, in this embodiment, before performing the above step S104, the following steps are included but not limited to: determining the normalized difference water index corresponding to the optical remote sensing image based on the green light band reflectance of the water body in the green light band and the infrared band reflectance of the water body in the near-infrared band; determining the land-water segmentation mask in the ship image; or determining the land-water segmentation mask based on the normalized difference water index corresponding to the radar image; or determining the land-water segmentation mask in the ship image based on the normalized difference water index corresponding to the optical remote sensing image and the normalized difference water index corresponding to the radar image.

[0091] In specific application scenarios, optical land-water segmentation results can be calculated solely from optical remote sensing images, and radar land-water segmentation results can also be calculated from radar images.

[0092] First, the Normalized Difference Water Index (NDWI) is calculated from the optical remote sensing imagery:

[0093]

[0094] Wherein, Green represents the reflectance of water in the green light band, and NIR represents the reflectance of water in the near-infrared band.

[0095] For NDWI results of optical remote sensing imagery, the OTSU (Otsu method) is used to determine the optimal binary segmentation threshold T. opt Optical images are divided into two categories: water bodies and non-water bodies, specifically:

[0096]

[0097] Among them, WaterMask opt It is an optical land-water separation mask.

[0098] In practical applications, water areas in radar images often exhibit low backscattering characteristics (low grayscale values), allowing the optimal binary segmentation threshold T to be directly determined using OTSU. sarThe radar images are divided into two categories: water bodies and non-water bodies, specifically:

[0099]

[0100] Among them, WaterMask sar It is a radar land-sea separation mask.

[0101] In this embodiment, the land-water segmentation result is determined based on both optical remote sensing images and radar images. The optical land-water segmentation result corresponding to the optical remote sensing image and the radar land-water segmentation result corresponding to the radar image can be fused to generate the final land-water segmentation mask.

[0102] The final water mask is generated by fusing the land-sea segmentation results from optical and SAR imagery:

[0103] WaterMask final =WaterMask opt ∩WaterMask sar

[0104] Among them, WaterMask final A mask for separating land and water.

[0105] In this embodiment, after obtaining the initial candidate box list, for each initial candidate box, if its center part is inside the water body of the land-water separation mask, it is removed; otherwise, it is retained.

[0106] Through the above embodiments, a land-sea segmentation mask is determined based on optical remote sensing images and / or radar images. The mask is then used to remove or identify false ship targets that fall on land, thereby avoiding interference from complex backgrounds such as ports, docks, and shorelines on the ship detection frame and further improving the accuracy of ship target detection on the sea surface.

[0107] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or its equivalents are included in this invention.

[0108] At least one embodiment of the present invention provides a ship detection device, such as Figure 2 As shown, the device includes:

[0109] Image recognition module 20 is used to perform image recognition on the ship image of the target ship to obtain an initial candidate box set corresponding to the target ship, wherein the initial candidate box set includes multiple initial candidate boxes corresponding to the target ship, and the initial candidate boxes are used to locate the boundary of the target ship;

[0110] The determination module 22 is used to determine the target detection box corresponding to the target ship based on the multiple confidence scores corresponding to the multiple initial candidate boxes and the multiple areas corresponding to the multiple initial candidate boxes.

[0111] Optionally, in this embodiment, it further includes:

[0112] The acquisition module is used to acquire optical remote sensing images and radar images of the target ship before performing image recognition on the ship images of the target ship.

[0113] An image processing module is used to preprocess the optical remote sensing image and the radar image to obtain joint input data, wherein the joint input data includes multi-channel data composed of the optical remote sensing image and the radar image.

[0114] Optionally, in this embodiment, the image recognition module 22 includes:

[0115] The feature extraction submodule is used to input the joint input data, the optical remote sensing image, and the radar image into a pre-trained ship detection neural network, and obtain the first image feature corresponding to the optical remote sensing image, the second image feature corresponding to the radar image, and the third image feature corresponding to the joint input data through the feature extraction module in the ship detection neural network.

[0116] The feature fusion submodule is used to obtain the fused features of the target ship based on the first image features, the second image features, and the third image features through the feature fusion module in the ship detection neural network.

[0117] The processing submodule is used to obtain the initial candidate box set based on the fusion features.

[0118] Optionally, in this embodiment, the initial candidate box includes the coordinates of multiple vertices of the initial candidate box and the confidence score.

[0119] The determining module 22 includes:

[0120] The first determining submodule is used to determine the area corresponding to each initial candidate box based on the coordinates of multiple vertices corresponding to each initial candidate box in the initial candidate box set;

[0121] The sorting submodule is used to sort the multiple initial candidate boxes according to the confidence score and the area corresponding to each initial candidate box in the initial candidate box set, so as to obtain an initial candidate box list;

[0122] The second determining submodule is used to determine the initial candidate boxes that meet the preset conditions in the initial candidate box list as the target detection boxes.

[0123] Optionally, in this embodiment, the preset conditions include at least one of the following:

[0124] The aspect ratio of the initial candidate box is within a preset ratio range;

[0125] The area of ​​the initial candidate box is within a preset area range;

[0126] The initial candidate box is located within the water area in the ship image.

[0127] Optionally, in this embodiment, it further includes:

[0128] The first processing module is used to determine the normalized difference water body index corresponding to the optical remote sensing image before determining the target detection box corresponding to the target ship based on the multiple confidence scores corresponding to the multiple initial candidate boxes and the multiple areas corresponding to the multiple initial candidate boxes.

[0129] The second processing module is used for

[0130] The land-water segmentation mask in the ship image is determined based on the normalized difference water index corresponding to the optical remote sensing image; or, the land-water segmentation mask is determined based on the normalized difference water index corresponding to the radar image; or, the land-water segmentation mask in the ship image is determined based on both the normalized difference water index corresponding to the optical remote sensing image and the normalized difference water index corresponding to the radar image.

[0131] Another embodiment of the present invention relates to an electronic device, such as Figure 3 As shown, it includes: at least one processor 301; and a memory 302 communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions executable by the at least one processor 301, the instructions being executed by the at least one processor 301 to enable the at least one processor 301 to perform the ship detection methods in the above embodiments.

[0132] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0133] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0134] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0135] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for inspecting ships, characterized in that, include: Image recognition is performed on the ship image of the target ship to obtain an initial candidate box set corresponding to the target ship. The initial candidate box set includes multiple initial candidate boxes corresponding to the target ship, and the initial candidate boxes are used to locate the boundary of the target ship. Based on the multiple confidence scores corresponding to the multiple initial candidate boxes and the multiple areas corresponding to the multiple initial candidate boxes, the target detection box corresponding to the target ship is determined.

2. The method according to claim 1, characterized in that, Before performing image processing on the target ship's image, the method further includes: Acquire optical remote sensing images and radar images of the target ship; The optical remote sensing image and the radar image are preprocessed to obtain joint input data, wherein the joint input data includes multi-channel data composed of the optical remote sensing image and the radar image.

3. The method according to claim 2, characterized in that, The step of performing image recognition on the target ship's image to obtain an initial candidate box set corresponding to the target ship includes: The joint input data, the optical remote sensing image, and the radar image are input into a pre-trained ship detection neural network. The feature extraction module in the ship detection neural network obtains the first image feature corresponding to the optical remote sensing image, the second image feature corresponding to the radar image, and the third image feature corresponding to the joint input data. The feature fusion module in the ship detection neural network obtains the fused features of the target ship based on the first image features, the second image features, and the third image features. The initial candidate box set is obtained based on the fusion features.

4. The method according to claim 1, characterized in that, The initial candidate box includes the coordinates of multiple vertices of the initial candidate box and the confidence score. The step of determining the target detection box corresponding to the target ship based on the multiple confidence scores corresponding to the multiple initial candidate boxes and the multiple areas corresponding to the multiple initial candidate boxes includes: Based on the coordinates of multiple vertices corresponding to each initial candidate box in the initial candidate box set, determine the area corresponding to each initial candidate box; Based on the confidence score and area corresponding to each initial candidate box in the initial candidate box set, the multiple initial candidate boxes are sorted to obtain an initial candidate box list; The initial candidate boxes that meet the preset conditions in the initial candidate box list are determined as the target detection boxes.

5. The method according to claim 4, characterized in that, The preset conditions include at least one of the following: The aspect ratio of the initial candidate box is within a preset ratio range; The area of ​​the initial candidate box is within a preset area range; The initial candidate box is located within the water area in the ship image.

6. The method according to claim 2, characterized in that, Before determining the target detection box corresponding to the target ship based on the multiple confidence scores corresponding to the multiple initial candidate boxes and the multiple areas corresponding to the multiple initial candidate boxes, the method further includes: Based on the green light band reflectance of the water body in the optical remote sensing image and the infrared band reflectance of the water body in the near-infrared band, the normalized differential water body index corresponding to the optical remote sensing image is determined. The land-water segmentation mask in the ship image is determined based on the normalized differential water index corresponding to the optical remote sensing image; or... The land-water separation mask is determined based on the normalized difference water index corresponding to the radar image; or... Based on the normalized difference water index corresponding to the optical remote sensing image and the normalized difference water index corresponding to the radar image, the land-water segmentation mask in the ship image is determined.

7. A ship detection device, characterized in that, include: An image recognition module is used to perform image recognition on the ship image of the target ship to obtain an initial candidate box set corresponding to the target ship. The initial candidate box set includes multiple initial candidate boxes corresponding to the target ship, and the initial candidate boxes are used to locate the boundary of the target ship. The determination module is used to determine the target detection box corresponding to the target ship based on the multiple confidence scores corresponding to the multiple initial candidate boxes and the multiple areas corresponding to the multiple initial candidate boxes.

8. The apparatus according to claim 7, characterized in that, The initial candidate box includes the coordinates of multiple vertices of the initial candidate box and the confidence score, wherein the determining module includes: The first determining submodule is used to determine the area corresponding to each initial candidate box based on the coordinates of multiple vertices corresponding to each initial candidate box in the initial candidate box set; The sorting submodule is used to sort the multiple initial candidate boxes according to the confidence score and the area corresponding to each initial candidate box in the initial candidate box set, so as to obtain an initial candidate box list; The second determining submodule is used to determine the initial candidate boxes that meet the preset conditions in the initial candidate box list as the target detection boxes.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the ship detection method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the ship detection method according to any one of claims 1 to 6.

Images